Computer Science II

1
Computer Science II

• Algorithm Analysis

2
Big-O Notation

• Definitionf(n) is O(g(n)) if there exists
  positive numbers c and N such that f(n) ? cg(n)
  for all n ? N
• Big-O is an upper bound

3
Big-O Notation

• Given f(n) n2 100n 1og10n 1000
• When n is very small, the last term is most
  important
• When n 10, 100n 1000
• When n 100, n2 100n
• When n gt 100, n2 becomes the dominant term

4
Big-O Notation

• Given f(n) n2 100n 1og10n 1000
• Therefore, for all n ? 100
• f(n) O(n2)
• Can we prove this?

5
Big-O Notation

• Given f(n) n2 100n 1og10n 1000
• Starting with f(n) ? cg(n)
• or n2 100n 1og10n 1000 ? cn2

6
Big-O Notation

• Dividing both sides by n2 yields 1 100/n
  1og10n/n2 1000/n2 ? c
• As n increases, the left side of this inequality
  will just keep getting smaller and smaller, so
• If we set N 100, we can compute c by 1
  100/100 1og10100/1002 1000/1002 ? c
• The smallest value for c is found when we replace
  ? by

7
Try this!

• Given f(n) 3n3 2n2 1
• Find a g(n), c and N, such that f(n) ? cg(n),
  for all n ? N

8
Big-O NotationProperties

• If f(n) is O(g(n)) and g(n) is O(h(n)),then f(n)
  is O(h(n)) (transitivity)
• If f(n) is O(h(n)) and g(n) is O(h(n)),then
  f(n)g(n) is O(h(n))
• The function ank is O(nk)
• The function nk is O(nkj) for any positive j

9
Big-O NotationProperties

• If f(n) cg(n), then f(n) is O(g(n))
• The function loga n is O(logb n) for any positive
  number a and b ? 1
• loga n is O(lg n) for any positive a ? 1,where
  lg n log2 n

10
Big-? and ? Notations

• (lower bound)f(n) is ?(g(n)) if there exists
  positive numbers c and N such that f(n) ? cg(n)
  for all n ? N
• (upper lower bounds)f(n) is ?(g(n)) if there
  exists positive numbers c1, c2 and N such that
  c1g(n) ? f(n) ? c2g(n)for all n ? N

11
Analysis Pitfalls

• Given two algorithms with timing equations of
  f1(n) n2 and f2(n) 108nWhich is a better
  algorithm, time-wise?
• Using big-O notation f1(n) O(n2) f2(n)
  O(n)
• But f1(n) lt f2(n) for n lt 108
• So, be careful!

12
Types of Complexity

• O(1) constant
• O(log n ) logarithmic
• O(n log n)
• O(n) linear
• O(n2) quadratic
• O(nc) polynomial
• O(cn) exponential

13
Determining Complexity

• Q How?
• A Count the number of assignments and
  comparisons
• Try this sum 0 for (i 0 i lt n
  i) sum ai cout ltlt sum ltlt endl
• How many comparisons? How many assignments?

14
Determining Complexity

• How about this for (i 0 i lt n
  i) sum a0 for (j 1 j lt i
  j) sum aj cout ltlt sum ltlt endl
• How many comparisons? How many assignments?

15
Determining Complexity

• Or this one for (i 4 i lt n i) sum
  ai-4 for (j i-3 j lt i j) sum
  aj cout ltlt sum ltlt endl
• How many comparisons? How many assignments?

16
Determining Complexity

• How about a sequential search algorithm?
• Or a binary search algorithm?

17
Best, Worst and Average Cases

• Best case
• Least amount of time to complete a task
• Worst case
• Greatest amount of time
• Average case
• Average amount of time

18
Best, Worst and Average Cases

• Example Linear search through an unsorted array
• Strategy Compare target to array elements, one
  at a time, until target located or array
  exhausted
• Best case
• First element matches target
• One comparison
• Worst case
• Target matches last element or not found
• n comparisons

19
Best, Worst and Average Cases

• Example Linear search through an unsorted array
• Average case
• Given each element has an equal likelihood of
  matching the target
• TAverage(n) (?ii) / n
• (1 n) / n
• (n 1) / 2

20
Best, Worst and Average Cases

• Example Binary search through a sorted array
• Strategy
• Compare target to middle element (key)If found,
  then stopElse, if target less than key, then
  loop using lower portion of rangeElse loop using
  upper portion of range

21
Best, Worst and Average Cases

• Example Binary search through a sorted array
• Best case
• Target matches first element
• One comparison
• Worst case
• Target matches last middle element, or not found
• lg n comparisons

22
Best, Worst and Average Cases

• Example Binary search through a sorted array
• Average case
• Do for homework!

23
Amortized Complexity

• Timings for operation sequences is difficult
• Timing for subsequent operations is often
  dependent on results of previous operations
• Naïve approach is to sum best/worse case timings
  for all operations
• Better approach is to analyze operation sequences
  as opposed to individual operations

24
Amortized Complexity