Chapter Two Algorithm Analysis

1
Chapter TwoAlgorithm Analysis

• Empirical vs. theoretical
• Space vs. time
• Worst case vs. Average case
• Upper, lower, or tight bound
• Determining the runtime of programs
• What about recursive programs?

2
Whats the runtime?

• int n
• cin gtgt n
• for (int i0 iltn i)
• for (int j0 jltn j)
• for (int k0 kltn k)
• cout ltlt Hello world!\n

2n3n2n2?
O(n3) runtime
What if the last line is replaced by string
snew string(Hello world!\n)
O(n3) time and space
3
Resource Analysis

• Runtime wed like to count the steps but that
  would be machine dependent
• Space we may also be interested in space usage

• ? ignore constant factors, use O() notation
• ? count steps equivalent to machine language
  instructions
• ? count the bytes used

4
Asymptotic notation

• g(n) is said to be O(f(n)) if there exist
  constants c and n0 such that g(n) lt c f(n) for
  all n gt n0
• g(n) is said to be W(f(n)) if there exist
  positive constants c and n0 such that 0 lt c f(n)
  lt g(n) for all n gt n0
• g(n) is said to be Q(f(n)) if g(n) O(f(n)) and
  g(n) W(f(n))
• O like lt for functions (asymptotically
  speaking)
• W like gt
• Q like
• ? for all n gt n0
• ? ignore constant factors, lower order terms

5
Asymptotic notation examples

• Asymptotic runtime, in terms of O, W, Q?
• Suppose the runtime for a function is
• n2 2n log n 40
• 0.0000001 n2 1000000n1.999
• n3 n2 log n
• n2.0001 n2 log n
• 2n 100 n2
• 1.00001n 100 n97

6
Asymptotic comparisons

• 0.0000001 n2 O(1000000n1.999 )?
• n1.000001 O(n log n)?
• 1.0001n O(n943)?
• lg n Q(ln n)?
• (Compare the limit of the quotient of the
  functions)

No a polynomial with a higher power dominates
one with a lower power
No all polynomials (n.000001) dominate any
polylog (log n)
No all exponentials dominate any polynomial
Yes different bases are just a constant factor
difference
7
Whats the runtime?

• int n
• cin gtgt n
• for (int i0 iltn i)
• for (int j0 jltn j)
• for (int k0 kltn k)
• cout ltlt Hello world!\n
• for (int i0 iltn i)
• for (int j0 jltn j)
• for (int k0 kltn k)
• cout ltlt Hello world!\n

Q(n3) Q(n3) Q(n3) Statements or blocks in
sequence add
8
Whats the runtime?

• int n
• cin gtgt n
• for (int i0 iltn i)
• for (int jn jgt1 j/2)
• cout ltlt Hello world!\n

Loops add up cost of each iteration (multiply
loop cost by number of iterations if they all
take the same time) log n iterations of n steps
? Q(n log n)
9
Whats the runtime?

• int n
• cin gtgt n
• for (int i0 iltn i)
• for (int j0 jlti j)
• cout ltlt Hello world!\n

Loops add up cost of each iteration 1 2 3
n n(n1)/2 O(n2)
10
Whats the runtime?

• template ltclass Itemgt
• void insert(Item a, int l, int r)
• int i
• for (ir igtl i--) compexch(ai-1,ai)
• for (il2 iltr i)
• int ji Item vai
• while (vltaj-1)
• aj aj-1 j--
• aj v

11
Whats the runtime?

• void myst(int n)
• if (nlt100)
• for (int i0 iltn i)
• for (int j0 jltn j)
• for (int k0 kltn k)
• cout ltlt Hello world!\n
• else
• for (int i0 iltn i)
• for (int j0 jltn j
• cout ltlt Hello world!\n

12
Estimate the runtime

• Suppose an algorithm has runtime Q(n3)
• suppose solving a problem of size 1000 takes 10
  seconds. How long to solve a problem of size
  10000?
• Suppose an algorithm has runtime Q(n log n)
• suppose solving a problem of size 1000 takes 10
  seconds. How long to solve a problem of size
  10000?

runtime 10-8 n3 if n10000, runtime 10000s
2.7hr
runtime 10-3 n lg n if n10000, runtime 133 secs
13
Worst vs. average case

• You might be interested in worst, best, or
  average case analysis of an algorithm
• You can have upper, lower, or tight bounds on
  each of those functions.
• Eg. For each n, some problem instances of size n
  have runtime n and some have runtime n2.
• Worst case
• Best case
• Average case

Q(n2), W(n), W(log n), O(n2), O(n3)
W(n), W(log n), O(n2), Q(n)
W(n), W(log n), O(n2), O(n3)
Average case need to know distribution of inputs
14
The Taxpayer Problem

• Tax time is coming up. The IRS needs to process
  tax forms. How to access and update each
  taxpayers info?
• ADT?
• ADT Dictionary find(x), insert(x), delete(x)
• Implementation?

15
Array Implementation

• Insert(x)
• Find(k)
• Delete(I)

Time for n Operations?
RecordsnumRecs x Runtime
O(1)
O(n2)
For (I0 IltnumRecs I) if (recordsI.key
k) return I Runtime
O(n)
recordsIrecords--numRecs Runtime
O(1)
16
Sorted Array Implementation

• Find(x)
• Runtime?

• int bot1, topnumRecs-1, mid
• while (bot lt top)
• mid (bot top)/2
• if (datamidx) return mid
• if (datamidltx) topmid-1 else botmid1
• return 1

17
Analysis of Binary Search

• How many steps to search among n items?
• Number of items eliminated at each step?
• Definition of lg(x)?
• Runtime?

O(log n)
18
Sorted Array, cont.

• Insert(x)?
• Delete(x)?
• Time for n insert, delete, and find ops?

O(n)
O(n)
O(n2)
19
Which implementation is better?

• find(x) insert(x) delete(x)
• Array
• S. Array
• Worst case for n operations?
• Array
• Sorted Array

O(n) O(1) O(1)
O(log n) O(n) O(n)
O(n2)
O(n2)
What if some operations are more frequent than
others?
20
Molecule viewer example

• Java demos molecule viewer
• Example1
• Example2
• Example3

21
Molecule Viewer Source Snippet

• /
• I use a bubble sort since from one iteration
  to the next, the sort
• order is pretty stable, so I just use what I
  had last time as a
• "guess" of the sorted order. With luck, this
  reduces O(N log N)
• to O(N)
• /
• for (int i nvert - 1 --i gt 0)
• boolean flipped false
• for (int j 0 j lt i j)
• int a zsj
• int b zsj 1
• if (va 2 gt vb 2)
• zsj 1 a
• zsj b
• flipped true
• if (!flipped) break

22
Merge sort runtime?

• void mergesort(first, last)
• if (last-first gt 1)
• mid(last-first)/2 first
• mergesort(first, mid)
• mergesort(mid1,last)
• merge(first, mid, last)

T(n) 2T(n/2) c n T(1) b
Called a recurrence relation
23
Recurrence relations

• In Discrete Math youll learn how to solve
  these.
• In this class well say Look it up.
• But you will be responsible for knowing how to
  write down a recurrence relation for the runtime
  of a program.

Divide-and-conquer algorithms like merge sort
that divide problem size by 2 and use O(n) time
to conquer T(n) 2T(n/2) c n have runtime
O(n log n)
24
Hanoi runtime?

• void hanoi(n, from, to, spare
• if (n gt 0)
• hanoi(n-1,from,spare,to)
• cout ltlt from ltlt ltlt to ltlt endl
• hanoi(n-1,spare,to,from)

T(n) 2T(n-1) c T(0) b
Look it up T(n) O(2n)
25
Hanoi recurrence solution

• T(n)2T(n-1)c
• T(n-1) 2T(n-2) c
• T(n-2) 2T(n-3) c
• _______
• T(n) 2T(n-1) c
• 2 2T(n-2) c c
• 22 T(n-2) 2 c c
• 23 T(n-3) 22 c 21 c 20 c
• 2k T(n-k) 2k-1 c 2k-2 c 21 c 20
  c
• 2k T(n-k) c(2k 1)
• Done when n-k0 since we know T(0).
• T(n) 2n b c 2n - c
• Q(2n)

26
Binary Search recurrence?

• Recurrence relation?
• T(n)T(n/2)c T(1) b
• Look it up Q(log n)