CS100J Lecture 27

1
CS100J Lecture 27

• Previous Lecture
• Interfaces
• Comparable
• Reflection
• super
• Re-Reading
• Lewis Loftus, Section 5.4
• Savitch, Appendix 7
• This Lecture
• Analysis of algorithms and their complexity
• Time Complexity
• Space Complexity
• Asymptotic Complexity
• (Worst, Best, Average) Case Analysis
• Examples
• Sequential Search vs. Binary Search
• Selection Sort vs. Merge Sort
• Recursion

2
Complexity Analysis

• Time or space needed to run an algorithm.
• Absolute measures e.g., seconds or bytes.
• Weaknesses
• depends on hardware/software specifics
• depend on input data
• size of data
• specific data values

3
Complexity Analysis, continued

• Parametric measures
• To avoid hardware/software dependencies
• Count abstract steps or values, not seconds or
  bytes. E.g., of additions and multiplications
  rather than seconds.
• To avoid dependence on specific data
• Consider worst possible data,
• Or best possible data,
• Or average complexity over all possible data.
• To avoid dependence on size of data
• Let n be the size of the problem.
• Indicate complexity by some analytical function
  of n. E.g., steps needed to sort n integers is 7
  n2 23.
• Weakness
• Writing an exact function is too hard due to
  irregularities in code. Also, exact coefficients
  are not so important.

4
Complexity Analysis, continued

• Asymptotic measures
• Focus on upper bounds.
• The running time is no worse than ..."
• Ignore finite number of exceptions.
• "For all inputs at least as large as N, the
  running time is no worse than ..."
• Focus on growth rates. Ignore coefficients and
  low-order terms.
• "For all inputs at least as large as N, the
  running time to sort n values is no worse than k
  n2, for some k gt 0.
• This is written as
• "Time to sort n values is O(n2)."

5
Asymptotic Bounds

• Let f(x) and g(x) be mathematical functions.
• Definition. Function f is asymptotically bounded
  by g if, for some N ³ 0 and k ³ 0, the inequality
• k g(x) ³ f(x)
• holds for all x ³ N.
• If f is asymptotically bounded by g, we write
• f O(g)
• read "f is big Oh of g".

6
Sequential Search

• / Given A0..n sorted in non-decreasing order,
  return the subscript of an occurrence of val in A
  (if val occurs in A) or n1 otherwise. /
• int find(int A, int n, int val)
• int k 0
• while (k lt n val ! Ak) k
• return k
• Worst case O(n)
• Best case O(1)
• Average case O(n)

7
Binary Search

• / Given A0..n sorted in non-decreasing order,
  return the subscript of an occurrence of val in A
  (if val occurs in A) or n1 otherwise. /
• int find(int A, int n, int val)
• int L 0
• int R n
• / Make LR s.t. if val is in A0..n, then
• AL val. /
• while ( L ! R )
• / Reduce the size of the interval AL..R
• approximately in half, preserving the
• property that if val was in the
• original interval, then it is in the
• new interval. /
• int M (L R) / 2
• if ( AM gt val )
• R M
• else
• L M1

8
Selection Sort

• / Sort A0..n into non-decreasing order. /
• public void sort(int A, int n)
• for ( int k 0 k lt n k )
• / Given that A0..k-1 is finished,
• finish A0..k. /
• int minLoc // subscript of
• // smallest in Ak..n
• / Set minLoc so AminLoc is
• smallest in Ak..n. /
• minLoc k
• for (int ik1 i lt n i)
• if (Ai lt AminLoc)
• minLoc i
• / Exchange Ak and AminLoc /

9
Merge Sort (in MatLab)

• function s mysort(a)
• sort vector a into nodecreasing order.
• n length(a)
• if n lt 1
• s a
• else
• s merge( mysort( a(1 n/2) ), ...
• mysort( a(n/21 n) ) )
• end
• function m merge(a,b)
• Given sorted vectors a and b,
• return sort(a,b).
• n length(a) length(b)
• m zeros(1,n)
• a a, inf b b, inf
• ia 1 ib 1

10
Merge Sort, continued

• Let T(n) be number of "steps" needed to sort n
  items using Merge Sort.
• Then
• T(1) 1
• T(n) 2 T(n / 2) n, for even n.
• n T(n) n(log2 n 1 n2
• 1 1 1 0 1 1 1
• 2 2 1 2 4 2 1 1 4 4
• 4 2 4 4 12 4 2 1 12 16
• 8 2 12 8 32 8 3 1 32 64
• 16 2 32 16 80 16 4 1 80 256
• 32 2 80 32 192 32 5 1 192
  1024
• 1024 2 5120 1024 1024 101
  1,048,576
• 11,264 11,264
• All cases O(n log n)