Algorithm Analysis Review

1
Algorithm Analysis Review

• CS341
• Western Washington University

2
Why Measure Algorithms?

• Algorithms make up computer applications that
  have a great impact on humans
• I.e. life support systems, AEDs, grocery checkout
  systems, automatic teller machines

3
Criteria for Measurement

• Time
• execution time
• usually measured by the number of instructions
• Space
• amount memory algorithm occupies
• usually measured in bytes or MB
• How does space usage affect overall algorithm
  performance?

4
How to Measure

• Hardware
• should compare algorithms using consistent
  hardware
• Why?
• Data
• need to supply all runs of algorithms identical
  data sets
• Why?
• Time units

5
The Towers of Hanoi

• Requires a recursive solution, where N disks
  requires 2N - 1 moves
• if each move require m time units, the solution
  requires (2N - 1)m time units
• What happens as N grows larger?
• What is the growth rate for this algorithm?

6
Big O Notation

• Big O notation defines the order of magnitude for
  an algorithm
• Definition
• Algorithm A is order (Big O) f(N) denoted
  O(f(N)) if constants K and n exist such that A
  requires no more than K f(N) time units to
  solve a problem of size N gt n
• Why the requirement N gt n?

7
An Example

• An algorithm takes N2 - 2 N 10 seconds to
  solve a problem of size N
• What is the order of the algorithm?
• What are K and n?

8
Common Growth Rate Functions

• 1 constant
• growth is independent of the problems size N
• log2N logarithmic
• growth increases slowly compared to the problem
  size
• binary search
• N linear
• growth is directly proportional to the size of
  the problem
• N log2N
• typical of some divide and conquer approaches
• mergesort
• N2 quadratic
• typical of nested loops
• N3 cubic
• more nested loops
• 2N exponential
• growth is extremely rapid and possibly
  impractical
• N! factorial

9
Worst-case and Average-case

• The growth rate of an algorithm can differ even
  though the problem size remains constant
• I.e. search algorithms often require a best case
  and worst case analysis depending on the order of
  the original data set
• Worst-case
• may rarely happen in practice
• algorithm performance will never be slower than
  this
• Average-case
• determines average amount of time to solve a
  problem of size N

10
How to Choose An Algorithm

• After you determine the time (order of magnitude)
  complexity of an algorithm, need to consider how
  it will be used
• Consider the size of N
• I.e. Compare an algorithm that is O(N) with one
  that is O(log2N)?
• Provide values for K and N where O(N) performs
  better
• Dont forget the space requirements

11
Asymptotic Notation

• Big O is an example of asymptotic notation
• Big O provides an upper bound for an algorithms
  time performance
• Other Asymptotic Notation
• Omega notation provides a lower bound for an
  algorithms time performance
• Theta notation is used when an algorithm can be
  bounded both from above and below by the same
  function
• Little oh defines a loose upper bound