Analysis of Algorithms

1
Analysis of Algorithms

• CPS212
• Gordon College

2
Measuring the efficiency of algorithms

• There are 2 algorithms algo1 and algo2 that
  produce the same results.
• How do you choose which algorithm to keep.
• Algorithm Efficiency

3
Measuring the efficiency of algorithms

• How do you determine an algorithms inherent
  efficiency?
• Code them up and compare their running times on a
  lab machine.
• How were they coded? We want to compare the
  algorithms - not their implementations
• What computer do we use? What else is running?
• What data is a fair test of an algorithms
  efficiency.

4
Measuring the efficiency of algorithms

• Goal
• Measure the efficiency of an algorithm -
  independent of implementation, hardware, or data.
• Solution
• Analyze and represent the number of operations
  the algorithm will perform as a function of
  input.
• Example
• Copying an array with n elements requires x
  operations. What is x?

5
Growth rates

• Algo1 requires n2 / 2 operations to solve a
  problem with input size n
• Algo2 requires 5n 10 operations
• Which is the better algorithm for the job?

6
Order of magnitude growth analysis

• A function f(x) is O(g(x)) if and only if there
  exist 2 positive constants, c and n, such that
  f(x) cg(x) for all x gt n

cg(x)
f(x)
number of operations
size of input set
7
Order of magnitude growth analysis

• Asymptotic growth - as the growth approaches
  infinity
• An asymptote of a real-valued function y f(x)
  is a curve which describes the behavior of f as
  either x or y tends to infinity.

cg(x)
f(x)
number of operations
size of input set
8
Order of magnitude growth analysis

• Important points
• Focus only on the growth
• Shape of g(x) is essential
• As the input set grows large (ignore the shape
  for small x)

cg(x)
f(x)
number of operations
size of input set
9
Standard function shapes constant

• O(1)
• Examples?

cg(x)
f(x)
number of operations
size of input set
10
Standard function shapes linear

• O(x)
• Examples?

cg(x)
f(x)axb
number of operations
size of input set
11
Standard function shapes logarithmic

• O(log x) (base 2)

bc a logba c 23 8 log28
3 log(xy) log x log y log(xa) a log x
Examples?
12
Standard function shapes Polynomial

• O(x2)

Examples?
13
Polynomial vs. Linear
14
Polynomial vs. Linear
Example 50n 20 n2
At what point will n2 surpass 50n20? n2
50n20 Solve for xquadratic formula
n2 - 50n - 20 0
n 101/2 50.5 n -1/2
15
Standard function shapes Exponential

• O(cx)

Examples? Hamiltonian Circuit Traveling
Salesman Optimization problems Solution Limit
to small input sets Isolate special cases Find
approximate solution (near optimal)
16
Complexity in action
17
Real Examples

• Searching (sequential)
• Unit of work comparisons
• Best case O(1) theta
• Worst case O(n) theta
• Average Case O(n) theta

18
Real Examples

• Sort (selection)
• Unit of work comparisons and exchanges
• Best case O(n2) theta
• Worst case O(n2) theta
• Average Case O(n2) theta

19
Real Examples

• Search (binary)
• Unit of work comparisons
• Best case O(1) theta
• Worst case O(log n) theta
• Average Case O(log n) theta