CS 312: Algorithm Analysis

1
CS 312 Algorithm Analysis

• Lecture 07 Recurrence Relations
• a.k.a. Difference Equations

2
Announcements

• HW 5 Due Today
• Questions about the Master Theorem
• Project 1
• Help Session with Michael D.
• Howd it go?
• Another Help Session with David M.
• 1/25/07 (Thu)
• 1pm
• 1121 TMCB
• Questions?
• Early Day is next Friday
• Due Date is the Monday thereafter

3
Objectives for Today

• Lecture 6 Divide and Conquer
• Introduce Divide and Conquer
• Apply to Multiplication
• Analyze by defining a recurrence relation
• Analyze for Big-theta bound
• Introduce the Convex Hull problem
• If time permits, analyze binary search using
  Master Theorem

• Lecture 7 Recurrence Relations
• Go beyond the Master Theorem
• Analyze Recursive Algorithms, especially for
  Divide and Conquer
• Recurrence Relations
• Also known as Difference Equations
• Today Special Type
• Homogeneous
• Linear
• Constant Coefficients

4
Recall Analysis Framework

• Define measure of input size
• Identify algorithms elementary operations
• Time efficiency is measured by counting the
  number of times an elementary operation is
  executed.
• Space efficiency is measured by counting the
  number of memory units consumed by the algorithm
• The efficiency of some algorithms may differ
  significantly for inputs of the same size.
• Distinguish between worst-case, average-case, and
  best-case efficiencies by plotting efficiency vs.
  input size
• Establish order of growth. Use asymptotic
  notation.

5
Asymptotic Notation
6
Nonrecursive Algorithms

• Example Element Uniqueness Problem, check
  whether all elements in an array are distinct.

function UniqueElements(A0n-1) for i0
to n-2 for j i1 to n-1
if AiAj return false return true

• Define input size
• Define elementary operation
• Distinguish worst-case

7
Recursive Algorithms

• Example Compute Factorial function N!

function Factorial(N) if n0 return 1 else
return Factorial(N-1)N

• Define input size
• Define elementary operation
• Distinguish worst-case

What series is generated by this recurrence
relation?
In order to answer that question we need an
initial condition!
Appeal to Theory of Recurrence Relations /
Difference Equations
8
Recursive Algorithms

• Example Tower of Hanoi, move all disks to third
  peg without ever placing a larger disk on a
  smaller one.

9
Recursive Algorithms

• Example Tower of Hanoi, move all disks to third
  peg without ever placing a larger disk on a
  smaller one.

10
Recursive Algorithms

• Example Tower of Hanoi, move all disks to third
  peg without ever placing a larger disk on a
  smaller one.

11
Recursive Algorithms

• Example Tower of Hanoi, move all disks to third
  peg without ever placing a larger disk on a
  smaller one.

12
Recursive Algorithms

• Example Tower of Hanoi, move all disks to third
  peg without ever placing a larger disk on a
  smaller one.

13
Recursive Algorithms

• Example Tower of Hanoi, move all disks to third
  peg without ever placing a larger disk on a
  smaller one.

Can you figure out an explicit formula for this
sequence?

• Substitution
• Linear Analysis (Appeal to Theory of Recurrence
  Relations)

14
Recurrence Relations

• Given a sequence of (real) numbers y(k), a
  recurrence relation is an equation relating the
  value y(k), at a point k, to values at other
  points (usually neighboring points).

15
Linear Recurrence Relations

• Theorem a recurrence relation of order n is
  linear if and only if it can be written as
• where x(k) is arbitrary and ai(k) are functions
  independent of y for all i0,1,2,.

16
Linear Recurrence Relations

• Theorem a recurrence relation of order n is
  linear if and only if it can be written as
• where x(k) is arbitrary and ai(k) are functions
  independent of y for all i0,1,2,.

17
Terminology

• The ai(k)s in these equations are called the
  coefficients of the linear equation.
• If these coefficients do not depend on k, the
  equation is said
• to have constant coefficients
• or to be time-invariant.
• The function x(k) is called
• the forcing term
• the driving term
• the system input
• or simply the right-hand side.
• A solution of the recurrence relation is a
  function y(k) (in terms of k) that satisfies the
  recurrence relation.
• A recurrence relation is said to be homogeneous
  ify(k) 0 (for all k) is a solution of the
  equation.

18
Big Picture

• Bear with me now while we look at the big picture

19
Recurrence Relations
So why should we care about recurrence relations?
Generalizes the idea of a function to an operator
or dynamic system
A function is like a table that takes a number in
and gives a number out.
20
Recurrence Relations
So why should we care about recurrence relations?
Generalizes the idea of a function to an operator
or dynamic system
Functions can also take in a vector of numbers or
yield a vector out.
21
Recurrence Relations
So why should we care about recurrence relations?
Generalizes the idea of a function to an operator
or dynamic system
What if the input and output vectors of a
function were infinitely long?
Then x and y themselves are functions!
f becomes a map from one function to another!
We call maps that take functions as inputs or
generate functions as outputs operators, or
systems.
An operator is like a rule, its the mathematical
description of a subroutine!
22
Recurrence Relations
So why should we care about recurrence relations?
Generalizes the idea of a function to an operator
or dynamic system
What if the input and output vectors of a
function were infinitely long?
Then x and y themselves are functions!
y(k)
x(k)
S
f becomes a map from one function to another!
We call maps that take functions as inputs or
generate functions as outputs operators, or
systems.
An operator is like a rule, its the mathematical
description of a subroutine!
23
Recurrence Relations
So why should we care about recurrence relations?
Generalizes the idea of a function to an operator
or dynamic system
What if the input and output vectors of a
function were infinitely long?
Then x and y themselves are functions!
y(k)
x(k)
S
f becomes a map from one function to another!
We call maps that take functions as inputs or
generate functions as outputs operators, or
systems.
Unlike a simple function, an operator can have
local variables that store intermediate
calculationsjust like a subroutine. In other
words, S is a map that can have memory!
An operator is like a rule, its the mathematical
description of a subroutine!
24
Recurrence Relations
So why should we care about recurrence relations?
Generalizes the idea of a function to an operator
or dynamic system
What if the input and output vectors of a
function were infinitely long?
Then x and y themselves are functions!
y(k)
x(k)
S
f becomes a map from one function to another!
We call maps that take functions as inputs or
generate functions as outputs operators, or
systems.
Subroutine Description function S(x,k) if
k0 then y f yo else y f w x
w f y return y
An operator is like a rule, its the mathematical
description of a subroutine!
25
Recurrence Relations
So why should we care about recurrence relations?
Generalizes the idea of a function to an operator
or dynamic system
What if the input and output vectors of a
function were infinitely long?
Then x and y themselves are functions!
y(k)
x(k)
S
f becomes a map from one function to another!
We call maps that take functions as inputs or
generate functions as outputs operators, or
systems.
An operator is like a rule, its the mathematical
description of a subroutine!
26
Recurrence Relations
So why should we care about recurrence relations?
Generalizes the idea of a function to an operator
or dynamic system
What if more than one number is stored in memory?
y(k)
x(k)
S
Subroutine Description function S(x,k) if
k0 then y f yo else if k1 then y f y1
else y f a1w1 aow0x w0 f w1 w1
f y return y
27
Order of Recurrence Relations
So why should we care about recurrence relations?
Generalizes the idea of a function to an operator
or dynamic system
What if more than one number is stored in memory?
y(k)
x(k)
S
We call the number of values stored in
memory the order of the system.
Subroutine Description function S(x,k) if
k0 then y f yo else if k1 then y f y1
else y f a1w1 aow0x w0 f w1 w1
f y return y
28
Recurrence Relations
So why should we care about recurrence relations?
Generalizes the idea of a function to an operator
or dynamic system
What if more than one number is stored in memory?
y(k)
x(k)
S
We call the number of values stored in
memory the order of the system.
29
Recurrence Relations
So why should we care about recurrence relations?
Generalizes the idea of a function to an operator
or dynamic system.
Characterizes a function y(k) by rules instead of
by a table of its values!
Example Complexity of solution for Tower of
Hanoi problem
Example Capital growth in your savings account
y(n) - in account starting day n i -
interest earned each day D(n) - Deposit on day
n W(n) - Withdraw on day n
30
Big Picture

• How was that?

31
Terminology
32
Example
33
Solution?
34
Assignment

• Read Recurrence Relations Notes, Part II
• HW 6 Part I Exercises (Section 1.3)