Title: Introduction to Algorithms
1Introduction to Algorithms
Sept. 2013
2Todays task
- Develop more asymptotic notations
- How to solve recurrences
3T-notation
- Math
- T(g(n)) f(n) there exist positive
constants c1, c2, and n0 such that 0 c1g(n)
f (n) c2g(n) for all n n0 - Engineering
- Drop low-order terms
- ignore leading constants.
4O-notation
- Another asymptotic symbol used to indicate upper
bounds. - Math
- Ex 2n2 O(n3)
- (c 1, n0 2)
- means one-way equality
5Set definition of O-notation
- Math
- Looks better?
- EX 2n2 ? O(n3)
- O-notation corresponds roughly to less than or
equal to. -
6Usage of O-notation
- Macro substitution A set in a formula represents
an anonymous function in the set, if O-notation
only occurs on the right hand of formula. - Ex
- f(n) n3 O(n2)
- Means f(n) n3 h(n)
- for some h(n) ?O(n2), here we can think of h(n)
as error term.
7Usage of O-notation
- If O-notation occurs on the left hand of formula,
such as n2 O(n) O(n2), which means - for any f(n) ?O(n)
- there exists some h(n) ?O(n2),
- makes n2 f(n) h(n)
- O-notation is an upper-bound notation.
- It makes no sense to say f(n) is at least O(n2).
-
8O-notation
- How can we express a lower bound?
- Ex
9T-notation
- O-notation is like
- O-notation is like .
- And T-notation is like
- Now we can give another definition of T-notation
- We also call T-notation as tight bound.
10A Theorem on Asymptotic Notations
- Theorem 3.1
- For any two functions f(n) and g(n), we say
f(n)T(g(n)) if and only if f(n)O(g(n)) and
f(n)O(g(n)). - Ex , how can we
prove that?
11Two More Strict Notations
- We have just said that
- O-notation and O-notation are like and .
- o-notation and ?-notation are like lt and gt.
- Difference with O-notation This inequality must
hold for all c instead of just for 1.
12What does it mean? for any constant c
- With o-notation, we mean that no matter what
constant we put in front of g(x), f(x) will be
still less than cg(x) for sufficiently large n.
No matter how small c is. - Ex 2n2 o(n3)
- An counter example
- 1/2n2 O (n2) does not hold for each c.
(n0 2/c)
13Two More Strict Notations
- We have just said that
- O-notation and O-notation are like and .
- o-notation and ?-notation are like lt and gt.
- Difference with O-notation This inequality must
hold for all c instead of just for 1.
14Solving recurrences
- The analysis of merge sort from Lecture 1
required us to solve a recurrence. - Lecture 3 Applications of recurrences to
divide-and-conquer algorithms. - We often omit some details inessential while
solving recurrences - n is supposed to be an integer because the size
of input is an integer typically . - Ignore the boundary conditions for convenience.
15Substitution method
- The most general method Substitution method
- Guess the form of the solution.
- Verify by induction.
- Solve for constants.
- Substitution method is used to determine the
upper or lower bounds of recurrence.
16Example of Substitution
- EXAMPLE T(n) 4T(n/2) n (n1)
- (Assume that T(1) T(1) )
- Can you guess the time complexity of it?
- Guess O(n3). (Prove O and O separately.)
- We will prove T(n) cn3 by induction.
- First ,we assume that T(k) ck3 for k lt n
- T(k) ck3 holds while kn/2 by assumption.
17Example of Substitution
-
- All we need is
- this holds as long as c
2 - for n 1
18Base Case in Induction
- We must also handle the initial conditions, that
is, ground the induction with base cases. - Base T(n) T(1) for all n lt n0, where n0 is a
suitable constant. - For 1 nlt n0, we have T(1) cn3, if we pick c
big enough. - Here we are!
BUT this bound is not tight !
19A tighter upper bound?
- We shall prove that T(n) O(n2) .
- Assume that T(k) ck2 for k lt n.
- Anybody can tell me
why? - Now we need
n 0 - But it seems
impossible - for n 1
20A tighter upper bound!
- IDEA Strengthen the inductive hypothesis.
Subtract a low-order term. - Inductive hypothesis T(k) c1k2 c2k for klt
n. -
Now we need -
(c2-1) n 0, -
it holds if c2 1
21For the Base Case
- We have proved now that for any value of c1, and
provided c2 1. - Base We need T(1) c1 c2
- Assumed T(1) is some constant.
- We need to choose c1 to be sufficiently larger
than c2, and c2 has to be at least 1. - To handle the initial conditions, just pick c1
big enough with respect to c2.
22About Substitution method
- We have worked for upper bounds, and the lower
bounds are similar. - Try it yourself.
- Shortcomings
- We had to know the answer in order to find it,
which is a bit of a pain. - It would be nicer to just figure out the answer
by some procedure, and that will be the next two
techniques.
23Recursion-tree method
- A recursion tree models the costs (time) of a
recursive execution of an algorithm. - The recursion-tree method can be unreliable, just
like any method that uses dot,dot,dots (). - The recursion-tree method promotes intuition,
however. - The recursion tree method is good for generating
guesses for the substitution method.
24Example of recursion tree
- Solve T(n) T(n/4) T(n/2) n2
25Example of recursion tree
- Solve T(n) T(n/4) T(n/2) n2
26Example of recursion tree
- Solve T(n) T(n/4) T(n/2) n2
27Example of recursion tree
- Solve T(n) T(n/4) T(n/2) n2
?
How many leaves?
ltn
We just need an upper bound.
28Example of recursion tree
- Solve T(n) T(n/4) T(n/2) n2
- lt2n2
- So T(n) T(n2)
?
?
?
?
Recall 11/21/41/8
29The Master Method
- It looks like an application of the recursion
tree method but with more precise. - The sad part about the master method is it is
pretty restrictive. - It only applies to recurrences of the form
- T(n) a T(n/b) f(n) ,
- where a 1, b gt1, and f(n) is asymptotically
positive. - aT(n/b) means every problem you recurse on
should be of the same size.
30Three Common Cases
31Three Common Cases
32Three Common Cases
33Three Common Cases
34Examples
- Ex T(n) 4T(n/2) n
- a 4, b 2
- and
-
-
35Examples
- Ex T(n) 4T(n/2) n2
- a 4, b 2
- and
-
-
36Examples
- Ex T(n) 4T(n/2) n3
- a 4, b 2
- and
-
-
37Homework
- Read Chapter 3 and 4 to be prepared for
applications of recurrences.
38Proof of Master Method
Height ?
logbn
(leaves) ?
39Proof of Master Method
- CASE1 The weight increases geometrically from
the root to the leaves. The leaves hold a
constant fraction of the total weight.
Height ?
logbn
(leaves) ?
40Proof of Master Method
- CASE2(k 0) The weight is approximately the same
on each of the logbn levels.
Height ?
logbn
(leaves) ?
41Proof of Master Method
- CASE3 The weight decreases geometrically from
the root to the leaves. The root holds a constant
fraction of the total weight.
Height ?
logbn
(leaves) ?
42Have FUN !