Merge Sort Recurrence

1
Merge Sort Recurrence

• Actual recurrence is
• But we typically assume that n 2k where k is an
  integer and use the simpler recurrence.

2
Substitution Method(Constructive Induction)

• Guess the form of the solution.
• Use mathematical induction to find constants or
  show that they can be found and to prove that the
  answer is correct.

3
Substitution

• Goal
• Derive a function of n that is not a recurrence
  so we can establish an upper and/or lower bound
  on the recurrence

4
Constructive Induction

• Suppose T includes a parameter n and n is a
  natural number (positive integer)
• Instead of proving directly that T holds for all
  values of n, prove
• T holds for a base case b (often n 1)
• For every n gt b, if T holds for n-1, then T holds
  for n.
• Assume T holds for n-1
• Prove that T holds for n follows from this
  assumption

5
Example of Substitution

• Determine upper bound for
• T(n) 2T( ) n
• Guess solution T(n) O(n lg n)
• Confirm by proving that T(n) cn lg n for
  appropriate choice of c.

6
Substitution Example (continued)

• Assume bound holds for , i.e.
• T( ) c lg ( )
• Substituting into recurrence
• T(n) 2(c lg ( )) n
• cn lg (n/2) n
• cn lg n cn lg 2 n
• cn lg n cn n
• cn lg n (as long as c 1)

7
Substitution Example (continued)

• Choose c large enough so the bound
• T(n) cn lg n
• works for the boundary conditions as well.
• Assume boundary condition T(n) 1 if n 1
• Substituting T(1) c1 lg 1 0
• Conflicts with T(1) 1.

8
Substitution Example (continued)

• Need only prove
• T(n) cn lg n for n n0.
• Observe Can replace base case T(1) with base
  cases T(2) and T(3).
• I.e., let n0 2.

9
Iteration

• Expand (iterate) the recurrence and express it as
  a summation of terms dependent only on n and the
  initial conditions.
• Use techniques for evaluating summations to
  provide bounds on the solution.

10
Example of Iteration

• Determine upper bound for
• T(n) 3T( ) n
• Iterate
• T(n) n 3T( )
• n 3( 3T( ))
• n 3( 3( 3T(
  )))
• n 3 9 27T(
  )

11
Iteration Example (continued)

• The ith term in the series is 3i
  .
• Iteration hits n 1 when i gt log4n.
• T(n) n 3n/4 9n/16 27n/64 3?log4n
  T(1)
• n T(n?log43)
• 4n o(n)
• O(n)

12
Recursion Tree

• Each node represents cost of a single subproblem
  somewhere in the set of recursive function
  invocations.
• Sum costs within each level of tree to obtain set
  of per-level costs.
• Sum per-level costs to determine total cost.

13
Example of Recursion Tree

• Recurrence T(n) 3T( ) T(n2)
• First Find upper bound for the solution.
• (Ignore floor. Write out the implied
• constant c gt 0.)
• Create tree for T(n) 3T(n/4) cn2
• Assume (for simplicity) n is power of 4

14
Recursion Tree Example (continued)

• Begin with T(n)

15
Recursion Tree Example (continued)

• Expand each node with cost T(n/4)

16
Recurrence Tree Example (continued)

• Determine cost at each level of tree.
• Number of nodes at depth i is 3i.
• Cost of node at depth i is c(n/4i)2 for i 0, 1,
  2, , log4n 1.
• Therefore total cost of all nodes at depth i is
  3ic(n/4i)2 (3/16)icn2.
• Last level (depth log4n) has 3?log4n n?log43
  nodes, each contributing a cost of T(1), for
  total of n?(log43)T(1), which is T(n?log43).

17
Example of Recurrence Tree (continued)

• Now add up the costs over all levels.
• T(n) cn2 (3/16)cn2 (3/16)2cn2
  (3/16)?(log4n 1) cn2 T(n?log43)
• 
  T(n?log43)
• cn2
  T(n?log43)

18
Example of Recurrence Tree (continued)

• Simplify by backing up one step and using a
  geometric series as an upper bound
• T(n)
  T(n?log43)
• lt T(n?log43)
• T(n?log43)
  T(n?log43) O(n2)

19
Example of Recurrence Tree (continued)

• Use substitution to verify the guess that T(n)
  O(n2) is an upper bound for the recurrence.
• I.e., show T(n) dn2 for some constant d gt 0.
• Using same constant c as before
• T(n) 3T( ) cn2
• 3d 2 cn2
• 3d(n/4)2 cn2
• (3/16) dn2 cn2
• dn2 as long as d (16/13)c

20
Master Method

• Provides cookbook for solving recurrences of
  form T(n) aT(n/b) f(n) where a 1 and b gt 1
  are constants and f(n) is an asymptotically
  positive function.
• Requires memorization of 3 cases.

21
Master Theorem

• Let a 1 and b gt 1 be constants, let f(n) be a
  function, and let T(n) be defined on the
  nonnegative integers by the recurrence
• T(n) aT(n/b) f(n)
• where we interpret n/b to mean either
  or . Then T(n) can be bounded
  asymptotically as follows.
• If f(n) O(n?(logba-e)) for some constant e gt
  0, then T(n)
  T(n?logba).
• If f(n) T(n?logba), then T(n) T(n?(logba) lg
  n).
• If f(n) O(n?(logbae)) for some constant e gt
  0, and if af(n/b) cf(n)
• for some constant c lt 1 and all sufficiently
  large n, then T(n) T(f(n)).

22
Examples of Master Method

• Example 1 T(n) 9T(n/3) n
• a 9, b 3, f(n) n
• n?logba n?log39 T(n2)
• Since f(n) O(n?(log39-e)), where e1,
• case 1 applies, yielding solution
• T(n) T(n2)

23
Examples of Master Method (continued)

• Example 2 T(n) T(2n/3) 1
• a 1, b 3/2, f(n) 1
• n?logba n?log3/21 n0 1
• Since f(n) T(n?logba) T(1), case 2
• applies, yielding solution
• T(n) T(lg n)

24
Examples of Master Method (continued)

• Example 3 T(n) 3T(n/4) n lg n
• a 3, b 4, f(n) n lg n
• n?logba n?log43 O(n0.793)
• Since f(n) O(n?((log43 e) where e 0.2,
• case 3 applies if we can show the regularity
• condition holds for f(n).
• For sufficiently large n, af(n/b) 3(n/4) lg
  (n/4)
• (3/4)n lg n cf(n) for c ¾.
• Therefore case 3 applies, yielding solution
• T(n) T(n lg n)

25
Examples of Master Method (continued)

• Example 4 T(n) 2T(n/2) n lg n
• a 2, b 2, f(n) n lg n
• n?logba n?log22 n
• f(n) n lg n is asymptotically larger than
• n?logba n, but its not polynomially larger.
• f(n)/(n?logba) (n lg n)/n is asymptotically
• less than ne for any positive constant e.
• So T(n) falls in gap between cases 2 and 3.