Solving%20recurrence%20equations

1
Solving recurrence equations

• 5 techniques for solving recurrence equations
• Recursion tree
• Iteration method
• Induction (Guess and Test)
• Master Theorem (master method)
• Characteristic equations
• We concentrate on the recursion tree method
  (others in the book)

2
Deriving the count using the recursion tree method

• Recursion trees provide a convenient way to
  represent the unrolling of recursive algorithm
• It is not a formal proof but a good technique to
  compute the count.
• Once the tree is generated, each node contains
  its non recursive number of operations t(n) or
  DirectSolutionCount
• The count is derived by summing the non
  recursive number of operations of all the nodes
  in the tree
• For convenience we usually compute the sum for
  all nodes at each given depth, and then sum these
  sums over all depths.

3
Building the recursion tree

• The initial recursion tree has a single node
  containing two fields
• The recursive call, (for example Factorial(n))
  and
• the corresponding count T(n) .
• The tree is generated by
• unrolling the recursion of the node depth 0,
• then unrolling the recursion for the nodes at
  depth 1, etc.
• The recursion is unrolled as long as the size of
  the recursive call is greater than
  DirectSolutionSize

4
Building the recursion tree

• When the recursion is unrolled, each current
  leaf node is substituted by a subtree containing
  a root and a child for each recursive call done
  by the algorithm.
• The root of the subtree contains the recursive
  call, and the corresponding non recursive
  count.
• Each child node contains a recursive call, and
  its corresponding count.
• The unrolling continues, until the size in the
  recursive call is DirectSolutionSize
• Nodes with a call of DirectSolutionSize, are not
  unrolled, and their count is replaced by
  DirectSolutionCount

5
Example recursive factorial

• Initially, the recursive tree is a node
  containing the call to Factorial(n), and count
  T(n).
• When we unroll the computation this node is
  replaced with a subtree containing a root and one
  child
• The root of the subtree contains the call to
  Factorial(n) , and the non recursive count for
  this call t(n) ?(1).
• The child node contains the recursive call to
  Factorial(n-1), and the count for this call,
  T(n-1).

6
After the first unrolling
depth nd T(n)
0 1 ?(1)
1 1 T(n-1)
nd denotes the number of nodes at that depth
7
After the second unrolling
depth nd T(n)
0 1 ?(1)
1 1 ?(1)
2 1 T(n-2)
8
After the third unrolling
depth nd T(n)
0 1 ?(1)
1 1 ?(1)
2 1 ?(1)
3 1 T(n-3)
For Factorial DirectSolutionSize 1 and
DirectSolutionCount ?(1)
9
The recursion tree
depth nd T(n)
0 1 ?(1)
1 1 ?(1)
2 1 ?(1)
3 1 ?(1)
?(1)
n-1 1 T(1) ?(1)
The sum T(n)n ?(1) ?(n)
10
Divide and Conquer

• Basic idea divide a problem into smaller
  portions, solve the smaller portions and combine
  the results.
• Name some algorithms you already know that employ
  this technique.
• DC is a top down approach. Often, you use
  recursion to implement DC algorithms.
• The following is an outline of a divide and
  conquer algorithm

11
Divide and Conquer

• procedure Solution(I)
• begin
• if size(I) ltsmallSize then calculate solution
• return(DirectSolution(I)) use direct solution
• else decompose, solve each and combine
• Decompose(I, I1,...,Ik)
• for i1 to k do
• S(i)Solution(Ii) solve a smaller problem
• end for
• return(Combine(S1,...,Sk)) combine solutions
• end Solution

12
Divide and Conquer

• Let size(I) n
• DirectSolutionCount DS(n)
• t(n) D(n) C(n) where
• D(n) count for dividing problem into
  subproblems
• C(n) count for combining solutions

13
Divide and Conquer

• Main advantages
• Simple code
• Often efficient algorithms
• Suitable for parallel computation

14
Binary search

• Assumption - the list Slowhigh is sorted, and
  x is the search key
• If the search key x is in the list, xSi, and
  the index i is returned.
• If x is not in the list a NoSuchKey is returned
• Books version returns an index i where x would
  be inserted into the sorted list

15
Binary search

• The problem is divided into 3 subproblems
  xSmid, xÎ Slow,..,mid-1, xÎ Smid1,..,high
• The first case xSmid) is easily solved
• The other cases xÎ Slow,..,mid-1, or xÎ
  Smid1,..,high require a recursive call
• When the array is empty the search terminates
  with a non-index value

16

• BinarySearch(S, k, low, high)
• if low gt high then
• return NoSuchKey
• else
• mid ? floor ((lowhigh)/2)
• if (k Smid)
• return mid
• else if (k lt Smid) then
• return BinarySearch(S, k, low, mid-1)
• else
• return BinarySearch(S, k, mid1, high)

17
Worst case analysis

• A worst input (what is it?) causes the algorithm
  to keep searching until lowgthigh
• We count number of comparisons of a list element
  with x per recursive call.
• Assume 2k ? n lt 2k1 k ëlg nû

• T (n) - worst case number of comparisons for
  the call to BS(n)

18
Recursion tree for BinarySearch (BS)

• Initially, the recursive tree is a node
  containing the call to BS(n), and total amount of
  work in the worst case, T(n).
• When we unroll the computation this node is
  replaced with a subtree containing a root and one
  child
• The root of the subtree contains the call to
  BS(n) , and the nonrecursive work for this call
  t(n).
• The child node contains the recursive call to
  BS(n/2), and the total amount of work in the
  worst case for this call, T(n/2).

19
After first unrolling
depth nd T(n)
0 1 1
1 1 T(n/2)
20
After second unrolling
depth nd T(n)
0 1 1
1 1 1
2 1 T(n/4)
21
After third unrolling
depth nd T(n)
0 1 1
1 1 1
1 1 1
T(n/8)
For BinarySearch DirectSolutionSize 0, 1 and
DirectSolutionCount0 for 0, and 1 for 1
22
Terminating the unrolling

• Let 2k? n lt 2k1
• k? ??lg n?
• When a node has a call to BS(n/2k), (or to
  BS(n/2k1))
• The size of the list is DirectSolutionSize since
  ? ?? n/2k ? 1, (or ?? n/2k1 ? 0)
• In this case the unrolling terminates, and the
  node is a leaf containing DirectSolutionCount
  (DFC) 1, (or 0)

23
The recursion tree
depth nd T(n)
0 1 1
1 1 1
2 1 1
k 1 1
T(n)k1?lgn? 1
24
Merge Sort

• Input S of size n.
• Output a permutation of S, such that if i gt j
  then S i ? S j
• Divide If S has at least 2 elements divide into
  S1 and S2. S1 contains the the first ? n/2 ?
  elements of S. S2 has the last ? n/2? elements
  of S.
• Recurs Recursively sort S1 , and S2.
• Conquer Merge sorted S1 and S2 into S.

25
Merge Sort

• Input S1 if (n ? 2) 2 moveFirst( S, S1,
  (n1)/2))// divide3 moveSecond( S, S2, n/2)
  // divide4 Sort(S1) // recurs 5
  Sort(S2) // recurs6 Merge( S1, S2, S ) //
  conquer

26
Merge( S1 , S2, S ) Pseudocode

• Input S1 , S2, sorted in nondecreasing order
• Output S is a sorted sequence.1. while (S1 is
  Not Empty S2 is Not Empty)2. if (S1.first() ?
  S2.first() )3. remove first element of S1
  and move it to the end of S4. else
  remove first element of S2 and move it
  to the end of S5. move
  the remaining elements of the non-empty
  sequence (S1 or S2 ) to S

27
Recurrence Equation for MergeSort.

• The implementation of the moves costs Q(n) and
  the merge costs Q(n). So the total count for
  dividing and merging is Q(n).
• The recurrence relation for the run time of
  MergeSort is T(n) T( ?n/2? ) T( ?n/2? )
  Q(n) . T(1) Q(1)

28
Recursion tree for MergeSort (MS)

• Initially, the recursive tree is a node
  containing the call to MS(n), and total amount of
  work in the worst case, T(n).
• When we unroll the computation this node is
  replaced with a subtree
• The root of the subtree contains the call to
  MS(n) , and the nonrecursive work for this call
  t(n).
• The two children contain a recursive call to
  MS(n/2), and the total amount of work in the
  worst case for this call, T(n/2).

29
After first unrolling of mergeSort
depth nd T(n)
MS(n)
t(n)?(n)
0 1 ?(n)
2T(n/2)
30
After second unrolling
depth nd T(n)
0 1 ?(n)
1 2 ?(n)

4T(n/4)
31
After third unrolling
depth nd T(n)
0 1 ?(n)
1 2 ?(n)

2 4 ?(n)



8T(n/8)
32
Terminating the unrolling

• For simplicity let n 2k
• ?lg n k
• When a node has a call to MS(n/2k)
• The size of the list to merge sort is
  DirectSolutionSize since n/2k 1
• In this case the unrolling terminates, and the
  node is a leaf containing DirectSolutionCount
  ?(1)

33
The recursion tree (excluding the calls)
d nd T(n)
0 1 Q (n)
1 2 Q (n)
2 4 Q (n)
3 8 Q (n)
k 2k Q (n)
T(n) (k1) Q (n) Q( n lg n)
34
The sum for each depth