Lectures on Recursive Algorithms - PowerPoint PPT Presentation

About This Presentation
Title:

Lectures on Recursive Algorithms

Description:

Graphs - basic definitions and examples. These lectures: Recursive algorithms ... In other words: recursive algorithm must store a path from the root to the ... – PowerPoint PPT presentation

Number of Views:28
Avg rating:3.0/5.0
Slides: 49
Provided by: DarekKo5
Category:

less

Transcript and Presenter's Notes

Title: Lectures on Recursive Algorithms


1
Lectures onRecursive Algorithms
  • COMP 523 Advanced Algorithmic Techniques
  • Lecturer Dariusz Kowalski

2
Announcements
  • Web page
  • www.csc.liv.ac.uk/darek/comp523.html
  • Slides constantly updated based on your feedback
    check for a new version a few hours after the
    lecture(s)

3
Overview
  • Previous lectures
  • Algorithms correctness, termination, performance
  • Asymptotic notation
  • Popular asymptotic complexities log n, n, n log
    n, n2, ...
  • Graphs - basic definitions and examples
  • These lectures
  • Recursive algorithms - basic recursion
  • Searching algorithm in time O(log n)
  • Finding majority in time O(n)
  • Sorting in time O(n log n)
  • Other examples

4
Algorithms based on recurrence
  • Algorithmic scheme
  • Reduce the input to smaller sub-input(s)
  • Solve the problem for (some) sub-inputs
  • Merge the obtained solution(s) into one global
    solution

5
Complexity analysis of recursive process
  • Let T(m) denote time (or other complexity
    measure) of solving a given problem by a given
    algorithm working on input length m
  • Simple recursive formula
  • T(n) ? qT(n/2) f(n)
  • Solve the problem for half of the input q times,
    and spend at most f(n) time for partitioning
    and/or merging

6
Example 1Searching in time O(log n)
7
Searching problem and algorithm
  • Input sorted array A of n numbers and number x
  • Problem find if x is in array A
  • Solution Algorithm SEARCH(x,1,n)
  • Procedure SEARCH(x,i,j)
  • If i j then
  • if x Ai then answer YES
  • if x ? Ai then answer NO
  • Else compare x with element A(ij)/2
  • If x A(ij)/2 then answer YES
  • If x lt A(ij)/2 then SEARCH(x, i, (ij)/2-1)
  • If x gt A(ij)/2 then SEARCH(x, (ij)/21, j)

8
Tree structure of searching
root
4
find if value 5 is there
6
2
height 3
1
3
5
7
1
2
3
4
5
6
7
8
8 leaves
9
Time and memory consumption
  • Each recursive call
  • needs additional constant memory, and
  • the size of input decreases by factor 2
  • Formula for time ( comparisons)
  • T(n) ? T(n/2) 4
  • Additional memory size log n O(log n)
  • In other words recursive algorithm must store a
    path from the root to the searched leaf, and it
    has logarithmic length

10
Case q 1, f(n) c
  • T(n) ? T(n/2) c , T(2) ? c
  • T(n) ? T(n/2) c
  • ? (T((n/2)/2) c) c T(n/4) 2c
  • ? (T((n/4)/2) c) 2c T(n/8) 3c
  • ? (T(n/2i) c) (i-1) . c T(n/2i) i . c
  • ? (T(n/2log n 1) c) (log n - 2) . c
  • T(n/(n/2)) (log n - 2) . c T(2) (log n
    1) . c
  • ? c (log n 1) . c c log n

11
Case q 1, f(n) cformal analysis
  • T(n) ? T(n/2) c
  • T(2) ? c
  • Solution T(n) ? c log n
  • Proof by induction.
  • For n 2 straightforward T(2) ? c ? c log 2
  • Suppose T(n/2) ? c log (n/2) then
  • T(n) ? T(n/2) c ? c log (n/2) c
  • ? c log n - c c ? c log n

12
Example 2Finding a majority in time O(n)
13
Finding a majority
  • Input array A of n elements
  • Problem find if there is an element x that
    occurs in array A more than n/2 times (a majority
    value)
  • Solution Algorithm Majority(A) returning a
    majority element in A, if it exists, or null
    otherwise

14
Finding a majority algorithm
  • Algorithm Majority(A1..n)
  • If A 0 then output null, else if A 1 then
    output A1 else
  • If n A is odd then
  • check whether An is a majority in A by counting
    the number of occurrences of value An
  • if yes then output An, otherwise decrease n by
    one
  • Initialize additional array B of size A/2
  • Set j to 0
  • For i 1,2,,n/2 do
  • if A2i-1 A2i then
  • increase j by one
  • set Bj to A2i
  • Find if there is a majority in B1..j by
    executing Majority(B1..j)
  • If a majority value x in B1..j is returned then
    check whether x is a majority in A, by going
    through array A and counting the number of
    occurrences of value x in A if successful output
    x otherwise null

15
Tree structure of finding majority
root
2
odd, no majority
height 2
2
2
1
2
2
2
2
1
2
1
1
8 leaves
16
Correctness of Majority Algorithm
  • Lemma (to be proved later)
  • If x is a majority in A then x is a majority in
    B.
  • Correctness (proof by induction, based on the
    Lemma)
  • Inductive assumption
  • Majority(B) works correctly for all Bs of size
    smaller than A
  • Case 1 There is a majority in A.
  • Then it is a majority in B, by Lemma, therefore
    it will be found in the recursive stage of the
    algorithm Majority(B) (by inductive argument),
    and will be checked (successfully) in the last
    point of the algorithm Majority(A).
  • Case 2 There is no majority in A.
  • Then even if there is a candidate value found in
    the recursive stage Majority(B) of the algorithm,
    it will be rejected anyway in the last point of
    the algorithm Majority(A).

17
Proof of the Lemma
  • Suppose, to the contrary, that x is a majority in
    A but is not a majority in B. Let
  • m be the number of occurrences of x in A, and
  • k be the number of occurrences of x in B.
  • It follows that the other values appear at least
    k times in B.
  • Consequently, the other values appear in A
  • at least 2k times those pairs that are
    represented in B by a value different than x
    plus
  • m-2k times each occurrence of x in A that is
    not paired by another x (there are k pairs xx in
    A, thus m-2k of other occurrences of x in A) is
    paired by some other value (different than x)
  • which gives at least 2k(m-2k) m occurrences in
    total, and contradicts the fact that x is a
    majority in A. Contradiction! Therefore x must be
    also a majority in B.

18
Memory consumption
  • Each call to the recursive procedure
  • needs additional memory of half of the current
    input size
  • reduces the size of the input by factor at least
    two
  • Formula for time ( comparisons)
  • T(n) ? n -1 3n/2 T(n/2) n T(n/2) 7n/2
  • Additional memory used (for smaller arrays B)
  • M(n) ? n/2 M(n/2) ? n/2 n/4 M(n/4)
  • ? n/2 n/4 1 n - 1 O(n)
  • Intuition about memory size the algorithm needs
    a binary tree of linear size, in the worst case,
    to store winners

19
Case q 1, f(n) cn
  • T(n) ? T(n/2) cn , T(1) ? c
  • T(n) ? T(n/2) cn
  • ? (T(n/4) cn/2) cn T(n/4) (3/2)cn
  • ? (T(n/8) cn/4) (3/2)cn T(n/8)
    (7/4)cn
  • ? (T(n/2i) cn/2i-1) ((2i-1-1)/2i-2) . cn
  • T(n/2i) ((2i-1)/2i-1) . cn
  • ? T(n/2log n) ((2log n-1)/2log n-1) . cn
  • T(1) (n 1)/(n/2) . cn ? c 2cn 2c
    ? 2cn

20
Case q 1, f(n) c nformal analysis
  • T(n) ? T(n/2) c n
  • T(1) ? c
  • Solution T(n) ? 2c?n
  • Proof by induction.
  • For n 1 straightforward T(1) ? c ? 2c ? 1
  • Suppose T(n/2) ? 2c?(n/2) then
  • T(n) ? T(n/2) c?n ? 2c?(n/2) c n
  • 2c?n

21
Textbook and exercises
  • READING Chapter 5, up to Section 5.2
  • EXERCISES
  • Solve the following recursive formulas the best
    you can
  • T(n) ? T(n/2) 4
  • T(n) ? T(n/2) 5n
  • T(n) ? T(n/2) 3n2
  • T(n) ? (3/2)?T(n/2) 1
  • Compare and sort the obtained (asymptotic)
    formulas according to the asymptotic order. For
    each of them try to find an algorithmic example
    with performance giving by this formula.
  • Sort the following formulas according to
    big/small Oh order
  • log (n1/2), log (9n), log (n3), 2log n, 23log n,
    2log (9n), n2, n log n
  • Propose and analyse an algorithm for checking if
    there is a majority in a given sorted array A.

22
Example 3Sorting in time O(n log n)
23
Divide and conquer
  • Algorithm
  • Partition input into two halves in (at most)
    linear time
  • Solve the problem for each half of the input
    separately
  • Merge solutions into one in (at most) linear time
  • Recursive formula on time complexity
  • T(n) ? 2 T(n/2) c?n

24
Case q 2, f(n) cn
  • T(n) ? 2T(n/2) cn , T(2) ? c
  • T(n) ? 2T(n/2) cn
  • ? 2(2T(n/4) cn/2) cn 4T(n/4) 2cn
  • ? 4(2T(n/8) cn/4) 2cn 8T(n/8) 3cn
  • ? 2i-1 (2T(n/2i) cn/2i-1) (i-1) . cn
  • 2i T(n/2i) i . cn
  • ? 2log n - 1 T(2) (log n - 1) . cn ? cn log
    n

25
Solution for timeformal analysis
  • T(n) ? 2 T(n/2) c?n
  • T(2) ? c
  • Solution T(n) ? c?n?log n
  • Proof by induction.
  • For n 2 straightforward T(2) ? c ? c?2?log
    2
  • Suppose T(n/2) ? c?(n/2)?log (n/2) then
  • T(n) ? 2T(n/2) c?n ? 2c?(n/2)?log (n/2)
    c?n
  • 2c?(n/2)?log n 2c?n/2 c?n c?n?log n

26
Example - sorting by merging
  • Input list L of n numbers
  • Problem Sort list L
  • Algorithm MergeSort(L)
  • If L has at most two elements then sort them by
    comparison and exit
  • Else
  • Split L into two lists prefix and suffix, each of
    size n/2
  • Sort, by merging, the prefix and the suffix
    separately
  • MergeSort(prefix) and MergeSort(suffix)
  • Merge sorted prefix with sorted suffix as
    follows
  • Initialize final list as empty
  • Repeat until either prefix or suffix is empty
  • Compare the first elements on the lists prefix
    and suffix and then move the smaller one to the
    end of the final list
  • Concatenate final list with the remaining
    non-empty list (prefix or suffix)

27
MergeSort procedure
1
2
4
7
8
3
5
6
9
10
1
2
3
4
5
6
7
8
9
10
28
Tree structure of sorting
root
merging operations
height 3
2
4
1
7
8
3
6
5
8 leaves
29
Time and memory consumption
  • Each recursive call
  • needs additional constant memory, and
  • the number of inputs increases by factor 2, and
  • the size of each input decreases by factor 2
  • Formula for time ( comparisons and list
    operations)
  • T(n) ? n/2 2T(n/2) 2n 2T(n/2) 5n/2
  • Additional memory size M(n) ? 2 M(n/2) 2
  • M(n) ? 2 M(n/2) 2 ? 4 M(n/4) 4 2 ? 8 M(n/8)
    8 4 2
  • ? ? n/2 n/4 4 2 n - 2 O(n)
  • In other words divide and conquer algorithm must
    store the binary tree of pointers during the
    computation

30
Quick sort - algorithmic scheme
  • Generic Quick Sort
  • Select one element x from the input
  • Partition the input into part containing elements
    not greater than x and part containing elements
    bigger than x
  • Sort each part separately
  • Concatenate these two sorted parts
  • Problem how to choose element x to balance the
    sizes of these two parts? (to get the similar
    recursive equations as for MergeSort)

31
Why parts should be balanced?
  • Suppose they are not balanced, but, say, the
    smallest element is chosen
  • T(n) ? T(1) T(n-1) c n
  • T(1) ? c
  • Solution T(n) ? (c/2) . n2
  • Proof by induction.
  • For n 1 straightforward T(1) ? c ? (c/2) .
    12
  • Suppose T(n-1) ? (c/2) . (n-1)2 then
  • T(n) ? T(n-1) c cn ? (c/2) . (n-1)2 c
    (n1)
  • ? (c/2) . (n-1)2 (c/2) . 2n ? (c/2) . n2

32
Two approaches to be fast
  • Deterministic
  • Additional tree structure of size O(n) needed for
    identifying the median value
  • Time O(n log n), additional memory O(n)
  • Randomized
  • Select separation element x uniformly at random
  • Time (expected) O(n log n), additional memory
    O(n)

33
Randomized approach - analysis
  • Let T(n) denote the expected time sum of all
    possible values of time weighted by the
    probabilities of these values
  • T(n) ? 1/n (T(n-1)T(1) T(n-2)T(2)
    T(0)T(n)) cn
  • T(0) T(1) 1, T(2) ? c
  • Solution T(n) ? d n log n, for some constant d
    ? 8c
  • Proof by induction.
  • For n 2 straightforward T(2) ? c ? d . 2
    . log 2
  • Suppose T(m) ? d m log m, for every m lt n then
  • (1-1/n)?T(n) ? (2/n)?(T(0) T(n-1)) c n
  • ? (2d/n)?(1 1 2log 2
    (n-1)log(n-1)) c n
  • ? (2d/n)?((n2/2) log n n2/8) c n
  • ? d n log n - d (n/4) c n ? d n
    log n - d n/2
  • T(n) ? (n/(n-1))?(d n log n - d n/2) ? d n log n

34
Tree structure of random execution
root
1
6
5
7
height 5
3
2
4
1
2
3
4
5
6
7
8
8 leaves
35
Textbook and Exercises
  • READING Section 5.3
  • EXERCISES
  • Solved exercises 1, 2 from the textbook, chapter
    5 Divide and Conquer
  • Prove that
  • 1 1 2 log 2 (n-1)log(n-1) ? n(n/2)
    log(4n) , and
  • (n/(n-1))?(d n log n - d n/2) ? d n log n , for n
    gt 16

36
Example 4Finding closest pair in time O(n log
n)
37
Problem
  • Closest pair of points
  • Input n points on the plane
  • Output two points having the closest distance
  • Remarks
  • Exhaustive search algorithm gives time O(n2)
  • Similar approach gives ALL pairs of closest points

38
Solution
  • Preprocessing
  • Sort points according to the first coordinate
    (obtain horizontal list H) and according to the
    second coordinate (obtain vertical list V)
  • Main Algorithm
  • Partition input list H into halves (H1,H2) in
    linear time (draw vertical line L containing
    medium point) split list V accordingly into
    V1,V2, where Vi contains the same elements as Hi
    and inherits the initial order from V (for i
    1,2)
  • Solve the problem for each half of the input
    separately, obtaining two pairs of closest
    points let ? be the smallest distance from the
    obtained ones
  • Find if there is a pair which has distance
    smaller than ? - if yes then find the smallest
    distance pair

39
Main difficulty
  • Find if there is a pair which has distance
    smaller than ?
  • if yes then find the smallest distance pair
  • How to do it in linear time?

?
?
?
L
40
Closest pair in the strip
  • Find a sub-list P of V containing all points with
    a distance at most ? from the line L
  • go through V and remove elements not ?close to L
  • 2. For each element in P check its distance to
    the next 8 elements and remember the closest pair
    ever found

?
?
?
P
L
41
Why to check only 8 points ahead?
  • 1. Partition the strip into squares of length ?/2
    each, as shown in the picture
  • 2. Each square contains at most 1 point - by
    definition of ?
  • 3. If there is at least 2 full squares between
    points then they can not be the closest points
  • 4. There are at most 8 squares to check

?
?/2
?/2
?/2
?/2
?/2
?/2
L
42
Time complexity
  • Preprocessing sorting in time O(n log n)
  • Main Algorithm recursion in time O(n log n)
  • T(n) ? 3n/2 2T(n/2) 8n 2T(n/2) 9.5n
  • T(2) 1

43
Example 5Integer multiplication in time
O(n1.59)
44
Beyond ?(n log n) integers multiplication
  • Input two integers x and y, each consisting of
    at most n bits
  • Output multiply these integers
  • Naïve Algorithm
  • Multiply each bit i of x by y, add (i-1) zeroes
    at the end
  • Add the obtained at most n numbers
  • Time ?(n2) of bit operations

45
Divide and Conquer approach
  • Let x x1 2n/2 x2 and y y1 2n/2 y2
  • Let p x1 x2 and q y1 y2. Then
  • x y (x1 2n/2 x2) (y1 2n/2 y2)
  • x1 y1 2n/2 (x2 y1 x1 y2) 2n x2 y2
  • x1 y1 2n/2 (pq - x1 y1- x2 y2) 2n
    x2 y2
  • Algorithm
  • Compute p and q (in linear time)
  • Compute recursively x1 y1 , x2 y2 , pq
  • Perform x1 y1 2n/2 (pq - x1 y1- x2 y2) 2n x2
    y2 in linear time

46
Time complexity
  • T(n) ? 3T(n/2) c n
  • T(1) ? c
  • Solution T(n) ? d nlog 3, for some constant d ?
    3c
  • Proof For n 1 straightforward T(1) ? c ? d
    nlog 3
  • General case
  • T(n) ? 3T(n/2) c n ? 9T(n/4) 3c(n/2) c
    n
  • ? ? c n (1 3/2 (3/2)2 (3/2)log n)
  • c n 2(3/2)log n 1 ? 3c n? nlog 3 - 1
  • 3c nlog 3 ? d? nlog 3 O(n1.59)

47
Textbook and Exercises
  • READING Chapter 5 from Section 5.4
  • OBLIGATORY EXERCISES
  • How to add two n-bit integers in linear number of
    bit operations?
  • ADDITIONAL EXERCISES
  • Solved Exercise 1 from the textbook, chapter 5
    Divide and Conquer
  • Exercises 1, 6, 7 from the textbook, chapter 5
    Divide and Conquer

48
Conclusions
  • Searching in time O(log n)
  • Finding majority value in time O(n)
  • Divide and conquer in time O(n log n)
  • sorting algorithms (MergeSort, RandQuickSort)
  • closest points algorithm
  • Beyond ?(n log n) O(nlog 3) time algorithm for
    integer multiplication
Write a Comment
User Comments (0)
About PowerShow.com