ICS 353: Design and Analysis of Algorithms

1
ICS 353 Design and Analysis of Algorithms
King Fahd University of Petroleum
Minerals Information Computer Science Department

• Divide and Conquer

2
Reading Assignment

• M. Alsuwaiyel, Introduction to Algorithms Design
  Techniques and Analysis, World Scientific
  Publishing Co., Inc. 1999.
• Chapter 6 (Except Section 6.9)

3
Divide and Conquer

• The divide and conquer paradigm consists of the
  following steps
• Divide step Input is partitioned into p ? 1
  partitions, each of size strictly less than n.
• Most common value of p 2.
• p could be equal to one when part of the input is
  discarded (example?)
• Conquer step Perform p recursive calls if the
  problem size is greater than a specific threshold
  n0.
• n0 is usually 1, but could be greater than 1.
• Combine step The solution to the p recursive
  calls are combined to solve the problem for the
  union of the p partitions of the input.
• In many, but not all cases, this step determines
  the time complexity of the algorithm.

4
Recursive Merge Sort

• MergeSort(A,p,r)
• if p lt r then
• q ?(pr)/2?
• MergeSort(A,p,q)
• MergeSort(A,q1,r)
• Merge(A,p,q,r)
• end if
• Assume that n is a power of two
• What is the cost of MergeSort(A,j,j)?
• What is the cost of MergeSort(A,1,n/2)?
• What is the cost of MergeSort(A,n/21,n)?
• What is the cost of Merge(A,1,n/2,n)?
• What is the recurrence equation(s) describing the
  time complexity? What is the solution?

5
Recursive Binary Search Algorithm

• BinarySearchRec(A,low,high)
• if (low gt high) then
• return 0
• else
• mid ? ?(low high)/2?
• if x Amid then return mid
• else if x lt Amid then
• return BinarySearchRec(A,low,mid-1)
• else return BinarySearchRec(A,mid1,high)
• end if
• end if
• Identify the divide, conquer and combine
  operations.
• If n 2k, what is the recurrence equation and
  its solution?

6
Recursive MinMax algorithm

• Procedure MinMax(A,low,high)
• if high low 1 then
• if Alow lt Ahigh then return
  (Alow,Ahigh)
• else return (Ahigh,Alow)
• end if
• else
• mid ? ?(low high)/2?
• (x1,y1) ? MinMax(A,low,mid)
• (x2,y2) ? MinMax(A,mid1,high)
• x ? minx1,x2
• y ? maxy1,y2
• return (x,y)
• end if

7
Analysis of Recursive MinMax

• Identify the divide, conquer, and combine steps
  in the algorithm.
• Assuming n is a power of two, what is the
  recurrence equation describing the time
  complexity? What is the solution?
• What is the cost of the straightforward
  algorithm?

8
Selection Problem

• Problem Statement Find the kth smallest element
  in the array
• A special case is to find the median of the array
• In case n is odd, the median is the (n1)/2 th
  smallest element
• In case n is even, the median is the n/2 th
  smallest element
• What is the straightforward algorithm? What is
  its time complexity?

9
A Better Algorithm

• If we can discard a constant fraction of the
  elements after the divide step of every recursive
  call, and recur on the rest of the elements, the
  size of the problem decreases geometrically
• E.g. if we assume that 1/3 of the elements are
  discarded and that the algorithm spends a
  constant time per element, we get
• cn (2/3)cn (2/3)2 cn (2/3)j cn

10
Basic Idea of the Algorithm

• If the number of elements is less than a
  threshold, sort and find the kth element
• Otherwise, partition the input into ?n/5? groups
  of five elements each
• You may have a group of less than 5 elements if n
  does not divide 5.
• Sort each group and extract its median.
• The median of medians is computed recursively.
• Partition the elements in A around the median
  into three sets A1, A2, and A3
• Where to look for the kth smallest element?

11
Example

• Find the 14th smallest element in the array
• 8 , 33 , 17 , 51 , 57 , 49 , 35 , 11 , 25 , 37 ,
  14 , 3 , 2 , 13 , 52 , 12 , 6 , 29 , 32 , 54 , 5
  , 16 , 22 , 23 , 7 , 8 , 19 , 44 , 66

12
Algorithm Select

• Input Array A1..n and an integer k, 1 k n
• Output kth smallest element in A
• Procedure select (A, low, high, k)
• p high low 1
• if p lt 44 then sort A and return(Ak)
• Let q ? p/5 ?. Divide A into q groups of 5
  elements each, discarding the possibly one
  additional group with less than 5 elements
• Sort the q groups individually extracting the
  median. Let the set of medians be M
• mm select(M,1,q,?q/2?)
• Partition Alow..high into three array,
  A1aaltmm, A2aamm, and A3aagtmm
• case
• A1 ? k return select(A1,1,A1,k)
• A1 A2 ? k return mm
• A1 A2 lt k return select(A3,1,A3,k(A1
  A2))

13
Analysis of the Selection Algorithm (1)
W
W
Z
Z

• Estimating the sizes of A1 and A3.
• A1 x ? A x ? mm
• The size of W will give us the minimum number of
  elements in A1
• Hence, knowing the minimum size of A1 will give
  us an upper bound on the size of which is equal
  to .

14
Analysis of the Selection Algorithm (2)

• Now, we can write the recurrence relation for the
  selection algorithm as follows
• What do we use to solve this recurrence?

15
Quick Sort

• Procedure QuickSort(A,low,high)
• if low lt high then
• w split(A,low,high)
• QuickSort(A,low,w-1)
• QuickSort(A,w1,high)
• end if

16
Split Algorithm

• Split(A,low,high)
• i low
• x Alow
• for j low 1 to high do
• if Aj ? x then
• i i 1
• if i ? j then
• swap(Ai,Aj)
• end if
• end if
• end for
• swap (Alow,Ai)
• return i

17
Quick Sort Complexity Analysis

• How many element comparisons are carried out by
  the split procedure?
• Worst case analysis of Quick Sort
• Best case analysis of Quick Sort

18
Average Case Analysis of Quick Sort

• Assumption All permutations of the input are
  equally likely
• The input consists of n distinct elements
• This implies that the probability that any
  element will be picked as a pivot is
• Let C(n) denote the number of comparisons done by
  QuickSort on the average on input of size n

19
Comparative Results of Various Sorting Algorithms
20
Multiplication of Large Integers

• Let u and v be two n-bit integers, where n is a
  power of 2
• The traditional multiplication algorithm takes
• Divide and conquer can be used to carry out the
  multiplication as follows
• Divide each integer into 2 n/2-bit
• portions as shown here
• u w2n/2 x and v y2n/2 z
• The product now becomes
• Note that multiplying a number t by 2k is
  equivalent to shifting t k bits to the left,
  which is ?(k) time.
• How many additions and multiplications we have?
  What is the recurrence equation?

u
w
x
y
z
v
21
Can We Do Better?

• Note that if we can reduce the number of
  multiplications by 1, we will have asymptotic
  improvement
• Consider evaluating wz xy as follows
• wz xy (w x) (y z) wy xz
• Note that wy and xz have already been computed.
  So no need to compute again
• What is the total number of multiplications now?
  Rewrite the recurrence equation and solve.

22
Matrix Multiplication

• Let A and B be 2 n ? n matrices, assuming n to be
  a power of 2. We would like to employ divide and
  conquer to carry out the multiplication of A and
  B.
• What is the traditional algorithm? How much does
  it cost?

23
Recursive Version

• Let and
  then
• What is the recurrence describing the cost of
  carrying out the multiplication by computing the
  number of multiplication operations? What is the
  solution to the recurrence? Did we achieve
  anything?

24
Strassens algorithm

• The idea is again exactly similar to that in
  multiplying large numbers we would like to
  rewrite the product in a manner that will reduce
  the number of multiplications needed (even by
  one!)

25
Strassens Algorithm
26
Analysis of Strassens Algorithm

• How many multiplications and additions/subtraction
  s are carried out?
• What is the recurrence equation describing the
  cost? What is the recurrence solution?
• Is there any improvement?

27
Empirical Comparison
n Multiplications Additions
Traditional Alg. 100 1,000,000 990,000
Strassens Alg. 100 411,822 2,470,334
Traditional Alg. 1,000 1,000,000,000 999,000,000
Strassens Alg. 1,000 264,280,285 1,579,681,709
Traditional Alg. 10,000 1012 9.99 ? 1011
Strassens Alg. 10,000 0.169 ? 1012 1012