CSE 780: Design and Analysis of Algorithms

1
CSE 780 Design and Analysis of Algorithms

• Lecture 5
• Lower bound
• comparison model
• decision tree
• Order statistics
• randomized alg.

2
Lower Bound for Sorting

• Model
• What types of operations are allowed
• E.g partial sum
• Both addition and subtraction
• Addition-only model
• For sorting
• Comparison-based model

3
Decision Tree
4
A Non-Comparison-Based Model?

• Can we have such a model at all ?
• Eg
• Given n integers in range 1, , m

5
Decision Tree

• Not necessary same height
• Worst case complexity
• Longest root-leaf path
• Each leaf
• A possible outcome
• I.e., a permutation of input
• Every possible outcome should be some leaf
• leaves n!

6
Lower Bound

• Any binary tree of height h has at most 2h
  leaves
• A binary tree with m leaves is of height at least
  lg m
• Worst case complexity for any algorithm sorting
  n elements is
• ?(lg (n!) )
• ?(n lg n) (by Stirling approximation)

7
Order Statistics

• Rank
• The order of an element in the sorted sequence
• Selection
• Return the element with certain rank

8
Examples

• Return rank(1), or rank(n)
• n-1 operations
• Return both rank(1) and rank(n)
• Better than 2n-2?
• Yes
• Return the median

9
Select ( r )

• Return the element with rank r.
• Straightforward approach
• Sort, then return Ar
• Compute a lot of extra information

10
Divide and Conquer Again!

• Similar to QuickSort
• Select ( A, 1, n, r )

Case 1 r m
return Am
11
Select (A, 1, n, r )
Case 2 r lt m
return Select ( A, 1, m-1, r )
12
Select (A, 1, n, r )
Case 3 r gt m
return Select ( A, m1, n, r-m )
13
Pseudo-code
Rand-Select ( A, s, t, r ) if ( s t )
return As m Rand-Partition ( A, s, t
) if ( m r) return Am if ( m gt
r ) return Rand-Select ( A, s, m-1, r
) else return Rand-Select ( A,
m1, t , r - m )
Rand-Select(A, 1, n, r) T(n) T( max(m-1,
n-m) ) O(n)
14
Analysis

• Worst case
• ? ( n2 )
• Average case Rand-Select(A, 1, n, r)
• Indicator random variable
• X(k) 1 if m k
• 0 Otherwise
• E X(k) 1 / n

15
Rand-Select ( A, 1, n, r)
m Rand-Partition ( A, 1, n ) if
( m r) return Am if ( m gt r )
return Rand-Select ( A, 1, m-1, r ) else
return Rand-Select ( A, m1, n, r - m )

T(n) ? X(k) (T( max(k-1, n-k) ) O(n)
16
Solving Recursion

• T(n) ? X(k) T( max(k-1, n-k) ) O(n)
• ET(n) E ? X(k) T( max(k-1, n-k) ) O(n)
• ? EX(k) T( max(k-1, n-k) ) O(n)
• ? EX(k) ET(max(k-1, n-k) )
  O(n)
• 1/n ? ET(max(k-1, n-k) ) O(n)

17
Recursion cont.

• max(k-1, n-k) k-1 if k gt n/2
• n-k otherwise

Proof by substitution method
18
Finding k Largest Numbers

• Given A1,..n
• Return k largest numbers

• Sorting
• T(n) O (n lg n k) O(n lg n)
• Heap
• T(n) O (n k lg n)
• Selection
• T(n) O (n k )

19
Deterministic

• Selection
• O(n) deterministic algorithm
• How about Quicksort?

20
Summary

• Sorting lower bound
• Comparison model
• Decision tree
• Selection
• Min/max
• Any rank -- randomized approach