Chapter 2: Fundamentals of the Analysis of Algorithm Efficiency

1
Randomized QS. Ch5, CLRS Dr. M. Sakalli, Marmara
University Picture 2006, RPI
2
Randomized Algorithm

• Randomize the algorithm so that it works well
  with high probability on all inputs
• In this case, randomize the order that candidates
  arrive
• Universal hash functions randomize selection of
  hash function to use

3
Probabilistic Analysis and Randomized Algorithms

• Hiring problem
• n candidates interviewed for a position
• One interview each day, if qualified hire the
  new, fire the current employed.
• Best Meet the most qualified one at the 1st
  interview.
• for k 1 to n
• Pay ci interview cost.
• if candidate(k) is better
• Sack current, best?k // sack current one and
  hire(k)
• Pay ch hiring (and sacking) cost
• Worst-case cost is increased quality nch nch,
  chgt ci, then O(nch)
• Best-case, the least cost ?((n-1) ci ch),
  prove that hiring problem is ?(n)
• Suppose applicants arriving in a random
  qualification order and suppose that the
  randomness is equally likely to be any one of the
  n! of permutations 1 through n.
• A uniform random permutation.

4
Probabilistic Analysis and Randomized Algorithms

• An algorithm is randomized, not just if its
  behaviors is controlled by but if the output is
  generated by a random-number generator.
• C rand(a,b), with boundaries are inclusive..
  Equally likely (probable) outputs of X Xi for
  i1n.
• Expected value EX Sin xi (probability
  density of each x) EX Sin xi Prxxi
• In the hiring case,
• Defining an indicator, Indicator random variable,
  Xi I indicator of candidate if hired (or of
  coin), 1(H) or 0(T)

5

• H and T, three fair coins, head 3s, every tail
  2s, and expected value of earnings.
• HHH 9s, 1/8,
• HHT 4s, 3/8,
• HTT -1s, 3/8,
• TTT -6s, 1/8
• Eearnings 9/812/8-3/8-6/812/81.5

6
Probabilistic Analysis and Randomized Algorithms

• Lemma 1. Given a sample space S, and an event e
  in the sample space, let XeI(e) be indicator of
  occurrence of e, then, EXePre.
• Proof Fr the definition of expected value,
• EXA Sen xe Prxe 1 PrXe 0
  PrnotXe PrXe, where noteS-e.
• In the binary case, equally distributed, uniform
  distribution, Ee PrXe 1/2.

7

• X Sj1nxi
• Expected value of a candidate been hired is the
  probability of candidate hired.
• Exj Prif candidate(i) is hired,
• Candidate i is hired if better than previous i-1
  candidates.
• The probability of being better is 1/i.
• Then, Exi Prxi 1/i
• Expected value of hiring (the average number of
  hired ones out of n arriving candidates in random
  rank) EX. Uniform distribution, equally likely,
  1/i.

8

• Expected value of hiring (the average number of
  hired ones out of n arriving candidates in random
  rank) EX. Uniform distribution, equally likely,
  1/i.
• EX ESj1n Ii Prxi, Ii1, 0 indicator
  random value here.
• EX SinExi, from linearity of expected
  value. .
• Si1(n)(1/i) 11/21/3, harmonic number
  (divergent).. ?Int 1 to (n1), (1/x)ln(n1)ltln(n
  )O(1)
• Si2(n)(1/i) 1/21/3.. ?Int 1 to (n1),(1/x)
  ln(n)ltln(n)
• Si1(n)(1/i) 11/21/3.. ? ln(n)1
• E(X) ln(n)O(1),
• Expected value of all hirings.. Upper boundary is
  ln(n).
• Lemma5.2 When candidates presented in random,
  the cost of hiring is O(chlgn). Proof from
  Lemma5.1.
• How to randomize.. Some random outputs of
  permutations will not be random. Would it
  matter?..

9

• In the case of dice, Prheads 1/2, in which
  case, for n tries, EX ESj1n1/2 n/2.
• Biased Coin
• Suppose you want to output 0 and 1 with the
  probabilities of 1/2
• You have a coin that outputs 1 with probability p
  and 0 with probability 1-p for some unknown 0 lt p
  lt 1
• Can you use this coin to output 0 and 1 fairly?
• What is the expected running time to produce the
  fair output as a function of p?
• Let Si be the probability that we successfully
  hire the best qualified candidate AND this
  candidate was the ith one interviewed
• Let M(j) the candidate in 1 through j with
  highest score
• What needs to happen for Si to be true?
• Best candidate is in position i Bi
• No candidate in positions k1 through i-1 are
  hired Oi
• These two quantities are independent, so we can
  multiply their probabilities to get Si

10

• Characterizing the running time of a randomized
  algorithm.
• ET(n) Sj1n tiPrti
• void qksrt( vectorltintgt data)
• RandGen rand
• qksrt(data, 0, data.lngth() - 1, rand )
• void qksrt( vectorltintgt dt, int srt, int end,
  RandGen rand )
• if( start lt end )
• int bnd partition(dt, srt, end,
  rand.RandInt(srt, end ) )
• qksrt(dt, start, bnd, rand)
• qksrt(dt, bnd 1, end, rand )

11

• we are equally likely to get each possible split,
  since we choose the pivot at random.
• Express the expected running time as
• T(0) 0
• T(n) Sk0n-1 1/nT(k)T(n-k1)n
• T(n) n Sk0n-1 1/nT(k)T(n-k1)
• Bad split, choosing from 1st or 4th quarter, that
  is
• i ? k ? i ? ¼j-i1 or j - ¼j-i1 ? k ? j
• Good split i ¼i-j1 ? k? j - ¼j-i1

12

• Mixing good and bad coincidence with equal
  likelihood.
• T(n) n Sk0n-1 1/nT(k)T(n-k1)
• T(n) n (2/n) Sjn/2n-1 T(k)T(n-k1)
• n (2/n)Skn/23n/4 T(k)T(n-k1)
  Sk3n/4n-1 T(k)T(n-k1)
• ? n (2/n) Skn/23n/4 T(3n/4)T(n/4)
  Sk3n/4n-1 T(n-1)T(0)
• ? n (2/n)(n/4)T(3n/4)T(n/4) T(n-1)
• ? n (1/2)T(3n/4)T(n/4) T(n-1)
• Prove that for all n T(n) ? cnlog(n), T(n) is
  the statement obtained above. Inductive proof,
  for n0, and nn,
• Probability of many bad splits is very small.
  With high probability the list is divided into
  fractional pieces which is enough balance to get
  asymptotic n log n running time.
• ? n (1/2)c(3n/4) log(3n/4) c(n/4) log(n/4)
  (1/2)c(n-1)log(n-1)

13

• MIT notess
• L(n) 2U(n/2) T(n) lucky
• U(n) L(n 1) T(n) unlucky
• L(n) 2U(n/2 1) 2T(n/2) T(n)
• L(n) 2U(n/2 1) T(n)
• L(n) T(nlogn)
• And there is more there..

14
Computing S

• Bi 1/n
• Oi k/(i-1)
• Si k/(n(i-1))
• S Sigtk Si k/n Sigtk 1/(i-1) is probability of
  success
• k/n (Hn Hk) roughly k/n (ln n ln k)
• Maximized when k n/e
• Leads to probability of success of 1/e