Randomized Algorithms

1
Randomized Algorithms

• Iman SadeghiHamid Reza Vaezi
• Advanced Algorithms Course Prof. Ghodsi

2
Agenda

• Introduction
• Sample problems
• Monte Carlo vs. Las Vegas methods
• Randomized complexity classes
• Markov Chains and Random Walks
• Probabilistic method
• Examples
• Dominating Sets
• Min Cut
• MST
• References

3
Quick Sort

• Problem who do not know ?
• Worst case
• bound expected number of comparisons C
• if compare i to j otherwise 0.
• linearity of expectation
• Consider smallest recursive call involving both i
  and j.
• Si and Sj get compared if pivot is Si or Sj.
• probability is at most

4
Quick Sort (contd)
5
Min Cut

• Compute the cut with minimum number of edges in
  the graph. Namely, find such that
  is as small as possible, and S
  is neither empty nor all the vertices of the
  graph G(V,E).

6
Min Cut (contd)

• edge contraction G/xy

• Observation 1 The size of the cut in G/xy is at
  least as large as the cut in G (as long as G/xy
  as at least one edge). Since any cut in G/xy has
  a corresponding cut of the same cardinality in G.

7
Min Cut (contd)

• Observation 2 Let e1, . . . ,en-2 be a sequence
  of edges in G, such that none of them is in the
  min-cut, and such that G0 G/e1, . . . ,en-2
  is a single multi-edge. Then, this multi-edge
  correspond to the min-cut in G.
• Idea Let us find a sequence of edges e1, . . .
  ,en-2, such that G/e1, . . . ,en-2 corresponds
  to the minimum cut.
• Lemma 1 If a graph G has a min-cut of size k, and
  it has n vertices, then E(G) gtkn/2.
• Prove vertex degree is at least k. so count them
  .
• Lemma 2 If we pick in random an edge e from a
  graph G, then with probability at most 2/n it
  belong to the min-cut.
• Prove more than nk/2 edge in graph and k edges
  in cut

8
Min Cut (contd)

• Observation 5 MinCutInner runs in O(n2) time.
• Observation 6 The algorithm always outputs a
  cut, and the cut is not smaller than the minimum

9
Min Cut (contd)

• Lemma 3 MinCutInner outputs the min cut in
  probability 2/n(n-1).
• Proof Let hi be the event that ei is not in the
  min-cut of Gi. Clearly, MinCut outputs the
  minimum cut if h0, . . . ,hn-3 all happen
  (namely, all edges picked are outside the min
  cut).
• We have
• Thus

10
Min Cut (contd)

• Definition (informal) Amplification is the
  process of running an experiment again and again
  till the things we want to happed with good
  probability.
• Let MinCut be the algorithm that runs MinCutInner
  n(n-1) times and return the minimum cut computed.
• Lemma 4 The probability that MinCut fails to
  return the min-cut is lt 0.14.
• Proof The probability of failure is at most

11
Min Cut (contd)

• Theorem One can compute the min-cut in O(n4) time
  with constant probability to get a correct
  result.
• Solved.

To be continued.
12
Monte Carlo vs. Las Vegas methods

• Definition A Las Vegas algorithm is a randomized
  algorithms that always return the correct result.
  The only variant is that its running time might
  change between executions.
• An example for a Las Vegas algorithm is the
  QuickSort algorithm.
• Definition A Monte Carlo algorithm is a
  randomized algorithm that might output an
  incorrect result. However, the probability of
  error can be diminished by repeated executions of
  the algorithm.
• The MinCut algorithm was an example of a Monte
  Carlo algorithm.

13
Randomized complexity classes

• Definition The class RP (for Randomized
  Polynomial time) consists of all languages L that
  have a randomized algorithm A with worst case
  polynomial running time such that for any input x
  belong S,

An RP algorithm is Monte Carlo, but the mistake
can only be if . co-RP is all the
languages that have a Monte Carlo algorithm that
make a mistake only if . A problem
which is in has an
algorithm that does not make a mistake, namely a
Las Vegas algorithm.
14
Markov Chains and Random Walks

• Markov property (first order)
• Transition Probability
• Random walks
• Equal probability
• Linear Random Walks
• Pi,i1p-pi1,I

15
Markov Chains Applications

• Markov Chains
• Connectivity
• Visiting every edge?
• S-t Connectivity
• Expanders
• Sparse Graphs O(n) edges
• Mixing probobility
• huv, Cuv
• Electrical Networks
• Random Walks
• Simulation
• Computer Graphics
• Genetics
• Statistics

16
Randomized 2SAT Algorithm

• Assign variables arbitrary
• While there is an unsatisfied clause Do
• choose an unsatisfied clause C
• Negate one of its participants randomly.
• Return the feasible assignment!

17
Randomized 2-SAT Analysis

• What is the expected runtime of our randomized
  algorithm?
• Av1, v2 vn
• xnumber of correct variables lt n
• In each step value of x changes by 1.
• Our algorithm runs in O(mn2)

18
s-t Connectivity

• Any path from s could be shown by strings
  consisting of 1,2,n-1
• Checking all O(nn) strings and O(n lg n) space
• A Monte Carlo algorithm with O(lg n) space
• Starting at s, simulate a random walk of n-1
  steps
• If t is reached return true
• Else if you stuck or you reach n-1th node return
  to s.(lg n) space.
• Flip lg nn coins. If all of them are Head return
  No. (lg(lg nn)lg n space

19
s-t Conectivity Analysis

• Probability of erroneous answer
• Our Algorithm outputs
• No if s-t are not connected
• Yes with probability of ½ if there is such a path

20
Probabilistic method

• It can actually be used to demonstrate the
  existence of algorithms.
• Any random variable assumes at least one value
  that is no smaller that its expectation, and at
  least one value that is no greater that its
  expectation.
• If an object chosen randomly from a universe
  satisfies a property with positive probability,
  then there must be an object in the universe that
  satisfies that property.

21
Probabilistic method (contd)

• Example 1 For any undirected graph G(V,E) with n
  vertex and m edges, there is a partition of
  vertex set V into sets A and B such that
• Prove each vertex of G is independently and
  equiprobably assigned to either A or B. for edge
  (u,v) the probability that its end-points are in
  different sets is ½. By linearity of expectation,
  the expected number of edges with end-points in
  different set is thus m/2. It follows that there
  must be a partion satisfyiung the example.

22
Probabilistic method (contd)

• Example 2 For any set of m clauses, there is a
  truth assignment for variables that satisfies at
  least m/2 clauses.
• Prove
• Suppose that each variable set to TRUE or FALSE
  independently.
• For any clause containing k literals, the
  probability that it is not satisfied by this
  random assignment is
• So the probability that clause is satisfied is
  more than ½.
• The expected number of satisfied clause is m/2.
• So there exist at least one assignment that at
  least m/2 of clauses are satisfied.

23
Min Dominating Sets

• Definition
• Approximation within 1lgV (MINIMUM SET COVER)
• No approximation within clgV for some cgt0
• (Alon 90) If G has minimum vertex degree kgt1 then
  G has a dominating set with at most

24
Min Dominating Sets (contd)

• Form a random vertex subset S, including each
  vertex by
• If T is not covered Add T to S.
• Expected size of the union is
• There must be some S no larger than this.

25
Min Cut

• As the graph get smaller, the probability to make
  a bad choice increases.
• Intuitive idea Run the algorithm more times when
  the graph get small.

Lemma The probability that Contract(G, n/ 2½ )
had not contracted the min-cut is at least 1/2.
26
Min Cut (contd)

• The running time of FastCut(G) is O(n2logn),
  where n V(G).

27
Min Cut (contd)

• Theorem FastCut finds the min-cut with
  probability larger than W(1/logn).
• Proof Let P(t) be the probability that the
  algorithm succeeds on a graph with t vertices.
  And the probability to succeed on the call on H1
  is at least
• So

28
Min Cut (contd)

• Exercise Prove, that running of FastCut c log2 n
  times, guarantee that the algorithm outputs the
  min-cut with probability greater than 1-1/n2
  for c constant large enough.
• So it is an RP algorithm with running time

29
Minimum Spanning Tree

• Well known classic problem
• Prim, Kruskal and Boruvka O(n lgn)
• Best deterministic algorithm
• We present RMST which is O(m) with high
  probability.

30
Boruvka MST

• Lemma Lightest edge incident to each vertex
  belongs to MST
• At each Boruka phase, contract these lightest
  edges simultaneously. It could be done in O(mn)
• After one phase there are at most n/2 vertices
• Reapeat the process until reaching a vertex
• Analysis T(n)2T(n/2)O(mn) T(n)O(m lgn)

31
Definitions and lemmas

• Definition wF(u,v) denotes the maximum weight of
  an edge of the path P(u,v) in forest F.
• Definition an edge (u,v) is said to be F-heavy if
  w(u,v)gtwF(u,v) and F-Light otherwise.
• Theorem all F-heavy edges of G can be found in
  O(mn)
• Lemma let F be the minimum spanning forest of
  G(p), the expected number of F-light edges in G
  is atmost n/p

32
Linear MST(G)

• By three Boruvka phases, compute G1with at most
  n/8 vertices and Ccontracted edges O(nm)
• If G1 is empty then return FC
• G2G1(p), p ½ O(nm)
• F2 Linear MST(G2) T(n/8,m/2)
• Identify F2-Heavy edges in G1 and remove them to
  obtain G3 O(mn)
• F3 Linear MST(G3) T(n/8,n/4)
• Return forest FC U F3 O(n)
• .

33
References

• Randomized Algorithmes R.Motwani
• Chapters 1,5,6,10
• Introducton to Algorithms CLRS
• Chapter 5
• A compendium of NP webpage http//www.nada.kth.se/
  viggo/problemlist/compendium.html
• Introduction to Graph Theory D.West
• Section 8.5