Instructor: Shengyu Zhang

1
CSC3160 Design and Analysis of Algorithms
Week 9 Randomized Algorithms

• Instructor Shengyu Zhang

2
About the course

• Three extra lectures would be added.
• Look at the lecture page for more details.
• This week
• NP-completeness An introduction without proof.
• Only 15 minutes.
• A bait rather than an intro. Got interested? Be
  sure to go to Andrejs class late this semester!
• Randomized algorithms.

3
NP
4
NP

• P, NP, examples in NP.
• question P vs. NP.
• Fact Cook-Levin Theorem.
• About the completeness.
• A view of reduction.
• Questions vs. answers.
• Impact on TCS, CS and other sciences.

5
SAT

• 3SAT
• n variables (taking values 0 and 1)
• m clauses, each being OR of 3 variables or their
  negations
• E.g. (x1) ? x5 ? (x7)
• 3SAT formula AND of these m clauses
• E.g. ((x1) ? x5 ? x7) ? ((x2) ? x5 ? (x7)) ?
  (x1 ? x7 ? x8)
• 3SAT Problem Is there an assignment of variables
  s.t. the formula evaluates 1?

6
P, NP

• P Problems solvable in deterministic polynomial
  time
• NP Problems checkable in deterministic
  polynomial time
• SAT just defined.
• Each to check Given any assignment, each to
  check whether it satisfies all clauses.
• Factoring Factor a number n.
• Easy to check For any m, each to check whether
  mn.

7
More examples in NP

• TSP On a weighted graph, find a closed cycle
  visiting each vertex exactly once, with the total
  weight on the path no more than k.
• Easy to check Given a cycle, easy to calculate
  the total weight.
• Graph Isomorphism Given two graphs G1 and G2,
  decide whether we can permute vertices of G1 to
  get G2.
• Easy to check For any given permutation, easy to
  permute G1 according to it and then compare to G2.

8
Question of P vs. NP

• Is P NP?
• The most notorious question in computer science.
• Has staggering practical implications
• Has withstood attack since forever
• Clay Mathematics Institute recognized it as one
  of seven great mathematical challenges of the
  millennium. US1M.
• Want to get rich (and famous)? Here is a simple
  way!

9
The P vs. NP question intuition

• Is producing a solution essentially harder than
  check a solution?
• Coming up with a proof vs. verify a proof.
• Compose a song vs. appreciate a song.
• Cook good food vs. recognize good food

10
What if P NP?

• The world becomes a Utopia.
• Mathematicians are replaced by efficient
  theorem-discovering machines.
• It becomes easy to come up with the simplest
  theory to explain known data
• But at the same time,
• Many cryptosystems are insecure.

11
Completeness

• Cook-Levin If you can solve 3SAT in polynomial
  time, then you can solve all problems in NP in
  polynomial time.
• We say that 3SAT is NP-complete.
• In other words, 3SAT is the hardest problem in
  NP.
• Karp 21 other problems such as TSP are also
  NP-complete
• Later thousands of NP-complete problems.

12
Meanings of NP-completeness

• Reduce the number of questions without increasing
  the number of answers.
• Huge impacts on almost all other sciences such as
  physics, chemistry, biology,
• The biggest export of TCS.

13
Randomized algorithms
14
Randomized Algorithms

• We use randomness in our algorithms.
• Example that youve seen
• quick sort pick a random pivot.
• Well see more in this week.

15
Example 1 A simple approximation algorithm for
3SAT
16
7/8-approximation of 3SAT

• Recall that 3SAT asks whether there is an
  assignment satisfying all clauses.
• Lets aim lower this time Can you find an
  assignment satisfying half of the clauses?
• Lets check the random assignment For each
  variable xi, we randomly choose a number from
  0,1 for it.
• How good is this assignment?

17
Analysis

• For each clause, there are 8 possible assignments
  for these three variables, and only 1 fails.
• E.g. (x1) ? x5 ? (x7) only (x11, x50, x71)
  fails.
• Thus if you assign randomly, then with each
  clause fails with expectation only 1/8.
• Thus the expected number of satisfied clauses is
  7m/8.

18
Formally

• Define a random variable Yi for each clause i.
• If clause i is satisfied, then Yi 1, otherwise
  Yi 0
• Define another random variable Y ?i Yi
• Y has a clear meaning the number of satisfied
  clauses
• EY // expected of satisfied clauses
  E?i Yi // linearity of expectation ?iEYi
  ?i 7/8 7m/8. // m of clauses

19
Turning expectation into small-error

• This means that if we choose randomly, then we
  can satisfy 7/8 fraction of clauses on average.
• If we try O(m) times, then with probability 0.99,
  we can obtain an assignment satisfying 7/8
  fraction of clauses.
• Pr one random assignment satisfies 7/8 fraction
  1/(8m)
• Details see chapter 13 in textbook KT or go to
  tutorials.

20
bait

• Can you imagine that we can also have a
  deterministic algorithm that always output an
  assignment satisfying at least 7/8 fraction of
  clauses?
• Come to tomorrows class for an elementary
  introduction to derandomization.

21
Example 2 global minimum cut
22
Min-cut for undirected graphs

• Given an undirected graph, a global min-cut is a
  cut (S,V-S) minimizing the number of crossing
  edges.
• Recall a crossing edge is an edge (u,v) s.t. u?S
  and v?V-S.
• In last homework, you basically solves this
  problem already.
• The way is by running max-flow algorithm V-1
  times
• Fix s, try all other t.

23
Better algorithm

• It turns out that we can do better.
• than running the Max Flow V-1 times.
• Well introduce a Contraction Algorithm, which is
  so simple that its hard to believe to work at
  the first glance.

24
Graph Contraction
b
a

• For an undirected graph G and two vertices u, v.
• We contract u and v and form a new graph G
• u and v become one vertex u,v
• Of course, the edge (u,v) thus disappear.
• Other edges incident to u or v in G naturally
  change to edges incident to u,v in G.
• Now we may have more than one edge between two
  vertices. But well thats fine. We just keep
  them there

v
u
e
d
c
b
a
u,v
e
d
c
25
Kargers algorithm

• For i 1 to 100n2
• repeat
• randomly pick an edge (u,v)
• contract u and v
• until two vertices are left
• ci ? the number of edges between them
• Output mini ci
• Essentially
• key fact If we contract an random edge until
  two vertices are left, then the number of edges
  between them achieve the minimum in min cut with
  prob. O(1/n2).

26
Analysis of the key fact

• Fix a min cut (S,V-S) If we never picked a
  crossing edge in the algorithm, then ok.
• i.e. then finally the number of edges between two
  last vertices is the correct answer.
• Lets analyze the probability step by step.

27
Step 1

• In step 1 whats the prob. that a crossing edge
  is not picked?
• (E-c)/E.
• c the number of edges of min cut.
• Lets analyze this quantity
• By def of min cut, we know that each vertex v has
  degree at least c.
• Otherwise the cut (v, V-v) is lighter.
• Thus E nc/2, where n V
• And (E-c)/E 1 - c/E 1-2/n.

c
c
28
Step 2

• Similarly, in step 2,
• Pr no crossing edge picked 1-2/(n-1)
• Note that now the number of vertices is n-1.
• In general, in step j,
• Pr no crossing edge picked 1-2/(n-j1)

29
Together

• Whats the prob. that the n-2 contractions never
  contract a crossing edge?
• Pr first step good Pr second step good after
  surviving first stepPr third step good after
  surviving first two stepsPr (n-2)-th step
  good after surviving first n-3 steps (1-2/n)
  (1-2/(n-1)) (1-2/3) (n-2)/n (n-3)(n-1)
  1/3 2/n(n-1) O(1/n2)

30
Random Walk on Graphs
31
Random walk on graphs

• Graph G.
• Starting vertex v0
• Each step
• Go to a random neighbor.
• Simple but powerful.

32
Typical questions about random walk

• Hitting time How long it takes to hit a
  particular vertex?
• H(s,t) Expected time needed to hit t, starting
  from s
• General graph H(s,t) O(n3)
• On a line (v1, , vn) H(v1, vn) T(n2)
• Covering time How long it takes to visit all
  other vertices (at least once)?
• C(s) Expected time needed to visit all other
  vertices, starting from s.
• General graph C(s) O(n3).
• On a line (v1, , vn) H(vi) T(n2), for all i.

33
Algorithm 1 2-SAT

• 2SAT each clause has two variables/negations
• Alg
• Pick any assignment
• Repeat O(n2) time
• If all satisfied, done
• Else
• Pick any unsatisfied clause
• Pick one of the two literals each with ½
  probability, and flip the assignment on the
  variable

(x1?x2)?(x2?x3) ?(x4?x3) ?(x5?x1)
x1, x2, x3, x4, x5 0, 1, 0, 1, 0
34
Analysis

• If satisfiable, fix a satisfying assignment
• For any unsatisfied clause, at least one of the
  two assignments is wrong
• Randomly picking one assignment and flipping it
  increase correct assignments by 1 w.p. ½
• This is a random walk on a line
• Hitting time (1 ? n) O(n2)
• Else never find an satisfying assignment

35
Discussion time

• Confused? Ask me now.
• Clear? Think about the following question Can we
  use the same algorithm for 3SAT?
• Whats the complexity of the same algorithm?
• Can you improve it?

36
Algorithm 2 Connectivity

• We know that BFS/DFS can solve connectivity.
• Now I want to use only O(log n) working space.
• BFS/DFS O(n).
• A simple randomized algorithm for
  st-Connectivity
• Starting from s, do random walk for O(n3) steps
• If weve never seen t, output NO otherwise
  output YES.

37
Tomorrow

• Randomized algorithm
• Polynomial Identity Testing.
• Primality Testing.
• Derandomization
• by k-wise independence approximation algorithm
  for 3SAT