Randomized Algorithms

1
?????Randomized Algorithms

• ??? (Hsueh-I Lu)
• http//www.iis.sinica.edu.tw/hil/

2
Today

• About midterm
• Fingerprinting techniques (?????)

3
Yaos ? O(n log n) for randomized sorting

• Find a probabilistic distribution over all
  possible inputs and argue that any deterministic
  algorithm has expected running time O(n log n)
  with respect to this probabilistic distribution.

4
Uniform distribution

• There are exactly n! possible ordering for the
  length-n input numbers.
• Let each possible ordering appears with
  probability exactly 1/(n!).
• It remains to show that with this probabilistic
  distribution, any deterministic algorithm runs in
  expected O(n log n) time.
• Note that this is about probabilistic analysis
  of a deterministic algorithm.

5
Sorting by comparison

• Each deterministic algorithm can be described by
  a binary tree with exactly n! leaves.

6
Sorting by comparison
7
Leaf ? an input

• The depth of a leaf is the number of comparisons
  required for the corresponding input.
• The average depth of all n! leaves is the
  expected running time for the deterministic
  algorithm described by the binary tree.

8
Binary tree with n! leaves

• There can be at most 2k leaves with depth k or
  less.
• Let k 0.5 log2 n!, so 2k (n!)0.5.
• At most (n!)0.5 leaves have depths 0.5 log (n!)
  or less.
• At least n! - (n!)0.5 leaves have depths O(n log
  n).
• The average depth of leaves is O(n log n).

9
MAXCUT

• Let x1, x2, , xn be the nodes of G.
• Let Gi be the sub-graph of G induced by x1, x2,
  , xi.
• For i 1 to n do
• Assign xi to L or R depending on which choice
  maximizes the number of crossed edges in Gi with
  respect to the choices made for x1, x2, , xi-1.

10
Derandomization
11
The decision for xi
xi
L
R
12
0.5 Dumb 0.5 Dumber Clever

• Yes, its possible. Just let problem Pi be to
  minimize 1 over the number of satisfied clauses. ?

13
Fingerprinting
14
Is X equal to Y?
?

15
Are Two Matrices Identical?
16
The problem

• Input two n-by-n matrices A and B.
• Output determining whether A B?

?
17
Naïve approach

• Comparing all elements.
• Needs n2 comparisons.

Can we reduce the number of comparisons if a
small probability of error is allowed?
18
Fingerprinting approach

• Choose an n-element column vector r,
• where each element of r is either 0 or 1
  independently and equally likely.
• Comment As a matter of fact, 0 and 1 can be any
  two distinct numbers.
• Compare Ar and Br, and output whether they are
  identical.

19
Fingerprinting
Only n comparisons.
20
Error Probability?
21
Proof
22
Bounding the probability
23
Therefore,
24
Criticism Computing Ar takes O(n2) time.
Only n comparisons.
25
More interesting exampleIs AB C?

• Input three n-by-n matrices A, B, C.
• Output determining whether AB C.
• Naïve approach takes O(n3) time for matrix
  multiplication and O(n2) time for comparison.
• Or, the fast matrix multiplication takes O(n2.37)
  time.

26
Fingerprinting approach

• Choose an n-element column vector r,
• where each element of r is either 0 or 1
  independently and equally likely.
• Comment As a matter of fact, 0 and 1 can be any
  two distinct numbers.
• Compare ABr and Cr, and output whether they are
  identical.

27
Fingerprinting
A
C
B
?

B
r
A
Only n comparisons.
Only O(n2) comparisons.
Only O(n2) operations
28
Error Probability?
29
Are two Polynomials Identical?
30
Is A(x)B(x) C(x)?

• Input
• A(x) and B(x) are two degree-n polynomials.
• C(x) is a degree-2n polynomial.
• Output
• Determine whether the product of A(x) and B(x) is
  equal to C(x).

31
Naïve approach

• Multiplying A(x) and B(x) takes O(n log n) time
• e.g., using Fast Fourier Transform.
• Comparing A(x)B(x) and C(x) takes O(n) time.

32
Fingerprinting approach

• Let R consist of arbitrary 4n distinct numbers.
• Choose a number r from R uniformly at random.
• Evaluate A(r)B(r) and C(r), and output whether
  they are identical.
• Time complexity
• O(n) for computation
• O(1) for comparison.

33
Error Probability?
34
Proof
35
Are Two Strings Identical?
36
The problem

• Input Two n bit strings A and B.
• Output determining whether A B.
• Naïve approach requires comparing all n bits of
  both strings.

37
????? (Alice and Bob)
38
The story

• Each of Alice and Bob maintains a copy of the
  same n-bit database. They have to make sure that
  both copies are indeed identical.
• Can they discover inconsistency with high
  probability by comparing only poly-log number of
  bits?

39
Fingerprinting approach

• Let a (resp., b) be the n-bit number whose binary
  representation is A (resp., B).
• Randomly choose a polylog(n)-bit prime number p.
• (For example, p ?2n2 log2 n.)
• Alice compute the remainder r of a divided by p.
• Bob compute the remainder s of b divided by p.
• Output whether r s.

40
Error Probability?
41
Two Facts

• Prime Number Theorem
• The number of primes smaller than m is
  asymptotically m/ln m.
• Observation
• The number of distinct prime divisors of an n-bit
  number is O(n). (Why?)

42
Proof
43
Verifying the number of satisfying truth
assignments
44
The SAT problem
45
Examples
46
The class NP

• L?NP Each member x of L has a short and easily
  verifiable proof for showing x?L (succinet
  certificate for membership)
• e.g. graph isomorphism ?NP

47
SAT belongs to NP,

• because each satisfiable set of clauses has a
  polynomial-sized proof for its satisfiability,
  i.e., one of its satisfying truth assignments.

48
The SAT problem
49
Examples
50
SAT belongs to IP
51
Proof system
Accept
Prover
Verifier
Short proof
Reject

• ???
• ????
• unlimited resource
• exp. computing power

• ??
• ???
• limited resource

52
Proof system
Accept
P
V
Short proof
Reject

• ?? completeness
• x?L ? ltP,Vgt(x)1.
• ?? soundness
• x?L ? ltP,Vgt(x)0, ? (malicious) P.
• ??efficiency
• V runs in polynomial time.

53
Proof system ? Interactive proof system.

• L?NP ? L has such a proof system
• ? generalization
• Interactive Proof System
• Allowing interaction
• Allowing V to throw dice

54
Definition

• A pair of interactive (randomized) algorithms P
  and V forms an interactive proof system for L if
• completeness
• ? x?L, PrltP,Vgt(x)1 ?
• soundness
• ? x?L, ? P, PrltP,Vgt(x)1 ?
• tractability
• V runs in polynomial time.

55
IP consists of problems having interactive proof
systems
NP
IPPSPACE Shamir JACM 92
56
Interactions in computation can be powerful

• in, e.g., zero-knowledge proofs.
• The Coke Problem
• The Labyrinth Problem
• The Graph Non-Isomorphism Problem.

57
The Coke Problem
58
The Labyrinth Problem
59
Graph Non-Isomorphism

• Input G0 and G1
• Output
• 1, if G0 and G1 are non-isomorphic
• 0, if G0 and G1 are isomorphic.

60
GNI belongs to IP

• An interactive proof for GNI
• V1
• Select b from 0,1 uniformly at random
• Select a random permutation ? over 1, 2, , n
• Send G ?(Gb) to P
• P1 compute a number a?0,1, with Ga G
• V2 if a b, then accept otherwise reject.

61
Is this an interactive proof system?

• Q1 tractability? Yes!!??verifier???
• Q2 completeness?
• (G0,G1)?GNI G0 and G1 arent isomorphic.
• ? a b ? PrltP,Vgt(G0,G1) 1 1
• Q3 soundness?
• G0 G1
• ?One can prove that
• ?P, PrltP,Vgt(G0, G1) 1 ½.

62
Illustration
GNI
IPPSPACE
NP
63
Goal SAT belongs to IP
SAT
IPPSPACE
P
64
Technique Representing a clause set by a
polynomial
65
Q(x1,x2,,xn)
66
The goal