The PCP Theorem by Gap Amplification

1
The PCP Theorem byGap Amplification

• Proven by Irit DinurExposition by
  Radhakrishnan-Sudan
• Lecture by Nick Harvey

2
How to give a proof?
Prover
Verifier
Is 728950327 prime?

• Prover writes a proof
• Verifier checks all of it.
• If theorem is true, Verifier is convinced.
• If theorem is false, no proof can convince her.

3
Probabilistically Checkable Proofs
Prover
Lazy verifier
Is 728950327 prime?

• Prover writes a proof
• Verifier rolls dice checks a few bits of proof.
• If theorem is true, Verifier is convinced.
• If theorem is false, Pr Verifier is fooled lt
  ½.(This probability is called the Soundness.)

4
PCP Classes

• What theorems can be proven by PCPs?

5
PCP Classes

• What languages can be decided by PCPs?
• Depends on dice, queries, alphabet size
• The PCP Theorem Arora, Lund, Motwani, Sudan,
  Szegedy 92NP PCP O(log n) dice, O(1)
  queries, alphabet size 2,
  soundness 1/2

6
Toy PCP Example
Is this graph 3-colorable?
Prover
Lazy verifier
7
Toy PCP Example
Is this graph 3-colorable?
Prover
Lazy verifier

• Prover produces coloring
• Verifier picks a random edge and checks colors
• If graph is 3 colorable, Verifier is always
  convinced
• If graph not 3 colorable, Verifier can be
  fooled.Prob Victor is fooled 1 O(1/n).

8
Toy PCP Example
Is this graph 3-colorable?
Prover
Lazy verifier

• Prover produces coloring
• Verifier picks a random edge and checks colors
• If graph is 3 colorable, Verifier is always
  convinced
• If graph not 3 colorable, Verifier can be
  fooled.Prob Verifier is fooled 1 - O(1/n)

Soundness is too low!Want ?(n) bad edges! Want
Gap to be ?(1)!
GAP
9
Ideal Reduction
Graphs onpoly(n) vertices
0,1
Any NP Language
3 colorablegraphs
L
Transformation T
Far from 3 colorablegraphs
L

• No such transformationis known.
• But we can do something similar!

In every 3 coloring, a constantfraction of edges
are bad
10
Generalized Graph Coloring
cu,v
Constraint cu,v
vs color
0 1 1
1 1 0
0 0 1
us color

• G ( V, E, ?, C ) where? is the alphabet of
  colorsC ce e in E is a set of
  constraintswhere ce ?? ? 0,1
• Let GK instances with ?K .

11
Generalized Graph Coloring
cu,v
Constraint cu,v
vs color
0 1 1
1 1 0
0 0 1
us color

• G is satisfiable if there is a coloring
  satisfying all constraints.
• G is ?-far from satisfiable if every coloring
  leaves at least ?E constraints unsatisfied.

12
Main Theorem

• There exists a constant ?gt0and a polynomial time
  transformation T s.t.

G16 (generalized graph coloringinstances with
?16)
0,1
Satisfiablegraphs
Any NP Language
L
Transformation T
?-far fromsatisfiable graphs
L
13
Main Theorem

• There exists a constant ?gt0and a polynomial time
  transformation T s.t.

G16 (generalized graph coloringinstances with
?16)
All graphs
Satisfiablegraphs
3-colorablegraphs
Transformation T
?-far fromsatisfiable graphs
Non-3-colorablegraphs
14
Main Theorem

• There exists a constant ?gt0and a polynomial time
  transformation T graphs ? G16
  s.t.3-colorable graphs map to satisfiable
  graphsand non-3-colorable graphs map to ?-far
  fromsatisfiable graphs.

Corollary

• NP ? PCP O(log n) dice, 2 queries,
  alphabet size 16, soundness 1-?

15
How To Prove Main Theorem?

• Trivial for ?O(1/n) (Call ? the Gap)
• Idea Find a transformation T that
• Increases gap by a constant factor
• Increases vertices edges by a constant factor
• Maintains alphabet size 16
• Apply T only O(log n) times
• Done!

16
How To Prove Main Theorem?

• Find a transformation T that
• Increases gap by a constant factor c
• Increases vertices edges by a constant factor
• Maintains alphabet size 16
• Handy Lemma Arora et al.There exists a
  constant ?gt0 such thatfor any constant alphabet
  size K,there exists a map M GK ? G16that
  shrinks gap by a factor ?.
• Pro Alphabet size shrinks Con Gap
  also shrinks

17
Dinurs Goal

• Find a transformation T that
• Maps G16 ? GK, where K is a constant
• Increases gap by a large constant factor c
• Increases vertices edges by a constant factor
• Note 0 ? gap ? 1, so the transformationnecessari
  ly fails once gap is large enough.

18
What to do?

• Setup
• Given G(V,E,?,C), construct G(V,E,?,C).
• Want 2 queries in G to somehow test
  constraintsfor many edges in G.
• Silly Approach
• Let G be the complete graph
• Increase the alphabet size to ??n
• Each vertex stores an opinion of other
  vertices colors
• Constraint in C for edge u,v checks thatus
  opinions all match vs opinions, and thatall
  constraints in C are satisfied.

19
What to do?
G
G

• Silly Approach
• Let G be the complete graph
• Increase the alphabet size to ??n
• Each vertex stores an opinion of other
  vertices colors
• Constraint in C for edge u,v checks thatus
  opinions all match vs opinions, and thatall
  constraints in C are satisfied.

20
What to do?
Colors of allnearby vertices

• Dinurs Approach
• Let t be a constant
• Each vertex stores opinions of its neighbors at
  distance ? t
• Want storage at each vertex to be a constant
• ? Vertices should have constant degree d.
  Storage ? 16dt
• Verifier queries two nearby vertices and checks
  someof their nearby edges

21
Implementing Dinurs Approach

• Let t be a large constant
• Pick vertex a at random
• Do a random walk from a. Halt at each step w.p.
  1/t.

B(a,t)
(Ball of radius taround a)
e1
e2
e3

v1
v2
v3
vTb
av0
Elength of walkt

• Both a and b have opinions about colors of
  verticesin B(a,t) ? B(b,t)
• Verifier tests if as and bs opinions agree, and
  if theconstraints are satisfied on those edges.

22
When does Verifier reject?

• Let t be a large constant
• Pick a vertex a at random
• Do a random walk. Stop at each step w.p. 1/t.

FAILURE!Conflicting Opinions
B(a,t)
B(b,t)
e1
e2
e3

v1
v2
v3
av0
vTb

• Both a and b have opinions about verticesin
  B(a,t) ? B(b,t)
• Verifier tests if as and bs opinions agree, and
  if theconstraints are satisfied on those edges.

23
When does Verifier reject?

• Let t be a large constant
• Pick a vertex a at random
• Do a random walk. Stop at each step w.p. 1/t.

FAILURE!Violated Constraint
B(a,t)
B(b,t)
e1
e2
e3

v1
v2
v3
av0
vTb

• Both a and b have opinions about verticesin
  B(a,t) ? B(b,t)
• Verifier tests if as and bs opinions agree, and
  if theconstraints are satisfied on those edges.

24
Analysis of Dinurs Approach

• If G is satisfiable, then prover supplies valid
  proofand verifier accepts with probability 1.
• Main Lemma Let G be ?-far from satisfiable.
  Then verifier rejects every coloringwith
  probability ?(?t) (until ? reaches 1/t).
• Caveats (1) G must have constant degree d.
  (2) G must have good expansion.
• Corollary Dinurs goal is achieved! (Modulo
  caveats)We have a transformation T that
• Maps G16 ? GK, where K?16dt
• Increases gap by a factor t
• Increases vertices edges by a constant factor

25
View of Dinurs Transformation
G
G
New Edges(connect vertices at distance ? t)

• Corollary Dinurs goal is achieved! (Modulo
  caveats)We have a transformation T that
• Maps G16 ? GK, where K?16dt
• Increases gap by a factor t
• Increases vertices edges by a constant factor

26
Remaining tasks for this talk

• Proof of Main Lemma
• Transforming graphs to constant degree
• Transforming graphs to expanders

Postponeuntil end
27
Breaktime Puzzle!

• I have two (biased) dice Da and Db
• Say their outcomes have probabilities
• a1 ? a2 ? ? a6 where Si ai 1
• b4 ? b2 ? ? b5 where Si bi 1
• (ordering is irrelevant)
• Roll Da and Db once each, independently
• What is Pr outcomes differ ?
• (I want a very simple lower bound, depending on
  ais bis)

28
Review of first half

• Want a transformation T that
• Maps G16 ? GK, where K is a constant
• Increases gap by a large constant factor c
• Increases vertices edges by a constant factor

29
Analysis of Dinurs Approach

• Suppose G is ?-far from satisfiable
• Let A be any coloring of G
• Want to show Pr verifier detects fault in G
  ? ?(?t)
• How?
• Show that A induces a coloring A of G
• A has many faulty edges by assumption
• Show that faults in A also cause A to have
  faults
• Formally
• Let N be an RV faulty edges of G detected by
  verifier
• Want to show Pr Ngt0 ? ?(?t)
• Via second-moment method Pr Ngt0 ? E N 2 /
  E N2
• Will show E N is large and E N2 is small

30
Remaining Tasks

• Define coloring A of G induced by coloring A of
  G
• Let F be faulty edges of coloring A
• Let N be an RV edges of F detected by
  verifier
• Show E N ? ?( tF / E )
• Show E N2 ? O( tF / E )
• Thus Pr Ngt0 ? ? ?( tF / E
  ) ? ?(t?)

(end of Main Lemma)
31
Induced Coloring A

• Notation
• Au(v) us opinion of vs color in coloring A
• A(v) vs color in A
• Intuition for defining A
• Color A(u) should be a majority opinion of us
  color in A
• Definition of A should be easy to analyzewhen
  considering verifiers behavior
• Verifier uses random walk ? define A via random
  walk
• Formally
• Perform random walk of length ? t-1 from v in G.
• Let final vertex be u. Then u votes for color
  Au(v).
• Let A(v) be the majority vote.

32
How to pick A(v)?
r
Votes for vs color
z
As(v)
s
Az(v)
y
R
G
B
1
1
6 3 1
w
u
x
v

• Formally
• Perform random walk of length ? t-1 from v in G.
• Let final vertex be u. Then u votes for color
  Au(v).
• Let A(v) be the majority vote.

33
Remaining Tasks

• Define coloring A of G induced by coloring A of
  G
• Let F be faulty edges of coloring A
• Let N be an RV edges of F detected by
  verifier
• Show E N ? ?( tF / E )
• Show E N2 ? O( tF / E )
• Thus Pr Ngt0 ? ? ?( tF / E
  ) ? ?(t?)

(end of Main Lemma)
34
E N is large
Let F be faulty edges of coloring A Let N be
an RV edges of F detected by verifier

• Focus on a single edge e?F
• Suppose walk traverses e and u,v ? B(a,t) ?
  B(b,t)
• Verifier performs 3 tests
• Is Aa(u) Ab(u)? Is Aa(v) Ab(v)?
  Is ce( Aa(u), Ab(v) )1?
• Intuition
• If opinions of nearby vertices differ then Tests
  1 and 2 will fail
• If opinions of nearby vertices match then Test 3
  will fail

e
B(a,t)
B(b,t)


u
v
a
b
Test 1
Test 2
Test 3
35
E N is large
Let F be faulty edges of coloring A Let N be
an RV edges of F detected by verifier

• Focus on a single edge e?F
• Suppose walk traverses e and u,v ? B(a,t) ?
  B(b,t)
• Verifier performs 3 tests
• Is Aa(u) Ab(u)? Is Aa(v) Ab(v)?
  Is ce( Aa(u), Ab(v) )1?
• Pr verifier detects fault ? maxi Pr Test i
  detects fault
• Define pu Pr Aa(u) A(u) d(a,u)ltt
• pv Pr Ab(v) A(v) d(b,v)ltt

e
B(a,t)
B(b,t)


u
v
a
b
Test 1
Test 2
Test 3
Unknown parameters
36
E N is large

• Verifier performs 3 tests
• Is Aa(u) Ab(u)? Is Aa(v) Ab(v)?
  Is ce( Aa(u), Ab(v) )1?
• Define pu Pr Aa(u) A(u) d(a,u)ltt

Test 1
Test 2
Test 3
(Unknown)
Pr Test 1 detects fault
? Pr d(a,u)ltt ? d(b,v)ltt ? Test 1 detects fault

? (1-1/e)2 Pr Test 1 detects fault d(a,u)ltt ?
d(b,v)ltt
(1-1/e)2 Pr Aa(u)?Ab(u) d(a,u)ltt ?
d(b,v)ltt
Roll two independent, biased dice Da and Db.What
is Pr values differ ?
? (1 (prob of most likely value for Da)
37
E N is large

• Verifier performs 3 tests
• Is Aa(u) Ab(u)? Is Aa(v) Ab(v)?
  Is ce( Aa(u), Ab(v) )1?
• Define pu Pr Aa(u) A(u) d(a,u)ltt

Test 1
Test 2
Test 3
(Unknown)
Pr Test 1 detects fault
? Pr d(a,u)ltt ? d(b,v)ltt ? Test 1 detects fault

? (1-1/e)2 Pr Test 1 detects fault d(a,u)ltt ?
d(b,v)ltt
(1-1/e)2 Pr Aa(u)?Ab(u) d(a,u)ltt ?
d(b,v)ltt
? (1 (prob of most likely value for Da)
By definition, A(u) is the most likely value for
Aa(u)!
? (1-1/e)2 (1-pu)
38
E N is large

• Verifier performs 3 tests
• Is Aa(u) Ab(u)? Is Aa(v) Ab(v)?
  Is ce( Aa(u), Ab(v) )1?
• Define pu Pr Aa(u) A(u) d(a,u)ltt
• pv Pr Ab(v) A(v) d(b,v)ltt

Test 1
Test 2
Test 3
Unknown
Pr Test 3 detects fault
? Pr d(a,u)ltt ? d(b,v)ltt ? Test 3 detects fault

? (1-1/e)2 Pr Test 3 detects fault d(a,u)ltt ?
d(b,v)ltt
? (1-1/e)2 Pr Aa(u)A(u) ? Ab(v)A(v)
d(a,u)ltt ? d(b,v)ltt
a and b are independent
(1-1/e)2 pu pv
39
E N is large

• Verifier performs 3 tests
• Is Aa(u) Ab(u)? Is Aa(v) Ab(v)?
  Is ce( Aa(u), Ab(v) )1?
• Define pu Pr Aa(u) A(u) d(a,u)ltt
• pv Pr Ab(v) A(v) d(b,v)ltt

Test 1
Test 2
Test 3
Unknown
Pr Test 1 detects fault ? (1-1/e)2 (1-pu)
Pr Test 2 detects fault ? (1-1/e)2 (1-pv)
Pr Test 3 detects fault ? (1-1/e)2 pu pv
Pr Verifier detects fault ? maxi Pr Test i
detects fault
? (1-1/e)2 max 1-pu, 1-pv, pupv
v
? (1-1/e)2 ( 5 - 1) / 2 ? 1/8
40
E N is large

• Verifier performs 3 tests
• Is Aa(u) Ab(u)? Is Aa(v) Ab(v)?
  Is ce( Aa(u), Ab(v) )1?
• Define pu Pr Aa(u) A(u) d(a,u)ltt
• pv Pr Ab(v) A(v) d(b,v)ltt

Test 1
Test 2
Test 3
Punchline e is a faulty edge for G under
coloring A If verifiers random walk traverses
e, then it will detect a fault with probability
1/8.
Unknown
Pr Test 1 detects fault ? (1-1/e)2 (1-pu)
Pr Test 2 detects fault ? (1-1/e)2 (1-pv)
Pr Test 3 detects fault ? (1-1/e)2 pu pv
Pr Verifier detects fault ? maxi Pr Test i
detects fault
? (1-1/e)2 max 1-pu, 1-pv, pupv
v
? (1-1/e)2 ( 5 - 1) / 2 ? 1/8
41
Wrapup E N is large

• Notation
• Let F be faulty edges of coloring A
• Let N be an RV edges of F detected as faulty
  by verifier
• Focus on a single edge e?F
• Let Me times e is traversed by verifiers
  random walk
• Let Ne times e is traversed by verifiers
  random walkand is detected to be faulty
• Thus N Se?F Ne

E Ne
E Ne Me0 Pr Me0 E Ne Me?1 Pr
Me?1
? (1/8)E Me Me?1 Pr Me?1 (1/8)E Me
ith edge of walk isdistributed uniformlysince G
is regular
(1/8)E length of random walk / E
(1/8) t / E
?
E N Se?F E Ne tF / 8E ?t / 8
42
Remaining Tasks

• Define coloring A of G induced by coloring A of
  G
• Let F be faulty edges of coloring A
• Let N be an RV edges of F detected by
  verifier
• Show E N ? ?( tF / E )
• Show E N2 ? O( tF / E )
• Thus Pr Ngt0 ? ? ?( tF / E
  ) ? ?(t?)

(end of Main Lemma)
43
E N2 is small

• Let e1, e2, , be edges chosen by verifier
• Let ?i be indicator that ? i edges chosen and ei
  ? F
• Simplifying Assumption
• Edges chosen independently, not by random walk
• ? Pr?j1 ?i1 (1-1/t)j-i F/E
  (if jgti)

EN2 E (?i?1 ?i)2
? 2 ?j?i E ?i ?j
2 ?i?1Pr?i1 ?j?i Pr?j1 ?i1
2 ( tF / E ) ( 1 ?k?1 (1-1/t)k F/E )
lt 2 ( tF / E ) ( 1 t F / E )
lt 4 ( tF / E )
So long as ?F/Elt1/t
44
E N2 is small

• Let e1, e2, , be edges chosen by verifier
• Let ?i be indicator that ? i edges chosen and ei
  ? F
• Claim If G is an expander then eis on random
  walkare almost pairwise independent.
• Pr?j1 ?i1 (1-1/t)j-i ( F/E
  (1-?2/d2)j-i-1 ) (if jgti)

Expansion of G
EN2 E (?i?1 ?i)2
? 2 ?j?i E ?i ?j
2 ?i?1Pr?i1 ?j?i Pr?j1 ?i1
2 ( tF / E ) (1 ?k?1 (1-1/t)k
(F/E(1-?2/d2)k-1) )
lt 2 ( tF / E ) (1 t F/E ?2/d2 )
lt 4 ( tF / E ) (1 ?2/d2)
45
E N2 is small

• Let e1, e2, , be edges chosen by verifier
• Let ?i be indicator that ? i edges chosen and ei
  ? F
• Claim If G is an expander then eis on random
  walkare almost pairwise independent.
• Prej?F e1?F F/E (1-?2/d2)j-2 (if
  jgt1)

EN2 E (?i?1 ?i)2
? 2 ?j?i E ?i ?j
2 ?i?1Pr?i1 ?j?i Pr?j1 ?i1
2 ( tF / E ) (1 ?k?1 (1-1/t)k
(F/E(1-?2/d2)k-1) )
lt 2 ( tF / E ) (1 t F/E ?2/d2 )
lt 4 ( tF / E ) (1 ?2/d2)
46
Remaining Tasks

• Define coloring A of G induced by coloring A of
  G
• Let F be faulty edges of coloring A
• Let N be an RV edges of F detected by
  verifier
• Show E N ? ?( tF / E )
• Show E N2 ? O( tF / E )
• Claim If G is an expander then eis on random
  walkare almost pairwise independent.
• Prej?F e1?F F/E (1-?2/d2)j-2 (if
  jgt1)
• Thus Pr Ngt0 ? ? ?( tF / E
  ) ? ?(t?)

(end of Main Lemma)
47
Random Walks on Expanders

• Claim Prej?F e1?F F/E (?2/d)j-2
  (if jgt1)
• Notation
• ?2 ? d-?2/d is the second-largest eigenvalue
• ?F(v) edges in F incident on v
• v1, v2, are the vertices on random walk
• ?i is the distribution on vi, conditioned on e1?F
• M is transition matrix of the random walk
• Preliminaries
• ?1(v) ?F(v) / 2F
• ?1 (1/n)1 ?F / 2Fwhere ?F is component
  of ?F orthogonal to 1
• ?j-1 Mj-2 ?1 Mj-2 ( (1/n)1 ?F / 2F )
• Prej?F e1?F ?v (?F(v)/d)?j-1(v) (?FT
  ?j-1)/d

48
Random Walks on Expanders

1. ?1(v) ?F(v) / 2F
2. ?1 (1/n)1 ?F / 2F
3. ?j-1 Mj-2 ?1 Mj-2 ( (1/n)1 ?F / 2F )
4. Prej?F e1?F ?v (?F(v)/d)?j-1(v) (?FT
   ?j-1)/d

• Prej?F e1?F (?FT ?j-1)/d (from (4))
• (?FT Mj-2 ( (1/n)1 ?F/2F ) ) / d
  (from (3))
• (1/nd)?FT 1 (1/2Fd) ?FT Mj-2 ?F
  (linearity)
• ? F/E (1/2Fd) ?F Mj-2 ?F
  (Cauchy-Schwarz)
• ? F/E (1/2Fd) ?F (?2/d)j-2
  ?F (since ?F - 1)
• F/E (1/2Fd) (?2/d)j-2 (?v ?F(v)2)
• ? F/E (1/2Fd) (?2/d)j-2 (d ?v ?F(v))
• F/E (?2/d)j-2

(end of Claim)
49
Remaining tasks for this talk

• Proof of Main Lemma
• Transforming graphs to constant degree
• Transforming graphs to expanders

50
Transforming Graphs to Constant Degree
Introduce dummyvertices!
v
51
Transforming Graphs to Constant Degree
Want to forcedummies to have same color
52
Transforming Graphs to Constant Degree
Clique? Degree too large! Too many edges
Add constraints forcing equality
53
Transforming Graphs to Constant Degree
Cycle? Half vertices can halfwrong color but
onlytwo violated edges.
Want a graphwhere cutshave many edges
54
Transforming Graphs to Constant Degree
Expander? Just right!
55
Transforming Graphs to Expanders
G is not an expander
H is an expander
56
Transforming Graphs to Expanders
G ? H is an expander