Complexity Theory Lecture 6

1
Complexity TheoryLecture 6

• Lecturer Moni Naor

2
Recap

• Last week
• Probabilistic Complexity
• Schwatz-Zippel
• Approximating Counting Problems
• Plus
• Alternation
• This Week
• Non-Uniform Complexity Classes
• Polynomial Time Hierarchy

3
Machines that Take Advice

• Turing Machine with advice extra read only tape
• Advice string
• Machine M decides language L with advice A(n) if
• X 2 L implies M(x,A(x) yes
• X ? L implies M(x,A(x) no
• Advice A(n) is specific to the length
• Complexity Class with advice C is a complexity
  class
• and fN ? N.
• Language L 2 C/f(n) if there is a TM M and
  advice A(n) that decides language L and
• A(n) f(n)
• Example P/Poly(n) k P/nK

4
P/Poly
1index(x) x?L where L is undecidable

• Every unary language is in P/poly
• There are undecidable languages in P/poly
• Observation a language is in P/poly iff it has a
  polynomial sized circuit
• There exists a sequence of circuits C1, C2, and
  a polynomial p(n) such that
• circuit Cn computes f on inputs of size n
• circuit Cn is of size at most p(n)
• What is the power of P/poly
• is NP 2 P/poly?
• Homework show that if NP 2 P/log then PNP

5
Circuits

• Let B be a collection of Booleans functions
• A Circuit for on n Boolean variables x1, x2,
  xn over Basis B is a DAG with
• Each source is labeled with a literal
• Unique sink
• Each internal node is labeled with a function
  from B and has the appropriate indegree
• A circuit compute a function in the natural way
• Which (families of) functions have circuits whose
  size is bounded by polynomial in the input size?
• B , Ç, Æ
• Claim all f 2 P have polynomial size circuits
• Proof via the usual tableau. Size of circuit
  T(n)S(n).
• Using Oblivious Turing Machines T(n) log T(n)
• Claim most Boolean function f on n variables do
  not have polynomial sized circuits
• For any polynomial p(n) most functions need more
  p(n) gates
• Need ?(2n/n) gates to compute all functions on n
  variables, which is tight

Can determine the inputs to a given gate by a log
space machine
6
Polynomial Sized Circuits for BPP

• Theorem any f 2 BPP has a polynomial size
  circuit
• There exists a sequence of circuits C1, C2, and
  a polynomial p(n) such that
• circuit Cn computes f on inputs of size n
• circuit Cn is of size at most p(n)
• Several proofs (various derandomization methods)
• Construction of hitting set (for f 2 RP)
• Claim that a sequence r1, r2, rn exists where
  for each x 2 L at least one ri says yes
• Simulating large sample spaces by small ones
• Amplification
• Reduce the error so that a single assignment will
  be good for all x

7
Derandomization I

• Theorem any f 2 BPP has a polynomial size
  circuit
• Simulating large sample spaces
• Want to find a small collection of strings on
  which the PTM behaves as on the large collection
• If the PTM errs with probability at most ?, then
  should err on at most ?? of the small collection
• Choose m random strings
• For input x event Ax is more than (??) of the m
  strings fail the PTM
• PrAx e-2?2m lt 2-2n
• Prx Ax ?x PrAx lt 2n 2-2n1

Collection that should resemble probability of
success on ALL inputs
Bad ?
Good 1-?
Chernoff
8
Derandomization

• A major research question
• How to make the construction of
• Small Sample space resembling large one
• Hitting sets
• Efficient.
• Successful approach randomness from hardness
• (Cryptographic) pseudo-random generators
• Complexity oriented pseudo-random generators

9
Alternation

• Non-determinism a configuration leads to
  acceptance if there is an accepting leaf in the
  computation
• Similar to Or
• Can also consider leads to acceptance if all
  leaves are accepting
• Similar to And
• What if we alternate between the modes

10
Alternating Turing Machines

• An Alternating Turing Machines (ATM) is a
  non-deterministic TM whose state S SAND SOR
  are partitioned into two sets
• For input x consider the tree of computations
  where each node is a configuration
• The ATM is accepting if the root leads to an
  accepting configuration
• A leaf is accepting or not
• A node in state s 2 SAND leads to acceptance if
  both its children leads to acceptance
• A node in state s 2 SOR leads to acceptance if
  one of its children leads to acceptance

11
Alternating Time Classes

• ATIME(f(n)) class of language decided by an ATM
  where
• All computations halt after at most f(x) steps
• ASPACE(f(n)) class of language decided by an ATM
  where
• All computation halt
• Never use more than f(x) cells
• AL ASPACE(log n)
• Theorem AL P
• Theorem by simulating circuit
• Important point if f 2 P then the circuit is
  constructable in log space
• Simulate the circuit from the output towards the
  inputs

12
Oracle Machines

• Recall Oracle Turing Machine (OTM) have a
  special query tape where can ask membership
  queries and obtain answers
• Oracle for A can ask whether x 2 A
• Oracle Machine with oracle for A MA
• Can consider time classes with oracle CA
• PA Lthere is an OTM MA that decides L in
  polynomial time
• NPA Lthere is an OTM MA that decides L in non
  deterministic polynomial time
• Is PSAT NPSAT ? Is PPSPACE NPPSPACE
  ?
• Can consider Time Classes with oracle from
  another time class
• PNP Lthere is an OTM M AND a language A2 L
  where MA decides L in polynomial time

Homework
13
Motivation minimizing circuits

• Central problem in logic synthesis
• What is the complexity of this problem?
• NP-hard?
• Is it in NP, coNP, PSPACE?
• Complete for any of these classes?

?
Given a Boolean circuit C Is there a circuit C
of size smaller than C computing the same
function that C does?
?
?
?
?
?
x1
x2
x3

xn
14
The Polynomial-Time Hierarchy

• Can define many complexity classes using oracles
• Concentrate on classes that
• have natural complete problems
• have a natural interpretation in terms of
  alternating quantifiers
• help state consequences and containments

15
The Polynomial Time HierarchyVersion I Oracles

• ?0P ?0P ?0P P
• ?i1P P?iP
• Si1P NP?iP
• Pi1P coNP?iP
• PH i 0 SiP

Notation sometimes ?iP
16
Polynomial time hierarchy

• First level of the hierarchy
• ?1P P ?0P P
• S1P NP ?0P NP
• P1P CoNP ?0P CoNP
• If we believe the first level classes to be
  different from each other (and from the
  zero-level class(es)), are other levels also
  different?
• Does equality at one level imply one at the other
  levels?
• Are there interesting problems higher than the
  first level?

17
Polynomial time hierarchy

• Proposition ?iP??iP ? ?i1P ? ?i1P??i1P
• Proof
• ?iP? ?iP ? ?i1PP?iP
• Example NP?coNP ? PNP?2P
• We have an oracle for L ? ?iP
• Ask the oracle and return answer for ?iP
• Ask the oracle and return the opposite for ?iP
• (which is co-?iP).

18
Polynomially Balanced Relations

• A binary relation R µ 0,1n 0,1n is
  polynomially balanced if
• (x,y) 2 R, then y xk for some k
• Characterization of NP in terms of witnesses
• Claim L 2 NP iff there is a polynomially
  balanced relation such that
• R2 P
• Lx 9 y such that (x,y) 2 R
• Claim L 2 CoNP iff there is a polynomially
  balanced relation R2 P such that
• Lx 8 y such that y xk, (x,y) 2 R

19
The Polynomial Time HierarchyVersion II
Quantifiers

• Theorem Let L be a language and i 1. L 2
  SiP iff there is a polynomially balanced relation
  R such that
• R 2 Pi-1P
• and
• Lx 9 y such that (x,y) 2 R
• Proof by induction on i
• Interesting direction show how to build a
  certificate checkable in Pi-1P from computation
  of NP machine with Si-1P oracle

20
Proof of Equivalence

• Proof of Theorem
• induction on i
• Have seen base case
• Oracle definition implies quantifier definition
• By definition SiP NPSi-1P NPPi-1P
• Given Lx 9 y such that (x,y)2R and R2Pi-1P
  to see why it is in SiP
• Guess y, ask oracle if (x, y) ? R
• Can do since SiP NPPi-1P

21
Proof of Equivalence

• Quantifier definition implies oracle definition
• Given L ? Si NPSi-1 decided by an Oracle NTM M
  running in time nk
• First try R (x, y) y describes a valid path
  of Ms computation
• Valid path leading to qaccept
• Problem how to recognize valid computation path
  when it depends on results of oracle queries?

22
Proof of Equivalence

• R (x, y) y describes valid path of Ms
  computation
• Try
• valid path step-by-step description including
  correct yes/no answer for each A-oracle query zj
  (A ? Si-1P)
• e.g z1 ?A, z2 2 A, z3 ?A, zm ?A
• verify no queries in Pi-1P
• e.g z1 ?A ? z3 ?A ? ? zm ?A
• for each yes query zj , by induction.
• 9 wj, wj zjk with (zj, wj) ? R for some
  R ? Pi-2P
• For each yes query zj put wj in description of
  path y

23
Proof of Equivalence

• Single relation R in Pi-1P
• (x, y) ? R
• IFF
• all no zj ?A and
• all yes zj have (zj, wj) ? R and
• y is a path leading to qaccept.
• Key Point the AND of Pi-1 predicates is in Pi-1.

24
The Polynomial Time HierarchyVersion II

• Corollary Let L be a language and i 1. L 2
  PiP iff there is a polynomially balanced relation
  R such that
• (x,y)(x,y) 2 R 2 Si-1P
• and
• Lx 8y with y xk we have (x,y) 2 R
• Proof recall that Pi P is CoSiP
• Corollary Let L be a language and i 1. L 2
  SiP iff there is a polynomially balanced and
  polynomially decidable (i1)-ary relation R such
  that
• Lx 9 y1 8 y2 9 y3 Q yi such that (x,y1,
  y2, yi) 2 R
• Quantifier Q is 9 if i is odd and Q is 8 if i is
  even

Complete problem for ?iP
25
Collapse of the Hierarchy

• Theorem If for some i 1
• SiPPiP,
• then for all j i
• SjPPjP
• Proof suffices to show SiPPiP implies SiPSi1P
• Corollary If NPCoNP then the polynomial
  hierarchy collapses to the first level
• Similarly if PNP

26
Interesting Languages in the hierarchy

• Minimum equivalent problems
• Minimum circuit given a circuit C is it true
  that there is no circuit C with fewer gates that
  computes the same function as C
• Claim Minimum circuit is in P2P
• 8 C, C lt C 9 x such that C(x) ? C(x)
• Minimum circuit is not known to be lower in the
  hierarchy or complete for P2P

27
Complete Problems for S2P

• Minimum DNF given DNF f, integer k is there a
  DNF f of size at most k computing same function
  f does?
• Example
• x1x2x3 ? x1?x3 ? x4 ? x1x2 ? x1x3 ? x4
• Theorem (Umans 98) Minimum DNF is S2P-complete

28
Complete Problems for D2P

• Odd TSP given a weighted graph G,
• is the length of the shortest TSP tour in G odd?
• Theorem Odd TSP is ?2P-complete
• Homework Show Odd TSP is in ?2P

29
Oracles vs. Algorithms

• Given a polynomial time algorithm for SAT
• Can you solve Minimum Circuit efficiently?
• What other consequences?
• Given an oracle for SAT
• same input/output behavior as algorithm!
• Can you solve Minimum Circuit efficiently?
• Key issue is access to a program as a black box
  give you more power then given the code itself
• Recent developments in cryptography show a
  difference (Barak 2001)

30
BPP in the Hierarchy

• Do not know whether BPP µ NP
• Do not know whether BPP?EXP
• Theorem BPP µ P2P? S2P
• Sufficient to show BPP µ S2P and then by symmetry
  BPP µ P2P
• L 2 BPP Poly time PTM M
• x ? L ? PryM(x,y) accepts 2/3
• x ? L ? PryM(x,y) rejects 2/3
• Consider a game between ? and ? players
• ? player wants to prove x ? L and ? player wants
  to prove x ? L
• Need to choose y cooperatively
• First attempt
• ? player chooses ? 2 0,1n
• ? player chooses t 2 0,1n
• Set y ? ? t run M(x,y)
• There is a winning strategy for ? player as long
  as PryM(x,y) accepts lt1

31
BPP in the Hierarchy

• Since game was unfair to ? player give him
  several chances
• Several (parallel) instances of the game
• Force the ? player to use the same choice in all
  instances
• Second attempt
• ? player chooses ?1, ?1, ?m where each ?i 2
  0,1n
• ? player chooses t 2 0,1n
• Set yi ?i ? t run M(x,y1), M(x,y2),, M(x,ym)
  accept if any of the runs accepts
• Each ?i invalidates 2/3rds of possible t 2 0,1n
• Invalidates not a useful response for ? player
• Due to randomization by Xor
• By a hitting set argument there is a set of ?1,
  ?2, ?m that invalidates all t 2 0,1n for m n
  log3 2
• But what happens when x ? L?
• There might be ?1, ?2, ?3 such that
• i1,2,3 t M(x,y) accepts for y ?i ? t
  0,1n

Happens when bad witnesses are a coset of an ECC
Each of size at most 1/3
32
BPP in the Hierarchy

• Can apply error reduction Poly time PTM M
• use n random bits (y n)
• fraction of strings y for which M(x, y) is
  incorrect is at most 1/n
• Now by the hitting set argument there is a set of
  ?1, ?2, ?m that invalidates all t 2 0,1n for
  m n /log n
• But when x ? L for any choice of ?1, ?2,, ?m
  
• 1/ 2n i1m t M(x,y) accepts for y ?i ?
  t
• lt n /log n 1/n
• Final form
• ? player chooses ?1, ?1, ?m where each ?i 2
  0,1n
• ? player chooses t 2 0,1n
• Set yi ?i ? t run M(x,y1), M(x,y2),,
  M(x,ym) accept if any of the runs accepts
• clearly a S2P process

Each of size at most 1/n
33
More on BPP in the HierarchySimultaneous Provers

• To prove BPP ½ S2P we considered a game between ?
  and ? players
• ? player wants to prove x ? L and ? player wants
  to prove x ? L
• In S2P setting first ? makes a move and then ?
  player (based on ?s move)
• What if both players have to move simultaneously?
• Want strategy where
• ? player wins whenever x ? L even if ? player
  makes move after ? player
• ? player wins whenever x ? L even if ? player
  makes move after ? player
• Homework
• Phrase the above requirements in terms of a
  polynomial time relation R(x,y,z)
• Show that for all L ? BPP such a strategy exists

34
Polynomial sized circuits for SAT

• We know that P NP implies SAT has poly-sized
  circuits (SAT ? P/poly ).
• Showing SAT does not have poly-size circuits is a
  research direction for proving P ? NP
• Suppose SAT has poly-size circuits
• Can we show implies P NP ?
• Instead show SAT ? P/poly ? PH collapses (to
  second level)
• similar implication as if SAT ? P

35
Self Reducibility

• Lemma If SAT ? P/poly then there exists a
  polynomial p(n) and family of circuits C1, C2
  such that Cn gets an n sized formula ? and
  outputs
• A satisfying assignment if ? is satisfiable
• Null if ? is not satisfiable
• Corollary If SAT ? P/poly and R is polynomial
  time balanced relation, then for
• L x ?z (x,z) 2 R
• there is a family of circuits C1, C2 such that
  
• Lx xn implies (x,Cn(x)) 2 R
• Can define new polynomial relation R such that
  (x,C) 2 R iff (x,C(x))2R and C is of the right
  size.

36
Collapse of The Hierarchy

• Theorem if SAT ? P/poly then PH collapses to the
  second level.
• Proof
• Show that P2P µ S2P
• If L ? P2P then we know
• L x ?y ?z (x, y, z) 2 R
• for R ? P.
• ?z (x, y, z) ? R? is in NP
• Let Cn be the SAT solver produces certificate z
• ?z (x, y, z) ? R iff (x, y, Cn(x,y)) ? R
• L x ?Cn ?y (x, y, Cn(x,y)) 2 R
• x ?Cn ?y (x, y, Cn) 2 R ?
  S2P

37
Approximate Counting is in the Hierarchy

• Theorem For any f 2 P there is a g 2 ?3P such
  that on input x and ? g(x, ?) outputs a (1 ?)
  approximation to f(x)
• (1-?)g(x, ?) f(x) (1?)g(x, ?)
• Proof key point method for proving that sets
  are of certain size
• The set in question
• A Accept(M(x)) µ 0,1m

A function f is in P if there exists a NTM M
running in polynomial time where M(x) has f(x)
accepting paths for all x 2 0,1n
38
Pair-wise Independent Hash Functions

• A family of hash functions
• Hhh0,1m ? 0,1l
• is pair-wise independent if for all x ? y, for
  random h 2RH the random variables h(x), h(y) are
  uniformly distributed and independent
• In particular Prh(x)h(y) 1/2l
• Already saw the approximate version for
  distributed equality testing
• Prh(x)h(y) ?
• Constructing H
• Construction based on matrix multiplication.
• Treat x as a m 1 vector.
• The function is defined by a random m l matrix
  over GF2.
• HhAA 2 GF2m l and hA(x)Ax

39
Methods for proving sizes of sets

• Let H be family of hash functions where for h 2 H
  h0,1m ? 0,1l
• if h 2 H is 1-1 on A then clearly A 2l
• a bit difficult to expect full uniqueness
• Relaxed property
• if there are h1, h2, hl 2 H are such that
• for all x 2 A there is an 1 i l where for
  all x 2 A\x we have hi(x) ? hi(x)
• then we know A lt l 2l
• Claim for a pair-wise independent family H, if
  A 2l-1 , then there exist h1, h2, hl 2 H
  with the no collisions property. Denote with
  UH(A,l).
• Claim for AAccept(M) we have UH(A,l) 2 S2P
• Can eliminate the quantifier on i by explicitly
  going over all possibilities!
• Conclusion with oracles calls to UH(A,l) can
  find smallest l for which UH(A,l) holds
• Yields a 2l approximation for f(x)
• Can use amplification of f(x) by running M k
  times (from accepting positions).
• New machine M will have fk(x) by accepting paths

No collisions property
40
Proof of No Collisions Claim

• For any x 2 A and any x 2 A\x if h is chosen
  at random from H
• Prh(x)h(x) 1/2l
• By the Union bound
• Prh(x) is unique A 1/2l 1/2
• Hitting set Construct h1, h2, hl one by one,
  each time covering at least ½ of elements in A
  that have not been unique so far
• Let A be the set not covered so far
• For any x 2 A and any x 2 A\x for random h 2R
  H
• Prh(x) is unique A 1/2l 1/2

Note this is thefull A
41
Approximate Counting and Uniform Generation

• For self reducible problems
• exact counting implies exact uniform generation
  of witnesses/solutions
• Approximate counting almost uniform generation
  of solutions
• Theorem if PNP then given Boolean circuit C
  possible to sample almost uniformly at random
  satisfying assignment to C

42
The classes we discussed
EXP
PSPACE
P

• PSPACE
• Complete problems TQBF, 2-person games,
  generalized geography,
• 3rd level
• Complete problems VC dimension for succinct sets
• Containment Approximate P
• 2nd level
• Complete problems Min DNF, Succinct Set Cover
• Containment Min Circuit, BPP
• 1st level
• Complete problems SAT, UNSAT,lots
• Containment factoring, graph isomorphism

PH
S3P
?3P
?3P
S2P
?2P
?2P
NP
coNP
BPP
P
43
Some History

• Polynomial Time Hierarchy Meyer-Stockmeyer
• BPP in the Hierarchy Lautmann, Sipser
• Approximate Counting in the Hierarchy Stockmeyer
  
• SAT ? P/poly implies collapse Karp-Lipton,
  Sipser
• Stockmeyers papers available
• http//www.geocities.com/stockmeyer_at_sbcglobal.net/
  
• Survey on problems high in the hierarchy
  Schaefer and Umans
• http//www.cs.caltech.edu/umans