Complexity Theory Lecture 3

1
Complexity TheoryLecture 3

• Lecturer Moni Naor

2
Recap

• Last week
• Non deterministic communication complexity
• Probabilistic communication complexity
• Their Relationship
• Lots of techniques
• This Space Complexity

3
Summary of techniques and ideas

• Probabilistic method to show that a
  combinatorial object exists choose a random one
  and show that it satisfies desired properties
  with Probgt0
• Constructive vs. non-constructive methods
• Union bound define a collection of bad events
  A1, A2, An
• Pri Ai ?i PrAi n PrAi
• Simulating large sample spaces by small ones
• Reversing roles via Minimax Theorem

4
Perfect Hash Functions

• A family H of functions is (n,k,l) perfect if
• 8 h 2 H h1,,n ? 1,, l
• 8 S ½ 1,,n Sk 9 h 2 H such that h is 1-1 on
  S
• A non-deterministic protocol for k-disjointness
  using an (n,2k,2k) family H of functions
• Alice guess h 2 H and send description
• hoping that h is perfect for x y
• compute h(i) for all i 2 x and send 2k
  bit vector Cx where
• Cxj1 iff 9 i 2 x such that h(i)j
• Bob compute h(i) for all i 2 y and see whether
  Cx and Cy intersect
• Accept only if do not intersect
• If x ? y 1 always reject.
• If x ? y 0 and h is perfect for x y
  accept
• Complexity logH 2k

5
Existence of Perfect Hash FamiliesThe
Probabilistic Method

• For a fixed S ½ 1,,n of size 2k and choose
  random h1,,n ? 1,,2k
• Prh is perfect for S k!/(2k)2k ¼ e-2k
• Suppose we choose m random h1,,n ? 1,,2k
• Let event AS be no h in the collection is
  perfect for S
• PrAS (1- e-2k)m
• We are interested in showing PrS AS lt 1
• This implies that there is a choice of the that
  is a perfect family of hash function
• PrS AS ?S PrAS (n2k) PrAS

The probabilistic method!
Union Bound
6
The parameters

• Set m e2k log (n2k). Then
• PrAS (1- e-2k)m (1- e-2k) e2k log (n2k)
  1/(n2k)
• This means that communication complexity is
• 2k log m 2k 2k log (k log n)
• 2 O(klog log n)
• Classical constructions Fredman, Komlos
  Szemeredi
• More modern one Rasmus Pagh

7
Public coins model

• What if Alice and Bob have access to a joint
  source of bits.
• Possible view distribution over deterministic
  protocols
• Let R?pub(f) be the minimum cost of a public
  coins protocol computing f correctly with
  probability at least 1-? for any input (x,y)
• Example R?pub(Equality)(log ?)
• Theorem for any Boolean f
• R?d(f) is R?pub(f)O(log n log 1/d)
• Proof choose t 8n/d2 assignments to the public
  string

8
Simulating large sample spaces
Collection that should resemble probability of
success on ALL inputs

• Want to find among all possible public random
  strings a small collection of strings on which
  the protocol behave similarly on all inputs
• Choose m random strings
• For input (x,y) event Ax,y is more than (??) of
  the m strings fail the protocol
• PrAx,y e-2?2t lt 2-2n
• Prx,y Ax,y ?x,y PrAx,y lt 22n 2-2n1

Bad ?
Good 1-?
9
Distributional Complexity

• Let ? be a probability distribution on X x Y and
  ?gt0. The (?,?)-distributional complexity of f
  (D??(f)) is the cost of the best deterministic
  protocol that is correct on 1-? of the inputs
  weighted by ?
• Theorem for any f
• R?pub(f)max? D??(f)

Is the given protocol correct on the given input
Inputs
Von Neumanns Minimax Theorem For all matrices
M maxp minq pT M q minq maxp pT M q
Protocols of depth d
10
Space Complexity deterministic and
non-deterministic

• Non deterministic TM transition function ? ? X
  Q ? ? X Q X left,right may be
  non-deterministic.
• The TM is considered to accept if any
  combination of decisions leads to an accepts
  state
• can think of computation as tree and need one
  accepting leaf
• Space L 2 SPACE(f(n)) if there is a TM which
  recognizes L and
• uses no more than f(x) cells of its work tapes
  during the computation on input x
• Cell is used if it scanned by the tape head
• Non deterministic Space L 2 NSPACE(f(n)) if
  there is a non deterministic TM which recognizes
  L and
• Halts on all branches
• Uses no more than f(x) cells of its work tapes
  on any of the branches during the computation on
  input x
• Cell is used if it scanned by the tape head

11
NSPACE and Paths in Graphs

• NSPACE(f(n) models naturally search in
  implicit graphs on 2f(n) nodes

Example the Rush Hour Game given a
configuration is it possible to get the red car
out Can be solved in NSPACE(n)
nconfiguration size
12
Space and Time take I

• Theorem if a NTM N always halts and uses space
  f(n) log n, then it runs in time 2O(f(n))
• consider C, a configuration of N
• Heads positions
• Working tape content
• Output tape content
• Except for the input tape there are 2O(f(n)) x
  f(n)o(1) x n possible configurations
• For any input, in a computation cannot have two
  equal configurations occurring
• might lead to an infinite branch

13
Savitchs Theorem

• Theorem for f(n) proper complexity function
  f(n) log n
• NSPACE(f(n)) µ SPACE(f2(n))
• Cannot hope to consider all possible branchings
• There might be 2f(n) nondeterministic choices in
  a single branch and cannot store all of them
• Let L be decided by NTM N in NSPACE(f(n))
• C denotes a configuration of N
• Heads positions
• Working tape content
• Output tape content
• Define reachable(c1,c2,t) to be true iff N can
  transition from configuration c1 to c2 within t
  steps
• x 2 L iff reachable(cstart, caccept , 2f(n))

14
Solving reachable(cstart, caccept , 2f(n))

• Idea reachable(c1,c2, t) iff there is a
  midpoint configuration c3 such that
• reachable(c1,c3, t/2) and
• reachable(c3,c2, t/2)
• To compute both reachable(c1,c3, t/2) and
  reachable(c1,c3, t/2) can use the same space
  (except some accounting)

15
Program for reachable(cstart, caccept , t)

• reachable on input (c1, c2, t)
• If t1 test whether c1c2 or whether N can move
  from c1 to c2 in a single step.
• If tgt1 for each configuration c3 of N using
  space f(n)
• run reachable(c1,c3, t/2)
• run reachable(c3,c2, t/2)
• Accept if both steps accept
• If exhausted all configuration without accepting
  reject

16
Program for reachable(cstart, caccept , t)

• reachable on input (c1, c2, t)
• If t1 test whether c1c2 or whether N can move
  from c1 to c2 in a single step.
• If tgt1 for each configuration c3 of N using
  space f(n)
• run reachable(c1,c3, t/2)
• run reachable(c3,c2, t/2)
• Accept if both steps accept
• If exhausted all configuration without accepting
  reject

Runtime stack has depth f(n) with O(f(n)) space
per call
17
PSPACE NPSPACE

• Let PSPACE k11 SPACE(nk) and let NPSPACE
  k11 NSPACE(nk)
• Corollary of Savitchs Theorem
• PSPACE NPSPACE
• Open problem improve the theorem (even by small
  amount)

18
Reductions and Completeness

• Want to convert the problem of deciding A into
  deciding B
• Oracle for A can ask whether x 2 A
• Oracle Machine with oracle for A MA
• Cook-Turing reducible a language A is
  Cook-Turing reducible to B (A T B) if there
  exists a polynomial time M such that MB
  recognizes A
• Many-One (Karp Levin) reducible a language A is
  Many-One reducible to B (A M B) if there exists
  a function F computable in polynomial time such
  that x 2 A iff F(x) 2 B
• Log space reducible a language A is Many-One
  reducible to B (A L B) if there exists a
  function F 2 FSPACE(log n) such that x 2 A iff
  F(x) 2 B
• Property for all A and B A L B ) A M B ) A
  T B
• Property all reducibility notions are transitive

19
Log-space reductions
Read only Write only Log Space for read and
write This is what we count
Input Tape
Output Tape
Work Tape
Closed under composition! Do not write the
output explicitly. Instead consider G(x,i) ith
bit of G(X)
20
PSPACE Completeness

• Definition Language L is PSPACE-complete iff
• L is in PSPACE
• Every language L in PSPACE is reducible to L
  via a polynomial time reduction (Cook Turing)
• Complete problems are the hardest problems in a
  complexity class
• They represent it
• Reduction must be easy relative to the complexity
  class
• Property If L is PSPACE-complete and L is in P,
  then P PSPACE

21
The TQBF Problem Totally Quantified Boolean
Formula

• Boolean formula expression containing
• Boolean variables xi
• Constants 0 and 1
• Boolean operations Ç, Æ,
• In a Quantified Boolean formula in addition
  quantifiers
• Universal (for all) 8
• The statement 8 x ? means for every value of x
  the resulting statement ? is true
• Existential (there exists) 9
• The statement 9 x ? means for some value of x
  the resulting statement ? is true
• Can assume all quantifiers appear at the
  beginning of the formula (prenex normal form)
• If each variable appears within the scope of
  some quantifier fully quantified
• Statement always either true or false
• TQBF ? ? is a true fully quantified Boolean
  formula

22
Theorem TQBF is PSPACE Complete

• TQBF is in PSPACE
• to see if ? is in TQBF,
• If contains no quantifiers expression with only
  constants can evaluate it.
• If ?9 x ? recursively
• set x0 and check ?
• set x1 and check ?
• If either one answers true answer true.
  Otherwise false
• If ? 8 x ? recursively
• set x0 and check ?
• set x1 and check ?
• If both answer true answer true . Otherwise
  false

Reuse space!
Reuse space!
23
TQBF is PSPACE Complete

• Every language L in PSPACE is reducible to TQBF
  via a polynomial time reduction
• Suppose NTM N decides L is space nk
• Goal Given (N,w), construct a Boolean formula ?
  such that
• is in TQBF , N has an accepting path on w
• size of ? should be nk

24
Given (N,w), consider a 2nk by nk tableau
similar to that of Cooks theorem
Ns configuration
c1,1 c1,2 c1,3c1,4 c1,nk c2,1 c2,2
c2,3 c2,4 c2,nk . . . cnk,,1
cnk,2 cnk,3 cnk,4 cnk,nk
1 2 2nk
time
25
TQBF is PSPACE Complete
If N accepts w, then there is a legal tableau
that looks like this
Ns configuration
Current state and implicit location of the tape
head
q0 w1 w2 wn U U c2,2 c2,3 c2,4
c2,nk1 c3,2 c3,3 c3,4
c3,nk1 any accept
configuration
1 2 2nk
time
26

• Want ? c1,c2,t which is in TQBF iff N can
  transition between configurations c1 and c2
  within at most t steps
• Construct ?cstart,caccept,2nk recursively
• First try guess c3 as mid point
• ?c1,c2,t 9c3 ?c1,c3,t/2 Æ ?c3,c2,t/2
• Exponential size formula
• But only used existential quantifiers!
• ?c1,c2,t 9 c3 8 (c4,c5)2 (c1,c3),
  (c3,c2)?c4,c5,t/2

27
PSPACE Completeness

• Many problems proved to be PSPACE Complete
• In particular games
• Games closely related to quantifiers
• The formula game
• Let ? 9 x1 8x2 9x3 8 xn? be a prenex
  formula
• Player A and Player E take turns in selecting
  values for the variables
• A for the 8 bound variables and
• E for the 9 bound variables
• At the end Player E wins if ? is TRUE and A wins
  if ? is FALSE
• Formula-game? Player E has a winning strategy
  in the formula game played on ?
• Theorem Formula-game is PSPACE Complete

28
Game of Go

• To players alternate placing black and white
  pebbles
• Objective create large safe groups of your color
  
• Monochromatic connected subgraph of the grid
• and capture as many pebbles of the other color
  by surrounding them
• Theorem deciding whether a given configuration
  is a win for white is PSPACE-Complete

29
Pebbling a DAG

• Rules
• Inputs a pebble can be place on input (source)
  vertex at any time
• Computation a pebble can be placed on any
  non-input vertex if all its immediate predecessor
  have pebbles
• A pebble can be removed at any time
• Goal find a strategy to pebble all the outputs
  while using a few pebbles and a few steps
• Theorem Minimizing pebbles is PSPACE-complete
• Gilbert, Lengaur, Tarjan, Sicomp 1980

Sources
Outputs
Models computation with registers
30
Homework on Pebbling

• Depth of a DAG distance of the input node and
  output node that are furthest apart (if they are
  connected).
• Let d be the maximum indegree
• Show that any DAG of depth c and can be pebbled
  using O(c d ) pebbles.
• Show that any tree can be pebbled using O(d log
  n) pebbles.

31
Generalized Rush Hour

• Given a rush hour configuration can you get the
  special car out?
• Recently proved theorem Rush Hour is
  PSPACE-Complete (Baum and Flake)
• Implications
• Hard to solve
• Even if you invested/wasted time and solved it,
  how do you convince your friend that it is
  possible?
• How do you demonstrate that there is no solution?
• Homework suggest a probabilistic protocol for
  verifying that a solution exists.
• The prover may be willing to work hard, but not
  the verifier.
• the probability of catching a cheating prover is
  non-zero (but might be very small) for any
  cheater
• If the prover is not cheating then the verifier
  is always convinced by the proof.

32
L and NL

• Log-Space SPACE(log(n))
• NL NSPACE(log(n))

Log(n) space for work Input is read-only
33
Log-space reductions
Read only Write only Log Space for read and
write This is what we count
Input Tape
Output Tape
Work Tape
Closed under composition! Do not write the
output explicitly. Instead consider G(x,i) ith
bit of G(X)
34
NL-completeness

• Language L is NL-complete iff
• L is in NL
• Every language L in NL is log-space reducible to
  L
• PATH (G,s,t) G is a directed graph with a
  path that starts at vertex s and ends at vertex
  t
• How is the graph given (adjacency matrix, list of
  outgoing edges?
• Theorem PATH is NL-complete
• What about undirected connectivity?

35
Sublogarithmic Space

• Are there interesting languages requiring less
  than logarithmic space?
• Regular languages
• Other?
• A ?(log log n) language
• xb0 b2 bk bibinary representation of
  i
• Algorithm
• Check the first and last blocks
• Check that each block is 1 from previous
• To avoid using too much space on bogus strings
• check for i1.. that the last i bits cycle
  through all possibilities

36
No languages between regular and O(log log n)

• Theorem For any f(n) 2 o(log log n)
• Space(f(n)) Space(O(1))
• Proof Configuration is lti,cgt
• i position on tape
• c content of working tapes heads and direction
• No configuration can repeat itself
• Ow. Machine does not stop
• For word x let C(x,i) c1, c2, cr sequence of
  configurations machine passes through on position
  i
• Number of different C(x,i) 2f(n)2f(n) lt n for
  sufficiently large n
• Let xS be shortest word requiring more than S
  space
• Idea if there is 1 iltj n such that C(xS,i)
  C(xS,j) then can cut it out resulting in shorter
  string with similar behavior

C1
C2
C3
37
Is sublogarithmic space interesting

• No equivalent RAM
• Not clear that it models easiness/simplicity
• Unlike regular languages
• Other models for simple languages
• Parallel complexity

38
Non-deterministic space is closed under
complementation The Immerman-Szelepcsenyi Theorem

• For simplicity start with NLCo-NL
• States of NTM correspond to directed graph
• Problem how to prove non-reachability
• Idea to prove that a room is empty show the
  whereabouts of all potential occupiers
• Show how to count in NL
• Sufficient for non-reachability can make sure
  graph is acyclic
• For directed graph G(V,E) let
• R(G,s,i) u reachable(s,u,i)
• Main Lemma there exists a log space non
  deterministic machine for computing R(G,s,i)
• The machine never outputs a wrong answer but may
  output fail

u is reachable from s in i moves in G
39
NL Algorithm for R(G,s,i) given R(G,s,i-1)

• Simple go over all nodes u1,2 n
• if reachable(s,u,i) increase counter
• How to determine reachable(s,u,i) with no errors
  either way?
• NL Algorithm for determining reachable(s,u,i)
• Input R(G,s,i-1)
• Output yes, no, fail
• go over nodes v1,2 n
• if reachable(s,v,i-1)
• m m1
• If (v,u) 2 G output yes
• If m lt R(G,s,i-1) output fail
• Else output no

Not a recursive call! Guess the path and verify
40
Wrapping up

• For i1,2 n-1 compute R(G,s,i)
• If ever the output is fail then fail
• Each call requires simply R(G,s,i-1)
• same log n bits
• Knowing R(G,s,n) allows us to determine
  whether reachable(s,accept,n) with no errors
• Theorem If f(n) log n is a proper complexity
  function, then
• NSPACE(f(n)) Co-NSPACE(f(n))