Complexity Theory Lecture 12

1
Complexity TheoryLecture 12

• Lecturer Moni Naor

2
Recap

• Last week
• Hardness and Randomness
• Semi-random sources
• Extractors
• This Week
• Finish Hardness and Randomness
• Circuit Complexity
• The class NC
• Formulas NC1
• Lower bound for Andreevs function
• Communication characterization of depth

3
Derandomization

• A major research question
• How to make the construction of
• Small Sample space resembling large one
• Hitting sets
• Efficient.
• Successful approach randomness from hardness
• (Cryptographic) pseudo-random generators
• Complexity oriented pseudo-random generators

4
Extending the result

• Theorem if E contains 2?(n)-unapproximable
  functions then BPP P.
• The assumption is an average case one
• Based on non-uniformity
• Improvement
• Theorem If E contains functions that require
  size 2?(n) circuits (for the worst case), then E
  contains 2?(n)-unapproximable functions.
• Corollary If E requires exponential size
  circuits, then BPP P.

5
How to extend the result

• Recall the worst-case to average case reduction
• For permanent
• The idea encode the function in a form allowing
  you to translate a few worst case error into
  random errors

6
Properties of a code

• Want a code C0,12n ? 0,12l
• Where
• 2l is polynomial in 2n
• C is polynomial time computable
• efficient encoding
• Certain local decoding properties

7
Codes and Hardness

• Use for worst-case to average case
• truth table of f0,1n ? 0,1
• worst-case hard
• truth table of f0,1l ? 0,1
• average-case hard

0
1
0
0
1
0
1
0
mf
0
1
0
0
1
0
1
0
0
0
0
1
0
C(mf)
8
Codes and Hardness

• if 2l is polynomial in 2n, then
• f ? E implies f ? E
• Want to be able to prove
• if f is s-approximable, then f is computable
  by a
• size s poly(s) circuit

9
Codes and Hardness

• Key point circuit C that approximates f
  implicitly defines a received word RC that is not
  far from C(mf)
• Want the decoding procedure D to computes f
  exactly

0
1
1
0
0
0
1
0
RC
0
1
0
0
0
0
1
0
0
1
0
1
0
0
0
0
1
0
C(mf)
Requires a special notion of efficient decoding
C
D
10
Decoding requirements

• Want that
• for any received word R that is not far from
  C(m),
• for any input bit 1 i 2n
• can reconstruct m(i) with probability 2/3 by
  accessing only poly(n) locations in R
• Example of code with good local decoding
  properties Hadamard
• But exponential length
• This gives a probabilistic circuit for f of size
• poly(n) size(C) size of decoding circuit
• Since probabilistic circuit have deterministic
  version of similar size - contradiction

11
Extractor

• Extractor a universal procedure for purifying
  imperfect source
• The function Ext(x,y) should be efficiently
  computable
• truly random seed as catalyst
• Parameters (n, k, m, t, ?)

source string
2k strings
near-uniform
Ext
seed
m bits
0,1n
t bits
Truly random
12
Extractor Definition

• (k, e)-extractor for all random variables X with
  min-entropy k
• output fools all tests T
• PrzT(z) 1 Pry 2R 0,1t, x?XT(Ext(x, y))
  1 e
• distributions Ext(X, Ut) and Um are e-close (L1
  dist 2e)
• Um uniform distribution on 0,1m
• Comparison to Pseudo-Random Generators
• output of PRG fools all efficient tests
• output of extractor fools all tests

13
Extractors Applications

• Using extractors
• use output in place of randomness in any
  application
• alters probability of any outcome by at most e
• Main motivation
• use output in place of randomness in algorithm
• how to get truly random seed?
• enumerate all seeds, take majority

14
Extractor as a Graph
Want every subset of size 2k to see almost all of
the rhs with equal probability
2t
Size 2k
0,1m
0,1n
15
Extractors desired parameters
source string
2k strings
near-uniform
Ext
seed
m bits
0,1n
t bits
Allows going over all seeds

• Goals good optimal
• short seed O(log n) log nO(1)
• long output m k?(1) m ktO(1)
• many ks k n?(1) any k
  k(n)

16
Extractors

• A random construction for Ext achieves optimal!
• but we need explicit constructions
• Otherwise we cannot derandomize BPP
• optimal construction of extractors still open
• Trevisan Extractor
• idea any string defines a function
• String C over ? of length l define a function
  fC1 l ? ? by fC(i)Ci
• Use NW generator with source string in place of
  hard function
• From complexity to combinatorics!

17
Trevisan Extractor

• Tools
• An error-correcting code
• C0,1n ? 0,1l
• Distance between codewords (½ - ¼m-4)l
• Important in any ball of radius ½-? there are at
  most 1/?2 codewords.
• ? ½ m-2
• Blocklength l poly(n)
• Polynomial time encoding
• Decoding time does not matter
• An (a,h)-design S1,S2,,Sm ? 1t where
• hlog l
• a dlog n/3
• tO(log l)
• Construction
• Ext(x, y)C(x)yS1?C(x)yS2??C(x)ySm

18
Trevisan Extractor

• Ext(x, y)C(x)yS1?C(x)yS2??C(x)ySm
• Theorem Ext is an extractor for min-entropy k
  nd, with
• output length m k1/3
• seed length t O(log l ) O(log n)
• error e 1/m

010100101111101010111001010
C(x)
seed y
19
Proof of Trevisan Extractor

• Assume X µ 0,1n is a min-entropy k random
  variable failing to e-pass a statistical test T
• PrzT(z) 1 - Prx?X, y ? 0,1tT(Ext(x, y))
  1 gt e
• By applying usual hybrid argument
• there is a predictor A and 1 i m
• Prx?X, y?0,1tA(Ext(x, y)1i-1) Ext(x, y)i gt
  ½e/m

20
The set for which A predict well

• Consider the set B of xs such that
• Pry?0,1tA(Ext(x, y)1i-1) Ext(x, y)i gt
  ½e/2m
• By averaging Prxx 2 B e/2m
• Since X has min-entropy k
• there are at least
• e/2m 2k
• different x 2 B
• The contradiction will be by showing a succinct
  encoding for each x 2 B

21
Proof of Trevisan Extractori, A and B are fixed

• If you fix the bits outside of Si to ? and ? and
  let y vary over all possible assignments to bits
  in Si. Then
• Ext(x, y)i Ext(x, ?y?)i C(x)?y?Si
  C(x)y
• goes over all the bits of C(x)
• For every x 2 B short description of a string z
  close to C(x)
• fix bits outside of Si to ? and ? preserving the
  advantage
• PryP(Ext(x, ?y?)1i-1)C(x)y gt ½ e/(2m)
• ? and ? is the assignment to 1t\Si maximizing
  the advantage of A
• for j ? i, as y varies, ?y?Sj varies over
  only 2a values!
• Can provide (i-1) tables of 2a values to supply
• Ext(x, ?y?)1i-1

22
Trevisan Extractorshort description of a string
z agreeing with C(x)
Output is C(x)y w.p. ½ e/(2m) over Y
Y ? 0,1log l
A
y
23
Proof of Trevisan Extractor

• Up to (m-1) tables of size 2a describe a string
  z that has a ½ e/(2m) agreement with C(x)
• Number of codewords of C agreeing with z
• on ½ e/(2m) places is O(1/d2) O(m4)
• Given z there are at most O(m4) corresponding
  xs
• Number of strings z with such a description
• 2(m-1)2a 2nd2/3 2k2/3
• total number of x 2 B
• O(m4) 2k2/3 ltlt 2k(e/2m)

• Johnson Bound
• A binary code with distance (½ - d2)n has at most
  O(1/d2) codewords in any ball of radius (½ - d)n.
• C has minimum distance (½ - ¼m-4)l

24
Conclusion

• Given a source of n random bits with min entropy
  k which is n?(1)
• it is possible to run any BPP algorithm with and
  obtain the correct answer with high probability

25
Application strong error reduction

• L ? BPP if there is a p.p.t. TM M
• x ? L ? PryM(x,y) accepts 2/3
• x ? L ? PryM(x,y) rejects 2/3
• Want
• x ? L ? PryM(x,y) accepts 1 - 2-k
• x ? L ? PryM(x,y) rejects 1 - 2-k
• Already know if we repeat O(k) times and take
  majority
• Use n O(k)y random bits
• Of them 2n-k can be bad strings

26
Strong error reduction

• Better Ext extractor for k y3 nd, e lt 1/6
• pick random w ?R 0,1n
• run M(x, Ext(w, z)) for all z ? 0,1t
• take majority of answers
• call w bad if majzM(x, Ext(w, z)) is incorrect
• PrzM(x,Ext(w,z))b - PryM(x,y)b 1/6
• extractor property at most 2k bad w
• n random bits 2nd bad strings

27
Strong error reduction
Strings where the majority of neighbors are bad
Property every subset of size 2k sees almost all
of the rhs with equal probability
2t
Bad strings for input at most 1/4
Upper bound on Size 2k
All strings for running the original randomized
algorithm
0,1m
0,1n
28
Two Surveys on Extractors

• Nisan and Ta-Shma, Extracting Randomness A
  Survey and New Constructions 1999, (predates
  Trevisan)
• Shaltiel,    Recent developments in Extractors,
  2002,
•  www.wisdom.weizmann.ac.il/ronens/papers/survey.p
  s
• Some of the slides based on C. Umans course
• www.cs.caltech.edu/umans/cs151-sp04/index.html

29
Circuit Complexity

• We will consider several issues regarding circuit
  complexity

30
Parallelism

• Refinement of polynomial time via (uniform)
  circuits allow

depth ? parallel time
circuit C
size ? parallel work
depth of a circuit is the length of longest path
from input to output Represents circuit latency
31
Parallelism

• the NC Hierarchy (of logspace uniform circuits)
• NCk O(logk n) depth, poly(n) size circuits
• Bounded fan-in (2)
• NC k NCk
• Aim to capture efficiently parallelizable
  problems
• Not realistic?
• overly generous in size
• Does not capture all aspects of parallelism
• But does capture latency
• Sufficient for proving (presumed) lower bounds on
  best latency

What is NC0
32
Matrix Multiplication
n x n matrix A
n x n matrix B
n x n matrix AB

• Parallel complexity of this problem?
• work poly(n)
• time logk(n)?
• which k?

33
Matrix Multiplication

• arithmetic matrix multiplication
• A (ai, k) B (bk, j) (AB)i,j Sk (ai,k x bk,
  j)
• vs. Boolean matrix multiplication
• A (ai, k) B (bk, j) (AB)i,j ?k (ai,k ? bk,
  j)
• single output bit to make matrix multiplication
  a language on input A, B, (i, j) output (AB)i,j

34
Matrix Multiplication

• Boolean Matrix Multiplication is in NC1
• level 1 compute n ANDS ai,k ? bk, j
• next log n levels tree of ORS
• n2 subtrees for all pairs (i, j)
• select correct one and output

35
Boolean formulas and NC1

• Circuit for Boolean Matrix Multiplication is
  actually a formula.
• Formula fan-out 1. Circuit looks like a tree
• This is no accident
• Theorem L ? NC1 iff decidable by polynomial-size
  uniform family of Boolean formulas.

36
Boolean formulas and NC1from small depth
circuits to formulas

• Proof
• convert NC1 circuit into formula
• recursively
• note logspace transformation
• stack depth log n, stack record 1 bit left or
  right

?
?
?
37
Boolean formulas and NC1 from forumulas to small
depth circuits

• convert formula of size n into formula of depth
  O(log n)
• note size 2depth, so new formula has poly(n)
  size

?
?
?
key transformation
?
D
C
C1
C0
D
1
0
D
38
Boolean formulas and NC1

• D any minimal subtree with size at least n/3
• implies size(D) 2n/3
• define T(n) maximum depth required for any size
  n formula
• C1, C0, D all size 2n/3
• T(n) T(2n/3) 3
• implies T(n) O(log n)

39
Relation to other classes

• Clearly NC µ P
• P ? uniform poly-size circuits
• NC1 µ Logspace
• on input x, compose logspace algorithms for
• generating Cx
• converting to formula
• FVAL Cx(x)
• FVAL is given formula and assignment what is the
  value of the output
• logspace composes!

40
Relation to other classes

• NL µ NC2
• Claim Directed S-T-CONN ? NC2
• Given directed G (V, E) vertices s, t
• A adjacency matrix (with self-loops)
• (A2)i, j 1 iff path of length at most 2 from
  node i to node j
• (An)i, j 1 iff path of length at most n from
  node i to node j
• Compute with depth log n a tree of Boolean matrix
  multiplications, output entry s, t
• Repeated squaring!
• log2 n depth total

Boolean MM
41
NC vs. P

• Can every efficient algorithm be efficiently
  parallelized?
• NC P
• Common belief NC ( P

?
42
P-Completeness

• A language L is P-Complete if
• L 2 P
• Any other language in P is reducible to L via a
  Logspace reduction
• P-complete problems are the least-likely to be
  parallelizable
• if a P-complete problem is in NC, then P NC
• we use logspace reductions to show problem
  P-complete and we have seen Logspace in NC

43
Some P-Complete Problems

• CVAL Circuit value problem
• Given a circuit and an assignment, what is the
  value of the output of circuit
• Canonical P-Complete problem
• Lexicographically first maximal independent set
• Linear Programming
• Finding a happy coloring of a graph

44
NC vs. P

• Can every uniform, poly-size Boolean circuit
  family be converted into a uniform, poly-size
  Boolean formula family?
• NC1 P
• Is the NC hierarchy proper
• is it true that for all NCi ( NCi1
• Define ACk O(logk n) depth, poly(n) size
  circuits with
• unbounded fan-in ? and ? gates
• Is the following true
• ACi ( NCi1 ( ACi1

?
45
Lower bounds

• Recall
• NP does not have polynomial-size circuits (NP ?
  P/poly) implies P ? NP
• Major goal prove lower bounds on (non-uniform)
  circuit size for problems in NP
• Belief exponential lower bound
• super-polynomial lower bound enough for P ? NP
• Best bound known 4.5n
• dont even have super-polynomial bounds for
  problems in NEXP!

46
Lower bounds

• lots of work on lower bounds for restricted
  classes of circuits
• Formulas
• Outdegree of each gate is 1
• Monotone circuits
• No nots (even at the input level)
• Constant Depth circuits
• Polynomial size but unbounded fan-in

47
Counting argument for formulas

• frustrating fact almost all functions require
  huge formulas
• Theorem Shannon With probability at least 1
  o(1), a random function
• f0,1n ? 0,1
• requires a formula of size O(2n/log n).

48
Shannons counting argument

• Proof (counting)
• B(n) 22n functions f0,1n ? 0,1
• formulas with n inputs size s, is at most
• F(n, s) 4s2s(2n)s

n2 choices per leaf
4s binary trees with s internal nodes
2 gate choices per internal node
49
Shannons counting argument

• F(n, c2n/log n) lt (16n)(c2n/log n)
• lt 16(c2n/log n)2(c2n) (1 o(1))2(c2n)
• lt o(1)22n (if c ½)
• Probability a random function has a formula of
  size s (½)2n/log n is at most
• F(n, s)/B(n) lt o(1)

50
Andreevs function

• best lower bound for formulas
• Theorem (Andreev, Hastad 93) the Andreev
  function requires (?,?,?)-formulas of size at
• O(n3-o(1)).

51
Andreevs function
yi
selector
n-bit string y
. . .
XOR
XOR
log n copies n/log n bits each
n bits total x
The Andreev function A(x,y)
A0,12n ? 0,1
52
Andreevs function

• Theorem the Andreev function requires
  (?,?,?)-formulas of size at
• O(n3-o(1)).
• First show O(n2-o(1)).
• Two important ideas
• Random restrictions
• Using the existential counting lower bounds on a
  smaller domain

53
General Strategy

• Restrict the function and show
• This must simplify the formula (a lot)
• But
• The remaining function is still quite complex, so
  needs a relatively large formula
• Conclude we must have started with a large
  formula
• Definition L(f) smallest (?,?,?) formula
  computing f
• measured as leaf-size
• Directly related to formula size

54
Random restrictions

• key idea given function
• f0,1n ? 0,1
• restrict by ? to get f?
• ? sets some variables to 0/1, others remain free
• R(n, m) set of restrictions that leave m
  variables free

55
Random restrictions

• Claim Let m ?n. Then
• E??R(n, ?n)L(f?) ? L(f)
• each leaf survives with probability ?
• may shrink even more
• propagate constants
• What happens to the Xor of a subset under random
  restriction
• if at least one member of the Xor survives, the
  Xor is not fixed and can get obtain both values

56
shrinkage result

• From the counting argument there exists a
  function h
• h0,1log n ?0,1
• for which L(h) gt n/2loglog n.
• hardwire truth table of that function into y to
  get function A(x)
• apply random restriction from set
• R(n, m 2(log n)(ln log n))
• to A(x).

selector
n-bit string y
. . .
XOR
XOR
57
The lower bound

• probability a particular XOR is killed by
  restriction
• probability that we dont miss it m times
• (1 (n/log n)/n)m (1 1/log n)m
• (1/e)2ln log n 1/log2n
• probability even one of the log n XORs is killed
  by the restriction is at most
• log n(1/log2n) 1/log n lt ½.

58
The lower bound

• probability even one of XORs is killed by
  restriction is at most
• log n(1/log2n) 1/log n lt ½.
• by Markovs inequality
• Pr L(A?) gt 2 E??R(n, m)L(A?) lt ½.
• Conclusion for some restriction ? both events
  happen
• all XORs survive and
• L(A?) 2 E??R(n, m)L(A?)

59
The lower bound

• if all XORs survive, can restrict formula further
  to compute the hard function h
• may need additional ?s (free)
• L(h) n/2loglogn L(A?)
• 2E??R(n, m)L(A?) O((m/n)L(A))
• O( ((log n)(ln log n)/n)1 L(A) )
• Conclude O(n2-o(1)) L(A) L(A).

Shrinkage factor
60
Random restrictions and shrinkage factors

• Recall
• E??R(n, ?n)L(f?) ? L(f)
• each leaf survives with probability ?
• But may shrink even more by propagating constants
• Lemma Hastad 93 for all f
• E??R(n, ?n)L(f?) O(?2-o(1)L(f))

61
The lower bound with new shrinkage factor

• if all XORs survive, can restrict formula further
  to compute hard function h
• may need to add ?s
• L(h) n/2loglogn L(A?)
• 2E??R(n, m)L(A?) O((m/n)2-o(1)L(A))
• O( ((log n)(ln log n)/n)2-o(1) L(A) )
• Conclude O(n3-o(1)) L(A) L(A).

Shrinkage factor
62
What can be done in NC1

• Addition of two number each n bits
• In fact can be done in AC0
• Adding n bits
• Can compute majority or threshold
• Something that cannot be done in AC0
• Multiplication
• Adding n numbers
• Division

63
Two different characterization of NC1

• Through communication complexity
• Through branching programs

64
More on Depth a communication complexity
characterization of depth

• For Boolean function f0,1n ? 0,1 let
• Xf-1(1)
• Yf-1(0)
• Consider relation Rf µ X Y 1,,n where
• (x,y,i) are such that xi ? yi
• For Monotone Boolean functions define
• Mf µ X Y 1,,n
• where
• (x,y,i) are such that xi 1 and yi 0

65
A communication complexity characterization of
depth

• What is the communication complexity of Rf
• D(Rf)
• assuming
• Alice has x 2 Xf-1(1)
• Bob has y 2 Yf-1(0)
• Lemma Let C be a circuit for f. Then
• D(Rf) depth(C)
• Lemma Let C be a monotone circuit for f. Then
• D(Mf) depth(C)

66
From circuits to protocolsboth monotone and
non-monotone case

• For each ? gate Alice says which of the two
  inputs wires to the gate is 1 under x
• If both are 1 picks one
• This wire must be 0 under y
• For each ? gate Bob says which of the two inputs
  wires to the gate is 0 under y
• If both are 0 picks one
• This wire must be 1 under x
• At the leaves, find an i such that xi ? yi
• if the circuit is monotone, then we know that xi
  1 and yi 0
• Property maintained for the subformula
  considered Alice assignment yields 1 and Bobs
  assignment yields 0

67
From protocols to circuits both monotone and
non-monotone case

• Lemma Let P be a protocol for Rf. Then there is
  a formula of depth C(P)

• Label
• Alices move with ?
• Bobs move with ?
• Leaf with rectangle A B and output i with
  either zi or ? zi

z0 z1 z2 z3 z4 z5 z6 z7 ...
z5
68
A communication complexity characterization of
depth

• Theorem D(Rf)depth(f)
• Theorem for any monotone function
• D(Mf)depthmonotone(f)
• Applications
• depthmonotone(STCON)?(log2 n)
• depthmonotone(matching) ?(n)

69
Example Majority

• Input to Alice x1, x2, , xn such that the
  majority are 1
• Input to Bob y1, y2, , yn such that the majority
  are 0
• Partition the input into x1, x2, , xn/2 input
  into and xn/21, xn/22, , xn and report the
  result on each half