Paul Beame

1
Satisfiability and Unsatisfiability Proof
Complexity and Algorithms
Paul Beame University of Washington
2
Outline

• Intro to proof complexity
• Proof complexity and complete SAT algs
• Review Resolution/DPLL connection
• Application to random k-SAT
• Proof complexity and DPLL extensions
• Clause learning
• Caching
• Proof system survey

3
Proof Complexity

• Study of the number of symbols required for
  proofs of unsatisfiability (or tautology) in
  propositional logic
• Does not address algorithmic issue
• How would you find short proofs if they existed?
• Existence of short proofs for every unsatisfiable
  formula is equivalent to NP co-NP (and
  is implied by PNP)
• Generally believed that such proofs dont exist

4
Propositional proof systems

• A propositional proof system is a polynomial time
  algorithm V s.t. for all formulas F
• F is unsatisfiable ? there exists a string P
  s.t. V accepts input (P,F)
• Note
• ? direction is usually called soundness
• ? direction is usually called completeness
• Proof complexity is size of P as a function of
  the size of F

5
Sample propositional proof systems

• Truth tables
• proof is a fully filled out truth table
• easy to verify that it is filled out correctly
  and all truth assignments yield T
• Frege systems
• inference rules e.g. modus ponens
• A, (A ? B) B
• axioms e.g. excluded middle (A ? ?A)
• axioms inference rules are schemas
• can make consistent substitution of arbitrary
  formulas for variables in schema
• e.g. excluded middle yields ((x??y) ? ?(x??y))

6
The graph of a proof
Axioms/inputs are sources
Inference rule associated with each node
Sink labelled by tautology (or L for refutation)
7
Resolution

• Start with clauses of CNF formula F
• Resolution rule
• Given (A ? x), (B ? x) derive
  (A ? B)
• The empty clause is derivable
• ? F is unsatisfiable
• Proof size of clauses used

8
Restricted forms of Resolution

• Resolution
• In general, graph of inferences is a directed
  acyclic graph
• Tree Resolution
• Graph of inferences forms a binary tree
• i.e., if you want to use a clause more than once
  you need to re-derive it
• Regular Resolution
• Once a variable is resolved out of a clause you
  cant add it back later
• More general than Tree Resolution

9
Satisfiability Algorithms

• Incomplete Algorithms
• Local search
• GSAT Selman,Levesque,Mitchell 92
• Walksat Kautz,Selman 96
• Belief Propagation
• SP Braunstein, Mezard, Zecchina 02
• Complete Algorithms
• Backtracking search
• DPLL Davis,Putnam 60
  Davis,Logeman,Loveland 62
• DPLL clause learning GRASP, SATO, zchaff,
  berkmin
• etc

10
Proof Complexity and SAT Algorithms

• Proof Complexity
• Unsatisfiable formulas
• Nondeterministic algorithms
• SAT Algorithms
• Satisfiable formulas
• Deterministic or randomized algorithms

11
Proof complexity and complete satisfiability
algorithms

• In general
• The transcript of the run of a complete
  satisfiability algorithm on an unsatisfiable
  formula is a proof of its unsatisfiability
• For backtracking style algorithms
• Even runs on satisfiable formulas yield proofs of
  unsatisfiability of related formulas

12
Backtracking search/DPLL

• DPLL(F)
• while F contains a 1-clause x
  F ? Fx1
• if F is empty return 1
• if F has an empty clause return 0
• else select a literal x to branch on return
  DPLL(Fx1) ? DPLL(Fx0)

Residual formula
13
Resolution and DPLL

• DPLL tree with any variable selection rule on an
  unsatisfiable formula F generates a Resolution
  refutation of F
• of clauses ? running time
• Note Regular resolution corresponds to a similar
  DAG (read-once branching program) rather than a
  tree but general resolution has no such analogue

14
DPLL Refutation
Clauses 1. a? b? c 2. a??b 3. ?c 4. ?a? d 5. ?d? b
15
DPLL Refutation Tree-like Resolution Proof
Clauses 1. a? b? c 2. a??c 3. ?b 4. ?a? d 5. ?a? b
16
Random k-CNF formulas

• Randomly choose m clauses over n variables
  independently each of size k
• Each size k clause is equally likely
• Key parameter rm/n, the ratio of clauses to
  variables
• Threshold value rk of r
• Below, almost certainly satisfiable
• Above, almost certainly unsatisfiable
• Hardest problems near threshold

17
DPLL on random 3-CNF
Proof complexity shows 2Q(n/r) time is required
for unsatisfiable formulas for r ?
r3 B,Karp,Saks,Pitassi 98 Ben-Sasson 02
of DPLL backtracks
What about satisfiable formulas below threshold?
ratio of clauses to variables
Mitchell,Selman,Levesque 92
r
n 50 variables
18
Some simple select choices for DPLL algorithms

• UC Unit Clause/Ordered DLL
• Choose variables in a fixed order
• Always set True first
• UCwm Unit Clause with majority
• Choose variables in a fixed order
• Apply a majority vote among 3-clauses for
  assigning each value
• GUC Generalized Unit Clause
• Choose a variable v in a shortest clause C
• Set v to satisfy C

19
Parameters of residual formulas follow
trajectories
1
Chao,Franco 88
Frieze,Suen 95
Achlioptas 00
2-clause ratio
Achlioptas,Sorkin 00
UC
Kaporis et al 03
GUC
2/3
3.52
8/3
0
3-clause ratio
20
Satisfiability for mixed random formulas proven
properties
Achlioptas et al 96
1
Kaporis et al 03
?
?
Dubois 01
?
?
?
?
UNSAT
2-clause ratio
?
?
?
?
SAT
?
?
3.52
4.501
2/3
2.28
3-clause ratio
21
Resolution proof complexity of mixed random
formulas

• Theorem Achlioptas, B, Molloy 01 For any
  constants r2?1 and r3 ?0 random formulas with r2n
  2-clauses and r3n 3-clauses need
  exponential-size resolution proofs almost surely
• Extends Chvatal-Szemeredi 88

22
Long-running DPLL Executions and Proof Complexity
Residual formula at each node is a mix of 2- and
3-clauses
Residual formula at is unsatisfiable
2rn
Every resolution
Algorithms proof of unsatisfiability is
exponentially long
23
Trajectory on 3-CNF
UC Algorithm Trajectory
1
Provably UNSAT Hard
2-clause ratio
Provably SAT Easy
0
4.51
3.81
3.52
4.267
3-clause ratio
24
Exponential lower bounds for 3-CNF formulas below
ratio 4.267

• Corollary For almost all 3-CNF formulas, above
  ratio
• 3.81 UC takes exponential time
• 3.83 UCwm takes exponential time
• 4.01 GUC takes exponential time

25
Exponential lower bound below the proven k-CNF
threshold

• Achlioptas, Peres 02
  rk ? 2k ln 2 (k4)/2
• Achlioptas, B, Molloy 04 For k ? 4, for almost
  all k-CNF formulas UC takes exponential time
  above ratio c 2k/k
• Note These formulas have huge numbers of
  satisfying assignments (more than 2 (1-?) n out
  of a possible 2n) but still are hard

26
Open Problem

• Closing gap for unsatisfiability of mixed
  formulas would yield sharp threshold behavior for
  each algorithm A
• Below rA algorithm runs
  in linear time
• Above rA algorithm
  requires exponential time
• Conjecture no polynomial time selection
  heuristic will work up to the threshold
• rA ? r3 for all algorithms A
• Backtracking algorithms for other random problems
  with phase transitions?
• e.g. k-colorability on random graphs G(n,r/n)
• Unsatisfiable phase exp(cn/rak)
  
  B, Culberson, Mitchell, Moore 03

27
Outline

• Intro to proof complexity
• Proof complexity and complete SAT algs
• Review Resolution/DPLL connection
• Application to random k-SAT
• Proof Complexity and DPLL extensions
• Clause learning
• Caching
• Proof system survey

28
Clause Learning

• At each backtrack point in DPLL search add new
  clauses to the input formula that correspond to
  known causes of failure of the search
• Critical to the good behavior of the best current
  algorithms
• What is the power of clause learning?
• Intuitively it makes DPLL trees DAG-like
• ? regular resolution? ? general resolution?

29
Conflict Graph
Known Clauses (p ? q ? a) (? a ? ? b ? ? t) (t ?
?x1) (t ? ?x2) (t ? ?x3) (x1 ? x2 ? x3 ? y) (x2 ?
?y)
Current decisions p ? false q ? false b ? true
30
Proof complexity and clause learning

• Clause learning is no stronger than Resolution
• Implication graph yields resolution derivation of
  each learned clause
• There are examples where clause learning works in
  polynomial time but any Regular Resolution proof
  is exponential Sabharwal, B, Kautz 04
• Clause learning can sometimes be much stronger
  than Regular Resolution

31
Proof complexity and clause learning

• Separation is true for any proper subsystem R of
  Resolution for which refuting Fx is no harder
  than F
• Clause learning does not have this property
  unless it is as powerful as resolution
• Idea Take formula F easy for Resolution but hard
  for R and create new formula F that is F plus
  clauses for (C?pC) for each C in Resolution proof
  of F where each pC is a new propositional
  variable
• Setting each pC to true yields original F but
  branching on sequence of ?pC allows sequence of
  clauses C to be learned

32
Still Open

• Can clause learning do everything regular
  resolution can?
• Is clause learning as powerful as general
  resolution?
• Note Alan Van Gelder has pointed out that if
  one returns learned clauses up the tree and
  resolves them while backtracking then clause
  learning does efficiently simulate regular
  resolution

33
Proof Complexity and Formula Caching

• Formula Caching Memoization
• Cache residual formulas that have been already
  found to be unsatisfiable
• Do not make recursive call if the current
  residual formula is already cached

34
DPLL

• DPLL(F)
• if F is empty report satisfiable and halt
• if F contains the empty clause L
• backtrack
• else select a literal x
• DPLL(Fx)
• DPLL(F?x)

35
Basic Formula Caching

• FC(F,L)
• if F is empty report satisfiable and halt
• if F contains the empty clause L or F?L
• return
• else choose a literal x
• FC(Fx,L)
• FC(F?x,L)
• add F to L

Start with FC(F,?)
36
Proof Complexity of Formula Caching B,
Impagliazzo, Pitassi, Segerlind 03

• Basic Formula Caching can exponentially improve
  DPLL
• Basic Formula Caching is not as powerful as
  regular resolution
• but some of its natural extensions are
• These natural extensions can be exponentially
  more powerful than general resolution or even
  Res(k) for any k !
• Res(k) is like resolution but uses k-DNFs instead
  of clauses

37
Extensions of Formula Caching

• Dont just check membership in the cache
• F is trivially unsatisfiable given G iff
  every clause of G contains some clause of F
• Record the reasons why residual formulas are
  known to be unsatisfiable and check the reasons
  rather than the formulas

38
More General Formula Caching

• FC(F,L)
• if F is empty report satisfiable and halt
• if F is trivially unsatisfiable given L
• add F to L
• return
• else choose a literal x
• FC(Fx,L)
• FC(F?x,L)
• add F to L

39
Adding Reasons

• FCreason(F,L)
• if F is empty report satisfiable and halt
• if F is trivially unsatisfiable based on G?L
• add F to L
• return(G)
• else choose a literal x
• G?FCreason(Fx,L)
• H?FCreason(F?x,L)
• J? (?x?G) (x?H)
• add F,J to L
• return(J)

40
Combining Clause Learning and Formula Caching

• Cachet system used to count of satisfying
  assignments
• Sang, Bacchus, B., Kautz, Pitassi 04
• Built on zchaffs clause learning
• Uses decomposition of residual formula into
  components
• Caches components rather than the whole formula
• Subtle interactions

41
Outline

• Intro to proof complexity
• Proof complexity and complete SAT algs
• Resolution/DPLL review
• Application to random k-SAT
• DPLL extensions
• Clause learning
• Caching
• Proof system survey

42
Proof Systems Hierarchy
ZFC
Ext-Frege
Frege
LS
AC0-Frege
Cutting Planes
PCR
Res(k)
Polynomial Calculus
Resolution
DPLL
Nullstellensatz
Truth Tables
43
AC0-Frege

• Constant-depth Frege (AC0-Frege)
• Like Frege but formulas in proof only have
  constant of alternations between ?
  and ?
• Resolution is depth-1 Frege

44
Some Known Hard Problems for Resolution

• Exact/Modular Counting
• Pigeonhole principle n1 pigeons cant fit into
  n pigeonholes without getting cosy with each
  other
• Parity Principle e.g. cant pair up an odd
  number of people
• Tseitin Tautologies
• These problems are also hard for AC0-Frege
• Random Unsatisfiable
• k-CNF
• Graph k-colorability
• Existence of k-cliques in graphs

45
Cutting Planes/PseudoBoolean

• Introduced to relate integer and linear
  programming Gomory 59, Chvatal 73
• Objects are linear integer inequalities
• Clause (x1? ?x2 ? x3) becomes inequality
  x1(1-x2)x3 ? 1
• Add inequalities xi ? 0 and 1-xi ? 0
• Goal derive 0 ? 1

46
Cutting Planes rules

• addition
• multiplication by positive integer
• Division by positive integer
  

47
Limitations of Cutting Planes

• At least as good as resolution
• Can also handle
• pigeonhole principle
• parity principle
• But cant handle counting when it is a derived
  property
• Clique-Coloring
• A graph cant have a k-clique and be
  k-1-colorable
• Tseitin tautologies still open

48
Proof Systems Hierarchy
ZFC
Ext-Frege
Frege
LS
AC0-Frege
Cutting Planes
PCR
Res(k)
Polynomial Calculus
Resolution
DPLL
Nullstellensatz
Truth Tables
49
Hilberts Nullstellensatz

• System of polynomials Q1(x1,,xn)0,,Qm(x1,
  ,xn)0 over field K has no solution
  in any extension field of K
  ?
  there exist polynomials
  P1(x1,,xn),,Pm(x1,,xn) in Kx1,,xn s.t.

50
Nullstellensatz proof system
C

• Clause (x1? ?x2 ? x3)
  becomes equation (1-x1)x2(1-x3)0
• Add equations xi2-xi 0 for each variable
• Guarantees only 0-1 solutions
• A proof is polynomials P1,, Pmn proving
  unsatisfiability i.e. such that

QC
51
Polynomial Calculus

• Similar to Nullstellensatz except
• Begin with Q1,,Qmn as before
• Given polynomials R and S can infer
• a?R b?S for any a, b in K
• xi?R
• Derive constant polynomial 1
• Degree maximum degree of polynomial appearing
  in the proof
• Can find proof of degree d in time nO(d) using
  Groebner basis-like algorithm (linear algebra)

52
PCR PC Resolution

• Two variables x and x for each proposition x
• x stands for ?x
• include equations xx-10, x2-x0, and
  (x)2-x0
• Translate (x1? ?x2 ? x3) as x1 x2 x30
• Same proof rules and proof search as polynomial
  calculus
• Efficiently simulates resolution clause?monomial
  
• Degree-size relationship for PCR is same as
  width-size relationship of resolution

53
Lovasz-Schrijver (LS) Proofs

• Linear inequalities
• Steps
• Lift to get degree 2 inequalities
• E.g. multiply inequalities
• Project
• Combine degree 2 inequalities plus x2x to cancel
  out degree 2 terms
• Captures properties of semi-definite programming

54
Proof Systems Hierarchy
ZFC
Ext-Frege
Frege
LS
AC0-Frege
Cutting Planes
PCR
Res(k)
Polynomial Calculus
Resolution
DPLL
Nullstellensatz
Truth Tables
55
Conclusions

• Exchange of ideas between proof complexity and
  SAT algorithms can be fruitful
• Better understanding of existing algorithms
• Source of ideas for new algorithms and proof
  systems
• Potential to branch out beyond DPLL derivatives
• Wide range of proof systems to explore

56
Thank You
Thank you to Toby Walsh Fahiem
Bacchus Ian Miguel