MCS%20312:%20NP%20Completeness%20and%20Approximation%20algorthms

1
MCS 312 NP Completeness and Approximation
algorthms

• Instructor
• Neelima Gupta
• ngupta_at_cs.du.ac.in

2
Table of Contents

• Circuit Satisfiablity is NP Complete
• (CNF) SAT is NP Complete
• 3(CNF) SAT is NP Complete

3
C SAT is NP Complete

• The Cooks Theorem

4
The CIRCUIT-SAT is in NP

• Do it yourself

5
Circuit SAT or C SAT is a set of all
combinatorial circuits such that C is satisfiable
ltcgt ltcgt is satisfiable
All NP Hard problems should be reducible to CSAT
for it to be NP Hard
6
Proof
Let P be an arbitrary problem in NP. P is a
decision problem (say).
Let L be the language corresponding to P.
Since P ? NP, there exists an algorithm A that
verifies P (or L) in polynomial time.
This implies that for every x ? L, there exists a
certificate y, polynomial in the length of x,
such that A(x,y) 1 (by def. of NP)
7
Aim
To reduce the given NP problem, P to CSAT
The algorithm A changes the state of the
system from one configuration to another
Suppose x n Then y O(nk), const k (by
def.) A runs in time T(n) O(x y)
O(nk)
8
Reduction of an NP problem to Circuit-SAT

• Claim
• Cx is satisfiable if there exists a certificate y
  such that A(x, y) 1.
• If Cx is satisfiable then there exists a
  certificate y such that A(x, y) 1.
• Proof The claim follows from the construction
  that Cx (y) A(x, y).

9
Reduction of an NP problem to Circuit-SAT

• Claim Reduction runs in polynomial time.
• Proof Well prove that the size of the circuit
  Cx is polynomial in x.
• 1. Size of the program for A is independent of
  the size of x. (Size of any program is
  independent of its input size, its execution time
  may depend but not the size of the code.)
• 2. y is polynomial in x
• 3. Working Storage Area in each configuration
  O(T(n)) where T(n) is the running time of A.
  (Thats true for any algorithm)
• 4. Size of M is polynomial in the length of a
  configuration.
• 5. Number of configurations is T(n).
• Hence proved.

10
The CIRCUIT-SAT is in NP

• We shall construct a 2-input , polynomial
• time algorithm that can verify CIRCUIT -
• SAT.

11
The CNF-SAT Problem

• Terminology
• Boolean Formula is a parenthesized expression
  formed from Boolean variables using Boolean
  operations, such as AND, OR, NOT, IMPLIES, IF AND
  ONLY IF.
• A clause is formed as the OR of Boolean variables
  or their negation called literals.
• A Boolean Formula is in Conjunctive Normal
  Form(CNF) if it is formed as a collection of
  subexpressions, called clauses, which are
  combined using AND.
• For example, the following Boolean formula is in
  CNF
• (x1 x2 x4 x7) (x3 x5) (x2 x4 x6
  x8) (x1 x3 x5 x8)

12
The CNF-SAT Problem

• Statement CNF SAT takes a Boolean
• formula in CNF as input and asks if there is
• an assignment of Boolean values to its
• variables so that the formula evaluates to 1.

13
The CNF-SAT Problem

• To Prove CNF-SAT is NP Complete
• Step 1 Show that CNF-SAT belongs to NP
• It is possible to design a polynomial time
  algorithm
• which takes the following 2 inputs and checks
• whether the assignment specified by S satisfies
• every clause in I
• An instance I of the problem
• A proposed solution S

14
CNF-SAT is NP-Hard

• Step 2 Show that CNF-SAT is NP hard
• We shall do this through the Local-Replacement
  approach
• by trying to reduce the CIRCUIT-SAT problem to
  CNF_SAT
• in polynomial time.
• Given A Boolean circuit C
• Assume that each AND / OR gate has 2 inputs, and
• each NOT gate has 1 input

15
CNF-SAT is NP Hard Reduction from Circuit-SAT
16
CNF SAT is NP-Hard

• ? y4 ?(y1 ?? x1 ?x2)
• ?(y1 ?? x3)
• ?(y3 ?? x2 ?y2)
• ?(y4 ?? y3 ? y1)
• Note that for every assignment of xis there is
  an assignment of yjs. These values of xis and
  yjs will always saitisfy the last four clauses
  above by construction
• Suppose the the circuit is satisfiable. Then
  there exists an assignment of xis which makes y4
  true.. Hence the above formula is satisfiable.
• If the circuit is not satisfiable. Then every
  assignment of xis will force y4 to be set to
  false. Hence the above formula is not
  satisfiable.

17
Converting Implications to CNF

• Create a formula Bg corresponding to each gate g
  in C as follows
• 1. If g is an AND gate with inputs a and b
  (which could be either xis or yis)
• and output c, then Bg (c ? (a.b)).
• 2. If g is an OR gate with inputs a and b, and
  output c, then Bg (c ?
• (ab))
• 3. If g is a NOT gate with input a and output
  b, then Bg (b ? a)
• Convert each Bg to CNF as follows
• 1. Construct a truth table for Bg.
• 2. Derive a formula for Bg in CNF form.
• ( If you dont know how to do it directly then
  convert it into DNF and then convert it into CNF
  by De Margans Law)

18
CNF-SAT An Example

• Conversion of BG1
• (y1 ? (x1.x2)) to CNF

y1 x1 x2 Bg1 (y1 ? (x1.x2))
1 1 1 1
1 1 0 0
1 0 1 0
1 0 0 1
0 1 1 0
0 1 0 1
0 0 1 0
0 0 0 1
19
CNF-SAT An Example

• DNF formula for BG1
• x1.x2.y1 x1.x2.y1 x1.x2.y1 x1.x2.y1
• CNF formula for BG1
• (x1x2y1)(x1x2y1)(x1x2y1)(x1x2y1)
  
• Similarly CNF formula for BG3
• (x2y2y3)(x2y2y3)(x2y2y3
  )(x2y2y3)

20
CNF-SAT An Example

• Conversion of BG2
• (y2 ? x3) to CNF

x3 y2 Bg2 (y2 ? x3)
0 0 0
0 1 1
1 0 1
1 1 0
21
CNF-SAT An Example

• DNF formula for BG2
• x3.y2 x3.y2
• CNF formula for BG2
• (x3y2)(x3y2)
• BG3 is similar to BG1

22
CNF-SAT An Example
y1 y3 y4 Bg4 (y4 ? (y1 V y3))
0 0 0 1
0 0 1 0
0 1 0 0
0 1 1 1
1 0 0 0
1 0 1 1
1 1 0 0
1 1 1 1

• Conversion of BG4
• (y4 ? (y1 V y3)) to CNF

23
CNF-SAT An Example

• DNF formula for BG4
• y1.y3.y4 y1.y3.y4 y1.y3.y4 y1.y3.y4
  
• CNF formula for BG4
• (y1y3y4)(y1y3y4)(y1y3y4)(y1y3y4)

24
CNF-SAT An Example

• CNF for the entire circuit is given by
• ?y4(x1x2y1)(x1x2y1)(x1x2y1)(x1x2y1
  )(x3y2)(x3y2)(x2y2y3)(x2y2y3)(x2y2y
  3)(x2y2y3)(y1y3y4)(y1y3y4)(y1y3y4)(y1
  y3y4)
• Note that for every assignment of xis there is
  an assignment of yjs. These values of xis and
  yjs will always satisfy the above clauses (but
  y4) by construction.
• Suppose the circuit is satisfiable. Then there
  exists an assignment of xis which makes y4
  true.. Hence the above formula is satisfiable.
• If the circuit is not satisfiable. Then every
  assignment of xis will force y4 to be set to
  false. Hence the above formula is not
  satisfiable.

25
THE 3SAT PROBLEM

• Statement This problem takes a Boolean
• formula S in CNF, having exactly 3 literals,
• and asks if S is satisfiable.
• Observation This is a restricted version of
• the CNF-SAT problem.

26
THE 3SAT PROBLEM

• To prove 3SAT is NP-Complete
• Step 1 3SAT is obviously in NP (by restriction
  3 SAT is a special case of CNF SAT)
• Step 2 Show that 3SAT is NP hard
• Observe that Restriction form of NP hardness
  proof does
• not apply to this situation. WHY ?
• We shall use the local replacement form of proof
  for this
• and try to reduce the CNF-SAT problem to 3SAT in
• polynomial time.

27
THE 3SAT PROBLEM

• Given A Boolean formula C in CNF.
• Perform the following local replacement for each
  clause Ci in C
• If Ci (a), that is, it has a single term,
  replace Ci with
• Si (a b c) . (a b c) . (a b
  c) . (a b c)
• where b and c are new variables not used
  anywhere else.
• If Ci (ab), that is, it has 2 terms, replace
  Ci with
• Si (a b c) . (a b c)
• where c is a new variable not used anywhere
  else.
• If Ci (abc), that is, it has 3 terms, set Si
  Ci
• If Ci (a1 a2 . ak), that is, it has
  kgt3 terms, replace Ci with
• Si (a1 a2 b1) . (b1 a3 b2) .
  (b2 a4 b3)..( bk-3 ak-1 ak)
• where b1, b2,..... bk-1 are new variables
  not used anywhere else.

28
Reducing SAT to 3SAT

• If Ci 1 then let a_r be true..
• if r1 or 2, then (a1 \/ a2 \/ b1) is satisfied
  so set all b_is to false to satisfy all other
  clauses
• if rk-1 or k, then (b_r-3 \/ a_r-1 \/ a-r)
  is satisfied so set all b_is to true to satisfy
  all other clauses
• if 2 lt r lt k-1, then (b_r-2 \/ a_r \/ b_r-1)
  is satisfied so set b1 b2 ... b_r-2
  true to satisfy the first r-2 clauses and set
  b_i-1 b_i ... b_r-3 false to satisfy
  the remaining clauses.
• Hence, Si is satisfied.

29
Reducing SAT to 3SAT

• If Si is satisfiable then
• if none of the new variables b1, ..., br-3 is
  set to false, then (br-3 \/ ar-1
  \/ ar) can only be satisfied by setting either
  ar-1 true or ar true
• If b1 is set to false, then (a1 \/ a2 \/ b1) can
  only be satisfied by setting either a1
  true or a2 true
• else let b_r be the first new variable set to
  false i.e. b1 b2 ... br-1 true then
  (br-1 \/ ar1 \/ br) can only be
  satisfied by setting ar1 true
• Thus in all the cases, one of the original
  literals must be set to true. Hence Ci is
  satisfied.

30
Reducing SAT to 3SAT

• Reduction is polynomial time
• Each clause increases in size by at most a
  constant factor and the computations involved are
  simple substitutions.
• Thus 3SAT is NP-Complete.

31
POLYNOMIAL TIME VARIANTS OF SAT

• The following 2 variants of the SAT problem can
• be solved in polynomial time, and therefore
  belong
• to the complexity class P.
• If all clauses contain at most one positive
  literal, then the Boolean formula is called a
  Horn Formula, and a satisfying truth assignment
  can be found by greedy
• algorithm.
• If the clauses have only 2 literals, then SAT can
  be solved in linear time by finding the strongly
  connected components of a particular graph
• constructed from the instance.
• Well not do them here. Do them yourself.

32
The 2SAT Problem

• Instance Collection C c1, , cm of clauses
  on a set U of n Boolean variables such that ci
  2 for 1 i m.
• Question Is there a truth assignment for the
  variables in U that satisfies all clauses in C?

33
The 2SAT Problem

• 2SAT can be solved by formulating it as a graph
  algorithm
• Let ? be an instance of 2SAT. Construct a
  directed graph G(?) such that vertices of G(?)
  are the variables of ? and their negations.
• There is an arc (x, y) in G(?) if and only if
  there is a clause (x y) or (y x) in ?.

34
The 2SAT Problem

• Note a b is equivalent to each of a ? b and
  b ? a.
• Thus a 2SAT formula may be viewed as a set of
• implications. Accordingly, if we have the
  following formula
• (a b) (b c) (c d)
• then we have a string of implications a ? b ? c ?
  d, which
• leads to a ? d,
• If for some variable a, there is a string of
  implications
• a ? ... ? a, and another string of
  implications
• a ? ... ? a, then the formula is not
  satisfiable,
• otherwise the formula is satisfiable.

35
The 2SAT Problem

• Consider the following formula
• (x1x2) (x2x3) (x1x2) (x3x4) (x3x5)
• (x4 x5) (x3 x4).
• The implication graph is as follows

36
The 2SAT Problem

• x1 x2
• x4 x3
• x5 x5
• x3 x4
• x2 x1

37
The 2SAT Problem

• The 2SAT problem thus reduces to the graph
  problem of finding strongly connected components
  (SCC) in the implication graph.
• A 2SAT formula is unsatisfiable if and only if
  some variable and its complement reside in the
  same SCC.
• As SCC is known to have a linear-time solution
  and the implication graph is constructible in
• linear time, it is clear that 2SAT may be
  decided under the same time bound.

38
Horn FormulaReferences 1. Chapter 5 Greedy
Algorithms (S. Dasgupta, C.H. Papadimitriou, and
U.V. Vazirani)2. http//www.cs.berkeley.edu/daw/
teaching/cs170-s03/Notes/lecture15.pdf(UC
BerkeleyCS 170 Ecient Algorithms and
Intractable ProblemsLecturer David Wagner)

• In Horn formulas, knowledge about variables is
  represented by two kinds of clauses
• 1. Implications, whose left-hand side is an AND
  of any number of positive literals and whose
  right-hand side is a single positive literal.
  These express statements of the form
• x1 x2 xk implies xk1
• which can alternatively be written as
  follows
• x1 v x2 v v xk v xk1
• Here, if the conditions on the left hold, then
  the one on the right must
• also be true.
• A degenerate type of implication is the singleton
  implies x, meaning
• simply that x is true.
• 2. Pure negative clauses, consisting of an OR of
  any number of negative literals, as in (u v v v
  y)

39
Horn Formula

• The implications tell us to set some of the
  variables to true.
• The negative clauses encourage us to make them
  false.
• Strategy for solving a Horn formula
• start with all variables set to false.
• proceed to set some of them to true, one by one,
  only if an implication would otherwise be
  violated.
• once done with this phase, when and all
  implications are satisfied, turn to the negative
  clauses and make sure they are all satisfied.