MCS 312: NP Completeness and Approximation algorthms

1
MCS 312 NP Completeness and Approximation
algorthms

• Instructor
• Neelima Gupta
• ngupta_at_cs.du.ac.in

2
LP- Based Approximation
3
Table of Contents

• Lp rounding
• Dual Fitting
• LP-Duality

4
Linear Programming Problem

• A linear programming (LP) problem is an
  optimization problem in which we minimize or
  maximize a linear objective function subject to a
  given set of linear constraints.
• Example
• Minimize 3x1 - 5x2 3x3 2x4 subject to
• 3x1 4x2 6
• -x3 2x1 - x2 22
• x5 -3.5
• x3 .5x4 .8
• xi 0 for all i

5
Solutions

• Feasible Solution
• A feasible solution to a linear program is a
  solution that satis?es all constraints.
• Optimal Solution
• An optimal solution to a linear program is a
  feasible solution with the largest(smallest)
  objective function value for a maximization(minimi
  zation) problem.

6

• Many optimization problems involve selecting a
  subset of a given set of elements.
• Examples
• A vertex cover is a subset of vertices.
• A spanning tree is really a subset of edges.
• A knapsack solution is a subset of items.
• Can be formulated as LPs with integrality
  constraints.

7
Integer Program

• An Integer Program (IP) is an LP with
  Integrality Constraints
• Integrality Constraints Some or all the
  variables are constrained to be integers.

8
Solving Linear/Integer Programming Problems

• LPs can be solved efficiently (polynomially but
  slowly).
• IPs generally cannot be solved efficiently (it is
  NP hard). Some specific IPs can be solved
  efficiently. Actually, their LP optimal is
  guaranteed to be integral.

9
Using Indicator Variables

• Many selection problems can be formulated as IPs
  using indicator variables (or 0-1 variables).
• An indicator variable is defined for each element
  . A value of 1 indicating the selection of the
  element and a value of 0 indicating otherwise.

10
Few Examples are

• vertex cover
• Set Cover
• Knapsack

11
Example Unweighted Vertex Cover

• Variables xv v ? V .
• The IP
• Minimize ? xv
• s.t.
• xu xv 1 ? (u, v) ? E,
• xv ? 0, 1 ? v ? V.

12
Example Knapsack

• Let the item names be 1, . . . , n.
• Variables xi 1 i n.
• The IP
• Minimize ?i cixi
• s.t.
• ?i sixi K,
• xi ? 0, 1 ? 1 i n.

13
Solving Linear/Integer Programming Problems

• LPs can be solved efficiently (polynomially but
  slowly).
• IPs generally cannot be solved efficiently (it is
  NP hard). Some specific IPs can be solved
  efficiently. Actually, their LP optimal is
  guaranteed to be integral.

14
LP Relaxation (Drop the integrality constraint)

• Example Unweighted Vertex Cover
• The IP
• Minimize ?v xv
• s.t.
• xu xv 1 ? (u, v) ? E,
• xv ? 0, 1 ? v ? V.
• The LP relaxation
• Minimize ?v xv
• s.t.
• xu xv 1 ? (u, v) ? E,
• xv gt 0 ? v ? V.

15
Example Weighted Vertex Cover

• Variables xv v ? V .
• The IP
• Min ?Cv xv
• where
• Cv cost associated with vertex
• xv indicator variable
• s.t
• xu xv 1 ? (u, v) ? E
• xv ? 0, 1 ? v ? V

16
LP Relaxation (Drop the integrality constraint)

• Example Weighted Vertex Cover
• The IP
• Min ?Cv xv
• s.t
• xu xv 1 ? (u, v) ? E
• xv ? 0, 1 ? v ? V
• The LP relaxation
• Min ?Cv xv
• s.t
• xu xv 1 ? (u, v) ? E
• xv 0 ? v ? V

17
LP- Rounding
18
LP rounding

• If xv ½, round it up to 1
• Else round it down to 0.
• Here xv is the solution obtained from LP
• E.g
• LP ¼ c1 ½ c2 ¾ c3 4/5 c4
• IP c2 c3 c4

19
Claim 1 Solution Obtained is feasible

• Let (u,v) ? E
• Since the solution of LP is feasible, values of
  xv , v ? V, satisfy
• xu xv 1 (1)
• ? atleast one of xu and xv ½
• Assume xu and xv be the solutions obtained
  after rounding, then at least one of them must be
  1, i.e.
• xu xv 1
• So the solution, obtained after rounding, is
  feasible.

20
Claim 2 C(S) 2LOPT

• According to the strategy some of the variables
  have been increased to a maximum of double some
  have been reduced to 0, i.e Cv lt 2Cv.

21

• So,
• C(S) cost of solution obtained by IP
• C(S) ?v Cv xv
• 2 ?v Cv xv ( xv 2 Xv )
• 2 LPOPT
• Hence claim 2 follows

22
Set Cover Problem

• A ?nite set (universe) U of n elements, U e1,
  e2,, en, a collection of subsets of U i.e.
  S1, S2, ., Sk with some cost, select a minimum
  cost collection of these sets that covers all
  elements of U.

23

• IP
• Indicator variable xs, xs ? 0,1
• xs 0 if set S is not picked
• xs 1 if set S is picked
• Min ?s Cs xs
• s.t.
• ?se belongs to S xs 1 ? e ? U
• xs 0,1
• LP Relaxation
• Min ?s Cs xs
• s.t.
• ?se belongs to S xs 1 ? e ? U
• xs gt 0

24
LP rounding for SC

• Let f denote the maximum frequency of any element
  in U Si
• Find an optimal solution to LP-Relaxation
• xs gt1/f round it to 1
• xs lt1/f discard the set, i.e. round it
  down to 0.

25
Claims

• Claim 1 solution is feasible
• Claim 2 It gives factor f approximation

26
Claim 1 Solution is feasible

• Let,
• ei ? U , 1in
• S be the collection of subsets of U
• em 1ltm ltn belongs to l subsets of S
  where 1ltlltk
• Since the solution of LP is feasible i.e.
  values of xs s ? S obtained satisfies
• xs1 xs2 xs3 . xsl gt1 (1)
• ? atleast one of xs1, xs2, xs3,., xl gt1/f
• ? xs1 xs2 xs3 . xlgt 1
• Where xsi is the solution obtained after
  rounding. Thus it is feasible.

27
Claim 2 Factor f approximation

• For each set s ? Collection of picked sets(S), xs
  has been increased by a factor of atmost f.
• Let C(s) Cost of our solution
• Therefore,
• C(S) ?s Cs xs ? s ? S
• f ?s Cs xs ( xs
  f xs)
• f LPOPT
• Hence it is a factor f approximation.
• Note f factor could be large. Later well see a
  technique of rounding that gives O(log n) factor.

28
LP- Duality
29

• Linear Programming - Example
• Minimize 8x1 5x2 5x3 2x4 subject to
• 3x1 4x2 6
• 3x2 x3 x4 5
• xi 0 for all i
• x (2, 1,0, 3) is a feasible solution.
• 82 51 23 27 is an upper bound.

30
What is the Lower Bound?

• Minimize 8x1 5x2 5x3 2x4 subject to
• 3x1 4x2 6
• 3x2 x3 x4 5
• xi 0 for all i
• LB 8x1 5x2 5x3 2x4 3x1 4x2 6
• Better LB
• 8x1 5x2 5x3 2x4 (3x1 4x2 ) (3x2
  x3 x4) 65 11

31
How to compute a good LB

• Minimize 8x1 5x2 5x3 2x4 subject to
• 3x1 4x2 6 .y1
• 3x2 x3 x4 5y2
• xi 0 for all i
• Assign a non-negative coefficient yi to every
  inequality such that
• 8x1 5x2 5x3 2x4 y1 (3x1 4x2 ) y2(3x2
  x3 x4 )
• Then, LHS 6y1 5y2.
• We are interested in finding yis such that RHS
  is maximum. This leads to our dual problem.

32

• The corresponding dual for the given example will
  be
• max 6y1 5y2
• such that,
• 3y1 lt 8
• 4y1 3y2 lt 5
• y1 lt 5
• y2 lt 2
• and, yi gt 0 for all i

33
Weak Duality Theorem

• Theorem If x and y are feasible then,
• gt
• Proof
• gt gt

34
Set Cover

• xs is 1 iff set S in included in the cover.
• The Primal
• Objective min ? Cs xs
• s.t
  gt 1 U
• xs 0,1
• LP relaxation xs gt 0

35

• Introduce an indicator variable ye for each of
  the constraints in primal.
• The Dual
• objective max
• s.t lt
  CSi for i 1 to k

36
Example

• S x, y, z, w
• S1 x, y
• S2 y, z
• S3 x, w, y
• Let xs1 , xs2 , xs3 be an indicator variable for
  S1 , S2 , S3 respectively.
• Let Cs1 , Cs2 , Cs3 is the cost of S1 , S2 , S3
  respectively.

37
Primal

• Min Cs1 xs1 Cs2 x2 Cs3 x3
• Subject to
• xs1 xs3 gt 1 (yx)
• xs1 xs2 xs3 gt 1 (yy)
• xs2 gt 1 (yz)
• xs3 gt 1 (yw)
• xs1, xs2, xs3 gt 0

38
Dual

• Max yx yy yz yw
• Subject to
• yx yy lt Cs1
• yy yz lt Cs2
• yx yy yw lt Cs3
• yx , yy , yz , yw gt 0

39
Dual Fitting

• From set cover via lp

40
Primal-Dual Schema
41
Complementary Slackness Conditions
42
Relaxed Complementary Slackness Conditions
43
Example Weighted Vertex Cover

• Primal
• Min ?Cv xv
• s.t
• xu xv 1 ? (u, v) ? E
• xv ? 0, 1 ? v ? V
• Dual
• Max ?ye
• s.t
• ?ee is incident on v ye lt Cv ? v ?
  V
• ye ? 0, 1 ? e ? E

44
Primal Dual Schema 1

• U empty, y 0
• For each edge e (u, v)
• ye min c(u) - ?e'u?e' ye' , c(v) - ?e'v?e'
  ye'
• U U union argmin c(u) - ?e'u?e' ye' , c(v) -
  ?e'v?e' ye'
• Output U

45
3
5
2
7
4
3
1
3
2
46
3
5
2
7
4 (1)
Ye 3
3 (0)
1
3
2
For every edge pick minimum of two
vertices Min4,3 3 Set ye3 U has vertex
having red color
47
3
5(4)
2
Ye 1
7
4 (1) (0)
Ye 3
3(0)
1
3
2
Min1,5 1 Set ye1
48
3
5(4)
2
Ye 1
7
4 (1) (0)
Ye 3
3(0)
Ye 0
Ye 0
Ye 0
1
3
2
Min1,0 0 Set ye0 Min2,0 0 Set ye0
Min3,0 0 Set ye0
49
3
5(4)(0)
2
Ye 1
Ye 4
7(3)
4 (1) (0)
Ye 3
3(0)
Ye 0
Ye 0
Ye 0
50
3
Ye 0
5(4)(0)
2
Ye 1
Ye 4
7(3)
4 (1) (0)
Ye 3
3(0)
Ye 0
Ye 0
Ye 0
51
3 (1)
Ye 2
Ye 0
2 (0)
5(4)(0)
Ye 1
Ye 4
7(3)
4 (1) (0)
Ye 3
3(0)
Ye 0
Ye 0
Ye 0
Red-colored nodes form a vertex-cover
52
3 (1)
Ye 2
Ye 0
2 (0)
5(4)(0)
Ye 1
Ye 4
7(3)
4 (1) (0)
Ye 3
3(0)
Ye 0
Ye 0
Ye 0
Red-colored nodes form a vertex-cover
53
Solution is feasible

• Trivial, since the algorithm runs for every edge.
• Let e (u,v) be an edge.
• Suppose if possible, none of the xu and xv has
  been set to 1 i.e constraints corresponding to u
  and v have not yet gone tight and we have a ye
  that can be raised. That means the algorithm has
  not yet completed.

54
Solution is 2 factor

• For every xv gt 0, dual constraint is tight
  (trivially).
• For every edge e (u,v),
• 1 lt xu xv lt 2
• Hence, by relaxed CSC, cost of the solution is at
  most twice the OPT.

55
Primal-Dual Schema 2 (Ignore)

• Raise the dual variables uniformly until one or
  more of the constraints become tight. Freeze the
  dual variables contributing to these constraints.
  Set the corresponding primal variable to 1.
• If more than one constraint becomes tight, take
  them one by one in an arbitrary order.

56
3
5
2
7
4
3
1
3
2
57
3
Ye 3/4
Ye 3/4
5
2
Ye 3/4
Ye 3/4
7
4
Ye 3/4
3 (1)
Ye 3/4
Ye 3/4
Ye 3/4
1
3
2
58
3
Ye 3/2
Ye 3/2
5
2
Ye 3/2
Ye 3/2
7
4
Ye 3/4
3 (1)
Ye 3/4
Ye 3/4
Ye 3/4
1
3
2
59
3
Ye 3/2
Ye 3/2
5
2
Ye 7/4
Ye 7/4
7
4
Ye 3/4
3 (1)
Ye 3/4
Ye 3/4
Ye 3/4
1
3
2
60
Solution is feasible

• Let e (u,v) be an edge.
• Suppose if possible, none of the xu and xv has
  been set to 1 i.e consraints corresponding to u
  and v have not yet gone tight and we have a ye
  that can be raised. That means the algorithm has
  not yet completed.

61
Solution is 2 factor

• For every xv gt 0, dual constraint is tight
  (trivially).
• For every edge e (u,v),
• 1 lt xu xv lt 2
• Hence, by relaxed CSC, cost of the solution is at
  most twice the OPT.