TUE ppt versie 0.3

1
Algorithms (2IL15) Lecture 12 Linear Programming
2

• Today linear programming (LP)
• most used and most widely studied optimization
  method
• can be solved in polynomial time (input size
  measured in bits)
• can be used to model many problems
• also used in many approximation algorithms
  (integer LP rounding)
• we will only have a very brief look at LP
• what is LP? what are integer LP and 0/1-LP ?
• how can we model problems as an LP ?
• and not study algorithms to solve LPs

3

• Example problem running a chocolate factory
• Assortment
• Cost and availability of ingredients

cacao milk hazelnuts retail price (100 g)
Pure Black 1 0 0 1.99
Creamy Milk 0.6 0.4 0 1.49
Hazelnut Delight 0.6 0.2 0.2 1.69
Super Nuts 0.5 0.1 0.4 1.79
cost (kg) available (kg)
cacao 2.1 50
milk 0.35 50
hazelnuts 1.9 30
How much should we produce of each product to
maximize our profit ?
4

• Modeling the chocolate-factory problem

cacao milk hazelnuts price (100 g)
Pure Black 1 0 0 1.99
Creamy Milk 0.6 0.4 0 1.49
Hazelnut Delight 0.6 0.2 0.2 1.69
Super Nuts 0.5 0.1 0.4 1.79
cost (kg) available (kg)
cacao 2.1 50
milk 0.35 50
hazelnuts 1.9 30
variables we want to determine production (kg)
of the products b production of Pure Black m
production of Creamy Milk h production
of Hazelnut Delight s production of Super
Nuts
5

• Modeling the chocolate-factory problem (contd)

cacao milk hazelnuts price (100 g)
Pure Black 1 0 0 1.99
Creamy Milk 0.6 0.4 0 1.49
Hazelnut Delight 0.6 0.2 0.2 1.69
Super Nuts 0.5 0.1 0.4 1.79
cost (kg) available (kg)
cacao 2.1 50
milk 0.35 50
hazelnuts 1.9 30
profits (per kg) of the products Pure Black 19.9
2.1 17.8 Creamy Milk 14.9 0.6 x 2.1 0.4
x 0.35 13.5 Hazelnut Delight 15.19 Super
Nuts 16.055 total profit 17.8 b 13.5 m
15.19 h 16.055 s
6

• Modeling the chocolate-factory problem

cacao milk hazelnuts price (100 g)
Pure Black 1 0 0 1.99
Creamy Milk 0.6 0.4 0 1.49
Hazelnut Delight 0.6 0.2 0.2 1.69
Super Nuts 0.5 0.1 0.4 1.79
cost (kg) available (kg)
cacao 2.1 50
milk 0.35 50
hazelnuts 1.9 30
we want to maximize the total profit 17.8 b
13.5 m 15.19 h 16.055 s under the
constraints b 0.6 m 0.6 h 0.5 s 50
(cacao availability) 0.4 m 0.2
h 0.1 s 50 (milk
availability) 0.2 h 0.4
s 30 (hazelnut
availability) This is a linear program optimize
linear function, under set of linear constraints
7

• Linear programming
• Find values of real variables x, y such
  that
• x - 3 y is maximized
• subject to the constraints
• - 2 x y - 4
• x y 3
• - ½ x y 2
• y 0

y
x
8

• Linear programming
• Find values of real variables x, y such
  that
• x - 3 y is maximized
• subject to the constraints
• - 2 x y - 4
• x y 3
• - ½ x y 2
• y 0

y ½ x 2
1 -3
y 0
y 2x - 4
y - x 3
feasible region region containing feasible
solutions region containing solutions
satisfying all constraints feasible region is
convex polytope in n-dim space
9

• Linear programming
• Find values of real variables x1, , xn such that
  
• given linear function c1 x1 c2 x2 cn
  xn is maximized (or minimized)
• and given linear constraints on the variables
  are satisfied
• constraints equalities or inequalities
  using or , cannot use lt and gt
• Possible outcomes
• unique optimal solution vertex of feasible
  region

10

• Linear programming
• Find values of real variables x1, , xn such that
  
• given linear function c1 x1 c2 x2 cn
  xn is maximized (or minimized)
• and given linear constraints on the variables
  are satisfied
• constraints equalities or inequalities
  using or , cannot use lt and gt
• Possible outcomes
• unique optimal solution vertex of feasible
  region
• no solution feasible region in empty

11

• Linear programming
• Find values of real variables x1, , xn such that
  
• given linear function c1 x1 c2 x2 cn
  xn is maximized (or minimized)
• and given linear constraints on the variables
  are satisfied
• constraints equalities or inequalities
  using or , cannot use lt and gt
• Possible outcomes
• no solution feasible region in empty
• unique optimal solution vertex of feasible
  region
• bounded optimal solution, but not unique

12

• Linear programming
• Find values of real variables x1, , xn such that
  
• given linear function c1 x1 c2 x2 cn
  xn is maximized (or minimized)
• and given linear constraints on the variables
  are satisfied
• constraints equalities or inequalities
  using or , cannot use lt and gt
• Possible outcomes
• no solution feasible region in empty
• unique optimal solution vertex of feasible
  region
• bounded optimal solution, but not unique
• unbounded optimal solution

13

• Linear programming standard form
• Maximize c1 x1 c2 x2 cn xn
• Subject to a1,1 x1 a1,2 x2 a1,n xn
  b1
• a2,1 x1 a2,2 x2 a2,n xn b2
• 
  ....
• am,1 x1 am,2 x2 am,n xn bm
• x1 0
• x2 0
• xn 0
• Maximize cx subject to A x b and
  non-negativity constraints on all xi

14

• Lemma Any LP with n variables and m constraints
  can be rewritten as an
• equivalent LP in standard form with 2n variables
  and 2n2m constraints.
• Proof. LP may not be in standard form because
• minimization instead of maximization
• negate objective function
• minimize 2 x1 - x2 4x3 ? maximize -2
  x1 x2 - 4 xn
• some constraints are or instead
  of
• getting rid of replace 3 x x2 - x3
  5
• by 3 x x2 - x3 5
• 3 x x2 - x3 5
• changing to negate constraint
• 3 x x2 - x3 5 ? - 3 x - x2
  x3 - 5

15

• Lemma Any LP with n variables and m constraints
  can be rewritten as an
• equivalent LP in standard form with 2n variables
  and 2n2m constraints.
• Proof (contd). LP may not be in standard form
  because
• minimization instead of maximization
• some constraints are or instead
  of
• variables without non-negativity constraint
• for each such variable xi introduce two new
  variables ui and vi
• replace each occurrence of xi by (ui -
  vi )
• add non-negativity constraints ui 0 and
  vi 0

16

• Lemma Any LP with n variables and m constraints
  can be rewritten as an
• equivalent LP in standard form with 2n variables
  and 2n2m constraints.
• Proof (contd).
• variables without non-negativity constraint
• for each such variable xi introduce two new
  variables ui and vi
• replace each occurrence of xi by (ui -
  vi )
• add non-negativity constraints ui 0 and
  vi 0
• new problem is equivalent to original problem
• for any original solution there is new solution
  with same value
• if xi 0 then ui xi and vi 0, otherwise ui
  0 and vi - xi
• and vice versa
• set xi (ui - vi )

17

• Lemma Any LP with n variables and m constraints
  can be rewritten as an
• equivalent LP in standard form with 2n variables
  and 2n2m constraints.
• Instead of standard form, we can also get
  so-called slack form
• non-negativity constraint for each variable
• all other constraints are , not or
• Standard form (or slack form) convenient for
  developing LP algorithms
• When modeling a problem just use general form

18

• Algorithms for solving LPs
• simplex method
• worst-case running time is exponential
• fast in practice
• interior-point methods
• worst-case running time is polynomial in input
  size in bits
• some are slow in practice, others are
  competitive with simplex method
• LP when dimension (number of variables) is
  constant
• can be solved in linear time (see course
  Geometric Algorithms)

19

• Modeling a problem as an LP
• decide what the variables are (what are the
  choices to be made?)
• write the objective function to be optimized
  (should be linear)
• write the constraints on the variables (should
  be linear)

20

• Example Max Flow

21
3
2 /
2
2 /
10
4 /
2
2 /
3
3 /
t
sink
s
source
2
2 /
1 /
5
1
1 /
1 /
5
3
0 /
flow 1, capacity 5

• Flow function f V x V ? R satisfying
• capacity constraint 0 f (u,v) c(u,v
  ) for all nodes u,v
• flow conservation for all nodes u ? s, t we
  have flow in flow out
• 
  ?v in V f (v,u) ?v in V f (u,v)
• value of flow f ?v in V f (s,v) -
  ?v in V f (v,s)

22

• Modeling Max Flow as an LP
• decide what the variables are (what are the
  choices to be made?)
• for each edge (u,v) introduce variable xuv
  ( xuv represents f(u,v) )
• write the objective function to be optimized
  (should be linear)
• maximize ?v in V xsv - ?v in V xvs
  (note linear
  function)
• write the constraints on the variables (should
  be linear)
• xuv 0 for all pairs of nodes u,v
  
• xuv c(u,v) for all pairs of nodes u,v
• ?v in V xvu - ?v in V xuv 0 for all
  nodes u ? s, t
• (note linear
  functions)

23

• Modeling Max Flow as an LP
• Now write it down nicely
• maximize ?v in V xsv - ?v in V xvs
• subject to xuv 0 for all pairs
  of nodes u,v
• xuv c (u,v) for all pairs
  of nodes u,v
• ?v in V xvu - ?v in V xuv
  0 for all nodes u ? s, t
• Conclusion Max Flow can trivially be written as
  an LP
• (but dedicated max-flow algorithm are faster than
  using LP algorithms)

24

• Example Vertex Cover

25

• G (V,E) is undirected graph
• vertex cover in G subset C V such that
  for each edge (u,v) in E
• we have u in C or v
  in C (or both)
• Vertex Cover (optimization version)
• Input undirected graph G (V,E)
• Problem compute vertex cover for G with minimum
  number of vertices
• Vertex Cover is NP-hard.
• there is a 2-approximation algorithm running in
  linear time.

n
26

• Modeling Vertex Cover as an LP
• decide what the variables are (what are the
  choices to be made?)
• for vertex v introduce variable xv
  
• ( idea xv 1 if v in cover, xv 0
  if v not in cover)
• write the objective function to be optimized
  (should be linear)
• minimize ?v in V xv
  (note linear
  function)
• write the constraints on the variables (should
  be linear)
• for each edge (u,v) write constraint xu xv
  1 (NB linear function)
• for each vertex v write constraint xu in
  0,1

not a linear constraint
27

• integrality constraint xi must be integral
• 0/1-constraint xi must be 0 or 1
• integer LP LP where all variables have
  integrality constraint
• 0/1-LP LP where all variables have
  0/1-constraint
• (of course there are also mixed versions)

28

• Theorem 0/1-LP is NP-hard.
• Proof. Consider decision problem is there
  feasible solution to given 0/1-LP?
• Which problem do we use in reduction?
• Need to transform 3-SAT formula into instance of
  0/1-LP
• maximize y1 (not relevant for decision
  problem, pick arbitrary function)
• subject to y1 y2
  (1-y3) 1
• y2 (1-y4) (1-y5 ) 1
• (1 - y2) y3 y5
  1
• yi in 0,1 for all i

already saw reduction from Vertex Cover lets do
another one 3-SAT
( x1 V x2 V x3) ? (x2 V x4 V x5) ?
(x2 V x3 V x5 )
variable yi for each Boolean xi yi
1 if xi TRUE and yi 0 if xi FALSE
29

• Theorem 0/1-LP is NP-hard.
• problem can be modeled as normal LP
• ? problem can be solved using LP algorithms
• ? problem can be solved efficiently
• problem can be modeled as integer LP (or 0/1-LP)
• ? problem can be solved using integer LP (or
  0/1-LP) algorithms
• ? does not mean that problem can be solved
  efficiently
• (sometimes can get approximation algorithms
  by relaxation and rounding
• see course Advanced Algorithms)
• ? there are solvers (software) for integer LPs
  that in practice are quite efficient

30

• Summary
• what is an LP? what are integer LP and 0/1-LP?
• any LP can be written in standard form (or in
  slack form)
• normal (that is, not integer) LP can be solved
  in polynomial time
• (with input size measured in bits)
• integer LP and 0/1-LP are NP-hard
• when modeling a problem as an LP
• define variables and how they relate to the
  problem
• describe objective function (should be linear)
• describe constraints (should be linear, lt and gt
  not allowed)
• no need to use standard or slack form, just use
  general form