ESI 6448 Discrete Optimization Theory

1
ESI 6448Discrete Optimization Theory

• Section number 5643
• Lecture 4

2

• It has been said that there are three types of
  people                     
• Those who make things happen.
• Those who watch things happen.
• Those who wonder what happened.

3
Last class

• Easy problems
• LP relaxation solves IP if
• constraint matrix A is Totally Unimodular
• constraint is Totally Dual Integral (TDI)
• Greedy algorithm solves IP for
• Submodular optimization problem
• problem in matroid
• matric, graphic, partition matroid

4
Matroid

• r submodular rank function (i.e. r(S?j)
  r(S) ? 0, 1 for all S ? N and j ? N \ S )a set
  A ? N is independent if r(A) A.
• (N, F), where F is the set of independent sets,
  is called a matroid.
• Equivalently,
• (N, F) is an independence system (subset system)
  ifi) ? ? F andii) if A ? F then for all B ? A,
  B ? F.
• A ? F is called independent.
• Maximal independent set I ? A is called a basis
  of A.
• Rank of A is the size of any basis of A, r(A)

5
Properties of Matroids

• Given a matroid (N, F)1. If A is an independent
  set and B ? A, then B is an independent set.
  (independence system or subset system)2. If A
  and B are independent sets w/ A gt B, then
  there exists some j ? A \ B s.t. B ? j is
  independent.3. Every maximal independent set has
  the same cardinality.
• For a subset system S, greedy algorithm solves
  the maximum weight problem in S iff S is a
  matroid
• Matric matroid, Graphic matroid, Partition matriod

6
Matric matroid

• Ground set is the set of columns/rows of a matrix
• Independent sets are the sets of linearly
  independent rows/columns
• rank of an independent set is the number of
  linearly independent columns/rows of a
  corresponding matrix.

7
Example

• I e1, e2, e3, e4 are linearly
  independent,rank(I) I
• I e1, e4, e5, e6 are not linearly
  independent,rank(I) lt I
• I e4, e6, e7 are not linearly
  independent,rank(I) lt I

8
Graphic matroid

• Ground set is the set of edges of a graph
• Independent sets are the sets of edges of a graph
  that do not form a cycle
• rank of an independent set is V number of
  connected components in corresponding graph

9
Example
4
1
3
2
5
G (V, E)

• r(E) 3 E, E independent
• r(E) 2 lt E , E dependent

10
Partition matroid

• Ground set is the union of m finite disjoint sets
  Ei, for 1 ? i ? r
• Independent sets are sets formed by taking at
  most one element from each set Ei
• For each subset E, let ki min E ? Ei, 1
• rank of an independent set is ?mi1 ki

11
Example

• Given D (V, A) and nonnegative arc weight, find
  a subset B of A w/ largest weight s.t. no two
  arcs of B have the same head.
• subset system (A, B), where subset C of A is in
  B iff no two arcs of B have the same head

partition Ei set of arcs that have the same
head E1 21, 41, 51 E2 12, 32 E3 24,
54 E4 35 E5 43 solution 12, 24, 43,
35, 21
3
1
5
2
4
7
5
7
2
4
6
8
4
3
12
Simplex algorithm for LP

• Assumptions
• LP in standard form
• min cxAx b, A m?n matrix of integers, m lt
  nx ? 0
• rank of A is m
• basis B of A is of rank m
• basic solution x corresponding to B is xj 0 for
  Aj ? B, xj the k-th component of B-1b
  (feasible?)
• The set F of feasible points is not empty
• at least one bfs exists

13
Simplex algorithm for LP

• degeneracy
• a bfs is degenerate if it contains more than n
  m zeros
• Basic idea
• optimal solution is found at vertex (bfs) for
  convex polytope
• moving from bfs to bfs along a feasible
  direction, find optimal solution
• d ? R is a feasible direction at x if ? ? gt 0
  s.t. x ?d ? P
• A(x ?d) b ? Ad 0 ? BdB Aj 0 ? dB B-1A
• reduced cost for each direction cj cBB-1Aj
• if all reduced cost is nonnegative, the solution
  is optimal

14
Simplex algorithm for LP

• 1. Start with a basis AB(1), ..., AB(m) and a bfs
  x.
• 2. Compute cj cBB-1Aj for each nonbasic index
  j.If they are all nonnegative, current sol is
  optimal.O.w. choose j s.t. cj cBB-1Aj lt 0.
• 3. Compute u B-1Aj.If no component of u is
  positive, optimal cost is -?.
• 4. O.w. let ? mini 1,...,mui gt 0 xB(i) /
  ui.Let l be s.t. xB(l) / ul ?. Form a new
  basis by replacing AB(l) w/ Aj. (pivoting) New
  bfs y is s.t. yj ? and yB(i) xB(i) ?ui.

15
Simplex algorithm for LP (full tableau)

• 1. Start with a basis AB(1), ..., AB(m) and a bfs
  x.
• 2. Examine the reduced costs in the zeroth
  row.If they are all nonnegative, current sol is
  optimal.O.w. choose j s.t. cj lt 0.
• 3. Examine u ( B-1Aj) which is j-th column.If
  no component of u is positive, optimal cost is
  -?.
• 4. O.w. let ? mini 1,...,mui gt 0 xB(i) /
  ui.Let l be s.t. xB(l) / ul ?. AB(l) exits
  the basis and Aj enters the basis. (pivoting)
  Make ul 1 and all other ui 0 using
  elementary row operations.

16
Full tableau
17
Example

• min 10x1 12x2 12x3 x1 2x2
  2x3 ? 20 2x1 x2 2x3 ? 20
  2x1 2x2 x3 ? 20
  x1, x2, x3 ? 0
• Using slack variables
• min 10x1 12x2 12x3 x1 2x2
  2x3 x4 20 2x1
  x2 2x3 x5 20
  2x1 2x2 x3 x6 20
  x1, ..., x6 ? 0

18
Example
min 10x1 12x2 12x3 x1 2x2
2x3 x4 20 2x1
x2 2x3 x5 20
2x1 2x2 x3 x6 20
x1, ..., x6 ? 0 x (0, 0, 0, 20,
20, 20) cost 0
19
Example
min 10x1 12x2 12x3 x1 2x2
2x3 x4 20 2x1
x2 2x3 x5 20
2x1 2x2 x3 x6 20
x1, ..., x6 ? 0 x (10, 0, 0, 0,
10, 10) cost -100
20
Example
min 10x1 12x2 12x3 x1 2x2
2x3 x4 20 2x1
x2 2x3 x5 20
2x1 2x2 x3 x6 20
x1, ..., x6 ? 0 x (0, 0, 10, 0,
0, 10) cost -120
21
Example
min 10x1 12x2 12x3 x1 2x2
2x3 x4 20 2x1
x2 2x3 x5 20
2x1 2x2 x3 x6 20
x1, ..., x6 ? 0 x (4, 4, 4, 0, 0,
0) cost -136
22
Example
(0, 0, 10)
(0, 0, 0)
(4, 4, 4)
(10, 0, 0)
23
Anticycling

• Lexicographic pivoting rule
• Entering column any j w/ cj is negative
• Exiting column for each i with ui gt 0, divide
  i-th row by ui and choose the lexicographically
  smallest row l.
• Blands rule (smallest subscript pivoting rule)
• Entering column smallest j w/ cj is negative.
• Exiting column out of all ties, choose the
  smallest l.
• first row is selected.

24
Finding initial bfs

• min cx Ax b, x ? 0
• using artificial variables y, make auxiliary
  problemmin ?yi Ax y b, x ? 0, y ? 0
• Two-phase method
• Use simplex twice
• First solve the auxiliary problem
• Second solve the original problem
• The big-M method
• combine two phases by changing cost function
  as?nj1 cjxj M ?mi1yi, M large positive
  integer

25
Example

• min x1 x2 x3 x1 2x2 3x3
  3 -x1 2x2 6x3 2
  4x2 9x3 5
  3x3 x4 1 x1, ..., x4 ? 0
• min x1 x2 x3 Mx5 Mx6 Mx7 x1
  2x2 3x3 x5
  3 -x1 2x2 6x3 x6
  2 4x2 9x3
  x7 5 3x3
  x4 1 x1, ..., x7
  ? 0

26
Example

• min x1 x2 x3 Mx5 Mx6 Mx7 x1
  2x2 3x3 x5
  3 -x1 2x2 6x3 x6
  2 4x2 9x3
  x7 5 3x3
  x4 1 x1, ..., x7
  ? 0
  x (0, 0, 0, 1, 3, 2, 5)

27
Example

• min x1 x2 x3 Mx5 Mx6 Mx7 x1
  2x2 3x3 x5
  3 -x1 2x2 6x3 x6
  2 4x2 9x3
  x7 5 3x3
  x4 1 x1, ..., x7
  ? 0
  x (0, 0, 1/3, 0, 2, 0, 2)

28
Example

• min x1 x2 x3 Mx5 Mx6 Mx7 x1
  2x2 3x3 x5
  3 -x1 2x2 6x3 x6
  2 4x2 9x3
  x7 5 3x3
  x4 1 x1, ..., x7
  ? 0
  x (0, 0, 1/3, 0, 2, 0, 2)

29
Example

• min x1 x2 x3 Mx5 Mx6 Mx7 x1
  2x2 3x3 x5
  3 -x1 2x2 6x3 x6
  2 4x2 9x3
  x7 5 3x3
  x4 1 x1, ..., x7
  ? 0
  x (1, 1/2, 1/3, 0, 0, 0, 0)

30
Example

• min x1 x2 x3 Mx5 Mx6 Mx7 x1
  2x2 3x3 x5
  3 -x1 2x2 6x3 x6
  2 4x2 9x3
  x7 5 3x3
  x4 1 x1, ..., x7
  ? 0
  x (1/2, 5/4, 0, 1, 0, 0, 0)

31
Today

• matroids again
• greedy algorithm can solve problems in matroids
• Simplex algorithm for LP
• Full tableau method
• anticycling
• lexicography and Blands rule
• finding initial bfs
• two-phase method
• big-M method