Linear Programming

1
Linear Programming

• Lecture 33
• Linear Programming
• Review Assignment problem and how it can help TSP
• The Simplex Method for solving general Linear
  Programs

• Lecture 32 More TSP
• Another BB Problem Formulation
• State Space
• Cost function
• Bounds
• Lower Bounds
• Upper Bounds
• Example

2
Announcements

• Cryptography Lab is now available on blackboard
  for extra credit.
• 5 points extra credit for participating in course
  evaluation. You must turn in both the on-line
  and paper forms, and you must enter your name
  when you go on-line to get the extra credit. I
  will get a list of names of participants, but
  wont know which review was yours.
• Sign up for final presentations Dec. 5th, 8th,
  and 10th. Your attendance and participation on
  those days is requiredyou need to support each
  other.
• We will expand the Branch-and-Cut extra credit to
  include any LP Branch-and-Bound technique that
  uses a general LP solution technique (such as the
  simplex algorithm, an ellipsoidal method, or an
  interior point method). We will show you one
  approach today in class. A flash tutorial on LP
  is available at http//commerce.concordia.ca/bourj
  olly/lp.html, and some introductions to the
  simplex method for solving LPs are available
  from us.

3
Another Formulation

• Consider the Assignment Problem

4
Another TSP Formulation

• Consider the Assignment Problem

55
5
Another TSP Formulation

• Consider the Assignment Problem

59
6
Another TSP Formulation

• Consider the Assignment Problem

58
7
Another TSP Formulation

• Consider the Assignment Problem

40
8
Another TSP Formulation

• Consider the Assignment Problem

A solution is generally a collection of disjoint
cycles.
40
3
4
1
2
9
Another TSP Formulation

• Consider the Assignment Problem

A solution is generally a collection of disjoint
cycles. But if solution happens to be one cycle,
it is an optimal tour.
40
3
4
1
2
10
Another TSP Formulation

• Consider the Assignment Problem

A solution is generally a collection of disjoint
cycles. But if solution happens to be one cycle,
it is an optimal tour. The Assignment Problem is
a special Integer Programthe solution to the LP
relaxation is the solution to the IP!
40
We can compute it for the subgraph at each node
and use the solution for TSP
11
Another TSP Formulation

• Consider the Assignment Problem

Not only does the value of the assignment problem
provide a lower bound to the optimal tour cost,
but it can define a useful state space. At each
node of the state space we identify a graph
H(V,A). The root node is associated with the
original graph.
40
12
Another TSP Formulation

• Consider the Assignment Problem

Not only does the value of the assignment problem
provide a lower bound to the optimal tour cost,
but it can define a useful state space. At each
node of the state space we identify a graph
H(V,A). The root node is associated with the
original graph.
To find the children nodes, we construct a set of
subgraphs by solving the assignment problem,
choosing one of the resulting cylces (suppose the
cycle contains r nodes), then building a graph
for each of the r nodes. These r subgraphs
define the children nodes in the state space.
13
Another TSP Formulation

• Consider the Assignment Problem

Let Ww1, w2, , wr be the nodes of the cycle.
Define Now define the edge sets
To find the children nodes, we construct a set of
subgraphs by solving the assignment problem,
choosing one of the resulting cylces (suppose the
cycle contains r nodes), then building a graph
for each of the r nodes. These r subgraphs
define the children nodes in the state space.
14
Another TSP Formulation

• Consider the Assignment Problem

Let Ww1, w2, , wr be the nodes of the cycle.
Define Now define the edge sets
The r children of the current node correspond to
the graphs (V, Ei), 1ltiltr. Compute the
assignment problem for each child, expand the
child with lowest score.
15
Another TSP Formulation

• Consider the Assignment Problem

16
Another TSP Formulation

• Consider the Assignment Problem

21
17
Another TSP Formulation

• Consider the Assignment Problem

1
9
8
21
2
10
6
4
3
12
7
(1,4), (4,3), (3,2), (2,5), (5,1)
4
Assignment is a single cycle, so this method
finds the optimal tour for this instance in one
shot without expanding any children !
18
Linear Programming

• We use graphs and trees as useful modeling
  abstractions to help us develop computational
  solutions for a wide variety of problems
• A linear program is simply another modeling
  abstraction (tool in your toolbox)
• Developing routines that solve general linear
  programs allows us to embed them in sophisticated
  algorithmic solutions to difficult problems (e.g.
  like we did for TSP)
• The cutting edge algorithmic solutions to many
  problems use the ideas from mathematical
  programming, linear programming forming the
  foundation

19
Example A Linear Program
20
Example A Linear Program
y
20
0
x
21
Example A Linear Program
y
20
15
(5,5)
0
x
22
Example A Linear Program
y
20
15
(5,5)
0
x
23
Example A Linear Program
y
20
15
(5,5)
0
x
24
Example A Linear Program
y
20
15
0
x
25
Example A Linear Program
y
20
15
(5,5)
0
x
26
Example A Linear Program
The linear inequalities form a simplex. The
linear cost function creates a hyperplane. The
solution to a general LP is either a unique
vertex or a face of the simplex, including more
than one vertex.
y
20
15
(5,5)
0
x
The simplex method is a greedy algorithm that
moves from one vertex of the feasible set to
another until an optimum is attained.
27
The Simplex Method
Transform Linear Program into a system of linear
equations using slack variables
28
The Simplex Method
x y s1 s2 P RHS
29
The Simplex Method
Basic Idea Start from a vertex (x0, y0)
Move to next vertex that increases
profit as much as possible
x y s1 s2 P RHS
30
The Simplex Method
Basic Idea Start from a vertex (x0, y0)
Move to next vertex that increases
profit as much as possible
x y s1 s2 P RHS

• At (0,0), P 0

31
The Simplex Method
Basic Idea Start from a vertex (x0, y0)
Move to next vertex that increases
profit as much as possible
x y s1 s2 P RHS

• At (0,0), P 0
• Increasing x can increase P
• the most (x coefficient has
• larger magnitude than the
• y coefficient)

32
The Simplex Method
Basic Idea Start from a vertex (x0, y0)
Move to next vertex that increases
profit as much as possible
x y s1 s2 P RHS

• At (0,0), P 0
• Increasing x can increase P
• the most (x coefficient has
• larger magnitude than the
• y coefficient)
• Compute check ratios to find
• pivot row (smallest ratio)

33
The Simplex Method
Basic Idea Start from a vertex (x0, y0)
Move to next vertex that increases
profit as much as possible
x y s1 s2 P RHS

• At (0,0), P 0
• Increasing x can increase P
• the most (x coefficient has
• larger magnitude than the
• y coefficient)
• Compute check ratios to find
• pivot row (smallest ratio)

34
The Simplex Method
Basic Idea Start from a vertex (x0, y0)
Move to next vertex that increases
profit as much as possible
x y s1 s2 P RHS

• At (0,0), P 0
• Increasing x can increase P
• the most (x coefficient has
• larger magnitude than the
• y coefficient)
• Compute check ratios to find
• pivot row (smallest ratio)

35
The Simplex Method
Basic Idea Start from a vertex (x0, y0)
Move to next vertex that increases
profit as much as possible
x y s1 s2 P RHS

• At (0,0), P 0
• Increasing x can increase P
• the most (x coefficient has
• larger magnitude than the
• y coefficient)
• Compute check ratios to find
• pivot row (smallest ratio)
• Pivot around the element in
• both pivot column and row

Pivoting means solve for that variable, Then
substitute into the other equations.
36
The Simplex Method
Basic Idea Start from a vertex (x0, y0)
Move to next vertex that increases
profit as much as possible
x y s1 s2 P RHS

• At (0,0), P 0
• Increasing x can increase P
• the most (x coefficient has
• larger magnitude than the
• y coefficient)
• Compute check ratios to find
• pivot row (smallest ratio)
• Pivot around the element in
• both pivot column and row

Pivoting means solve for that variable, Then
substitute into the other equations.
37
The Simplex Method
Basic Idea Start from a vertex (x0, y0)
Move to next vertex that increases
profit as much as possible
x y s1 s2 P
RHS

• At (0,0), P 0
• Increasing x can increase P
• the most (x coefficient has
• larger magnitude than the
• y coefficient)
• Compute check ratios to find
• pivot row (smallest ratio)
• Pivot around the element in
• both pivot column and row

Pivoting means solve for that variable, Then
substitute into the other equations.
38
The Simplex Method
Basic Idea Start from a vertex (x0, y0)
Move to next vertex that increases
profit as much as possible
x y s1 s2
P RHS

• At (0,0), P 0
• Increasing x can increase P
• the most (x coefficient has
• larger magnitude than the
• y coefficient)
• Compute check ratios to find
• pivot row (smallest ratio)
• Pivot around the element in
• both pivot column and row

Pivoting means solve for that variable, Then
substitute into the other equations.
39
The Simplex Method
Basic Idea Start from a vertex (x0, y0)
Move to next vertex that increases
profit as much as possible
x y s1 s2
P RHS

• At (0,0), P 0
• Increasing x can increase P
• the most (x coefficient has
• larger magnitude than the
• y coefficient)
• Compute check ratios to find
• pivot row (smallest ratio)
• Pivot around the element in
• both pivot column and row

We have moved from (0,0) to (20/3,0) and
increased the profit from 0 to 400/3. Can we do
better?
40
The Simplex Method
Basic Idea Start from a vertex (x0, y0)
Move to next vertex that increases
profit as much as possible
x y s1 s2
P RHS

• At (0,0), P 0
• Increasing x can increase P
• the most (x coefficient has
• larger magnitude than the
• y coefficient)
• Compute check ratios to find
• pivot row (smallest ratio)
• Pivot around the element in
• both pivot column and row

We have moved from (0,0) to (20/3,0) and
increased the profit from 0 to 400/3. Can we do
better? Yes! Repeat the procedure.
41
The Simplex Method
Basic Idea Start from a vertex (x0, y0)
Move to next vertex that increases
profit as much as possible
x y s1 s2
P RHS

• At (0,0), P 0
• Increasing y can increase P
• the most (s1 can only hurt
• the total profit P).

We have moved from (0,0) to (20/3,0) and
increased the profit from 0 to 400/3. Can we do
better? Yes! Repeat the procedure.
42
The Simplex Method
Basic Idea Start from a vertex (x0, y0)
Move to next vertex that increases
profit as much as possible
x y s1 s2
P RHS
20

• At (0,0), P 0
• Increasing y can increase P
• the most (s1 can only hurt
• the total profit P).
• Compute check ratios to find
• pivot row (smallest ratio)

5
43
The Simplex Method
Basic Idea Start from a vertex (x0, y0)
Move to next vertex that increases
profit as much as possible
x y s1 s2
P RHS

• At (0,0), P 0
• Increasing y can increase P
• the most (s1 can only hurt
• the total profit P).
• Compute check ratios to find
• pivot row (smallest ratio)

5
44
The Simplex Method
Basic Idea Start from a vertex (x0, y0)
Move to next vertex that increases
profit as much as possible
x y s1 s2
P RHS

• At (0,0), P 0
• Increasing y can increase P
• the most (s1 can only hurt
• the total profit P).
• Compute check ratios to find
• pivot row (smallest ratio)
• Pivot around the element in
• both pivot column and row

5
Pivoting means solve for that variable, Then
substitute into the other equations.
45
The Simplex Method
Basic Idea Start from a vertex (x0, y0)
Move to next vertex that increases
profit as much as possible
x y s1 s2
P RHS

• At (0,0), P 0
• Increasing y can increase P
• the most (s1 can only hurt
• the total profit P).
• Compute check ratios to find
• pivot row (smallest ratio)
• Pivot around the element in
• both pivot column and row

Pivoting means solve for that variable, Then
substitute into the other equations.
46
The Simplex Method
Basic Idea Start from a vertex (x0, y0)
Move to next vertex that increases
profit as much as possible
x y s1 s2 P RHS

• At (0,0), P 0
• Increasing y can increase P
• the most (s1 can only hurt
• the total profit P).
• Compute check ratios to find
• pivot row (smallest ratio)
• Pivot around the element in
• both pivot column and row

Pivoting means solve for that variable, Then
substitute into the other equations.
47
The Simplex Method
Basic Idea Start from a vertex (x0, y0)
Move to next vertex that increases
profit as much as possible
x y s1 s2 P RHS

• At (0,0), P 0
• Increasing y can increase P
• the most (s1 can only hurt
• the total profit P).
• Compute check ratios to find
• pivot row (smallest ratio)
• Pivot around the element in
• both pivot column and row

We have moved from (0,0) to (20/3,0) and
increased the profit from 0 to 400/3. We then
moved from (20/3,0) to (5,5), Increasing the
profit from 400/3 to 145. Can we do better?
48
The Simplex Method
Basic Idea Start from a vertex (x0, y0)
Move to next vertex that increases
profit as much as possible
x y s1 s2 P RHS

• At (0,0), P 0
• Increasing y can increase P
• the most (s1 can only hurt
• the total profit P).
• Compute check ratios to find
• pivot row (smallest ratio)
• Pivot around the element in
• both pivot column and row

We have moved from (0,0) to (20/3,0) and
increased the profit from 0 to 400/3. We then
moved from (20/3,0) to (5,5), Increasing the
profit from 400/3 to 145. Can we do better? No!
Increasing s1 or s2 will only make things
worse. We are optimal.
49
The Simplex Method
Basic Idea Start from a vertex (x0, y0)
Move to next vertex that increases
profit as much as possible
x y s1 s2 P RHS
y
20
15
We have moved from (0,0) to (20/3,0) and
increased the profit from 0 to 400/3. We then
moved from (20/3,0) to (5,5), Increasing the
profit from 400/3 to 145. Can we do better? No!
Increasing s1 or s2 will only make things
worse. We are optimal.
(5,5)
0
x
50
Weve Come Full Circle