Linear Programming

1
Linear Programming

• Piyush Kumar

2
Graphing 2-Dimensional LPs
Optimal Solution
y
Example 1
4
Maximize x y
3
x 2 y ³ 2
Subject to
Feasible Region
x 3
2
y 4
1
x ³ 0 y ³ 0
0
x
3
0
1
2
These LP animations were created by Keely
Crowston.
3
Graphing 2-Dimensional LPs
Multiple Optimal Solutions!
y
Example 2
4
Minimize x - y
3
1/3 x y 4
Subject to
-2 x 2 y 4
2
Feasible Region
x 3
1
x ³ 0 y ³ 0
0
x
3
1
2
0
4
Graphing 2-Dimensional LPs
y
Example 3
40
Minimize x 1/3 y
30
x y ³ 20
Subject to
Feasible Region
-2 x 5 y 150
20
x ³ 5
10
x ³ 0 y ³ 0
x
0
Optimal Solution
30
10
20
0
40
5
Do We Notice Anything From These 3 Examples?
Extreme point
y
y
y
4
40
4
3
3
30
2
2
20
1
1
10
0
0
0
x
x
x
3
0
1
2
30
10
20
0
40
3
0
1
2
6
A Fundamental Point
y
y
y
4
40
4
3
3
30
2
2
20
1
1
10
0
0
0
x
x
x
0
1
10
20
0
0
1
2
3
2
30
40
3
If an optimal solution exists, there is always a
corner point optimal solution!
7
Graphing 2-Dimensional LPs
Optimal Solution
Second Corner pt.
y
Example 1
4
Maximize x y
3
x 2 y ³ 2
Subject to
Feasible Region
x 3
2
y 4
1
x ³ 0 y ³ 0
Initial Corner pt.
0
x
3
0
1
2
8
And We Can Extend this to Higher Dimensions
9
Then How Might We Solve an LP?

• The constraints of an LP give rise to a
  geometrical shape - we call it a polyhedron.
• If we can determine all the corner points of the
  polyhedron, then we can calculate the objective
  value at these points and take the best one as
  our optimal solution.
• The Simplex Method intelligently moves from
  corner to corner until it can prove that it has
  found the optimal solution.

10
Linear Programs in higher dimensions
maximize z -4x1 x2 -
x3 subject to -7x1
5x2 x3 lt 8
-2x1 4x2 2x3 lt 10
x1, x2, x3
? 0
11
In Matrix terms
12
LP Geometry

• Forms a d dimensional polyhedron
• Is convex If z1 and z2 are two feasible
  solutions then ?z1 (1- ?)z2 is also feasible.
• Extreme points can not be written as a convex
  combination of two feasible points.

13
LP Geometry

• Extreme point theorem If there exists an
  optimal solution to an LP Problem, then there
  exists one extreme point where the optimum is
  achieved.
• Local optimum Global Optimum

14
LP Algorithms

• Simplex. (Dantzig 1947)
• Developed shortly after WWII in response to
  logistical problemsused for 1948 Berlin
  airlift.
• Practical solution method that moves from one
  extreme point to a neighboring extreme point.
• Finite (exponential) complexity, but no
  polynomial implementation known.

Courtesy Kevin Wayne
15
LP Polynomial Algorithms

• Ellipsoid. (Khachian 1979, 1980)
• Solvable in polynomial time O(n4 L) bit
  operations.
• n variables
• L bits in input
• Theoretical tour de force.
• Not remotely practical.
• Karmarkar's algorithm. (Karmarkar 1984)
• O(n3.5 L).
• Polynomial and reasonably efficientimplementation
  s possible.
• Interior point algorithms.
• O(n3 L).
• Competitive with simplex!
• Dominates on simplex for large problems.
• Extends to even more general problems.

16
LP The 2D case
Wlog, we can assume that c(0,-1). So now we
want to find the Extreme point with the smallest
y coordinate.
Lets also assume, no degeneracies, the solution
is given by two Halfplanes intersecting at a
point.
17
Incremental Algorithm?

• How would it work to solve a 2D LP Problem?
• How much time would it take in the worst case?
• Can we do better?