Solving LP Models

1
Solving LP Models

• Improving Search
• Unimodal
• Convex feasible region
• Should be successful!
• Special Form of Improving Search
• Simplex method (now)
• Interior point methods (later)

2
Simple Example

• Top Brass Trophy Company
• Makes trophies for
• football
• wood base, engraved plaque, brass football on top
• 12 profit and uses 4 of wood
• soccer
• wood base, engraved plaque, soccer ball on top
• 9 profit and uses 2 of wood
• Current stock
• 1000 footballs, 1500 soccer balls, 1750 plaques,
  and 4800 feet of wood

3
Formulation
4
Graphical Solution
Optimal Solution
5
Feasible Solutions

• Feasible solution is a
• boundary point if at least one inequality
  constraint that can be strict is active
• interior point if no such constraints are active
• Extreme points of convex sets do not lie within
  the line segment of any other points in the set

6
Example
7
Optimal Solutions

• Every optimal solution is a boundary point
• We can find an improving direction whenever we
  are at an interior point
• If optimum unique the it must be an extreme point
  of the feasible region
• If optimal solution exist, an optimal extreme
  point exists

8
LP Standard Form

• Easier if we agree on exactly what a LP should
  look like
• Standard form
• only equality main constraints
• only nonnegative variables
• variables appear at most once in left-hand-side
  and objective function
• all constants appear on right hand side

9
Converting to Standard

• Inequality constraints
• Add nonnegative, zero-cost slack variables
• Add in ? inequalities
• Subtract in ? inequalities
• Variables not nonnegative
• nonpositive - substitute with negatives
• unrestrictive sign (URS) - substitute difference
  of two nonnegative variables

10
Top Brass Model
11
Why?

• Feasible directions
• Check only if active
• Keep track of active constraints
• Equality constraints
• Always active
• Inequality constraints
• May or may not be active
• Prefer equality constraints!

12
Standard Notation
13
LP Standard Form

• In standard notation
• In matrix notation

14
Write in Matrix Form
15
Extreme Points

• Know that an extreme point optimum exists
• Will search trough extreme points
• An extreme point is define by a set of
  constraints that are active simultaneously

16
Improving Search

• Move from one extreme point to a neighboring
  extreme point
• Extreme points are adjacent if they are defined
  by sets of active constraints that differ by only
  one element
• An edge is a line segment determined by a set of
  active constraints

17
Basic Solutions

• Extreme points are defined by set of active
  nonnegativity constraints
• A basic solution is a solution that is obtained
  by fixing enough variable to be equal to zero, so
  that the equality constraints have a unique
  solution

18
Example
Choose x1, x2, x3, x4 to be basic
19
Where is the Basic Solution?
20
Example

• Compute the basic solution for x1 and x2 basis
• Solve

21
Existence of Basis Solutions

• Remember linear algebra?
• A basis solution exists if and only if the
  columns of corresponding equality constraint form
  a basis
• (in other words, a largest possible linearly
  independent collection)

22
Checking

• The determinant of a square matrix D is
• A matrix is singular if its determinant 0 and
  otherwise nonsingular
• Need to check that the matrix is nonsingular

23
Example

• Check whether basic solutions exist for

24
Basic Feasible Solutions

• A basic feasible solution to a LP is a basic
  solution that satisfies all the nonnegativity
  contraints
• The basic feasible solutions correspond exactly
  to the extreme points of the feasible region

25
Example Problem

• Suppose we have x3, x4, x5 as slack variables in
  the following LP
• Lets plot the original problem, compute the basic
  solutions and check feasibility

26
Solution Algorithm

• Simplex Algorithm
• Variant of improving search
• Standard display

27
Simplex Algorithm

• Starting point
• A basic feasible solution (extreme point)
• Direction
• Follow an edge to adjacent extreme point
• Increase one nonbasic variable
• Compute changes needed to preserve equality
  constraints
• One direction for each nonbasic variable

28
Top Brass Example
Basic variables
Initial solution
29
Looking in All Directions
Can increase either one of those
Must adjust these!
30
So Many Choices ...

• Want to try to improve the objective
• The reduced cost of a nonbasic variable
• Want

Defines improving direction
31
Top Brass Example

• Improving x1 gives
• Improving x2 gives
• Both directions are improving directions!

32
Where and How Far?

• Any improving direction will do
• If no component is negative
• ? Improve forever - unbounded!
• Otherwise, compute the minimum ratio

33
Computing Minimum Ratio
34
Moving to New Solution
35
Updating Basis

• New basic variable
• Nonbasic variable generating direction
• New nonbasic variable(s)
• Basic variables fixing the step size

36
What Did We Do?
37
Where Will We Go?
Optimum in three steps!
Why is this guaranteed?
38
Simplex Algorithm (Simple)

• Step 0 Initialization. Choose starting feasible
  basis, construct basic solution x(0), and set t0
• Step 1 Simplex Directions. Construct directions
  Dx associated with increasing each nonbasic
  variable xj and compute the reduced cost cj c
  Dx.
• Step 2 Optimality. If no direction is
  improving, then stop otherwise choose any
  direction Dx(t1) corresponding to some basic
  variable xp.
• Step 3 Step Size. If no limit on move in
  direction Dx(t1) then stop otherwise choose
  variable xr such that
• Step 4 New Point and Basis. Compute the new
  solution
• and replace xr in the basis with xp. Let t
  t1 and go to Step 1.

39
Stopping

• The algorithm stop when one of two criteria is
  met
• In Step 2 if no improving direction exists, which
  implies local optimum, which implied global
  optimum
• In Step 3 if no limit on improvement, which
  implies problem is unbounded

40
Optimization Software

• Spreadsheet (e.g, MS Excel with Whats Best!)
• Optimizers (e.g., LINDO)
• Combination
• Modeling Language
• Solvers
• Either together (e.g., LINGO) or separate (e.g.,
  GAMS with CPLEX)
• LINDO and LINGO are in Room 0010 (OR Lab)
• Also on disk with your book

41
LINDO

• The main software that Ill ask you to use is
  called LINDO
• Solves linear programs (LP), integer programs
  (IP), and quadratic programs (QP)
• We will look at many of its more advanced
  features later on, but as of yet we havent
  learned many of the concepts that we need

42
Example
43
LINDO Program
MAX 12 x1 9 x2 ST x1 x2 1000 x2 x4
1500 x1 x2 x5 1750 4x1 2x2 x6
4800 x1gt0 x2gt0 x3gt0 x4gt0 x5gt0 x6gt0 END
44
(No Transcript)
45
(No Transcript)
46
Output
LP OPTIMUM FOUND AT STEP 4 OBJECTIVE
FUNCTION VALUE 1) 12000.00
VARIABLE VALUE REDUCED COST
X1 1000.000000 0.000000
X2 0.000000 3.000000 X4
1500.000000 0.000000 X5
750.000000 0.000000 X6
800.000000 0.000000 X3
0.000000 0.000000
47
Output (cont.)
ROW SLACK OR SURPLUS DUAL PRICES
2) 0.000000 12.000000
3) 0.000000 0.000000 4)
0.000000 0.000000 5)
0.000000 0.000000 6)
1000.000000 0.000000 7)
0.000000 0.000000 8)
0.000000 0.000000 9)
1500.000000 0.000000 10)
750.000000 0.000000 11)
800.000000 0.000000 NO. ITERATIONS
4
48
LINDO Basic Syntax

• Objective Function Syntax Start all models with
  MAX or MIN
• Variable Names Limited to 8 characters
• Constraint Name Terminated with a parenthesis
• Recognized Operators (, -, gt, lt, )
• Order of Precedence Parentheses not recognized

49
Syntax (cont.)

• Adding Comment Start with an exclamation mark
• Splitting lines in a model Permitted in LINDO
• Case Sensitivity LINDO has none
• Right-hand Side Syntax Only constant values
• Left-hand Side Syntax Only variables and their
  coefficients

50
Why Modeling Language?

• More to learn!
• More complicated to use than LINDO (at least at
  first glance)
• Advantages
• Natural representations
• Similar to mathematical notation
• Can enter many terms simultaneously
• Much faster and easier to read

51
Why Solvers?

• Best commercial software has modeling language
  and solvers separated
• Advantages
• Select solver that is best for your application
• Learn one modeling language use any solver
• Buy 3rd party solvers or write your own!

52
Example Problem
53
Problem Formulation
54
LINDO Solution
max 1.60 x1 1.40 x2 1.90 x3 1.20 x4 st
x1 x2 x3 x4 lt1200 x1 gt 310 x1
lt 434 x2 gt 245 x2 lt 343 x3 gt 255
x3 lt 357 x4 gt 190 x5 lt 266 end
55
LINGO Solution

• Capacity constraint
• _at_SUM(REGIONS(I) CASES(I))
• lt1200
• Minimum/maximum cases
• _at_FOR(REGIONS(I)
• CASES(I) lt UBOUND
• CASES(I) gt LBOUND)

56
LINGO Solution

• Objective function
• MAX _at_SUM(REGIONS(I)
• PROFITCASES(I))
• We also need to define REGIONS, CASES, etc, and
  type in the data.

57
LINGO Solution

• Defining sets
• SETS
• REGIONS / NE SE MW W/ LBOUND,
• UBOUND, PROFIT, CASES
• ENDSETS

58
LINGO Solution

• Enter the data
• DATA
• LBOUND 310 245 255 190
• UBOUND 434 343 357 266
• PROFIT 1.6 1.4 1.9 1.2
• ENDDATA

59
Sensitivity Analysis

• Basic Question How does our solution change as
  the input parameters change?
• The objective function?
• More/less profit or cost
• The optimal values of decision variables?
• Make different decisions!
• Why?
• Only have estimates of input parameters
• May want to change input parameters

60
What We Know

• Qualitative Answers for All Problems
• Quantitative Answers for Linear Programs (LP)
• Dual program
• Same input parameters
• Decision variables give sensitivities
• Dual prices
• Easy to set up
• Theory is somewhat complicated

61
Back to Example Problem
62
LINDO Formulation
max 1.60 x1 1.40 x2 1.90 x3 1.20 x4 st
x1 x2 x3 x4 lt1200 x1 gt 310 x1 lt
434 x2 gt 245 x2 lt 343 x3 gt 255
x3 lt 357 x4 gt 190 x5 lt 266 end
63
LINDO Solution (second half)
ROW SLACK OR SURPLUS DUAL
PRICES 2) 0.000000
1.600000 3) 98.000000
0.000000 4) 26.000000
0.000000 5) 0.000000
-0.200000 6) 98.000000
0.000000 7) 102.000000
0.000000 8) 0.000000
0.300000 9) 0.000000
-0.400000 10) 266.000000
0.000000
64
Dual Prices

• The Dual is Automatically Formed
• Also in LINGO
• Also in (all) other optimization software
• Report dual prices
• Gives us sensitivities to RHS parameter
• Know how much objective function will change
• When will the optimal solution change?
• Need to select that we want sensitivity analysis

65
LINDO Sensitivity Analysis (part)
RANGES IN WHICH THE BASIS IS UNCHANGED
OBJ COEFFICIENT RANGES
VARIABLE CURRENT ALLOWABLE
ALLOWABLE COEF
INCREASE DECREASE X1
1.600000 0.300000 0.200000
X2 1.400000 0.200000
INFINITY X3 1.900000
INFINITY 0.300000 X4
1.200000 0.400000 INFINITY
X5 0.000000 0.000000
INFINITY
66
Interpretation

• As long as prices for the NE region are between
  1.4 and 1.9, we want to sell the same quantity
  to each region, etc.

67
Example
An insurance company is introducing two new
product lines special risk insurance and
mortgages. The expected profit is 5 per unit on
special risk insurance and 2 per unit on
mortgages. Management wishes to establish a
sales target for the new product lines to
maximize the expected profit. The work
requirements are as follows
68
LINDO Formulation
max 5 x1 2 x2 st 3 x1 2 x2 lt 2400 x2 lt
800 2 x1 lt 1200 x1 gt0 x2 gt0 end
69
Graphical Solution
70
Solution
VARIABLE VALUE REDUCED COST
X1 600.000000 0.000000
X2 300.000000 0.000000 ROW
SLACK OR SURPLUS DUAL PRICES 2)
0.000000 1.000000 3)
500.000000 0.000000 4)
0.000000 1.000000 5)
600.000000 0.000000 6)
300.000000 0.000000
71
Sensitivity Analysis
RANGES IN WHICH THE BASIS IS UNCHANGED
OBJ COEFFICIENT RANGES
VARIABLE CURRENT ALLOWABLE
ALLOWABLE COEF
INCREASE DECREASE X1
5.000000 INFINITY 2.000000
X2 2.000000 1.333333
2.000000 RIGHTHAND
SIDE RANGES ROW CURRENT
ALLOWABLE ALLOWABLE
RHS INCREASE DECREASE 2
2400.000000 1000.000000 600.000000
3 800.000000 INFINITY
500.000000 4 1200.000000
400.000000 666.666687 5
0.000000 600.000000 INFINITY
6 0.000000 300.000000
INFINITY
72
New Decisions!
800 700 600 500 400 300 200 100
Optimum Moves!
73
What-If ? Solve New Problem
max 5 x1 2 x2 st 3 x1 2 x2 lt 2400 x2 lt
290 2 x1 lt 1200 x1 gt0 x2 gt0 end
74
New Solution
VARIABLE VALUE REDUCED COST
X1 600.000000 0.000000
X2 290.000000 0.000000 ROW
SLACK OR SURPLUS DUAL PRICES 2)
20.000000 0.000000 3)
0.000000 2.000000 4)
0.000000 2.500000 5)
600.000000 0.000000 6)
290.000000 0.000000
75
Interior Point Methods

• Simplex always stays on the boundary
• Can take short cuts across the interior
• Interior point methods
• More effort in each move
• More improvement in each move
• Much faster for large problems

76
Simple Example

• Frannies Firewood sells up to 3 cords of
  firewood to two customers
• One will pay 90 per half-cord
• Other will pay 150 per full cord

77
Graphical Solution
78
Improving Directions

• Which direction improves the objective function
  the most?
• The gradient
• Direction

79
Most Improving Direction?
80
Back to Example
81
Maintaining Feasibility

• At the initial point all directions are feasible
  because it is an interior point
• At the new point we have to make sure that a
  direction Dx at x(1) satisfies
• Interior point algorithms begin inside and move
  through the interior, reaching the boundary only
  at an optimal solution

82
Valid Interior Point Search?
83
Valid Interior Point Search?
84
Valid Interior Point Search?
85
LP Standard Form

• For Simplex used the form
• In Frannies Firewood

86
Benefits of Standard Form?

• In Simplex
• Made easy to check which variables are basic,
  non-basic, etc.
• Needed to know which solutions are on boundary
• Here quite similar
• Know which are not on boundary
• Check that nonnegativity constraints are strict!
• A feasible solution to standard LP is interior
  point if every component is strictly positive

87
Interior Points?
88
Projections

• Must satisfy main equality constraints
• Want direction Dx that satisfies this equation
  and is as nearly d as possible
• The projection of a vector d onto a system of
  equalities is the vector that satisfies the
  constraints and minimizes the total squared
  difference between the components

89
Obtaining Projection

• The projection of d onto ADx0 is
• where
• is the projection matrix.

90
Example Frannies Firewood
91
Example

• The cost vector is c(90 150 0)

92
Improvement

• The projection matrix is design to make an
  improving direction feasible with minimum changes
• Is it still an improving direction? Yes!
• The projection DxPc of c onto Axb is an
  improving direction at every x

93
Sample Exercise

• Determine the direction d of most rapid
  improvement
• Project it onto the main equality constraints to
  get Dx
• Verify that the move direction Dx is feasible
• Verify that the move direction Dx is improving