LINEAR PROGRAMMING

1
LINEAR PROGRAMMING
These visual aids will assist in class
participation for the section on Linear
Programming.

• Motivation
• Learning Objectives
• Basic LP assumptions and limitations
• Concepts and geometric interpretations
• The Simplex Algorithm
• Post-optimal analysis - Sensitivity
• Post-optimal analysis - Weird Events
• Useful LP model formulations
• Conclusions
• Workshops

2
Motivation
LINEAR PROGRAMMING
Multiple choice question Why study Linear
Programming? 1. The professor cannot formulate
N-L models 2. Linear programming is the most
frequently used optimization method. 3. Linear
programming was developed in the
1940s 4. Formulation and solution knowledge will
also help when addressing non-linear
models 5. Excellent software is available
3
Motivation
OPTIMIZATION is Needed to find the Best Solution
Motivating example
The cost of manufacturing is the same.
Therefore, the objective is to minimize the
transportation cost, using values in the figure.
Class exercise What is the best plan?
4
Motivation
plant
warehouse
customer
Sequential Analysis
Heuristic 1
In the first step, we rank the costs of
satisfying the customer demands from the
warehouses, from which we select the flows that
give the lowest cost alternates. From W2 to
C1 50,000 units From W2 to C2 100,000
units From W2 to C3 50,000 units In the
second step, we determine the flows that give the
minimum cost for the plant to warehouse, which
are given below. From P2 to W2 60,000
units (P2 ? 60,000) From P1 to W2 140,000
units This result satisfies all strict customer
requirements and does not exceed the capacity of
plant 2 we will call this a feasible solution.
The total cost is 1,200,000. Is this good is
this the best? Without optimization, we do not
know.
5
Motivation
plant
warehouse
customer
Analysis with lookahead
Heuristic 2
For C1 P1-W1-C1 For C2 P2-W2-C2 For
C3 P2-W2-C3 Observing these paths gives the
following flows from the warehouses. From W1 to
C1 50,000 units From W2 to C2 100,000
units From W2 to C3 50,000 units Second, we
select the lowest cost for plant-warehouse flows
to satisfy the above decisions, which are given
in the following. From P1 to W1 50,000
units From P2 to W2 60,000 units (P2 ?
60,000) From P1 to W2 90,000 units The total
cost is 920,000, which is less than the first
solution. Is this good is this the best?
Without optimization, we do not know.
6
Motivation
OPTIMIZATION is Needed to find the Best Solution
Motivating example
Linear programming will find the best - optimal -
plan with less than 1 second of computing. The
lowest cost is 740,000. Big improvement over
heuristics, little computing - lets learn LP!
7
Learning goals
LEARNING GOALS

• Attitudes
• An optimal solution is much better than an
  answer.
• Numbers without understanding are useless

• Knowledge
• Based of fundamental
• concepts, we will learn
• formulation for LP
• results analysis,
• including diagnosing
• weird events
• sensitivity analysis

• Skills
• Translate a
• complex problem
• into a mathematical
• formulation
• Communicate
• optimization results
• in engineering terms

8
Learning goals
LEARNING GOALS
Challenge Understand both the geometric
interpretation (excellent understanding) and the
matrix calculations (for the quantitative result)
- At the same time!
It is worth the effort!
9
LP assumptions limitations
LINEAR PROGRAMMING
Models must conform to these restrictions

• Linearity
• Divisibility
• Certainty

10
LP assumptions limitations
LINEAR PROGRAMMING
Models must conform to these restrictions
FA0 flow rate of A

• Linearity
• Divisibility
• Certainty

FA flow rate of A FB flow rate of B FC
flow rate of C FD flow rate of D
How do we formulate a linear model of this
complex process?
11
LP assumptions limitations
LINEAR PROGRAMMING
Models must conform to these restrictions

• Linearity
• Divisibility
• Certainty

FA flow rate of A FB flow rate of B FC
flow rate of C FD flow rate of D

• The alpha is for a specific reactor temperature
• Sum of alphas is 1.0, if these are only
  components
• Fs are mass units

12
LP assumptions limitations
LINEAR PROGRAMMING
Models must conform to these restrictions

• Linearity
• Divisibility
• Certainty

• Mathematician The variables must be continuous.
• Chemist The molecules cannot be divided.
• Engineer Continuous variables are a good enough
  approximation.

13
LP assumptions limitations
LINEAR PROGRAMMING
Models must conform to these restrictions

• Linearity
• Divisibility
• Certainty

We will develop a method for no uncertainty.
Then, we will extend with sensitivity analysis.
14
LP assumptions limitations
LINEAR PROGRAMMING
Linear programming is formulated using the
general problem statement. How do we satisfy the
modelling limitations?
Objective function Equality constraints Inequali
ty constraints Variable Bounds
15
LP assumptions limitations
LINEAR PROGRAMMING
Linear programming is formulated using the
general problem statement. We must learn some
formulation tricks.
Objective function Equality constraints Inequali
ty constraints Variable Bounds
16
LP assumptions limitations
LINEAR PROGRAMMING
Linear programming symbols
aij coefficient of variable j (column) in ith
constraint equation (row) A matrix with
elements aij (termed LHS) b vector of
constraint RHS values with elements bi cj
objective coefficient for variable j m number
of constraints n number of variables z
objective function ?cjxj
17
LP assumptions limitations
LINEAR PROGRAMMING
Linear programming requires inequality
constraints that require the LHS (left hand side)
to be less than or greater than or equal to the
RHS (right hand side).
Inactive (feasible) Active (feasible) Violated
(infeasible)
18
LP concepts geometry
LINEAR PROGRAMMING
The feasible region contains points in three
categories.
Dark blue dashed lines are objective contours,
lines with constant values of the objective
function.
Class exercise define adjacent corner points
Corner point
?
2
Variable x
Shaded area
is the feasible
region
Interior point
Boundary point, Edge
Variable x
1
19
LP concepts geometry
LINEAR PROGRAMMING
Corner Point A point is a corner point (p) if
every line segment in the set (feasible region)
containing p has p as an endpoint. When
explaining linear programming, various references
use the following terms, all having the same
meaning corner point, extreme point, and vertex.
20
LP concepts geometry
LINEAR PROGRAMMING Key geometric interpretation
Physical example Where does the optimum occur?
The feasible region is the set of points in
yellow, which is defined by four inequality
constraints
21
LP concepts geometry
LINEAR PROGRAMMING Key geometric interpretation
The best corner-point solution must be the
optimal value of the objective function! Thus,
if the problem has one optimal solution, it must
be a corner point (vertex) if it has multiple
optimal solutions, at least two must be located
at corner points (vertices).

• Class workshops
• Explain why the green interior point cannot be
  optimal

22
LP concepts geometry
LINEAR PROGRAMMING Key geometric interpretation
Linear programming is a convex optimization
problem therefore, in linear programming a local
optimum is a global optimum!

• Class workshops
• Can an LP have an infinite number of optimal
  points?

23
LP concepts geometry
LINEAR PROGRAMMING Key geometric interpretation
Using the Corner Point concept as the foundation
for a solution method. Proposed algorithm
calculate the objective function value at all
corner points and select the best.
With 20 variables and 10 constraints, 185,000
corner points exist! And, that is a small
problem!
Back to the drawing board!
24
LP concepts geometry
LINEAR PROGRAMMING Key geometric interpretation

• BASIC NUMERICAL OPTIMIZATION
• We only have local information at the current
  point in an iterative scheme.
• We determine a good next point, which will
  improve the objective value
• We check if further improvement is possible. If
  yes, continue
• Stop at local optimum.

Class Exercise Develop an algorithm that
combines the basic numerical approach with the
key geometric insight.
Starting point
25
LP concepts geometry
LINEAR PROGRAMMING Key geometric interpretation
Class Exercise Develop an algorithm that
combines the basic numerical approach with the
key geometric insight.

• One approach would
• Use the gradient to select the direction of
  improvement.
• Move in the interior along the greatest rate of
  improvement
• Move until constraint has been reached
• Must change the search direction, how?

We will not use this interior path concept
26
LP concepts geometry
LINEAR PROGRAMMING Key geometric interpretation
Class Exercise Develop an algorithm that
combines the basic numerical approach with the
key geometric insight.

• One approach would
• Consider only adjacent corner points for
  improvement direction.
• Move along the edge that yields the greatest rate
  of improvement
• Move until another corner point has been reached
• If further improvement is possible, iterate.

Basis of the Simplex Algorithm
27
Simplex algorithm
LINEAR PROGRAMMING
The Simplex algorithm - Using the Corner Point
concept as the foundation for an efficient
solution method.
We want to convert this general formulation to a
system of linear equations - we know how to solve
these!
28
Simplex algorithm
LINEAR PROGRAMMING
The Simplex algorithm - Using the Corner Point
concept as the foundation for a solution method.
We will add slack variables to all inequalities
to convert them to equalities.
When xs?0, what do the slack variables measure?
29
Simplex algorithm
LINEAR PROGRAMMING
The Simplex algorithm - Using the Corner Point
concept as the foundation for a solution method.
We have the LP problem in Standard Form. Note
that the system has more variables than equations.
The vector x of variables now includes the slack
variables.
30
Simplex algorithm
LINEAR PROGRAMMING The Simplex Algorithm Finding
a solution for non-square set of equations
31
Simplex algorithm
LINEAR PROGRAMMING The Simplex Algorithm Finding
a solution for non-square set of equations

• BASIS How do we select the variables to form a
  basis?
• A basis is formed by any selection of variables
  that yields a set of linearly independent
  equations. The equations will be linearly
  independent if
• Det (AB) ? 0
• with AB the square coefficient sub-matrix based
  on the variables selected.
• The basic variables will have a unique solution
  for all variables.
• The selection of basic variables is not unique.
  For the system of m equations with n variables
  (mgtn), typically many bases are possible.

32
Simplex algorithm
LINEAR PROGRAMMING The Simplex Algorithm Finding
a solution for non-square set of equations
A basic feasible solution satisfies the basis
equations and the non-negativity constraints for
the variables. What does a basis define in the
LP space?
33
Simplex algorithm
LINEAR PROGRAMMING
Relating the equations to the graphical
interpretation.
Non-basic variables which take the values gt0
Square set of equations that can be solved for
the basic variables
FEASIBLE SOLUTION Any selection of values for the
non-basic variables that result in all variables
being non-negative yields a feasible solution.
34
Simplex algorithm
LINEAR PROGRAMMING
Relating the equations to the graphical
interpretation.
Square set of equations that can be solved for
the basic variables
Non-basic variables which take the values gt 0
BASIC FEASIBLE SOLUTION A selection of 0.0 values
for the non-basic variables result in basic
variables being non-negative and yields a corner
point.
35
Simplex algorithm
LINEAR PROGRAMMING
Relating the equations to the graphical
interpretation.

• A BASIC FEASIBLE SOLUTION IS A CORNER POINT
• Any small, negative change in a non-basic
  variable leads to an infeasibility.
• Any point between the solution and a feasible
  point has the solution at the end point of a
  line.
• Therefore, a BFS is a corner point.

36
Simplex algorithm
LINEAR PROGRAMMING The Simplex Algorithm
Using the Corner Point concept as the foundation
for a solution using the Simplex Algorithm.
Start here 1. Are we optimal 2. Move along an
edge to an adjacent corner point. 3. This adds
one variable to the basis and removes another
variable from the basis.
37
Simplex algorithm
LINEAR PROGRAMMING
Moving between adjacent corner points is achieved
by changing one variable from basic to non-basic
and one non-basic to basic. (The new variables
must form a basis.)
38
Simplex algorithm
LINEAR PROGRAMMING
Using the Corner Point concept as the foundation
for a solution using the Simplex Algorithm.
39
Simplex algorithm
LINEAR PROGRAMMING The Simplex Algorithm
Defining the best corner point
Each corner point represents a set of
basic/non-basic
40
Simplex algorithm
LINEAR PROGRAMMING The Simplex Algorithm
Using the Corner Point concept as the foundation
for the Simplex Tableau. Finding an initial
feasible solution.
The Simplex method moves among feasible corner
points to find the best. But, it requires an
initial feasible corner point. The Big-M
method adds artificial variables.
Po c1x1 . cnxn the original objective
Pa c1x1 . cnxn Mxa1 .Mxam the
modified objective
The artificial variables get us started, but they
must not appear in the final optimal solution.
41
Simplex algorithm
LINEAR PROGRAMMING The Simplex Algorithm
The artificial variables give us an initial
solution in canonical form. The solution can be
determined by observation
42
Simplex algorithm
LINEAR PROGRAMMING The Simplex Algorithm
Give a meaning to the artificial variables. We
set all real variables to 0.0, and we use the
artificial variables to satisfy the equations.
The coefficient for every artificial var. is 1.0.
All artificial variables must be 0.0 at the
optimum!! Must have Big M penalty on every
artificial variable.
43
Simplex algorithm
LINEAR PROGRAMMING
These calculations are performed using a tableau
or matrix. The procedure involves a series of
row operations that move from corner point to a
better corner point.
Objective function coefficients
LHS problem constraint coefficients
Artificial variables
RHS problem constraint coefficients
44
Simplex algorithm
Quick View of the LP Tableau The problem is the
initial formulation in general form (Note This
is how you will enter it into a software package)
45
Simplex algorithm
Convert to standard form, with all equality
constraints)
Do values of slacks provide any useful
information?
46
Simplex algorithm
Add artificial variables to easily find an
initial feasible solution
Remember that our method works when moving among
feasible corner points
How can we add phony variables? What are the
units of the artificial variables?
47
Add high penalties to the artificial variables to
be sure that they do not affect the optimal
solution.
What is a reasonable value for M in this
problem?
48
Simplex algorithm
Set up the problem in a Tableau for row
operations
?
Which are the basic variables?
?
Canonical form We want the basic variables to
have a column with one 1.0 and all other elements
0.0. Is the above OK. If not, what do we do?
49
Simplex algorithm
Now, the problem is at a feasible corner point
and in canonical form. We want to begin
optimization by changing one variable in the
basis (moving to an adjacent corner point) to 1.
Improve the objective function, and 2. Remain
feasible.
Lets move along one edge and not too far this
will keep us (just) feasible.
What calculations do we perform so that x2 is a
basic variable??
50
Simplex algorithm
First, is the current point optimal? We want to
continue optimization by changing one variable in
the basis (moving to an adjacent corner point) to
1. Improve the objective function, and 2.
Remain feasible.
51
Now, is the current point optimal? We want to
continue optimization by changing one variable in
the basis (moving to an adjacent corner point) to
1. Improve the objective function, and 2.
Remain feasible.
52
Simplex algorithm
LINEAR PROGRAMMING The Simplex Algorithm
Defining the best corner point
The LP method will converge to an optimum. 1. In
a finite number of steps (cycling theoretically
possible) 2. Experience shows that the
convergence is much faster than the theoretical
maximum number of iterations 3. Because the
problem is convex, the local optimum is a global
optimum.
53
Simplex algorithm
LINEAR PROGRAMMING
Do I have to write a computer program?
Problem Statement
Solution
z 25
Objective function
inactive
3.75 ? 4.0
active
20 ? 20
active
10 10
inactive
x1 5 ? 0
variables
inactive
x2 5 ? 0
54
Simplex algorithm
LINEAR PROGRAMMING

• In this course, we will not design or implement
  computer programs to solve the LP. We will
  concentrate on
• Formulating problems as LPs
• Using specialized modelling approaches to give
  reasonable models within the LP limitations
• Diagnosing results for failures
• Interpreting results for sensitivity
• We need to understand the algorithm at the level
  presented to become a skilled used.

The calculations can be performed by specialized
computer systems that enable the user to define
and solve the LP problem without computer
programming for the Simplex algorithm.
55
Simplex algorithm
LINEAR PROGRAMMING
IFORS The Simplex Place enables user to
participate in the step-by-step solution of an LP.
Great learning resource. Not for use in
solving practical problems
56
Simplex algorithm
LINEAR PROGRAMMING
Microsoft EXCEL Solver function uses data in
cells for LP
OK for small LPs
57
Simplex algorithm
LINEAR PROGRAMMING
GAMS Very powerful equation manager and solver.
Basic input and output are in a text file.
Requires the engineer to learn a new language.
Sounds tough, but its worth the effort.
A pipestill is a crude distillation unit, with
several products from the two (atmospheric and
vacuum) towers Table pipestill_yield(flow, crude)
pipestill yield (proportions by weight)
arabian-l arabian-h brega lv-naphtha
.035 .030 .045 iv-naphtha .100
.075 .135 v-heat-oil .390 .300
.430 vacuum-dst .285 .230 .280
res-arab-l .165 res-arab-h
.335 res-brega .100
OK for LPs with 10,000 variables
purchase_cost .. purchase E
sum(crude, crude_oil(crude)crudedat(crude,
'price')) flowrate('fuel-oil',
'in', 'fuel-imp')fuel_imp_price transport_cost
.. transport E sum(crude,
crude_oil(crude)crudedat(crude,
'transport')) profit_equ .. profit E
revenue - operating - purchase -
transport model exxon /all/
58
Simplex algorithm
LINEAR PROGRAMMING
The LP Simplex Algorithm Extensions

• Variables can be positive, zero, or negative.
  Replace one unrestricted variable with two
  non-negative variables
• x x x with x ? 0 and x ? 0
• Variables can have upper and lower bounds
• (We can do this without adding constraints)
• Efficient Calculations
• - Do not recalculate a big matrix every time
• - Restart from a previous solution

These features are included in software packages.
59
Simplex algorithm
LINEAR PROGRAMMING
Is there more about the solution that we would
like to know?
Problem Statement
Solution
z 25
Objective function
inactive
3.75 ? 4.0
active
20 ? 20
active
10 10
inactive
x1 5 ? 0
variables
inactive
x2 5 ? 0
60
Sensitivity analysis
LINEAR PROGRAMMING
Yes, SENSITIVITY or What if - Since many
parameters are uncertain, we would like to know
how the solution changes when a parameter changes.
Problem Statement
Example of sensitivity information
What is the affect of a change in this parameter
on z and all x? The result is the same as
changing the parameter and resolving, but without
the work!
61
Sensitivity analysis
LINEAR PROGRAMMING
SENSITIVITY - Since many parameters are
uncertain, we would like to know how the solution
changes when a parameter changes.
WHY? - We could always change parameter(s) and
resolve the LP. Sensitivity analysis yields some
useful information with little computation and
helps build insight.
Parameter changes
Structure changes
Qualitative Sensitivity
- direction of z change - relative magnitude of z
change
Y Y
- exact change in z (over range) - range of
validity
Quantitative Sensitivity
Y N
62
Sensitivity analysis
LINEAR PROGRAMMING
QUALITATIVE SENSITIVITY A. Changes in the
feasible region - direction of z change 1. For
changes that increase the feasible region, the
objective function is either improved (for
maximum, increased or for minimum, decreased) or
unchanged 2. For changes that decrease the
feasible region, the objective function is either
unchanged or worse (for maximum, decreased or for
minimum, increased)
Note The change is after reoptimization. The
values of the variables (x) can change from the
base case to the sensitivity case.
63
Sensitivity analysis
LINEAR PROGRAMMING
QUALITATIVE SENSITIVITY A. Class Exercise - For
each of the following single changes to a
minimization problem, what would be the effect of
the optimum value of the objective function? 1.
Change the RHS of the first constraint to 4.5 2.
Eliminate the second constraint 3. Add a
constraint 7 x1 3 x2 lt 8 4. Allow variable
x2 to be unrestricted, from -? to ? 5.
Reducing the LHS coefficient in the second
constraint from 3.0 to 1.5 6. Change the RHS of
the second constraint to 22 7. Change the first
constraint to an equality
64
Sensitivity analysis
Solution
QUALITATIVE SENSITIVITY A. Class Exercise 1.
Change the RHS of the first constraint to 4.5 -
decrease or same 2. Eliminate the second
constraint - decrease or same 3. Add a
constraint 7 x1 3 x2 lt 8 - increase or
same 4. Allow variable x2 to be unrestricted,
from -? to ? - decrease or same 5. Reducing the
LHS coefficient in the second constraint
from 3.0 to 1.5 - decrease or same (note xi ?
0) 6. Change the RHS of the second constraint to
22 - increase or same 7. Change the first
constraint to an equality - increase or same
65
Sensitivity analysis
LINEAR PROGRAMMING
QUALITATIVE SENSITIVITY B. Changes in the
feasible region - magnitude of z change 1. The
objective function changes monotonically as the
feasible region is enlarged or reduced. 2. The
rate of change in z decreases as the feasible
region increases and active constraints change.
Note The change is after reoptimization. The
values of the variables (x) can change from the
base case to the sensitivity case.
66
Sensitivity analysis
LINEAR PROGRAMMING
For increasing the feasible region 1. The
objective function changes monotonically 2. The
rate of change in z decreases
No change in active set
Change in active set
67
Sensitivity analysis
LINEAR PROGRAMMING
B. Class Exercise - For a maximization problem
and an active constraint of the following
type, Sketch the typical qualitative behavior of
the objective function as the value of the RHS is
increased and decreased from the base case.
Assume that the active set changes over the range
of values of the RHS.
OBJ
RHS
68
Sensitivity analysis
B. Class Exercise - Solution See Rardin (1998)
for further discussion
Active constraint
Max OBJ
Min OBJ
69
Sensitivity analysis
LINEAR PROGRAMMING
QUALITATIVE SENSITIVITY C. Changes in the cost
coefficients - magnitude of z change 1. Over a
limited range of cost coefficients, the corner
point (x) does not change. 2. The rate of
change in z increases as the cost coefficient
improves and active constraints change.
Note The change is after reoptimization, if
required. The values of the variables (x) can
change from the base case to the sensitivity case.
70
Sensitivity analysis
LINEAR PROGRAMMING
QUALITATIVE SENSITIVITY C. Changes in the cost
coefficients - magnitude of z change 1. Over a
limited range of cost coefficients, the corner
point (x) does not change. 2. The rate of
change in z increases as the cost coefficient
improves and active constraints change.
After a large change in the cost coefficient, the
optimal corner point changes.

• For limited changes in cj, the optimal corner
  point does not change.
• The optimal z changes because of ?cj
• When the change is large enough, the corner point
  changes - now z changes because of ?cj and ?xj

The
Original problem
original
optimum
71
Sensitivity analysis
LINEAR PROGRAMMING
C. Class Exercise - For a maximization problem,
the cost coefficient changes. Sketch the typical
qualitative behavior of the objective function as
the value of the cost coefficient is increased
and decreased from the base case. Assume that
the active set (optimal corner point) changes
over the range of values of the RHS.
OBJ
Cost coefficient, cj
72
Sensitivity analysis
C. Class Exercise - Solution See Rardin (1998)
for further discussion
Max OBJ
Min OBJ
73
Sensitivity analysis
LINEAR PROGRAMMING
Quantitative Sensitivity

• Sensitivity ?z/?? with all x constant
• Sensitivity ?z/?? with all basic variables, xB,
  allowed to change so the results represent a new
  optimal solution, i.e., corner point, for the
  modified problem.

1. The results are limited to optima with the
same active constraints as the base case, i.e.,
without requiring a basis change. 2. The
results define the range of parameter changes
which result in no change to the active
constraints. 3. The results provide an exact,
quantitative value of ?OBJ/?parameter
74
Sensitivity analysis
LINEAR PROGRAMMING
The LP Simplex Algorithm - Quantitative
Sensitivity
One-at-a-time changes to a RHS parameter
Change to RHS, e.g., max production rate, min
flow, etc.
0
non-basic are not changed if ?bi results in same
basis
Resulting change in basic variables
75
Sensitivity analysis
LINEAR PROGRAMMING
The LP Simplex Algorithm - Quantitative
Sensitivity
Changes to one RHS parameter
How much can we change constraint 1 RHS without
changing the basis?
Note that x changes, even though the active set
does not change!
76
Sensitivity analysis
LINEAR PROGRAMMING
The LP Simplex Algorithm - Quantitative
Sensitivity
One-at-a-time changes to one RHS parameter -
Software products report the value and range of
the sensitivity for every constraint! Outside of
the range, the basis changes what do we do?
Remember to check for constraint degeneracy!
Shadow price ?OBJ/?RHS, has units!
77
Sensitivity analysis
LINEAR PROGRAMMING
The LP Simplex Algorithm - Quantitative
Sensitivity
Class Exercise SolutionOne-at-a-time changes to
one RHS parameter. Use the table on the
preceding slide.
1. What is the effect on the profit of changing
the reflux flow rate capacity by 2.1? 2. Are the
pump slack and shadow price values
consistent? 3. What does a maximum allowable
increase of 10E30 mean? 4. A proposal has been
made to increase the capacity of pump 7.
Discuss your response to the proposal. 5. What
is the effect of reducing the reflux flow by 125?
78
Sensitivity analysis
LINEAR PROGRAMMING
The LP Simplex Algorithm - Quantitative
Sensitivity
1. What is the effect on the profit of changing
the reflux flow rate capacity by 2.1? First,
2.1 lt max increase, therefore, shadow price is
valid. ?profit 2.13.74 2. Are the pump slack
and shadow price values consistent? Yes, since
the constraint is not active, the slack gt 0 and
the shadow price is 0.0 (no effect for a change
within range). 3. What does a maximum allowable
increase of 10E30 mean? It really means infinity.
Since the constraint is inactive, increasing its
RHS to infinity does not affect the solution 4. A
proposal has been made to increase the capacity
of pump 7. Discuss your response to the
proposal. This would not be profitable for the
operation considered in this case. The pump
already has more capacity than needed at the
optimal operation. 5. What is the effect of
reducing the reflux flow by 125? The effect for
reducing by 123 is (3.74)(-123). We do not know
the cost for the next 2 units of reduction, but
we know that it is greater (a larger loss be unit
flow in this case). In the limit, the problem
could become infeasible.
79
Sensitivity analysis
LINEAR PROGRAMMING
The LP Simplex Algorithm - Quantitative
Sensitivity
objective function z ?cjxj
Sensitivity to one change in cost coefficient of
a basic variable

• For small changes in cj, does x change?
• For small changes in cj, does z change?
• For changes in cj, describe how the feasible
  region is affected.
• Describe the size of changes in cj, that result
  is a change in the optimal basis.

80
Sensitivity analysis
LINEAR PROGRAMMING
The LP Simplex Algorithm - Quantitative
Sensitivity
objective function z ?cjxj
Sensitivity to one change in cost coefficient of
a basic variable
For changes in cj that retain the same basis, ?x
0 ? z ? ?cjxj ? ? cj(xj) ? ck(xk)
with k the cost coefficient changed
81
Sensitivity analysis
LINEAR PROGRAMMING
The LP Simplex Algorithm - Quantitative
Sensitivity
A cost change beyond the max/min range will cause
a change to the optimal basis (active set). The
range is given in the solution. If basis
changes, resolve.
82
Sensitivity analysis
LINEAR PROGRAMMING
The LP Simplex Algorithm - Quantitative
Sensitivity

• What if more than one parameter changes?
• 100 Rules are available to test whether the
  base case sensitivity results can be applied.
  (See the course notes and references for
  details).
• These rules find the worst case direction for
  all changes to see if the corner point could
  change.
• - If the worst case does not result in a corner
  point change, the sensitivity results can be
  evaluated using
• total change in z ? (change from each
  parameter)
• - If the basis (corner pt.) changes, reoptimize.

83
Sensitivity analysis
LINEAR PROGRAMMING
The LP Simplex Algorithm - Quantitative
Sensitivity
Great, but everything has its limits.
However, for general changes in the LHS A
coefficients, the problem must be resolved for
quantitative results
84
Sensitivity analysis
LINEAR PROGRAMMING
The LP Simplex Algorithm - Quantitative
Sensitivity

• We have covered the interpretation of the
  qualitative and quantitative sensitivity results.
  These are critical skills for the engineer.
• We have not cover how the quantitative
  sensitivity analysis is calculated. For further
  details, see Rardin (1998) for introduction to
  the DUAL formulation.

85
Solution Diagnosis
LINEAR PROGRAMMING
The LP Simplex Algorithm Weird Events
We must monitor and diagnose the LP solution. We
could have made a formulation error, or we could
have defined a problem that is correct but has
special, unusual, properties. We must monitor for
weird effects.

• Lets learn to
• diagnose and
• correct (if possible).

86
Solution Diagnosis
LINEAR PROGRAMMING
The LP Simplex Algorithm Weird Events

• NO FEASIBLE SOLUTION
• Diagnosis -
• Remedial Action -

87
Solution Diagnosis
LINEAR PROGRAMMING
The LP Simplex Algorithm Weird Events

• NO FEASIBLE SOLUTION
• Diagnosis - At least one artificial variable in
  optimal basis - software reports this as
  infeasible.
• Remedial Action - reformulate, if appropriate
• (See goal programming)

88
Solution Diagnosis
LINEAR PROGRAMMING
The LP Simplex Algorithm Weird Events

• UNBOUNDED SOLUTION
• Diagnosis -
• Remedial Action -

89
Solution Diagnosis
LINEAR PROGRAMMING
The LP Simplex Algorithm Weird Events

• UNBOUNDED SOLUTION
• Diagnosis - The distance to the best adjacent
  corner point is infinity - software will report.
• Remedial Action - Reformulate, which is always
  possible - realistic variables never go to ?

90
Solution Diagnosis
LINEAR PROGRAMMING
The LP Simplex Algorithm Weird Events

• ALTERNATIVE OPTIMA
• Diagnosis -
• Remedial Action -

91
Solution Diagnosis
LINEAR PROGRAMMING
The LP Simplex Algorithm Weird Events

• ALTERNATIVE OPTIMA
• Diagnosis 1 - The basis can change with no change
  in objective.
• One or more non-basic variables has a zero
  marginal cost.
• Software does not report warning

92
Solution Diagnosis
LINEAR PROGRAMMING
The LP Simplex Algorithm Weird Events

• ALTERNATIVE OPTIMA
• Diagnosis 2 - One or more active constraint rhs
  can be changed without affecting the objective.
• An active constraint has a zero marginal value
  and non-zero range (both ways).
• Software does not report warning

Constraint rhs can be changed with no change to
OBJ
93
Solution Diagnosis
LINEAR PROGRAMMING
The LP Simplex Algorithm Weird Events

• ALTERNATIVE OPTIMA
• Remedial action- We have found the best value of
  the objective function!
• We likely prefer one of the different sets of x
  values. We would like to know all solutions and
  select the best, using additional criteria.

94
Solution Diagnosis
LINEAR PROGRAMMING
The LP Simplex Algorithm Weird Events

• CONSTRAINT
• DEGENERACY Redundancy
• Diagnosis -
• Remedial Action -

95
Solution Diagnosis
LINEAR PROGRAMMING
The LP Simplex Algorithm Weird Events

• CONSTRAINT
• DEGENERACY Redundancy
• Diagnosis - The marginal value of an active
  constraint rhs depends upon the direction of
  change.
• The range of the (non-zero) marginal value is
  0.0 in one direction.
• Software does not report warning

96
Solution Diagnosis
LINEAR PROGRAMMING
The LP Simplex Algorithm Weird Events

• CONSTRAINT
• DEGENERACY Redundancy
• Remedial action- The solution is correct.
• The sensitivity information is not reliable!
• If you need sensitivity information, introduce
  the change (rhs, cost, etc.) and rerun the
  optimization.

97
Solution Diagnosis
LINEAR PROGRAMMING
The LP Simplex Algorithm Weird Events

• CONSTRAINT
• DEGENERACY
• Diagnosis -
• Remedial Action -

optimum
98
Solution Diagnosis
LINEAR PROGRAMMING
The LP Simplex Algorithm Weird Events

• CONSTRAINT
• DEGENERACY
• Diagnosis -
• More inequalities are active (slacks 0) than
  dimension of the problem.
• Software does not report warning

optimum
99
Solution Diagnosis
LINEAR PROGRAMMING
The LP Simplex Algorithm Weird Events

• CONSTRAINT
• DEGENERACY
• Remedial action - The solution is correct.
• The sensitivity information is not reliable!
• If you need sensitivity information, introduce
  the change (rhs, cost, etc.) and rerun the
  optimization.

optimum
100
Model formulations
LINEAR PROGRAMMING
Formulating Models for LPs
101
Model formulations
LINEAR PROGRAMMING
Formulating Models for LPs
How do we start?
What is special about optimization? For more
guidance, see lessons of Art of Modeeling and
Problem Solving.
102
Model formulations
LINEAR PROGRAMMING
Formulating Models for LPs
We know that systems can be very complex and are
seldom (never) truly linear. But, LPs must
conform to severe restrictions.
Fundamental Balances

• Material (ball bearings, fluid, people,
  airplanes, )
• Time (utilization of equipment, peoples work, .
  )
• Energy (vehicle travel, processing, )
• Space (volume, area)
• Lumped quantity (pollution, economic activity, ..)

103
Model formulations
LINEAR PROGRAMMING
Formulating Models for LPs
Lets do some examples Flow Splitting
F2
Feed mass fractions ?1, ?2, ...?n
What is the model?
F1
F3
104
Model formulations
LINEAR PROGRAMMING
Formulating Models for LPs
Lets do some examples Perfect Separator
All of components 1 2 in feed
F2
What is the model?
Feed mass fractions ?1, ?2, ...?n
F1
All of components 3 to n in feed
F3
105
Model formulations
LINEAR PROGRAMMING
Formulating Models for LPs
Lets do some examples CSTR Reactor
FA flow rate of A FB flow rate of B FC
flow rate of C FD flow rate of D
What is the model for the product flows?
Ff feed flow rate
T
A
Reactant
Coolant
106
Model formulations
LINEAR PROGRAMMING
Formulating Models for LPs
Lets do some examples CSTR Reactor
FA flow rate of A FB flow rate of B FC
flow rate of C FD flow rate of D
What is the model for the coolant duty?
Ff feed flow rate
T
A
Reactant
Coolant
107
Model formulations
LINEAR PROGRAMMING
Formulating Models for LPs
Lets do some examples Production Planning
Product 1
Feed 1
Product 2
Feed 2
Product 3
What is the model?
108
Model formulations
LINEAR PROGRAMMING
Formulating Models for LPs
Lets do some examples Production Planning
Product 1
Feed 1
Product 2
Feed 2
Product 3
Are there important issues not included?
109
Model formulations
LINEAR PROGRAMMING
Formulating Models for LPs
Lets do some examples Production Scheduling
warehouse
Periodic deliveries
Product 1
Feed 1
Product 2
Product 3
Feed 2
Product 1
Product 2
Product 3
The model must consider the inventories in tanks,
warehouses, factory. Also, delays in
transportation can be important.

• Describe this
• category of model.
• What type of
• balances are
• needed?

110
Model formulations
LINEAR PROGRAMMING
Formulating Models for LPs
What we have seen so far are straightforward
models. Straightforward model represents effects
of dominant variables and ignores all other
effects. Now, lets look at some extensions
(tricks).

• Base-Delta
• Disjunctive
• Separable Programming
• Goal Programming

• Linearizing transforms (Blending)
• MiniMax
• Linear-Maximum

111
Model formulations
LINEAR PROGRAMMING
Formulating Models for LPs
Base-Delta models include the effects of
secondary variables over a limited range
Fi ? (F) ? (T-T0) ? (P-P0)
? is the yield at T0 and P0
112
Model formulations
LINEAR PROGRAMMING
Formulating Models for LPs
Disjunctive Programming provides alternative
models for the same system.
113
Model formulations
LINEAR PROGRAMMING

• Class exercise Formulate LP models for the
  reformer reactor/separator process. You should
  build
• Straightforward
• Base-delta
• Disjunctive

114
Model formulations
LINEAR PROGRAMMING
The next modelling approach depends on the
functions being separable. OK, so what is a
separable function?
Are the following functions separable?
A function H(x) is separable if it can be
expressed as the sum of single-variable
functions, hi(xi).
115
Model formulations
LINEAR PROGRAMMING
Separable Programming extends the range over
which the linear models are accurate by using
piecewise linear models.
A function H(x) is separable if it can be
expressed as the sum of single-variable
functions, hi(xi).
Each single-variable function can be approximated
by a piecewise linear function.
h(xi)
xi
116
Model formulations
LINEAR PROGRAMMING
Formulating Models for LPs
Separable Programming extends the range over
which the linear models are accurate by using
piecewise linear models.
117
Model formulations
LINEAR PROGRAMMING
Formulating Models for LPs
Separable Programming extends the range over
which the linear models are accurate by using
piecewise linear models.
LP is applicable when the original function H(x)
is convex. Discuss with respect to the following
minimization examples.
A
B
cost
cost
x
x
118
Model formulations
LINEAR PROGRAMMING
Formulating Models for LPs
Goal Programming provides a method for getting
close to desired conditions that are not
possible. This is a penalty function.
We want to achieve three properties and the total
flow by mixing two flows. What do we do?
119
Model formulations
LINEAR PROGRAMMING
Formulating Models for LPs
Goal Programming
Extra variables are added to ensure math
feasibility they measure the amount of
infeasibility related to the original
problem. They are penalized to minimize
infeasibility.
120
Model formulations
LINEAR PROGRAMMING
Formulating Models for LPs
Linearizing Transformations - Some properties can
be transformed to behave linearly.
121
Model formulations
LINEAR PROGRAMMING
Formulating Models for LPs
Flow-property relationships- When properties
combine linearly and the properties do not depend
on the variables (flows), a special formulation
can be used.
Non-linear
Linear
122
Model formulations
LINEAR PROGRAMMING
Formulating Models for LPs
Mini-Max Problem- Often, we have a range of
outcomes possible from alternative systems. These
could systems could be - Alternative decisions
(e.g., investments) - possible outcomes from an
uncertain system Here, we would like to consider
every system and minimize the worst case, i.e.,
the maximum of the minimums.
123
Model formulations
LINEAR PROGRAMMING
Formulating Models for LPs
Mini-Max Problem- Minimize the worst case,
i.e., the maximum of the minimums from all
systems.
fi a set of linear equations and inequalities
with the parameters ?i yielding an objective
function fi ?i The parameters associated with
outcome I x The optimization variables, which
are used in every model I
124
Model formulations
LINEAR PROGRAMMING
Formulating Models for LPs
Minimum-Proportional The flow through the
compressor is the feed rate, except when the feed
rate is below a minimum. Then, the recycle flow
opens to maintain a the flow above the minimum to
prevent damage to the compressor.
125
Model formulations
LINEAR PROGRAMMING
Engineers performing different functions require
tailored windows to the software - the user
interface.
Algorithm development
Model formulation
Define input and results analysis
This is OK too, but you need more study of math
and numerical methods
This course prepares you for these tasks
126
Model formulations
LINEAR PROGRAMMING
Engineers performing different functions require
tailored windows to the software - the user
interface.
Model formulation
Define input and results analysis
Flexible equation management such as GAMS enables
formulation in natural manner.
Tabular and graphical interface displays results
for easy interpretation. Excel is a reasonable
choice.
purchase_cost .. purchase E
sum(crude, crude_oil(crude)crudedat(crude,
'price')) flowrate('fuel-oil',
'in', 'fuel-imp')fuel_imp_price transport_cost
.. transport E sum(crude,
crude_oil(crude)crudedat(crude,
'transport')) model exxon /all/
127
Model formulations
LINEAR PROGRAMMING
Documenting LP Results - See course notes for
guidance
Plot of results as parameter changes is
especially effective
Numbers indicate basis changes
128
Conclusions
LINEAR PROGRAMMING

• The LP solution method offers the following
• whether or not the feasible region is unbounded
  (giving an objective value of ? ?).
• whether or not the feasible region does not exist
  (no solution).
• the unique optimum value of the objective at a
  corner point, or the existence of multiple
  optima.

more
129
Conclusions
LINEAR PROGRAMMING

• The LP solution method offers the following
• excellent sensitivity information regarding the
  effects of changes in selected parameters.
• Excellent software
• Extremely fast solutions for small problems
• Solutions for problems with gt 100,000 variables

We will seek to formulate an optimization problem
as an LP, when the method provides adequate
accuracy for the problem being solved.
130
Conclusions
LINEAR PROGRAMMING
Nothing is linear!!! When and why do we apply
Linear Programming

• Simple material balances (shipping, storing, and
  selling Products) because LP approximations are
  very good.

• Complex (even highly NL) systems with large
  uncertainties. In some cases, LP approximations
  are less than uncertainties. We need to find
  ways to correct as we learn more information.

• Some problems require a fast solution. An
  important example is control systems. (Feedback
  corrects model errors.)

• Some problems are still too large to solve. (We
  will see some when we introduce integer
  decisions.)

131
Linear Programming - Workshop 1
The Simplex algorithm assumes that the LP problem
in standard form has more variables than
equations. Discuss the validity of this
assumption. (Do all realistic problems conform
to this condition?)
132
Linear Programming - Workshop 2
The Simplex algorithm moves among bases, and a
basis provides a unique solution. A model might
have linearly dependent equations. How does the
simplex algorithm deal with dependent equations?
133
Linear Programming - Workshop 3
The LP lessons introduced the Big-M method for
determining an initial basic feasible
solution. Design an alternative method for
determining an initial basic feasible
solution. Discuss the result of your problem if
the problem has no feasible solution.
134
Linear Programming - Workshop 4
Given the problem formulation and an initial
feasible solution, derive an equation for a
direction for (any) other feasible solution. Is
this direction adequate for selecting a direction
of improvement?
135
Linear Programming - Workshop 5
A graphical analysis is given in slide number 66
showing the effect of relaxing a constraint by
changing the RHS value. It shows that the rate
of change of the objective must decrease as the
basis changes. Extend the analysis for reducing
the feasible region by changing the RHS.
Generalize your result concerning the rate of
change of the objective.
136
Linear Programming - Workshop 6
We are evaluating a potential investment and its
sensitivity to parameter changes. We have a base
case LP solution and sensitivity results. We are
interested in the profit for parameter changes
that will cause the basis (active set of
constraints) to change. How can we use the base
case information to evaluate the financial
attractiveness for the parameter changes?
137
Linear Programming - Workshop 6 - SOLUTION
Let's consider an example in which we are
maximizing the objective, profit and the feasible
region is increased, ?rhs gt 0. We can calculate
an estimate of the profit using the
following. Pestimate Profitbase case
(marginal value)(?rhs) - If this profit
estimate (Pestimate) is less than the acceptable
rate of return, we know that the project is not
acceptable. We can reject it, because the
marginal value would decrease if a basis change
occurred within ?rhs. This would lower the
profit even further. - If this profit estimate
(Pestimate) is above the acceptable rate of
return, we are not sure whether the profit with
?rhs is high enough, because the marginal value
of the constraint would decrease (perhaps, to
zero) if a basis change occurred. Therefore, this
problem has to be reoptimized with the right-hand
side changed.
138
Linear Programming - Workshop 7
The sensitivity of the inequality RHS is often
interpreted as shown below. Shadow price
?OBJ/?RHS is the change in the objective for a
unit (1.0) change in the RHS. Define when this
interpretation is correct and not correct, and
explain why.
139
Linear Programming - Workshop 8
Explain why sensitivity effects of general
changes in the LHS coefficients are more
difficult calculate and usually require a
reoptimization.
However, for general changes in the LHS A
coefficients, the problem must be resolved for
quantitative results
140
Linear Programming - Workshop 9
Modify the problem below to maximize the maximum
value problem.
Mini-Max Problem- Minimize the worst case,
i.e., the maximum of the minimums from all
systems.
fi a set of linear equations and inequalities
with the parameters ?i yielding an objective
function fi ?i The parameters associated with
outcome I x The optimization variables, which
are used in every model I
141
Linear Programming - Workshop 10
The LP problem stated below has one coefficient
that is not know exactly. The coefficient a22 is
equal to 2.0 ? 0.55. Describe how you would
solve this optimization problem.
142
Linear Programming - Workshop 11
These notes contain the solution to the LP
problem given below. The RHS coefficient for the
equality constraint changes from 10 to 10.5.
Determine the effects on the variables.
143
Linear Programming - Workshop 12
These notes contain the solution to the LP
problem given below. The LHS coefficient for the
second (inequality) constraint changes from 1.0
to 1.02. Determine the effects on the objective
function and the variables.
144
Linear Programming - Workshop 13
An LP computer program is under development. The
results shown below are obtained for the
objective function as the algorithm iterates
(after all artificial variables have been reduced
to zero. Discuss the results and determine
whether they are possible.
No change
Objective function, z
optimum
Iteration
145
Linear Programming - Workshop 14

• In Optimization Basics I, we learned that convex
  optimization provides a strong result - a local
  optimum point is a global optimum.
• Is this result true for linear programming?
• Is the LP feasible region convex?
• Is the LP objective function convex?
• Hint Apply the definition of convexity to the
  objective function and to equations and
  inequalities defining the feasible region.

146
Linear Programming - Workshop 15

• Separators are typically imperfect. In many
  cases, the fraction of impurities are specified
  in the product streams, here ?3 and ?2.
• Derive a linear model of this system with four
  components, with the variables being the flows.
• Describe how the process is operated to conform
  to the model.

All of component 1, most 2, none of component 4,
and ?3 fraction component 3
F2
Feed mass fractions ?1, ?2, ...?4
?1, ?2, ?2, ?4
0
F1
All of component 4, most 3, none of component 1,
and ?2 fraction component 2
?1, ?2, ?2, ?4
F3
0
Note ?1, ?2, ...?n can not change!!