LINEAR PROGRAMMING

1
LINEAR PROGRAMMING
YU LIANG CMASS, NCAT
2
Note

• 3-4 people form a group (A,B,C). Each group turn
  to answer questions according to dice.
• Plans for today
• Introduction to LP
• Graphic formulation for LP
• Resume Gauss-Jordan
• Standard form of LP
• Plans for Thursday
• Simplex Algorithm
• Advanced topics
• Do not take the notes, the slides will be
  distributed
• All questions are welcome!
• Enjoy the party!

3
Outline

• Introduction to Linear Programming
• Terminology
• Solving LP Problems
• Graphical Method
• Gauss-Jordan Strategy, An Essential Operation in
  Simplex Method.
• Simplex Method
• Standard Form
• Simplex Tableau Iterative Method
• Advanced Topics
• Unconstrained Optimization
• Constrained Optimization

4
Introduction

• Mathematical programming is used to find the best
  or optimal solution to a problem that requires a
  decision or set of decisions about how best to
  use a set of limited resources to achieve a state
  goal of objectives.
• Steps involved in mathematical programming
• Conversion of stated problem into a mathematical
  model that abstracts all the essential elements
  of the problem.
• Exploration of different solutions of the
  problem.
• Finding out the most suitable or optimum
  solution.
• Linear programming requires that all the
  mathematical functions in the model be linear
  functions.

5
Mathematical Optimization is nearly everywhere

• Finance
• Marketing
• E-business
• Telecommunications
• Games
• Operations Management
• Production Planning
• Transportation Planning
• System Design

6
Examples Product Mix Problem (0)

• A manufacturer has fixed amounts of different
  resources such as raw material, labor, and
  equipment.
• These resources can be combined to produce any
  one of several different products.
• The quantity of the ith resource required to
  produce one unit of the jth product is known.
• The decision maker wishes to produce the
  combination of products that will maximize total
  income.

7
Example Product Mix Problem(A)
The N. Dustrious Company produces two products I
and II. The raw material requirements, space
needed for storage, production rates, and selling
prices for these products are given in Table 1.
(1) The total amount of raw material available
per day for both products is 15751b. (2) The
total storage space for all products is 1500 ft2,
(3) and a maximum of 7 hours per day can be used
for production.
8
Example Product Mix Problem(B)
Problem Description
Two products share the total raw material,
storage space, and production time. The company
wants to determine how many units of each product
to produce per day to maximize its total income.
Solution

• The company has decided that it wants to maximize
  its sale income, which depends on the number of
  units of product I and II that it produces.
• Therefore, the decision variables, x1 and x2 can
  be the number of units of products I and II,
  respectively, produced per day.

9
Example Product Mix Problem(C)
The linear programming model for this example can
be summarized as

• Maximize Z 13x1 11x2 (profit)
• Subject to 4x1 5x2 1500 (storage)
• 5x1 3x2 1575 (material)
• x1 2x2 420 (working hours)
• x1, x2 0 (nonnegative
  quantity)

10
Linear Programming
minimize or maximize a linear objective subject
to linear equalities and inequalities maximize
3x 4y subject to 5x 8y ? 24
x, y ? 0
A feasible solution satisfies all of the
constraints.
x 1, y 1 is feasible x 1, y 3 is
infeasible.
An optimal solution is the best feasible solution.
The optimal solution is x 4.8, y 0.
11
Terminology

• Decision variables e.g., x and y.
• In general, there are quantities you can control
  to improve your objective which should completely
  describe the set of decisions to be made.
• Constraints e.g., 5x 8y ? 24 , x ? 0 , y ?
  0
• Limitations on the values of the decision
  variables.
• Objective Function. e.g., 3x 4y
• Value measure used to rank alternatives
• Seek to maximize or minimize this objective
• examples maximize NPV, minimize cost

12
Linear Programming (LP) Problem

• A mathematical programming problem is one that
  seeks to maximize an objective function subject
  to constraints.
• If both the objective function and the
  constraints are linear, the problem is referred
  to as a linear programming problem.
• Linear functions are functions in which each
  variable appears in a separate term raised to the
  first power and is multiplied by a constant
  (which could be 0).
• Linear constraints are linear functions that are
  restricted to be "less than or equal to", "equal
  to", or "greater than or equal to" a constant.

13
General Form Of LP Problems

• MAX (or MIN) c1x1 c2x2 cnxn
• Subject to a11x1 a12x2.. a1nxn b1
  a21x1 a22x2.. a2nxn b2
  am1x1
  am2x2.. amnxn bm,
• and
• x1 0, x2 0, ....., xn 0.

14
Terminology for Solutions

• Feasible solution A solution for which all the
  constraints are satisfied. It is possible for a
  problem to have no feasible solutions. Given that
  there are feasible solutions, the goal of the
  linear programming is to find which one is the
  best, as measured by the value of the objective
  function in the model.
• Optimal solution A feasible solution that has
  the most favorable value of the objective
  function. Most favorable value means the largest
  or the smallest value, depending upon whether the
  objective is maximization or minimization.
• one optimal solution
• multiple optimal solutions
• no optimal solutions

15
Solving LP Problems

• Graphical Solution Approach - used mainly for 2D
  problems.
• Simplex Method - most common analytic tool.

16
Steps in Formulating a Linear Programming (LP)
Model

• Understand the problem.
• Identify the decision variables.
• State the objective function as a linear
  combination of the decision variables.
• State the constraints
• upper or lower bounds on the decision variables,
  including non-negativity constraints if
  applicable.
• linear combinations of the decision variables.

17
Graphical Method
18
A 2-D Maximization Problem
Max z 5x1 7x2
s.t. x1 lt 6
(constraint 1) 2x1 3x2
lt 19 (constraint 2) x1
x2 lt 8 (constraint 3)
x1, x2 gt 0 (sign constraint)
19
Graphical Solution Constraint 1
x2
8 7 6 5 4 3 2 1 1
2 3 4 5 6
7 8 9 10
x1 lt 6
(6, 0)
x1
20
Graphical Solution Constraint 2
x2
8 7 6 5 4 3 2 1 1 2
3 4 5 6 7
8 9 10
(0, 6 1/3)
2x1 3x2 lt 19
(9 1/2, 0)
x1
21
Graphical Solution Constraint 3
x2
(0, 8)
8 7 6 5 4 3 2 1 1
2 3 4 5 6
7 8 9 10
x1 x2 lt 8
(8, 0)
x1
22
Graphical Solution Combined-Constraint
x2
x1 x2 lt 8
8 7 6 5 4 3 2 1 1
2 3 4 5 6
7 8 9 10
x1 lt 6
2x1 3x2 lt 19
x1
23
Graphical Solution Optimal Solution
x2
8 7 6 5 4 3 2 1 1
2 3 4 5 6
7 8 9 10
5x1 7x2 46
Optimal Solution
x1
24
Graphical Solution Five Extreme Points
8 7 6 5 4 3 2 1 1
2 3 4 5 6
7 8 9 10
5
4
Feasible Region
3
1
2
x1
25
Maximization vs Minimization
The position of your optimal point differs,
depending upon whether the objective is
maximization or minimization.
Maximization
Minimization
Optimum (Max)
Isoprofit line
Y
Y
Isocost line
Optimum (Min)
X
X
25
26
Special Cases

• Alternative Optimal Solutions In the graphical
  method, if the objective function line is
  parallel to a boundary constraint in the
  direction of optimization, there are alternate
  optimal solutions, with all points on this line
  segment being optimal.
• Infeasibility A linear program which is over
  constrained so that no point satisfies all the
  constraints is said to be infeasible.
• Unbounded Objective function can be maximized
  infinitely.

27
Example 2 Unbounded Problem

• Solve graphically for the optimal solution (Group
  Discussion)
• Max z 3x1 4x2
• s.t. x1 x2 gt 5
• 3x1 x2 gt 8
• x1, x2 gt 0

28
Example 2 Unbounded Problem

• The feasible region is unbounded and the
  objective function line can be moved parallel to
  itself without bound so that z can be increased
  infinitely.

x2
3x1 x2 gt 8
8
Max 3x1 4x2
5
x1 x2 gt 5
x1
5
2.67
29
Example 3 A Minimization Problem (Group
Discussion)
Min z 5x1 2x2
s.t. 2x1 5x2 gt 10
4x1 - x2 gt 12
x1 x2 gt 4 x1,
x2 gt 0
30
Example 3 Graphical Solution
Constraints Graphed
x2
Feasible Region
5 4 3 2 1
4x1 - x2 gt 12 x1 x2 gt 4

2x1 5x2 gt 10
1 2 3 4 5
6
x1
31
Example 3 Graphical Solution

• Optimal Solution

Min z 5x1 2x2 4x1 - x2 gt 12 x1 x2 gt
4
x2
5 4 3 2 1

2x1 5x2 gt 10 Optimal x1 16/5
x2 4/5
1 2 3 4 5
6
x1
32
Graphical Solution to a 2-Variable LP

• It can be shown that
• The feasible region for any LP will be a convex
  set.
• The feasible region for any LP has only a finite
  number of extreme points.
• Any LP that has an optimal solution has an
  extreme point that is optimal.

33
Fundemental Theorem Of LP (basis of Simplex
Method)

• If the optimal value of the objective function in
  a linear programming problem exists, then that
  value must occur at one (or more) of the corner
  points of the feasible region.

34
Existence of Solution

• (A) If the feasible region for a linear
  programming problem is bounded, then both the
  maximum value and the minimum value of the
  objective function always exist.
• (B)   If the feasible region is unbounded, and
  the coefficients of the objective function are
  positive, then the minimum value of the objective
  function exists, but the maximum value does not.
• (C) If the feasible region is empty (that is,
  there are no points that satisfy all the
  constraints), the both the maximum value and the
  minimum value of the objective function do not
  exist.

35
Summary of Graphic Solution

• The optimal solution (if one exists) occurs at a
  corner point of the feasible region.
• In two dimensions with all inequality
  constraints, a corner point is a solution at
  which two (or more) constraints are binding.
• There is always an optimal solution that is a
  corner point solution (if a feasible solution
  exists).
• More than one solution may be optimal in some
  situations
• Graphic solution only works for 2D -gt Simplex
  method.

36
Gauss-Jordan Algorithm

• A Key Operation for Numerical LP Algorithm

37
Gauss-Jordan Method Principle
Gauss-Jordan method is used to solve system of
linear equations ( the solution maybe be unique,
nonexistent, or infinite ). According to, Ax b
? P1P2P k A x P1P2P k b IF
P1P2P k A I, Then x P1P2P k b
38
Gauss-Jordan Elementary Row Operations (ero)

• Type 1 ero Matrix A is obtained by multiplying
  any row of A by a nonzero scalar.
• Type 2 ero a row of A (say, row j) is added by
  another scalar-multiplied row of A (say, row i).
  
• For some j ? i, let row j of A c(row i of A)
  row j of A.
• Type 3 ero Interchange any two rows of A.

39
Gauss-Jordan Examples (Group Discussion?)
Solve Ax b where
Step 0 Write the augmented matrix
representation
Ab
40
Gauss-Jordan Column 1

• Making a11 the current entry
• 1. Multiply row 1 by ½ (type 1 ero).
• 2. Replace row 2 of A1b1 by -2(row1 of A1b1)
  row 2 of A1b1 (type 2 ero).
• 3. Replace row 3 of A2b2 by -1(row 1 of A2b2)
  row 3 of A2b2 (type 2 ero).

A1b1
A2b2
A3b3
41
Gauss-Jordan Column 2

• Making a22 the current entry
• 4. Multiply row 2 of A3b3 by -1/3 (type 1 ero)
• 5. Replace row 1 of A4b4 by -1(row 2 of A4b4)
  row 1 of A4b4 (type 2 ero).
• 6. Place row 3 of A5b5 by 2(row 2 of A5b5)
  row 3 of A5b5 (type 2 ero).

A4b4
A5b5
A6b6
42
Gauss-Jordan Column 3

• Repeating Step 3 again for the another new entry
  (a33) and performing an additional three eros
  yields the final augmented array

A9b9
Special Cases After application of the
Gauss-Jordan method, linear systems having no
solution or infinite number of solutions can be
recognized.
No solution example
Infinite solutions example
43
Gauss-Jordan Method Basic and Nonbasic variables

• Basic variables and solutions to linear equation
  systems
• For any linear system, a variable that appears
  with a coefficient of 1 in a single equation and
  a coefficient of 0 in all other equations is
  called a basic variable.
• Any variable that is not a basic variable is
  called a nonbasic variable.
• Let BV be the set of basic variables for Axb
  and NBV be the set of nonbasic variables for
  Axb. The character of the solutions to Axb
  (and Axb) depends upon which of following
  cases occur.

44
Gauss-Jordan Case 1 -- No Solution Exists
Case 1 Axb has at least one row of the form
0, 0 , , 0 c (c ? 0). Then Axb (and Ax
b) has no solution. In the matrix to the right,
row 5 meets this Case 1 criteria. BV x1,
x2, x3 NBV x4
Ab
45
Gauss-Jordan Case 2 -- Only One Solution Exists
Case 2 The set of nonbasic variables is empty.
Then Axb will have a unique solution. The
matrix to the right has a unique solution. BV
x1, x2, x3 NBV .
Ab
46
Gauss-Jordan Case 3 Infinite Solution
Case 3 Suppose Case 1 does not apply and NBV is
not empty. Then Axb (and Ax b) will have an
infinite number of solutions. BV x1, x2, and
x3 NBV x4 and x5.
Ab
47
The Gauss-Jordan Method
A summary of Gauss-Jordan method is shown to the
right. The end result of the Gauss-Jordan method
will be one either Case 1, Case 2, or Case 3.
48
Real LP Problems

• Real-world LP problems often involve
• Hundreds or thousands of constraints
• Large quantities of data
• Many products and/or services
• Many time periods
• Numerous decision alternatives
• and other complications

49
Standard LP Form

• An Indispensable Preprocess for Numerical
  Solution of LP

50
Linear Programs in Standard Form

• We say that a linear program is in standard form
  if
• 1. Non-negativity constraints for all variables,
• 2. All inequality constraints are replaced by
  equality constraints, and
• 3. The right hand side vector, b, is non-negative.

not equality
not equality
x3 may be negative
51
Standardization Strategy 1 Converting
Inequalities into Equalities Plus Non-negatives
s1 is called a slack variable, which measures
the amount of unused resource. Note that s1
5 - x1 - 2x2 - x3 x4.
To convert a ? constraint to an equality, add a
slack variable.
52
Standardization Strategy 2 Converting ?
constraints
Consider the inequality -2x1 - 4x2 x3 x4 ?
-1 Step 1. Eliminate the negative RHS
2x1 4x2 - x3 - x4 ? 1 Step 2. Convert
to an equality 2x1 4x2 - x3 -
x4 s2 1 s2 ?
0 The variable added will be called a surplus
variable.
To covert a ? constraint to an equality,
subtract a surplus variable.
53
Standardization Strategy 3 Maximization to
Minimization

• How can one convert a maximization problem to a
  minimization problem?
• Example Maximize 3W 2P
• Subject to constraints

Has the same optimum solution(s) as
Minimize -3W -2P Subject to constraints
54
Standardization Strategy 4 Non-negativity
constraints for all variables
Transforming variables that may take on negative
values. maximize 3x1 4x2 5 x3 subject
to 2x1 - 5x2 2x3 7
other constraints x1 ??0, x2 is
unconstrained in sign, x3 ? 0

• Transforming x1 replace x1 by y1 -x1 y1 ?
  0.
• One can recover x1 from y1.

max -3 y1 4x2 5 x3 -2 y1 -5 x2 2 x3
7 y1 ? ?0, x2 is unconstrained in sign, x3 ? 0
55
Another Example (group discussion)

• Exercise transform the following to standard
  form (maximization)
• Minimize x1 3x2
• Subject to 2x1 5x2 ? ? 12
• x1 x2 ? 1
• x1 ? 0

56
Another Example (answer to group discussion)

• Exercise transform the following to standard
  form (maximization)
• Minimize x1 3x2
• Subject to 2x1 5x2 s1 ? 12
• x1 x2 - s2 1
• x1, s1, s2 ? 0

57
A General Standard Form
Suppose an LP with m constraints and n
variables has been converted into standard form.
The form of such an LP is
max ( or min) z c1x1 c2x2 cnxn s.t.
a11x1 a12x2 a1nxn b1 a21x1 a22x2
a2nxn b2 . .
. . . . am1x1
am2x2 amnxn bm xi 0 ( i 1,2, , n)
58
Resume Tuesdays Class

• Definition of linear programming
• Graphic Method
• Two Steps feasible region and optimization.
• Two Observations convex and extreme point
• Preparation for Simplex Method
• Gauss-Jordan
• Ero (elementary row operation)
• Basic variables (BV) and nonbasic variables
  (NBV).
• Standard Form
• Inequality -gt Equality.
• All the variable and RHS are non-negative.

59
Simplex Method

• Derivation of Simplex Algorithm

60
Features of Simplex Method

• The simplex algorithm, which was discovered
  in1947 by George Dantzig, is a simple,
  straightforward method for solving linear
  programming problems.
• It has proved to be remarkably efficient method
  that is used to solve large-scale problems.
• Simplex method is a iterative method by beginning
  with a feasible solution and tests whether or not
  it is optimum. If not, the method proceeds a
  better solution.
• The first step in the setting up the simplex
  method is to convert inequality constraints into
  equality constraints. This conversion can be
  succeeded by introducing slack variables.

61
Preview of the Simplex Algorithm
If we define
The constraints may be written as a system of
equations Ax b.
Consider a system Ax b of m linear equations in
n variables (where n m). A basic solution to
Ax b is obtained by setting n m variables
equal to 0 and solving for the remaining m
variables. This assumes that setting the n m
variables equal to 0 yields a unique value for
the remaining m variables, or equivalently, the
columns for the remaining m variables are
linearly independent.
62
Preview of the Simplex Algorithm
Define
The constraints may be written as a system of
equations Ax b.
Consider a system Ax b of m linear equations in
n variables (where n m). A basic solution to
Ax b is obtained by setting n m variables
equal to 0 and solving for the remaining m
variables. This assumes that setting the n m
variables equal to 0 yields a unique value for
the remaining m variables, or equivalently, the
columns for the remaining m variables are
linearly independent.
63
Principle of Simplex Algorithm BFS and Extreme
Points (1)
Theorem 1 The feasible region for any linear
programming problem is a convex set. Also, if an
LP has an optimal solution, there must be an
extreme point of the feasible region that is
optimal.
Any basic solution in which all variables are
nonnegative is called a basic feasible solution (
or bfs).
Theorem 2 For any LP, there is a unique extreme
point of the LPs feasible region corresponding
to each basic feasible solution. Also, there is
at least one bfs corresponding to each extreme
point in the feasible region.
64
BFS and Extreme Points (2)
max z 4x1 3x2 (income) s.t. x1
x2 40 (Leather) 2x1 x2 60
(Labor) x1, x2 0
max z 4x1 3x2 s.t. x1 x2 s1
40 2x1 x2 s2 60
x1, x2, s1, s2 0
Two kinds of leather bags are produced (x1, x2).
Both inequalities are satisfied in the shaded
area. The extreme points are of the feasible
region are B, C, E, and F.
65
BFS and Extreme Points (3)
Basic Variables Nonbasic Variables Basic Feasible Solution Corresponds to Corner Point
x1, x2 s1, s2 s1 s2 0, x1 x2 20 E
x1, s1 x2, s2 x2 s2 0, x1 30, s1 10 C
x1, s2 x2, s1 x2 s1 0, x1 40, s2 - 20 ANot a bfs since s2 lt 0
x2, s1 x1, s2 x1 s2 0, s1 - 20 x2 60 DNot a bfs since s1 lt 0
x2, s2 x1, s1 x1 s1 0, x2 40, s2 20 B
s1, s2 x1, x2 x1 x2 0, s1 40, s2 60 F
The table above shows the correspondence between
the basic feasible solutions to the LP and the
extreme points of the feasible region. The basic
feasible solutions to the standard form of the LP
correspond in a natural fashion to the LPs
extreme points.
66
BFS and Extreme Points (4) Adjacent Basic
Feasible Solutions
For any LP with m constraints, two basic feasible
solutions are said to be adjacent if their sets
of basic variables have m 1 basic variables in
common.

• For example in the Leather Limited LP on the
  previous slide, the bfs corresponding to point E
  is adjacent to the bfs corresponding to point C.
  These points share (m 1 2 - 1 1) one basic
  variable, x1. Points E (BV x1,x2) and F (BV
  s1,s2) are not adjacent since they share no
  basic variables.
• Intuitively, two basic feasible solutions are
  adjacent if they both lie on the same edge of the
  boundary of the feasible region.

67
Go Through All the Extreme Point?

• Along the adjacent extreme points, we can access
  all extreme points (or BFS).
• It is mathematically inhibitive for large scale
  problem.
• Remedy Simplex algorithm
• optimization test during browsing
• Try the most possible extreme point first if
  failed, try the most possible neighbor corners
• Avoid the non-feasible.

68
The Simplex Algorithm (max LPs)

• Step 1 Convert the LP to standard form
• Step 2 Obtain a bfs (if possible) from the
  standard form
• Step 3 Determine whether the current bfs is
  optimal
• Step 4 If the current bfs is not optimal,
  determine which nonbasic variable should be come
  a basic variable and which basic variable should
  become a nonbasic variable to find a bfs with a
  better objective function value.
• Step 5 Use eros to find a new bfs with a
  better objective function value. Go back to Step
  3.

In performing the simplex algorithm, write the
objective function in the form
z c1x1 c2x2 - - cnxn 0
We call this format the row 0 version of the
objective function (row 0 for short).
69
Simplex Method

• A 3D Linear Programming Example

70
General description of the simplex algorithm
solving an LP in a maximization problem

• Step 1 Find a bfs (initial bfs) to the LP.
  The most recent bfs will be called the current
  bfs.

Simplex iteration
Step 2 Determine if the current bfs is an
optimal solution. If not, find an adjacent bfs
that has a larger z-value.
Step 3 Return to Step 2, using the new bfs as
the current bfs.
71
A Mixed Product Problem The Amount of Resource
Per Unit
The Greensboro Furniture company manufactures
desk, tables, and chairs. The manufacturer of
each type of furniture requires lumber and two
types of skilled labor finishing and carpentry.
The amount of each resource needed to make each
type of furniture is given in the table below.
Resource Desk Table Chair
Lumber 8 board ft 6 board ft 1 board ft
Finishing hours 4 hours 2 hours 1.5 hours
Carpentry hours 2 hours 1.5 hours 0.5 hours
72
A Mixed Product Problem Constrains on Resource
At present, 48 board feet of lumber, 20 finishing
hours, 8 carpentry hours are available. A desk
sells for 60, a table for 30, and a chair for
20. Greensboro believes that demand for desks
and chairs is unlimited, but at most 5 tables can
be sold. Since the available resources have
already been purchased, Greensboro wants to
maximize total revenue.
73
A Mixed Product Problem Mathematics Formulation
Define x1 number of desks produced x2
number of tables produced x3 number of chairs
produced.
The LP is max z 60x1 30x2 20x3
s.t. 8x1 6x2 x3 48 (lumber
constraint) 4x1 2x2 1.5x3 20
(finishing constraint) 2x1
1.5x2 0.5x3 8 (carpentry
constraint) x2 5
(table demand constraint)
x1, x2, x3 0
74
Step 0-1 Initial Standard Form
Canonical Form 0 Basic Variable
Row 0 z 60x1 30x2 20x3 0 z 0
Row 1 8x1 6x2 x3 s1 48 s1 48
Row 2 4x1 2x2 1.5x3 s2 20 s2 20
Row 3 2x1 1.5x2 0.5x3 s3 8 s3 6
Row 4 x2 s4 5 s4 5
If we set x1 x2 x3 0, we can solve for the
values s1, s2, s3, s4. Thus, BV s1, s2, s3,
s4 and NBV x1, x2, x3 . Since each
constraint is then in canonical form (BVs have a
coefficient 1 in one row and zeros in all other
rows) with a nonnegative rhs, a bfs can be
obtained by inspection.
75
Step 0-2 Obtain the Initial Basic Feasible
Solution
To perform the simplex algorithm, we need a basic
(although not necessarily nonnegative) variable
for row 0. Since z appears in row 0 with a
coefficient of 1, and z does not appear in any
other row, we use z as the basic variable. With
this convention, the basic feasible solution for
our initial canonical form has
BV z, s1, s2, s3, s4 and NBV x1, x2, x3 .
For this initial bfs, z 0, s1 48, s2 20, s3
8, s4 5, x1 x2 x3 0. As this example
indicates, a slack variable can be used as a
basic variable if the rhs of the constraint is
nonnegative.
76
Step 0-3 Determine if the Current BFS is Optimal
Once we have obtained a bfs, we need to
determine whether it is optimal. To do this, we
try to determine if there is any way z can
be increased by increasing some nonbasic variable
from its current value of zero while holding all
other nonbasic variables at their current values
of zero. Solving for z in row 0 yields
Z 60x1 30x2 20x3
For each nonbasic variable, we can use the
equation above to determine if increasing a
nonbasic variable (while holding all other
nonbasic variables to zero) will increase z.
Increasing any of the nonbasic variables will
cause an increase in z. However increasing x1
causes the greatest rate of increase in z. If x1
increases from its current value of zero, it will
have to become a basic variable. For this
reason, x1 is called the entering variable.
Observe x1 has the most negative coefficient in
row 0.
77
Comments about Step 0-3 Ratio Test

• When entering a variable into the basis, compute
  the ratio
• rhs of row / coefficient of entering variable
  in row
• For every constraint in which the entering
  variable has a positive coefficient. The
  constraint with the smallest ratio is called the
  winner of the ratio test. The smallest ratio is
  the largest value of the entering variable that
  will keep all the current basic variables
  nonnegative.
• Make the entering variable x1 a basic variable in
  row 3 since this row (constraint) was the winner
  of the ratio test (8/2 4).

78
Step 0-4 Choose the Entering Variable to the
Nonbasic Variable

• Choose the entering variable (in a max problem)
  to the nonbasic variable with the most negative
  coefficient in row 0 (ties broken arbitrarily).
  We desire to make x1 as large as possible but as
  we do, the current basic variables (s1, s2, s3,
  s4) will change value. Thus, increasing x1 may
  cause a basic variable to become negative.
  
• RATIO
• From row 1 we see that s1 48 8x1. Since s1
  0, x1 48 / 8 6
• From row 2, we see that s2 20 4x1. Since s2
  0, x1 20 / 4 5
• From row 3, we see that s3 8 2x1. Since
  s3 0, x1 8 / 2 4
• From row 4, we see that s4 5. For any x1, s4
  will always be 0
• This means to keep all the basic variables
  nonnegative, the largest we can make x1 is min
  6, 5, 4 4.

79
Step 0-5A Using eros to Turn Pivoting Term Into
Basic Variable

• To make x1 a basic variable in row 3, we use
  elementary row operations (eros) to make x1 have
  a coefficient of 1 in row 3 and a coefficient of
  0 in all other rows. This procedure is called
  pivoting on row 3 and row 3 is called the pivot
  row. The final result is that x1 replaces s3 as
  the basic variable for row 3. The term in the
  pivot row that involves the entering basic
  variable is called the pivot term.
• Step 5 - The Gauss-Jordan method using eros and
  simplex tableaus shown on the next slide makes x1
  a basic variable.

80
Step 0-5B Use eros and Simplex Tableaus to
transform x1 into basic variable
Pivot row
81
Step 1-0 Standard Form After 0th Iteration
Canonical Form 1 Basic Variable
Row 0 z 15x2 - 5x3 30s3 240 z 240
Row 1 - x3 s1 - 4s3 16 s1 16
Row 2 - x2 0.5 x3 s2 - 2 s3 4 s2 4
Row 3 x1 0.75x2 0.25x3 0.5s3 4 x1 4
Row 4 x2 s4 5 s4 5
BV z, s1, s2, x1, s4 and NBV s3, x2, x3 .
yielding the bfs z 240, s1 16, s2 4, x1
4, s4 5, s3 x2 x3 0.
The procedure from going to one bfs to a better
adjacent bfs is called an iteration (or sometime
pivot) of the simplex algorithm.
82
Step 1-3 Determine If the Current BFS is Optimal
Rearranging row 0 from Canonical Form 1, and
solving for a yields
z 240 15x2 5x3 -30s3
The current bfs is NOT optimal because increasing
x3 to 1 (while holding the other nonbasic
variable to zero) will increase the value of z.
Making either x2 or s3 basic will caused the
value of z to decrease.
83
Step 1-4 The Simplex Algorithm (max LPs)

• The entering variable is the row 0 coefficient
  with the greatest negative value. Since x3 is
  the only variable with a negative coefficient, x3
  should be entered into the basis.
• Performing the ratio test using x3 as the
  entering variable yields the following results
  (holding other NBVs to zero)

From row 1, s1 0 for all values of x3 since s1
16 x3 From row 2, s2 0 if x3 4 / 0.5
8 From row 3, x1 0 if x3 4 / 0.25 16 From
row 4, s4 0 for all values of x3 since s4 5
This means to keep all the basic variables
nonnegative, the largest we can make x1 is min
8,16 8. So, row 2 becomes the pivot row.
84
Step 1.5 Use eros to make x3 a basic variable
in row 2.
Pivot row
85
Step 2-0 Standard Form After 1st Iteration
Canonical Form 2 Basic Variable
Row 0 z 5x2 10s2 10s3 280 z 280
Row 1 - 2x2 s1 2s2 - 8s3 24 s1 24
Row 2 - 2x2 x3 2s2 - 4s3 8 x3 8
Row 3 x1 1.25x2 - 0.5 s2 1.5s3 2 x1 2
Row 4 x2 s4 5 s4 5
In Canonical Form 2, BV z, s1, x3, x1, s4 and
NBV s3, s2, x2 , yielding the bfs z 280, s1
24, x3 8, x1 2, s4 5, s2 s3 x2 0.
86
2.3 Determine Optimal
Solving for z in row 0 yields
Z 280 - 5x2 - 10s2 -10s3
We can see that increasing x2, s2, or s3 (while
holding the other NBVs to zero) will not cause
the value of z to decrease. The solution at the
end of iteration 2 is therefore optimal. The
following rule can be applied to determine
whether a canonical forms bfs is optimal
A canonical form is optimal (for a max problem)
if each nonbasic variable has a nonnegative
coefficient in the canonical forms row 0.
87
Advanced Discussions
88
Taxonomy of Optimization Problems
89
Thank You!!!
Lecture Notes
http//www-users.cs.umn.edu/yliang/teaching.html

• Yu Liang