Optimization Mechanics of the Simplex Method

1
OptimizationMechanics of the Simplex Method
2
Simplex Method

• Find initial corner point (basic) feasible
  solution.
• Usually, the origin is a basic feasible solution.
• Iterate until the stopping conditions are met.
• Are we optimal yet? Look at the current version
  of the objective function to see if an entering
  basic variable is available. If not, then the
  current basic feasible solution is the optimum
  solution.
• Select the entering variable choose the
  non-basic variable that gives the fastest rate of
  increase in the objective function value.
• Select the leaving variable by applying the
  Minimum Ratio Test.
• Update the equations to reflect new basic
  feasible solution.
• Go to step (a).

3
Tableau

• First, we create a "tableau" out of the augmented
  system of equations and pick the origin as the
  starting point.

4
Tableau (Proper Form)

• Characteristics of a tableau in proper form
• Exactly 1 basic variable per equation.
• The coefficient of the basic variable is always
  exactly 1, and the coefficients above and below
  the basic variable in the same column are all 0.
• Z is treated as the basic variable for the
  objective function row.

5
Tableau (Proper Form)

• When a tableau is in the proper form
• The RHS values are the value of the basic
  variables.
• What is the value of x1?
• x1 is a non-basic variable, so its value is 0.
• What is the value of Z?
• 0. (Z is a basic variable, so look at the RHS
  column)

6
Are we optimal yet?

• We are optimal if no entering basic variable is
  available (i.e., no basic variable, if
  introduced, can increase the value of Z).
• If there is any negative coefficient in the
  objective function row, then the current basic
  solution is not optimal.

7
Select the entering basic variable

• Select the variable with the most negative
  coefficient in the objective function row as the
  entering variable.
• We won't accidentally select a basic variable.
  Why?
• The column corresponds to the entering basic
  variable becomes the pivot column.

8
Select the leaving basic variable

• Perform Minimum Ratio Test (MRT)
• Special cases If the coefficient of the entering
  basic variable is not a positive number, enter
  "no limit".
• Select the variable corresponding to the smallest
  MRT value as the leaving basic variable. Call
  the corresponding row the pivot row.

9
Updating the tableau

• Step 1 Replace the leaving basic variable in the
  "Basic" column by the entering basic variable.
• Step 2 Turn the tableau into a proper form
• How?

10
Updating the tableau

• The coefficient of all the basic variables must
  be 1.
• Divide the row corresponds to the entering basic
  variable by its coefficient.
• The coefficients above and below the basic
  variable in the same column must be turned into
  0.
• Apply (Gaussian) elimination steps

11
End of iteration 1

• When x1 0, x2 6, Z 1050.
• Note The columns correspond to the "old" basic
  variables do not change. (Why?)
• Are we optimal yet?

12
Iteration 2

• Select the entering basic variable x1
• Perform MRT test and select the leaving basic
  variable S1

13
Iteration 2

• Updating tableau
• Replace S1 by x1 in the "Basic" column
• Divide pivot row by coefficient of x1 (7)

14
Iteration 2

• Updating tableau (continue)
• Apply elimination steps.
• (End of iteration 2)
• Are we optimal yet?

15
Iteration 3

• Select the entering basic variable S4
• Perform MRT test and select the leaving basic
  variable S2

16
Iteration 3

• Updating tableau
• Replace S2 by S4 in the "Basic" column
• Divide pivot row by coefficient of S4 (54/7)

17
Iteration 3

• Updating tableau (continue)
• Apply elimination steps.
• (End of iteration 3)
• Are we optimal yet?

18
Iteration 3

• Because there is no more negative coefficient in
  the objective function row, we cannot further
  improve the value of the objective function.
• The optimal value of the objective function is
  1413.889 when x1 4.889 and x2 3.889.

19
Special Cases in Tableau Manipulation

• In case of tie for the entering basic variable,
  select any of the tying variables.
• When will a tie occur for the entering variable?
• In case of tie for the leaving basic variable,
  select any of the tying variables.
• When will a tie occur for the leaving variable?
• What does it mean when all MRT yield "No limit"?
• Increasing the value of the entering variable
  does not affect any variable. i.e., the problem
  is unbounded.

20
Special Cases in Tableau Manipulation

• Note Here the coefficient is referring to the
  coefficient of a variable in the objective
  function row in the tableau.
• At the optimum, what's the coefficient of a
  non-basic variable?
• At the optimum, what's the coefficient of a basic
  variable?
• What does it mean when the coefficient of a
  non-basic variable is zero at the optimum?
• We have alternate optimal solutions. i.e.,
  increasing the value of that non-basic variable
  does not increase or decrease the value of Z.

21
Directions References

• Solving non-standard linear programming.
• Equality and greater-than-or-equal-to ()
  constraints
• Variables that can be negative or unrestricted
  variables
• Sensitivity Analysis
• LP in practice Revised Simplex method, interior
  point methods.
• Ref http//www.sce.carleton.ca/faculty/chinneck/p
  o.html

22
Summary

• Simplex Method for solving "Standard Form" LP
  problems
• Algorithm
• Understand the characteristics of the slack
  variables
• Understand how to select entering/leaving
  variables
• Performing simplex method in tableau form
• Understand the characteristics of the tableau
  (which can give you additional info of the
  problem).

23
Summary Optimization

• One dimensional unconstrained optimization
  function with one variable
• Newton's Method for finding x s.t. f'(x) 0.
• Bracketing methods (Iterative)
• Golden Section Search
• Quadratic Interpolation
• Hybrid methods

24
Summary Optimization

• Multi-dimensional unconstrained optimization
  function with 2 or more variables
• Random Search
• Its advantages and disadvantages
• The "General Iterative Algorithm" for finding
  optimal point using either non-gradient and
  gradient methods.
• Non-gradient Methods
• Univariate Search and Powell's Method
• Gradient Methods
• Evaluate partial derivative
• Derive gradients and Hessian matrix
• Determine if a point is a maximum, minimum, or
  saddle point using Hessian matrix
• Finding optimal point in a given direction
• Steepest Ascent and Newton's Method