DUALITY THEORY

1
DUALITY THEORY

• CONTENTS
• Defining the Dual Problem
• The Weak Duality Theorem
• The Strong Duality Theorem
• Simplex Algorithm and Dual Variables
• Complementary Slackness Conditions
• Dual Simplex Method
• Reference Chapter 6 in Bazaraa, Jarvis and
  Sherali.

2
Lower and Upper Bounds

• Maximize 4x1 x2 3x3
• subject to
• x1 4x2 ? 1
• 3x1 - x2 x3 ? 3, x1, x2, x3 ? 0
• A feasible solution x1 0, x2 0, x3 3, z
  9
• Each feasible solution gives a lower bound on the
  optimal objective function value.
• We can also obtain an upper bound easily.
• 2 (x1 4x2 ) ? (1) 2
• 3 (3x1 - x2 x3 ) ? (3) 3
• 11x1 5x2 3x3 ? 11
• Observe that 4x1 x2 3x3 ? 11x1 5x2 3x3 ?
  11. Hence z ? 11

3
Lower and Upper Bounds (contd.)

• Maximize 4x1 x2 3x3
• subject to
• x1 4x2 ? 1
• 3x1 - x2 x3 ? 3, x1, x2, x3 ? 0
• 4x1 x2 3x3 ? 11x1 5x2 3x3 ? 11. Hence z
  ? 11.
• How can we get the smallest possible upper bound?
• Find y1 and y2 such that
• ?1 (x1 4x2 ) ? (1) ?1
• ?2 (3x1 - x2 x3 ) ? (3) ?2
• (?13?2)x1 (4?1-?2)x2 (?2)x3 ? ?1 3?2
• ?1 3?2 ? 4, 4?1- ?2 ? 1, ?2 ? 3, ?1 ? 0, ?2 ? 0
  and ?13?2 is minimum

4
Defining the Dual

• Dual LP Primal LP
• Minimize ?1 3?2 Maximize 4x1 x2
  3x3
• subject to subject to
• ?1 3?2 ? 4 x1 4x2 ? 1
• 4?1 - ?2 ? 1 3x1 - x2 x3 ?
  3
• ?2 ? 3 x1, x2, x3
  ? 0
• ?1 ? 0, ?2 ? 0

5
The Dual Problem

• PRIMAL LP
• Maximize ?j1,n cjxj
• subject to
• ?j1,n aij xj ? bi for all i 1, 2, , m
  
• xj ? 0 for all j 1, 2, , n
• THE ASSOCIATED DUAL LP
• Minimize ?i1,m bi?i
• subject to
• ?i1,m ?i aij ? cj for all j 1, 2, , n
  
• ?j ? 0 for all i 1, 2, , m

6
The Dual Problem

• THE DUAL LP
• - maximize ?i1,m (-bi) ?i
• subject to
• ?i1,m (-aij) ?i ? (-cj) for all j
  1, 2, , n
• ?j ? 0 for all i 1, 2, , m
• DUAL OF THE DUAL LP
• -minimize ?j1,n (-cj)xj
• subject to
• ?j1,n (-aij) xj ? (-bj) for all i
  1, 2, , m
• xj ? 0 for all j 1, 2, , n
• which is the same as the primal problem.

7
A Diet Problem

• My diet consists of four items Brownie, ice
  cream, soda, and cheesecake.
• Each day, I must ingest at least 500 calories, 6
  oz. of chocolate, 10 oz. of sugar, and 8 oz. of
  fat.
• The nutritional contents of each type of per unit
  of food are given below. Find the minimum cost
  diet plan.

8
A Pill Manufacturers Problem

• A pill manufacturer is planning to manufacture
  pills for calories, chocolates, sugar, and fat.
• He wants to set the prices for these pills which
  maximizes his revenue and at the same time the
  pills are more competitive than each of the items
  I consume (brownie, chocolate ice cream, cola,
  and pineapple cheese cake)

9
The Weak Duality Theorem

• THEOREM If (x1, x2, , xn) is feasible for the
  primal problem and (?1, ?2, , ?m) is feasible
  for the dual problem, then
• ?j1,n cjxj ? ?i1,m ?i bi
• PROOF
• ?j1,n cjxj ? ?j1,n (?i1,m ?i aij) xj
• ?j1,n ?i1,m ?i aij xj
• ?i1,m (?j1,n aijxj ) ?i
• ? ?i1,m bi ?i

10
Implications of the Weak Duality Theorem

• THEOREM If (x1, x2, , xn) is feasible for the
  primal problem and (?1, ?2, , ?m) is feasible
  for the dual problem, and ?j1,n cjxj
  ?i1,m bi ?i, then x is optimal to the primal
  problem and ? is optimal to the dual problem.

11
Strong Duality Theorem

• THEOREM If the primal problem has an optimal
  solution
• then the dual also has an optimal solution
• such that
• PROOF To be proved using the simplex algorithm.

12
Strong Duality Theorem (contd.)

• The simplex algorithm maintains a set of dual
  variables ? at each iteration. Simplex
  multipliers are these dual variables.
• For the initial solution ? 0. As it performs
  iterations and updates BFS, the associated set of
  dual variables changes.
• PRIMAL LP
• Maximize ?j1,n cjxj
• subject to ?j1,n aij xj ? bi for all i
  1, 2, , m
• xj ? 0 for all j 1, 2, ,
  n
• or
• Maximize ?j1,n cjxj ?i1,m 0wj
• subject to ?j1,n aij xj wi bi for all i
  1, 2, , m
• xj ? 0 for all j 1, 2, ,
  n

13
Strong Duality Theorem (contd.)

• Maximize ?j1,n cjxj ?in1,nm 0xj
• subject to ?j1,n aij xj xni bi for all i
  1, 2, , m
• xj ? 0 for all j 1, 2, ,
  nm
• Alternatively,
• Maximize ?j1,nm cjxj
• subject to ?j1,nm aij xj - bi 0 for all i
  1, 2, , m
• xj ? 0 for all j 1, 2, ,
  nm
• As the algorithm proceeds, we modify the
  objective function by performing elementary row
  operations - we multiply the ith constraint by ?i
  and add it to the objective function.

14
Strong Duality Theorem (contd.)

• The modified objective function
• z ?i1,m bi ?i ?j1,nm (cj - ?i1,m aij
  ?i) xj
• ?i1,m bi ?i - ?j1,nm xj
• Optimality criteria ? 0, or
• ?i1,m aij ?i ? cj
  (These are the dual constraints.)
• The objective function value of the current
  primal solution
• z ?i1,m bi ?i (This is also the value
  of dual obj.)
• Primal objective function value equals the dual
  objective function value, establishing the strong
  duality theorem.

15
Complementary Slackness Theorem

• THEOREM Suppose that x (x1, x2, , xn) is
  primal feasible and that ? (?1, ?2, , ?n) is
  dual feasible. Let (w1, w2, , wm) denote the
  corresponding primal slack variables, and let
  (z1, z2, , zn) denote the corresponding dual
  slack variables. Then, x and y are optimal for
  their respective problems if and only if
• xjzj 0 for j 1, 2, , n, (1)
• wi ?i 0 for i 1, 2, , m. (2)
• PROOF
• Show that ?j1,n cjxj ?j1,m bi ?i if and only
  if (1) and (2) are satisfied.

16
Complementary Slackness Theorem (contd.)

• PRIMAL LP
• Maximize ?j1,n cjxj
• subject to
• ?j1,n aij xj ? bi for all i 1, 2, , m (dual
  var. yi slack wi)
• xj ? 0 for all j 1, 2, , n
• DUAL LP
• Minimize ?i1,m bi ?i
• subject to
• ?i1,m ?i aij ? cj for all j 1, 2, , n (primal
  var. xj slack zj)
• ?j ? 0 for all i 1, 2, , m

17
Dual for a Problem in General Form

• PRIMAL LP
• Maximize ?j1,n cjxj
• subject to
• ?j1,n aij xj bi for all i 1, 2, , m
• xj ? 0 for all j 1, 2, , n
• Equivalent Primal LP
• Maximize ?j1,n cjxj
• subject to
• ?j1,n aij xj ? bi for all i 1, 2, , m
• - ?j1,n aij xj ? -bi for all i 1, 2, , m
• xj ? 0 for all j 1, 2, , n

18
Dual for a Problem in General Form (contd.)

• Primal LP
• Maximize ?j1,n cjxj
• subject to
• ?j1,n aij xj ? bi for all i 1, 2, , m
  (Dual variable )
• - ?j1,n aij xj ? -bi for all i 1, 2, , m
  (Dual variable )
• xj ? 0 for all j 1, 2, , n
• DUAL LP
• Minimize ?i1,m bi - ?i1,m
  bi
• subject to
• ?i1,m aij - ?i1,m aij ? cj for all
  j 1, 2, , n
• , ? 0 for all i 1, 2, , m

19
Dual for a Problem in General Form (contd.)
DUAL LP Minimize Subject to for all

for all EQUIVALENT DUAL LP Minimize Subj
ect to for all where
20
Rules for Forming the Dual
21
Dual Simplex Method

• The dual simplex is an alternate method to solve
  linear programming problems.
• The method starts with a BFS which satisfies
  optimality conditions (dual feasible) but
  violates non-negativity feasibility conditions
  (is not primal feasible).
• The method performs a sequence of pivot
  operations where it maintains optimality
  conditions throughout and restores feasibility
  conditions.
• For several problems, dual simplex method runs
  faster than the primal simplex method.
• Dual simplex method is very useful for performing
  sensitivity analysis.

22
Dual Simplex Method (contd.)

• Initial simplex tableau
• z - 60x1 - 30x2 - 20x3
  0
• 8x1 6x2 x3 s1
  48
• 4x1 2x2 1.5x3
  s2 20
• 2x1 1.5x2 0.5x3
  s3 8
• Optimal tableau
• z 5x2
  10s2 10s3 280
• - 2x2 s1 2s2
  - 8s3 24
• - 2x2 x3
  2s2 - 4s3 8
• x1 1.25x2 -
  0.5s2 1.5s3 2
• In our LP, 20 hours of finishing time is
  available. Suppose that we 30 hours of finishing
  time instead of 20 hours.
• New updated RHS vector for this basis B-1b
  44, 28, -3 which is primal infeasible.

23
Dual Simplex Method (contd.)

• Dual simplex method allows us to restore primal
  infeasibility of a solution while maintaining its
  dual infeasibility.

24
Steps of the Dual Simplex Method

• Step 1. If the right-hand side of each constraint
  is nonnegative, an optimal solution has been
  found otherwise go to Step 2.
• Step 2. Choose the most negative basic variable
  as the leaving variable. This row is the pivot
  row. Select the entering variable as the nonbasic
  variable which has a negative coefficient in the
  pivot row and for which the following ratio
• is minimum. Perform the pivot operation as
  usual.
• Step 3. If there is no negative coefficient in
  the pivot row, then the LP is infeasible.

25
Uses of the Dual Simplex Method

• The dual simplex method is often used in the
  following situations
• Finding the new optimal solution after changing
  an LPs right-hand side.
• Solving a normal maximization or minimization
  problem.
• When a new constraint is added.

26
Solving a Normal LP by the Dual Simplex Method

• ORIGINAL LP
• Minimize z 4x1 x2
• subject to
• x1 4x2 ? 1
• 3x1 - x2 ? 3,
• x1, x2 ? 0
• LP with Surplus Variables
• Minimize z 4x1 x2
• subject to
• x1 4x2 - s1 1
• 3x1 - x2 - s2 3,
• x1, x2 ? 0

27
Solving a Normal LP by Dual Simplex Method
(contd.)

• LP with Surplus Variables
• Minimize z 4x1 x2
• subject to
• - x1 - 4x2 s1 -1
• -3x1 x2 s2 -3,
• x1, x2 ? 0
• We can apply dual simplex method as this
  solution as the starting solution.

28
Adding a New Constraint

• When a new constraint is added, one of the
  following three
• cases will occur
• Case 1 The current solution satisfies the new
  constraint.
• The current solution is optimal for the new LP.
• Case 2 The current solution does not satisfy the
  new constraint, but the LP still has a feasible
  solution.
• We perform dual simplex method to reoptimize.
• Case 3 The additional constraint causes the LP
  to have no feasible solution.
• We perform dual simplex method to identify this
  possibility.

29
Adding a New Constraint (contd.)

• Consider the following LP
• z 5x2
  10s2 10s3 280
• - 2x2 s1 2s2
  - 8s3 24
• - 2x2 x3
  2s2 - 4s3 8
• x1 1.25x2 -
  0.5s2 1.5s3 2
• Additional Constraint
• x1 x2 ? 12
• Or, equivalently
• x1 x2 s4 12
• or
• -x1 - x2 s4 -12

30
Adding a New Constraint (contd.)

• Add a row and make s4 as the basic variable of
  that row.
• Obtain the new B-1
• Obtain the new B-1b
• Perform dual pivot operations to restore the
  primal feasibility of the solution.