CS5321 Numerical Optimization

1
CS5321 Numerical Optimization

• 14 Linear Programming The Interior Point
  Method

2
Interior point methods for LP

• There are many variations of IPM for LP
• Primal IPM, dual IPM, primal-dual IPM
• Potential Reduction Methods
• Path Following Methods
• Short-step methods
• Long-step methods
• Predictor-corrector methods
• We will cover the path following method and the
  long-step method

3
The dual problem of LP

• minxzcTx subject to Ax b, x ? 0
• The dual problem is (see chap 12)
• max?bT? subject to AT ? s c, s ? 0
• Weak duality
• For any x and ? that are feasible in the primal
  and the dual problems, bT? ? cTx.
• Strong duality
• If x and ? are solutions to the primal and
  the dual problems, bT? cTx.

4
Optimality conditions of LP

• minxzcTx subject to Ax b, x ? 0
• The Lagrangian function
• L(x,?,s) cTx - ?T(Ax-b) - sTx
• The optimality conditions (KKT, see chap 12)
• The last one is the complementarity condition

5
Primal-dual method

• The optimality conditions can be expressed as
• Xdiag(x), Sdiag(s) and e(1,1,,1)T.
• F is a nonlinear equation R2nm ? R2nm.
• Solved by the Newtons method (J the Jacobian
  of F.)
• (x, ?, s)?(?x, ??, ?s),

6
Jacobian of F

• Let .
• The Jacobian J (x, ?, s) of F(x, ?, s) is

7
Solving the linear systems

• The Newtons direction is obtained by solving
• S is an artificial variable. Let
• The left hand side is symmetric, but indefinite.

8
Centering

• In practice, rXSXSe -??e rather than rXSXSe.
• Duality measure
• Centering parameter ? ? 0,1
• Decreasing as approaching solutions
• This is related to the barrier method (chap 15)

9
The path following algorithm

• Given (x0,?0,s0) with (x0,s0)gt0
• For k 0, 1, 2 until (xk)Tsklt ?
• Choose ?k?0,1 and let ?k(xk)Tsk/n.
• Solve the Newtons direction (?xk, ??k, ?sk)
• Set (xk1, ?k1, sk1)(xk, ?k, sk) ?k(?xk,
  ??k, ?sk), where ?k is chosen to make
  (xk1,sk1)gt0

10
Central path and neighborhood

• Feasible set F0(x,?,s) Axb, AT?sc,
  (x,s)gt0
• Point (x?,??,s?)?F0 is on the central path C if
  x?(i)s?(i) ? for all i1,2,,n.
• An example of the neighborhood of the central
  path
• ? is the duality measure (two slides before)
• Typically ?10-3.

11
Long-step path-following

• Given (x0,?0,s0)?N-? and ?, ?min, ?max.
• For k 0, 1, 2 until (xk)Tsklt ?
• Choose ?k??min,?max and let ?k(xk)Tsk/n.
• Solve the Newtons direction (?xk, ??k, ?sk)
• Set (xk1, ?k1, sk1)(xk, ?k, sk) ?k(?xk,
  ??k, ?sk), where ?k is chosen to make (xk1,
  ?k1, sk1)?N-?.

12
Complexity

• The long step following method can converge in
  O(nlog(1/?)) iterations (Theorem 14.4)
• The most effective interior point method for LP
  is the predictor-corrector algorithm (Mehrotra
  1992)