Lagrange Multipliers

1
Lagrange Multipliers
2
Lagrange Multipliers

• The method of Lagrange multipliers gives a set of
  necessary conditions to identify optimal points
  of equality constrained optimization problems.
• This is done by converting a constrained problem
  to an equivalent unconstrained problem with the
  help of certain unspecified parameters known as
  Lagrange multipliers.
• The classical problem formulation
• minimize f(x1, x2, ..., xn)
• Subject to h1(x1, x2, ..., xn) 0
• can be converted to
• minimize L(x, l) f(x) - l h1(x)
• where
• L(x, v) is the Lagrangian function
• l is an unspecified positive or negative constant
  called the Lagrangian Multiplier

3
Finding an Optimum using Lagrange Multipliers

• New problem is minimize L(x, l) f(x) - l
  h1(x)
• Suppose that we fix l l and the unconstrained
  minimum of L(x l) occurs at x x and x
  satisfies h1(x) 0, then x minimizes f(x)
  subject to h1(x) 0.
• Trick is to find appropriate value for Lagrangian
  multiplier l.
• This can be done by treating l as a variable,
  finding the unconstrained minimum of L(x, l) and
  adjusting l so that h1(x) 0 is satisfied.

4
Method

• Original problem is rewritten as minimize
  L(x, l) f(x) - l h1(x)
• Take derivatives of L(x, l) with respect to xi
  and set them equal to zero.
• If there are n variables (i.e., x1, ..., xn) then
  you will get n equations with n 1 unknowns
  (i.e., n variables xi and one Lagrangian
  multiplier l)
• Express all xi in terms of Langrangian multiplier
  l
• Plug x in terms of l in constraint h1(x) 0 and
  solve l.
• Calculate x by using the just found value for l.
• Note that the n derivatives and one constraint
  equation result in n1 equations for n1
  variables!
• (See example 5.3)

5
Multiple constraints

• The Lagrangian multiplier method can be used for
  any number of equality constraints.
• Suppose we have a classical problem formulation
  with k equality constraints
• minimize f(x1, x2, ..., xn)
• Subject to h1(x1, x2, ..., xn) 0
• ......
• hk(x1, x2, ..., xn) 0
• This can be converted in
• minimize L(x, l) f(x) - lT h(x)
• where
• lT is the transpose vector of Lagrangian
  multpliers and has length k

6
In closing

• Lagrangian multipliers are very useful in
  sensitivity analyses (see Section 5.3)
• Setting the derivatives of L to zero may result
  in finding a saddle point. Additional checks are
  always useful.
• Lagrangian multipliers require equalities. So a
  conversion of inequalities is necessary.
• Kuhn and Tucker extended the Lagrangian theory to
  include the general classical single-objective
  nonlinear programming problem
• minimize f(x)
• Subject to gj(x) ? 0 for j 1, 2, ..., J
• hk(x) 0 for k 1, 2, ..., K
• x (x1, x2, ..., xN)