ESI 4313 Operations Research 2

1
ESI 4313Operations Research 2

• Nonlinear Programming
• Multi-dimensional Problems
• Lecture 9

2
Multi-dimensional NLP

• A general multi-dimensional NLP is

3
Multi-dimensional derivatives

• As in the one-dimensional case, we can recognize
  convexity and concavity by looking at the
  objective functions derivatives
• The gradient vector is the vector of first-order
  partial derivatives

4
Recognizing multi-dimensional convex and concave
functions

• The Hessian matrix is the matrix of second-order
  partial derivatives

5
Recognizing multi-dimensional convex and concave
functions

• Suppose that
• H(x) exists for all x?S
• Then
• f(x) is a convex function on S if H(x) is
  positive semi-definite for all x?S
• f(x) is a concave function on S if H(x) is
  negative semi-definite for all x?S

6
Recognizing multi-dimensional convex and concave
functions

• Recall that
• a matrix H is positive definite if and only if
  zTHz gt 0 for all z?Rn /0
• a matrix H is negative definite if and only if
  zTHz lt 0 for all z?Rn /0
• a matrix H is positive/negative semi-definite if
  the strict inequalities are replaced by weak
  inequalities

7
Recognizing multi-dimensional convex and concave
functions

• An alternative characterization uses the concept
  of principal minors of a matrix
• A principal minor of order i of the matrix H is
  the determinant of an i?i submatrix obtained by
  removing n-i rows and the corresponding n-i
  columns

8
Recognizing multi-dimensional convex and concave
functions

• Suppose that
• H(x) exists for all x?S
• Then
• f(x) is a convex function on S if and only if all
  principal minors of H(x) are nonnegative for all
  x?S
• f(x) is a concave function on S if and only if
  the principal minors of H(x) of order k have the
  same sign as (-1)k for all x?S and all k

9
Unconstrained optimization

• A general unconstrained multi-dimensional NLP is

10
Unconstrained optimization

• A point x where ?f(x) 0 is called a stationary
  point of f
• These are called first-order conditions for
  optimality
• Let x be a stationary point, i.e., ?f(x) 0
• If H(x) is positive definite then x is a local
  minimum
• If H(x) is negative definite then x is a local
  maximum
• If H(x) is neither negative definite nor
  positive definite,
• If detH(x) 0 then x is a local minimum, local
  maximum, or saddle point
• If detH(x) ? 0 then x is not a optimum

11
Unconstrained optimization

• These characterizations can be strengthened a bit
  using the concept of leading principal minors
• The leading principal minor of order i of the
  matrix H is the determinant of the i?i submatrix
  formed by the first i rows and columns

12
Unconstrained optimization

• A point x where ?f(x) 0 is called a stationary
  point of f
• Let x be a stationary point, i.e., ?f(x) 0
• If all leading principal minors of H(x) are
  positive then x is a local minimum
• If the leading principal minors of H(x) of order
  k has the same sign as (-1)k (for all k) then x
  is a local maximum

13
Example 1

• There are two firms producing an item
• It costs Firm 1 q1 to produce q1 units
• It costs Firm 2 0.5q22 to produce q2 units
• If a total of q units are produced, consumers
  will pay 200 q per unit
• The manufacturers have decided to collaborate in
  order to maximize the sum of their profits
• Determine the optimal production quantities

14
Example 2

• Find and classify the stationary points of