AED Economics 702: Computational Economics

1
AED Economics 702 Computational
Economics Nonlinear Programming (NLP) Methods and
Models Winston Venkataramanan Chapter 12.
2
Nonlinear Programming _ An Introduction

• Learn the differences between the LP and the
  nonlinear program (NLP).
• Review solution schemes or approaches for solving
  common nonlinear programming model.
• Understand the wide range of real applications
  for which NLPs are used.
• Use LINGO to solve the Quadratic Programming
  Problem

3
An Overview of Chapter 12 in WVThis is really
a heavy-duty math chapter. You should
concentrate on those sections denoted with a

• Section 12.1 Review of Calculus (functions and
  derivatives)
• Section 12.2 Introduction to NLP
• Section 12.3 Convex and Concave Functions
• Section 12.4 Solving NLP models with one
  variable
• Section 12.5 Golden Section Search
• Section 12.6 Unconstrained Maximization /
  Minimization
• Section 12.7 The Method of Steepest Ascent
• Section 12.8 Lagrange Multipliers
• Section 12.9 Karush-Kuhn-Tucker Conditions
• Section 12.10 Quadratic Programming

4
What is a NLP?

• NLPs are closer to general and realistic (and
  possibly unsolvable) models than the LPs. Some
  LPs are the linearized versions of NLPs out of
  necessity. NLPs have non-proportional and
  non-additive relationships.
• The most general mathematical model is likely to
  have nonlinear terms with random (and possibly
  dependent) coefficients. These difficulties are
  some of the reasons why a deterministic LP is
  often used as an approximation to a stochastic
  NLP.

5
Why the term Non-linear?

• Up until this chapter, decision variables,
  anywhere in model, were always in additive (hence
  linear) form 3x14x2, etc. Xs always as
  polynomials of degree 1.
• The only mathematical operators used in the LP
  models are and -.
• In NLP, no such limitations exist. The LP is
  actually a subset of NLP.

6
What causes non- linearity?

• Common math operations such as multiplication
  (x1x2), power (x2),
• exponentiation, logarithms, etc., make a model
  nonlinear even if only one of them appears just
  once anywhere in the model.

7
NLP vs. LP Applications

• If possible, the analyst should strive to model a
  decision process as a LP. Many management and
  production type problems have long been solved as
  LPs.
• Different set of problems (cheese making,
  engineering design and stock selection, for
  example) must contain non-linear terms that can
  not be avoided.

8

• Unlike the LPs, one is not always sure if a given
  NLP solution is optimal or not. In NLP,
  decision variables are not automatically
  non-negative. This allows certain physical
  values such as temperature to assume negative
  values, if necessary.

9
The Importance of Calculus to NLP.

• It appears that geometry and algebra were
  sufficient until this chapter. This convenience
  ends with NLPs. WV Chapter 12 Examples 1, 2,
  and 3 will refresh your memory on necessary
  calculus needed in NLP. Calculus is the backbone
  of NLP much like matrix algebra was for the LP
  model.

10
Solvers for NLP models

• It is possible to use specialized software such
  as the LINGO to solve NLPs just like using LINDO
  for LPs without getting too involved with the
  underlying mathematics.
• We need to know enough math to determine whether
  or not the NLP can be solved for an optimum
  before we expend the time and effort to structure
  the problem.

11
LINGO is a great tool for NLP!

• LINGO can be used to solve (or attempt to solve)
  NLPs of any form. It is also possible to have
  negative and/or integer (even binary) decision
  variables with LINGO.

12
What makes a mathematical program problem
non-linear?

• maximize 3 sin x x y y3 - 3z log
  zSubject to x2 y2 1
  x 4z ? 2 z ? 0
• A non-linear program is permitted to have
  non-linear terms in the objective function and/or
  the constraints.
• By restricting the terms in the objective
  function and/or the constraints, an LP is a
  special case of non-linear programming!

13
Concave vs. Convex NLPs

• Both have the so called convex constraint sets.
  
• A concave NLP has a concave objective function
  and it is a maximizing model.
• A convex NLP has a convex objective function and
  it is a minimizing model.
• To tell which set we have, apply the classic
  second derivative test. Review WV pages
  676-678.

14
Local vs. Global Optima

• Defn Let x be a feasible solution, then
• x is a global max if f(x) ³ f(y) for every
  feasible y.
• x is a local max if f(x) ³ f(y) for every
  feasible y sufficiently close to x (i.e., xj-e
  yj xj e for all j and some small e).

• There may be several locally optimal solutions.

15
When is a local optimum also globally optimal?

• For maximization problems (theorem 1, p675)
• The objective function is concave.
• The feasible region is convex.
• For minimization problems (theorem 1, p675)
• The objective function is convex.
• The feasible region is convex.

16
Convexity and Extreme Points
P
We say that a set S is convex, if for every two
points x and y in S, and for every real number l
in 0,1, lx (1-l)y ? S.
The feasible region of a linear program is convex.
We say that an element w ? S is an extreme point
(vertex, corner point), if w is not the midpoint
of any line segment contained in S.
x
y
W
2
4
6
8
10
12
14
16
17
On convex feasible regions
If all constraints are linear, then the feasible
region is convex
18
On Convex Feasible Regions
The intersection of convex regions is convex
19
Recognizing convex sets
Rule of thumb suppose for all x, y ? S the
midpoint of x and y is in S. Then S is convex.
20
Which are convex?
B
A
C
D
B ? C
B ? C
B ? C
20
21
Convex Functions f(l y (1- l)z) l f(y)
(1- l)f(z) for every y and z and for 0 l
1. e.g., l 1/2 f(y/2 z/2)
f(y)/2 f(z)/2
22
Concave Functions f(l y (1- l)z) ? l f(y)
(1- l)f(z) for every y and z and for 0 l
1. e.g., l 1/2 f(y/2 z/2) ?
f(y)/2 f(z)/2
23
Convexity and the Optimal Solution for LP vs. NLP

• The feasible region for any LP is a convex set.
  If an LP has an optimal solution, then there
  exists an extreme point of the feasible (convex)
  region that is optimal.
• IF a NLP has a convex feasible set (not a
  certainty) the optimal solution need not be an
  extreme point of the NLPs feasible region.

24
Graphical Analysis of Non-linear programs in two
dimensions An example
Minimize
subject to (x - 8)2 (y - 9)2 ? 49
x ? 2
x ? 13
x y ? 24
25
(No Transcript)
26
(No Transcript)
27
WV Example 10 Facility Location Problem of
Section

• This example has a linear objective function and
  mildly non-linear constraints. Figure 7 shows
  how LINGO software is used in determining the
  optimal location (in x, y coordinates) of a new
  warehouse.

28
Unconstrained Facility Location
This is the warehouse location problem with a
single warehouse that can be located anywhere in
the x-y plane. Distances are Euclidean.
Loc. Demand. A (8,2) 19 B
(3,10) 7 C (8,15) 2 D (14,13)
5 P ?
29
An NLP example

• Costs proportional to distanceknown daily
  demands

minimize 19 d(P,A) 5 d(P,D)subject to
P is unconstrained
30
Here are the objective values for 55 different
locations.
31
Facility Location. What happens if P must be
within a specified region?
32
The modified nlp model

Minimize
Subject to x ? 7 5 ?
y ? 11 x y ? 24

33
Section 12.8 Lagrange Multipliers and NLPs

• We use this concept if the NLP comes with all
• equality constraints. WV, pages 706-712.
  Theorem 8 and 8. Example 31 shows the
  mathematics of the Lagrange multipliers. For
  actual problems setup ast NLPs this may be a
  tough task to complete.
• LINGO will implement the LM math. The solutions
  report will indicate whether or not a local
  minimum has been found and if the LM (as shown in
  equation 16 of WV, page 707) hold at that point.

34
LINGO Example Solution Report
35
Shadow Prices and NLP Problems

• Do you remember this concept from the LPs?
  Lagrange Multipliers are the equivalent of the
  shadow prices in NLP. They are the rate of
  change of the optimal value as a ?fraction of the
  change in the RHS values of the NLP model.
• Review Example 28 for a demonstration of this
  concept. LINGO output in Figure 42 provides the
  Lagrange Multipliers under the Price Column. Also
  see the previous slide for an example.

36
Section 12.9 Karush-Kuhn-Tucker Conditions and
NLPs

• We use this concept if the NLP comes with all
• inequality constraints. WV, pages 713-722.
  Theorem 9 through 11. Example 32 shows the
  mathematics of the Lagrange multipliers. Example
  33 shows an application of the KKTs.
• LINGO will implement the KKT math. The solutions
  report will indicate whether or not a local
  minimum has been found and if the Theorems 11 or
  11 hold and the KKT hold at the optimal
  solution.

37
LINGO Example 33 Solution Report
38
Non-linearities in Math Programming Models

• Non-linearities as a result of time
  considerations
• a) Discount rates
• b) Decreasing value of equipment over time
• wear and tear, improvements in technology
• c) Tax implications (Depreciation)
• d) Salvage values

39
An Example Time Value and Salvage Value NLP
Model

• Buy a machine and keep it for t years, and then
  sell it. (0 ? t ? 10)
• all values are measured in million
• Cost of machine 1.5
• Revenue 4(1 - .75t)
• Salvage value 1/(1 t)
• How long (t) should we keep the machine before
  selling for salvage value? (same general model
  used to model culling decisions on dairy farms!)

40
(No Transcript)
41
Non-linearities in Pricing

• The price of an item may depend on the number
  sold
• quantity discounts for a small seller
• price elasticity for monopolist
• Complex interactions because of substitutions
• Lowering the price of your product may or will
  decrease the demand for the competitors product,
  etc.

42
Non-linearities because of congestion

• The time it takes to go from OSU to Westerville
  by car depends non-linearly on the congestion on
  SR-315 and I-270.
• As congestion increases just to its limit, the
  traffic sometimes comes to a near halt.

43
Non-linearities because of penalties

• Consider any linear equality constraint
• e.g., 3x1 5x2 4x3 17
• Suppose it is a soft constraint and we permit
  solutions violating it. We can then write
• 3x1 5x2 4x3 - y 17
• And we may include a term of 10y2 in the
  objective function.
• This adds flexibility to the solution by
  discourages violation of our goals

44
General Cost Curve with Economies of Scale

• Fixed Charge Linear variable cost does not
  allow for the possibility of declining marginal
  cost
• Assuming a constant cost per unit/mile
• May be a good approximation over short distances
  and homogeneous geographic structure
• May be a good approximation with a fixed
  transport mode, e.g., trucks of only a fixed size
  or capacity, only one type of transportation mode
  available,
• Generalized Cost structure relaxes these
  assumptions and allows for a economies
  (diseconomies)
• Variable cost declines with greater distance,
  larger capacity shipments, combinations of mode
  types (truck and rail), etc.

45
Modeling Economies of Scale

• Two approaches can be utilized to capture the
  nonlinear structure in cost.
• Integer Approximation to the Cost Curve as a
  Concave Piecewise Linear Cost Curve.
• Has all of the advantages of LP optimal
  solutions
• Requires the specification of
• Direct specification of the nonlinear cost curve
  and the use of nonlinear programming.

46
QUADRATIC PROGRAMMING (QP) of WV Section 12.10

• This is a very special and a highly realistic
  form of NLP. The constraints are linear and the
  objective function is a mildly non-linear form.
  The terms of the objective function can be either
  the square of one variable or the multiplication
  of any of the two variables.

47
QP Contd.

• QPs application in portfolio optimization is
  important. LINGO has a special structure for QP
  models. In a QP portfolio model the goal is to
  find how to allocate our funds to several
  securities while minimizing the portfolio
  variance and achieving a minimum return of 12.
  WV Example 35 shows how LINGO can be used to
  solve QPs.

48
Risk vs. Return

• In finance, one trades of risk and return. For a
  given rate of return, one wants to minimize risk.
  
• For a given rate of risk, one wants to maximize
  return.
• Return is modeled as expected value. Risk is
  modeled as variance (or standard deviation.)

49
Some Required Algebra
For a review see WV page 723
50
A simple example

• Suppose that X and Y are random variables
• E(X Y) E(X) E(Y)
• Interpretation
• Suppose that the expected return in one year for
  Stock 1 is 9.
• Suppose that the expected return in one year for
  Stock 2 is 10
• If you put 100 in Stock 1, and 200 in Stock 2,
  your expected return is 9 20 29.

51
Variances of random variables

• Suppose that X and Y are random variables
• Var(aX bY) a2 Var(X) b2 Var(Y)
  2ab Cov(X, Y)
• In general
• Var(X1 X2 Xn)

52
Reducing risk

• Diversification is a method of reducing risk,
  even when investments are positively correlated
  (which they often are).
• If only two investments are made, then the risk
  reduction depends on the covariance.

53
Portfolio Selection (contd)

• Two Methods are commonly used
• Min Risk
• s.t. Expected Return ³ Bound
• Max Expected Return - q (Risk)
• where q reflects the tradeoff between return and
  risk.

54
Portfolio Selection A Simple Example

• There are 3 candidate assets for out portfolio,
  X, Y and Z. The expected returns are 30, 20 and
  8 respectively (if possible we would like at
  least a 12 return). Suppose the covariance
  matrix is

Let X,Y,Z be percentage of portfolio represented
by each asset.
55
Extensions to the Portfolio Model

• There can be institutional constraints as well,
  especially for mutual funds.
• Other types of constraints can be
• No more than 15 in the energy sector
• Between 20 to 25 high growth
• At most 3 in any one firm
• etc.
• For more detail information see the powerpoint
  slide set from the text Optimization Modeling
  with LINGO. Chapter 13, by Linus Schrage.