Mathematics for Economists

1
Mathematics for Economists

• Nature of Dynamic Optimization
• HLMRS 25

2
Example of a dynamic problem optimal investment
by the firm

• Suppose a firm uses a production function
  Q Q(K,L) with positive but diminishing
  marginal products. K capital, L labor
• The firm maximizes the net present value of the
  firm. This period profits are PQ(K,L) WL - mI,
  where P is the product price, W is the nominal
  wage rate, m is the cost of capital and I is
  gross investment
• I dK/dt ?K net investment plus depreciation

3
Investment Example (2)

• Suppose the firm has a discount rate ?. We assume
  exponential discounting exp(- ?t). Exponential
  discounting leads to a fast decay at the
  beginning of the period

exp(-?t)
t
0
4
Investment Example (3)

• So the net present value of the firm is
• ? ? PQ(K,L) WL m(dK/dt ?K) exp(-?t) dt
• The problem with this objective function is that
  it contains both K and dK/dt
• We want to compute the optimal K and L
• L is not a big problem, but K is more complex

5
Example 2 Ramsey model

• Population ( labor force) N grows with n
• Output Y is produced using capital K and labor N
  Y F(K,N) C dK/dt C is consumption and
  dK/dt is investment. We assume that there is no
  depreciation
• We go to per capita terms y Y/N,
  k K/N, c C/N

6
Ramsey (2)

• There is a transformation problem. How to go from
  dK/dt to dk/dt?
• dk/dt d(K/N)/dt dK/dt.N - KdN/dt/N2
  dK/dt/N - (K/N)dN/dt/N dK/dt/N - k.n
• So Y/N C/N dK/dt/N c dk/dt nk f(k)
• Inada conditions f(0) 0, f(0) ?, f(?) 0,
  and k0 gt 0
• We assume that consumers optimize utility
  U ?0? u(ct) exp(-?t) dt

7
Ramsey (3)

• So maximize U ?0?u(ct) exp(-?t) dt
• Subject to c dk/dt nk f(k) and kt, ct ? 0
• Again, this is a nasty problem
• The Ramsey model is an optimal savings model and
  currently a standard model of economic growth
• It is rather easy to solve the investment model
  that is what we do now the Ramsey-model needs a
  more advanced technique (namely Optimal Control).
  But first some general remarks on dynamic
  optimization

8
Dynamic optimization

• It applies to all problems that have an intrinsic
  dynamic nature. The state of the world at time t
  is connected with time t1, t2, etc.
• This holds for all investment, growth,
  investment, even consumption decisions and models
• One can represent these problems as multi-staging
  problems

9
Multi-stage decision making
State
7
8
2
Z
A
4
6
1
2
3
4
5
Stage
10
Continuous-variable version
State
B
A
Stage
0
11
General Dynamic Problem

• Max V ? f(x,y,t)dt
• Subject to dx/dt g(x,y,t)
• And some beginning and ending restrictions on x
  x(0) x0 and x(T) xT (see hereafter)
• x state variable, y control variable
• We are going to weight the restriction by a
  so-called co-state variable ?

12
Alternative techniques

• Suppose there is no control variable in this
  case Calculus of Variations (Euler equations) is
  the appropriate technique
• Otherwise we use (1) Optimal control using the
  Hamiltonian approach, (2) Dynamic Programming
  (Bellman equations) works for discrete problems

13
Calculus of Variations

• Classical approach to dynamic optimization
• Dates back to Newtons Principia (1687) and e.g.
  one of the Swiss Bernoulli brothers
• We transform the problem into
  max V ? f(t, x, dx/dt) dt
• Subject to begin and end restrictions on x
  x(0) x0 and x(T) xT
• We need the functional to be integrable,
  continuous, and differentiable. Note that V
  depends on x and dx/dt

14
Euler equation

• max V ? f(t, x, dx/dt) dt
• Euler equation fx d(fdx/dt)/dt
• So we need a couple of steps
• Take the first-order derivatives of f with
  respect to x an dx/dt
• Take the time derivative of fdx/dt

15
Calculus of Variations intuition
x
xT
x(t) x(t) ah(t) pertubation
x0
x(t) optimal path
h(t) pertubation function
0
t
16
Calculus of Variations intuition

• V depends on given x(t) and h(t) and an unknown
  a
• For small a x(t) goes to x(t). Given x(t) and
  h(t) we need to have that dV/da 0 for an
  extremal value of V
• Note that the pertubation function h(t) needs to
  have the properties h(0) 0 and h(T) 0
• Also note that we do not know too much about a or
  h(t). Calculus of variations helps we formulate
  the Euler condition, which is a convenient way of
  expressing dV/da 0 at values of a close to 0
• In doing so we need to know how to differentiate
  an integral

17
Integration tricks

• Suppose we have I(x) ?ab F(t,x) dt. Leibnizs
  rule dI / dx ?ab Fx(t,x) dt
• Or denote I(a,b) ?ab F(t,x) dt
• Then dI / da -F(a,x) and dI / db F(b,x)
• Integration by parts
  ?ab f(x)g(x) dx -?ab f(x)g(x)dx f(x)
  g(x) ab

18
Application to the pertubation integral

• Va ?0T f(t,xa,dxa/dt) dt
• dVa/da ?0T df/dx (dxa/da) df/d(dxa/dt)
  (d(dxa/dt)/da) dt ?0T fxh(t) fx h(t) dt
  since we have xa(t) x(t) ah(t) and dxa(t)/dt
  dx(t)/dt adh(t)/dt
• Note that the last expression still contains the
  unknown h(t) and h(t). Now we use integration by
  parts on the last terms
• ?0T fx h(t) dt - ?0T h(t) d(fx)/dt dt

19
The Euler condition

• So we get ?0T fxh(t) fx h(t) dt
  ?0T h(t)fx - d(fx)/dt dt 0 for all
  h(t), so
• fx d(fx)/dt for all t in 0,T
• Special case 1 (no x) suppose we have
  f(t,dxa/dt). We will get d(fx)/dt constant
• Special case 2 (no t) f(x,dxa/dt).
  We get f dxa/dt fx

20
Example Investment

• ? ? PQ(K,L) WL - m(dK/dt ?K)
  exp(-?t)dt
• d?/dL PQL - W exp(-?t) no dynamics L
• d?/dK PQK - m ? exp(-?t)
• d?/d(dK/dt) -m exp(-?t)
• So d/dt d?/d(dK/dt) ?.m exp(-?t)
• The Euler-condition for K
  PQK - m ? exp(-?t) ?.m.exp(-?t) or
  QK (? ?)m / P

21
Summary so far

• Dynamic problems are abundant in economics all
  decisions depend on future expectations. Famous
  examples optimal savings, investment, and growth
  models
• Without control variables, calculus of variations
  in general and the Euler condition in particular
  are useful techniques
• With control variables we need to extend the
  model to optimal control problems

22
Again Ramsey

• So maximize U ?0?u(ct) exp(-?t) dt
• Subject to ct dkt/dt nkt f(kt) and kt, ct
  ? 0
• The problem is difficult due to dkt/dt
• The Ramsey model is an optimal savings model and
  currently a standard model of economic growth
• We need an advanced technique namely the Theory
  of Optimal Control.
• First we review the solution to the Ramsey
  problem in full and after that we return to the
  techniques used

23
Optimum

• We use the so-called Hamiltonian
  Ht u(ct) exp(-?t) ?tf(kt) nkt -
  ct
• Note that the costate variable ?t (associated
  with the single state variable kt) is time
  dependent ct is the control variable.
  Trick use ?t ?t exp(?t) and get Ht u(ct)
  ?t (f(kt) nkt - ct) exp(-?t)
• We forget for the moment the non-negativity
  constraints on kt and ct

24
Optimum (2)

• We use the so-called Maximum Principle
• Condition 1 Hc 0, so u(ct) ?t
• Condition 2 d?t/dt -Hk
• d?t/dt d?t/dt exp(-?t) - ?t ? exp(-?t), and
  Hk - ?t (f(kt) - n), so
  d?t/dt ?t ? n - f(kt) this
  is the equation for the law of motion of the
  costate variable
• Condition 3 lim kt ?t 0 if t??

25
How to translate these conditions into economics?

• We combine the first two into
• (d u(ct)/dt) / u(ct) ? n - f(kt)
• Define 1/?(ct) -u(ct)ct/u(ct) elasticity of
  substitution (or the curvature of the utility
  function). For linear functions this ? is very
  large. We get
• dct/dt ?(ct) f(kt) - ? - n this is an Euler
  condition. It is the so-called Keynes-Ramsey
  rule. If productivity of capital is high,
  consumption will be increasing along the optimal
  path

26
Economics (2)

• The transversality condition
  lim kt ?t 0 if t??. Use ?t u(ct) exp(-?t)
• At the end of the planning period if
  u(ct) exp(-?t) were positive, it is not optimal
  to end with a positive capital stock (one could
  have increased consumption!)

27
Steady State and Dynamics

• Three key equations
• dct/dt ?(ct)f(kt) - ? - n
• dkt/dt f(kt) - nkt - ct
• lim kt u(ct) exp(-?t) 0 if t??
• So the steady state is described by f(k)
  ? n and c f(k) - nk the modified golden
  rule

28
Dynamics of capital and consumption
c
DD saddle point path
dc/dt0
D
E
dk/dt0
D
k
0
kg
29
Transitional Dynamics

• How does the model behave outside the steady
  state? We linearize the model around the steady
  state.
• dc/dt f(k) c ?(c)k - k -?k - k
• dk/dt -c - c (f(k) - n) k - k
• These can be combined into
• d2k/dt2 - ?(dk/dt) - ?k - ?k
• This differential equation has one positive and
  one negative root this implies Saddle point
  stability!

30
Lessons from the example

• Important to be able to set up the so-called
  Hamiltonian
• Use the Maximum conditions, derive the Euler
  equations, take care of terminal conditions
• Give an economic interpretation to the transition
  equations
• Analyze the steady state and the transitional
  dynamics
• Now we go to the more formal theory

31
Theory of Optimal Control The Maximum Principle

• Max J ?0T f(x(t),y(t),t) dt
  subject to dx/dt g(x(t),y(t),t),
  x(0) x0 gt 0, x(T) is free (this is an
  assumption that will be changed)
• Hamiltonian
  H f(x(t),y(t),t) ?(t) g(x(t),y(t),t)
• x state variable, y control variable, ? is
  the costate variable

32
Pontryagins Maximum Principle

• Optimality conditions are
• Hy 0 control variable is chosen to maximize H
• -Hx d?/dt path for the costate variable
• dx/dt H? g(x,y) this is the constraint!
• x(0) x0 and ?(T) 0 two boundary conditions

33
Mangasarian Theorem

• The necessary conditions are sufficient if
• If f(x,y,t) is concave in x and y and g(x,y,t) is
  linear in x and y
• If f(x,y,t) is concave in x and y and g(x,y,t) is
  also concave in x and y and ?(t) ? 0
• If f(x,y,t) is concave in x and y and g(x,y,t) is
  also convex in x and y and ?(t) ? 0

34
Example Investment again

• Production Q K aK2, a gt 0
• dK/dt I - ?K, ? gt 0
• Costs I2
• Profits ? K aK2 - I2
• Max ?0T (K aK2 - I2) dt , subject to
  dK/dt I - ?K and K(0) K0

35
Solution (1)

• Hamiltonian H K aK2 - I2 ?(I - ?K)
• H is concave in I HI -2I ? 0, so
  I(t) ?(t)/2 is a solution
• d?(t)/dt -HK - (1 - 2aK - ??)
• dK/dt ?/2 - ?K
• This is a system of differential equations!
• Determinant of the coefficient matrix is
  negative Saddle-point equilibrium

36
Solution (2)

• ?(t) C1 exp(r1t) C2 exp(r2t) ?p
• K(t) (r1 - ?)/2a C1 exp(r1t)
  (r2 - ?)/2a C2 exp(r2t) Kp
• Steady state values can be computed
• ?p ?/(?2 a) and Kp 1/(2(?2 a))
• ?(t) is the shadow price (or imputed value) of
  the state variable at time t
• So 2I 2? represents marginal costs equal
  marginal returns to investment