The Maximum Principle: General Inequality Constraints

1
Chapter 4

• The Maximum Principle General Inequality
  Constraints

2
4.1 Pure State Variable Inequality Constraints
Indirect Method

• It is common to require state variable to remain
• nonnegative, i.e.,
• i.e., xi(t) ? 0, i1,2,,n. Constraints
  exhibiting (4.1) are
• called pure state variable inequality
  constraints. The
• general form is
• With (4.1) at any point where a component
  xi(t)0, the
• corresponding constraint xi(t) ? 0 is not binding
  and can
• be ignored.

3

• In any interval where xi(t)0,we must have
  so
• that xi does not become negative.
• The control must be constrained to satisfy
• making fi ? 0, as a constraint of the mixed type
  (3.3) over
• the interval. We can add the constraint
• We associate multipliers ?i with (4.3) whenever
  (4.3)
• must be imposed, i.e., whenever xi(t)0. A
  convenient
• way to do this is to impose an either or
  condition
• ?i xi0. This will make ?i0 whenever xi 0.

4
We can now form the Lagrangian where H is as
defined in (3.7) and ?(?1, ?2 ,,?n ).

• We apply the maximum principle in (3.11) with
• additional necessary conditions satisfied by ?
• and the modified transversality condition
• where ? is a constant vector satisfying

5

• Since the constraints are adjoined indirectly (in
  this
• case via their first time derivative) to form the
  
• Lagrangian, the method is called the indirect
  adjoining
• approach. If on the other hand the Lagrangian L
  is
• formed by adjoining directly the constraints
  (4.1), i.e.,
• where? is a multiplier associated with (4.1),
  then the
• method is referred to as the direct adjoining
  approach.

6

• Remark 4.1 The first two conditions in (4.5) are
  
• complementary slackness conditions on the
  multiplier
• ?. The last condition is difficult to
  motivate. The
• direct maximum principle multiplier ? is related
  to ? as
• The complementary slackness
  conditions for
• the direct multiplier ? are ? ? 0 and ?x0.
  Since ? ? 0,
• it follows that
• Example 4.1 Consider the problem

7
Solution. The Hamiltonian is

• which implies the optimal control to be
• When x0, we impose , in order
  to insure
• that (4.10) holds. Therefore, the optimal control
  on the
• state constraint boundary is
• Now we form the Lagrangian

8

• where ?1,?2, and ? satisfy the complementary
• slackness conditions
• Furthermore, the optimal trajectory must satisfy
• From the Lagrangian we also get
• Let us first try ?(2) ? 0. Then the solution
  for ? is the
• same as in Example 2.2, namely,

9

• Since ?(t)?-1 on 0,1 and x(0)10, the original
  
• optimal control given by (4.11) is u(t)-1.
  Substituting
• this into (4.8) we get x(t)1-t, which is
  positive for t
• Thus
• In the time interval 0,1) by (4.14), ?20 since
  ?
• and by (4.15) ?0 because x0. Therefore,
  ?1(t)-?(t)
• 2-t 0 for 0?t(4.13).
• At t1 we have x(1)0 so the optimal control is
  given
• by (4.12), which is u(1)0. Now assume that we
• continue to use the control u(t)0 in the
  interval
• 1? t ? 2.

10
Figure 4.1 State and Adjoint Trajectories in
Example 4.1
11

• With this control, we can solve (4.8) beginning
  with
• x(1)0, and obtain x(t)0 for 1?t?2. Since ?(t) ?
  0 in the
• same interval, we see that u(t)0 satisfies
  (4.12)
• throughout this interval.To complete the
  solution, we
• calculate the Lagrange multipliers.since u0, we
  have
• ?1?20 throughout 1?t?2. Then from (4.16) we
  obtain
• ?-?2-t?0 which, with x0, satisfies (4.15).
  This
• completes the solution.
• Remark 4.2 In instances where the initial state
  or the
• final state or both are on the constraint
  boundary, the
• maximum principle may degenerate I.e., there is
  no nontrivial solution of the necessary
  conditions,i.e., ?(t)?0, t?0,T, where T is the
  terminal time.

12
4.1.1 Jump Conditions

• There may be a piecewise continuous ?(t)
  satisfying a
• jump condition
• at a time at which the state trajectory hits
  its
• boundary value zero.
• Example 4.2 Consider Example 4.1 with T3 and the
• terminal state constraint
• Clearly, the optimal control u will be the one
  that
• keeps x as small as possible, subject to the
  state
• constraint (4.1) and the boundary condition
• x(0)x(3)1. Thus,

13

• We only compute the adjoint function and
  multipliers
• that satisfy the optimality conditions. These are

14
4.2 A Maximum Principle Indirect Method

• Mixed constraints as in Chapter 3 and the pure
  state
• inequality constraints
• where h En x E1 ? Ep. By the definition of
  function h,
• (4.26) represents a set of p constraints
  hi(x,t)?0,
• i1,2,,p.It is noted that the constraint hi?0 is
  called
• a constraint of rth order if the rth time
  derivative of
• hi is the first time a term in control u appears
  in the
• expression by putting f(x,u,t) for after each
  
• differentiation.

15

• In case of first order constraints, h1(x,u,t) is
  as follows
• With respect to the ith constraint hi(x,t)?0, a
  sub
• Interval (?1,?2) ? 0,T with ?1interior
• interval if hi(x(t),t)0 for all t?(?1,?2). If
  the optimal
• trajectory satisfies hi(x(t),t)0 for
  for
• some i, then is called a boundary
  interval. An
• instant is called an entry time if there
  is an interior
• interval ending at t and a boundary
  interval
• starting at . Correspondingly, is
  called an exit
• time if a boundary interval ends and an interior
  interval
• starts at .

16

• If the trajectory just touches the boundary at
  time ,
• i.e., and if the
  trajectory is in
• the interior just before and just after ,then
  is called
• a contact time. Entry, exit and contact times are
  called
• junction times.

?1
?1
?1
Entry
Exit
Contact
17

• Full rank condition on any boundary interval
  is
• as follows
• where for
• and

18

• To formulate the maximum principle for the
  problem
• with mixed constraints as well as first-order
  pure state
• constraints, we form the Lagrangian as
• H is defined in (3.7), u satisfies the
  complementary
• slackness conditions stated in (3.9), and ??Eq
  satisfies
• the conditions
• The maximum principle sates that the necessary
• conditions for u to be an optimal control are
  that there
• exists multipliers ?,?,?,?,?,and ?, and the jump
• parameters ?, which satisfy (4.29) that follows.

19
(No Transcript)
20
Note that the jump conditions on the adjoint
variables in (4.29) generalize the jump condition
on H in (4.29)requires that the Hamiltonian
should be continuous at if ht 0.Example
4.3 Consider the following problem with the
discount rate ? ?0
21
Solution. As one can see from Figure 4.2 and 4.3,
the optimal solution is

• Figure 4.2 Feasible State Space and Optimal
  State Trajectory
• for Example 4.3

22
Figure 4.3 Adjoint Trajectory for Example 4.3
23
Note that at the entry time t1 to the state
constraint (4.34), the control u and, therefore,
h1u2(t-2) is discontinuous, i.e., the entry
is non-tangential. on the other hand, u and h1
are continuous at t2 so that the exit is
tangential. With u and x thus obtained, we
must obtain ?, ?1 , ?2, ?, and ? so that the
necessary optimality conditions (4.29) holds,
i.e.,
24
From (4.41), we obtain ?(1-)e-? . This with
(4.38) gives
25
and

• which, along with u and x, satisfy (4.29).
• Note, furthermore, that ? is continuous at the
  exit
• time t2. At the entry time so that (4.30) also
  holds.

26
4.3 Current-Value Maximum Principle Indirect
method

• With the Hamiltonian H as defined in (3.33), we
  can
• write The Lagrangian
• We can now state the current-value form of the
• maximum principle as given in (4.42)

27
(No Transcript)
28
4.4 Sufficiency Conditions

• The sufficiency results can be stated in the
  indirect
• adjoining framework. In order to do so, let us
  define the
• Hamiltonian H and the Lagrangian Ld in the
  direct
• method as
• where ?d, ?d, and ?d are multipliers in the
  direct
• formulation, corresponding to ?, ?, and ? in the
  direct
• formulation.

29
It can be shown that

• Theorem 4.1 Let satisfy
  the necessary
• conditions in (4.29) and let
• If is concave in (x,u) at
  each t? 0,T, S in
• (3.2) is concave in x,g in (3.3) is quasiconcave
  in (x,u)
• h in (4.26) and a in (3.4) are quasiconcave in x,
  and b
• in (3.5) is linear in x, then (x,u) is optimal.

30
Theorem 4.2 Theorem 4.1 remains valid if the
concavity of in (x,u) at each t
is replaced by the concavity of the maximized
Hamiltonian
in x, where
Theorem 4.1 and Theorem 4.2 are written for
finitehorizon problems and remains valid if the
transversality conditions on the adjoint
variables (4.29) is replaced by the following
limiting transversality condition
31

• Example 4.1 (Continued) First we obtain the
  direct
• adjoint variable
• It is easy to see that

is linear and hence concave in (x,u) at each
t?0,2.
32
Functions

• and
• are linear and hence quasiconcave in (x,u) and x,
  
• respectively. Functions S ? 0, a ? 0 and b ? 0
  satisfy
• The conditions of Theorem 4.1 trivially.