Optimal Control and Reachability with Competing Inputs

1
Optimal Control and Reachabilitywith Competing
Inputs

• Ian Mitchell
• Department of Computer Science
• The University of British Columbia
• research supported by
• National Science and Engineering Research Council
  of Canada

2
Competing Inputs

• What do we do when there are multiple parameters,
  some of which we can choose but some of which
  have unknown and uncontrolled value
• Control input denoted by u (or a)
• Disturbance input denoted by d (or b)
• Choose control input as before to optimize
  trajectory or achieve safety
• Due to disturbance input, system remains
  nondeterministic even if control signal is fixed

3
Two Treaments of Disturbance

• Stochastic perturbations d(t) D
• Discrete state Poisson processes
• Markov decision processes
• Stochastic differential equations
• Bounded value inputs d(t) 2 D
• Robust reach sets
• Two player zero sum games

4
Markov Decision Process

• Discrete time, discrete state model with
  probabilistic transitions
• Typically specified by
• Alternatively specified by x(t1) d(t)
• where d(t) is drawn from the Bernoulli Scheme
  with
• For discrete time systems, many distributions are
  supported

5
Stochastic Differential Equations (SDEs)

• Two mathematical frameworks Ito and Stratonovich
• Conversions exist between the two
• SDE is ODE with a Brownian motion (Wiener
  process) perturbation
• Restrictive class of distributions
• eg cannot guarantee bound on stochastic term
• Equivalent of Hamilton-Jacobi equation for SDEs
  is the Fokker-Planck equation

6
Continuous Backward Reachable Tubes

• Set of all states from which trajectories can
  reach some given target state
• For example, what states can reach G(0)?

Continuous System Dynamics
Target Set G(0)
Backward Reachable Set G(t)
7
Reachable Tubes (controlled input)

• For most of our examples, target set is unsafe
• If we can control the input, choose it to avoid
  the target set
• Backward reachable set is unsafe no matter what
  we do
• Minimal backward reach tube

Continuous System Dynamics
8
Reachable Tubes (uncontrolled input)

• Sometimes we have no control over input signal
• noise, actions of other agents, unknown system
  parameters
• It is safest to assume the worst case
• Maximal backward reach tube

Continuous System Dynamics
9
Two Competing Inputs

• For some systems there are two classes of inputs
  ? (u,d)
• Controllable inputs u ? U
• Uncontrollable (disturbance) inputs d ? D
• Equivalent to a zero sum differential game
  formulation
• If there is an advantage to input ordering, give
  it to disturbances

Continuous System Dynamics
10
Objective Function

• Extends in obvious way to the additional input
• eg discrete time discounted with fixed finite
  horizon tf
• eg continuous time no discount with target set T

11
Who Goes First?

• One input is chosen to maximize and the other to
  minimize the objective
• But what knowledge is available when choosing an
  input?
• Current state? Other input?
• Non-anticipative strategies
• One player gets to know the other players input
  value (as well as current state)
• However, that player must declare their strategy
  (reaction to every input) in advance

12
Zero Sum Game Value Function

• Value function is then defined as optimization
  over appropriate strategy and input signal pair
• Lower value function, since disturbance
  (minimizer) has the advantage
• Parallel upper value function can be defined
• If inputs are independent, optimal strategy will
  ignore additional information about the other
  input
• Upper and lower value functions will be equal

13
Competing Inputs Final Comments

• Feedback control is more realistic implementation
• If order of input decision is irrelevant (upper
  and lower value functions are equal), then
  nonanticipative strategy results will be
  equivalent to feedback results
• For robustness, give advantage (eg strategy) to
  the disturbance input if it matters (potentially
  pessimistic)
• Input signals still drawn from set of measureable
  functions
• Two player concepts have been extended to
  viability theory and set-valued analysis

14
Optimal Controland Reachability with Competing
Inputs

• For more information contact
• Ian Mitchell
• Department of Computer Science
• The University of British Columbia
• mitchell_at_cs.ubc.ca
• http//www.cs.ubc.ca/mitchell