Constructive Methods of Optimal Control under Uncertainty

1
Constructive Methods of Optimal Control under
Uncertainty

• Rafail Gabasov
• Belarussian State University

2
Outline

• Introduction
• Classical optimal feedback and its realization
• Optimal guaranteeing feedbacks
• Optimal control under imperfect information
• Optimal decentralized control
• Parallelizing of computations during optimal
  control of large systems
• Optimal on-line control with delays
• Optimal control of time-delay systems
• Optimal control of PDEs
• Nonlinear optimal control problems
• Stabilization of dynamical systems

3
Introduction

• Points of view on Optimal Control Theory
• Calculus of Variations
• Control Theory
• Principles of control
• Open-loop control
• Closed-loop control
• Real time (on-line) control
• Types of closed loops
• Feedforward
• Feedback
• Feedforward-feedback (combined)

4
Linear optimal control problem
(1)
state of the control system at the instant
t value of control at the instant t
piecewise continuous matrix functions
terminal set (g, g?Rm,
H?Rm?n) set of accessible values
of control
Discrete controls (h(t t)/N, Ngt0)
(2)
5
Classical optimal feedback
Imbed problem (1) into a family
(3)
depending on ??Th and z?Rn
u0(t?, z), t?T(?) optimal open-loop control
of (3) for a position (?, z) X? set of states
z for which optimal open-loop control exists
Optimal feedback
(4)
6
Realization of optimal feedback

• Real system closed by optimal feedback

(5)
w disturbances
Trajectory of (5) is a solution to linear
differential equation
(6)
Particular control process with w(t), t?T
(7)
Realization of optimal feedback in a particular
control process
(8)
7
Optimal Controller

• ?

(9)
Linear programming problem
(10)
8
Fast algorithms for optimal open-loop control

• Gabasov R., Kirillova F.M. (2001) Fast
  algorithms for positional optimization of dynamic
  systems. Proceedings of the Workshop "Fast
  solutions of discretized optimization problems".
  (K.-H.Hoffmann, R.Hoppe and V. Schulz eds.)
• Gabasov R., Kirillova F.M. and N.V. Balashevich
  (2000). On the Synthesis Problem for Optimal
  Control Systems. SIAM J. Control Optim.
• Gabasov, R., F.M. Kirillova and N.V. Balashevich
  (2000). Open-loop and Closed-loop Optimization
  of Linear Control Systems. Asian Journal of
  Control.

9
Analysis

• ? h u0(t? h, x(? h)), t?T(? h)

u(t) u0(? h ? h, x(? h)), t?? h,?
control fed into the system w(t), t?? h,?
realized disturbance
? u0(t? , x(? )), t?T(? )
10
Example optimal damping of two-mass
oscillating system
Real system
with disturbance
11
Example optimal damping of two-mass
oscillating system
12
Discussion

• Direct control system
• Gabasov R., Dmitruk N.M. and F.M.Kirillova
  (2004). Indirect Optimal Control of Dynamical
  Systems. Comput. Math. Math. Phys.
• Gabasov R., Kirillova F.M. and N.S. Pavlenok
  (2003). Design of Optimal Feedbacks in the
  Class of Inertial Controls.
• Automation and Remote Control
• Gabasov R., Kirillova F.M. and N.N.Kovalenok
  (2004) Synthesis of optimal signals for the
  control of dynamical systems with Lipschitz
  bang-bang actuators. Dokl. Akad. Nauk, Ross.
  Akad. Nauk
• No state or mixed constraints
• Gabasov R., F.M. Kirillova and N.V. Balashevich
  (2001). Algorithms for open-loop and closed-loop
  optimization of control systems with
  intermediate state constraints. Comput. Math.
  Math. Phys.
• Information on disturbances is not used
• Exact measurements of all states are available
• Mathematical model with lumped parameters, not
  large
• Problem is linear

13
Optimal guaranteeing feedbacks
(11)
disturbance set of possible
values of the disturbance

• Types of feedback

• closed

• closable

• unclosable

• R.Gabasov, F.M.Kirillova and N.V.Balashevich
  (2004). Guaranteed on-line control for linear
  systems under disturbances. Functional
  Differential Equations

14
Example optimal guaranteeing feedbacks
Parameters
Guaranteed values of the performance index 1)
unclosable feedback 2) one-time closable
feedback with closure instant t 8
2
1
15
Control System under Disturbances

• Tt,t control interval
• Dynamical system with disturbance

(12)
piecewise continuous matrix function
Measuring device
(13)
continuous matrix function
output
errors of the measuring device
Measurements are made at discrete instants
16
Elements of uncertainty
Initial state
(14)
given
set of possible values of parameters z
Disturbance
(15)
piecewise continuous functions (L1,2,nv)
vector of parameters of the disturbance
set of possible values of parameters v
Measurement errors
(16)
17
Classical control of the system under uncertainty
t t

• measurement y(t) is obtained (generated by
  x(t), ?(t))
• vector u(t) u(t,y(t)) ?U is chosen
• control function u(t) u(t), t ?t,th, is
  fed into the system

t th

• system moves to the state x(th)
• measurement y(th) is obtained (generated by
  x(th), ?(th))
• .

t ?

• measurement y(? ) is obtained
• signal is formed
• vector u(? ) u(?, y?()) ?U is chosen
• control function u(t) u(? ), t ?? ,? h, is
  fed into the system

totality of all signals y?() that can be
obtained under chosen u
18
Optimal classical feedback
Feedback under inaccurate measurements
(17)
set of all trajectories of
(18)
for a chosen feedback u and a fixed signal
y()(y(t), t?Th)

• Admissible feedback
  

Performance index
Optimal (guaranteeing) feedback
19
Optimal on-line control
Suppose that by the moment ?

• measurements
  has been made
• controls
  has been calculated in time
• 
  (neglected for simplicity)
• control function
• has been fed into the system

At the moment ?

• current measurement y(? ) is obtained

Aim

• calculate current value of control
• feed to the input of control object the control
  function

20
A priori and a posteriori distribution sets
A priori distribution sets

• Z a priori distribution set of parameters z of
  the initial state x(t)
• V a priori distribution set of parameters v of
  the disturbance w(t), t?T
• ?Z?V(?(z,v) z ? Z, v ?V) a priori
  distribution
• of unknown parameters ? of
  the system

A posteriori distribution set
set of all vectors ? to which there correspond
the initial condition x(t)x0Gz and the
disturbance w(t)?(t)v, t?t,?, able together
with some measurement error ?(t), t?Th(?), to
generate the signal
21
Admissible open-loop control (program)
Function is said to be an
admissible open-loop control if together with
it transfers the control
system (12) at the moment t on the terminal
set X for all ? from
Equivalent to The admissible control
transfers the determined system
(19)
state of this system with x(t)x0,
u(t)u(t), t?t,?
at the moment t to the terminal set
(20)
22
Accompanying optimal observation problems
To establish admissibility of control
it is required to solve extremal
problems
(21)
i-th row of matrix H
Problems (21) are called optimal observation
problems accompanying the optimal control
problem under uncertainty (accompanying
optimal observation problems)
23
Optimal open-loop control and accompanying
optimal control problem
The quality of the admissible open-loop control
is evaluated by
Optimal open-loop control
solves problem
(22)
called optimal control problem accompanying the
optimal control problem under uncertainty
(accompanying optimal control problem)
Let
24
Scheme of optimal on-line control of dynamical
system under uncertainty

• At the moment ?
• Solve 2m accompanying optimal observation
  problems
• Solve the accompanying optimal control problem

OE Optimal Estimator solves accompanying
optimal observation problem OC Optimal
Controller solves accompanying optimal
control problem
25
Optimal observation problems

• Gabasov R., Dmitruk N.M., Kirillova F.M. (2002).
• Optimal Observation of Nonstationary Dynamical
  Systems.
• Journal of Computer and Systems Sciences Int.
• Gabasov R., Dmitruk N.M., Kirillova F.M. (2004).
• Optimal Control of Multidimensional Systems by
  Inaccurate
• Measurements of Their Output Signals.
  Proceedings of the
• Steklov Institute of Mathematics.

26
Example optimal control under imperfect
information
Mathematical model
Control interval T0,15
Parameters
Initial condition
Disturbance
Sensor
27
Example optimal control under imperfect
information
Performance index
Terminal condition
Particular process
28
Example optimal control under imperfect
information
29
Example optimal control under imperfect
information
30
Optimal decentralized control

• Optimal control of a group of q objects

31
Optimal decentralized control

• For control of q subsystems q Optimal Controllers
  operating in parallel are used

At each moment ? ?Th i-th Optimal Controller
obtains

• current state of i-th subsystem
• results, obtained by all other OCs at
• previous moment ? h

Realization of optimal feedback
optimal open-loop control of problem with ri
inputs
32
Example optimal decentralized control
1
1
2
2
1) decentralized 2) centralized
33
Parallelizing of computations during optimal
control of large systems
(1)
e.g., two systems
34
Quasidecomposition of the fundamental matrix
(23)
(24)
35
Example parallelizing of computations
Number of parameters of approximation of F(t) Terminal state Value of performance index
40 0.00102613 0.00410210 -0.00028247 -0.00129766 7.041025176
50 -0.00017364 0.00015934 -0.00010114 0.000031591 7.047378611
No approximation (exact) (10-7, 10-7, 10-7, 10-7) 7.046875336
36
Optimal on-line control with delays
Every Optimal Controller calculates u(?) in time
? Kh, Kgt1
37
Optimal control of time-delay systems
(25)

• a(t)?R, t?T x10(t)?R, t?t?,t piecewise
  continuous functions
• ? delay e(1,0,,0) ?Rn

Optimal feedback
state of system
u0(t?, z?()), t?T(? ) optimal open-loop
control of (25) for (?, z?())
Realization of optimal feedback
38
Quasireduction of the fundamental matrix
pi1(t), t?T finite-parametric approximations
of fi1(t), t?T ,
fij(t), t?T solutions to n 1 systems of
ODEs
R.Gabasov, O.Yarmosh. Fast algorithm of open-loop
solution in a linear optimal control problem for
dynamical systems with delays. Today, Section C-1.
39
Example optimal control of time-delay systems
,
Real system
with disturbances a)
b)
c)
.
40
Optimal control of PDEs

• Problem of optimal heating

(26)
x(s,t), (s,t)?Q temperature at point s at
instant t u(t), t?T control a, ? , a, ßgt0
given constants h(s)?Rm, s?S g, g?Rm
41
Approximation of PDE

• S?0, ?,, l- ?, l, l/K, Kgt0
• ys(t)x(s,t), s?S?, t?T

(27)
y0(x0(s), s?S?), H (h(s), s?S?),
b(0,0,,0,a?/?)
42
Nonlinear optimal control problems

• Nonlinear dynamics
• f(x), x?X
• N.V. Balashevich, R. Gabasov, A.I. Kalinin, and
  F.M. Kirillova (2002).Optimal Control of
  Nonlinear Systems. Comp. Mathematics and
  Math.Physics

43
Nonlinear optimal control problems

• Nonlinear performance index
• f(x), f0(x), x?X convex functions
• Nonlinear input
• Arbitrary set U, convex terminal set X

44
Stabilization of dynamical systems

• Gabasov R. Kirillova F.M. and O.I. Kostyukova
  (1995). Dynamic system stabilization methods.
  Journal of Computer and Systems Sciences
  International.
• Gabasov, R.F. Ruzhitskaya, E.A. (1999). A
  method of stabilization of dynamic systems under
  persistent perturbations. Cybernetics and
  Systems Analysis