Linear Quadratic Optimal Control

1
Linear Quadratic Optimal Control

• Digital Control Theory
• Lecture 12

2
Outline

• Introduction to linear quadratic optimal control
• Quadratic cost function
• Principle of optimality and example
• Linear quadratic optimal control and example

3
Introduction to linear quadratic optimal control
The pole-assignment design results in the
closed-loop pole locations that yield the best
control system. A different technique that
yields the best control system is performed
through minimsing a mathematical function, called
cost function. Hence it is an optimal design
technique. The cost function (or performance
index) is defined as a combination of state
vector, control input and/or output vector. In
most applications, the cost function is in
quadratic form
where k is the sample instant, N is the terminal
sample instant, x(k) is the state vector, u(k) is
the control input to the plant, Q(k) and R(k) are
known as weighting matrices as they control the
relative importance of x(k) and u(k),
respectively, in optimisation.
4
Introduction to linear quadratic optimal control
(contd)
The linear quadratic optimal control problem can
be stated as

• The plant is allowed to be linear time-varying
  (LTV).
• Q(k) and R(k) can be time-varing.
• Control law is LTV.

5
Quadratic cost function
The quadratic cost function
is considered because the development is simple
and the cost function is logical. For example,
consider a 2nd-order single output system
Suppose we want to drive the output to zero and
hence choose the cost function to contain y2(k),
that is
So the cost function in this example appears in
quadratic form naturally.
6
Consider the quadratic function

• F is a scalar.
• Matrix Q can be assumed to be symmetric with no
  loss of generality.
• If the quadratic form F is positive semidefinite,
  then

Minimising F will minimise the magnitude of
states that contribute to F. For example, if
then minimising F will tend to minimise x1 and
x2. But if
then minimising F will also minimise x1 and
x2, but x1 should be much smaller than x2.
Thus Q is a weighting matrix.
7
Consider now the contribution of control input
u(k)
Matrix R can be assumed to be symmetric with no
loss of generality. If the quadratic form G is
positive definite, then
Minimising G will minimise the control input. G
should NOT be allowed to be positive semidefinite.
Consider the total quadratic cost function
If R(k)0, minimising JN would force x(k) toward
zero quickly, which would require a large u(k).
But for physical systems, u(k) is always bounded.
Thus, R(k) is added to the cost function to
limit u(k) to physically realisable values.
8
Principle of optimality
The optimal control design problem can be solved
by different approaches. The approach to be used
here is through the principal of optimality. The
principal can be stated as
If a closed-loop control u(k)fx(k) is optimal
over the interval 0kN, it is also optimal over
any subinterval mkN, where 0mN.
Define the scalar Fk as
Then the cost function
can be expressed as
9
Let Sm be the cost from the (N-m1)th sample
instant to the Nth sample instant,
where m can vary from 1 to (N1). The principle
of optimality states that if JN is the optimal
cost, so is Sm.
The principle is applied as follows. First
minimise S1FN, then choose FN-1 to minimise
where the superscript o denotes optimal. Then
choose FN-2 to minimise
and so on, until SN1JN is minimised. This
procedure is also known as dynamic programming.
10
Example - principle of optimality
Consider a first-order plant
Determine the control law u(k) that minimises
First, minimise S1.
Choose u(2) to minimise S1 and u(2) is obtained by
Next, minimise S2.
Choose u(1) to minimise S2 and u(1) is obtained by
11
Last, minimise S3J2.
Choose u(0) to minimise S3 and u(0) is obtained by
Thus the min cost is
and the optimal control law is generated by

• Observations from the example
• Although the plant is LTI, the control law
  u(k)-K(k)x(k) is time-varying.
• Optimisation is done is reverse time.

12
Linear quadratic optimal control
We recall that the linear quadratic optimal
control problem is formulated as
We consider only an LTI plant and cost function,
hence
13

• Remarks about the derivation (P390-P392
  textbook)
• From the principle of optimality, Sm1, expressed
  by

is minimised by u(N-m) and u(N-m) is obtained by

• The optimal gain matrix K(N-m) is obtained from

• Matrices P(N-m) and K(N-m) are solved recursively
  with P(N)Q and K(N)0.

14
Example - linear quadratic optimal control
Design an optimal control law for the 1st-order
plant
that minimises
From the equations given, the required parameters
in the design equations are
Since the design process goes backward in time,
we start from N2.
15
With K(1) and P(2), we can get P(1).
From P(1), K(0) is obtained.
With K(0) and P(1), we can get P(0).
So the min cost is
The optimal control law is given by