Manchester University

1
Manchester University
Electrical and Electronic Engineering
Control Systems Centre (CSC)
A COMPARITIVE STUDY BETWEEN A DATA BASED MODEL
OF LEAST SQUARES AND PARTIAL LEAST SQUARES
ALGORITHMS
Prepared by AWAD R. SHAMEKH
Under supervision of Dr. BARRY LENNOX

10-5-2006
2
The Presentation Contents

• MODELLING AND IDENTIFICATION
• LEAST SQUARES ALGORITHMS
• MULTIVARIATE STATISTICAL METHODES
• SIMULATION RESULTS AND CONCLUSIONS
• FURTHER WORK

3
Modelling and Identification
Modelling is a useful way to consolidate
information about a system and to explore its
characteristics. a model can be constructed
either theoretically or by identification.
The Steps in identifying a model of a process are
as follows

• Collection of data

2.Selection of identification algorithm
3.Selection of a model structure
4.Specifying of Criteria
4

Least squares algorithms
Since its invention in1795 by Gauss, the least
squares technique remains the most popular tool
in the identification field.

The reasons for its popularity are that it does
not contain 1. high-level mathematical analysis
2. it is easy to implement and 3. modification
and extensions have been made to it that
make it extremely robust and applicable
5
Recursive Least Squares algorithm (RLS)
In the recursive computation technique, the
identification of the current parameters is
performed based on the old estimated
parameters and therefore the capacity of memory
storage will be significantly reduced.
Summary of the recursive least squares
1. It is commonly used in on-line
controlling systems. It could be performed
explicitly as in the self tuning regulators or
implicitly as in case of model reference
control.
2. Dose not required a large memory size.
3. It can provide an over view about a system
behaviour, such advantage arises when the
system is subjected to drastic changes in the
operating conditions.


6
Multivariate Statistical Methods
The multivariate statistics is a modern data
analysis technique that has been widely used in
Industry with good results. By using the
multivariate statistical algorithms the data can
be compressed in a manner that retains the
essential information in small number of factors
which describe of how the variables are related
to each other. Principle component analysis
(PCA) and Partial least squares (PLS) are
dominant techniques in multivariate statistics.
Partial Least squares PLS
PLS regression originated in social science by
Herman Wold, 1966, and then Entered in
chemometrics by his son Svante. The PLS
decomposes X and Y data into orthogonal sets of
scores (T,U), loadings (P,Q) and Weights
(W,C) which are evaluated to maximize covariance
between the scores of X and Y.

7
Non-Iterative Partial Least Squares (NIPALS)
The regression coefficient b for the inner
relation is
the X and Y block residuals are calculated as
follows
The same procedure for all Y columns should be
repeated, results PLS parameters vector
8
Recursive Partial Least Squares (RPLS)

In this study the modified kernel PLS
algorithm is implemented to develop a RPLS
Model. As introduced by Dayal and MacGregor, the
algorithm contains the following steps

The covariance matrices should update as
9
the output covariance matrix deflated as
The RPLS model Coefficients are calculated by
10
Simulation Results
Three cases are under taken to demonstrate the
performance of the four types of identification
algorithm, OLS, PLS, U-D RLS and RPLS
1. Correlated data
A set of highly correlated variables denoted
by X-matrix and observations of y-vector
are used to test a model of four different ways
of identification, Ordinary Least Squares (OLS),
NIPALS Partial Least Squares (PLS), U-D
Recursive Least Squares (RLS), and modified
kernel Recursive Partial Least squares (RPLS).
The X-data and y-outputs are
11
Table (4.1.a), parameters of the estimated models
from X y. (LV3)
Table (4.1.b), parameters of the estimated models
from X y. (LV2)
12
(No Transcript)
13
2. Artificial data
The following process transfer function has been
excited by GBN signal And its out put is
estimated by means of OLS, PLS, U-D RLS and RPLS



The objective is to identify the ARX model for
the considered process as in the structure below
at different signal to noise ratio.
Remark In the case of recursive
identification, the output error criterion is
applied
14
(No Transcript)
15
(No Transcript)
16
Table (4.2.a), the estimated parameters compared
with the actual at signal-to-noise ratio (0.5)
all latent variables (4) are considered in the
PLS's estimation.
Table (4.2.b), the estimated parameters compared
the actual .The PLS's estimation are performed
with (LV4) at signal-to-noise ratio (0.05).
17
3. Non-isothermal Continuous Stirred Tank Reactor
(CSTR)
An ARX model of each output variable is
identified individually where the system is
driven by random walk of Arrhenius rate
constant the desired model has the following
structure

The prediction is carried out recursively by U-D
RLS and RPLS
18
(No Transcript)
19
(No Transcript)
20
Conclusions
The study reveals some notes about the studied
and applied algorithms, these are summarized as
1. The motivation behind using the ARX model
is its simplicity and to ensure model
accuracy number of lags should be increased. In
contrast, this leads to a huge regression
vector especially in the fat system data, as in
the case of a distillation column.
2. A system can be identified perfectly with OLS
algorithm assuming that the variables of
regression matrix are independent. But such
conditions in many situations are not
guaranteed, which is related to an unstable
model.
21
3. As it has been documented in the literature
that the importance of the PLS appears in
heavy multivariable systems.
4. From the results apparently, there is on
difference in the model accuracy of U-D RLS
and RPLS. However, in some studies, it has been
shown that RPLS-based model is better than its
competitor, U-D RLS, when they are used in
control design.
22
Further work
1. Survey and analysis of General Model
Predictive control (GPC). 2. Design of Dynamic
Matrix control for the CSTR case study. 3.
Comparison between U-D RLS and RPLS using the
error optimization. technique.
23
Thank you