Jnos Madr, Jnos Abonyi and Ferenc Szeifert

1
Genetic Programming forSystem Identification

• János Madár, János Abonyi and Ferenc Szeifert

Department of Process Engineering University of
Veszprém Hungary
ISDA 2004, August 26-28, Budapest, Hungary
2
Contents

• Introduction
• Genetic Programming
• Linear in parameter models and the OLS
• Application Examples
• Summary

3
Introduction
MODEL OF THE PROCESS
PROCESS
Model Structure
Model Parameters
Known parameters From the engineering knowledge
White-box modeling The modeler design
Black-box modeling The modeler select ablack-box
model structure
Identification From input-output databy
optimization
4
Genetic Programming
The Genetic Programming (GP) is able to generate
model structures from input-output measurement of
the system. So the GP is potentially useful for
system identification.

• Main features of GP
• Stochastic nature(like evolutionary algorithms)
• Symbolic optimization algorithm

5
Representation of Equations
One of the most popular method for representing
structures is the binary tree.
Tree structure
Non terminal nodes Operators ,-,,/ Functions
exp(),cos()
Terminal nodes Variables x1, x2 Parameters
p1, p2
6
Genetic Operators Mutation
7
Genetic Operators Crossover
-

x1
/
x2


p1
x1
p1
x2
x1
-

/
x2
x1


p1
x1
p1
x2
x1
8
Scheme of GP
Creation of initialpopulation
Parameteroptimization
Evaluation
Fitnessvalue
End?
Selection
Direct reproduction
Crossover
Mutation
New generation
End
9
Linear in Parameter Models

• Every new potential solution (model structure)
  represents a nonlinear equation.
• In the nonlinear equations there are model
  parameters which must be optimized.

Ill-conditioned nonlinear optimization problems,
may stuck in local minima, etc.
Linear in parameter models
The parameter identification Least Squares (LS)
10
Representation



x1

x3

x1
x1
F2
x2
x2
F3
F1
11
Fitness Evaluation and OLS

• The GP tends to result in complex models, but we
  must balance the transparency against accuracy.
• Noisy input-output data.

Over-parameterized modelwith unnecessary terms.

• Penalty function
• Decreases the fitness value of complex models.
• Not very effective.

• Orthogonal Least Squares
• Linear in parameter models
• Help select the significant model terms.

New approach
Classical approach
12
OLS

• We calculate the error reduction ratio values of
  the terms.
• Based on these values, we eliminate the less
  significant terms.

x1

x1
x3


x1
F1
x2
F2
x1
x1
F2
x2
x2
F3
F1
13
Example 1 input-output model
Simulation
(y) Output 4 noise
(u) Input
Identification by GP
14
Example 1 input-output model
10-10 runs were performed, max. fun. evaluations
1000
Method 1 GP without penalty function and
OLS Method 2 GP with penalty function, without
OLS Method 3 GP with penalty function and OLS
15
Example 2 Polymerization Reactor
Polymerization of methyl-methacrylate(in a
jacketed continuously stirred tank reactor)
According to the literature the process can be
write in the following input-output model
Where the G is a nonlinear function
Input the volumetric flow rate of the
initiator.Output the average molecular weight
of the product.
Input-output order
16
Example 2 Polymerization Reactor
Method 1Every possible polynomial terms, max.
degree 2 y(k) u(k-1) u(k-2)
u(k-1)y(k-1) y(k-4)y(k-4) Free-run mse
Inf One-step ahead mse 7.86
Method 2Method 1 model, but error reduction
ratio gt 0.01 y(k) u(k-1) y(k-1) y(k-2)
Free-run mse 26.8 One-step ahead mse 30.3
17
Example 2 Polymerization Reactor
Method 3Polynomial GP-OLS ,x, error
reduction ratio gt 0.02 Found models with correct
model order 6 The best y(k) u(k-1)u(k-1)
y(k-1) u(k-2) u(k-1)y(k-1) Free-run mse
7.15 One-step ahead mse 6.63
Method 4GP-OLS ,x,/,v, error reduction ratio
gt 0.02 Found models with correct model order 3
The best y(k) u(k-1)u(k-1) y(k-1) u(k-2)
u(k-1)y(k-1) Free-run mse 7.15 One-step
ahead mse 6.63
18
Conclusions

• The GP is an effective algorithm to generate
  model structures from input-output data.
• To avoid the difficult and ill-conditioned
  optimization problems it is worth uses linear in
  parameter models.
• We proposes the use of OLS to balance the
  transparency and accuracy of the model.