CHEE825

1
CHEE825

• System Identification
• J. McLellan
• Fall 2005

2
Module 1

• Introduction and Motivation

3
What do dynamic data look like?
4
Example - Average Weekly Gasoline Prices in the
U.S.
5
Example - Gasoline Prices

• How can we model this?
• Inputs?
• Inertia?
• Continuous vs. discrete time?
• The Time Series model framework
• Structure
• Lagged regression perspective

6
Outline

• dynamic models
• role
• sources of variability and need for disturbance
  models
• components in data
• statistical properties of a model - sensor
  analogy
• making a link to linear regression
• problem formulation and assumptions
• parameter estimates properties
• role of experimental design
• motivating example - 1st order process
• lagged regression problem
• impact of non-ideality in disturbances - serial
  correlation

7
Role of Dynamic Models

• process characterization
• analysis of process dynamics
• analysis of disturbance dynamics
• process control
• loop configuration and tuning (e.g., PID control)
• model-based control algorithms
• cf. Internal Model Control (IMC) formalism
• minimum variance control (MVC)
• model predictive control (MPC) - e.g., Dynamic
  Matrix Control
• process monitoring
• controller performance assessment, process
  tracking

8
Control Applications

• models implemented as
• step response models - primarily for MPC
• impulse response models - MPC
• transfer functions - MVC, IMC, occasionally MPC
• range of disturbance estimation approaches
• step disturbances - compare predicted vs.
  current measured value - e.g., in MPC
• time series approaches
• MPC reference - survey of Badgwell and Qin

9
Sources of Variability

• process disturbances
• internal
• flow fluctuations
• deterioration - e.g., channeling in a reactor
• external
• from upstream units
• ambient conditions - e.g., air temperature
• operator interventions

10
Sources of Variability

• instrumentation and measurements
• electronic noise
• physical location of instrument
• single thermocouple for tank - impact of mixing
• sampling - e.g., for gas chromatograph
• models and process representation
• unmodeled components become lumped as
  disturbances
• e.g., ignoring radial profiles in a reactor
• model simplification
• ignored dynamic behaviour

11
Steady-State Data

• no time variation
• does this mean the time trace is a straight line?
• no - we have noise !
• focus on whether variability patterns
  (distributions) are changing
• is mean value constant?
• is variance constant?

12
Dynamic Data

• responses changing in time
• typically have process inertia
• serial or autocorrelation
• values now depend on earlier values
• denominator in a transfer function
• variability patterns changing - e.g., variance

13
Components in Data

• deterministic
• non-random relationships
• physical relationships
• e.g., energy/material balance relationships in
  column influence composition in overhead
• stochastic
• random fluctuations - variability pattern
• frequently disturbances
• can evolve in time

14
Matrix of Model Scenarios
15
Examples

• step response white noise
• dynamic deterministic static stochastic
• white noise is purely uncorrelated noise
• step response integrating noise
• dynamic deterministic dynamic stochastic
• inferred property model
• static deterministic dynamic stochastic

16
The Task of Dynamic Model Building

• partitioning process data into a deterministic
  component (the process) and a stochastic
  component (the disturbance)

time series model
process
disturbance
transfer function model
?
17
Dynamic vs. Steady State Disturbance Cases

Static (steady state) Disturbance
Dynamic Disturbance
local trends
no consistent local trends
18
Example - Random Shocks into a Tank

• consider tank and vary the time constant
• approach an integrator
• transfer function
• referred to as an autoregressive (AR) time
  series model
• d is the AR parameter

Note the vertical scales of the plots
19
Example Industrial Step Response Data

• Deterministic component
• Step response
• Limited amount of noise
• Variation about the step response

some variabilityabout deterministictrend
20
Outline

• dynamic models
• role
• sources of variability and need for disturbance
  models
• components in data
• statistical properties of a model - sensor
  analogy
• making a link to linear regression
• problem formulation and assumptions
• parameter estimates properties
• role of experimental design
• motivating example - 1st order process
• lagged regression problem
• impact of non-ideality in disturbances - serial
  correlation

21
Sensor Analogy

• hard sensor - e.g., thermocouple
• consider accuracy and reproducibility/precision
• accuracy
• is there persistent bias?
• where is the mean of the readings relative to
  true value?
• reproducibility
• how consistent are the measurements?
• variability of the sensor

22
Sensor Accuracy

number of samples
value
true value
sensor avg
bias
23
Sensor Reproducibility

better consistency
number of samples
poor consistency
value
24
How does accuracy apply to dynamic models?

• Bias in model predictions due to
• incorrect model form
• for process component
• for disturbance component
• poor data
• experiment too short
• inadequate dynamic content in MV signal
• violation of model estimation assumptions
• white noise (uncorrelated) vs. correlated noise
  present

25
How does reproducibility apply to dynamic models?

• consistency of the predictions
• consistency variability
• arises from uncertainty associated with the
  parameter estimates

parameter 1
parameter 2
26
How does reproducibility apply to dynamic models?

• variability associated with parameter estimates
  is influenced by data collection, model structure
• form of summation dictated by model type

27
Outline

• dynamic models
• role
• sources of variability and need for disturbance
  models
• components in data
• statistical properties of a model - sensor
  analogy
• making a link to linear regression
• problem formulation and assumptions
• parameter estimates properties
• role of experimental design
• motivating example - 1st order process
• lagged regression problem
• impact of non-ideality in disturbances - serial
  correlation

28
Linear Regression Model
response
regressor input
noise (disturbance)

• Given -
• N sets of (x,y) data
• estimate parameters

29
Least Squares Estimation

• Minimize sum of squared prediction errors

o
responses
o
o
o
o
o
x
30
Linear Regression Assumptions

• independent noise at each measurement point
• normally distributed noise
• constant variability patterns
• mean, variance
• independent, identically distributed (IID) Normal
• noise is white
• constant distribution
• no trends

31
Matrix Formulation - Linear Case

• Model
• Least squares estimates

operating points for msmts. design matrix
observations
32
Interpretation - Columns of X

• values of a given variable at different operating
  points -
• entries in XTX
• dot products of vectors of regressor variable
  values
• related to correlation between regressor
  variables
• form of XTX is dictated by experimental design
• e.g., 2k design - diagonal form

33
Parameter Estimation - Graphical View

• approximating observation vector

residual vector
observations
34

Parameter Estimation - Nonlinear Regression Case
approximating observation vector
residual vector
observations
model surface
35
Properties of LS Parameter Estimates

• Key Point - parameter estimates are random
  variables
• because of how stochastic variation in data
  propagates through estimation calculations
• parameter estimates have a variability pattern -
  probability distribution and density functions
• Unbiased
• average of repeated data collection /
  estimation sequences will be true value of
  parameter vector

36
Properties of Parameter Estimates

• Consistent
• behaviour as number of data points tends to
  infinity
• with probability 1,
• distribution narrows as N becomes large
• Efficient
• variance of least squares estimates is less than
  that of other types of parameter estimates

37
Properties of Parameter Estimates

• Covariance Structure
• summarized by variance-covariance matrix

structure dictated by experimental design
variance of noise
38
Prediction Variance

• in matrix form -
• where is vector of conditions at k-th data
  point

39
Joint Confidence Regions

• Variability in data can affect parameter
  estimates jointly depending on structure of data
  and model

section of sum of squares (or likelihood) functio
n
marginal confidence limits
40
Role of Experimental Design

• role of regressor values in data set is clearly
  seen from covariance and prediction variance
  expressions
• choose experimental factor levels to -
• produce uncorrelated parameter estimates
• XTX will be diagonal
• minimize parameter estimate variances
• minimize prediction variance

41
Outline

• dynamic models
• role
• sources of variability and need for disturbance
  models
• components in data
• statistical properties of a model - sensor
  analogy
• making a link to linear regression
• problem formulation and assumptions
• parameter estimates properties
• role of experimental design
• motivating example - 1st order process
• lagged regression problem
• impact of non-ideality in disturbances - serial
  correlation

42
Hot Oil Tank
TI
TIC
Input Sequence
Step
OR
PRBS
FI
43
Process Model 1

• In difference equation form
• impulse response model with additive white noise
• current output depends on past control moves
• noise structure satisfies assumptions of standard
  regression
• for deterministic input, matches standard
  regression problem
• deterministic non-random

white noise
In all cases, the et s are independent,
identically distributed Gaussian (Normal) random
shocks
44
Formulation as Linear Regression Problem

matrix of deterministic elements
uncorrelated (white) noise
impulse weights as regression parameters
45
Properties of Impulse Estimates - Case 1

• In this scenario, the impulse weight estimates
  are
• unbiased
• consistent
• best linear unbiased estimator
• essentially, possess standard properties of least
  squares linear regression estimates
• if input is white noise, the analysis is more
  complex
• prove properties by examining dependence of
  estimates on stochastic inputs

46
Process Model 2

• In difference equation form
• AutoRegressive with eXogenous input (ARX) model
• dependence of current temperature on temperature
  at previous sample time
• dependence of current disturbance component on
  value at previous time - disturbance is no longer
  simply white noise
• inertia in disturbance exactly matches that in
  process

47
Another look at the Autoregressive disturbance

• Known as an AR(1) disturbance 1st order
• transfer function
• referred to as an autoregressive (AR) time
  series model
• d is the AR parameter

Note the vertical scales of the plots
48
Process Model 2

• In transfer function form
• OR

process
disturbance
dependence of process and disturbance on
previous values is the same
49
Formulation as Linear Regression Problem

regressor matrix now contains random elements
noise satisfies regression assumptions
50
Properties of the Estimates - Case 2

• In this case, the coefficient estimates are
• consistent
• asymptotically unbiased
• properties proved by examining dependence of the
  estimates on the lagged outputs - stochastic
  regressor elements
• key point is the manner in which disturbance
  appears relative to process dynamics - same
  autoregressive dynamics (denominator term)

51
Process Model 3
nt

• In difference equation form
• Output Error model-
• dependence of current temperature on temperature
  at previous sample time
• form of immediate dependence of current
  disturbance component on value at previous time
  cancels process inertia term
• scenario - disturbance free process output with
  added white noise added - measurement noise
  model

52
Process Model 3
nt

• In transfer function form
• OR

process
disturbance
53
Formulation as Linear Regression Problem

regressor matrix now contains random elements
noise no longer satisfies regression
assumptions - dependence on previous shocks
54
Properties of the Estimates - Case 3

• In this case, the coefficient estimates are
• biased
• not consistent
• likely to have poor precision - variance

55
Conclusions

• need framework to investigate dynamics of
  disturbances and process components, and how
  random components influence estimation method and
  properties of estimates
• standard least squares estimation not always best
  approach - need to examine validity of
  assumptions, particularly nature of the
  disturbances