Title: GUARANTED SET COMPUTATION Ellipsoidal approach
1GUARANTED SET COMPUTATION Ellipsoidal approach
- S. Lesecq
- GIPSA-Lab, Département Automatique
- INPG-UJF-CNRS
- Suzanne.lesecq_at_gipsa-lab.inpg.fr
2Outline
- Motivation
- Notations and definitions
- Outer bounding ellipsoid
- Algorithm(s)
- Parameterised family
- Simple example
- Factorisation
- Convergence conditions
- Common difficulties
- Choice of the bound
- Industrial context input?
- Extra
- Inner bounding
- Uncertainty in the model
- References
3Motivation - 1
- Discrete-time dynamic system
- Observation
- yk measured output vector
- uk known input vector
- unknown state, process perturbation,
measurement noise - Identification (output linear in the parameters)
-
4Motivation - 2
- Consider a SISO model
- dk regressor, ? parameter vector
- Error bound ek
5Motivation - 3
- Possible simpler sets
- Polytopes
- Parallelotopes
- Orthotopes
- Ellipsoids
- Outer bounding ellipsoid
- Mininum size
- Inner bounding ellipsoid
- Maximum size
?2
? Criterion to be optimised
?1
6Motivation - 4
- Usual criteria
- Determinant criterion (actually log(det))
- Trace criterion
7Notations definitions - 1
- Norm Euclidian
- Unit ball centred on the origin
- Bounded ellipsoid E(c,P) with non empty interior
- c centre
- P PT gt 0 shape and orientation
8Notations definitions - 2
- Particular cases
- Strip unbounded ellipsoid
- Empty interior ellipsoid
Centre c not unique, just satisfy y Cc
?2
?1
9Notations definitions - 3
- Sum of K ellipsoids (prediction)
- Intersection of K ellipsoids (correction)
10Outline
- Motivation
- Notations and definitions
- Outer bounding ellipsoid
- Algorithm(s)
- Parameterised family
- Simple example
- Factorisation
- Examples Identification
- Convergence conditions
- Common difficulties
- Choice of the bound
- Industrial context input?
- Extra
- Inner bounding
- Uncertainty in the model
- References
11Outer bounding ellipsoid - 1
- Recursive algorithm (Fogel and Huang, 1982)
- not normalised form (intersection)
-
P PT gt 0
12Outer bounding ellipsoid - 2
Choice of ?k, ?k ?
13Outer bounding ellipsoid - 3
- Basic OBE algorithm not optimal (Belforte and
Bona, 1985 and 1990)
- Void intersection
-
- ? must be detected
- ? add intersection test (e.g. Pronzato and
Walter, 1994)
- Other classical situation
14Outer bounding ellipsoid - 4
- Usual criteria
- In the litterature, two sets of programs
- (Favier and Arruda, 1996 Tran Dinh, 2005)
- Set 1 minimize the geometrical size of Ek
- Set 2 insure the convergence of ? 2
15Outer bounding ellipsoid - 5
16Outer bounding ellipsoid - 6
17Outer bounding 7 (parameterised family)
- Normalised problem (Durieu, et al., 2001)
- MIMO models
- State estimation (identification) ? Sum,
Intersection - Analytical results K ellipsoids
- Optimisation problem addressed in details
Empty interior
Unbounded (strip)
18Outer bounding 8 (parameterised family)
- Sum of K ellipsoids (Durieu, et al., 2001)
- Problem find
- Theorem 4.1
- The centre of the optimal ellipsoid E for both
problems is given by
Empty interior possible
19Outer bounding 9 (parameterised family)
- Parameterized family ? optimisation can be done
- Theorem 4.2
- c independent of ?
- Solve for ? the problem
- ? generally, suboptimal solution of
20Outer bounding 10 (parameterised family)
- Necessary condition (Lemma 4.1) for E? to be
optimal solution of - Theorem 4.4 Trace criterion ? explicit solution
- Theorem 4.5 recursive ? nonrecursive
approximating ellipsoid - Determinant criterion ? no explicit solution
21Outer bounding 11 (parameterised family)
- Intersection of K ellipsoids (Durieu, et al.,
2001) -
- Problem find
22Outer bounding 12 (parameterised family)
- Theorem 5.1
- Proposition 5.1
23Outer bounding ellipsoid - 13
- Ellipsoid with parallel cuts algorithm (Goldfarb
and Todd, 1982) - Sequential algorithm
- Equivalent to (modified) OBE (Pronzato and
Walter, 1994) - Recursively optimal
- ? i.e. minimal volume ellipsoid containing
E(ck-1,Pk-1) ? Bk
x2
Hk
Ek
Ek-1
x1
24Outer bounding ellipsoid - (example)
- Simulated data
- yk measured output vector
- unknown measurement noise
- (uniform distribution)
- SNR ? 50 dB
- Identification
25Outer bounding ellipsoid - (example)
- Results
- xtheoretic 0.95, 0.05
- c0 0, ?02P 106I
Algo Param min max c ? ?V
FH a 0.9496 0.9509 0.9502 6.51 10-4 1.85 10-14
FH b 0.0497Â 0.0502 0.0499 2.76 10-4 1.85 10-14
DH a 0.9094 0.9914 0.9504 4.10 10-2 6.14 10-10
DH b 0.0380 0.0618 0.0499 1.19 10-2 6.14 10-10
TAN a 0.0509 1.3912 0.9504 4.41 10-1 1.04 10-3
TAN B -0.0499 0.1498 0.0499 9.99 10-2 1.04 10-3
LO A -60.8 62.7 0.9495 61.79 1.12 106
LO B -39.9 40. 1 0.0510 40.04 1.12 106
Set 1 determinant
Set 2
26Outer bounding ellipsoid - (example)
- Evolution of Det (?k2P)
- LO Lozano-Leal and Ortega, 1987
- TAN Tan, et al., 1997
- DH Dasgupta and Huang, 1987
- FH Fogel and Huang, 1982
27Outer bounding ellipsoid - (example)
Figure 2.4 Evolution des paramètres estimés.
28Outer bounding ellipsoid - (example)
29Outer bounding ellipsoid summary
- Two sets of algorithms (Favier and Arruda, 1996
Tran Dinh, 2005) - Minimise the geometrical size of E
- Minimise ?2
- Different formulations of the algorithm
- Equivalence provable for some of them (OBE-EPC)
(Pronzato and Walter, 1994, Tran Dinh, 2005) - (Durieu, et al., 1996 and 2001)
- Parameterised family
- K ellipsoids
- Convexity of criteria
30Outline
- Motivation
- Notations and definitions
- Outer bounding ellipsoid
- Algorithm(s)
- Parameterised family
- Simple example
- Factorisation
- Examples Identification
- Convergence conditions
- Common difficulties
- Choice of the bound
- Industrial context input?
- Extra
- Inner bounding
- Uncertainty in the model
- References
31Factorisation - 1
- Prerequisite
- Orthogonal matrix HT H-1
- Numerically highly suitable
- Orthogonal factorisation
- Factorisation of a product of matrices
32Factorisation - 2
- Factorisation of a sum of matrices
- Least Square problem solution
- Recursive Least Square problem also factorised
33Factorisation - 3
- Parameterised family (Durieu, et al., 1996)
- hypothesis (not restrictive) y? R
- Sum of 2 symmetric matrices ? factorise!
Intersection
- Goldfarb and Todd, 1982
- LDL factorisation
- Cholesky suggested
34Factorisation - 4
- Reformulation ? optimisation problem (Lesecq and
Barraud, 2002) - Let then
35Factorisation - 5
- Factorised algorithm (Lesecq and Barraud, 2002)
- Theoretical property ? ? 0, 1 simpler
demonstration
36Factorisation - 6
- Directly with P (general formulation) (Tran Dinh,
2005)
ck, Mk dependent
37Factorisation - summary
- Absolutely necessary to ensure numerical
stability - Academic example (Lesecq and Barraud, 2002)
- n 8, M hilb(8)hij 1/(ij-1), c ones
(8,1), y 1 - temp invhilb(9), d temp(18,9)
- ? 0.001 ? ?not factorised - 47.6 and
?factirised 1.7 10-2 - Practical problem (identification)
- Theoretical properties easier demonstration
- Parameterised family (Durieu, et al., 1996) ? P
and M algorithms (Lesecq and Barraud, 2002) - General formulation ? P and M algorithms (Tran
Dinh, 2005)
38Outline
- Motivation
- Notations and definitions
- Outer bounding ellipsoid
- Algorithm(s)
- Parameterised family
- Simple example
- Factorisation
- Examples Identification
- Convergence conditions
- Common difficulties
- Choice of the bound
- Industrial context input?
- Extra
- Inner bounding
- Uncertainty in the model
- References
39Examples - 1
- Industrial Data
- 1st example industrial
- Looks like 1st order, 2 parameters
- aim model identification ? diagnosis
- 2nd example LIRMM robot
- 14 parameters
- 14 000 regressors!
- aim model identification, large problem
40Examples - 2
- 1st example recorded on a process (valve)
output
input
41Examples - 3
- 1st example
- Data re-used several times
- Determinant criterion
- ? 0.002
- Measurement
- and regressor
- known
Determinant criterion 1st circulation of data
42Examples - 4
Ellipsoid updating
Circulations param. min max centre ? determinant R()
1 a 0.7967 1.0205 0.9086 0.1119 4.85 10-7 13.7
1 b -0.0133 0.0637 0.0252 0.0385 4.85 10-7 13.7
10 a 0.8479 0.9462 0.8971 0.0491 1.08 10-7 2.4
10 b 0.0145 0.0441 0.0293 0.0148 1.08 10-7 2.4
43Examples - 5
det(10)
det(1)
Gain properly identified
Trace(10)
Trace(1)
44Examples - 7
- 2nd example LIRMM parallel robot
N 3500
45Examples 8
- 2nd example Recorded data (for instance)
Sequential algorithm
46Examples - 9
criterion
No empty intersection
? 6 Nm
- 2nd example parameters (60 circulations)
Parameter min max c ? ?() a priori
Imot1 -0.0251Â 0.1158 0.0454 0.0704 155.2 0.012
Imot2 0.0418Â 0.0580 0.0499 0.0081 16.28 0.012
Imot3 -0.0504 0.0998 0.0247 0.0752 304.1 0.012
Imot4 -0.0469 0.1035 0.0283 0.0753 266. 2 0.012
Mnac 0.1395 0.2893 0.2144 0.0749 34.92 1.0
Inac -0.03613 0.03524 -0.00045 0.0357 799.2 0.0008
F?1 -2.9720 3.7908 0.40939 3.38 825.9 X
F?2 -0.2689 1.2673 0.4992 0.768 153.9 X
F?3 -2.8062 2.9045 0.0491 2.86 581.0 X
F?4 -2.6674 2.7023 0.0174 2.68 1539.1 X
Fs1 -8.0750 9.8545 0.8898 8.96 1007.5 X
Fs2 -1.2478 1.4495 0.1008 1.35 1337.5 X
Fs3 -16.212 16.868 0.3282 16.5 5039.5 X
Fs4 -14.437 15.390 0.47671 14.9 3128.4 X
47Factorisation interest
Parallel robot, det, family of ellipsoids
No factorisation
Thanks to N. Ramdani Work done by N. Ramdani P.
Poignet LIRMM
Factorisation
48Examples - 10
- 2nd example Model reconsidered
- ? split in several models
? 6 Nm
1st model
Parameter min max c ? ?() a priori
Imot1 0.0063 0.0805 0.0434 0.037 85.4 0.012
Mnac -0.1299 0.6809 0.2755 0.405 147. 2 1.0
Inac -0.1994 0.2864 0.0435 0.243 558.4 0.0008
F?1 -1.6799 2.3432 0.3316 2.01 606.6 X
Fs1 -2.9494 5.1822 1.1164 4.07 364. 2 X
49Examples - 11
Center, obtained with ? 6 Nm and 60 circulations
- 2nd example Adaptation of the bound
Far from hypothesis!
- Analysis of data
- Reject outliers
- heavy tail
50Examples - 12
60 circulations
- 2nd example Adaptation of the bound ? heavy
tail
Param. ? 6 Nm ? 6 Nm ? 6 Nm heavy heavy heavy
Param. centre min max ? centre min max ?
Imot1 4.54 10-2 -0.025Â 0.116 7.04 10-2 1.66 10-2 0.014Â 0.019 2.40 10-3
Imot2 4.99 10-2 0.042 0.058 8.12 10-3 1.64 10-2 0.0117 0.021 4.62 10-3
Imot3 2.49 10-2 -0.050 0.099 7.52 10-2 1.75 10-2 0.011 0.024 6.11 10-3
Imot4 2.84 10-2 -0.047 0.104 7.53 10-2 2.32 10-2 0.019 0.027 3.52 10-3
Mnac 2.14 10-1 0.139 0.289 7.49 10-2 9.87 10-1 0.923Â 1.051 6.42 10-2
Inac -6.48 10-4 -0.036 0.035 3.57 10-2 2.94 10-3 0.002Â 0.004 1.19 10-3
F?1 4.15 10-1 -2.972 3.791 3.38 2.13 10-1 0.145Â 0.281 6.78 10-2
F?2 4.99 10-1 -0.269 1.267 7.68 10-1 1.23 10-1 0.067Â 0.179 5.64 10-2
F?3 5.37 10-2 -2.806 2.905 2.86 1.29 10-1 0.059Â 0.199 7.04 10-2
F?4 1.61 10-2 -2.667 2.702 2.68 1.15 10-1 0.011Â 0.219 1.04 10-1
Fs1 8.81 10-1 -8.075 9.855 8.96 1.22 1.059Â 1.374 1.58 10-1
Fs2 1.01 10-1 -1.248 1.449 1.35 1.02 0.807Â 1.239 2.16 10-1
Fs3 2.38 10-1 -16.21 16.87 1.65 101 7.68 10-1 0.378Â 1.159 3.90 10-1
Fs4 4.82 10-1 -14.44 15.39 1.49 101 1.03 0.840Â 1.219 1.89 10-1
det 3.1312 10-9 3.1312 10-9 3.1312 10-9 1.5719 10-45 1.5719 10-45 1.5719 10-45
51Examples - 13
- 2nd example reorganise data
Angle between 2 regressors
52Examples - 14
criterion
?(Para.) 1st circulation 1st circulation 60th circulation 60th circulation
?(Para.) Initial Reorganised Initial Reorganised
?(Imot1) 1.6423 1.5004 10-1 7.0448 10-2 6.9419 10-3
?(Imot2) 5.3308 10-1 1.5788 10-1 8.1202 10-3 7.9782 10-3
?(Imot3) 2.1952 10-1 9.4689 10-2 7.5151 10-2 7.4153 10-2
?(Imot4) 9.6392 10-2 8.9899 10-2 7.5253 10-2 7.5824 10-2
?(Mnac) 3.5286 3.1971 7.4868 10-2 7.3429 10-2
?(Inac) 4.0685 10-2 5.7488 10-2 3.5688 10-2 3.6735 10-2
?(F?1) 7.2910 101 5.5816 3.3814 3.3284
?(F?2) 2.7041 101 5.3418 7.6816 10-1 7.5179 10-1
?(F?3) 7.1152 3.2917 2.8553 2.8145
?(F?4) 2.6920 2.9740 2.6849 2.6892
?(Fs1) 1.9207 102 2.0027 101 8.9647 8.8031
?(Fs2) 8.3393 101 1.8595 101 1.3487 1.3250
?(Fs3) 2.6252 101 1.7720 101 1.6540 101 1.6520 101
?(Fs4) 1.1166 101 1.5965 101 1.4913e 101 1.4925 101
det 7.0667 1014 6.2676 103 3.1312 10-9 2.5372 10-9
53Criterion
T17
det
Initial data
Outliers removed
T23
Nb of Circulations
54Summary
- Ellipsoidal approach
- Large size problem
- (14 parameters,
- 14 000 regressors)
- Data analysis
- Several strips together
- (optimisation)
- Algorithm
- Factorisation
- Uncertain model
- Industrial/real data
- Assumptions? Convergence condition?
- Known inputs?
- Choice of the bound?
- Data analysis
- Choice of the input?
- Excitation
- Analysis of results
- Physical meaning
Bound?
55References
- Bai E. W., Tempo R., Cho H. (1995), Membership
Set Estimators Size, Optimal Inputs, Complexity
and Relations with Least Squares, IEEE
Transactions on Circuits and Systems I
Fundamental Theory and Application, Vol. 42(5),
pp. 266-277. - Bai E. W., Huang Y. F. (1999), Convergent of
optimal sequential outer bounding sets in bounded
error parameter estimation, Mathematics and
Computers in Simulation, Vol. 49, pp. 307-317. - Belforte G. Bona B. (1985), An improved parameter
identification algorithm for signals with
unknown-but- bounded error, 7th IFAC/IFORS Symp.
on identification and system parameter
estimation, pp. 1507-1512. - Belforte G. Bona B. (1990), Parameter estimation
algorithms for a set-membership description of
uncertainty, Automatica, Vol. 26, pp. 887-898. - Durieu C., Polyak B., Walter E. (1996), Trace
versus determinant in ellipsoidal outer bounding,
with application to state estimation, 13th IFAC
World Congress, San Francisco, USA, Vol. I, pp.
9-12. - Durieu C., Walter E., Polyak B. (2001),
Multi-input multi output ellipsoidal state
bounding, Journal of Optim. Theory and
Application, Vol. 111(2), pp. 273-303. - Fogel E., Huang Y. (1982), On the Value of
Information in System Identification Bounded
noise case, Automatica, Vol. 18, pp. 229-238. - Nayeri M., Liu M. S, Deller J. R (1993), An
interpretable and converging set-membership
algorithm, Proceeding IEEE of International
Conference on Acoustics, Speech and Signal
Processing, Vol IV, Minneapolis, New York, pp.
472-475. - Pronzato L., Walter E. (1994), Minimal volume
ellipsoids, International Journal of Adaptive
Control and Signal Processing, Vol. 8, , pp.
15-30. - Schweppe C. F. (1973), Uncertain dynamic systems,
Englewood Cliffs, NJ, Prentice-Hall.
56References
- Dasgupta S., Huang Y. F. (1987), Asymptotically
Convergent Modified Recursive Least-square with
Data-dependent Updating and Forgetting Factor for
Systems with Bounded Noise, IEEE Trans. on
Information Theory, Vol. IT 33(3), pp. 383-393. - Lozano-Leal R., Ortega R. (1987), Reformulation
of the Parameter Identification Problem for
Systems with Bounded Disturbances, Automatica,
Vol. 23(2), pp. 247-251. - Favier G., Arruda L.V.R. (1996), Review and
Comparison of Ellipsoidal Algorithms, In Bounding
approaches to System Identification, edited by M.
Milanese et al., Plenium Press, New York, pp.
43-68. - Tran Dinh Q.K. (2005), Contribution Ã
lestimation ensembliste ellipsoidale, PhD
thesis, INPG, Grenoble, France (in French). - Goldfarb D., Told M.J. (1982), Modification and
implementation of the ellipsoid algorithm for
linear programming, Mathematical Programming,
Vol. 23, pp. 1-19. - Lesecq S., Barraud A. (2002), Une approche
factorisée plus simple et numérique-ment stable
pour l'estimation ensembliste, JESA, Vol. 36(4),
pp. 505-518. - Veres S.M., Norton J.P. (1991), Structure
selection for bounded-parameter models
consistency conditions and selection criterion,
IEEE Transactions on Automatic Control, Vol.
36(4), pp. 474-481. - Sedda E. (1998), Estimation en ligne de létat et
des paramètres dune machine asynchrone par
filtrage à erreur borne et par filtrage de
Kalman, PhD thesis, ENS Cachan (in French).
57References
- Lin T.M., Nayeri M., Deller J.R. (1998), A
consistently convergent OBE algorithm with
automatic estimation of error bounds,
International Journal of Adaptive Control and
Signal Processing, Vol. 12, pp. 305-324. - Polyak B., Nazin S., Durieu C., Walter E. (2004)
Ellipsoidal parameter or state estimation under
model uncertainty, Automatica, Vol. 40, pp.
1171-1179. - Clement T., Gentil S. (1990), Recursive
membership set estimation for output error
models, Mathematics and Computers in Simulation,
Vol. 32, pp. 505-513. - Ferreres G., MSaad M. (1997), Estimation of
output error models in the presence of unknown
but bounded disturbances, International Journal
of Adaptive Control and Signal Processing, Vol.
11, pp. 115-140.