Introduction to Model Order Reduction

1
Introduction to Model Order Reduction
II.1 Reducing Linear Time Invariant Systems
Luca Daniel
Thanks to Jacob White, Peter Feldmann
2
Model Order Reduction Linear Time Invariant
Systems

• II.1.a via Modal Analysis
• II.1.b via Rational Function Fitting (point
  matching)
• II.1.c. via Quasi Convex Optimization
• II.1.d via Pade approximation and AWE

3
Introduction to Model Order Reduction
II.1.a Reduction using Modal Analyis
Luca Daniel
Thanks to Jacob White, Peter Feldmann
4
State-Space Description
Dynamic Linear case

• Original Dynamical System - Single Input/Output
• Reduced Dynamical System
• q ltlt N, but input/output behavior preserved

5
Defining Accuracy

• Time-domain response should be close
• For which possible inputs?
• Frequency response should match
• At what frequencies?

6
Matching Frequency Response

• Ensure accuracy for only some inputs?
• Example
• low frequency inputs,
• or some band,
• or some points in the frequency response

matching some part of the frequency response
Original
7
Reminder about Eigenanalysis
8
Reminder about Eigenanalysis Cont.
Decoupled Equations
Output Equation
9
Reminder about Eigenanalysis Cont.
Solving Decoupled Equations
Assuming Zero Initial Conditions
Output Equation
10
Reduced models via mode truncation
Dynamic Linear Case
Output Equation
11
Reduced models via mode Truncation
Dynamic Linear Case
Why?

• Certain modes are not affected by the input
• Certain modes do not affect the output
• Keep least negative eigenvalues (slowest modes)
• Look at response to a constant input

12
Reduced models via mode truncation
Dynamic Linear Case
Heat Conducting bar Results
N100
q1
q3
q10
Exact
Keep qth slowest modes
13
Another way to look at Reduction by Modal Analysis
Transfer Function
Apply Eigendecomposition
elimitate each mode for which this term is small
14
Model Order Reduction via Eigenmode Analysis
Pole-Residue Form
Pole-Zero Form (SISO)

• Ideas for reducing order
• Drop terms with small residues
• Drop terms with large negative (fast
  modes)
• Remove pole/zero near-cancellations
• Cluster poles that are together

15
Eigenmode Analysis Based Reduction Summary

• Advantages
• Conceptually familiar
• Simple physical interpretation retains dominant
  system modes/poles
• Drawbacks
• Relatively expensive have to find the
  eigenvalues first
• Relatively inefficient. For a given model size,
  many other approaches can provide better accuracy
  for the same computational cost
• e.g. Hankel Model Order Reduction
• e.g. Truncated Balance Realization

O(n3)
16
Model Order Reduction Linear Time Invariant
Systems

• II.1.a via Modal Analysis
• II.1.b via Rational Function Fitting (point
  matching)
• II.1.c. via Quasi Convex Optimization
• II.1.d via Pade approximation and AWE

17
Introduction to Model Order Reduction
II.1.b Reduction using Fitting
Luca Daniel
Thanks to Jacob White
18
A canonical form for model order reduction
Assuming A is non-singular we can cast the
dynamical linear system into one canonical form
for model order reduction Note not necessarily
always the best, but the simplest for educational
purposes
19
Model Order Reduction via Rational Transfer
Function Fitting
Original System Transfer Function
rational function
Model Reduction Find a low order (q ltlt N)
rational
function matching
reduced order rational function
20
Rational Transfer Function Fitting Degrees of
Freedom
Reduced Model Dynamical System
coefficients
Reduced Model Transfer Function
coefficients
21
Rational Transfer Function Fitting Degrees of
Freedom (cont.)
Reduced Model Transfer Function
Apply any invertible change of variables to the
state
I
I
Many Dynamical Systems have the same transfer
function!!
22
Rational Transfer Function Fitting via Point
Matching
23
Rational Transfer Function Fitting Point
Matching matrix can be ill-conditioned

• Columns contain progressively higher powers of
  the test frequencies problem is numerically
  ill-conditioned
• also... missing data can cause severe accuracy
  problems

24
Fitting Example
Hard to Solve Systems
Polynomial Interpolation
Table of Data
f
t0 f (t0) t1 f (t1) tN f (tN)
f (t0)
t
t0
t1
t2
tN
Problem fit data with an Nth order polynomial
25
Example Problem
Hard to Solve Systems
Matrix Form
26
Fitting f(t) t
Coefficient Value
Coefficient number
27
Perturbation Analysis
Hard to Solve Systems
Geometric Approach is clearer
Case 1
Columns orthogonal
Case 1
Columns nearly aligned
28
The power series polynomials are nearly linearly
dependent
29
Course Outline
Numerical Simulation Quick intro to PDE
Solvers Quick intro to ODE Solvers Model Order
reduction Linear systems Common
engineering practice Optimal techniques
in terms of model accuracy Efficient
techniques in terms of time and memory
Non-Linear Systems Parameterized Model Order
Reduction Linear Systems Non-Linear Systems
Yesterday
Today
Tomorrow
Thursday
Friday
30
Introduction to Model Order Reduction

• Luca Daniel
• Massachusetts Institute of Technology
• luca_at_mit.edu
• http//onigo.mit.edu/dluca/2006PisaMOR

www.rle.mit.edu/cpg
31
Course Outline
Numerical Simulation Quick intro to PDE
Solvers Quick intro to ODE Solvers Model Order
reduction Linear systems Common
engineering practice Optimal techniques
in terms of model accuracy Efficient
techniques in terms of time and memory
Non-Linear Systems Parameterized Model Order
Reduction Linear Systems Non-Linear Systems
Monday
Yesterday
Today
Tomorrow
Friday
32
Model Order Reduction Linear Time Invariant
Systems

• II.1.a via Modal Analysis
• II.1.b via Ratianal Function Fitting (point
  matching)
• II.1.c. via Quasi Convex Optimization
• II.1.d via Pade approximation and AWE

33
Introduction to Model Order Reduction
II.1.c Reduction using Optimization
Luca Daniel
Thanks to Kin C. Sou, Alexander Megretski
34
Overview

• Optimization based reduction
• Quasi-convex optimization MOR setup
• Solving the MOR setup
• Application examples
• Conclusions

35
Recall Rational Transfer Function Fitting via
Point Matching
36
Optimization based rational fit Model Order
Reduction Setup
From field solver Or measurements
Small stable and passive reduced order model

• Least Square method
• Cast as nonlinear least squares (solved by
  Gauss-Newton)

• Quasi-convex method
• Cast as quasi-convex program (solved by convex
  optimization algorithm)

• Do not consider stability or passivity while
  finding poles (need post-processing)

• Explicitly take care of stability and passivity
  while finding poles

37
Change of variables

• To make our program tractable, we introduce a
  change offrequency variables (bilinear transform)

z frequency variable
Laplace frequency variable
z
s
38
Modified optimal H-inf norm MOR setup
Stability q(z) Schur polynomial (roots inside
unit circle) Passivity, and possibly other
constraints

• Desirable MOR setup to solve
• Feasible set is not convex if m ? 3
• For example,
  
• but
• Problem has not been proved to be NP complete
  either

39
Overview

• Optimization based reduction
• Quasi-convex optimization MOR setup
• Solving the MOR setup
• Application examples
• Conclusions

40
Relaxation
General idea

• Original problem is difficult
• Made easier if some constraints are dropped
  (relaxed)
• Solve the relaxed problem
• Construct original solution from relaxation
• For example, LP relaxation (polynomial time) of
  IP problems (exponential time).

optimal solution
-c

feasible set
41
Relaxation of the H-inf norm MOR setup
Anti-stable term
Stability q(z) Schur polynomial (roots inside
unit circle) Passivity, and possibly other
constraints
Benefit Relaxation equivalent to a quasi-convex
program. Drawback May obtain suboptimal solutions
42
How bad is this relaxation?
THEOREM
Let
such that deg(q) m, q(z) is Schur polynomial
Then
m1th Hankel singular value
43
Change of variables
where a(z) b(z) and c(z) are trigonometric
polynomials
when
Prop Stability ?
44
Passivity

• For SISO systems, passivity means
• H(z) is analytic for zgt1
• H(z)H(z)
• Re(H(z))gt0 for z1 for impedance,

for all frequencies!
Conclusion Stability and passivity
positivity of trigonometric polynomials
45
Equivalent quasi-convex setup
convex set
This is a quasi-convex program, because
defines an intersection of halfspaces and ?
sub-level set is
is again intersection of halfspaces parameterized
by ? and ?
46
Additional constraints

• Can model additional constraints such as

• Bounded real passivity (for scatter parameters)
• Explicit minimization of quality factor error
  (for inductors)
• Weighting of frequency responses
• Point-wise transfer function (and/or
  derivatives) matching

47
Overview

• Optimization based reduction
• Quasi-convex optimization MOR setup
• Algorithm Summary
• Application examples
• Conclusions

48
Summary of QCO algorithm
Step 1 Compute optimal solution a(z),b(z),c(z)
of the relaxation
subject to stability, passivity
Solved for example by the ellipsoid algorithm
Step 2 Compute coefficients of q(z) using the
relation
and q(z) being a Schur polynomial
Step 3 Compute coefficients of p(z) by solving
,stability, passivity
Solved for example by the ellipsoid algorithm
49
Solving quasi-convex programs
(a,b,c,?) current iterate
localization set (e.g. ellipsoid)
Objective oracle, stabilityoracle, passivity
oracle
N
Termination?
Y
N
Stability?
Update localization set
N
Y
and so on
Passivity?
Generate cut
N
N
Y
Decrease ?

All Yes
50
Overview

• Optimization based reduction
• Quasi-convex optimization MOR setup
• Algorithm summary
• Application examples
• Conclusions

51
Example 1 RLC line (MNA)

• RLC line full model 20th order Vasilyev 2004
• Open circuit terminal
• 10th order reduced model by existing PRIMA and
  our QCO

4
2
4
PRIMA (Moment Matching) Model Order Reduction
Quasi Convex Optimization Model Order Reduction
52
Example 2 RF inductor with substrate(from field
solver)

• RF inductor with substrate effect captured by
  layered Greens function Hu Dac 05
• System matrices are frequency dependent
• Full model has infinite order
• Reduced model has order 6

53
Example 3 RF inductor model (from measurement)
Fabricated 7 turn spiral inductor Blue
measurement Red 10th order reduced model
(positive real part constraint imposed)
54
Example 4 Model of graphic card package (from
measurement)

• Industry example of a multi-port device (390
  frequency samples)
• 12th order SISO reduced models are constructed
• Bounded realness constraint is imposed
• Frequency weight is employed

S13
S11
Solid ROM Dot measurement
Solid ROM Dot measurement
55
Example 5 Large IC power distribution grid(from
field solver)

• Power distribution grid (dimension size 7mm,
  wire width 2 µm)
• Blue full model (order 2046)
• Green PRIMA 40th order reduced model
• Red QCO 40th order reduced model (positive real)

3 curves on top of each other
3 curves on top of each other
56
Conclusion

• QCO competes reasonably well in terms of accuracy
  with moment matching (e.g. PRIMA) for reducing
  large systems
• But in addition QCO can reduce models with
  frequency dependent matrices
• QCO is very flexible in imposing constraints such
  as stability and passivity
• QCO can be extended to parameterized MOR problems
  (see IV.2)

57
Model Order Reduction Linear Time Invariant
Systems

• II.1.a via Modal Analysis
• II.1.b via Ratianal Function Fitting (point
  matching)
• II.1.c. via Quasi Convex Optimization
• II.1.d via Pade approximation and AWE

58
Point matching vs. Moment Matching
Point matching can be very inaccurate in between
points
Moment (derivatives) matching accurate around
expansion point, but inaccurate on wide
frequency band
59
Frequency Domain "Moments" (or Taylor
coefficients) of the transfer function
Taylor Series Expansion of the original transfer
function around s0
The Taylor coef. frequency domain moments
derivatives of the transfer function (up to a
constant)
60
Time domain moments of the impulse response

• Definition

61
Connection to the time-domain moments of the
circuit response
Time-domain moments
62
Rational function fitting via moment matching
Pade Approximation (AWE)
63
Rational function fitting via moment matching
Pade Approximation (AWE)

• Step 1 calculate the first 2q moments of H(s)
• Step 2 calculate the 2q coeff. of the Pade
  approx, matching the first 2q moments of H(s)

64
Step 1 calculation of moments simulating
equivalent circuits (AWE)

• Historical note
• Electrical engineers calculated freq. domain
  Taylor coef. by calculating time domain moments,
• synthesizing and simulating circuit networks.
• Specifically the momets can be calculated
  evaluating the asymptotic behaviors of the
  circuit waveforms,
• Hence the name AWE (Asymptotic Waveform
  Evaluation)

65
Step 1 Calculation of moments (algebraically)

• For sparse system
• can use one initial LU decomposition on A
• then solve 2q linear triangular systems for the
  2q moments
• For dense systems
• can use iterative methods and matrix implicit
  matrix-vector products

66
Step 2 Calculation of Pade coeff. (AWE)
For coeff. as solve the following linear system
For coeff. bs simply calculate
67
Heat Conducting Bar
Demonstration Example
State-Space Description
Given the right scaling
68
Keeping Eigenmodes versus matching moments
Dynamic Linear Case
Heat Flow Results
q1
q1
q3
q3
q10
Exact
Exact
Keep qth slowest eigenmodes
Matches q moments
N100
69
Numerical problem for q gt20 (cannot get accuracy)

• matrix powers converge to the eigenvector
  corresponding to the largest eigenvalue.

Vectors will line up with dominant eigenspace!
70
Pade matrix can be very ill-conditioned

• matrix powers converge to the eigenvector
  corresponding to the largest eigenvalue.

Columns become linearly dependent for large q the
problem is numerically very ill-conditioned!
71
Pade matrix can be very ill-conditioned

• matrix powers converge to the eigenvector
  corresponding to the largest eigenvalue.

Columns become linearly dependent for large q the
problem is numerically very ill-conditioned!
72
Example simulation of voltage gain of a filter
with Pade via AWE
73
Model Order Reduction Summary. Linear Time
Invariant Systems

• II.1.a via Modal Analysis
• II.1.b via Rational Function Fitting (point
  matching)
• II.1.c. via Quasi Convex Optimization
• II.1.d via Pade approximation and AWE