Predictions from models and data

1
Predictions from models and data

• Michael Frenklach, Andy Packard, Pete Seiler
• Mechanical Engineering
• University of California
• Berkeley, CA
• Support from NSF-CTS. Additional thanks to
  Laurent ElGhaoui, Bernd Sturmfels, Pablo Parrilo
  and Eric Wemhoff.

The Mohammed Dahleh Symposium, Feb 8-9
2
Model-Based Uncertainty Prediction

• Goal Model-based prediction of the range of
  possible outcomes of a physical process knowing
  the outcomes of several related, but different
  processes.
• Motivation Model-based and experimental research
  in chemical reaction networks

D0?e0?
300 Reactions, 53 Species, and 102
Active Parameters
Process P0
Process PN
Process P1
DN?eN
D1?e1
Experiments /Outcomes
Physics-based models of Processes
3
Modeling of Chemical Reaction Networks (CRN)

• CRN develop high-fidelity chemical reaction
  math model, enhancing the predictive capability
  of complex multi-physics models that involve
  (methane) combustion.
• Characterized by several interrelated, relevant
  facts
• Processes are complex, though physics-based
  governing
  equations are widely accepted
• Uncertainty in process behavior exists, but much
  is known
  regarding where the uncertainty lies in the
  governing
  equations (uncertain parameters)
• Semi-isolated aspects of processes are studied
  
  experimentally
• Numerical simulations of processes, with
  uncertain
  parameters fixed to
  certain values, may be performed
  reliably.
• The perspectives weve learned from Robust
  Control
  Theory (RCT) are useful in addressing
  this goal.

4
Reporting of Experimental Results

• The canonical structure of a technical report (a
  paper) is
• Description of experiment apparatus, conditions,
  measured observable
• flow-tube reactors, laminar premixed flames,
  ignition delay, flame speed
• Care in eliminating unknown biases, and assessing
  uncertainty in outcome measurement
• Transport and chemistry models (involve uncertain
  parameters)
• momentum, diffusion, heat transfer
• 10-100s reactions, uncertainty in rate constant
  parameters kATn exp(-E/RT)
• Sensitivities of modeled outcome to parameters in
  chemistry model
• evaluate sensitivities at nominal parameter
  values
• Focus on parameter(s) resulting in high
  sensitivities on the outcome
• Assumptions on parameters not being studied
• freeze low-sensitivity parameters at nominal
  values
• Predict one or two parameter values

5
CRN Data Processing

• Given
• Model Y S(r) with r ??n
• Experimental Data, (outcome,tolerance) (D,e)
• A priori knowledge -1? rk ?1 ?k ?n.
• From this, all that can be concluded is
  S(r)-Dlte.
• But, typically, the procedure is
• Freeze all parameters except one, at the nominal
  
• rk0 for k ? k0
• Find range of the unfrozen parameter
• max/min rk0
• subject to rk0 for k ? k0
• -1? rk0 ?1
  
• S(r)-Dlte
• The reported range is a subset of what can
  actually be inferred from (S,D,e), but the
  implied higher dimensional cube neither contains,
  nor is a subset of the feasible parameter set.

6
Consistent CRN Data Processing

• Instead, find a bound on range in a consistent
  manner. For each k0 (ie., parameter)
• max/min rk0
• subject to -1? rk ?1 ?k
  
• S(r)-Dlte
• Test this on a large database of models and
  measured outcomes
• Models with n102 uncertain parameters
• 77 different experiments (outcome, tolerance)
• Perform this optimization at e0.02D and
  e0.05D.
• Message You cant reduce the uncertainty in any
  parameter from any one experiment. Some
  collaborative data processing is necessary.

Clearly, few experiments, on their own, are able
to reduce the uncertainty of any given
parameter.
7
GRI-Mech and GRI Data Set

• GRI-Mech (1992-present) addresses the
  collaborative data processing for methane
• Methane reaction model 53 Species/300
  Reactions/102 Uncertain constants
• Chemistry(r)
• 77 peer-reviewed, published Experiments/Measured
  Outcomes
• Processes Pi, measured outcomes Di, measurement
  uncertainties ei
• Models of (Experiments/Measured Outcomes)
• Math models involving CFD/chemistry/other
  phenomena
• Surrogate models of (Experiments/Measured
  Outcomes)
• Factorial design of computer experiments, leading
  to quadratic Si(r)
• Optimization to get best fit single parameter
  vector
• Validation
• Features
• Only "raw" data is used - none of the potentially
  erroneous conclusions.
• Treats the experiments as information, and
  combines them all.
• Addresses the "lack-of-collaboration" in the post
  experimental data processing.
• www.me.berkeley.edu/gri_mech

8
Criticisms

• There are criticisms of GRI-Mech. Typically, the
  objections read like
• It's too early to undertake such a project,
  because some fundamental knowledge is still
  lacking
• I am unwilling to rely on a flame measurement to
  extract the value of some fundamental reaction's
  rate properties -- I prefer to do that in
  isolation
• Not all relevant data was included
• The result (one particular number) is different
  from mine
• Root cause for objection is mostly psychological
• distributed effort dilutes any one specific
  contribution (promotion tenure?)
• protection of individuals territory
• complex geometry of feasible set is unappreciated
• ownership
• information technology (no longer valid)
• And...

9
Limitation in GRI-Mech

• GRI-Mech returns a single parameter vector,
  without uncertainty information.
• The nonlinear dependence makes it difficult to
  translate the uncertainty in experimental outcome
  space (77 dimensional) to an uncertainty in the
  parameter-space (102 dimensional).
• GRI-Mech could return the smallest cube in
  parameter-space that is consistent with the
  GRI-Mech Data Set. Computing this yields
• Only 24 of the dimensions have been shrunk, and
  many of those only by a few percent.

10
Predictive capability associated with Reduced Cube

• Now use this method for prediction.
• First find the smallest cube in the parameter
  space that is consistent with 76 of the
  experiments.
• On this cube, compute the range of the 77th
  model.
• Also compute the range of the 77th model on the
  original unit-cube.
• Compare the two predictions (normalized range
  reduction)
• Repeat this for all cases
• Roughly, an outcome prediction of 1?0.5
  
  has been improved to 1?0.4, knowing the
  
  76 experiments as 1?0.05. That
  is
  modest improvement, at
  best.

11
Approaches to Prediction

• Overbounding the feasible parameter set is
  decoupled from the prediction.
• But the benefit is small.

Models/Experiments
A coupled approach that bypasses the
coarse description of the parameter set may
prove more effective.
Overbound of Feasible Parameter Set
Model
Predicted Range
12
An Approach

• The message from the previous results is that the
  data processing should
• Use all the experimental data and models
• Avoid the intermediate coarse description of the
  feasible set
• Facts
• Any r consistent with all experiments yields a
  
  possible outcome of Po, So(r).
• All such possible outcomes constitute the
  predicted
  outcome set of Po.
• Our goal is to understand the extremes of this
  set,
• Problem involves polynomial objective and
  constraints,
  and is nonconvex. Take the viewpoint
  from RCT, and
  go for two types of bounds.

13
Inner Bounds/Outer Bounds

• Compute outer and inner bounds satisfying
• Inner bounds
• Find feasible points, and compute objective
• Local search from randomly selected initial
  conditions
• Off-the-shelf nonlinear, constrained minimization
  tools
• NPSOL (www.sbsi-sol-optimize.com/NPSOL.htm)
• Outer bounds
• For quadratic surrogates, use S-procedure (going
  back to Yacobovitch in control)
• (almost obvious) relying on the S-procedure, it
  is always worse to take the intermediate step,
  overbounding the feasible set with quadratic
  constraints
• As surrogates become general polynomials, move to
  the sum-of-squares hierarchy (Parrilo,
  Zelentsovsky, Barkin)
• SeDuMi (J. Sturm, www.unimass.nl/sturm/software/s
  edumi.html)

14
S-procedure, SOS

• Solving for one endpoint of the predicted outcome
  set reduces to an indefinite quadratic program
• Use the S-procedure to find an outer bound on
  this extreme point
• To find the best upper bound, solve a
  semi-definite programming problem
• Remarks
• In fact, the S-procedure only uses one side of
  each absolute value constraint.
• The sum-of-squares hierarchy can also be used to
  improve the upper bound.

15
New Predictive capability

• Use this method for prediction on the GRI-Mech
  Data Set
• Compute the range of the ith model, consistent
  with the other 76 experiments models.
• Also compute the range of the ith model on the
  original unit-cube.
• Compare the two predictions.
• Repeat this for all i
• Recall old prediction, using intermediate cube.
• New prediction, using experimental constraints
  directly
• Now, an outcome prediction of 1?0.5 has been
  improved to 1?0.15, knowing the 76 experiments
  as 1?0.05.

16
Connections to RCT robustness analysis

• Some similarities with robust control theory
  (specifically, worst-case analysis)
• In particular,
• high-order, uncertain differential equation
  models
• S-procedure to yield outer bounds
• heuristic searches to get inner bounds
• use of experimental data to refine bounds
• evidence of what appears to be good performance
  on a real problem
• Nevertheless, there are differences
• data used to correlate uncertainty, not merely
  shrink a-priori norm bound
• uncertain parameters the same across all
  experiments
• Other differences are
• systems considered are (at least as modeled)
  nonlinear
• unconventional model transformations that in RCT,
  with linear fractional uncertainty on linear
  models, and H? criterion are more rigorous.

17
Possible Implementation...

• In order to do predictions using others surrogate
  models and experimental data, you pay a fee.
• The fee is distributed between
• the organization maintaining the infrastructure,
  and
• the contributors of the surrogate model/data you
  choose to condition on
• Contribution of surrogate model and data is
  voluntary, but beneficial
• explicit model and data are kept secret from
  users
• revenue realized from others relying on your
  information
• No charge to run predictions on extensive, fixed
  training surrogate model database
• from published experiments, with known measured
  outcomes
• assists users in assessing the potential benefits
  of including your (non-free) information in their
  calculation
• Motto Ill pay for your info, but I dont want
  your conclusions