Lecture Outline:

1

• Lecture Outline
• A Typical Filtering Problem
• Sequential Estimation
• Foramal Bayesian Solution to the Nonlinear
  Filtering Problem
• The Linear Case -- Classical Kalman Filtering
• Practical Solution of the Nonlinear problem
  Ensemble Kalman Filtering
• SGP97 Case Study

2
Problem is to characterize time-varying soil
saturation at nodes on a discrete grid.
Estimates rely on scattered meteorological
measurements and on microwave remote sensing
observations (brightness temperature). Estimate
must be compatible with relevant mass and energy
conservation equations. State and meas.
equations are nonlinear.
3
Variational methods provide an effective way to
estimate the mode of f yz(yz) for
interpolation and smoothing problems. However,
they have the following limitations

• The adjoint equations for a complex model may be
  difficult to derive, especially if the model
  includes discontinuous functions.
• Most variational methods rely on gradient-based
  search algorithms which may have trouble
  converging.
• Variational data assimilation algorithms tend to
  be complex, fragile, and difficult to implement,
  particularly for large time-dependent problems.
• Variational algorithms generally provide no
  information on the likely range of system states
  around the conditional mode.
• Most variational methods are unable to account
  for time-varying model input errors when problem
  size is large.

Considering these limitations, it seems
reasonable examine other alternatives for the
filtering problem. One option is sequential
estimation.
4
For filtering problems, where the emphasis is on
characterizing current rather than historical
conditions, it is natural to divide the data
assimilation process into discrete stages which
span measurement times.
To illustrate, suppose that the measurements
available at all times through the current time
ti are assembled in a large array Zi Zi z1,
z2, , zi Set of all measurements through time
ti Then the conditional PDF of y(ti), given all
measurements through ti , is f yiZi y(ti)Zi
and the conditional PDF of the state at some
later time (e.g. ti1 ) given the same
measurement information is f y,i1Zi
y(ti1)Zi .
With these defintions we can define two distinct
types of estimation steps
A propagation step -- Describes how the
conditional PDF changes between two adjacent
measurement times (e.g. between ti and ti1). An
update step -- Describes how the conditional PDF
changes at a given time (e.g. ti1) when new
measurements are incorporated.
These steps are carried out sequentially,
starting from the initial time between t0
through the final time tI.
5
The propagation and update steps of the
sequential estimation process are related as
follows
6
Bayes theorem provides the basis for deriving the
PDF transformations required during the
propgation and update steps of the sequential
filtering procedure. For example, when the
random input and the measurement error are
additive and independent from time to time the
transformation from f yi Zi yiZi to f
yi1 Zi yi1Zi is
The transformation from f yi1 Zi yi1Zi to
f yi1 Zi1 yi1Zi1 is
These expressions constitute a formal sequential
solution to the nonlinear filtering problem since
everything on the right-hand side of each
equation is known from the previous
step. Although it is generally not possible to
evaluate the required PDFs these Bayesian
equations provide the basis for a number of
practical approximate solutions to the filtering
problem.
7
When the state and measurement equations are
linear and the input and measurement error PDFs
are multivariate normal the Bayesian propagation
and update expressions yield a convenient
sequential algorithm known as the Kalman filter.
Suppose that the state and measurement equations
have the following forms
If ui , y0 , and ?i are independent multivariate
normal random variables then the conditional PDFs
appearing in the Bayesian propagation and update
equations are all multivariate normal. These PDFs
are completely defined by their means and
covariances, which can be derived from the
general expressions for the moments of a
conditional multivariate normal PDF.
8
The conditional mean and covariance given by the
Bayesian filter for the linear multivariate case
are
Propagation from ti to ti1
Kalman gain
Update at ti1
9
A simple example illustrates the forecast -
correction tradeoff at the heart of the Kalman
filter. Consider the following scalar state and
measurement equations
In this special case the only input is a random
initial condition with a zero mean and a
specified covariance (actually a scalar variance)
Cyy,0. The state remains fixed at this initial
value but is observed with a set of noisy
measurements. The filter equations are
Note that the filter tends to ignore the
measurements (Ki1 ?0 ) when the measurement
error variance is large and it tends to ignore
the forecast (Ki1 ?1) when this variance is
small..
10
In this example the covariance and Kalman gain
both decrease over time and the updated estimate
converges to the true value of the unknown
initial condition. The Kalman gain is smaller
and the convergence is slower when the
measurement error variance is larger.
Note that the Kalman gain approaches 1/t for
large time. This implies that the estimate of
the unknown initial condition is nearly equal to
the sample mean of the measurements for the
conditions specified here (variance of both
initial condition and measurement error 1.0).
11
The general Bayesian nonlinear filtering
algorithm is not practical for large problems.
The Kalman filter is intended for linear problems
but can be applied to weakly nonlinear problems
if the state and measurement equations are
linearized about the most recent updated
estimate. Unfortunately, the resulting extended
Kalman filter tends to be unstable and
computationally inefficient. In order to solve
the large nonlinear problems encountered in data
assimilation we need to take a different
approach.
One option is to use ensemble (or Monte Carlo)
methods to approximate the probabilistic
information conveyed by the conditional PDFs f
yi Zi yiZi and f yi1 Zi
yi1Zi . The basic concept is to generate
populations of random inputs, measurement errors,
and states which have the desired PDFs. This is
easiest to illustrate within the two step
sequential structure adopted earlier
12
Propagation of conditional probability density
(formal Bayesian filtering)
Evolution of random replicates in ensemble
(Ensemble filtering)
y l(ti1 Zi)
Ensemble filtering propagates only replicates (no
PDFs). But how should update be performed? It
is not practical to construct complete
multivariate PDF and update with Bayes theorem.
13
The updating problem simplifies greatly if we
assume py(ti1) Zi1 is Gaussian. Then update
for replicate l is
Ki1 Kalman gain derived from propagated
ensemble sample covariance Covy(ti1) Zi.
After each replicate is updated it is propagated
to next measurement time. No need to update
covariance. This is the ensemble Kalman filter
(EnKF).

• Potential Pitfalls
• Appropriateness of the Kalman update for
  non-Gaussian density functions?
• Need to construct, store, and manipulate large
  covariance matrices (as in classical Kalman
  filter)

14

• Plot below shows ensemble Kalman filter
  trajectories for a simple two state time series
  model with perfect initial conditions, additive
  input error, and noisy measurements at every
  fifth time step
• 20 replicates are shown in green
• Measurements are shown as magenta dots
• The true state (used to generate the
  measurements) is shown in black
• The mean of the replicates is shown in red
• Note how ensemble spreads over time until update,
  when replicates converge toward measurement.

15
It is difficult and not very useful to try to
estimate the complete multivariate conditional
PDF of y from the ensemble. However, it is
possible to estimate means, median, modes,
covariances, marginal PDFs and other useful
properties of the state. Plot below shows
histogram (approximate marginal PDF) for state 1
of the time series example. This histogram is
derived from a 200 replicate ensemble at t 18.
16
SGP97 Experiment - Soil Moisture Campaign
17
Ensemble Kalman Filter Test SGP97 Soil Moisture
Problem
Observing System Simulation Experiment (OSSE)
Measured radiobrightness
18
Synthetic experiment uses real soil, landcover,
and precipitation data from SGP97 (Oklahoma).
Radiobrightness measurements are generated from
our land surface and radiative transfer models,
with space/time correlated model error (process
noise) and measurement error added.
SGP97 study area, showing principal inputs to
data assimilation algorithm
19
Normalized error for open-loop prediction (no
microwave meas.) 1.0 Compare jumps in EnKF
estimates at measurement time to variational
benchmark (smoothing solution). EnKF error
generally increases between measurements.
Increasing ensemble size
20
Error decreases faster up to 500 replicates but
then levels off. Does this reflect impact of
non-Gaussian density (good covariance estimate is
not sufficient)?
21
Ensemble Kalman filter consistently
underestimates rms error, even when input
statistics are specified perfectly. Non-Gaussian
behavior?
22
(No Transcript)
23
Ensemble Kalman filter provides discontinuous but
generally reasonable estimates of model error but
sample problem. Compare to smoothed error
estimate from variational benchmark. Specified
error statistics are perfect.
24

• Although variational methods can work well for
  interpolation and smoothing problems, they
  conceptual and computational deficiencies that
  limit their applicability to filtering problems
• Sequential Bayesian filtering provides a formal
  solution to the nonlinear filtering problem but
  is not feasible to use for large problems.
• Classical Kalman filtering provides a good
  solution to linear multivariate normal problems
  of moderate size but it is not suitable for
  nonlinear problems.
• Ensemble filtering provides an efficient option
  for the solving large nonlinear filtering
  problems encountered in data assimilation
  applications.
• Ensemble propagation characterizes distribution
  of system states (e.g. soil moisture) while
  making relatively few assumptions. Approach
  accommodates very general descriptions of model
  error.
• Most ensemble filter updates are based on
  Gaussian assumption. Validity of this assumption
  is problem-dependent.