Postprocessing Ensembles Using Stochastic Enlargement

1
Post-processing Ensembles Using Stochastic
Enlargement
Craig H. Bishop, Naval Research Laboratory,
Monterey, CA Xuguang Wang, CIRES, Boulder
CO Kevin Shanley, Naval Research Laboratory,
Monterey, CA October 12, 2004 National Centers
for Environmental Prediction
2
What is an Ensemble Forecast?

• A collection of forecasts from multiple numerical
  model runs (different initial conditions,
  different models/parameterizations, etc)

NOGAPS Navy Operational Global Atmospheric
Prediction System
Iso-height lines of 500 hPa surface from
FNMOCs NOGAPS ensemble
3
What is a probability forecast?
36 hr probability forecast from NOGAPS ensemble

• Shows the probability of occurrence of an event
  (here, sfc wnds gt 25 kt )
• P(event) ( exceeding threshold) / (total )

4
The Distribution of Verifying Observations Given
an Ensemble Forecast Category

• Suppose one had access to an infinitely long
  historical record of ensemble forecasts and their
  corresponding verifying observations/analyses.
• With such a record, one could define
  distributions of verifying observations for any
  ensemble forecast category.

Example Find distribution of verification for
forecast events in which 4.9Clt (ensemble mean T)
lt 5.1C, and 3.9 lt (ensemble standard deviation of
T) lt 4.1C from historical record.

• If ensemble members drawn from distribution of
  verifications then the probability of an event
  occurring, given the forecast category, is
• P(event) ( members exceeding threshold) /
  (total )

5
Ideal Single model Ensemble Ensemble members
formed by adding random perturbations to control
forecast. Each perturbation drawn from
distribution of errors in control forecast.
Ideal Multi-model Ensemble Ensemble members
formed by adding random perturbations to
verification. Each perturbation drawn from
distribution of errors in control forecast.
Ideal Single Model Ensemble with Model
Error Ensemble members are random perturbations
to control forecast. Each perturbation drawn from
narrower distribution of errors in control
forecast.
6
4-member ensemble
True error Variance
Sample Variance
7
128-member ensemble
True error Variance
Sample Variance
8

• Typical problem of ensemble

Observations fall outside the range of ensemble
with a margin and frequency that cannot be
explained by observational errors.

• Postprocess ensemble with stochastic
  enlargement.
• Add statistical perturbations to each dynamic
  ensemble member to augment the spread (Roulston
  and Smith 2003).

9
Post-processing Ensembles Using Stochastic
Enlargement
Seasonal distribution of ensemble members about
ensemble mean
Seasonal distribution of truth about ensemble mean
(a)
(b)
Addition of stochastic perturbations to single
member
Seasonal distribution of stochastically enlarged
ensemble about ensemble mean
(c)
(d)
10
Best member dressing (Roulston and Smith 2003)

• Idea Statistical perturbations are drawn from
  historical archive of best member errors.

• limitation May still not reliably predict
  forecast error (co)variance. Best-member dressed
  ensemble may be overdispersive or underdispersive.

11
The New Dressing Kernel
Idea let , choose so that
Derived
(a)
(b)
(c)
(d)
12
Experiment on CCM3 ETKF ensemble

• Variables 500mb U over 14 sites of eastern USA
• Verifications NCEP/NCAR reanalysis
• Cross-sample test Training statistics for bias
  and dressing perturbations are built from 1999
  summer. Forecasts and evaluations are made for
  2001 summer.
• Another limitation of the best member method
  Sub-space to identify the best member is
  uncertain.

13
500mb U rank histogram
14
500mb U Brier Score

• All dressed ensembles have significantly better
  scores than the undressed at longer lead times.
• New kernel has significantly better score than
  RS-10d-globe at 1-2day lead times.
• Score of RS-10d-globe is the worst among dressed
  ensembles.
• Bootstrap resampling to test the statistical
  significance

15
Application Cooling degree days (CDD) forecasts
at Boston
CDD definition where Ti is daily average 2m T
for the ith day of Nd-day period

• Application of dressing
• Dress Ti ensemble from CCM3 ETKF outputs to
  augment CDD ensemble.
• Temporal correlation of forecast errors of Ti
  needs to be considered.
• New kernel designed to produce reliable estimate
  of covariance of Ti forecast errors.

16
Rank histogram for CDD ensembles

• New kernel dressed CDD ensemble has more
  reliable spread

17
TABLE 2. CALL/PUT RETURN MEASUREMENTS ON THE WB
DRESSED AND FIX 3-DAY CDD ENSEMBLES.
18
Summary

• The new stochastic enlargement procedure produces
  reliable estimates of the second moment of the
  forecast errors, whereas best member enlargement
  does not.
• ETKF ensemble becomes more skillful after
  stochastic enlargement.
• CDD ensemble augmented by the new kernel is more
  skillful than that augmented by the best member
  method.

19
Stochastic enlargement currently confined to
space of routinely observed variables

• Problem Routinely observed variables are few and
  far between in typical Naval battle-space
  environment.
• Solution Use globally balanced but quick to
  obtain domain filling perturbations to
  stochastically enlarge the ensemble.
• Examples of balanced but quick to obtain
  perturbations
• Random draws from rescaled histories of forecast
  perturbations or from parameterizations of such
  distributions.
• Random draws from distribution based on
  space-time deformation.

20
Space-time deformation stochastic perturbation
normal
Surface pressure change solely due to horizontal
field shift of potential temperature field.
shifted
21
Boundary of mesoscale model
22
Future stochastic enlargement work should account
for fact that there are skilful predictors of
forecast error variance other than ensemble
variance.
23
4-member ensemble
True error Variance
Sample Variance
24
128-member ensemble
True error Variance
Sample Variance
25
End