Ensemble Probabilistic Forecast and Uncertainty

1
Ensemble Probabilistic Forecast and Uncertainty

• Yuejian Zhu and Zoltan Toth
• Environmental Modeling Center
• EPS Meeting and Discussion
• December 16 2003
• Acknowledgements
• Tim Marchok, Bill Bua and Lacey Holland

2
References

• 1. Toth, Talagrand, Candille and Zhu, 2003
• "Probability and ensemble forecasts" book
  chapter.
• 2. Zhu, Iyenger, Toth, Tracton and Marchok, 1996
• "Objective evaluation of the NCEP global
  ensemble forecasting system"
• AMS conference proceeding.
• 3. Toth, Zhu and Marchok, 2001
• "The use of ensembles to identify forecasts
  with small and large uncertainty".
• Weather and Forecasting
• 4. Zhu, Toth, Wobus, Rechardson and Mylne, 2002
• "The economic value of ensemble-based weather
  forecasts"
• BAMS
• more related articles

3
Ensemble Forecasts

• 1. Why do we need ensemble forecast?
• Look at following schematic diagrams

4
Ensemble Forecasts (continue)
Deterministic forecast
Initial uncertainty
Forecast probability

Verified analysis
5
Simple Measurement
6
Probabilistic Forecast (1)

• Introduce two characteristics
• reliability and resolution
• reliability --- forecast lt-gt
  observation
• the property of statistical consistency
  between predicted
• probabilities and observed frequencies of
  occurrence of
• the event under consideration.
• resolution --- forecast/observation lt-gt
  climatology
• the ability of a forecast system to discern
  sub-sample
• forecast periods with different relative
  frequencies of event
• (the difference between sub-sample and
  overall sample
• climatology).
• reliability and resolution together determine the
  usefulness
• of a probabilistic forecast system.

7
Probabilistic Forecast (2)

• 1. Talagrand Distribution (histogram
  distribution)
• Sorting forecast in order, to check where
  the analysis is falling
• Reliability measurement, system bias
  detected.
• If analysis and model are bias-free
• U sharp --- spread is too small
• flat line --- perfect spread to cover
  uncertainty

avg distribution
8
Probabilistic Forecast (3)

• 1. Talagrand distribution (continue).
• outlier evolution by different leading
  time
• adding up two outliers subtract the
  average.
• ideal forecasts will have zero outliers.

Due to inability of ensemble to capture model
related errors?
9
Probabilistic Forecast (4)

• 2. Outlier maps (2-dimension).
• flow dependent systematic model errors
• . 40 members at 4 different lead time.
• . measured by normalized distance.
• d (AVGe-ANL)/SQRT(1/(n-1) SPRDe)
• The model errors evaluation ( outliers ) is a
  part of model bias estimation ( see map ). The
  "Normalized Distance" is defined as the
  difference of AVE and ANL over the square root of
  normalized ensemble spread. Therefore, the
  positive distance means there are positive biases
  for ensemble forecasts, the negative means there
  are negative biases for ensemble forecasts.
• ---gt show example maps.
• . area where consecutive ensembles fail ( missed
  )
• -- initial errors ? / errors in boundary
  forcing ?
• model development - identify problem
  area.
• . weather system where consecutive ensembles
  fail ( missed )
• -- inability to capture model related
  errors?

10
Probabilistic Forecast (Useful tools)

• 1. Small and large uncertainty
• Productions spaghetti diagram,
• spread plots, norm. spread and RMOP
• for example of evaluation

11
Probabilistic Forecast (useful tools)

• ... Small and large uncertainty.
• 1 day (large uncertainty) 4 days (control)
  10-13 days (small uncertainty)

12
Ensemble Forecast for Uncertainty (1)
13
Ensemble Forecast for Uncertainty (2)
14
Ensemble Forecast for Uncertainty (3)
15
Ensemble Forecast for Uncertainty (4)
16
1. By using equal climatological bins
(e.g. 10 bins, each grid points)2. Counts of
ensember members agree with ensember mean, (same
bin)3. Construct n1 probabilities for n
ensemble members from (2).3. Regional (NH,
weighted) Normalized Accumulated Probabilities
(n1)4. Calculate RMOP based on (3), but 30-d
decaying average.5. Verification information
(blue numbers) historical average (reliability)
17
Ensemble mean
10 Climatological equally likely bins
Example of 1 grid point
10 ensemble forecasts
The value of ensemble members agree to ensemble
mean is 4/10 or 40 (probability) There are 10512
points ( values ) for global at 2.5 2.5 degree
resolution
10 ensemble members could construct 11
probabilities categories, such as 0/10 (0),
1/10(10), 2/10(20), 3/10(30), 4/10(40),
5/10(50), 6/10(60), 7/10(70), 8/10(80),
9/10(90), 10/10(100) Sum of each grid point for
above 11 probabilistic categories by area
weighted and normalized for global or specified
region Get 0/10 1/10 2/10 3/10
4/10 5/10 6/10 7/10 8/10 9/10
10/10 .029 .047 .077
.085 .100 .135 .116 .089 .081
.070 .177 sum of these 1.0
(1.007 here) 2.9 7.6 15.3 23.8
33.8 47.3 58.9 67.8 75.9 82.9 100
accumulated values There is 30-day decaying
average of above values ( last line ) in the
data-base and updated everyday. Assume these are
30-day decaying average values In this case,
point value is 4/10, RMOP value of this point is
33.8
18
Applications and Discussion1.
Spaghetti diagram, spread/norm spread, RMOP
are all updated daily in global ensemble
web-page.2. HPC will add RMOP to their
discussion3. NRL will use our RMOP technique.
Any other possible measurements for probabilistic
forecast and uncertainty?. Any other
variables for RMOP?. How to apply to regional
ensemble?