Probabilistic Prediction

1
Probabilistic Prediction
2
Uncertainty in Forecasting

• All of the model forecasts I have talked about
  reflect a deterministic approach.
• This means that we do the best job we can for a
  single forecast and do not consider uncertainties
  in the model, initial conditions, or the very
  nature of the atmosphere. These uncertainties
  are often very significant.
• Traditionally, this has been the way forecasting
  has been done, but that is changing now.

3
A Fundamental Issue

• The work of Lorenz (1963, 1965, 1968)
  demonstrated that the atmosphere is a chaotic
  system, in which small differences in the
  initialization, well within observational error,
  can have large impacts on the forecasts,
  particularly for longer forecasts.
• In a series of experiments found that small
  errors in initial conditions can grow so that all
  deterministic forecast skill is lost at about two
  weeks.

4
Butterfly Effect a small change at one place in
a complex system can have large effects elsewhere
5
Uncertainty Extends Beyond Initial Conditions

• Also uncertainty in our model physics.
• And further uncertainty produced by our numerical
  methods.

6
Probabilistic NWP

• To deal with forecast uncertainty, Epstein (1969)
  suggested stochastic-dynamic forecasting, in
  which forecast errors are explicitly considered
  during model integration.
• Essentially, uncertainty estimates were added to
  each term in the primitive equation.
• This stochastic method was not computationally
  practical, since it added many additional terms.

7
Probabilistic-Ensemble NWP

• Another approach, ensemble prediction, was
  proposed by Leith (1974), who suggested that
  prediction centers run a collection (ensemble) of
  forecasts, each starting from a different initial
  state.
• The variations in the resulting forecasts could
  be used to estimate the uncertainty of the
  prediction.
• But even the ensemble approach was not possible
  at this time due to limited computer resources.
• Became practical in the late 1980s as computer
  power increased.

8
Ensemble Prediction

• Can use ensembles to estimate the probabilities
  that some weather feature will occur.
• The ensemble mean is more accurate on average
  than any individual ensemble member.
• Forecast skill of the ensemble mean is related to
  the spread of the ensembles
• When ensemble forecasts are similar, ensemble
  mean skill is higher.
• When forecasts differ greatly, ensemble mean
  forecast skill is less.

9
A critical issue is the development of ensemble
systems that provide probabilistic guidance that
is both reliable and sharp.
10
Elements of a Good Probability Forecast

• Reliability (also known as calibration)
• A probability forecast p, ought to verify with
  relative frequency p.
• Forecasts from climatology are reliable (by
  definition), so calibration alone is not enough.

11
Elements of a Good Probability Forecast

• Sharpness (a.k.a. resolution)
• The variance or confidence interval of the
  predicted distribution should be as small as
  possible.

Probability Density Function (PDF) for some
forecast quantity
Sharp
Less Sharp
12
Early Forecasting Started Probabilistically

• Early forecasters, faced with large gaps in their
  nascent science, understood the uncertain nature
  of the weather prediction process and were
  comfortable with a probabilistic approach to
  forecasting.
• Cleveland Abbe, who organized the first forecast
  group in the United States as part of the U.S.
  Signal Corp, did not use the term forecast for
  his first prediction in 1871, but rather used the
  term probabilities, resulting in him being
  known as Old Probabilities or Old Probs to
  the public.
• A few years later, the term indications was
  substituted for probabilities and by 1889 the
  term forecasts received official sanction
  (Murphy 1997).

13
Ol Probs

• Cleveland Abbe (Ol Probabilities), who led the
  establishment of a weather forecasting division
  within the U.S. Army Signal Corps,
• Produced the first known communication of a
  weather probability to users and the public.

Professor Cleveland Abbe, who issued the first
public Weather Synopsis and Probabilities on
February 19, 1871
14
History of Probabilistic Prediction

• The first operational probabilistic forecasts in
  the United States were produced in 1965. These
  forecasts, for the probability of precipitation,
  were produced by human weather forecasters and
  thus were subjective predictions. The first
  objective probabilistic forecasts were produced
  as part of the Model Output Statistics (MOS)
  system that began in 1969.

15
Ensemble Prediction

• Ensemble prediction began an NCEP in the early
  1990s. ECMWF rapidly joined the club.
• During the past decades the size and
  sophistication of the NCEP and ECMWF ensemble
  systems have grown considerably, with the
  medium-range, global ensemble system becoming an
  integral tool for many forecasters.
• Also during this period, NCEP has constructed a
  higher resolution, short-range ensemble system
  (SREF) that uses breeding to create initial
  condition variations.

16
Major Global Ensembles

• NCEP GEFS (Global Ensemble Forecasting System)
  GFS, 21 members every 6 hr, T254 (roughly 50 km
  resolution), 64 levels
• http//www.esrl.noaa.gov/psd/map/images/ens/ens.ht
  ml)
• Canadian CEFS GEM Model, 21 members, 100 km
  grid spacing, 0 and 12Z
• ECMWF 51 members, 62 levels, 0 and 12Z, T399
  (roughly 27 km)
• http//www.ecmwf.int/products/forecasts/d/charts/m
  edium/eps/

17
Variety of Ways to View Ensembles and Their Output
18
(No Transcript)
19
(No Transcript)
20
(No Transcript)
21
Verification
The Thanksgiving Forecast 2001 42h forecast
(valid Thu 10AM)
SLP and winds

• Reveals high uncertainty in storm track and
  intensity
• Indicates low probability of Puget Sound wind
  event

1 cent
11 ngps
5 ngps
8 eta
2 eta
3 ukmo
12 cmcg
9 ukmo
6 cmcg
4 tcwb
13 avn
10 tcwb
7 avn
22
Box and Whiskers NAEFS
23
Major International Global/Continental Ensembles
Systems

• North American Ensemble Forecasting Systems
  (NAEFS) Combines Canadian and U.S. Global
  Ensembles
• http//www.meteo.gc.ca/ensemble/naefs/EPSgrams_
  e.html

24
NCEP Short-Range Ensembles (SREF)

• Resolution of 16 km
• Out to 87 h twice a day (09 and 21 UTC
  initialization)
• Uses both initial condition uncertainty
  (breeding) and physics uncertainty.
• Uses the NMM, NMM-B, and WRF-ARW models (21
  total members)
• http//www.emc.ncep.noaa.gov/SREF/
• http//www.emc.ncep.noaa.gov/mmb/SREF/FCST/COM_US/
  web_js/html/mean_surface_prs.html

25
(No Transcript)
26
NARRE (N. American Rapid Refresh Ensemble)
27
(No Transcript)
28
(No Transcript)
29
British Met Office MOGREPS

• 24 members, 18 km

30
Ensemble Post-Processing

• Ensemble output can be post-processed to get
  better probabilistic predictions
• Can weight better ensemble members more.
• Correct biases
• Improve the width of probabilistic distributions
  (pdfs)

31
BMA (Bayesian Model Averaging) is One Example
32
(No Transcript)
33
There is a whole theory on using probabilistic
information for economic savings

• C cost of protection
• L loss if bad event event occurs
• Decision theory says you should protect if the
  probability of occurrence is greater than C/L

34
Decision Theory Example
Forecast?
YES NO
Critical Event sfc winds gt 50kt Cost (of
protecting) 150K Loss (if damage ) 1M
Hit False Alarm
Miss Correct Rejection
YES NO
150K
1000K
Observed?
150K
0K
35
The Most Difficult Part Communication of
Uncertainty
36
Deterministic Nature?

• People seem to prefer deterministic products
  tell me exactly what is going to happen
• People complain they find probabilistic
  information confusing. Many dont understand
  POP.
• Media and internet not moving forward very
  quickly on this.

37
(No Transcript)
38
Commercial sector is no better
39
A great deal of research and development is
required to develop effective approaches for
communicating probabilistic forecasts which will
not overwhelm people and allow them to get value
out of them.