Probabilistic Prediction

1
Probabilistic Prediction
2
Uncertainty in Forecasting

• All of the model forecasts I have talked about
  reflect a deterministic approach.
• This means that we do the best job we can for a
  single forecast and do not consider uncertainties
  in the model, initial conditions, or the very
  nature of the atmosphere. These uncertainties
  are often very significant.
• Traditionally, this has been the way forecasting
  has been done, but that is changing now.

3
A More Fundamental Issue

• The work of Lorenz (1963, 965, 1968) demonstrated
  that the atmosphere is a chaotic system, in which
  small differences in the initializationwell
  within observational error can have large
  impacts on the forecasts, particularly for longer
  forecasts.
• Similarly, uncertainty in model physics can
  result in large forecast differences..and errors.
• Not unlike a pinball game.
• Often referred to as the butterfly effect

4
Probabilistic-Ensemble NWP

• One approach would be to add uncertainty terms to
  all terms in the primitive equations. Not
  practical.
• Another Instead of running one forecast, run a
  collection (ensemble) of forecasts, each starting
  from a different initial state or with different
  physics. Became practical in the late 1980s as
  computer power increased.

5
Ensemble Prediction

• The variations in the resulting forecasts could
  be used to estimate the uncertainty of the
  prediction. Can use ensembles to provide a new
  generation of products that give the
  probabilities that some weather feature will
  occur.
• Can predict forecast skill or forecast
  reliability!
• It appears that when forecasts are similar,
  forecast skill is higher.
• When forecasts differ greatly, forecast skill is
  less.
• The ensemble mean is usually more accurate on
  average than any individual ensemble member.

6
Probabilistic Prediction

• A critical issue will be the development of
  mesoscale ensemble systems that provide
  probabilistic guidance that is both reliable and
  sharp.

7
Elements of a Good Probability Forecast

• Reliability (a.k.a. calibration)
• A probability forecast p, ought to verify with
  relative frequency p.
• Forecasts from climatology are reliable (by
  definition), so calibration alone is not enough.

8
Elements of a Good Probability Forecast

• Sharpness (a.k.a. resolution)
• The variance, or confidence interval,of the
  predicted distribution should be as small as
  possible.

Probability Density Function (PDF) for some
forecast quantity
Sharp
Less Sharp
9
Early Forecasting Started Probabilistically

• Early forecasters, faced with large gaps in their
  nascent science, understood the uncertain nature
  of the weather prediction process and were
  comfortable with a probabilistic approach to
  forecasting.
• Cleveland Abbe, who organized the first forecast
  group in the United States as part of the U.S.
  Signal Corp, did not use the term forecast for
  his first prediction in 1871, but rather used the
  term probabilities, resulting in him being
  known as Old Probabilities or Old Probs to
  the public.
• A few years later, the term indications was
  substituted for probabilities and by 1889 the
  term forecasts received official sanction
  (Murphy 1997).

10
Ol Probs

• Cleveland Abbe (Ol Probabilities), who led the
  establishment of a weather forecasting division
  within the U.S. Army Signal Corps,
• Produced the first known communication of a
  weather probability to users and the public.

Professor Cleveland Abbe, who issued the first
public Weather Synopsis and Probabilities on
February 19, 1871
11
History of Probabilistic Prediction

• The first operational probabilistic forecasts in
  the United States were produced in 1965. These
  forecasts, for the probability of precipitation,
  were produced by human weather forecasters and
  thus were subjective predictions. The first
  objective probabilistic forecasts were produced
  as part of the Model Output Statistics (MOS)
  system that began in 1969.

12
Ensemble Prediction

• Ensemble prediction began an NCEP in the early
  1990s. ECMWF rapidly joined the club.
• During the past decades the size and
  sophistication of the NCEP and ECMWF ensemble
  systems have grown considerably, with the
  medium-range, global ensemble system becoming an
  integral tool for many forecasters.
• Also during this period, NCEP has constructed a
  higher resolution, short-range ensemble system
  (SREF) that uses breeding to create initial
  condition variations.

13
NCEP Global Ensemble System

• Begun in 1993 with the MRF (now GFS)
• First tried lagged ensembles as basisusing
  runs of various initializations verifying at the
  same time.
• For the last ten years have used the breeding
  method to find perturbations to the initial
  conditions of each ensemble members.
• Breeding adds random perturbations to an initial
  state, let them grow, then reduce amplitude down
  to a small level, lets them grow again, etc.
• Give an idea of what type of perturbations are
  growing rapidly in the period BEFORE the
  forecast.
• Does not include physics uncertainty.
• Coarse spatial resolution..only for synoptic
  features.

14
NCEP Global Ensemble

• At 00Z
• T254L64 high resolution control) out to 7 days,
  after which this run gets truncated and is run
  out to 16 days at a T170L42 resolution
• T62 control that is started with a truncated T170
  analysis
• 10 perturbed forecasts each run at T62 horizontal
  resolution. The perturbations are from five
  independent breeding cycle.
• At 12Z
• T254L64 control out to 3 days that gets truncated
  and run at T170L42 resolution out to 16 days
• Two pairs of perturbed forecasts based on two
  independent breeding cycles (four perturbed
  integrations out to 16 days.

15
(No Transcript)
16
(No Transcript)
17
(No Transcript)
18
(No Transcript)
19
Verification
The Thanksgiving Forecast 2001 42h forecast
(valid Thu 10AM)
SLP and winds

• Reveals high uncertainty in storm track and
  intensity
• Indicates low probability of Puget Sound wind
  event

1 cent
11 ngps
5 ngps
8 eta
2 eta
3 ukmo
12 cmcg
9 ukmo
6 cmcg
4 tcwb
13 avn
10 tcwb
7 avn
20
NCEP Short-Range Ensembles (SREF)

• Resolution of 32 km
• Out to 87 h twice a day (09 and 21 UTC
  initialization)
• Uses both initial condition uncertainty
  (breeding) and physics uncertainty.
• Uses the Eta and Regional Spectral Models and
  recently the WRF model (21 total members)

21
SREF Current System
Model Res (km) Levels Members Cloud
Physics Convection RSM-SAS 45 28 Ctl,n,p
GFS physics Simple Arak-Schubert RSM-RAS
45 28 n,p GFS physics Relaxed
Arak-Schubert Eta-BMJ 32 60 Ctl,n,p Op
Ferrier Betts-Miller-Janjic Eta-SAT
32 60 n,p Op Ferrier BMJ-moist
prof Eta-KF 32 60 Ctl,n,p Op
Ferrier Kain-Fritsch Eta-KFD 32 60 n,p Op
Ferrier Kain-Fritsch with
enhanced detrainment
PLUS NMM-WRF control and 1 pert. Pair
ARW-WRF control and 1 pert. pair
22
There is a whole theory on using probabilistic
information for economic savings

• C cost of protection
• L loss if bad event event occurs
• Decision theory says you should protect if the
  probability of occurrence is greater than C/L

23
Decision Theory Example
Forecast?
YES NO
Critical Event sfc winds gt 50kt Cost (of
protecting) 150K Loss (if damage ) 1M
Hit False Alarm
Miss Correct Rejection
YES NO
150K
1000K
Observed?
150K
0K
24
The Most Difficult Part Communication of
Uncertainty
25
Deterministic Nature?

• People seem to prefer deterministic products
  tell me exactly what is going to happen
• People complain they find probabilistic
  information confusing. Many dont understand
  POP.
• Media and internet not moving forward very
  quickly on this.

26
(No Transcript)
27
(No Transcript)
28
Icons are not effective in providing probabilities
29
Even worsethey use the same icons for likely
rain and rain as they do for chance rain. Also,
they used likely rain for 70 on this page and
chance rain for 70 in the example on the
previous page
30
And a slight chance of freezing drizzle reminds
one of a trip to Antarctica
31
Commercial sector is no better
32
A great deal of research and development is
required to develop effective approaches for
communicating probabilistic forecasts which will
not overwhelm people and allow them to get value
out of them.