P1247676901iSnQR

1

Forecast Combination in s2d Simulations
Francisco J. Doblas-Reyes f.doblas-reyes_at_ecmwf.int
European Centre for Medium-Range Weather
Forecasts (ECMWF)
2
Contributors

• DEMETER Consortium
• David B. Stephenson, Caio Coelho (UREADMM)
• Renate Hagedorn, Tim Palmer, Antje Weisheimer
  (ECMWF)

3
Multi-model ensemble approach
4
Multi-model ensemble system

• DEMETER system 7 coupled global circulation
  models

9 member ensembles ERA-40 initial conditions
SST and wind perturbations 4 start dates per
year 6 months hindcasts

• Hindcast production for 1980-2001 (1958-2001)

5
Calibration and combination in s2d

• Calibration, or the ability to issue
  probabilistic forecasts that, on average, verify
  with a similar proportion of cases.
• Combination, or the need to merge information
  from different sources.
• Calibration bias-correction (mean), inflation
  (variance), dressing, CCA- or SVD-based
  adjustments, etc.
• Combination multiple regression, subjective
  weights (model elimination), etc.
• Predictions are calibrated and combined using
  hindcasts.

6
Calibration and combination in s2d

• The information sources for the combination could
  be
• climatology, persistence or any other prior
  information
• dynamical model simulations
• empirical predictions
• previously released predictions (e.g., a
  prediction issued the previous month) or
  predictions from different forecast systems
  (monthly, seasonal, annual, decadal, etc.)
• Calibration and combination can be undertaken in
  a single step using forecast assimilation.

7
Calibration and combination in s2d

• The basic difference with climate change
  experiments is the possibility of performing
  verification.
• Therefore, our basic rule Whatever the method
  used, calibrated and/or combined forecasts
  performance needs to be checked against raw
  single or simple multi-model forecasts.

8
Simple multi-model benefits Reliability
Reliability for T2mgt0, 1-month lead, May start,
1980-2001
Multi-model
9
Calibration versus model combination
Brier skill score (BSS) of simple multi-model
(SMM) probabilistic predictions versus other
multi-model options
1) a simple multi-model constructed from adjusted
single models (ADI)
Several variables, lead times and start dates are
shown in each plot. Plus, diamond and cross
symbols correspond to the tropics, North America
and Europe regions, respectively.
10
Normal multi-variate forecast assimilation

Prior
Likelihood
Posterior
Prior
Likelihood
Posterior
From Stephenson et al. (2005)
11
Forecast assimilation of precipitation
PAGE agricultural extent
PAGE agroclimatic zones
From Coelho et al. (2005)
12
FA gives calibrated downscaled predictions
Southern box
From Coelho et al. (2005)
13
Predictions for different time scales

• Need to combine information from different
  models, but also different time scales, e.g.
  probabilistic seamless forecast system at ECMWF
• ? 1-10 days medium range EPS (TL399L60)
• ? 10 days-1 month monthly forecast system
  (TL255L60)
• ? 1 month-12 months seasonal forecast system
  (TL159L40)

12mth
1mth
10d
01/01
01/02
01/03
15/01
29/01
12/02
26/02
14
Some answers from s2d

• Is weighting appropriate? Yes, when robust
• How should the weights be computed? Using
  hindcasts and in a robust way
• How do weights relate to PDFs? The combination
  method should be probabilistic
• Can weights from different systems be combined?
  Good question
• Can weights from impact models also be combined?
  Combination/calibration can be carried out at any
  stage of the forecast process
• The performance should be compared with an
  unweighted/uncalibrated prediction

15
Further reading

• Barnston, A.G. , S.J. Mason, L. Goddard, D.G.
  Dewitt and S.E. Zebiak, 2003 Multimodel
  ensembling in seasonal climate forecasting at
  IRI. BAMS, 84, 1783-1796.
• Coelho C.A.S., S. Pezzulli, M. Balmaseda, F.J.
  Doblas-Reyes and D.B. Stephenson, 2004 Forecast
  Calibration and Combination A Simple Bayesian
  Approach for ENSO. Journal of Climate, 17,
  1504-1516.
• Doblas-Reyes, F.J., R. Hagedorn and T.N. Palmer,
  2005 The rationale behind the success of
  multi-model ensembles in seasonal forecasting.
  Part II Calibration and combination. Tellus A,
  57, 234-252.
• Stephenson, D.B., C.A.S. Coelho, F.J.
  Doblas-Reyes and M. Balmaseda, 2005 Forecast
  Assimilation A Unified Framework for the
  Combination of Multi-Model Weather and Climate
  Predictions. Tellus A, 57, 253-264.
• Yun, W.T., L. Stefanova, A.K. Mitra, T.S.V.
  Vijay, A. Kumar, W. Dewar and T.N. Krishnamurti,
  2005 A multi-model superensemble algorithm for
  seasonal climate prediction using DEMETER
  forecasts. Tellus A, 57, 280-289.

16
Questions and comments
17
Calibrated South American Precipitation
Forecast Assimilation
Observations
Multi-model

• 3 DEMETER coupled models
• 1-month lead time DJF precipitation
• ENSO composites for 1959-2001
• 16 warm events
• 13 cold events

r0.51
r0.97
r0.28
r0.82
(mm/day)
From Coelho (2005)