Diapositiva 1

1
Probabilistic Cuantitative Precipitation Forecast
Calibration and Verification over South America
Juan Ruiz1,2, Celeste Saulo1,2, Soledad
Cardazzo1, Eugenia Kalnay3 1Departamento de Cs.
de la Atmósfera y los Océanos (FCEyN-UBA), 2
Centro de Investigaciones del Mar y la Atmósfera
(CONICET-UBA), 3 University of Maryland
GOAL The aim of this work is to quantify the
improvement in forecasts due to the
implementation of a short range regional ensemble
system over South America. The focus is in the
probabilistic quantitative precipitation forecast
(PQPF) where different methodologies are tested
in order to improve forecast reliability.
Although these methodologies have been applied to
short range forecasts, they can be easily applied
to longer range forecasts and to other variables
as well.
ENSEMBLE GENERATION
RESULTS BREEDING (summer 2002-2003 SALLJEX)

• Several techniques have been tested for ensemble
  generation
• Ensembles based on initial and boundary
  condition perturbation In this case we selected
  methods to generate perturbations with
  information of the errors of the day in order
  to represent the uncertainty in the initial and
  boundary conditions. We used SLAF (Scaled Lagged
  Averaged Forecast) (Dalcher et. al. 1988) and
  Breeding of the Growing Modes (Toth and Kalnay
  1993). Both of them were implemented in the WRF
  regional model.
• Ensemble based on combination of different
  models The MASTER Laboratory Super Model
  Ensemble (SMES) has been used. (Silva Dias et.
  al. 2006). (http//www.master.iag.usp.br/graf_phpl
  ot/index.php) The SMES combines the operational
  forecasts available over South America. Regional
  and global models are included in this ensemble
  as well as forecasts started from different data
  assimilation systems.
• In all cases the 24 and 48 ensemble forecasts
  were verified and calibrated.

The SALLEX and ANA (Agencia Nacional del Agua,
Brasil) rain gauge network data is used for the
verification and calibration of the breeding
ensemble (11 members and 40 km horizontal
resolution). Forecast verification / calibration
is performed independently over 3 different
regions
24 hour forecasts (BSS decomposition)
Uncalibrated (dark red) and its confidence
limits (bootstrap). Grey (control
forecast) Violet (ensemble mean) Green
(HC98) Blue (HC98 with bias correction) Yellow
(Modified HC98). The calibration applied to the
ensemble mean and the control forecast shows the
best results.
PQPF CALIBRATION

• Why we need to calibrate? Because
  probabilities estimated directly from the
  ensemble are not reliable. This is because the
  different members of the ensemble have biases
  which should be corrected before the computation
  of the probabilities.
• How do we calibrate the probabilities? There
  are several methodologies but most of them use a
  dataset composed of forecasts and its
  verification to quantify model biases and to
  construct an statistical model to remove those
  errors from the forecasted probabilities.
• Tested methodologies
• Calibration based in the rank histogram Hamill
  y Colucci (1998) HC98-
• Calibration using the a conditional probability
  approach Gallus and Seagal 2007.

48 hour forecasts (region 1) In this case the
ensemble mean is significantly better than the
control forecast.
Forecast value ( 2.5 mm threshold) Forecast value
is computed using a very simple decision making
model. The score shows the relative increment in
the forecast value over the climatology. The
value is plotted as a function of the cost / loss
ratio (i.e. the ratio between the cost of
protection and the loss produced by the
unexpected occurrence of a particular phenomena)
RESULTS SLAF AND SMES (October December 2006)
Southern Region (South of 20º S), Northern Region
(north of 20º S). Red Uncalibrated Blue Rank
histogram calibration. Circles Conditional
probability calibration for the ensemble
mean. Triangles Conditional probability
calibration for the control forecast. SLAF(left)
and SMES(rigth)
Forecast calibration increases forecast value
over all regions and mainly in cases where the
cost / loss relation is high.
Brier Skill Score
Operational implementation The calibrated and
uncalibrated PQPF obtained from the SMES ensemble
and from the WRF-CIMA forecasts are being
computed every day. These experimental products
are available at CIMA web page
http//wrf.cima.fcen.uba.ar

• The results show that
• SLAF seems to be better over the southern
  region.
• A single forecast can achieve results which are
  similar to the ones obtained with the ensemble
  for very short forecast lead times. (i.e. less
  than 24 hours).
• Calibration significantly improve forecast
  reliability.