Intercomparison of low visibility prediction methods

1
Intercomparison of low visibility prediction
methods

• COST-722 (WG-i)
• Frédéric Atger Thierry Bergot (Météo-France)

2
Proposal

• Horizontal surface (2m) visibility
• Main goal learn about the value of the
  different existing methods
• Not a competition

3
Observations

• Participants propose airports where hourly
  visibility observations are available
• WG-i select one airport (validated by MC)
• Selected participant provides observations for 4
   winter  seasons (October to March) from Oct.
  1999 to March 2003
• 2 seasons for adjusting statistical methods and
  models characteristics
• 2 seasons for evaluating existing methods

4
Observed parameters

• Minimum required hourly visibility (reference
  for verification)
• Any potentially useful parameters (according to
  local observation capabilities)
• for initializing models
• as predictors for statistical methods

5
Forecasts

• Forecast basis 00 UTC and 12 UTC
• Lead times 3h to 24h by 3h step
• It is not a competition
• Participants indicate the main characteristics of
  the forecasting method (e.g.  1D model coupled
  to ECMWF  or  MOS based on Aladin )
• Participants are encouraged to provide
  alternative sets of forecasts obtained by
  modifying these characteristics

6
Visibility thresholds

• Depend on WG-ii conclusions (requirements from
  the forecasters and from the customers)
• Proposal
• 200 m (roads)
• 600 m (airports)
• 1000 m (fog)
• 5000 m (mist)
• Participants provide probabilistic or
  deterministic forecasts for as many thresholds as
  possible

7
Verification

• Comparison for a given validity (e.g. 06 UTC) and
  a given lead time (e.g. 6h)
• 2 verification aspects
• Contingency tables ? hit rate and false alarm
  rate ? ROC or  pseudo-ROC  diagram
• Reliability diagram ? Brier Score reliability
  and resolution components
• Deterministic forecasts are considered as a
  special case of probabilistic forecasts

8
ROC and pseudo-ROC curves

• 2 definitions for the False Alarm Rate !

9
Reliability diagram and Brier score decomposition

• BS?(pi-oi)2/N
• BSREL-RESUNC
• REL?nk(pk-ok)2/N
• RES?nk(ok-o)2/N
• UNCo(1-o)

10
Evaluation tasks

• More efficient if performed centrally
• Could be performed by a WG-i participant not
  involved in the intercomparison
• Data (observations and forecasts) should be
  provided in due time and in a defined format by
  the participants
• Alternatively, each participant conducts the
  evaluation of its own forecasts (following common
  verification rules)
• All data and results should circulate among the
  participants

11
Appendix list of additional observed parameters

• 2m temperature
• 2m humidity
• 10m wind
• 1h/3h rainfall
• Total cloud cover
• Soil temperature
• Surface pressure
• Net short wave radiation near the ground
• Soil type (soil vegetation)
• Radio-sounding observations