Evaluation of AMPS Forecasts Using SelfOrganizing Maps SOMs

1
Evaluation of AMPSForecasts UsingSelf-Organizing
Maps (SOMs)

• John J. Cassano
• Cooperative Institute for Research in
  Environmental Science
• and
• Program in Atmospheric and Oceanic Sciences
• University of Colorado at Boulder

Beardmore Glacier, Jan 2004
2
Outline

• What are SOMs?
• Application of SOMs for model evaluation studies
• Application of SOM Analysis to AMPS data
• Conclusions / Future Work

3
What are SOMs?

• SOM - Self-Organizing Map
• SOM technique uses an unsupervised learning
  algorithm (neural net)
• Clusters data into a user selected number of
  nodes
• SOM algorithm attempts to find nodes that are
  representative of the data in the training set
• More nodes in areas of observation space with
  many data points
• Fewer nodes in areas of observation space with
  few data points
• SOMs are in use across a wide range of
  disciplines

4
Application of SOMsfor Model Evaluation Studies

• Synoptic pattern classification
• Frequency of occurrence of synoptic patterns
• Determine model errors for different synoptic
  patterns

5
Application of SOM Analysis to AMPS Data

• Train SOM with AMPS SLP data
• Result is a synoptic pattern classification
• Evaluate frequency of occurrence of synoptic
  patterns predicted by AMPS as a function of
  forecast duration
• Map 0h, 24h, and 48h forecasts to SOM
• Mis-mapping of AMPS forecasts
• Model validation statistics for specific synoptic
  patterns (ongoing work)
• Calculate model error statistics at points of
  interest (Willie Field) for different synoptic
  patterns
• Are certain synoptic patterns prone to bias (e.g.
  error in predicted wind speed or direction)?

6
AMPS Data for SOM Analysis

• SLP over Ross Sea sector of AMPS 30 km model
  domain
• Summer only (NDJ)
• 00Z AMPS simulations from Jan 2001 through Feb
  2003
• 186 model simulations
• Evaluate 0, 24, and 48 h AMPS forecasts

7
AMPS SOM Analysis Domain
8
Synoptic Pattern Classification
9
Frequency of Occurrence
10
Misprediction of Synoptic Patterns

• Consider all of the time periods for which the
  model analyses map to a particular node
• For these time periods determine which nodes the
  model predictions map to
• From this analysis we can determine biases in the
  model predictions of specific synoptic patterns
  relative to the model analyses
• Percent of cases that map to the correct node
• Mis-mapping of model predictions between nodes

11
AMPS 24h Forecasts
12
AMPS 48h Forecasts
13
Model Errors for Synoptic Patterns

• Compare model predictions to in-situ atmospheric
  measurements
• Calculate model validation statistics for all
  time periods that map to each node
• Look for model errors that vary from node to node
• This is ongoing work using AMPS data
• This technique has been applied to ARCMIP data

14
An ARCMIP exampleSHEBA Surface Pressure (DJF)
15
Conclusions / Future Work

• The use of SOMs provides an alternate method of
  evaluating model performance
• Identify synoptic patterns which are over or
  underpredicted
• Determine model tendency for mis-prediction of
  certain synoptic types
• Provide information on model errors related to
  specific synoptic patterns
• Complete SOM analysis for entire AMPS archive
  (Jan 2001 - present)
• Calculate model biases as a function of forecast
  time and synoptic patterns