Thomas Karl Director

1

NOAA Reanalysis User Needs AMS Town Hall Meeting

• Thomas Karl Director
• National Oceanic and Atmospheric Administration
• National Climatic Data Center
• AMS 88th Annual Meeting
• Town Hall Meeting
• January 21,2008

2
Outline

• Purpose of Meeting
• Overview of NOAA Reanalysis Efforts
• Climate Forecast System Reanalysis and Reforecast
  (CFSRR)
• Data Availability Plans
• Data Access Optimization
• User Input and Open Discussion

3
Open Forum and User Input
Tonights discussion is a focus on user needs

• Has the proposed file organization and groupings
  reflect current user needs and expectations?
• Which data need to be placed on-line given e.g.,
  150TB of disk?
• Are we on the right track? Do we need to re-group
  data sets to create needed products such as
  initialization files, analysis, or by forecast
  projections?
• Can some other mechanism (i.e. LDM) provide some
  CFSRR output to users in near real-time, shifting
  the load from the archive?
• NCDC and NCEP are committed to provide the
  highest resolution if possible. Tonights
  discussion is a focus on user needs tempered by
  available resources.

4
New NOAA Reanalysis Projects

• 1) Historical SFC Reanalysis (Compo et al.,)
• 1850 to present. 60TB
• 2) Post WW-II Reanalysis (NCEP)
• 1944 to present. 235TB
• 3) Climate Forecast System Reanalysis and
  Reforecast (CFSRR) Project (Saha et al.)
• 1978-2008 Reanalysis and Seasonal Reforecast
  915TB (Grib1)
• Development of a Reanalysis Clearinghouse
• Capability for an on-going analysis of the
  climate system

5
CFSRR Overview (Saha, EMC/NCEP)

• Coupled Climate Forecast System (CFS) Reanalysis
  and Reforecast (CFSRR) (Atm - Ocn - Land
  SeaIce) has two parts
• Reanalysis 31-year (1979-2009) T384L64 (32 km
  and 64 hybrid layers for the atmosphere, 0.50 and
  40 levels for the ocean, 4 soil levels)
• Reforecast 28-year (1982-2009) T126L64 (32 km
  and 64 hybrid layers for the atmosphere, 0.50 and
  40 levels for the ocean, 4 soil levels)
• 6 hourly Reforecast for 1 year
• NCDC and NCEP are seeking community input for
  archive and access priorities to this massive
  dataset
• Approximately 915 Terabytes in GRIB1.
• Note this presentation assumes conversion is
  needed to Grib1 as many user applications cannot
  yet deal with GRIB2 directly (including NCDC).

6
Proposed CFS Reanalysis Data

• CFS High Resolution Initial Conditions 72 TB
• CFS Low Resolution Initial Conditions 35 TB
• Full Data Ingest 46TB (satellite radiances,
  RAOB, in-situ, etc.)
• Pgbh Pressure Grib Files 0.5 x 0.5 Hourly 91TB
• Flxf T382 Gaussian Hourly 25TB
• Ocnh 0.5 x 0.5 Hourly 18TB
• Diabf 1.0 x 1.0 Hourly 27TB
• Ipvh 0.5 x 0.5 Hourly 17TB
• Monthly 1TB
• TOTAL 332 TB

7
Proposed CFS Reanalysis Hi-Resolution File Level
Organization

1. Siganl 3-D Hybrid Analysis
2. Sfcanl Surface Analysis
3. Ocnanl 3D Ocean Analysis
4. Pgbh 3-D Pressure level data
5. Flxf SFC fluxes, radiative fluxes, precip.
6. Ocnh 3-D Ocean data
7. Ipvh 3-D Isentropic level data
8. Diabf Diabatic Heating, Moistening rates, etc.
9. Egyh Energetics, u v, TKE, etc.

T382L64 Ocean (1/4 x ½) 6 hourly
72TB T126L64 Ocean (1/2 x 1) 6 hourly
35 TB
0.5 x 0.5 degree Hourly 91TB
T382 Gaussian Hrly 25TB
0.5 x 0.5 degree Hourly 18TB
0.5 x 0.5 degree Hourly 17TB
1.0 x 1.0 6-Hourly 27TB
0.5 x 0.5 Monthly 1 TB
8
CFS 28-Year Reforecast Data1982-2009

• 6-hourly Pressure Grib 412TB
• 6-hourly Ocean 102TB
• 6-hourly Ipv 78TB
• Monthly and Time-series 3TB
• 1.0 x 1.0 degree for first 6 months of forecast
• 2.5 x 2.5 for next 6 months of forecast
• TOTAL 595 TB

9
National Academies Recommendation

• National Research Council, Board of Atmospheric
  Sciences and Climate (BASC)
• Completing the Forecast Characterizing and
  Communicating Uncertainty for Better Decisions
  Using Weather
• and Climate Forecasts
• Recommendation 3.4
• The NOAA National Operational Model Archive and
• Distribution System (NOMADS) should be maintained
  and
• extended to include (a) long-term archives of
  global and regional
• ensemble forecasting systems and their native
  resolution, and
• (b) re-forecast datasets to facilitate
  post-processing

10
Open Discussion
Proposed Data and System Access Priorities
11
CFSRR System Access Prioritization 1

• Basic ftp and http scripting services to the
  highest resolution data though the NCDC tape
  archive services.
• NOMADS on-line sub-setting services for most
  requested data (tonights discussion).
• Advanced GIS and other Web Based Portal Services
  as resources and time permit
• Issue Data access may be throttled based on
  number of concurrent users, I/O restraints, and
  communications bandwidth at NCDC.

12
Reanalysis Data Access Prioritization 1

• Highest Resolution CFSRR data provided
• Low Res users can subset or use Post WW-II
• Access assisted by user feedback as to file level
  organization (this meeting!)
• Historical 20th Century SFC Reanalysis
• 60TB (Grib and HDF)
• FY10 Post WW-II U/A Reanalysis
• Low resolution 6-hourly (235TB Grib-1)
• When funded

13
FEEDBACK and QUESTIONS
OPEN DISCUSSION
David.Easterling_at_noaa.gov

• Suranjana.Saha_at_noaa.gov
• CFSRR PI
• Glenn.Rutledge_at_noaa.gov
• nomads.ncdc.noaa.gov

14
Backup Slides
15
Overview

• To overcome a deficiency in model data access,
  some of the Nations top scientists are actively
  engaged in a grass-roots framework to share data
  and research findings over the Internet.

• NCDC, NCEP and GFDL initiated the NOAA
  Operational Model Archive and Distribution
  System.
• NOMADS is a distributed data services pilot for
  format independent access to climate and weather
  models and data.

16
The NOAA Operational Model Archive and
Distribution System
NOMADS Goals

• Establish a unified climate and weather model
  archive providing format independent access to
  retrospective models

• promote model evaluation and community feedback

• foster research within the geo-science
  communities (ocean, weather, and climate) to
  study multiple earth systems using collections of
  distributed data

• develop institutional partnerships and access
  via distributed open standard technologies.

17
Multiple paths to format independent data access
The NOMADS System Design
18
NCDC HDSS Access System (HAS NCDC Archive)
Other incoming data
Data request from access serviceeg, NOMADS
Data delivery to access serviceeg, NEXRAD
NOMADS Model data
NEXRAD Radar
Major Access Services
CLASS Satellite
CDO, GIS Services In-situ
NOAA Customers
A very simplified diagram of data flow from
NCDCs HDSS tape robotics system incoming
sources