NCDC: On the Forefront of E-Government

1
DATA ARCHIVES AND ACCESS
NCDC
Data Archiving and Access at the National
Climatic Data Center David R. Easterling
Principal Scientist National
Climatic Data Center
Asheville, NC

National Climatic Data Center
2
DATA ARCHIVES AND ACCESS
NCDC
BASE / MANDATED FUNCTIONS OF NCDCS MISSION

• Archive
• Data Sources
• Description of the Archives
• Data Sets Held
• Data Migration Goals to Improve Access

• Access
• Off-line
• On-line
• Customers
• Applications

National Climatic Data Center
3
NCDC
DATA ARCHIVES

• Archive
• Data Sources
• Description of the Archives
• Data Sets Held
• Data Migration Goals to Improve Access

4
NCDC
OPERATIONAL DATA RECEIVED FROM U.S. SOURCES
Climate Reference Network l NWS Coop Observers
l Hourly Airways l NCEP Charts l Ship
Reports l Rocketsonde l Radiosonde l Storm
Data l NEXRAD l Satellite l Aircraft l
Profiler l ASOS l
5
NCDC
OPERATIONAL DATA RECEIVED FROM INTERNATIONAL
SOURCES
Daily max/min temp precip l Surface Synoptic
3/6 Hrly l Hourly Airways l Ship Reports l
Rocketsonde l Radiosonde l Aircraft l
6
NCDC
DATA HELD IN MANY FORMS
l Original manuscripts l Diskettes l 35mm
microfilm l 16mm microfilm l Publications l
Autographic charts rolls
l Magnetic tape 8mm, 3480, 3590E LTO l
Optical disks l Paper copies l Mass Store
Robotics System
7
NCDC
HARDCOPY DATA ARCHIVE
Manuscript / Autograph 200 Million Pages -
37,878 Miles (1.5 times around the earth)
35mm 16mm Film 125,129 Rolls 2,340
Miles (Washington D.C. to Los Angeles)
10 NOW IN MASS STORE SYSTEM NEAR ON-LINE
Microfiche 1.2 Million Pages 114
Miles (Washington D.C. to Philadelphia)
8
NCDC
CLIMATOLOGICAL PUBLICATIONS
Available electronically on-line and furnished in
paper form on demand
9
NCDC
DIGITAL DATA ARCHIVE
10
NCDC
NCDC MAJOR BASIC IN-SITU DATA SETS

• In-situ Data
• Monthly means/extremes of temp. total precip
  U.S. 1831-current
  5GB
• Daily max/min mean temperatures, total
  precipitation U.S. 1881-current 20GB
• Hourly data U.S.
  
  1936-current 45GB
• Hourly 3/6 hrly surface synoptic Global
  1900-current
  500GB
• Upper air synoptic - U.S.
  
  1912-current 40GB
• Upper air synoptic Global (CARDS)
  1948-current
  5GB
• Marine surface observations (ships/buoys)
  Global 1800s-current
  14GB
• Hourly Precipitation Data U.S.
  
  1900-current 2GB
• 15 Minute Precipitation Data U.S.
  1970-current
  2GB
• International Experiments (U.S. sponsorship or
  involvement)
• Aircraft observations
  
  1973-current 18GB

11
NCDC
USA PRECIPITATION NETWORK
Number of Reporting Stations
Data prior to 1941 digitized under CDMP
12
NCDC
USA DAILY DATA
Number of Reporting Stations
Data from stations added by CDMP
13
NCDC
USA UPPER AIR NETWORK
Number of Reporting Stations
14
NCDC
NCDC REMOTELY SENSED DATA SETS

• Satellite Data
• Polar Orbiting Environmental Satellites (POES)
  1978-current
  45,000GB
• Geostationary Operational Environmental
  Satellites (GOES) 1978-current
  300,000GB
• Radar Data
• Radar (NEXRAD) U.S.
  Majority 1995-current
  360,000GB
  
• Profiler
• Wind Profiler
  
  1991-current 54GB

15
NCDC
NCDC DIGITAL ARCHIVE GOALS

• Improve Access to Data by Migrating all Data to
  Mass Store Robotics System
• Migrate data from IBM 3480 tapes, 8MM tapes, WORM
  discs to Mass Store
  Robotics
• NEXRAD by Jan. 2003
• In-situ by Jan 2003
• POES by Jan. 2004
• GOES by Jul. 2005
• Initiate placing current GOES data on Mass
  Store Robotics by Jan. 2002
• Implement GOES Active Archive - FY 2003

16
NCDC
DIGITAL DATA ARCHIVE -NEXRAD
NEXRAD LEVEL II Migration from 8MM Tapes to HDSS
(Near On-line)

• 25,000 tapes migrated to date
• 76,599 8MM tapes to migrate will be migrated to
  700 IBM 3590E tapes in HDSS (approximately 300TB
  compressed to 25TB)
• Estimated completion Jan 2003

NEXRAD LEVEL III Migration from WORM disc to HDSS
(Near On-line)

• 15,000 discs to migrate
• Estimated completion Jan 2003

17
NCDC
GLOBAL BASELINE DATA SETS
NCDC Global Historical Climatology Network
(GHCN) U.S. Historical Climatology Network
(USHCN) Comprehensive Ocean-Atmosphere Data
Set (COADS) Comprehensive Aerological
Reference Data Set (CARDS) Blended Space
Based In-Situ Data 1971-2000 Climate Normals
(U.S.) 1961-1990 Climate Normals
(Global) World Weather Records Global
Temperature Index Various Maximum / Minimum
Temperature Series Snow Cover Wind
Climatologies Drought Related Data NCDC
Distributed Trace Gas Data Sets Model
Output 100-Year Doubling of CO2, etc
Microwave Sounding Unit Tropospheric Temperature
Former USSR Climate Data IPCC Related
Data Sets
18
Is the Earths surface getting warmer?Data from
the GHCN indicate warming temperatures in many
regions of the world, particularly in the past 25
years.
NCDC
INTERNATIONAL USA SURFACE WEATHER NETWORK
Global Historical Climatology Network (GHCN)
19
NCDC
CLIMATE DATA ACCESS
20
NCDC
CLIMATE DATA ACCESS
NCDC On the Forefront of E-Government
Offline access Continues to exhibit a sharp
decline
Online web access Continues to exhibit a rapid
increase
21
NCDC
CLIMATE DATA ACCESS
22
NCDC
DATA ACCESS
NCDC User Statistics Offline and Online
23
The NOAA Operational Model Archive and
Distribution System (NOMADS)

• NOMADS provides for distributed model data access
  over the Internet and planning for NGI access.
• Lack of access to climate and weather models
  initiated the highly collaborative NOMADS
  project.
• Near format independent access using the eXtended
  Markup Language (XML). Core s/w DODS/LAS/GDS.
• Many institutions both inside and outside of NOAA
  have understood the importance of NOMADS and have
  collaborated to guarantee its success.
• Currently there are 5 NOAA Lab/Data Centers and 9
  external collaborators (DOE, NCAR, Unidata,
  et.al.) under the NOMADS framework.

24
NOMADS Current Status

• NOMADS servers are operational at the following
  sites CDC, COLA, GFDL, NCAR, NCDC, PMEL,
  UCAR/Unidata, Lawrence Livermore Natl Lab,
  et.al.
• NOMADS server software has the following
  functionality Browse, Download, Graphics, Data
  Format Conversion, and Diagnostic Analysis. Other
  data types can be served.
• NOMADS has proven the concept of distributed
  serving model climate and weather data over the
  internet.
• A NOMADS site nomads.gfdl.noaa.gov/ck/nomads.htm
  l
• An FY04 NOAA Budget Initiative is being initiated
  across NESDIS, NWS, and OAR Line Offices.

25
NCDC
CUSTOMERS AND CLIMATE DATA APPLICATIONS
26
NCDC
CUSTOMERS BY INDUSTRY GROUP
CONSULTING METEOROLOGIST Provide expert
testimony in court LEGAL Certified evidence
in court cases INSURANCE INDUSTRY Rate
determination claims settlement
INDIVIDUAL Recreation, retirement,
entertainment, vacations ENGINEERING
Provide guidance in the design and
construction of facilities / infrastructure BUSI
NESS Sales impact, site relocation,
environmental impacts NOAA / GOVERNMENT
Research, investigations, legal
claims, risk management, policy guidance
/ support RESEARCH Environmental issues,
science studies, weather,
prediction
27
NCDC
CLIMATE DATA APPLICATIONS Climate Atlas

• Climate Atlas of the Contiguous United States
  CD-ROM
• CD-ROM Description
• Updates paper copy atlas published in 1968
• Contains 737 maps (monthly annual)
• Developed using latest GIS technology
• Includes temperature, precipitation, snowfall,
• HDD CDD
• Alaska and Hawaii maps in early 2002
• PDF maps available on-line

28
NCDC
CLIMATE DATA APPLICATIONS Climate Normals

• 1971-2000 U.S Climate Normals
• Description
• Official Decadal Product
• Nearly 8,000 Stations in the U.S.,
  Puerto Rico, Virgin Is., Pacific Is.
• Data
• Maximum Temperature
• Minimum Temperature
• Mean Temperature
• Precipitation
• Heating Degree Days
• Cooling Degree Days

1971-2000 Normals Station Locations in the
United States
29
NCDC
CLIMATE DATA APPLICATIONS Extreme Weather
Climate
Extreme Weather and Climate Events Web Page

• Description
• One of NCDCs most popular web pages
• More than 100K accesses per month
• Central NOAA web page for information/links
  on hurricanes, tornadoes, storm events, drought,
  extreme temperatures, heavy precipitation, etc.
• Billion dollar weather disasters
• Applications
• Natural hazards mitigation
• Insurance claims
• Agriculture
• Many others

http//lwf.ncdc.noaa.gov/oa/climate/severeweather/
extremes.html Hurricane Mitch
30
NCDC
CLIMATE DATA APPLICATIONS - FEMA
Return period statistics are useful for planning
purposesand also help in determining federal
snow disaster areas