Global Earth Observation Integrated Data Environment GEOIDE

1
Global Earth ObservationIntegrated Data
Environment (GEO-IDE)
Presentation to DMAC-ST
David McGuirk Representing NOAA GEO-IDE
2
NOAA Top Ten Challenges1

• 10) Alphabet Soup
• 9) Stove Pipes
• 8) Integration
• 7) Architecture
• 6) Data Sharing

• 5) User Needs
• 4) Maximizing Benefits
• 3) Communication
• 2) Data Management
• 1) Execution

• Vice Admiral Lautenbacher in address to American
  Meteorological Society Annual Meeting, January
  30, 2006

3
Data Management - a top priority

• Improving data management is among the highest
  priority challenges facing NOAA integrated data
  management is at the heart of the GEOSS concept

4
Why is improved integration needed? Important
societal issues require data from many observing
systems
Discipline Specific View
Whole System View
Current systems are program specific, focused,
individually efficient. But incompatible, not
integrated, isolated from one another and from
wider environmental community
4
5
Todays Challenges

• Incompatible syntax (formats) and semantics
  (terminology) among science disciplines within
  NOAA. Thousands exist. Several examples
• Naming standards Surface Air Temperature
• Meteorology (WMO) named Temperature/dry bulb
  temperature
• Meteorology (air pollution) named Boundary layer
  temperature
• Oceanography named Air Temperature
• Location standards (latitude, longitude,
  elevation)
• Lat/Lon can be degrees/minutes/seconds or degrees
  to tenths and hundredths
• Latitude E/W, 0-180 positive and negative, or
  0-360 running east or west
• Z used to designate elevation in both atmosphere
  and ocean but positive is up in the atmosphere
  and down in the ocean
• Formats (gt50 formats used within NOAA
  translators and standards needed)
• GRIB, NetCDF, HDF and others used for gridded
  data
• BUFR, NetCDF, and many others used for
  observations
• Potential for no answer or the wrong answer to
  important societal questions due to separate NOAA
  data management systems

6
NOAAs GEO-IDE

• Scope NOAA-wide architecture development to
  integrate legacy systems and guide development of
  future NOAA environmental data management systems
• Vision NOAAs GEO-IDE is envisioned as a
  system of systems a framework that provides
  effective and efficient integration of NOAAs
  many quasi-independent systems
• Foundation built upon agreed standards,
  principles and guidelines
• Approach evolution of existing systems into a
  service-oriented architecture
• Result a single system of systems (user
  perspective) to access the data sets needed to
  address significant societal questions

7
Goals

• Through GEO-IDE NOAA will
• Identify and address integration gaps in data
  management systems
• Create interoperability across existing data
  management systems
• Develop and adopt data standards for formats and
  terminology
• Integrate measurements, data, and products
• and will achieve
• Cost avoidances in NOAA business through improved
  efficiency and reduced duplication
• Better integration of data and products across
  disciplines
• Reduced risks for US IEOS and GEOSS

8
GEO-IDE - an essential component ofenvironmental
information management for NOAA
Integrated observing, data processing and
information management systems Connected by
NOAAs Integrated Data Environment Contributes
to U.S. Global Earth Observation System (USGEO)
and International Global Earth Observing System
of Systems (GEOSS).
9
Why Now?

• Societal Benefits
• Improve weather forecasting
• Reduce Loss of Life and Property from
  Disasters
• Protect and Monitor our Ocean Resources
• Understand, Assess, Predict, Mitigate and Adapt
  to Climate Variability and Change
• Support Sustainable Agriculture and
  Combat Land Degradation
• Understand the Effect of Environmental Factors on
  Human Health Well Being
• Develop the Capacity to Make Ecological
  Forecasts
• Protect and Monitor Water Resources
• Monitor and Manage Energy Resources

• Critical to USGEO
• six near term opportunities
• Uncoordinated development leads to
  inefficiencies, incompatibilities, and
  duplication of effort.
• Integration of data among systems is needed to
  answer questions that address diverse societal
  benefits
• Increased efficiency is needed to handle the
  expected exponential increase in data volumes
  that will occur over the next decade

9
10
Scope

• Concerned with environmental and geospatial data
  and information obtained or generated from
  worldwide sources to support NOAA's mission (as
  defined in NOAA Administrative Order 212-15)
• Does not consider administrative support systems
  such as finance, personnel, acquisition or
  facilities management
• Includes all aspects of data management,
  including data acquisition, ingest, data
  processing, archival and access

11
Vision

• System of systems a framework to effectively
  and efficiently integrate NOAAs many systems
• Minimize impact on legacy systems
• Utilize standards
• Work towards a service-oriented architecture

12
Approach

• Each NOAA LO/program/project continues to manage
  its data independently
• Standards
• Adopt, adapt and only as a last resort, create
• Open, inclusive process for adoption
• Inclusive not exclusive use of standards
• Service Oriented Architecture

Reference Federal CIO Council, Jan 06
"Services and Components Based Architectures A
Strategic Guide for Implementing Distributed and
Reusable Components and Services in the Federal
Government"
13
Standards

• Standard names and terminology
• Metadata standards
• e.g. FGDC and ISO 19115 w/ remote sensing
  extensions
• Standard formats for delivery of data/products
• WMO, NetCDF, HDF, GeoTIF, JPEG, etc.
• Web Services Standards
• World Wide Web Consortium
• OGC (Features, Coverage, GML)
• OPeNDAP

14
Service-Oriented Architecture

• Under an SOA, capabilities are built one at a
  time to create Web Services
• The fabric of the SOA is built upon standards
  for
• discovery (e.g. CF, FGDC, ISO, SQL)
• transport (e.g. HTTP, FTP, OPeNDAP)
• use (e.g. netCDF, HTML, etc.)
• Can be tightly coupled (SOAP) or loosely coupled
  (REST)

15
Key Development Strategies

• Maintain and minimize impact on legacy systems
• Evolutionary development through pilot projects
• Coordinate activities through Communities of
  Interest organized by Data Types
• Grids, time-series, moving-sensor
  multi-dimensional, profiles, trajectories,
  geospatial framework, point data and metadata

16
Project Management
Undersecretary for Atmosphere and Oceans
NOAA Goal ThemeCommerce Transportation
NOAA Goal ThemeWeather Water
NOAA Goal ThemeEcosystems
NOAA Goal ThemeClimate
17
Future Direction - Priorities

• FY07 Work with scientists/data system managers to
  assess requirements and systems
• Develop enterprise architecture and GEO IDE
  Implementation Plan
• Implement standards process
• Active out-reach activities
• FY08/09 Incrementally execute work packages
• Develop data standards and interoperability
  mechanisms, e.g., translators and directory
  services
• Direct, test and evaluate changes being made to
  data management systems
• FY10/11 Re-evaluate architecture related to new
  data systems (across NOAA with national
  /international partners)

18
Conclusions

• NOAA faces daunting challenges of vastly
  increasing data volumes and an increasing need
  for interdisciplinary use of data
• NOAA is committed to enhancing access and
  ensuring data and products of enduring value are
  preserved for future generations
• NOAA has initiated several activities to actively
  respond to these challenges. GEO-IDE is a key
  component