Global Earth Observation Integrated Data Environment GEOIDE

1
Global Earth ObservationIntegrated Data
Environment (GEO-IDE)
Presentation to
Presenter name and affiliation
2
The information revolution
Need a quote that conveys the significance of the
period of history we are in -- the Information
Revolution.  "Get with it or become fossilized
and irrelevant." maybe some reference to how
much the nation as a whole is investing in
integrating systems.  (Contrast with NOAA
...) Anyone have a gem to go here?
3
NOAA Top Ten Challenges1

• 10) Alphabet Soup
• 9) Stove Pipes
• 8) Integration
• 7) Architecture
• 6) Data Sharing

• 5) User Needs
• 4) Maximizing Benefits
• 3) Communication
• 2) Data Management
• 1) Execution

• Vice Admiral Lautenbacher in address to American
  Meteorological Society Annual Meeting, January
  30, 2006

4
Data Management - a top priority

• Improving data management is among the highest
  priority challenges facing NOAA integrated data
  management is at the heart of the GEOSS concept

5
Why is improved integration needed? Important
societal issues require data from many observing
systems
Discipline Specific View
Whole System View
Current systems are program specific, focused,
individually efficient. But incompatible, not
integrated, isolated from one another and from
wider environmental community
5
6
Example Problem Coastal Inundation Erosion
involves multiple systems
Temperature
Human Population Distribution Scenarios
Societal Infrastructure Coastal Scenarios
Permafrost
Bathymetry
Coastal Inundation Erosion Scenarios
Winds
Reanalysis Coupled Climate Models
Biochemical Impacts(Ecosystems)
Winds
Temperature
Sea Ice
Tides

• Ocean
• Volume
• Circulation
• Quantity

Temperature
Sea Level Change
RiverRun-off
Glaciers
Observations (75)
Models (25)
Land Mass Ocean Properties (20)
Human Impacts
Physical Impacts
Sea Level Change (12)
( in parenthesis represents data sets which must
be integrated)
6
7
Data Management a top Federal Priority

• Congressional mandate
• Public Law 102-567, section 106 directs NOAA to
  (among other things)
• Develop effective interfaces among the
  environmental information systems of NOAA and
  other departments and agencies
• Develop and use nationally-accepted formats and
  standards
• Integrate and interpret data from different
  sources

8
Todays Challenges

• Incompatible syntax (formats) and semantics
  (terminology) among science disciplines within
  NOAA. Thousands exist. Several examples
• Naming standards Surface Air Temperature
• Meteorology (WMO) named Temperature/dry bulb
  temperature
• Meteorology (air pollution) named Boundary layer
  temperature
• Oceanography named Air Temperature
• Location standards (latitude, longitude,
  elevation)
• Lat/Lon can be degrees/minutes/seconds or degrees
  to tenths and hundredths
• Latitude E/W, 0-180 positive and negative, or
  0-360 running east or west
• Z used to designate elevation in both atmosphere
  and ocean but positive is up in the atmosphere
  and down in the ocean
• Formats (50 formats used within NOAA
  translators and standards needed)
• GRIB, NetCDF, HDF and others used for gridded
  data
• BUFR, NetCDF, and many others used for
  observations
• Potential for no answer or the wrong answer to
  important societal questions due to separate NOAA
  data management systems

9
NOAAs GEO-IDE

• Scope NOAA-wide architecture development to
  integrate legacy systems and guide development of
  future NOAA environmental data management systems
• Vision NOAAs GEO-IDE is envisioned as a
  system of systems a framework that provides
  effective and efficient integration of NOAAs
  many quasi-independent systems
• Foundation built upon agreed standards,
  principles and guidelines
• Approach evolution of existing systems into a
  service-oriented architecture
• Result a single system of systems (user
  perspective) to access the data sets needed to
  address significant societal questions

10
Goals

• Through GEO-IDE NOAA will
• Identify and address integration gaps in data
  management systems
• Create interoperability across existing data
  management systems
• Develop and adopt data standards for formats and
  terminology
• Integrate measurements, data, and products
• and will achieve
• Cost avoidances in NOAA business through improved
  efficiency and reduced duplication
• Better integration of data and products across
  disciplines
• Reduced risks for US IEOS and GEOSS

11
GEO-IDE - an essential component ofenvironmental
information management for NOAA
Integrated observing, data processing and
information management systems Connected by
NOAAs Integrated Data Environment Contributes
to U.S. Global Earth Observation System (USGEO)
and International Global Earth Observing System
of Systems (GEOSS).
12
Why Now?

• Societal Benefits
• Improve weather forecasting
• Reduce Loss of Life and Property from
  Disasters
• Protect and Monitor our Ocean Resources
• Understand, Assess, Predict, Mitigate and Adapt
  to Climate Variability and Change
• Support Sustainable Agriculture and
  Combat Land Degradation
• Understand the Effect of Environmental Factors on
  Human Health Well Being
• Develop the Capacity to Make Ecological
  Forecasts
• Protect and Monitor Water Resources
• Monitor and Manage Energy Resources

• Critical to USGEO
• six near term opportunities
• Uncoordinated development leads to
  inefficiencies, incompatibilities, and
  duplication of effort.
• Integration of data among systems is needed to
  answer questions that address diverse societal
  benefits
• Increased efficiency is needed to handle the
  expected exponential increase in data volumes
  that will occur over the next decade

12
13
Scope

• Concerned with environmental and geospatial data
  and information obtained or generated from
  worldwide sources to support NOAA's mission (as
  defined in NOAA Administrative Order 212-15)
• Does not consider administrative support systems
  such as finance, personnel, acquisition or
  facilities management
• Includes all aspects of data management,
  including data acquisition, ingest, data
  processing, archival and access

14
Vision

• System of systems a framework to effectively
  and efficiently integrate NOAAs many systems
• Minimize impact on legacy systems
• Utilize standards
• Work towards a service-oriented architecture

15
Approach

• Each NOAA LO/program/project continues to manage
  its data independently
• Standards
• Adopt, adapt and only as a last resort, create
• Open, inclusive process for adoption
• Inclusive not exclusive use of standards
• Service Oriented Architecture

Reference Federal CIO Council, Jan 06
"Services and Components Based Architectures A
Strategic Guide for Implementing Distributed and
Reusable Components and Services in the Federal
Government"
16
Standards

• Standard names and terminology
• Metadata standards
• e.g. FGDC and ISO 19115 w/ remote sensing
  extensions
• Standard formats for delivery of data/products
• WMO, NetCDF, HDF, GeoTIF, JPEG, etc.
• Web Services Standards
• World Wide Web Consortium
• OGC (Features, Coverage, GML)
• OPeNDAP

17
Service-Oriented Architecture

• Under an SOA, capabilities are built one at a
  time to create Web Services
• The fabric of the SOA is built upon standards
  for
• discovery (e.g. CF, FGDC, ISO, SQL)
• transport (e.g. HTTP, FTP, OPeNDAP)
• use (e.g. netCDF, HTML, etc.)
• Can be tightly coupled (SOAP) or loosely coupled
  (REST)

18
SOA Concept
19
Key Development Strategies

• Maintain and minimize impact on legacy systems
• Evolutionary development through pilot projects
• Coordinate activities through Communities of
  Interest organized by Data Types
• Grids, time-series, moving-sensor
  multi-dimensional, profiles, trajectories,
  geospatial framework, point data and metadata

20
Project Management
Undersecretary for Atmosphere and Oceans
NOAA Goal ThemeCommerce Transportation
NOAA Goal ThemeWeather Water
NOAA Goal ThemeEcosystems
NOAA Goal ThemeClimate
21
Project ManagementRoadmap to Implementation
22
Project ManagementPerformance Measures

• Performance Measure Outcomes
• Ability to address multi-disciplinary societal
  issues
• Cost avoidance of future investments in Data
  Management
• Metrics
• Number of interoperable Data Systems as measured
  by
• Data Sets within science disciplines
• Data Sets among science disciplines
• Number of users accessing cross-discipline data
  to address societal issues
• DOC Strategic Goal Supported
• Observe and Manage the Earth's Environment to
  Promote Sustainable Growth

23
Future Direction - Priorities

• FY07 Work with scientists/data system managers to
  assess requirements and systems
• Develop enterprise architecture and GEO IDE
  Implementation Plan
• Implement standards process
• Active out-reach activities
• FY08/09 Incrementally execute work packages
• Develop data standards and interoperability
  mechanisms, e.g., translators and directory
  services
• Direct, test and evaluate changes being made to
  data management systems
• FY10/11 Re-evaluate architecture related to new
  data systems (across NOAA with national
  /international partners)

24
Conclusions

• NOAA faces daunting challenges of vastly
  increasing data volumes and an increasing need
  for interdisciplinary use of data
• NOAA is committed to enhancing access and
  ensuring data and products of enduring value are
  preserved for future generations
• NOAA has initiated several activities to actively
  respond to these challenges. GEO-IDE is a key
  component

25
Backup Slides

• Risk Management
• Architectural compliance

26
Risk Management

• NOAAs Integrated Data Management System is not a
  traditional IT Investment Project
• Integration as opposed to new development
• No initial hardware investment and Modest
  software investment
• Large up front effort to do Enterprise
  Architecture
• Hardware/software from existing data management
  systems will be integrated
• This Enterprise Architecture Development IS a
  risk reduction effort

27
Risk Management Architecture
28
Architectural Compliance and Standards

• The project creates and executes the integrated
  management component of the NOAA Enterprise
  Architecture
• Consistency with NOAA Architecture Standards
• NOAA Planning, Programming, Budgeting, and
  Execution Process (PPBES)
• Federal Geographic Data Committee (FGDC) metadata
  standards,
• NOAA Observing System Architecture (NOSA)
• Geospatial One-Stop
• COTS Software and Commodity Hardware
• COTS applications software and previously
  developed government software
• Commodity hardware
• Information Quality Standard (Section 515). NOAA
  Guidelines to ensure and maximize Quality,
  Objectivity, Utility, Integrity (in support of
  Reproducibility)
• Accessibility Standard (Section 508) compliance
• Government Paperwork Elimination Act (GPEA)

29
Architectural ComplianceObserving System
Architecture Component -ERD
Mission Goal
Stake-holder/User
drives
drives
Programs
owned by
Owner
have
provides info to
supported by
Support
Capability Requirements
Observing System
Data Handling System
provides data to
Operator
is type of
operated by
contains
Environmental Parameter Requirement
Sensing Element
senses
Platform / Station
Drives the need to measure
situated on
located at
is a
Location
Environmental Parameter
Human
Sensor
is
Mobile
Fixed
30
Architectural ComplianceIntegrated Data
Management Component -ERD
produce
Data Handling System
Observing Systems Other Data Sources
Applications
contain
contain
Products
provide
delivers to
Peripherals
Data
delivered via
Network Services
travels over
yield
Services
contains
delivers to
Peripherals
Peripherals
delivered via
contains
delivers to
delivers to
Web Servers
Archive Systems
User
delivers to
delivers to
31
Architectural Compliance
OMB Business Model
Services to Citizens
Program Admin
Compliance
Support Delivery of Services
Internal Operations/Infrastructure
Inter-Agency
Intra-Agency