Architecture

1

Architecture Data Management (ADM) Working
Group February 22, 2007 Status Update
2
OUTLINE

• Architecture Data Management Goals
• Activities Status
• Broad Challenges and Issues
• Service Oriented Architecture
• An important first step for IEOS Data Management

3
Architecture Data ManagementGoals

• ADM Goals Support to all NTOs IEOS data
  management systems
• Enable increased interoperability across existing
  data management systems
• Identify address integration gaps in data
  management systems
• Utilize community standards for data metadata
• Enable integrated measurements, data, products
  predictive models
• Examine the need for future data management
  requirements

Integration of Earth Observation Systems
4
Architecture Data ManagementUpdate on
Activities

• FY05
• Cross-cutting Survey of Data Management Needs for
  5 NTOs
• 7 ADM objectives identified portals,
  interoperability, terminology, metadata, others
• FY06
• 20 page ADM document and related 2-page summary
• Originally prepared as part of the NTO process
• Determination that ADM is not an NTO but more a
  long term opportunity
• Document summarized ADM requirements for other
  NTOs and USGEO
• Proposed milestone and schedule for ADM
  activities related to NTOs
• 10 year goal was to expand past 5 NTOs toward all
  IEOS systems
• ADM coordination with NTOs
• Assigned an ADM POC to each NTO.
• Workshop on NIDIS Portal Development in an
  Integrated Earth Observation System Architecture
  May 2006
• Frequent interactions among NOAA, NASA, EPA, ESIP
  exploring how to use existing portal technologies
  to support air quality NTO
• Coordinated with Disaster NTO Co-chairs on
  document portal data management content
• FY07 (current activities)
• Concentrated effort on Service Oriented
  Architecture (SOA)
• White paper Toward the achievement of IEOS
  interoperability
• Highlight SOA as a first step toward a system of
  systems data management infrastructure

5

• ADM Broad Challenges and Issues

6
Architecture Data ManagementBroad Challenges
and Issues
We currently have hundreds of separate, but
individually efficient, data management systems
within among agencies

• Agency Contributions
• Many key observing systems are in place
• Other systems are in place to acquire, process,
  archive, distribute data examples, AirNow,
  GEON, NOMADS, NEON, SciFlow
• Integration is needed in 2 ways
• Technical IT infrastructure for sharingdata
  products (data, metadata, information)
• Shared Web services via Service Oriented
  Architecture
• Knowledge required to properly use data products

Integrated data management is not possible
without the existing agency infrastructure
resources that support these separate pieces to
function
7
Architecture Data ManagementBroad Challenges
and Issues
1) Technical IT infrastructure for sharing data
products

• Service Oriented Architecture (SOA)
• Definition
• Generic A SOA is an underlying structure
  supporting communications between services, or
  units of work.
• IEOS A SOA is a style of system design based
  upon loosely coupled connections among
  independent programs to create a scalable,
  extensible, interoperable, reliable and secure
  framework.
• SOA is an enabler for an integrated data
  management system of systems
• ADM white paper on IEOS approach to SOA
  described in detail later

8
Architecture Data ManagementBroad Challenges
and Issues
2) Knowledge required to properly use data
products
Garbage in, garbage out is still the rule for
information systems. SOA is worthless if it
becomes just a more efficient way of
communicating poor quality data. Federal Times
January 2007

• Comprehensive Principles, Guidelines, Challenges
• Data and Information Quality
• Defined levels of data quality
• Conforms to Law Data Quality Act, OMBs
  Information Quality Guidelines
• Information Integrity/Security
• Information Assurance data have not been
  tampered with (malicious or non-malicious)
• Common Services for Data Interoperability
  Definitions
• Data Provenance
• Lineage of data describing historic details of
  change (how, when, why, who)
• Rich Metadata
• Descriptions of the potential difficulties
  advantages of using specific data

9

• Service Oriented Architecture
• Toward achieving IEOS interoperability

10
Service Oriented ArchitectureAn important first
step

• SOA provides an underlying structure that
  supports the opportunity for integration among
  the many IEOS systems
• SOA defines protocols, linkages, standards for
  shared web services that can be reused among
  existing agency systems to provide
• Data transport (including data formats)
• Discovery
• Delivery and user services
• Data manipulation
• Visualization

A single system to manage all data from all
global observing systems is not viable.
Integrating systems is doable for both technical
practical reasons
11
Service Oriented Architecture
Enabling Data Product Interoperability Goal of
SOA for IEOS A multi-faceted framework

• Interoperability through SOA is achieved via a
  selected set of standards and protocols
  functioning at the interface between system
  elements.
• Users engage a distributed set of loosely
  coupled data services through connections
  made just-in-time between data sources, services
  user clients.

12
Service Oriented ArchitectureChallenges
SOA Challenges for New Legacy Data Management
Systems

• New systems
• More easily built using SOA since starting
  from the ground up
• Legacy systems
• More constraints due to existing system
  capabilities
• Requires an additional layer to interface
  with existing components
• More daunting task

13
Service Oriented ArchitectureBenefits/Challenges

• SOA Benefits
• Facilitates interoperability
• Rapid assembly of new applications
• Shared web services can be reused by leveraging
  existing services
• More code reuse
• Developers can find a service in a directory and
  reuse it
• More efficient interdisciplinary use of data
• SOA Challenges
• Specialized technical skills required
• SOA is a cutting edge trend. Few currently have
  this expertise in house developers with skill in
  this area are increasingly in demand.
• Once a service is published, the interface must
  remain stable and reliably available
• Changes must be rigorously controlled and
  coordinated.
• Verification and Validation
• There are no accepted guidelines defining who is
  responsible for verifying/certifying that a
  service produces the proper result.

14
Architecture and Data ManagementNext Steps
Next Steps in FY07/FY08 for ADM Working Group

• Complete Draft white paper for further review by
  USGEO Apr 07
• Technical IT infrastructure for sharing data
  products (Highlighting SOA)
• Entitled Toward achieving IEOS interoperability
• Develop ADM Strategic Approach Document
  4thQ FY07
• Using the 20- and 2- page documents created in
  FY06 as the basis
• Develop white paper 4thQ
  FY07
• Knowledge required to properly use data products
• Accelerate achievement of IEOS through near-term
  deployment of services under
  the SOA approach, working with the NTOs
• Hold as needed workshops with other NTOs
  FY07/08

15

• Questions