SPASE Status

1
SPASE Status

• J. R. Thieman
• SPASE RAL Meeting
• Coseners House
• July 9, 2007

2
What is SPASE ?

• Space Physics Archive Search and Extract (SPASE)
  is
• An international, community-based organization
  with the goals of
• Facilitating data search and retrieval across the
  Space and Solar Physics data environment
• Defining and maintaining a standard Data Model
  for Space and Solar Physics interoperability
• Demonstrating the Models viability
• Providing tools and services to assist SPASE
  users
• Providing a group for other heliophysics data
  management and services coordination as needed
• SPASE is the only international group supporting
  global data management for Solar and Space Physics

3
History of SPASE

• SPASE had its start at an International Solar
  Terrestrial Physics (ISTP) meeting at RAL in 1998
• A group of data centers agreed to work together
  to enable searching for and retrieving data
  across the solar terrestrial data archives
• The group gradually expanded and discussions
  became more frequent
• In October 2003 an international consortium,
  SPASE, was formally created by these and other
  data centers
• In 2004 a three-year grant was funded by the
  Living With a Star (LWS) program to mainly create
  the SPASE Data Model
• In 2007 NSSDC was asked by NASA Hq. to provide
  permanent support for SPASE within the NSSDC
  funding at the same time that NASA Virtual
  Observatories were being organized

4
International Effort
5
SPASE Community
SPASE Core Group - Historic Partners -
Virtual Observatory Representatives - Special
Interest Representatives Heliophysics Data
Centers and Archives Heliophysics Service
Providers Users
6
Heliophysics Data Search and Retrieval
7
The SPASE Data Model

• Version 1.0.0 released in November, 2005
• Version 1.1.0 released in August, 2006
• Version 1.2.0 released in May, 2007
• This version is frozen for use by the community,
  but
• development is continuing with small changes (to
  be indicated by Versions 1.2.x)
• SPASE Resource Types Catalog
• Display Data
• Numerical Data
• Granule
• Instrument
• Observatory
• Person
• Registry
• Repository
• Service

8
SPASE Data Model Access
Data Model Version 1.2
http//www.spase-group.org
9
SPASE in the Virtual Space Physics Observatory
Product Finder
http//vspo.gsfc.nasa.gov/websearch/dispatcher
10
SPASE Issues
SPASE and the SPASE Data Model are intended to be
an interlingua among virtual observatories and
data archives. Relative to this goal, there are
the following issues that can be discussed
concerning SPASE usage in a Virtual Observatories
environment. How effective is SPASE in
describing data sets for data finding and
usage? How much should SPASE be "inside" vs.
"outside" the observatories, etc. to be
effective? To what level of detail should data
descriptions be created in order to make SPASE
effective as an interlingua? What else is
needed in SPASE for the non-NASA virtual
observatories environment?
11
Future Plans

• SPASE Data Model
• Use the SPASE V1.2 data description to describe
  as many data sets as possible
• Test the functionality and incorporate necessary
  modifications into V1.3
• Improve the description at the physical parameter
  level to a level adequate to support API
• Overall
• Develop or facilitate tool development to ease
  the application of the Data Model
• Work with Virtual Observatories to ensure utility
  for their purposes
• Promote the use of SPASE throughout the community

12
BACKUP
13
Example Hierarchy
14
Data Model Dictionary
15
Sample XML Data Description
16
Application Tools
Generator Create SPASE descriptions using
external sources of information. Ruleset
Description Generator Simple scripting language
and templates. Validator Determines compliance
with SPASE data model. XML Validate Test a web
accessible SPASE description against a selected
version of the data dictionary. Parser
Convert SPASE descriptions to internal
representations. Parser Parse SPASE XML Wrapper
Convert or embed SPASE metadata in other
descriptions or form. Data Dictionary Lookup
Output SPASE data dictionary in XML.
SPASE-to-OAI mapping Map SPASE metadata to the
Open Archive Initiative (OAI) metadata using
stylesheets. All tools available from
http//www.spase-group.org/tools/
17
SPASE Activities

• Assembled a team of domain experts (scientists),
  information specialists and technologists (others
  are welcome to join)
• Established a forum for discussion of a Space
  Physics Data Model and Data Dictionary
• Fully discuss (e-mail biweekly teleconferences)
  each change to the model
• Meet face-to-face regularly (at least annually)
  for concentrated discussion, formal consensus,
  release of new versions, and planning for future
  versions of the data model.
• Define and support useful representational forms
  of the data model (XML)
• Test data model and its representation (XML) with
  user scenarios and real world resources
• Refine the data model in response to community
  needs
• Encourage and support adoption of data model by
  providing tools and reference implementation
• Advocate ever wider adoption of SPASE within the
  heliophysics community

18
EXAMPLE GENERATOR/VALIDATOR
SPASE Assistant - SwRI
19
EXAMPLE EDITOR
Created at Augsburg College
20
Prime Users

• The following data centers plan to provide SPASE
  descriptions of their holdings
• All Virtual Observatories supported by NASA
• VHO (Heliospheric)
• ViRBO (Radiation Belt)
• VITMO (Ionosphere, Thermosphere and Mesosphere)
• VMO/G (Magnetospheres)
• VMO/U (Magnetospheres)
• VSPO (Space Physics)
• VSO (Solar)
• ESA Cluster Data Archive
• Centre de Données de la Physique des Plasmas
  (CDPP)

21
Outline

• What is SPASE ?
• SPASE Goals
• SPASE Activities
• The SPASE Data Model
• Application Tools
• Prime Users
• Future Plans

22
SPASE Objectives

• To conceptualize the domain of space physics data
  and resources.
• Provide a formal dictionary (set of
  representational terms) to describe space physics
  resources.
• Provide a standard method of describing
  resources.
• Facilitate
• - One-stop searching for data in multiple
  data repositories
• - Intercomparison of similar quantities from
  different data sets in different data centers
  through common terminology mapping
  (e.g. visible radiation vs. optical radiation)
• - Identification of disparate data granules
  overlapping in time.
• - Extraction of data granules together with
  all the information necessary for their immediate
  analysis.

23
Submitted Abstract

• SM11C-02?
• The Application of SPASE to Heliophysics Data
  Description?
• Thieman, J R?
• james.r.thieman_at_nasa.gov?
• NASA/GSFC, Code 690.1 NASA/GSFC, Greenbelt, MD
  20771 United States ?
• King, T A?
• thieman_at_mail630.gsfc.nasa.gov?
• UCLA, Institute of Geophysics and Planetary
  Physics University of California 5881 Slichter
  Hall, Los Angeles, CA 90095-1567 United States ?
• Roberts, D A?
• thieman_at_mail630.gsfc.nasa.gov?
• NASA/GSFC, Code 612.2 NASA/GSFC, Greenbelt, MD
  20771 United States ?
• The Space Physics Archive Search and Extract
  (SPASE) Project has developed a Data Model for
  the description of heliophysics data sets. The
  Data Model is intended as a standard means for
  describing data so that the descriptions could be
  used for interoperable data search and retrieval
  across the varied archives and data centers that
  hold heliophysics data throughout the world. The
  success of this project depends on the wide usage
  of the Data Model in the community. In this
  presentation we will talk about the development
  of the Model through dedicated committee work and
  input from major international data centers and
  archives. We will show where to get the Model,
  how to apply it, and examples of its use. Version
  1.1 is available presently and has been "frozen"
  for usage in a stable environment. (See
  http//www.spase-group.org) The Model will
  ultimately evolve as the needs of the community
  dictate. We invite feedback and participation in
  the evolution from interested members of the
  community, especially those associated with the
  heliophysics Virtual Observatory effort. ?
• http//www.spase-group.org ?

24
The Space Physics Data Environment

• Thousands of relatively small datasets as well
  as several large datasets
• Tens to hundreds of data centers or data
  providers (repositories), scattered worldwide
• Very loose (if any) coupling of data centers
• Very diverse metadata terminology and data
  formats
• Data which is not archived will effectively be
  lost.
• Preservation of the data often becomes the task
  of a national data centre (NSSDC, CDPP)

25
Recent Space and Solar Physics Missions
26
Example Tools Instrument Description

lt?xml version"1.0" encoding"UTF-8"?gt ltSpase
xmlnsx0"http//www.spase-group.org/data/schema/s
pase-1_1_0.xsd"gt ltVersiongt1.1.0lt/Versiongt ltInstrum
entgt ltResourceIDgtCDPPIRWINDTNRlt/ResourceIDgt
ltResourceHeadergt ltResourceNamegtWind
Waves Thermal Noise Receiverlt/ResourceNamegt
ltReleaseDategt2006-09-01T000000lt/ReleaseDategt
ltDescriptiongtThe Thermal Noise Receiver
(TNR) on the WIND spacecraft is an instrument
of a new generation, which is flown for
the first time. It utilizes a digital spectrum
analysis technique similar to a
wavelet transform analysis. This permits a high
time resolution, associated to a high
frequency resolution. It was essentially designed
to measure the thermal noise at a
high rate. The interest of this receiver also
covers other events, both in situ (plasma
waves, ion acoustic waves) and remotely
detected (solar and magnetospheric radio bursts).
However, this receiver uses a
compression technique which limits the study of
high intensity events. Namely, the
digital dynamic range is about 45 dB, added to an
AGC range of more than 60 dB, the AGC
acting as a sensitivity adjustment. (e.g. an 80
dB event will not saturate the receiver, but the
computed spectrum will only cover its
upper 45 dB.) lt/Descriptiongt
ltAcknowledgement/gt ltContactgt
ltPersonIDgtCDPPPRIssautierIsabellelt/PersonIDgt
ltRolegtCoInvestigatorlt/Rolegt
lt/Contactgt lt/ResourceHeadergt
ltInstrumentTypegtSpectralPowerReceiverlt/InstrumentT
ypegt ltInvestigationNamegtThermal Noise
Receiverlt/InvestigationNamegt
ltObservatoryIDgtspase//observatory/examplelt/Observ
atoryIDgt lt/Instrumentgt lt/Spasegt
27
System Model for Data Access

• Information flows from repositories to
  applications through access points and gateways
  in the form of SPASE descriptions as indicated by
  the S
• Registries contain inventories of resources
  available at one or more repositories.
• Virtual Observatories may be Gateways or Access
  Points.
• Access points may map internal metadata into
  SPASE descriptions.

28
European Considerations

• There are currently three operational European
  data activities (in order of conception)
• CDPP (Centre de Données de la Physique des
  Plasmas), a national centre for archiving space
  plasma data which opened in October 1999
• EGSO (European Grid of Solar Observations), a
  virtual observatory led by UCL-MSSL and
  established under the Information Systems
  Technology Priority of the EC FP5 (which has now
  ended).
• CAA (Cluster Active Archive), opened in February
  2006 and funded by ESA to make publicly available
  data of the highest possible quality from the
  four-spacecraft Cluster mission.
• How do these three entities fit into the scheme
  presented previously ?
• EGSO is a virtual observatory, and should
  participate as such.
• CDPP and CAA are magnetospheric data centres,
  which are respectively multi-mission and
  project-oriented. How do they fit in ?
• Is some European coordination desirable, or will
  European data integration be managed from NSSDC ?

29
Virtual Observatories and Space Weather

• The requirements for Virtual Observatory are
  primarily driven by the research community, whose
  priorities are
• location of data, generally long after the event
• being able to exploit it without undue software
  development
• The requirements for an operational space weather
  service are different. The primary requirement
  is rapid access to the required data, which
  implies that
• its location is known, and
• processing software already exists.
• Despite this difference, Virtual Observatory
  services will be increasingly used for the
  research activities associated with Space
  Weather.
• Use of standard formats and data descriptions
  would probably accelerate (TBC) the integration
  of new data sources or services into an
  operational service.

30
SPASE is not free !

• Usually, individual data centers and repositories
  have their own internal data model and
  dictionary, to exercise control over their own
  system. For example, dates of upgrades depend
  upon
• operating system upgrades,
• major updates to support (e.g., database or
  graphical) software, and
• requirements to support launch and commissioning
  campaigns, special scientific workshops or
  similar projects.
• Therefore to be SPASE-compatible a data
  repository must convert as many as possible of
  its internal data descriptions to the current
  version of the SPASE model and dictionary. This
  is
• Relatively easy if the data model resembles the
  SPASE model (essentially translation of some
  keywords), but
• Much more difficult if the data model is
  different in addition to keywords, at some
  level the metadata syntax must be understood.
• The conversion need not be perfect, but
  information not converted cannot be used by
  SPASE.
• Should Virtual Observatories which are deemed
  sufficiently SPASE-compliant use the logo
  SPASE inside ?