Hayabusa Data Archiving

1
Hayabusa Data Archiving

• Hayabusa Joint Science Team Meeting
• August 17-19, 2003
• ISAS, Tokyo

2
Introduction to PDS

• The Planetary Data System (PDS) is sponsored by
  NASA to insure the long-term usability of NASA
  and other data and to stimulate advanced
  research. PDS archives and distributes planetary
  data from missions and from groundbased
  telescopes, as well as laboratory data.

3
PDS Philosophy

• Maximize usability of data by the science
  community, including future generations
• Work with missions to achieve archiving goals
• Observe whatever proprietary or data validation
  periods are set by the sponsoring agency

4
(No Transcript)
5
Structure of PDS

• PDS is divided into seven discipline nodes each
  responsible for archiving data of a different
  planetary discipline or data type. The
  discipline nodes are located at universities or
  research centers and led by planetary scientists
  with expertise in the applicable field. Data
  archiving is coordinated through the Central Node.

6
(No Transcript)
7
Small Bodies Node

• The Small Bodies Node (SBN), led by Dr. Mike
  AHearn, is responsible for archiving data on
  comets, asteroids, and interplanetary dust. The
  Asteroid Subnode of SBN, led by Dr. Don Davis and
  located at the Planetary Science Institute in
  Tucson, is the lead node for the archiving of
  Hayabusa data for PDS.

8
Structure of the Small Bodies Node
9
PDS Web Sites

• Small Bodies Node pdssbn.astro.umd.edu
• Asteroid Subnode www.psi.edu/astsubnode.html
• Main PDS web site pds.jpl.nasa.gov

10
PDS Contacts

• Dr. Mike AHearn Head, Small Bodies Node.
  ma_at_astro.umd.edu
• Dr. Don Davis Head, Asteroid Subnode.
  drd_at_psi.edu
• Dr. Carol Neese Archiving Specialist, Asteroid
  Subnode. neese_at_psi.edu

11
Archive Planning Needs

• To plan the archiving of data from Hayabusa, PDS
  will need information and documentation from the
  mission and instrument teams, in advance of the
  data. See the Proposers Archive Guide at
• pds.jpl.nasa.gov/documents/documents.html
• for information and examples.

12
PDS Needs

• A list of expected data products from each
  instrument
• Estimated data volumes and file sizes
• Definition of data formats
• Data samples

• Documentation describing the mission, instruments
  and data
• Ancillary data such as filter profiles and
  performance parameters to support the primary data

13
Archive Process

• Archive plan
• Design archive - file formats, archive structure
  in consultation with PDS (SBN)
• Review of calibration data and sample flight data
  by PDS
• Submit Archive Interface Document Data to PDS
• Peer review by outside scientific peers -
  produces liens against archival data
• Project corrects liens
• Final archive with PDS (via SBN)

14
Archive Content

• Everything one needs to understand the data
• Data (raw and calibrated)
• Geometric information
• Calibration information
• Documents describing instruments, S/C, mission,
  etc.
• Catalog files to enable computer searching
• Simple, user-friendly formats
• Consistency across the whole mission with
  related missions

15
Required Archive Documents

• Archive Plan (AP).
• Archive Interface Documents (AID), also called
  Software Interface Specifications (SIS).
• Examples are given on the web at
  pds.jpl.nasa.gov/documents/documents.html

16
Archive Plan (AP)

• Outlines data pipeline in sufficient detail to
  determine scope of effort required at each stage
• Defines schedule for all tasks, including data
  deliveries
• Assigns responsibilities for all tasks
• Signed by all those with assigned
  responsibilities
• Overview of mission and instruments

17
AP Content

• Overview of mission and instruments for context
• Description of data pipeline from spacecraft to
  archive, assigning responsibility for each
  component
• Schedule for delivery at each stage
• Data volume and types at each stage

18
Archive Interface Documents

• Defines the detailed content of the archive
  organization, data products, file types and
  formats, data volumes, documentation.
• One or more separate documents that collectively
  describe the archive in sufficient detail that
  the end user knows what everything means.

19
AID Content

• Inventory of data products
• Detailed description of all file types
• Definitions of all keywords
• Outline of archive directory structure
• List of ancillary files and their structure
• Description of calibration procedures

20
Schedule for AP

• Preliminary draft at Preliminary Design Review
• Complete plan negotiated with PDS at Critical
  Design Review
• Final signed plan before Launch Readiness Review

21
Schedule for AIDs

• Draft at Launch Readiness Review
• Signed version within 3 months of launch checkout
  phase or one month prior to first delivery of
  data, whichever occurs first
• Updates if needed with every data delivery

22
PDS Format

• Archived data are required to be in PDS format,
  but what does this mean? PDS can accept data in
  many different formats, but formats must be
  consistent, well-defined, and described by a PDS
  data label. Data cannot be accepted in
  proprietary data formats.

23
PDS Labels

• One label for each file (data, doc, etc.)
• Detached labels (with rare exceptions)
• Label describes the format and content of the
  data file
• Labels are in Object Description Language (ODL),
  a machine-readable language
• Example label next slide ?

24
NEAR MSI Image Label
25
PDS Catalog Files

• In a PDS archive, the data are supported by
  metadata called catalog files. These are
  summaries which pop up when querying the archive
  to find out which data one wants to retrieve.
  The following catalog files are needed for the
  Hayabusa archive.

26
Catalog Files

• Mission Cat describes the mission
• Inst Host Cat describes the spacecraft
• Instrument Cats one for each instrument to
  describe the instruments
• Data Set Cats one for each data set to describe
  the data sets

27
Sample instrument catalog file NEAR Infrared
Spectrometer (one slide of seven for entire
file) PDS_VERSION_ID PDS3
RECORD_TYPE
STREAM


OBJECT INSTRUMENT

INSTRUMENT_HOST_ID "NEAR"
INSTRUMENT_ID
"NIS"

OBJECT
INSTRUMENT_INFORMATION
INSTRUMENT_NAME
"NEAR INFRARED SPECTROMETER"
INSTRUMENT_TYPE "SPECTROMETER"
INSTRUMENT_DESC
"

Instrument
Overview



Launched on February
17, 1996, the Near Earth Asteroid Rendezvous
(NEAR) spacecraft will go into orbit
around the asteroid 433 Eros on
February 14, 2000. The first launch of the
Discovery class missions, NEAR will
study this S-class asteroid up close, determining
its geological characteristics,
physical properties and composition
CHENGETAL1997. The reflected-light spectral
data gathered by the Near Infrared
Spectrometer (NIS) is a key component of an
integrated strategy for the study of
Eros. This strategy also includes high
resolution imaging from the multispectral
imager (MSI)
VEVERKAETAL1997A, elemental abundance
measurements from the
X-ray/gamma-ray spectrometer (XGRS)
TROMBKAETAL1997, magnetic field
data from the magnetometer (MAG) ACUNAETAL1997,
topographic data from the NEAR laser
rangefinder (NLR) ZUBERETAL1997, and mass
and gravity data from the on-board radio
science experiment
YEOMANSETAL1997. NIS will conduct spectral
mapping of Eros by measuring
reflected sunlight in the wavelength range from
800 to 2600 nm. Reflectance spectra
of Eros will be used to identify the
surface mineral assemblages, to constrain the
origin and evolution of the surface,
and, in combination with other NEAR data, to
explore the links between asteroids and
meteorites.


28
- page 2 NEAR has traveled a
circuitous path on its way to its orbital
encounter with Eros FARQUHARETAL1995,
including a swing through the inner
main asteroid belt that allowed a flyby of the
C-type asteroid 253 Mathilde, a
gravity-assist swingby of Earth, and an
unplanned initial flyby of Eros itself
following an aborted
orbit-insertion burn YEOMANSETAL1999,
VEVERKAETAL1999B. Due to inadequate
power margins at the relatively large
heliocentric distance during the
target-of-opportunity encounter with Mathilde on
June 27, 1997, NIS was not turned on at
that time VEVERKAETAL1997B. The NIS
instrument's protective cover was not opened
until later in 1997, allowing
observations of the on-board calibration target
(caltarget) and of the Earth and
Moon during the January 23, 1998
swingby maneuver. NIS first observed Eros in late
December 1998 VEVERKAETAL1999A.
Data from the Eros flyby provided a valuable
opportunity to evaluate instrument
performance. Observations of Eros will
resume on final approach in early 2000 and
continue throughout the mission as the
orbit is changed in steps from 500 to 35 km
radius.
NIS
was designed to fulfill four key NEAR measurement
requirements VEVERKAETAL1997A 1)
Map the mineralogical composition of Eros
using reflected sunlight 1) Map the
distribution and abundance of
minerals at scales as small as 300 meters 1)
Complement high resolution MSI
images and low resolution XGRS elemental
distribution maps for definitive
identification of rock types composing Eros'
surface 1) Provide information on the
physical and textural
properties of these surface materials


In order to fulfill these requirements,
the instrument's behavior must be
characterized both from rigorous preflight
calibration and testing and from
in-flight observations over the mission lifetime.
This paper reviews the data and
techniques used to calibrate the
NIS, and discusses initial in-flight results for
observations by NIS during the cruise,
Earth swingby, and Eros flyby phases of the NEAR
mission. The calibration of the
instrument will be refined
throughout the mission. Table two in the NIS
calibration paper contains the science
rationale for the phases and can be found in the
NIS/DOCUMENT/INSTRUMENT/CALPAPER
directory.
29
- page 3 Instrument Description


NIS is composed of
a grating infrared spectrometer, a scan mirror,
two passively cooled detector modules,
a mounting bracket,
spectrometer electronics, and a data processing
unit (DPU) that controls both the
NIS and the NEAR magnetometer. NIS has a one-time
deployable opaque cover, which was
opened on September 24, 1997, three
months after the Mathilde flyby
VEVERKAETAL1997B.

Hardware

--------
The NIS design
(Fig. 1) is copied from an earlier Applied
Physics Laboratory (APL)
spectrometer - the Defense Meteorological
Satellite Special Sensor Ultraviolet
Spectrographic Imager (SSUSI) - and
modified for an infrared wavelength range.
WARRENETAL1997 detail the design
characteristics, engineering, and construction of
the instrument. A gold-coated scan
mirror controls the direction of
viewing over a range of 140 degrees. Light
reflected from the scan mirror passes
through a 20 x 25-mm aperture stop and is imaged
by a telescope mirror at a slit. The
field of view is selectable at the
slit to 'narrow' (0.38 degrees x 0.76 degrees) or
'wide' (0.76 degrees x 0.76
degrees) settings. A shutter actuates the narrow
slit, with the smaller slit opening
coming down over the fixed wide slit.
The two slits provide field-of-view sizes of 0.65
x 1.3 km or 1.3 x 1.3 km from 100 km
distance. A second shutter can be actuated
to completely block the slit for dark current
measurements. After passing through
the slit, light is dispersed and re- imaged off a
gold toroidal diffraction grating (a
Rowland circle configuration
spectrometer) and hits a dichroic beamsplitter
mounted at 45 degrees to the beam which
transmits or reflects the energy to fall on two
32- element linear detector arrays.
Reflected 2nd order wavelengths
(804-1506 nm) fall on a germanium (Ge) array.
Each Ge channel has a bandwidth of
21.6 nm. The germanium detector has a selectable
gain of 1x or 10x. Transmitted 1st
order wavelengths (1348-2732 nm) go to
an indium-gallium-arsenide (InGaAs) array with
43.1 nm channel bandwidth.
WARRENETAL1997 calibrated the central
wavelengths of all NIS channels at
three operational temperatures (-7 degrees C,
-17 degrees C, and -23 degrees C) and
found best-fit Ge spectral
calibration given by

30
- page 4 lambda (nm) 794.6
21.61n (Equation 1)

where n is Ge
element number 1-32. The uncertainty of the
wavelength calibration over the range
of temperatures examined is /-0.5 nm.
InGaAs element center wavelengths are given by


lambda (nm) 43.11n - 50.8
(Equation 2)

where n is InGaAs element
number 33-64. The temperature-dependent
wavelength uncertainty is approximately /-3.5
nm.

The lower two channels of the InGaAs
detector (channels 33 and 34, at 1372
and 1315 nm respectively) are below the
transition wavelength of the
dichroic beamsplitter, and therefore always
register very low signal. The upper 3
channels (channels 62-64 at 2622,
2665, and 2708 nm) are near or in detector
cutoff, making the effective upper
bound for good signal-to-noise ratio (SNR) around
2500 nm for the signal level
expected at Eros. Additionally, two
InGaAs channels (47 and 57, at 1975 nm and 2406
nm) have been extremely noisy
since manufacture and do not produce easily
usable data. Default operations for
NIS utilize the narrow slit, providing
a critically sampled spectral resolution of 22 nm
in the Ge detector, and 44 nm
in the InGaAs detector. The wide slit
configuration provides half the spectral
resolution (44 nm and 88 nm
respectively), but passes twice the light and
therefore has a higher SNR.


The NIS
scan mirror can rotate the line of sight over 350
steps in 0.4-degree increments in the
spacecraft Z-X' plane. The Z axis is
perpendicular to the plane of NEAR's solar
panels, and X' is the boresight of
the instruments. Mirror position 0 (nominal
caltarget observation geometry) is 30
degrees towards the Z-axis from the
boresight. The boresight is aligned with mirror
position 75. Position 300
points in the -Z (anti-Sun) direction.

31
- page 5 For optimum performance
the detectors are operated near -35 degrees
C, maintained by passive cooling or active
heaters, depending on the thermal
environment. A solar-illuminated gold calibration
plaque (caltarget) is mounted to
the instrument for radiometric stability
calibration. Table I summarizes the NIS
specifications. More detailed
descriptions of the NIS instrument design are
presented in PEACOCK1997 and
WARRENETAL1997.

Flight
Software
---------------

In-flight NIS data are acquired through the
use of command sequences to the
instrument that specify ten instrument parameters
NISAPLSRS1995,
IZENBERGETAL1998 as follows 1. Spectrometer
sequence ID (0-15). Sixteen sequences
can be uploaded and stored while
the instrument is powered. Sequences 0, 1, and 2
are hard-coded, but can be
redefined. 2. Repeats the number of times
the commanded observations will repeat.
3. Seconds between repeats.
4. Number of observations. This is the number
taken during a single repeat of the
sequence. 5. Calibration interval
(1-65535). Number of observations before
acquisition of dark spectra.
This is used to interleave shutter-closed dark
observations with data observations.
6. Number of seconds to co-add spectral
data in each observation (0-63). 7. Number
of rest spectra (0-63). Used when
interleaved darks are taken, between spectra
acquisition and dark acquisition. 8.
Number of co-added seconds of dark signal
for interleaved dark spectrum. 9. Number of
scan mirror steps between
observations (sign indicates direction). 10.
Seconds between observations.

32
- page 6 For the purposes of the
software discussion, a 'spectrum' is the
result of a one-second integration of the
instrument as it gathers data. An
'observation' is 0-63 consecutive one-second
spectra summed together. The
example sequence ( 15 3 5 10 5 16 2 4 2
2 ) is interpreted as follows Sequence
ID is 15. The sequence will repeat
three times, each iteration to contain 10
observations consisting of 16
co-added spectra of accumulated data. The
calibration interval of five means
that every 5th observation
(numbers 5 and 10 in each repeat) will instead
co-add 10 spectra of the target, then
rest 2 seconds while the shutter closes, then
take four co-added spectra of dark
signal. After each accumulated
observation, the scan mirror will move 2 steps
and the instrument will rest two
seconds. During each repeat, the mirror will move
20 steps. Each repeat is separated by
five seconds to allow the mirror to
return to the start position. This example
sequence would generate 30 NIS
observations of the target, and 6 dark
observations. Choice of the wide or
narrow NIS slit, the high or low Ge
detector gain, and the starting NIS scan mirror
position are specified through
separate commands to the instrument."



END_OBJECT
INSTRUMENT_INFORMATION

OBJECT
INSTRUMENT_REFERENCE_INFO
REFERENCE_KEY_ID
"ACUNAETAL1997"
END_OBJECT
INSTRUMENT_REFERENCE_INFO

OBJECT
INSTRUMENT_REFERENCE_INFO
REFERENCE_KEY_ID
"CHENGETAL1997"
END_OBJECT
INSTRUMENT_REFERENCE_INFO
33

• page 7
• OBJECT
  INSTRUMENT_REFERENCE_INFO
• REFERENCE_KEY_ID
  "VEVERKAETAL1999A"
• END_OBJECT
  INSTRUMENT_REFERENCE_INFO
• 
  
• OBJECT
  INSTRUMENT_REFERENCE_INFO
• REFERENCE_KEY_ID
  "VEVERKAETAL1999B"
• END_OBJECT
  INSTRUMENT_REFERENCE_INFO
• 
  
• OBJECT
  INSTRUMENT_REFERENCE_INFO
• REFERENCE_KEY_ID
  "WARRENETAL1997"
• END_OBJECT
  INSTRUMENT_REFERENCE_INFO
• 
  
• OBJECT
  INSTRUMENT_REFERENCE_INFO
• REFERENCE_KEY_ID
  "YEOMANSETAL1997"
• END_OBJECT
  INSTRUMENT_REFERENCE_INFO
• 
  
• OBJECT
  INSTRUMENT_REFERENCE_INFO

34
Data supporting sample return

• Times of bullet firing and collection events
• Full geometric data including SC attitudes,
  location on target, etc.
• Imaging data on target in region of sample
  collection

• All relevant data from other instruments at the
  time of sample collection
• Temperature and pressure of sample canister from
  time of collection until return

35
NEAR Archiving

• Small Bodies Node and the Asteroid Subnode led
  the archiving of the NEAR asteroid rendezvous
  mission to the asteroid Eros. The archiving was
  completed successfully and we learned many
  lessons enabling us to improve future archiving
  efforts.

36
NEAR Archiving Issues

• PDS not allowed to interact with mission
  scientists directly until late in the mission
• Data producers contacted PDS with proposed
  archive products too late to be negotiated in any
  depth
• Some teams delivered data products too late to be
  included in Peer Review

37
Proposed Solutions

• Involve PDS in science archival design from
  beginning of mission data flow design
• Hold peer reviews per timeline to be shown
• Get early input from science team PIs on all
  delivered data and supporting information

38
Acronym Definitions

• PDS Planetary Data System
• SBN Small Bodies Node of PDS
• NEAR Near Earth Asteroid Rendezvous
• ODL Object Description Language

• AP Archive Plan
• AID Archive Interface Document
• SIS Software Interface Specification
• Cat PDS Catalog File