AstroGrid and Starlink Data Modelling

1
AstroGrid and Starlink Data Modelling
2
AstroGrid Domain Model
also referred to as AstroGrid Conceptual
architecture
3
Detailed view
4
Structured Object

• Need Structured Object in addition to Data
  Objects
• to separate the Structure from the Data
• avoid stressing FITS
• allow use of distributed data
• Easily extensible to allow additional metadata
  e.g. relationships
• Need something moderately standardised because
• applications from many sources must have some
  rules about what to do with things they do not
  understand

5
Structured Object HDX

• Flexible data model based on many years
  experience with HDS using local data
• also STPDF
• Aim to support distributed processing and data
  holding
• Uses URIs to point to data
• Low (or even zero) overhead e.g. bare FITS files
  OK
• Can be serialised as XML
• Platform independent
• Format agnostic
• Natural formats are XML and FITS

6
HDX

• An HDX is a W3C DOM which has a top-level element
  lthdxgt, and which is valid in HDX terms.
• valid if each of the document element's children
  is either unknown to the HDX system or, if known,
  is valid
• when a new type is registered with the HDX system
• the software registering it declares a validator
• the software implements the DOM API

7
Data access layer

• The abstract data model has been implemented in a
  Java data-access library.
• The support for the HDX system is distinct from
  the support for the various HDX types which are
  defined.
• It is easy to extend the system to support new
  types
• It is easy to extend the system to support new
  data storage resources, such as new file formats
  or a database serving an archive. These
  extensions can be implemented in very efficient
  ways.

8
HDX UML
9
NDX

• NDX represents an N-dimensional chunk of
  astronomical data and contains
• Image pixel array
• Variance pixel array
• Quality pixel array
• Bad bit mask
• WCS information
• History information
• Title
• Units
• User-defined extensions (stored as XML)
• ...

10
More details NDX/NDArray

• The philosophy and design goals behind
  NDArray/NDX are
• Very large arrays can be processed.
• Arrays of unlimited size can be processed in
  limited memory. 
• Bad value processing is comprehensive and
  transparent. 
• Array access is direct and transparent between
  different formats. 
• Resource naming is location transparent. 
• Deferred processing. 
• Etensibility. 
• Type independence where possible.

11
NDArray

• An NDArray represents a hyper-rectangular array
  of primitive values byte, short, int, float,
  double. As well as the numeric data itself, it
  has the following attributes
• Shape 
• Type 
• BadHandler 
• Flags

12
NDX UML
13
IHDA