XSIL Extensible Scientific Interchange Language

1
XSIL Extensible Scientific Interchange Language
XSIL Extensible Scientific Interchange Language
http//www.cacr.caltech.edu/XSIL
R. Williams, T. Prince, A. Lazzarini, K.
Blackburn
Caltech CACR, LIGO Laboratory
Roy Williams
California Institute of Technology

• Notebook for Scientific Data and Metadata
• Data and metadata exchange
• Creating and editing metadata
• Catalogue applications
• Electronic Log applications

• Notebook for Scientific Data and Metadata
• Data and metadata exchange
• Creating and editing metadata
• Catalogue applications
• Electronic Log applications

2
LIGO
Laser Interferometric Gravitational
wave Observatory
Listening to Collisions of Black Holes
and Neutron Stars
3
LIGO
Detection of cosmic gravitational waves
4
LIGO Data Objects
Multichannel time-series Interferometer
Seismometer Microphone Magnetometer
Feedback systems Frequency series Power
spectra Noise spectra Events Astrophysics,
Instrument glitch, ... Images beam
shape Tables Parameters
5
What is that Binary Data?
Question What does this binary file
mean? Answer Look at the metadata! Where is
it? In the XSIL file!
6
Tools for Scientific Data Archives
Leverage E-Commerce!
Web Java XML Servlets EJB E-services
E-speak Software components Application
servers Security Databases
Search for this book In my shopping basket My
authentication Download Giftwrap Make it in size
10, with the pink edging
Login Find events like this Cache from tape to
disk As a Mathematica workbook Mine this Tbyte
with this parametrized search
7
XML Extensible Markup Language
8
XML personal HTML
Separation of structure from presentation
ltFromgtAntonio Stadivariuslt/Fromgt ltTogtDomenico
Scarlattilt/Togt ltDategt ltDaygt13lt/Daygt ltMonthgt4lt/Mo
nthgt ltYeargt1723lt/Yeargt lt/Dategt ltBodygt Io bisogno
una appartamento acoglienti a Cremona lt/Bodygt
13/4/23 April 13, 1723 17.iv.1723
9
Why XML
The cost is not the hardware The cost is not the
software The cost is the DEPENDENCIES The
solution is STANDARD INTERFACES
XML is a standard way to represent structured
documents, including metadata and
data Platform neutral / Open Vendor supported
/ Vendor neutral Proven -- decades with
SGML Extensible Syntax checking -- Explicit
Schema Embraces Java -- the software standard
10
XML for science
The library or the simulation should
output Binary objects glued together
with Structured documents XML can provide the
syntax for this structure Like HTML but
extensible for metadata, programs,
formatting for searching, archiving,
summarizing -- now we need to define Document
Types (what are the fields on the catalogue
card?)
11
XSILExtensible Scientific Interchange Language
http//www.cacr.caltech.edu/XSIL/
Base Objects Param, Table, Array, Comment,
Event, ... Hierarchy Streams Local/Remote,
Mirrorred Typed, Mime, Raw Extensibility How
can I represent MyObject
12
XSIL Example Pulsar Search in LIGO Data
lt?xml version"1.0"?gt lt!DOCTYPE XSIL SYSTEM
"XSIL.dtd"gt ltXSIL Name"fold3212711Agt
ltCommentgtCreator Stuart Anderson and Roy
Williamslt/Commentgt ltCommentgtThis time series
has been folded at freqfoldlt/Commentgt ltParam
Name"Dataset"gtfold3212711Alt/Paramgt lt!--
REQUIRED DIMENSIONS AND VARIABLES --gt ltParam
Name"n_elem Typeintgt 100 lt/Paramgt
ltParam Name"n_chan Typeintgt 1
lt/Paramgt lt!-- VARIABLES AND ATTRIBUTES --gt
ltParam Name"ligo_type"gt time series
lt/Paramgt ltParam Name"phase TYpedoublegt
21773750.252132lt/Paramgt ltParam Name"tphase
Typedoublegt 0.252132 lt/Paramgt
ltParam Name"sample_freq Typedoublegt2000.00000
0 lt/Paramgt continued...
13
XSIL Example Pulsar Search in LIGO Data
...continued lt!-- HERE IS THE LINK TO THE
DATA --gt ltStream TypeRemotegt
file//fold3212711A.bin lt/Streamgt ltArray
Name"data" Type"double"gt ltDim
Namexgt100lt/Dimgt ltDim Nameygt20lt/Dimgt
lt/Arraygt ltArray Name"spectrum"
Type"double"gt ltDimgt131072lt/Dimgt
lt/Arraygt lt/XSILgt
14
XSIL Base Objects
ltXSIL NameApple TypeFruitgt The container,
which may contain other containers
ltParam Name... Unit...gt ltTime Type...gt ltArray
Name... Type...gt ltDimgt... ltTable
Name...gt ltColumn Name... Unit...
Type...gt ltURLgt
ltStreamgt Local/Remote Typed/Raw/MIME Encoding
Actuation
15
XSIL Streams
Data is Explicit
ltStream TypeLocal Delimiter,gt
1.28374, 1.23453, 1.94847, 2.148474
2.39484, 2.84746, 3.10928, 4.92827 lt/Streamgt
Data is External
ltStream TypeRemote EncodingBigendian"
Timeout"600/gt http//hpss.cacr.caltech.edu
/magval_09_25_97.bin lt/Streamgt
16
Extending XSIL
Look for Java class of this name Expected
internal structure Helper application Queries,
Filters
ltXSIL TypeTimeSeriesgt ltParam
NameStartTimegt943873726.92837lt/Paramgt ltParam
NameEndTimegt943873876.83736lt/Paramgt ltArray
NameDatagt ltDimgt1024lt/Dimgt lt/Arraygt ltStream
TypeRemotegt file//mydata.dat lt/Streamgt lt/XSI
Lgt
17
XML Pro -- An XML Editor
18
Xeena from IBM
19
Parsed to HTML
20
XSL Stylesheets
XML with XSL stylesheet applied
Raw XML
21
XML Parsing
SAX Event-Based Handlers for StartElement, Text,
EndElement, etc. DOM Document Object
Model Returns a tree-like Document object with
data attached
22
XSIL Information Flow
GUI Editor Presentation
Scripting Tcl, Javascript, Python, ...
Metadata
Filter
Filter
XSIL notebook
Data
Numerical computation or other formats
Local and remote data Servers, Databases, Binary
files
23
World Domination Plan
XSIL
XML tools
own code, own parser
combining tools
supercomputer tape robot
extends org.xsil.XSIL
Matlab IDL Excel FITS hdf etc . etc.
standard filters
extends org.xsil.GUI
Xlook control panel
24
XSIL filter Matlab
XSIL
Matlab
xsil2matlab lt file.xml gt file.mat
25
Some Java
package org.xsil.XSIL public class TimeSeries
extends XSIL double startTime 0.0
double endTime 1.0 int ndata 0
double data public void
constructor(Element e) ...
for(int ichild0 ichild lt children.size()
ichild) Object o
children.elementAt(ichild) if(o
instanceof Param) Param p
(Param)o if(p.getName().equals("S
tartTime")) startTime new
Double(p.getText()).doubleValue()
if(p.getName().equals("EndTime"
)) endTime new
Double(p.getText()).doubleValue()
if(o instanceof
Array ((Array)o).getNdim() 1)
....
inherited tree structure
base objects
26
Dynamic Loading
Code is not linked
Just throw it in the box and it will be used!
MyObject.class
CLASSPATH
27
Xlook Scientific Notebook
Implemented with Java Swing classes -- standard
with Java2
28
Xlook Scientific Notebook
Implemented with LiveTable Bean from KL Group
(www.klg.com)
29
Xlook Scientific Notebook
Implemented with JChart Bean from KL Group
(www.klg.com)
30
Xlook Scientific Notebook
Browser services for MIME-typed data (URLs)
31
Distributed Science Data
XML/XSIL
Java RMI
Server EJB or Servlet Service Computing,
Data, Metadata Transport RMI or XML
32
Digital Puglia Synthetic Aperture Radar Atlas
Multichannel Image Retrieval Format
Selection Processing Menu
Bari, Multitemporal
33
NPACI Digital Sky Project
34
The Sky is a Database
35
Cross-Identification
2MASS versus DPOSS cross-identification 2MASS
j_m , 15 DPOSS I_mtotn lt 18