Experiments with Remote Access to Scientific Data using HDF5

1
Experiments with Remote Access to Scientific Data
using HDF5

• Kun Yan
• Robert E. McGrath
• (mcgrath_at_ncsa.uiuc.edu)
• NCSA
• May 15, 2002

2
Todays Talk

• Background
• Experiments with JSP, XML, HDF4, HDF5, etc.
• Experiments with CORBA, Java, HDF5
• Discussion

3
Acknowledgements
Kun Yan conducted these experiments as a Research
Assistant at NCSA. This report is based upon work
supported in part by a Cooperative Agreement with
NASA under NASA grants NAG 5-2040 and NCC5-599.
Any opinions, findings, and conclusions or
recommendations expressed in this material are
those of the author(s) and do not necessarily
reflect the views of the National Aeronautics and
Space Administration. Other support provided by
NCSA and other sponsors and agencies.
4
Client/Remote
Server/Local
Distributed Product Demonstrated in
Research Should work, but not demonstrated
5
2. Experiments with JSP

• Experiments with JSP, XML, XSL, etc.
• Web access to Data and services.
• Demonstration of browsing HDF5 datasets
• Generate HTML or XML
• Convert data from HDF4 to HDF5 on demand
• http//arabica.ncsa.uiuc.edu7070/hdf5/

6
Tomcat

• is a standalone web server that supports the
  Servlet and JSP specifications.
• developed under the Jakarta project at the Apache
  Software Foundation.
• integrated into the J2EE Reference
  Implementation.
• Tomcat 4.x (Catalina, Servlet 2.3, JSP 1.2)
• Tomcat 3.3

7
JavaServer Pages

• JavaServer PagesTM (JSPTM) technology
• rapidly develop and easily maintain, dynamic web
  pages
• uses XML-like tags and Java scriptlets
• application logic can reside in server-based
  resources

8
The Flow of a Request
9
response
request
HTML
Xalan.jar Xerces.jar
client
JSP
response
XML
beans
beans
beans
Application
XSL
DB
data
lib
How JSP and Beans Work together
10
3. Experiments with CORBA

• Created a demonstration CORBA server that reads
  HDF5
• Written in C, linked directly to HDF5 library.
• Created a pure-java client
• No CORBA required on the client
• Can be used from an applet within a standard Web
  browser

11
Technology used

• Orbacus 4.1.0 (C) (free, full source)
• Java 1.3 or above
• HDF5.1.4.3, HDF5 C interface

12
HDF5 CORBA Experiment -- Design Diagram CORBA
Object Hierarchy
13
HDF5 CORBA Experiment IDL File

• void getNumAttributes(out long attrNum)
• void openAttributes (out long attr)
• interface H5DatasetObjH5Obj
• void getDatasetInfo(out DatasetInfo dsinfo)
• long long getStorageSize()
• interface H5GroupObjH5Obj
• ObjInfo getGMember(in long fid, in string
  gName, in long index)
• void openGroup(in string gName, out
  H5GroupObj h5gobj)
• void getDatasetInfo(in string dsName, out
  DatasetInfo dsinfo)
• void openDataset(in string dsName, out
  H5DatasetObj h5ds)
• void getDataTypeInfo(in string ttName, out
  DatatypeInfo dtinfo)
• interface H5FileObj

• module HDF5
• struct ObjInfo
• long oid
• long fid
• long type
• string name
• long numOfMembers
• typedef sequenceltlonggt Sizes
• struct DatasetInfo
• string name
• long numOfDim
• Sizes dimSizes
• Sizes dimMaxSizes
• long dataTypeClass
• long dataTypeSize

14
HDF5 CORBA Experiment -- Design Diagram C
Servant Object Hierarchy
H5Obj
H5FileAccess
H5FileObj
H5DatasetObj
H5GroupObj
H5Obj_impl
ObjInfo poa_
getObjInfo setObjName
H5FileAccess_impl
H5FileObj_impl
H5H5File poa_
H5H5File ObjInfo poa_
openFile closeFile
getTOC openRootGroup setFileName
H5DatasetObj_impl
H5GroupObj_impl
H5H5Daraset DatasetInfo
H5H5Group poa_
getDatasetInfo getStorageSize
openGroup getGMembers openDataset getDatasetInfo
getDatatypeInfo
15
HDF5 CORBA Experiment -- How to Use the Client
Applet

• Select file name from drop-down list (file
  resides corba server side)
• Select Flags and Access Property of the selected
  file
• Click on open file to open the file
• Click on get file TOC to get the file object
  info
• Click on open root to open the root group
• Click on get group TOC to get the root group
  object info
• Click on iterate group to get the group
  members info of the root and each subgroup
• Input dataset name including path, then click on
  get dataset to get the dataset object
• Click on get dataset TOC to get the dataset
  object info
• Click on get storage size to get the storage
  size of the opened dataset object
• Input dataset name including path, then click on
  get dataset info to get object info of the
  given dataset
• Input datatype name including path, then click on
  get datatype info to get object info of the
  given datatype

16
URLs

• JSP experiments http//hdf.ncsa.uiuc.edu/HDF5/XML
  /JSPExperiments/
• http//hdf.ncsa.uiuc.edu
• http//hdf.ncsa.uiuc.edu/h4toh5/
• http//hdf.ncsa.uiuc.edu/HDF5/XML/
• http//hdf.ncsa.uiuc.edu/hdf-java-html

17
4. Discussion
18
Client/Remote
Server/Local
Distributed Product Demonstrated in
Research Should work, but not demonstrated
19

• HDF5 CORBA Experiment -- User Interface

20
HDF5 CORBA Experiment -- Examples of Object Info
window
21
(No Transcript)
22
Other HDF Tools
23
Java
HDF4
HDF5 file
HDF5 file
Viewer/Editor
h4toh5
XML
XML
ASCII
HDF4
HDF5 file
Generator
XML
Source code
24
HDF DDL
Dumper
HDF5 file
XML
Non-Java (C)
h4toh5
HDF4 file
HDF5
H4 to H5 LIbrary
h5toh4
HDF5 file
HDF4
gif2hdf
GIF file
HDF5
fp2hdf
flat file
HDF5
25
Silly XMLTricks
Netcdf to HDF5, via XML using XSL
ncxdump
nc.xml
netcdf
netcdf.dtd
xalan
nctoh5.xsl
h5gen
h5.xml
hdf5
26
URLs

• JSP experiments http//hdf.ncsa.uiuc.edu/HDF5/XML
  /JSPExperiments/
• http//hdf.ncsa.uiuc.edu
• http//hdf.ncsa.uiuc.edu/h4toh5/
• http//hdf.ncsa.uiuc.edu/HDF5/XML/
• http//hdf.ncsa.uiuc.edu/hdf-java-html