Title: Other Services in Hyrax
1Other Services in Hyrax
2Other Services in Hyrax
- HTML Web-page form interface
- ASCII Data responses
- Info Metadata as formatted HTML
- Catalogs Directory and THREDDS responses
- GridFTP Direct access to the BES
3HTML Form Interface
- The Form interface provides a uniform way to
access data - Its quite primitive because it must work for all
data - even data with no semantic metadata - When a constraint is formed with this interface,
the subsetting must be done using only syntactic
information like the physical size of an array. - This interface is part of every Hyrax and most
other DAP servers, although there are some slight
variations among different implementations - It provides a way to view the structure of the
data and form a constraint so that a complete URL
can be cut and pasted into another application
4ASCII
- ASCII is the lowest common denominator for data
since many programs can read it - Hyrax has a handler that transforms the DAP data
response (binary data with structural metadata)
into a CSV ASCII response. - You can use a spreadsheet to read data
- Try it!
5Find some data in the form interface
Build a constraint with the form (remember this
is ASCII were talking about, so data volume is
a consideration!) and then hit Get ASCII
6FireFox puts the new data into a tab, other
browsers open a separate window
7Try it with a spreadsheet type (or paste) the
URL into the Location box. Then hit OK. This
example is using Kspreadsheet from the Virtual
machine Excel works too.
8Data come back in the spreadsheet cells. You may
be prompted to supply information Depending on
the spreadshhet
9Info
- The DDS (structural) and DAS (semantic) metadata
responses can be combined and returned as an HTML
page - Add the extension .info to the DAP URL
- Try it
10Catalogs
- OPeNDAP directory is a kind of catalog
- Conceptually based on the HTML directory pages
- Provides a click-to-navigate interface
- Different servers implement it differently and
this has created problems with uniformity for
clients - THREDDS provides a framework for cataloging data
sources - Provides a way to list several different access
protocols - Listings iin XML
11Catalogs, cont.
- To get the OPeNDAP directory listing from an old
server, end the URL in a slash (/) - To get this from Hyrax, use either a slash or
contents.html - To get a THREDDS catalog from any DAP server, use
catalog.xml for the XML listing or
catalog.html for the HTML version - The HTML THREDDS catalog supports a
click-to-navigate interface like the old OPeNDAP
directory - Try it
12THREDDS examples
- Switch Hyrax to the THREDDS HTML view
Choose the HTML view
13The THREDDS HTML view
- The top-level THREDDS catalog on our test server
defines a single data root directory (SVN Test
Data Archive) - This illustrates how THREDDS can be used to
control the view of data presented by the server - Use catalog.xml in place of catalog.html to
get the catalog data in an XML document.
14Traverse the links to find data
15THREDDS data set page
- THREDDS catalogs can list more than one access
mechanism - here we see on the DAP, but WCS, WMS,
et c., are other possibilities
16GridFTP
- Hyrax does not actually support GridFTP but the
BES does. - The BES was originally developed as a DAP-capable
server for the Earth Systems Grid project (now
ESG II) - The BES can be run standalone in appropriate
environments and the GridFTP protocol can be used
to connect to it. - Using GridFTP, DAP requests can be made and DAP
responses transferred to ESG clients.
17Numerical Grid Computations within BES
- CSAC - Community Specro-polarimetric Analysis
Center - Two part problem
- Compute the so called Stokes vector for a given
vector parameter space P using a model F(x,y,?
P) - Correct P to improve the fit of the theoretical
signal to the observed signal using non-linear
optimization - The purpose is to determine a vector parameter
space for which the model produces the best fit
in the Least Square sense. - The complete problem is solved with a Numerical
Library called MERLIN - Very computational Intensive, once took up to 6
hours now takes 16 minutes.
18High Performance Solution
- Create multiple versions of the computing node
waiting for data to crunch - Create a central place (controller) to browse
spectra- polarimetry data inventories and launch
inversions upon users request. - Let the controller decide how to distribute the
problem. - Requires multiple identical processes waiting to
be called for duty.
19Distributing the Work
GRID
BES Server 1
Client
BES Server 2
APACHE TOMCAT MODULE
. . .
. . .
Linux Dualcore Xeon 64-bit processor with 10
Terabyte RAID
BES Server 10
Merlin Inversion Engines
20 64-bit AMD Opteron processors running on
Solaris 10. Can also plug in MacPro EightCore
machines