Other Services in Hyrax - PowerPoint PPT Presentation

1 / 19
About This Presentation
Title:

Other Services in Hyrax

Description:

The complete problem is solved with a Numerical Library called MERLIN ... Merlin Inversion Engines. Linux Dualcore Xeon 64-bit processor with 10 Terabyte RAID ... – PowerPoint PPT presentation

Number of Views:90
Avg rating:3.0/5.0
Slides: 20
Provided by: jamesga1
Category:
Tags: hyrax | merlin | services

less

Transcript and Presenter's Notes

Title: Other Services in Hyrax


1
Other Services in Hyrax
2
Other Services in Hyrax
  • HTML Web-page form interface
  • ASCII Data responses
  • Info Metadata as formatted HTML
  • Catalogs Directory and THREDDS responses
  • GridFTP Direct access to the BES

3
HTML Form Interface
  • The Form interface provides a uniform way to
    access data
  • Its quite primitive because it must work for all
    data - even data with no semantic metadata
  • When a constraint is formed with this interface,
    the subsetting must be done using only syntactic
    information like the physical size of an array.
  • This interface is part of every Hyrax and most
    other DAP servers, although there are some slight
    variations among different implementations
  • It provides a way to view the structure of the
    data and form a constraint so that a complete URL
    can be cut and pasted into another application

4
ASCII
  • ASCII is the lowest common denominator for data
    since many programs can read it
  • Hyrax has a handler that transforms the DAP data
    response (binary data with structural metadata)
    into a CSV ASCII response.
  • You can use a spreadsheet to read data
  • Try it!

5
Find some data in the form interface
Build a constraint with the form (remember this
is ASCII were talking about, so data volume is
a consideration!) and then hit Get ASCII
6
FireFox puts the new data into a tab, other
browsers open a separate window
7
Try it with a spreadsheet type (or paste) the
URL into the Location box. Then hit OK. This
example is using Kspreadsheet from the Virtual
machine Excel works too.
8
Data come back in the spreadsheet cells. You may
be prompted to supply information Depending on
the spreadshhet
9
Info
  • The DDS (structural) and DAS (semantic) metadata
    responses can be combined and returned as an HTML
    page
  • Add the extension .info to the DAP URL
  • Try it

10
Catalogs
  • OPeNDAP directory is a kind of catalog
  • Conceptually based on the HTML directory pages
  • Provides a click-to-navigate interface
  • Different servers implement it differently and
    this has created problems with uniformity for
    clients
  • THREDDS provides a framework for cataloging data
    sources
  • Provides a way to list several different access
    protocols
  • Listings iin XML

11
Catalogs, cont.
  • To get the OPeNDAP directory listing from an old
    server, end the URL in a slash (/)
  • To get this from Hyrax, use either a slash or
    contents.html
  • To get a THREDDS catalog from any DAP server, use
    catalog.xml for the XML listing or
    catalog.html for the HTML version
  • The HTML THREDDS catalog supports a
    click-to-navigate interface like the old OPeNDAP
    directory
  • Try it

12
THREDDS examples
  • Switch Hyrax to the THREDDS HTML view

Choose the HTML view
13
The THREDDS HTML view
  • The top-level THREDDS catalog on our test server
    defines a single data root directory (SVN Test
    Data Archive)
  • This illustrates how THREDDS can be used to
    control the view of data presented by the server
  • Use catalog.xml in place of catalog.html to
    get the catalog data in an XML document.

14
Traverse the links to find data
15
THREDDS data set page
  • THREDDS catalogs can list more than one access
    mechanism - here we see on the DAP, but WCS, WMS,
    et c., are other possibilities

16
GridFTP
  • Hyrax does not actually support GridFTP but the
    BES does.
  • The BES was originally developed as a DAP-capable
    server for the Earth Systems Grid project (now
    ESG II)
  • The BES can be run standalone in appropriate
    environments and the GridFTP protocol can be used
    to connect to it.
  • Using GridFTP, DAP requests can be made and DAP
    responses transferred to ESG clients.

17
Numerical Grid Computations within BES
  • CSAC - Community Specro-polarimetric Analysis
    Center
  • Two part problem
  • Compute the so called Stokes vector for a given
    vector parameter space P using a model F(x,y,?
    P)
  • Correct P to improve the fit of the theoretical
    signal to the observed signal using non-linear
    optimization
  • The purpose is to determine a vector parameter
    space for which the model produces the best fit
    in the Least Square sense.
  • The complete problem is solved with a Numerical
    Library called MERLIN
  • Very computational Intensive, once took up to 6
    hours now takes 16 minutes.

18
High Performance Solution
  • Create multiple versions of the computing node
    waiting for data to crunch
  • Create a central place (controller) to browse
    spectra- polarimetry data inventories and launch
    inversions upon users request.
  • Let the controller decide how to distribute the
    problem.
  • Requires multiple identical processes waiting to
    be called for duty.

19
Distributing the Work
GRID
BES Server 1
Client
BES Server 2
APACHE TOMCAT MODULE
. . .
. . .
Linux Dualcore Xeon 64-bit processor with 10
Terabyte RAID
BES Server 10
Merlin Inversion Engines
20 64-bit AMD Opteron processors running on
Solaris 10. Can also plug in MacPro EightCore
machines
Write a Comment
User Comments (0)
About PowerShow.com