Hyrax Architecture

1
Hyrax Architecture

• Two cooperating processes
• Front-end provides DAP interface
• Back-end reads data
• Both parts can be customized
• Front-end different network protocols
• Back-end different data formats/systems
• N-Tier design is flexible, secure

2
Hyrax Architecture

• Front-end (OLFS)
• Authentication Authorization (via Tomcat)
• DAP2 compliant interface.
• SOAP interface.
• Extensible via pluggable dispatch handlers
• Back-end (BES)
• Data resource access.
• Extensible (see Patricks slides)
• Single/Multiple Machine Installations
• Security

3
Hyrax Architecture
BES Commands
DAP2
THREDDS
XML- encapsulated object
HTML
Data Store
File system with data files, SQL Database,
Optional THREDDS catalogs
4
OLFS and Authorization/Authentication

• Authorization Authentication (Az/Ac) actually
  provided by Apache or Tomcat
• Apache/Tomcat provides a range of Az/Ac features
• Realms Role-based Az/Ac
• Usernames passwords -gt Authorization
• Roles -gt Authentication
• Realm info in RDBs, XML docs, LDAP,
• TLS/SSL
• Server and client Az/Ac
• Secure data transmission
• Tomcat provides single sign-on capabilities
• Clients must support cookies
• Suitable for portals

5
Possible Installations

• Both OLFS and BES run on one machine
• OLFS on one machine and BES on another
• One OLFS and several instances of BES on
  different machines
• OLFS communicating with one or more BESs and
  other backend processors

6
Hyrax Architecture Summary

• Hyrax can be installed on one machine or several
• Installation security merits serious
  consideration
• Authentication Authorization are handled by the
  web servers (Tomcat and Apache)

7
Extending the OLFS

• Extension modules written in Java
• Added to a directory within Tomcat
• The new modules have complete access to the
  request information
• Both HTTP GET and POST requests
• There is some significant processing done before
  the handler is called
• Conditional GET Requests
• Authorization Authentication

8
OLFS Dispatch Handlers

• The OLFS uses an ordered list of Dispatch
  Handlers.
• Each handler on the list is asked if it can
  handle the incoming request.
• The first handler on the list that claims the
  request gets to handle it.
• List order is important, as some handlers (for
  example THREDDS) may claim to handle requests
  that could (should) be handled by a different
  handler.
• Handlers are free to do anything they need to
  handle a request Contact a remote
  system/process, read files, spawn threads, et
  cetera.

9
Extending the BES

• Extension modules written in C or C
• Download already existing modules
• Write new modules
• Add these modules to the BES configuration file
  to be dynamically loaded
• Modules interact with the BES and can interact
  with other modules
• Modules add to, or modify functionality in, the
  BES framework

10
BES
BES Framework
Network Protocol and Process start/stop activities
PPT
Initialization/ Termination
DAP2 Access
Data Catalogs
BES Commands/ XML Documents
Commands
NetCDF3
HDF4
FreeForm
Data Store Interfaces

PPT point to point transport Some commands
are built in, most load at run-time
11
BES Extensibility

• In what ways can you extend the BES?
• New request handlers (data handlers like netcdf,
  hdf4)
• New response handlers (Cedar's Flat, Tab, Info)
• New commands (ESGs commit command)
• Container and Definition storage
• Aggregation engines
• Methods of returning your data (return as nc)
• Initialization/Termination callbacks
• Exception Handlers
• Reporters

12
THREDDS Catalogs
13
THREDDS responses

• Use THREDDS to define a logical hierarchy thats
  distinct from the set of directories that
  actually hold the data.
• We can request THREDDS catalog XML files using
  catalog.xml or HTML pages using catalog.html
  after a directory name.
• While the directory browser works for any
  directory, THREDDS catalogs are valid only for
  the logical hierarchy they define
• Files/Directories not included in that hierarchy
  have no catalogs

14
THREDDS examples

• Switch Hyrax to the THREDDS HTML view

Choose the HTML view
15
The THREDDS HTML view

• The top-level THREDDS catalog on our test server
  defines a single data root directory (SVN Test
  Data Archive)
• This illustrates how THREDDS can be used to
  control the view of data presented by the server
• Use catalog.xml in place of catalog.html to
  get the catalog data in an XML document.

16
Traverse the links to find data
17
THREDDS data set page

• THREDDS catalogs can list more than one access
  mechanism - here we see on the DAP, but WCS, WMS,
  et c., are other possibilities

18
Choosing DAP access leads to the HTML form
19
Security Authentication Authorization
20
Authentication Authorization

• Hyrax currently relies on the security features
  implemented by Tomcat for authentication and
  authorization services.
• The Tomcat authentication model is based on
  Realms and roles.
• A Realm is a database of usernames and
  passwords that identify valid users of a web
  application.
• A role is similar to a UNIX group because access
  to to resources is granted to all users possesing
  a particular role. A particular user can have any
  number of roles.

21
Authentication Authorization

• Tomcat supports 5 standard plug-ins that support
  connections to various sources of authentication
  (Realms)

22
Authentication Authorization

• JDBCRealm - Accesses authentication information
  stored in a relational database, accessed via a
  JDBC driver.
• DataSourceRealm - Accesses authentication
  information stored in a relational database,
  accessed via a named JNDI JDBC DataSource.
• JNDIRealm - Accesses authentication information
  stored in an LDAP based directory server,
  accessed via a JNDI provider.

23
Authentication Authorization

• MemoryRealm - Accesses authentication information
  stored in an in-memory object collection, which
  is initialized from an XML document
  (conf/tomcat-users.xml).
• JAASRealm - Accesses authentication information
  through the Java Authentication Authorization
  Service (JAAS) framework.

24
Authentication Authorization

• Realm and Role configuration is achieved by
  editing XML files in the Tomcat distribution.
• Passwords may be saved as clear text, or if
  desired they may be stored in a digested form.
• The standard Realms support SHA, MD2, and MD5
  digest algorithms.

25
Hyrax Logging
26
Hyrax Installation Logging

• Access Logging - Many people will want to record
  access logs for their Hyrax server. We want you
  to keep access logs for your Hyrax server. The
  easiest way to get a simple access log for Hyrax
  is to utilize the Tomcat/Catalina Valve Component
• Informational/Debug Logging - In general you
  shouldn't have to modify the default logging
  configuration for Hyrax. It may become necessary
  if you encounter problems, but otherwise I
  suggest you leave it be. Enabling it can both
  consume disk space and increase your security
  vulnerability.

27
Hyrax Logging

• Access Logging
• Since Hyrax's public facade is provided by the
  OLFS running inside of the Tomcat servlet
  container you may utilize Tomcat's handy access
  logging which relies on the org.apache.catalina.va
  lves.AccessLogValve class. By default Tomcat
  comes with this turned off. It can be easily
  enabled by editing an XML file in the Tomcat
  distribution.

28
Hyrax Logging

• Informational and Debug Logging in the OLFS
• The OLFS uses the Log4j logging package to
  provide an easily configurable and flexible
  logging environment. All "console" output is
  routed through the Log4j package and can be
  controlled using the Log4j configuration file.
• Log4j allows the user to control logging output
  in a hierarchical manner from the (java) package
  down to the individual class level.

29
Hyrax Logging

• Informational and Debug Logging in the BES
• The BES has two forms of logging.
• Event logging in a bes.log file. This is
  configured in the bes.conf file
• BES.LogName/path/to/bes.log
• BES.LogVerboseno
• Debug logging for the BES
• besctl start -d /path/to/bes.debug,nc,bes,ppt

30
BES Debugging

• Format of the command line option -d
• cerrltlog_file_namegt,ltcontext1gt,,ltcontextngt
• E.g. -d /tmp/bes.debug,nc,bes
• Specify a context of all to turn on debugging
  for everything
• Writing debug statements in your code
• Macro BESDEBUG( ltcontextgt, ltoutput to ostreamgt
  )
• E.g. BESDEBUG( nc, reading DAS for ltlt file
  ltlt endl )
• Chunk of debugging
• If( BESDebugIsSet( nc ) )
• listltstringgtconst_iterator i
  _somelist.begin()
• while( i ! _somelist.end() i )
• BESDEBUG( nc, ltlt (i) )

31
BES Debugging

• Most BES classes derived from BESObj, and some
  libdap class derived from DapObj
• You can dump BESObj instances and DapObj
  instances to ostream
• virtual void dump( ostream strm )
• E.g.
• DAS das
• BESDEBUG( nc, DAS ltlt endl ltlt das )
• Allows for indentation as well

32
BES Debugging - dump method example