Java Root IO Part of the FreeHEP Java Library

1
Java Root IOPart of the FreeHEP Java Library

• Tony Johnson
• for the SLAC FreeHEP team
• Mark Dönszelmann, Joseph Perl, Victor Serbo, Max
  Turri

2
Contents

• Goals
• Java IO library for Root
• Implementation
• Test Suite
• Benchmarks
• Networked Root IO (rootd)
• Demo tools
• Root plugin for AIDA
• Examples of Use
• JAS3
• AIDA Web Interface
• Feedback for Root team
• Future plans
• Links for more information

3
Java Root IO Goals

• Goals
• Pure Java package for reading Root Files
• Will be extended to writing later
• High Performance
• We want to do data analysis so we need very good
  performance
• Should not need to know about objects ahead of
  time(no need for dlls, .so files etc.)
• But should provide full support for reading
  user-defined objects
• Easy to use
• Provide access to Root data, not C code
• Separate j2root exists for accessing in memory
  root objects

4
Implementation

• Use TStreamerInfo objects inside Root files
  (since Root 3.0) to decode contents of file
• Dynamically generate Java proxies for each Root
  object read
• Each proxy has custom streamer
• Build as Java bytecode using BCEL
  (http//jakarta.apache.org/bcel)
• Converted at runtime to machine code by
  Just-in-Time compiler HotSpot
• Uses java.nio package (new since Java 1.4)
• Designed to support high-performance binary IO

5
Implementation Details
Java Interface Accessor method for each data
member
.java file
Interface Generator
Implements
TStreamerInfo
Java Proxy Custom version based on how object is
stored in file
6
Example of Use
7
Test Suite

• Most classes are dynamically generated from info
  in Root file, so are fairly robust against
  changes
• Need bootstrap process, in particularly need
  enough custom Java classes to read the streamer
  info, and interpret it
• Relatively few classes, but
• This tends to change from one Root release to
  another
• Typically have to make small changes in the
  bootstrap code for each new Root release
• We have test suite that creates a set of test
  files from each new root release, and verifies
  that our library can read the files.
• Currently test 3.00.06, 3.01.06, 3.02.07,
  3.03.07, 3.05.07, 3.10.02
• In addition we have a set of Root files collected
  from users, and the test suite verifies that we
  can (continue) to read them with each release.

8
Java Root IO - Benchmark

• Task Loop over all events, loop over all tracks,
  sum px for all tracks. 5000 event files created
  with event test program distributed with Root.

• 1.4GHz AMD Athlon, Windows XP, 512MByte DDR,
  ATA100 disk..
• S Split level 0,1,2
• C Compression 0,1
• Each test run 5 times in succession, lines show
  range in results
• in later runs file is typically cached in memory
  faster if task IO limited

9
Network Root IO (rootd)

• Recently added ability for Java library to talk
  directly to rootd server
• Defined root protocol handler for URL class.
• Defined Authenticator for authenticating to
  rootd
• Extended RootFileReader to accept (any) URL
  argument to constructor
• If URL corresponds to file read the file
  directly
• If URL corresponds to root protocol use rootd
  (random access)
• Otherwise buffer URL contents into memory and
  read from there (suitable for small files only).

10
Authentification Issues

• Authentification can be set programmatically, or
  via GUI Authenticator
• Rootd supports many authentification schemes
• Anonymous (c.f. anonymous ftp)
• User/Password (clear text)
• User/Password (RSA encrypted)
• SCP
• SSH like
• GSI certificate
• UID/GID
• Currently Java library supports only 1 and 2
• Work underway on 3 (but some compatibility
  problems between Root and Java RSA
  implementations)
• Java libraries exist for all other methods, so
  can be supported in future.

11
Network Root IO Example of Use
12
Demo Apps Root Object Browser
13
Demo Apps Root Histogram Browser
14
Root Plug-in for JAIDA

• JAIDA Java implementation of AIDA
• AIDA Abstract Interfaces for Data Analysis
  Histograms Tuples Fitting Plotting IO
  data analysis package
• C and Python implementations exist as well
• No time for more details
• Root plugin for AIDA makes it possible to read
  root files via the AIDA interfaces and treat the
  objects contained in the root file as AIDA
  objects
• Useful if
• You like AIDA interfaces better than root
  interfaces
• You want to use language not supported by Root
• You want to analyze data in mixed formats

15
Root Plug-in for JAIDA Example
16
Root Java IO Examples of Use

• JAS3
• AIDA Web Interface

17
JAS3 Root

• JAS3 Modular, Extensible, Data Format
  Independent data analysis system.
• Among many other formats JAS3 supports Root
• Contents of Root files can be accessed directly,
  or via Root-gtAIDA interface
• Full access to AIDA functionality for fitting,
  projections, plotting etc.
• Can operate on Root data
• using GUI
• using scripting (Python, Pnuts)
• using compiled Java code

18
JAS3 access to Root Files
19
JAS3 Root Scripting
20
JAS3 Root Real life example

• Babar data quality monitoring (Carlos Chavez)

21
Put your Root Data on the Web
22
Put your Root Data on the web
23
Put your Root Data on the web
24
Put your Root Data on the web
25
Put your Root Data on the web
26
Future Plans

• Continue to track new Root releases
• Root network protocol
• More work on authentification
• Implement xrootd protocol
• should be transparent to user
• Root file writing
• Should be much easier than reading
• Most of work already done by Petr Vokac/Julius
  Hrivnac
• Support Root XML format?
• Read Babar Kanga files via root IO library
• Almost works, but some critical classes (e.g.
  KanTree) dont appear to have TStreamerInfo in
  file.

27
Suggestions for Root team

• Read Andy Hanushevskys documentation on xrootd
  protocol
• Generate more documentation like this!
• Particularly useful would be
• Documentation on rootd authentification
  mechanisms
• Documentation on low level IO storage,
  particularly different sub-classes of
  TStreamerInfo
• Currently only way to understand format is via
  reverse engineering the source code
• Very hard to know what changes between releases.

28
More Info

• Freehep Java Library
• http//java.freehep.org/
• Root IO component of library
• http//java.freehep.org/lib/freehep/doc/root/
• (will be updated soon to reflect support for
  rootd)
• JAIDA
• http//java.freehep.org/jaida/index.html
• JAS3
• http//jas.freehep.org/jas3