DICE Group UCSD SRB and iRODS

1
DICE Group UCSD SRB and iRODS
Wayne Schroeder Data Intensive Computing
Environments, San Diego Supercomputer
Center, University of California San Diego
2
Storage Resource Broker A Data Grid Solution

• Collaborative client-server system that federates
  distributed heterogeneous resources using uniform
  interfaces and metadata
• Provides a simple tool to integrate data and
  metadata handling attribute-based access
• Blends browsing and searching
• Developed at SDSC
• Operational for 9 years
• Under continual development since 1997
• Customer-driven

3
SRB Projects

• Astronomy
• National Virtual Observatory
• Data Grids
• UK e-Science CCLRC
• Teragrid
• Digital Libraries and Archives
• National Archives and Records Administration
• National Science Digital Library
• Persistent Archive Testbed
• Ecological, Environmental, Oceanographic
• ROADnet
• Southern California Earthquake Center
• SIO Digital Libraries
• Molecular Sciences
• Synchrotron Data Repository
• Alliance for Cellular Signaling
• Neuro Sciences
• Biomedical Information Research Network
• Physics and Chemistry

Over 650 Tera Bytes in 106 million files
4
(No Transcript)
5
IRODS - the Next Generation of Data Grid
Technology
6
iRODS

• A New Paradigm in Middleware Development
• Flexible Collection management
• Can be customized at user/collection-levels,
• Language for Collection management
• As in stored procedures, triggers (RDB)
• Administrative ease
• Lots of potential beyond SRB
• adaptive middleware architectures
• Is fully Open Source (BSD)

7
Rule-Oriented Data Systems Framework
Resources
8
Installation

• IRODS Is Middleware - depends on other components
• Download/Install PostgreSQL and ODBC
• Installation/Admin/ICAT
• Testing

9
Testing

• iCommand test suite from IN2P3, France
• Thomas Kachelhoffer, Jean-Yves Nief
• ICAT test suite all 205 SQL Forms
• Layers of Scripts
• Tinderbox
• installation
• irodsctl test the above two test suites
• Much More Needed/Planned

10
DICE Uses of NMI Build Test Lab

• SRB build/test with help from NMI staff
• SRB Client builds, many platforms (March/April
  2006)
• iRODS testing, port to additional platforms
  (Aug/Sept 2006)
• Port SRB to HP-UX, Spawar Project (March/April,
  2007)
• for Joint Virtual Network Centric Warfare
  (JVNCW),
• Space and Naval Warfare System Command
  (Spawar)
• iRODS WAN testing (Mike Wan, May 2007)
• iRODS testing/fixing on Intel Macs (June, 2007)
• iRODS testing, other platforms, 64-bit, etc (Aug,
  2007)
• Test/Port DICE PDU Software for SPAWAR on HP
  (Nov/Dec, 2007)
• Additional testing PDU SPAWAR (Jan/Feb, 2008)
• iRODS WAN testing (Sheau-Yen, Feb 2008)
• iRODS Tinderbox testing (interactive)
  (George/Wayne, Jan 2008-present)

11
Future NMI Uses

• Like above, as needed
• iRODS Tinderbox Build via Condor-Batch
• Or Directly Using NMI Web Site/Batch Control