Grid Software Detection

1
Grid Software Detection

• Maximilian Berger
• David Gstir
• Thomas Fahringer
• Distributed and parallel Systems Group
• University of Innsbruck
• Austria

Oct, 13, 2009. Krakow, PL.
2
Outline

• Motivation
• Architecture
• Example Use Case
• Conclusions

3
Motivation

• The Grid is heterogeneous
• Clusters are (mostly) homogeneous
• Multiple Architectures, Distributions, Versions,
  Libraries
• 32 / 64 Bit
• Intel / Sparc
• Linux / Solaris
• SL Linux, Redhat, Debian, etc.
• Will my software run on this Grid site?

4
Possible Solutions

• Homogenization
• Make all sites equal
• Same Hardware
• Same Software
• Partially done in EGEE (Scientific Linux, fixed
  version)
• Virtualization
• Provide a virtual machine which provides a known
  environment
• Large Overhead!

5
Possible Solutions (cont.)

• Resource Descriptions
• Every Resource published information about itself
• Grid middleware uses this information to select
  sites
• Example GLUE schema
• Problem
• Often Incomplete!
• Mostly done manually
• Automated Software detection
• Automatic detect installed software
• Provide this information back to the user

6
Softcrawl Architecture
7
Crawler

• Contains a list of plug-ins
• Each plug-in describes
• Key-Value pairs about software
• Test script
• Parser component
• Results are stored in database (e.g. BDII)
• Tests are run periodically
• In the context of a normal user
• Multiple tests can be combined into one Grid job

8
Example Use Case

• Java Runtime Environment
• Application is written in Java
• Is Java installed?
• If so, which Version?
• Does JAVA_HOME point to a valid location?
• Prototype implementation
• Uses internal DB
• Runs in context of a personal user

9
Example Use Case Script

• !/bin/sh
• echo "-- WHICH OUTPUT BEGIN --"
• which java
• echo "-- WHICH OUTPUT END --"
• echo "-- VERSION OUTPUT BEGIN --"
• which java -version
• echo "-- VERSION OUTPUT END --"
• echo "-- JAVA_HOME OUTPUT BEGIN --"
• echo JAVA_HOME
• echo "-- JAVA_HOME OUTPUT END --"
• echo "-- JAVA_HOME_VALID OUTPUT BEGIN --"
• -x JAVA_HOME/bin/java echo true echo
  false
• echo "-- JAVA_HOME_VALID OUTPUT END --"

10
Example Use Case Values
Key Example Value Type
Environment.java_home.exists True Boolean
Java.binary.path /usr/lib/jvm/java/bin/java String
Environment.java_home.valid True Boolean
Environment.java_home.value /usr/lib/jvm/java String
Java.version.number 16013.0 Double
Java.version.string java version "1.6.0 13" Java (TM) SE Runtime Environment (build 1.6.0 13-b03) Java HotSpot(TM) Server VM (build 11.3-b02, mixed mode) String
11
Example Use Case Results
12
Conclusions

• Homogenization is not an option
• Software description helps
• Software description must be automated
• Users must be able to create tests
• Prototype implementation successful
• Collected data can be used to create better
  resource descriptions
• Jobs are now only run on sites where they
  actually work!