EasyGrid Job Submission System and Gridification Techniques

1
EasyGrid Job Submission System and Gridification
Techniques

• James Cunha Werner
• Christmas Meeting 2006 - University of Manchester

2
Develop grid software for BaBar experiment at
University of Manchester

• BaBar is a high-energy physics experiment running
  since 1999 at Stanford University/SLAC to throw
  light on how the matter-antimatter symmetric Big
  Bang can have given rise to todays
  matter-dominated universe.
• BaBar analysis was a conventional centralized
  software (850 packages).
• The project goal was to study grid performance
  and develop gridification algorithms 3 papers
  published 1 paper submitted.
• Data management was a main issue Distributed
  datasets around the world and several other
  support databases (conditions, configuration,
  bookkeeping metadata, and parameters).
• Distributed and heterogeneous hardware platform
  around the world (standards).
• Users do not have grid skills.Their interests
  were high energy physics, not grid.
• Reliability should be at least the same as SLAC.
  Users have a fixed time to do their research,
  they will use the more efficient resource.

3
EasyGrid Job Submission system for grid
It is an intermediate layer between Grid
middleware and users software. It integrates
data, parameters, software, and grid middleware
doing all submission and management of several
users software copies to grid doing Data and/or
Functional parallelism.
4
Dataset name
Gridification Process from conventional to grid
computing.
gt Easygrid BetaMiniApp Tau11-Run3
Grid enabled software User software
Gridification algorithms
File name
Data Gridification
Functional Gridification
gt BetaMiniApp Tau11-Run3.tcl

• EasyGrid Job Submission system
• Submit jobs
• Manage datasets
• Recover results
• Recover reports

User software
User computer
Workload management
Data Management
Performance analysis
Datasets
Grid resources
Many files in sequence
Many files in parallel -gt Fast
See http//www.hep.man.ac.uk/u/jamwer/Grid2006.pdf
for more information
5
Data parallelism in Grid

• Each data file will be read by each copy of the
  binary code in parallel.
• EasyGrid Tasks
• Copy binary code at closest storage elements.
• Set environment in each worker node.
• Start the binary code.
• Recover results in users directory.
• Provide information in case software fails.
• Tools for data management and replication.

6
Data gridification in action
7
Functional parallelism in grid

• One master binary code (or client) requesting
  services and managing load flow.
• EasyGrid Tasks
• Set a task queue.
• Search information system for services published
  in grid.
• Establish sections in each worker node.
• Start services and initialize software.
• Send data for processing in each server.
• Manages processing and re-submit in case of fail.
• Manages notification and recover results in
  master.

8
Functional gridification in action
9
Summary

• Easygrid is available and can be use for
  distributed analysis in any experiment or grid
  project.
• Published at AHM2006, Nottingham.
• Supported the development of evolutionary
  discriminate functions for neutral pion
  (EuroGP2007).
• I am not the only user

10
Eta from Hadronic Tau Decays(Marta Tavera PhD
Student)
She knows nothing about grid, but using EasyGrid
she did her analysis!!! Merry Christmas and Happy
New Year!!!