Computational Grids and Computational Economy: Nimrod/G Approach

1
Computational Grids and Computational Economy
Nimrod/G Approach
David Abramson Rajkumar Buyya Jonathan Giddy
2
Parametric Execution of Applications

• Coarse-grained SPMD model
• Execute one application repeatedly for many
  combinations of input parameters
• Legacy applications add iteration and
  distribution without modifying code
• New applications remove iteration and
  distribution from design
• Parametrised modeling experiments
• Require very high levels of performance
• Generate
• Large amounts of work concurrency
• Uncoupled computations
• Tolerate - moderately high latencies

3
Description of Parameters
4
Working with Small Clusters

• Nimrod (1994 - )
• DSTC funded project
• Designed for department level clusters
• Proof of concept
• Clustor (Activetools) (1997 - )
• Commercial version of Nimrod
• Re-engineered
• Features
• Workstation orientation
• Access to idle workstations
• Random allocation policy
• Password security

5
Clustor limitations

• Manual resource location
• static file of machine names
• No resource scheduling
• first come first served
• No cost model
• all machines cost alike
• Single access mechanism

6
Towards Grid Computing.
Source www.globus.org updated
7
Nimrod/G - Nimrod over Globus/Grid

• Wide-Area Network Support
• redesigned architecture
• use of high-performance networks
• Scalable Scheduling
• guaranteed deadline
• use of existing schedulers
• Computational Economy
• I am willing to pay , can you complete the job
  by given deadline
• trading, bidding, resource reservation...

8
Layered Architecture (Grid Components)
Applications
High-level Services and Tools
GlobusView
Testbed Status
Nimrod/G
DUROC
globusrun
MPI
MPI-IO
CC
Core Services
GRAM
Nexus
Metacomputing Directory Service
Globus Security Interface
Heartbeat Monitor
Gloperf
GASS
Local Services
Condor
MPI
TCP
UDP
LSF
NQE
Easy
Solaris
Irix
AIX
Source www.globus.org
9
Nimrod/G Architecture
Nimrod/G Client
Nimrod/G Client
Nimrod/G Client
Parametric Engine
Schedule Advisor
Resource Discovery
Persistent Info.
Dispatcher
Grid Directory Services
Grid Middleware Services
GUSTO Test Bed
10
Nimrod/G Interactions

• Additional services used implicitly
• GSI (authentication authorization)
• Nexus (communication)

Resource location
MDS server
Scheduler
Resource allocation (local)
Prmtc.. Engine
Queuing System
Job Wrapper
Dispatcher
GRAM server
Root node
Gatekeeper node
Computational node
11
Scheduling Algorithm

• Find a set of machines (MDS search)
• Distribute jobs from root to machines
• Establish job consumption rate for each machine
• For each machine
• Can we meet deadline?
• If not, then return some jobs to root
• If yes, distribute more jobs to resource
• If cannot meet deadline with current resource
• Find additional resources

12
A Nimrod/G Client
13
Sample Applications of Nimrod

• Bioinformatics Protein Modeling
• Sensitivity experiments on smog formation
• Parametric study of Laser detuning
• Combinatorial Optimization Simulated Annealing
• Ecological Modeling Control Strategies for
  Cattle Tick
• Electronic CAD Field Programmable Gate Arrays
• Computer Graphics Ray Tracing
• High Energy Physics Searching for Rare Events
• Physics Laser-Atom Collisions
• VLSI Design SPICE Simulations
• Radiation Protection and Nuclear Safety

14
Electronic CAD
15
Some early results -
16
(No Transcript)
17
Related Works

• AppLeS (UC. San Diego)
• application level scheduling case-by-case
• NetSolve (UTK/ORNL)
• API for creating farms
• DISCWorld (U. Adelaide)
• remote information access
• Millennium (UC. Berkeley)
• remote execution environment on clusters and
  supports computational economy

18
Conclusions

• Nimrod/G architecture offers a scalable model
  for resource management and scheduling on
  computational grids
• Supports Computational Economy
• The current model supporting Parametric
  Computing can be extended to support parallel
  jobs or any other computational model.
• Plan to use the concept of Advance Resource
  Reservation in order to offer the feature wherein
  the user can say I am willing to pay , can you
  complete my job by this time
• Further Information www.csse.monash.edu.au/david
  a/nimrod.html