GREASE Grid Aware Endtoend Analysis and Simulation Environment

1
GREASEGrid Aware End-to-end Analysis and
Simulation Environment

• Mark ter Linden, Ruud Grim21 May 2003

2
Content

• Current situation of the Grid
• Aim of GREASE
• Architecture
• How does it work?
• Demonstration environment
• Near future plans
• More information

3
Current Situation of the Grid

• Globus has command-line tools to submit jobs and
  transfer files
• For a simple job the user has to
• Obtain a proxy certificate
• Find a suitable grid node
• Transfer all required input to the node
• Submit the job
• Transfer all output back
• For complex jobs (workflows) this has to be done
  many times
• Problems with firewalls (lot of openings needed
  for Globus)

4
Aim

• Aim of GREASE is to make it easier for scientists
  to use their existing applications in a Grid
  environment but hide the underlying technology.
• Focus will be on the simulation, validation and
  calibration of data processing applications for
  remote sensing instruments.
• High Level Requirements
• No or minimal re-coding of existing applications
• Use existing Grid environments (based on Globus
  Toolkit 2)
• Easy to use interfaces

5
Example

• Example of data processing applications for an
  Earth Observing Instrument
• Workflow !
• Data transfer via Files

Orbit Model
Atmosphere Model
Instrument Model
Downlink Model
L0 Processor
L1b Processor
L2 Processor
L2 Processor
6
Architecture (1)

• Three major elements in GREASE

7
Architecture (2)

• Core of the GREASE system is the Controller which
  includes the Workflow Engine
• Receives complex jobs in the form of workflows
• Selects Grid nodes for execution
• Submits each of the applications in the workflow
• Handles file transfer between the applications
• Provides information for monitoring

8
Architecture (3)

• Workflows can be created with the Workflow Tool.
• Progress of submitted jobs can be monitored with
  the Monitoring Tool.
• The Registry contains all the information about
  the applications that the Workflow Engine and the
  Workflow Tool needs.
• Information in the Registry can be edited with
  the Registry Tool

9
How does it work?
Application
Data
Grid cluster
Application
The Internet
Grid cluster
Firewall
Workflow engine
Workflow tool
Intranet
Scientist
10
Demonstration environment
32-Node DAS-2 Cluster at Leiden University
32-Node DAS-2 Cluster at NIKHEF
The Internet
4-Node Cluster at Dutch Space
32-Node Cluster at ESA/ESRIN
Firewall
Workflow engine
Dutch Space Intranet
Workflow tool
Workflow tool
User
User
11
Screenshots

• Workflow Tool

12
Screenshots (2)

• Monitoring Tool

13
Near future plans

• Objective transfer GREASE into a truly Grid
  assisting software package
• Method open source and co-operation
• ESA, Grid Community

14
GridAssist should

• . assist the user to use his or her application
  in the Grid in an user friendly and easy way
• . allow researchers to define work flows based
  on research applications and data sets
• . allow operators of Data Centers to construct
  and operate work flows for operational data
  production in the Grid

15
GridAssist example
HIRLAM - a numerical weather prediction system

• Approx. 100 components
• Preferred platforms
• Security

16
More information

• Website http//tphon.dutchspace.nl/grease
• Ruud Grim, Dutch Space, r.grim_at_dutchspace.nl
• Eric Boom, Dutch Space, e.boom_at_dutchspace.nl
• Mark ter Linden, Dutch Space, m.ter.linden_at_dutchsp
  ace.nl