Grid Scheduling Architecture

1
Grid Scheduling Architecture

• BOF
• GGF-5, Edinburgh 2002
• July 23, 2002

2
BOF Agenda

• What is the subject of this BOF?
• Is this subject important for the Grid community?
• Is it useful to start an initiative now and in
  this area?
• Is the proposed approach reasonable?
• Is the time frame realistic?
• Who will participate in this initiative?

3
Execution of a Grid Job

• Transfer of experimental input data from a large
  shared data store to the local memory of the
  target computer.
• The data transfer must be successfully completed
  before starting the evaluation.
• Data evaluation by simulation with the target
  computer and an additional parallel computer.
• The expected computation time depends on the
  available network bandwidth.
• Visualization of the results by using a (local)
  virtual reality cave.
• The local memory of the VR-cave is not enough to
  hold all result data.
• A guaranteed network bandwidth between the target
  computer and the VR-cave is required.
• The visualization can start as soon as sufficient
  data are available.
• Transfer of the simulation results to the shared
  data store.

4
Resource Allocation for the Job
Data Store
Storing Data
Data Access
Network 1
Data Transfer
Data Transfer
Computer 1
Loading Data Parallel
Computation Providing Data
Communication for Computation
Network 2
Computer 2
Parallel Computation
Network 3
Communication for Visualization
VR-Cave
Visualization
time
5
Role of Grid Scheduling

• Grid Scheduling is a service that determines,
  reserves and allocates all Grid resources that
  are required and available for the execution of a
  job.
• To this end a Grid Scheduler needs to interact
  and co-operate with various other systems in a
  Grid
• Local (hardware and system software) resource
  management systems
• Data management system
• Grid information system
• Grid monitoring system
• Grid accounting system
• User portals
• Grid security system

6
Subject of the BOF

• How does a Grid Scheduler work together with
  other functional units in the Grid?
• Identification of the functional units
• Functional description of the co-operation
  between the scheduler and those units
• Which components form a Grid Scheduler?
• Description of the tasks of the components
• Functional description of the interactions
  between different components

7
BOF Agenda

• What is the subject of this BOF?
• Is this subject important for the Grid community?
• Is is useful to start an initiative now in this
  area?
• Is the proposed approach reasonable?
• Is the time frame realistic?
• Who will participate in this initiative?

8
Importance of the Issue

• Is there any fully functional Grid Scheduler
  available?
• Has a Grid Scheduler architecture been described
  in a publication?
• Is it easy to create such a scheduler?
• Is there a need for a Grid Scheduler?
• Will Grid users benefit from such a scheduler?
• Will such a scheduler be accepted by resource
  providers?
• Will the definition of an architecture help to
• create an implementation?
• support the acceptance of such an architecture?

9
Part of an Architecture Proposal
Information Service
static scheduled
Query for resources
Scheduling Service
Resources
Reservation of resources
GIIS/GRIS,..
Data Management Service
Maintain information
Data
NWS
Job Supervision Service
...
Network
Network Management Service
Maintain information
Globus GRAM/GRIS
IBM
ManagementSystem
Compute/ Storage /Visualization
10
BOF Agenda

• What is the subject of this BOF?
• Is this subject important for the Grid community?
• Is is useful to start an initiative now in this
  area?
• Is the proposed approach reasonable?
• Is the time frame realistic?
• Who will participate in this initiative?

11
Time and Affiliation

• Would it be better to wait for more input?
• More applications and new use cases?
• More GGF documents with a more restricted focus?
• Scheduler Attributes Document
• Definition of more protocols?
• Advance reservation protocol
• Distributed Resource Management Application API
• Grid Resource Allocation Agreement Protocol
• What is the most appropriate area for a Grid
  Scheduler?
• Shall this initiative belong to more than one
  area?

12
BOF Agenda

• What is the subject of this BOF?
• Is this subject important for the Grid community?
• Is is useful to start an initiative now in this
  area?
• Is the proposed approach reasonable?
• Is the time frame realistic?
• Who will participate in this initiative?

13
Different Approaches

• Working group within the GGF
• Goal Rapid production of a document describing
  the Grid Scheduler Architecture with substantial
  influence from the Grid community
• less focus on scientific content and originality
• Research group within the GGF
• Goal Exploration of various possible
  architectural alternatives
• long term focus
• Research or commercial activity outside of the
  GGF
• Goal Scientific and/or commercial achievements
• Possible initiation but little control from the
  Grid community

14
Purpose of the Working Group
Definition of a Grid Scheduler Architecture that
allows co-operation between local resource
management systems for network, storage and
processing units and additional Grid services as
provided, for instance, by Grid information and
monitoring systems. Specifically, the
interaction between hardware resource management
and data management shall be considered in the
architecture.
15
BOF Agenda

• What is the subject of this BOF?
• Is this subject important for the Grid community?
• Is is useful to start an initiative now in this
  area?
• Is the proposed approach reasonable?
• Is the time frame realistic?
• Who will participate in this initiative?

16
Milestones

• GGF 5, July 2002 Initiation of the working group
  and presentation of the outline of a document
  defining a Grid Scheduler architecture
• GGF 6, October 2002 Delivery of the a first
  draft of the document and first plenary
  discussion
• GGF 7, February 2003 Delivery of a revised
  document and second plenary discussion
• GGF 8, July 2003 Delivery of the final document
  and third plenary discussion
• September 2003 Final revision and submission of
  the document for the review process

17
BOF Agenda

• What is the subject of this BOF?
• Is this subject important for the Grid community?
• Is is useful to start an initiative now in this
  area?
• Is the proposed approach reasonable?
• Is the time frame realistic?
• Who will participate in this initiative?

18
Structure of the Working Group

• Working group chair(s)
• Uwe Schwiegelshohn, University Dortmund ???
• Uwe.Schwiegelshohn_at_udo.edu
• Working group secretary
• Ramin Yahyapour, University Dortmund
• Ramin.Yahyapour_at_uni-dortmund.de
• Document authors
• Achim Streit, University Paderborn
• Volker Sander, Research Center, Julich
• Florian Schintke, Konrad Zuse Center, Berlin
• ???
• Providing additional input via GGF and mailing
  list sched-wg_at_gridforum.org
• ???