Virtual Grid Execution System: Present

1
Virtual Grid Execution System Present Future

• Yang-Suk Kee, Ken Yocum, Jerry Chou, Andrew Chien
• Computer Science Engineering
• University of California, San Diego
• Henri Casanova
• Information and Computer Sciences Department
• University of Hawaii

2
Virtual Grid Initiative

• Separation of concerns
• Application scheduling and execution management
• Complex Grid resource environment management

Schedule
3
Key Concepts of Virtual Grid

• Application-driven resource management
• Application-level resource abstraction
  (CCGRID05)
• Programmatic abstraction to underlying resources
• vgDL (Virtual Grid Description Language)
• Binding-aware resource selection
• Integrated selection and binding (In submission
  to HPDC06)
• vgFAB (Virtual Grid Finder Binder)
• Distributed stream processing
• Queries application-defined data streams,
  monitoring faults, progress, debugging (In
  submission to HPDC06)
• vgMON (Virtual Grid Monitor)

4
The Problems
Pegasus
LEAD
Plush
EMAN
Mortar
2-level scheduling

• vgDL exploration
• Job submission to front node
• Dynamic features of VG

VGES
Time-sharing
5
The Problems
Pegasus
LEAD
Plush
EMAN
Mortar
2-level scheduling

• Support of common RMs
• Provisioning beyond RM capability

VGES
Time-sharing
6
Requirements for Applications
Has
Needs
Three aggregates Ranking function Two
connectivity operators
vgDL
Static VG Multiple instances
Dynamic VG (add, remove, merge, split,
replacement)
VG
Per-node job submission Monolithic architecture
Per-aggregate job submission Client-server
architecture
vgES
Static controller
vgDL synthesis Event-driven controller
App
7
Requirements for Resources
Has
Needs
Time Resource properties
vgDL
VG status (unfound, found, pending, bound) VG
types (timely, guaranteed, reliable)
VG
Time-sharing
Space-sharing Provisioning Scavenging
vgES
Provisioning
Library
8
Virtual Grid World
Pegasus
LEAD
Plush
EMAN
Mortar(K.Y.)
2-level scheduling
vgDL synthesis
Provisioning system (G.S.)
VGES (Y.-S.K.)
Time-sharing
Space-sharing
Provisioning
Scavenging
9
LEAD on VG

• Dynamic workflow
• Rapid and automatic configuration change in
  response to weather and resources
• Dynamic resource allocation close to existing
  resources
• Dynamic VG, notification

Initial parameters
Time
Space
10
VG Application Development Cycle
vgDL description
Resource type Ranking function Connectivity
Resource allocation
Performance evaluation
Deadline Quality Throughput Response time
Time-sharing Space-sharing Provisioning Private Sc
avenging
Application management
Scheduling Planning Execution management