Convergence Characteristics for Clusters, Grids, and P2P networks

1
Convergence Characteristics for Clusters, Grids,
and P2P networks
Panel at the P2P conference, Linkoping, Sweden

• Rajkumar Buyya

Grid Computing and Distributed Systems (GRIDS)
Lab. The University of MelbourneMelbourne,
Australiawww.gridbus.org
2
Where Did I come From ?
StockholmLinkoping


London
Singapore


Melbourne
3
GRIDS Lab _at_ the U. of Melbourne, The Gridbus
Project www.gridbus.org

• Conduct RD in Distributed Computational Economy
  for Service-Oriented Computing
• Architecture, Specification, and Open Source
  Reference Implementation in in Collaboration with
  Any Global Peers.
• Platform Cluster, Grid, P2P for diverse
  applications.
• Distributed Application Composition Tools
• Grid Economy Scheduling (via Nimrod-G Broker)
• GridSim Toolkit Grid Modeling and Simulation
  (Java based)
• Libra Economic Cluster Scheduler
• Grid Bank Accounting and Transaction Management
• World Wide Grid (WWG) A Global Testbed
• P2P/Jxta-based Compute Power Market
• Application Enabler Projects
• Virtual Laboratory Toolset for Drug Design
• High-Energy Physics and the Grid Network
  (HEPGrid)
• Brain Activity Analysis on the Grid
• Future GridEmail for Spam Managementread only
  if sender pays!

4
Books
5
Technology Evolution Cluster, Grid, P2P
6
Scalable Seamless Computing Breaking
Administrative Barriers
?
PERFORMANCE
Administrative Barriers

• Individual
• Group
• Department
• Campus
• State
• National
• Globe
• Inter Planet
• Galaxy

Desktop
SMPs or SuperComputers
Local Cluster
Global Cluster/Grid
Inter Planetary Grid!
Enterprise Cluster/Grid
7
Basic Elements
Application Development Tools
8
Cluster Architecture
Parallel Applications
Parallel Applications
Parallel Applications
Sequential Applications
Sequential Applications
Sequential Applications
Parallel Programming Environment
Cluster Middleware (Single System Image and
Availability Infrastructure)
Cluster Interconnection Network/Switch
9
Cluster Architecture
Parallel Applications
Parallel Applications
Parallel Applications
Sequential Applications
Sequential Applications
Sequential Applications
Parallel Programming Environment
Cluster Middleware (Single System Image and
Availability Infrastructure)
Cluster Interconnection Network/Switch
10
A Typical Grid Computing Environment
Grid Information Service
Grid Resource Broker
Application
R2
R3
2
R4
R5
RN
Grid Resource Broker
R6
R1
Resource Broker
Grid Information Service
11
A Typical Grid Computing Environment
Grid Information Service
Grid Resource Broker
Application
R2
R3
2
R4
R5
RN
Grid Resource Broker
R6
R1
Resource Broker
Grid Information Service
12
A Typical P2P Computing Environment
Peer Discovery Service
Peer Agent
Application
P3
pM
Who can help ?
Peer P2, P7 can help!
pN
Request
P2
Sorry, I am busy.
Peer Agent
Request
Peer Agent
Response
P1
R7
p4
p5
13
A Typical P2P Computing Environment
Peer Discovery Service
Peer Agent
Application
P3
pM
Who can help ?
Peer P2, P7 can help!
pN
Request
P2
Sorry, I am busy.
Peer Agent
Request
Peer Agent
Response
P1
R7
p4
p5
14
Cluster, Grid, P2P Characteristics
Characteristic Cluster Grid P2P
Population Commodity Computers High-end computers Edge of network (desktop PC)
Ownership Single Multiple Multiple
Discovery Membership Services Centralised Index Decentralised Info Decentralized
User Management Centralised Decentralised Decentralised
Resource management Centralized Distributed Distributed
Allocation/Scheduling Centralised Decentralised Decentralised
Inter-Operability VIA based? No standards yet No standards
Single System Image Yes No No
Scalability 100s 1000? Millions? _at_Home
Capacity Guaranteed Varies, but high Varies
Throughput Medium High Very High
Speed(Lat. Bandwidth) Low, high High, Low High, Low
15
Nimrod-G Modeling Distributed Computing in
Economic Terms
Nimrod/G Client
Nimrod/G Client
Nimrod/G Client
Nimrod-G Engine
Schedule Advisor
Trading Manager
Grid Store
Grid Dispatcher
Grid Explorer
Grid Middleware
TM TS
Globus, Legion, Condor, etc.
GE GIS
Grid Information Server(s)
RM TS
RM TS
RM TS
G
C
L
G
Legion enabled node.
Globus enabled node.
L
G
C
L
RM Local Resource Manager, TS Trade Server
Condor enabled node.
16
On-Demand Computing
Cost
Deadline
Legion hosts
Globus Hosts
Bezek is in both Globus and Legion Domains
17
Active Sheet ? Nimrod-GTransparent Spreadsheet
Processing from PCs to Clusters to World-Wide
Grids
Nimrod Proxy
Nimrod-G
World-Wide Grid
18
MEG(MagnetoEncephaloGraphy) Data Analysis on the
Grid Brain Activity Analysis
64 sensors MEG
Analysis All pairs (64x64) of MEG data by
shifting the temporal region of MEG data over
time 0 to 29750 64x64x29750 jobs
2
3
Data Analysis
1
5
Nimrod-G
4
Life-electronics laboratory, AIST
World-Wide Grid

• Provision of expertise in
• the analysis of brain function
• Provision of MEG analysis

Collaboration with Osaka University, Japan
19
Backup Slides
20
Evolution Time Line
21
What is Cluster/Grid ?

• A type of parallel and distributed system that
  enables the sharing, selection, aggregationof
  resources distributed in _____/_______
  administrative domains depending on their
  availability, capability, performance, cost, and
  users quality of service requirements.

Multiple
A Single
Cluster
Grid
22
CPM DC Economy-based P2P Computing(Jxta based
Implementation)

• Discovery
• - Membership

User (Consumer)
Trader
Job Management
Resources (Provider)
Accounting
23
Drug Design Data Intensive Computing on Grid
Chemical Databases (legacy, in .MOL2 format)

• It involves screening millions of chemical
  compounds (molecules) in the Chemical DataBase
  (CDB) to identify those having potential to serve
  as drug candidates.

Collaboration with WEHI for Medical Science,
Melbourne
24
Virtual Drug DesignA Virtual Lab for Molecular
Modeling for Drug Design on P2P Grid
Grid Info. Service
Grid Market Directory
Data Replica Catalogue
Give me list PDBs sources Of type aldrich_300?
service cost?
service providers?
GTS
Resource Broker
Screen 2K molecules in 30min. for 10
mol.5 please?
(RB maps suitable Grid nodes and Protein DataBank)
get mol.10 from pdb1 screen it.
PDB2
mol.10 please?
(GTS - Grid Trade Server)
PDB1