How do We Make Grid Useful

1
How do We Make Grid Useful?

• Xiaodong Zhang
• National Science Foundation
• This talk does not necessarily reflect NSFs
  official opinions

2
Client/Server based Grids A Starting Point

• Original vision and state-of-the-art Grid
• a global networking infrastructure connecting
  multiple high performance computational
  resources.
• Targeted applications
• Supercomputing across the globe.
• Collaborative computing
• Global data repository and data-intensive
  computing
• Core Technology
• centralized administration (e.g. resource
  registrations)
• centralized management (e.g. job scheduling)

3
NSF Sponsored Grid Efforts

• 1997 to 2002
  Two Partnerships for Adv.
  Comp. Infras. (PACI) NCSA at Illinois
  and NPACI at San Diego
  leading 60 institutions from 27 states.
• Missions
  - providing grid computing
  and data resources
  - developing grid
  software tools
  - applications on grids
  - education
  outreach and training.

4
Building National Grid Infrastructure

• 2001 to 2004
  Distributed Terascale
  Facility (DTF) 4 DTF sites
  NCSA, NPACI, Argonne, and Caltech
  providing aggregated 14 teraflops
  and 450 terabytes.
• Tasks
  - NCSA 6 TFs 240TBs
  Linux cluster of Itaniums
  - NPACI
  4 TFs 225 TBs
  - Angonne 1 TF IBM cluster, grid viz.
  software
  - Caltech 86 TB on-line storage.

5
Large NSF Sponsored Grid Projects

• GIOD (Globally Interconnected Object Databases)
• global data storage and accesses of particle
  collider experiments
• GriPhyN (Grid Physics Network)
• building global grids for experimental
  physics studies.
• iVDgL (international Virtual-Data grid Lab)
• grids for physics/astronomy experiments
• data-intensive science, US EU collaboration
• NEES (Network for Earthquake Engineering
  Simulation)
• shifting from physical tests to simulation
  (20 grid sites)

6
Additional NSF Grid Efforts

• 2003 to 2005
  Enhanced Distributed
  Terascale Facility 4
  original DTF sites plus Pittsburgh SC.
  
• Tasks
  - Enhancing the existing
  DTFs software and hardware
  -
  Testing large scale applications.
  - Widely connecting to
  users.

7
Limits of Current Grid Infrastructure

• Deployment of grid is still not easy.
  
  
  
• High cost and case by case (e.g. NSF grid
  projects)

• Application scope is narrow, and killer apps are
  limited
• More and more local clusters will satisfy
  applications.
• Special ones go to custom-designed SC (ES,
  Blue-gene).
• Global supercomputing is not cost- and
  performance-effective storing data is much
  cheaper than transferring.

• Centralized administration and management
• limiting the scalability.
• Creating single points of failures.

8
Limits of Current Grid Infrastructure

• Lack of resource sharing among different type
  services
  
  
• A grid or collaborative grids provide single
  type service.

• Lack of security/trusts among different grid
  services.
• Each service has its own standards and protocols.

• Grid business model needs to be further
  refined.

9
Enhanced Grid Vision and Challenges

• Grid scope is extended to global Internet
  covering both major components (servers) and
  edges (peers).
• Keys Resource administration management.
• Keeping merits of client/server on security and
  reliable services.
• Keeping merits of P2P on scalability and avoiding
  single point of failures.
• Challenges resource virtualization across
  regions.

10
Heterogeneous Grid Members Co-exist

• Billions of clients in the format of c-phones,
  PDAs, laptops, PCs at home (Internet/wireless).
• Millions of clients become termed super-peer
  nodes.
• Millions of powerful clusters for local services.
  
• Millions of trusted/independent grid nodes serve.
  
• Millions of trusted/collaborative grid nodes
  serve.
• Dozens of supercomputers for science advancement.

11
Future of Distributed Computing

• Grid infrastructure will provide reliable
  computing resources for large collaborations.
• In a grid region, P2P techniques will be
  integrated for resource administration and
  management.
• P2P paradigm will play a major role for
  information retrievals.
• The demand for data accesses/transfers will be
  higher than cycles.