What is Grid Computing

1
What is Grid Computing?

• Richard Hopkins
• rph_at_nesc.ac.uk
• NGS Induction National e-Science Centre,
  Edinburgh, 29th Sept 2005

2
Acknowledgements

• This talk was prepared by Richard Hopkins of NeSC
  and includes slides from previous tutorials and
  talks delivered by
• Dave Berry, Mike Mineter, Guy Warner (National
  e-Science Centre)
• the EDG training team
• Ian Foster, Argonne National Laboratories
• Jeffrey Grethe, SDSC
• EGEE colleagues
• Mark Baker, The Distributed Systems Group,
  University of Portsmouth, http//dsg.port.ac.uk/ma
  b
• Talks at 3rd EGEE conference by
• Kyriakos Baxevanidis,Deputy Head,Unit of Research
  Infrastructures,European Commission, DG INFSO
• Dr Spyros Konidaris, European Commission DG
  INFSO

3
Policy for re-use

• This presentation can be re-used, in part or in
  whole, provided its sources are acknowledged.
• However if you re-use a substantial part of this
  presentation please inform training-support_at_nesc.a
  c.uk. We need to gather statistics of re-use
  number of events and number of people trained.
  Thank you!!

4
Goals and Content

• Goal - To introduce the concepts of Grid
  computing assuming no previous knowledge
• What is a grid ?
• Drivers of grid computing
• Current status of grids

5
The Grid Metaphor
Mobile Access
G R I D M I D D L E W A R E
Supercomputer, PC-Cluster
Workstation
Data-storage, Sensors, Experiments
Visualising
Internet, networks
6
The grid vision

• The grid vision is of Virtual computing (
  information services to locate computation,
  storage resources)
• Compare The web virtual documents ( search
  engine to locate them)
• MOTIVATION collaboration through sharing
  resources (and expertise) to expand horizons of
• Research
• Commerce engineering, the knowledge economy
• Public service health, environment,

7
A grid

• The initial vision The Grid
• The present reality Many grids
• Each grid is an infrastructure enabling one or
  more virtual organisations (VOs) to share
  computing resources
• Whats a VO?
• People in different organisations seeking to
  cooperate and share resources across their
  organisational boundaries
• Why establish a Grid?
• Share data
• Share computers
• Share instruments
• Collaborate

VO
8
Single Computer

• The Operating System enables easy use of
• Input/Output devices
• Processor
• Disks
• Display
• Instruments

9
Local Area Network
User just perceives shared resources, with no
regard to location in the organisation LAN
resources act like a single virtual
computer Middleware (LAN O/S) presents that image
10
A grid

• Users join VOs
• Virtual organisation negotiates with sites to
  agree access to resources
• Distributed services (both people and middleware)
  enable the grid

INTERNET
11
Grid

• Grid middleware creates the image of the Grid
  being a single virtual computer (Ideally)
• Issues
• Heterogeneity hardware, software, culture
• Scalability
• Reliability tolerate permanent partial failure
• Viable computing model - batch processing
• Access control
• Authentication
• Authorisation
• Single sign on

Application Software
Interface between app. and grid
Grid Middleware collective services
Grid Middleware on each resource
Operating System on each resource
Resources connected by internet
12
What characterises a grid?

• Co-ordinated resource sharing
• No centralised point of control
• Different administrative domains.
• Standard, open, general-purpose protocols and
  interfaces
• NOT specific to an application
• EGEE, NGS support multiple VOs
• Delivering non-trivial qualities of service
• Co-ordinated to deliver combined services,
  greater than sum of the individual components
• http//www.gridtoday.com/02/0722/100136.html

13
The components of a Grid

• Resources
• networking, computers, storage, data,
  instruments,
• Grid Middleware
• the operating system of the grid
• Operations infrastructure
• Run enabling services (people software)
• Virtual Organization management
• Procedures for gaining access to resources

14
DRIVERS OF GRID COMPUTING

• Goal - To introduce the concepts of Grid
  computing assuming no previous knowledge
• What is a grid ?
• Drivers of grid computing
• Current status of grids

15
The first driver e-Science

• What is e-Science? Collaborative science that is
  made possible by the sharing across the Internet
  of resources (data, instruments, computation,
  peoples expertise...)
• Often very compute intensive
• Often very data intensive (both creating new data
  and accessing very large data collections) data
  deluges from new technologies
• Crosses administrative boundaries
• Examples.

16
Astronomy

• No. sizes of data sets as of mid-2002,
  grouped by wavelength
• 12 waveband coverage of large areas of the
  sky
• Total about 200 TB data
• Doubling every 12 months
• Largest catalogues near 1Billion objects

Data and images courtesy Alex Szalay, John
Hopkins University

17
Large Hadron Collider at CERN

• Data Challenge
• 10 Petabytes/year of data !!!
• 20 million CDs each year!
• Simulation, reconstruction, analysis
• LHC data handling requires computing power
  equivalent to 100,000 of today's fastest PC
  processors!
• Operational challenges
• Reliable and scalable through project lifetime of
  decades

Mont Blanc (4810 m)
Downtown Geneva
18
BLAST gridification
Computing element
Input file
UI
Computing element
19
DAME Grid based tools and Infer-structure for
Aero-Engine Diagnosis and Prognosis

• A Significant factor in the success of the
  Rolls-Royce campaign to power the Boeing 7E7 with
  the Trent 1000 was the emphasis on the new
  aftermarket support service for the engines
  provided via DSS. Boeing personnel were shown
  DAME as an example of the new ways of gathering
  and processing the large amounts of data that
  could be retrieved from an advanced aircraft such
  as the 7E7, and they were very impressed, DSS
  2004

XTO
Companies Rolls-Royce DSS Cybula
Universities York, Leeds, Sheffield, Oxford
Engine Model
Case Based Reasoning
20
Academic drivers not only e-science!!
The impact of grids when they support
21
Academic drivers

• Centrality of curation, preservation
• Under-recognised by many researchers
• Virtual Digital Data Libraries needed for
  research as well as learning

• Digital libraries

• E-research

• E-learning

Derived from a slide by the UKs JISC

• AAA Services

• e-Infrastructure

22
Political drivers

• Entering the knowledge society from the
  industrial society
• Industrial society Transportation
  Infrastructure
• Knowledge society Communications infrastructure
• Lisbon strategy Research and Innovation will be
  the most important factors in determining
  Europes success through the next decades
• THE GOAL UNLEASH CREATIVITY- by investment in
• Human skills
• Infrastructures
• Growth of e-infrastructure ( networks grid
  operations)
• phase 1 mainly academia, some in industry an
  elite, privileged to do this job
• phase 2 ordinary people doing distributed work
  SMEs, adopt, adapt and use
• phase 3 the next generations
• Will transform e-infrastructure and its uses
• We dont know how others will use what we devise
• Just as current use of WWW not predictable by its
  initiators

23
EGEE building e-infrastructure

• EGEE is building a large-scale production grid
  service to
• Underpin research, technology and public service
• Link with and build on national, regional and
  international initiatives
• Foster international cooperation both in the
  creation and the use of the e-infrastructure

24
CURRENT STATUS OF GRIDS

• Goal - To introduce the concepts of Grid
  computing assuming no previous knowledge
• What is a grid ?
• Drivers of grid computing
• Current status of grids

25
If The Grid vision leads us here
then where are we now?
26
Grid projects

• Many Grid development efforts all over the
  world

• UK OGSA-DAI, RealityGrid, GeoDise,
  Comb-e-Chem, DiscoveryNet, DAME, AstroGrid,
  GridPP, MyGrid, GOLD, eDiamond, Integrative
  Biology,
• Netherlands VLAM, PolderGrid
• Germany UNICORE, Grid proposal
• France Grid funding approved
• Italy INFN Grid
• Eire Grid proposals
• Switzerland - Network/Grid proposal
• Hungary DemoGrid, Grid proposal
• Norway, Sweden - NorduGrid

• NASA Information Power Grid
• DOE Science Grid
• NSF National Virtual Observatory
• NSF GriPhyN
• DOE Particle Physics Data Grid
• NSF TeraGrid
• DOE ASCI Grid
• DOE Earth Systems Grid
• DARPA CoABS Grid
• NEESGrid
• DOH BIRN
• NSF iVDGL

• DataGrid (CERN, ...)
• EuroGrid (Unicore)
• DataTag (CERN,)
• Astrophysical Virtual Observatory
• GRIP (Globus/Unicore)
• GRIA (Industrial applications)
• GridLab (Cactus Toolkit)
• CrossGrid (Infrastructure Components)
• EGSO (Solar Physics)

27
Grids where are we now?

• Many key concepts identified and known
• Many grid projects have tested, and benefit from,
  these
• Major efforts now on establishing
• Standards (a slow process) (e.g. Global Grid
  Forum, http//www.gridforum.org/ )
• Production Grids for multiple VOs
• Production Reliable, sustainable, with
  commitments to quality of service
• In Europe, EGEE
• In UK, National Grid Service
• In US, Teragrid
• One stack of middleware that serves many research
  (and other!!!) communities
• Operational procedures and services (people!,
  policy,..)
• New user communities
• whilst research development continues

28
The key for new VOs

• The tools, services used by the VOs applications
  
• Application development environment, portals,
  semantics
• Insulate applications from changing middleware

29
The vision of 2001 convergence of Web Services
and Grids
Open Grid Services Architecture
web developments
big Science research
OGSIGrid prototypes
Web services
INTERNET
World-wide web
High-end computing High throughput-computing
Massively parallel computing
30
Key concepts

• Virtual organisation
• people and resources collaborating - across
  admin, organisational boundaries
• Individual joins VO
• VO negotiations with resource providers
• Grid middleware
• running on each resource to interface it to the
  Grid
• providing specific services
• Single Virtual Computer
• User just perceives shared resources with no
  concern for location or owning organisation
• Issues
• Heterogeneity
• Scalability
• Reliability
• Computing model
• Access control

31
Key Concepts

• Drives are towards
• Production services (reliable, sustainable,
  against which research projects can plan with
  confidence)
• In Europe, EGEE
• In UK, National Grid Service
• Standards convergence with WWW mainstream
• Empowering new user communities