The Semantic Grid A Future eScience Infrastructure

1
The Semantic GridA Future e-Science
Infrastructure

• Presented by
• David De Roure
• University of Southampton, UK

www.semanticgrid.org
dder_at_ecs.soton.ac.uk
2
Structure of the talk

• Evolution of the Grid
• The Semantic Web
• The Semantic Grid
• The story so far
• Semantic Grid Projects
• Closing Remarks

3
The Semantic Grid

• Evolution of the Grid
• The Semantic Web
• The Semantic Grid
• The story so far
• Semantic Grid Projects
• Closing Remarks

• The Evolution of the Grid

4
The Collaboratory Concept

• In 1989, William Wulf, then with the U.S.
  National Science Foundation, defined a
  collaboratory as

"a center without walls, in which the nation's
researchers can perform their research without
regard to geographical location, interacting with
colleagues, accessing instrumentation, sharing
data and computational resources, and accessing
information in digital libraries."
5
A short history of the Grid

• Science as a team sport
• Grand Challenge Problems of the 80s
• Gigabit Testbed program
• Focus on applications for the local to wide area
• FAFNER
• Factoring via Network-Enabled Recursion
• I-Way at SC 95
• First large-scale grid experiment
• Provided the basis for modern grid infrastructure
  efforts

(Fran Berman, SDSC)
6
Datagrid perspective
www.griphyn.org www.ppdg.net
www.eu-datagrid.org
7
1995 2000 Grid Computing

• Grid book gave a comprehensive view of the
  state of the art
• Important infrastructure and middleware efforts
  initiated
• Globus, Legion, Condor, SRB, etc.
• 2000 Beginnings of a Global Grid
• Evolution of the Global Grid Forum
• Some projects evolving to de facto standards
  (e.g. Globus, Condor)

8
The Grid Problem

• Resource sharing coordinated problem
  solvingin dynamic, multi-institutional virtual
  organizations

Foster, Kesselman, Tueke
9
Open Grid Services Architecture

• Anatomy vs Physiology
• Present Grid Architecture is a services
  architecture
• Implemented using Web Services Technology
• OGSA will provide
• Naming /Authorization / Security / Privacy
• Higher level services Workflow, Transactions,
  DataMining,Knowledge Discovery,
• Exploiting Synergy Commercial Internet with Grid
  Services

10
UNiform Interface to COmputing REsources

• UNICORE is a vertically integrated Grid
  environment offering seamless, secure and
  intuitive access to distributed computing
  resources

11
e-Science

• e-Science is about global collaboration in key
  areas of science, and the next generation of
  infrastructure that will enable it.
• e-Science will change the dynamic of the way
  science is undertaken.
• John Taylor, DG of UK OST
• The Grid intends to make access to computing
  power, scientific data repositories and
  experimental facilities as easy as the Web makes
  access to information.
• Tony Blair, 2002

12
UK e-Science Initiative

• 180M Programme over 3 years
• 130M is for Grid Applications in all areas of
  science and engineering
• Particle Physics and Astronomy (PPARC)
• - 25M GridPP and 8M AstroGrid
• Engineering and Physical Sciences (EPSRC)
• - funding 6 projects at around 5M each
• Biology, Medical and Environmental Science
• - funding projects with total value of 35M
• 50M Core Program to encourage development of
  generic industrial strength Grid middleware

13
Some UK e-Science Projects

• Climateprediction.com (NERC)
• Oceanographic Grid (NERC)
• Molecular Environmental Grid (NERC)
• NERC DataGrid (NERC OST-CP)
• Biomolecular Grid (BBSRC)
• Proteome Annotation Pipeline (BBSRC)
• High-Throughput Structural Biology (BBSRC)
• Global Biodiversity (BBSRC)

• GRIDPP (PPARC)
• ASTROGRID (PPARC)
• Comb-e-Chem (EPSRC)
• DAME (EPSRC)
• DiscoveryNet (EPSRC)
• GEODISE (EPSRC)
• myGrid (EPSRC)
• RealityGrid (EPSRC)

• Biology of Ageing (BBSRC MRC)
• Sequence and Structure Data (MRC)
• Molecular Genetics (MRC)
• Cancer Management (MRC PPARC)
• Clinical e-Science Framework (MRC)
• Neuroinformatics Modeling Tools (MRC)

• Interdisciplinary Research Collaborations Grand
  Challenge
• Advanced Knowledge Technologies
• Medical Images and Signals
• Equator
• DIRC (Dependability)

14
UK e-Science Grid
Edinburgh
Glasgow
Newcastle
DL
Belfast
Manchester
Cambridge
Oxford
Hinxton
RAL
Cardiff
London
Southampton
15
Access Grid nodes
Access Grid

16
Todays Grid activities across the world
17
Observations

• The Grid has been about large scale computation
• But the applications are also about collaboration
• Middleware has provided some computational
  interoperability
• But we now need semantic interoperability
• The old problem was lots of different computers
• The new problem is lots of different projects!

18
The Semantic Grid

• Evolution of the Grid
• The Semantic Web
• The Semantic Grid
• The story so far
• Semantic Grid Projects
• Closing Remarks

• The Semantic Web

19
Vision

• The Semantic Web is an extension of the current
  Web in which information is given a well-defined
  meaning, better enabling computers and people to
  work in cooperation. It is the idea of having
  data on the Web defined and linked in a way that
  it can be used for more effective discovery,
  automation, integration and reuse across various
  applications. The Web can reach its full
  potential if it becomes a place where data can be
  processed by automated tools as well as people
• From the W3C Semantic Web Activity statement

20
Resource Description Framework
21
Semantic Web
Richer semantics
Classical Web
22
OWL Web Ontology Language
The World Wide Web as it is currently
constituted resembles a poorly mapped geography.
Our insight into the documents and capabilities
available are based on keyword searches, abetted
by clever use of document connectivity and usage
patterns. The sheer mass of this data is
unmanageable without powerful tool support. In
order to map this terrain more precisely,
computational agents require machine-readable
descriptions of the content and capabilities of
web accessible resources. These descriptions must
be in addition to the human-readable versions of
that information. The OWL Guide
23
SW Tools
24
Observations

• Semantic Web requires a metadata-enabled Web
• Where will the metadata come from?
• Semantic Web requires ontologies
• Where will the ontologies come from?
• What will motivate the generation of ontologies
  and metadata?

25
The Semantic Grid

• Evolution of the Grid
• The Semantic Web
• The Semantic Grid
• The story so far
• Semantic Grid Projects
• Closing Remarks

• The Semantic Grid

26
Grid vision

• "Grid computing has emerged as an important new
  field, distinguished from conventional
  distributed computing by its focus on large-scale
  resource sharing, innovative applications, and,
  in some cases, high-performance orientation...we
  review the "Grid problem", which we define as
  flexible, secure, coordinated resource sharing
  among dynamic collections of individuals,
  institutions, and resources - what we refer to as
  virtual organizations."
•  
• From "The Anatomy of the Grid Enabling Scalable
  Virtual Organizations" by Foster, Kesselman and
  Tuecke

27
Classical Web
Classical Grid
More computation
28
Grid is metadata based middleware
Astronomy Sky Survey Data Grid
1. Portals and Workbenches
2.Knowledge Resource Management
Bulk Data Analysis
Metadata View
Data View
Catalog Analysis
3.
Standard APIs and Protocols
Concept space
4.Grid Security Caching Replication Backup Schedu
ling
Information Discovery
Metadata delivery
Data Discovery
Data Delivery
5.
Standard Metadata format, Data model, Wire format
Catalog Mediator
6.
Data mediator
Catalog/Image Specific Access
7.
Compute Resources
Catalogs
Data Archives
Derived Collections
29
For example
Annotations of results, workflows and database
entries could be represented by RDF graphs using
controlled vocabularies described in RDF Schema
and OWL Personal notes can be XML documents
annotated with metadata or RDF graphs linked to
results or experimental plans Exporting results
as RDF makes them available to be reasoned
over RDF graphs can be the glue that associates
all the components (literature, notes, code,
databases, intermediate results, sketches,
images, workflows, the person doing the
experiment, the lab they are in, the final paper)
The provenance trails that keep a record of how
a collection of services were orchestrated so
they can be replicated or replayed, or act as
evidence
30
More
Represent the syntactic data types of e-Science
objects using XML Schema data types Represent
domain ontologies for the semantic mediation
between database schema, an applications inputs
and outputs, and workflow work items Represent
domain ontologies and rules for parameters of
machines or algorithms to reason over allowed
configurations Use reasoning over execution
plans, workflows and other combinations of
services to ensure the semantic validity of the
composition Use RDF as a common data model for
merging results drawn from different resources or
instruments Capture the structure of messages
that are exchanged between components
31
And more
At the data/computation layer classification of
computational and data resources, performance
metrics, job control, management of physical and
logical resources At the information layer
schema integration, workflow descriptions,
provenance trail At the knowledge layer problem
solving selection, intelligent portals Governance
of the Grid, for example access rights to
databases, personal profiles and security
groupings Charging infrastructure, computational
economy, support for negotiation e.g. through
auction model
32
Semantic Grid

• There is currently a gap between grid computing
  endeavours and the vision of Grid computing in
  which there is a high degree of easy-to-use and
  seamless automation and in which there are
  flexible collaborations and computations on a
  global scale.
• To support the full richness of the grid
  computing vision we need Semantic Web
  technologies to Grid middleware and applications
  i.e. the Semantic Grid 
• www.semanticgrid.org

33
Semantic Grid
Semantic Web
Richer semantics
Classical Web
Classical Grid
More computation
Source Norman Paton
34
The Semantic Grid

• Evolution of the Grid
• The Semantic Web
• The Semantic Grid
• The story so far
• Semantic Grid Projects
• Closing Remarks

• The Story So Far

35
The Semantic Grid Initiative

• Originally motivated by
• 3 layer model
• Agent-based computing
• In 2001 we aimed to
• promote service-oriented architecture
• bridge Grid and Semantic Web communities
• clarify knowledge layer
• encourage holistic approach

36
Source Keith Jeffery
37
Agent Technology A Canonical View
Agent
Interactions
Organisational relationships
Environment
Sphere of influence
Source Jennings, CACM
38
Research Agenda for theSemantic GridA Future
e-Science Infrastructure

• Technical Report of the National e-Science Centre
• UKeS-2002-02, 2001.

• David De Roure - Distributed Systems, Web
• Nigel Shadbolt - Advanced Knowledge Technologies
• Nick Jennings - Agent Based Computing
• Mark Baker - Grid technologies

39
Aim

• A Research Agenda aiming to move from the
  current state-of-the-art in e-Science
  infrastructure to the future infrastructure that
  is needed to support the full richness of the
  e-Science vision.

40
The report

• Commissioned for UK e-Science Programme
• Draft distributed in July 2001, samizdat
  publication was influential
• Completed in December, and report now split into
  two documents
• The Evolution of the Grid (De Roure, Baker,
  Jennings, Shadbolt)
• The Semantic Grid (De Roure, Jennings, Shadbolt)
• See also Semantic Web and Grid Computing (Goble,
  De Roure)

41
Grid -gt Semantic Web community

• WWW2002
• Semantic Web Track
• Chairs Carole Goble Eric Miller
• 12 refereed papers, 2 panels, 2 workshops, lots
  of posters
• 50 developers day at SW track.

1st International Semantic Web Conference
ISWC Chairs Ian Horrocks Jim Hendler 4
tutorials, 40 refereed papers
42
Keynotes
Ian Foster at WWW2002 Carl Kesselman at
ISWC2002 Plus panel at WWW2002
43
Semantic Grid Panel
WWW2002 Semantic Grid Panel

• What do grid computing and semantic web have in
  common? Where do they differ?
• Does the Grid need the Semantic Web?
• Does the Semantic Web need the Grid?
• Where do you think it's going in 50 years?
• What is the biggest challenge we must address to
  realise the semantic grid?

Grid meets Web
44
(No Transcript)
45
SW -gt Grid Community

• Global Grid Forum 5 in Edinburgh
• Semantic Grid BOF
• Ontologies and the Grid tutorial
• Semantic Web keynote

Dave Carole Nigel
46
Semantic Grid RG Charter

• Goal
• The goal of this RG is to realise the added
  value of Semantic Web technologies for Grid users
  and developers.
• It will provide a forum to track Semantic Web
  community activities and advise the Grid
  community on the application of Semantic Web
  technologies in Grid applications and
  infrastructure, to identify case studies and
  share good practice.

47
Semantic Grid RG Charter

• When the scientist seeks answers to problems
  such as the following, we wish to have the answer
  obtained by automatic linking of databases and
  computational resources by means of metadata,
  ontologies and reasoning over both i.e. Semantic
  Web and Grid technologies

48
Semantic Grid RG Charter

• "Correlate the new molecular structure with the
  existing structural databases what are the
  likely physical properties of the crystal?"
• "Retrieve align 2000nt 5' from every
  serine/threonine kinase in Fabacae expressed
  exclusively in the root cortex whose expression
  increases 5x or more upon infection by Rhizobium
  but is not affected by osmotic or heavy-metal
  stresses is lt40 homologous in the active site
  to kinases known to be involved in cell-cycle
  regulation in any other species"
• How many cows in Texas? And how many will there
  be if we increase land tax? ?

49
Semantic Grid RG Charter

• Projected Tasks
• Track semantic web activities and inform the Grid
  community on what tools and ideas to use now and
  which to watch
• Provide a forum to discuss and share best
  practice in 'semantic grid' projects
• Create links with other RG and WG to both push
  Semantic Grid expertise and to offer a service of
  expertise. For example, participation in the
  proposed working group on scheduling ontology.
• Operate a community web portal
• Encourage engagement between the Grid and
  Semantic Web communities.

50
The Semantic Grid

• Evolution of the Grid
• The Semantic Web
• The Semantic Grid
• The story so far
• Semantic Grid Projects
• Closing Remarks

• Semantic Grid Projects

51
Semantic Grid projects

• We are investigating Semantic Grids in e-Science
  projects
• myGrid
• Comb-e-Chem
• Geodise
• CoAKTinG
• GRIA

Combechem
52
Semantic Grid Aspects

• Comb-e-Chem automation requires
  machine-processable descriptions
• myGrid ontologies in bioinformatics domain,
  plus ontologies for service description
• Geodise knowledge in a problem solving
  environment
• CoAKTinG ontologies to augment collaboration
  and communicate events
• GRIA descriptive and operational metadata,
  description of negotiations

53
Comb-e-Chem Project - Automation
Video
Simulation
Properties
Analysis
StructuresDatabase
Diffractometer
X-Raye-Lab
Propertiese-Lab
Grid
54
myGrid Project - bioinformatics

• Imminent deluge of genomics data
• Highly heterogeneous
• Highly complex and inter-related
• Convergence of data and literature archives
• Database access from the Grid
• Process enactment on the Grid
• Personalisation services
• Metadata services
• Grid Services Ontologies

55
Geodise Project
Geodise will provide grid-based seamless access
to an intelligent knowledge repository, a
state-of-the-art collection of optimisation and
search tools, industrial strength analysis codes,
and distributed computing data resources
56

• CoAKTinG will provide tools to assist scientific
  collaboration by integrating
• intelligent meeting spaces
• ontologically annotated media streams from online
  meetings
• decision rationale and group memory capture
• meeting facilitation
• issue handling, planning and coordination support
• constraint satisfaction
• instant messaging/presence.

http//www.aktors.org/coakting/
57
Industrial applications
58
The Semantic Grid

• Evolution of the Grid
• The Semantic Web
• The Semantic Grid
• The story so far
• Semantic Grid Projects
• Closing Remarks

• Closing remarks

59
The Grid as a killer app for SW?

• Grid apps are a very good example of the type of
  application envisaged for the Semantic Web.
• Grid is a real application the emphasis is on
  deployment and on high performance, and is on a
  large scale and has established communities of
  users.
• The Grid genuinely needs Semantic Web
  technologies.
• It will stress Semantic Web solutions
• It is self-contained, with a well-defined
  community who already work with common tools and
  standards.
• Aspects of the Semantic Web could be applications
  of grid computing, for example in search, data
  mining, translation and multimedia information
  retrieval.

60
Grid and Pervasive computing

• The Grid shares many distributed systems issues
  with pervasive (ubiquitous) computing (e.g.
  service discovery and composition)
• e-Science needs pervasive computing, e.g. the
  smart lab and support for collaboration
• The Equator project investigates the convergence
  of the digital and physical worlds the Grid is
  a digital world!

Technical innovation in physical and digital life
61
Limited DigitalEnvironment
Mainframes
FTPShared InfoStores
Multi User Machines
Conferencingand GroupwareSystems
NetworkedPCS
Growing Presence of the Digital in the Physical
World
Increasingly RichDigital environments
Web andVirtual Worlds
Mobile DevicesWearablesNovel Displays
Grid
Fully Converged Digital and Physical Environment
Seamless Meshing of Digital and Physical
Interaction
62
Grid-based Devicesfor everyday health

• Providing medical information onto the Grid
• Focus on combining medical information with
  motion information to provide context
• Requires timely Grid computation
• Information reported remotely to mobile devices

63
Summary

• Middleware enables interoperable use of
  heterogeneous computer systems
• Grid applications involve a wide range of problem
  driven pioneering and provide challenges in
  Information, Knowledge and Collaboration as well
  as high performance computation
• Semantic Grid enables interoperable use of
  heterogeneous Grid projects!
• Semantic Web technologies should be applied now
  for machine-processable descriptions and future
  semantic interoperability
• Need to
• Track Semantic Web developments e.g. OWL tools
• Investigate enhanced collaboration environments
• Aim is to accelerate the scientific process and
  not just scientific computation this is the
  reward that will motivate SG

64
Credits

• Mark Baker, University of Portsmouth
• Carole Goble, University of Manchester
• Nick Jennings, University of Southampton
• Nigel Shadbolt, University of Southampton
• Many colleagues in Grid and Semantic Web
  communities