Delivering Grid Interoperability

1
Delivering Grid Interoperability

• Dr Alistair Dunlop, Project Manager, OMII-Europe
• a.dunlop_at_ecs.soton.ac.uk

2
Outline

• Macro view What is the project trying to
  achieve? (4)
• The project organisation and structure (3)
• The role of standards in grid interoperability
  (3)
• Micro view Standardised Components from
  OMII-Europe (11)
• Putting OMII-Europe components together to
  deliver real grid interoperability (5)
• Summary (3)

3
The big picture

• A definition of e-Science
• e-Science is about global collaboration in key
  areas of science and the next generation
  infrastructure that will enable it
• Next generation infrastructures represented by
  Grids
• Foundation for seamless and secure collaborations
  over networks
• National Grid infrastructures UK NGI, D-Grid,
  others
• New science will require the combined resources
  of all existing infrastructures

4
Problem is e-Infrastructure Islands

• E.g., DEISA Grid (Supercomputing/HPC community)
• Non WS-based UNICORE 5 Proprietary jobs
  (AJO/UPL)
• No Virtual Organization Membership (VOMS), Full
  X.509
• Suitable for parallel scientific jobs (MPI, many
  interactions)
• E.g., EGEE Grid (mainly HEP community others)
• Non WS-based gLite Proprietary jobs (JDL)
• Proxy-based X.509 security, but proprietary VOMS
  support
• Suitable for embarrassingly parallel jobs (few
  interactions)
• Scientists cannot use one middleware to access
  both (Little adoption of standards)

5
OMII-Europe Vision Interoperability Highway
End-users via clients portals
GOAL Transparencyof Grids for end-users
Emerging OpenStandards
Interoperability highwaybased on open standards
others
Grid Middlewares
others
Grid Resources
6
OMII-Europe in context

• A guide to flagship e-infrastructure projects...
• Connectivity (Geant)
• Cluster grids (EGEE)
• Supercomputer Grid (DEISA)
• Middleware (OMII-Europe)
• Software that enables you to use and easily
  access these above distributed EGEE and DEISA
  distributed grid infrastructures
• Together these elements form the
  e-infrastructures that create Global Virtual
  Research Communities...

7
The OMII-Europe Project

• OMII-Europe stands for
• Open Middleware Infrastructure Institute for
  Europe
• It is an EU-funded project FP6, RI
• It has an initial duration of 2 years
• May 2006 -gt April 2008
• It has been granted a contribution of 8M
• It involves 16 partners
• 8 EU
• 4 USA
• 4 China

8
Partners
9
Project Structure and Effort Allocation

• Networking activities
• Management, Outreach, Training
• 8 Person Effort
• Service Activities
• Repository, QA, Support
• 25 Person Effort
• Joint Research Activities
• Re-engineering, new services, integration,
  benchmarking
• 67 Person Effort

10
OMII-Europe Standards-based approach to Grid
Interoperability

• Adapters-based
• The ability of Grid middleware to interact via
  adapters that translate the specific design
  aspects from one domain to another
• Standard-based
• the native ability of Grid middleware to interact
  directly via well-defined interfaces and common
  open standards

definition inspired by OGF GIN CG
11
Participation in Middleware Standardisation

• Most project participants involved as
  member/observer in many OGF WG
• 11 project participant hold senior positions in
• OGSA DAIS WG (Database Access and Integration
  Services)
• OGSA RUS WG (Resource Usage Server)
• OGSA BES WG (Basic Execution Service)
• OGSA JSDL WG (Job Submission Description
  Language)
• GIN CG (Grid Interoperability Now)
• OGSA-AuthZ-WG (Authorization)
• GLUE WG
• GFSG WG (Grid File System)
• RM WG (Reference Model)
• OGSA Naming WG
• Technical Standards Committee
• GSA RG (Grid Scheduling Architecture)
• GRAAP WG (Grid Research Agreement Allocation
  Protocol)
• OGSA BYTE IO WG
• OGSA D WG (Data)
• OGSA DMI WG (Data Movement Interface)

12
The Virtuous Cycle Technology transfer with
Grid projects and standards organisations
Standards Compliance Testing and QA
JRA2
New Components
Standards Implementation
Components
JRA1
IN
Globus
Benchmarking
Repository
OUT
OMII-UK
Components
CROWN
Supported Components on Eval. Infrastructure
Integrated Components
13
Standards Components from OMII-Europe

• Initial focus on providing common interfaces and
  integration of major Grid software
  infrastructures
• Common interoperable services
• Database Access
• Virtual Organisation Management
• Accounting
• Job Submission and Job Monitoring
• Information modelling
• Infrastructure integration
• Initial gLite/UNICORE/Globus interoperability
• Interoperable security framework
• Access these infrastructure services through a
  portal

14
Job Submission

• Unify Job Submisson and Monitoring interface
• Adoption of emerging OGSA-BES and JSDL standards
• Alpha BES and JSDL implementations for
• UNICORE 6, gLite 3.1, Globus 4, OMII-UK,
  CROWNgrid
• Interoperability demonstrated through use of a
  BES compliant meta-scheduler

15
VO Management

• To provide a common Virtual Organisation (VO)
  management solution across middleware
  distributions
• Extend VOMS Interface to support emerging AuthZ
  standard
• compliance with SAML Authorisation model
• Extension, not a replacement interface
• Public release of VOMS integrated with UNICORE

16
Accounting

• Unify accounting information across middleware
  distributions
• Provide standardized interfaces for accessing
  that information
• Information standard
• Usage Record Format (URF)
• Service interface standard
• Resource Usage Service (OGSA-RUS)
• Alpha versions RUS
• gLite (DGAS)
• Globus (SGAS)
• UNICORE

17
Data Access

• Port OGSA-DAI 3.0 from Globus to other middleware
  distributions available throughout Europe and
  China
• UNICORE
• gLite
• CROWN

18
Information Modelling

• There is a lack of a common description of Grid
  resources suitable for discovery, monitoring and
  scheduling
• Many descriptions exist
• e.g. GLUE Schema, NorduGrid Schema
• Working on the definition of next-generation GLUE
  Information Model in the context of OGF GLUE WG
  and its implementation

19
Portal

• Deliver tools for developing Grid portals and
  support for key Web and Grid standards and
  technologies
• Objectives
• Develop gateway to OMII Evaluation Infrastructure
• Develop tools for portal and grid software
  training
• Explore new approaches for grid portal
  development

20
Repository of Open-Source Software

• Make available software reengineered within
  OMII-Europe and contributed by third parties
• Single services/tools complete distributions
• Provide an interface to select software from the
  repository based on user requirements
• By capability/standards/provider/
• Support the upload, download and installation of
  the software
• Document platform portability pre-requisites
• Verify the software through compliance metrics
  tests

21
Behind the Repository

• Leverage existing infrastructure projects
• ETICS
• Capture build test configuration data for
  repeatability
• NMI Build Test Framework
• Manage cross-platform environment for build
  tests
• Condor
• Underlying execution infrastructure
• Provides reports to be displayed within the
  portal
• Builds Pre-requisites platforms
• Testing Conformance Interoperability

22
Tests For Standards Conformance

• Job Submission and Job Monitoring
• Job Submission Description Language (JSDL)
• Basic Execution Service (BES)
• Accounting
• Usage Record (UR)
• Resource Usage Service (RUS)
• Database Access
• WS-DAI, WS-DAIX, WS-DAIR (OGSA-DAI)
• Virtual Organisation Management
• Move towards SAML2

23
Standards status March 2008

• Accounting
• SGAS/DGAS/UNICORE-RUS
• OGSA-Resource Usage Service (RUS OGF draft
  specification) - implies
• Usage Record Format (UR) (OGF recommendation
  status awaiting implementation)
• Job Submission and Job monitoring
• CREAM-BES, GLOBUS-BES, UNICORE-BES
• OGSA-Basic Execution Service (BES) version 1.0
  (OGF final specification) implies
• Job Submission Description Language (JSDL)
  version 1.0 (OGF final specification)
• Database Access
• OGSA-DAI
• WS-DAIX and WS-DAIR are being implemented in
  OGSA-DAI (both currently OGF candidate standards
  at 1.0 awaiting implementation)
• Virtual Organisation Membership Service
• VOMS implements the following
• Security Assertion Markup Language (SAML) Version
  2.0 OASIS (March 05)
• Extensible Access Control Markup Language (XACML)
  version 2.0 OASIS (February 05)
• Information modelling
• GLUE 2.0
• standard is currently draft, expected to go out
  for public comments by the end of April 2008,
  final version by September 2008

24
Using OMII-Europe components to deliver grid
interoperability WISDOM scenario

• Wide In Silicio Docking On Malaria (WISDOM)
  projects
• Developing new drugs for neglected and emerging
  diseases with a particular focus on malaria
• Accelerated Research Development for emerging
  and neglected diseases
• Massive computation reduces RD costs

25
Dr. Nicolas Jacq 16
26
WISDOM in the context of DEISA EGEE

• WISDOM uses EGEE for large scale in-silicio
  docking
• Comp. method for prediction of whether one
  molecule will bind to another
• Using AutoDock and FlexX software provided via
  gLite in EGEE (FlexX licensed software on EGEE
  with 6k licenses)
• Output is a list of best chemical compounds
  (potential drugs NOT final solution)
• DEISA to find best compound from potential drugs
  list
• Fast molecular dynamics computations
• Using highly scalable AMBER (Assisted Model
  Building with Energy Refinement) in DEISA
• Goal Accelerate drug discovery using EGEE DEISA

27
Interoperability Scenario WISDOM
SAML- based Attribute Authority (AA) VOMS gets
central role middleware independent
28
Further Interoperability Scenarios EU - IndiaGrid
Work in progress
29
What can you do Now and Later

• Now
• Most products at Beta stage becoming publicly
  available
• They provide basic interoperability of multiple
  grid middleware systems focusing on job execution
• Available to early adopters working with
  OMII-Europe partners
• Spring 2008 (end of current project)
• Further security integration work between
  different middleware platforms (SAML-VOMS, TLS
  (Transport level security))
• Completed QAd services and demonstrated
  end-to-end solutions
• Availability of GLUE 2 information model service
  implementations

30
Caveat...

• So you can use EGEE and DEISA seamlessly? Not
  quite...
• OMII-Europe has provided much of the required
  technical interoperability, but
• Needs EGEE and DEISA sites to deploy either the
  latest versions of the grid middleware, or to
  deploy the OMII-Europe services available from
  the repository
• AND...
• Currently need to apply for resources
  independently. Intention is that EGI will do this
  in the future

31
Summary (1/2)

• OMII-Europe is a 24 Month EU funded project with
  16 partners to establish grid infrastructure
  interoperability through implementing a set of
  agreed open standards on all middleware platforms
  
• OMII-Europe is implementing a number of
  components that will allow identically specified
  jobs to be run, managed and migrated to different
  middleware platforms
• Initial versions of BES, VOMS/SAML and security
  service have already enabled UNICORE and gLite
  managed resources to be used by the same job
• Users can try interoperability on the OMII-Europe
  evaluation infrastructure, or obtain services for
  installation on their own resources from the
  OMII-Europe repository

32
Summary (2/2)

• We anticipate OMII-Europe services to be
  integrated into standard middleware distributions
  as well as deployed on large scale
  e-infrastructures such as EGEE and DEISA
• OMII-Europe requested continuing funding in the
  September EU call to support the existing
  services and provide further services in the
  areas of data and Grid management
• We are interested in working with projects that
  have real grid interoperability issues to ensure
  our solutions match your needs.

33
Further Information

• General information http//omii-europe.org
• Repository http//repository.omii-europe.org
• Support http//support.omii-europe.org

• With special thanks to the following project
  members for diagrams and slides
• Morris Riedel
• Stephen Brewer