The EGEE project and the future of European grids

1
The EGEE project and the future of European grids

• Bob Jones
• EGEE-II Project Director
• CERN

2
How e-Infrastructrures help e-Science

• e-Infrastructures provide easier access for
• Small research groups
• Scientists from many different fields
• Remote and still developing countries
• To new technologies
• Produce and store massive amounts of data
• Transparent access to millions of files across
  different administrative domains
• Low cost access to resources
• Mobilise large amounts of CPU storage on short
  notice (PC clusters)?
• High-end facilities (supercomputers)?
• And help to find new ways to collaborate
• Develops applications using distributedcomplex
  workflows
• Eases distributed collaborations
• Provides new ways of community building
• Gives easier access to higher education

3
EGEE
Flagship grid infrastructure project co-funded
by the European Commission Now in 2nd phase with
91 partners in 32 countries

• Main Objectives
• Operate a large-scale, production quality grid
  infrastructure for e-Science
• Attract new resources and users from industry as
  wellas sciences

4
EGEE What do we deliver?

• Infrastructure operation
• Currently includes 250 sites across 45 countries
• Continuous monitoring of grid services
  automated site configuration/management
• Support many Virtual Organisations from diverse
  research disciplines
• Middleware
• Production quality middleware distributed under
  business friendly open source licence
• Implements a service-oriented architecture that
  virtualises resources
• Adheres to recommendations on web service
  inter-operability and evolving towards emerging
  standards
• User Support - Managed process from first contact
  through to production usage
• Training
• Expertise in grid-enabling applications
• Online helpdesk
• Networking events (User Forum, Conferences etc.)?

5
240 sites 45 countries 41,000 CPUs 5
PetaBytes gt5000 users gt100 VOs gt100,000 jobs/day
Archeology Astronomy Astrophysics Civil
Protection Comp. Chemistry Earth
Sciences Finance Fusion Geophysics High Energy
Physics Life Sciences Multimedia Material
Sciences
32
6
Types of applications

• Simulation
• LHC Monte Carlo simulations Fusion WISDOM
• Jobs needing significant processing power Large
  number of independent jobs limited input data
  significant output data
• Bulk Processing
• HEP Processing of satellite data
• Distributed input data Large amount of input and
  output data Job management (WMS) Metadata
  services complex data structures
• Parallel Jobs
• Climate models, computational chemistry
• Large number of independent but communicating
  jobs Need for simultaneous access to large
  number of CPUs MPI libraries
• Short-response delays
• Prototyping new applications grid Monitoring
  grid Interactivity
• Limited input output data processing needs but
  fast response and quality of service
• Workflow
• Medical imaging flood analysis
• Complex analysis algorithms complex dependencies
  between jobs
• Commercial Applications
• Non-open source software Geocluster (seismic
  platform) FlexX (molecular docking) Matlab,
  Mathematics Idl,
• License server associated to an application
  deployment model

7
EGEE User Forum 2007-2008
Co-located with OGF20 900 attendees50 booths
user forum30 sessions100 presentations20
demos60 posters
Next 22 to 26 September 2008 in Istanbul -
Turkey
8
Registered Collaborating Projects
25 projects have registered as of September 2007
web page
9
Collaborating infrastructures
10
EGEE-II to EGEE-III

• EGEE-III proposal currently under negotiation
  with European Commission
• Key objectives
• Expand/optimise existing EGEE infrastructure,
  include more resources and user communities
• Prepare migration from a project-based model to a
  sustainable federated infrastructure based on
  National Grid Initiatives
• 2 year period spring 2008 to spring 2010
• No gap between EGEE-II and EGEE-III
• Similar consortium
• Now structured on a national basis (National Grid
  Initiatives/Joint Research Units)?

11
European Grid Initiative

• Need to prepare permanent, common Grid
  infrastructure
• Ensure the long-term sustainability of the
  European e-Infrastructure independent of short
  project funding cycles
• Coordinate the integration and interaction
  between National Grid Infrastructures (NGIs)?
• Operate the production Grid infrastructure on a
  European level for a wide range of scientific
  disciplines

Must be no gap in the support of the production
grid
12

• EGI Design Study proposal approved to the
  European Commission (started 1st September07)?
• Supported by 30 National Grid Initiatives
  (NGIs)?
• 2 year project to prepare the setup and operation
  of a new organizational model for a sustainable
  pan-European grid infrastructure
• Federated model bringing together NGIs to build a
  European organisation
• Well defined, complimentary responsibilities
  between NGIs and EGI

http//www.eu-egi.org
13
Characteristics of NGIs

• Each NGI
• recognized national body with a single
  point-of-contact
• mobilise national funding and resources
• operate the national e-Infrastructure
• support user communities (application
  independent, and open to new user communities and
  resource providers)?
• contribute and adhere to international standards
  and policies
• Responsibilities between NGIs and EGI are clearly
  
• separated and complementary

14
37 European NGIs Asia, US, Latin America
PRACE OGF-Europe
15
Upcoming Deliverables Milestones

• Dec 2007 D2.1 EGI consolidated requirements
  and use cases
• Feb 2008 D4.2 Options analysis of different
  legal structures
• Mar 2008 D5.1/5.2 Draft definition and
  convention of EGI organization
• ? EGI Workshop List of EGI functions and
  working model
• Apr 2008 D4.3 Guidelines for NGIs
• June 2008 M4.1 EGI Blueprint publication

16
Summary

• Grids are all about sharing they are a means of
  working with groups around the world
• Today we have a window of opportunity to move
  grids from research prototypes to permanent
  production systems (as networks did a few years
  ago)?
• Interoperability is key to providing the level of
  support required for our user communities
• EGEE operates the worlds largest
  multi-disciplinary grid infrastructure for
  scientific research
• In constant and significant production use
• A third phase of EGEE is under preparation
• Need to prepare the long-term
• EGEE, collaborating projects, national grid
  initiatives and user communities are working to
  define a model for a sustainable grid
  infrastructure that is independent of short
  project cycles

www.eu-egee.org