Stefan Heinzel

1
Towards a European HPC Infrastructure DEISA
and its contributions

• Stefan Heinzel
• DEISA, Stefan.Heinzel_at_rzg.mpg.de
• IS-ENES - Kick-off-Meeting
• 30th March 2009, Paris

2
European Strategy Forum on Research
Infrastructures
PRACE
ESFRI Report 2006, p. 65
European High-Performance Computing Service
A European strategic approach to
high-performance computing, concentrating the
resources in a limited number of world top-tier
centres in an overall infrastructure connected
with associated national, regional and local
centres, forming a scientific computing network
to utilise the top-level machines.
DEISA
3
Mario Campolargo, OGF23, June 2008
4
Tier0 / Tier1 CentersAre there implications for
the services?
Main difference between T0 and T1 centers
policy and usage models ! T1 centers can
evolve to T0 for strategic/political reasons T0
machines automatically degrade to T1 level by
aging

• T0 Centers
• Leadership-class European systems in competition
  to the leading systems worldwide, cyclically
  renewed
• Governance structure to be provided by European
  organization
• T1 Centers
• Leading national Centers, cyclically renewed,
  optionally
• surpassing the performance of older T0
  machines
• National Governance structure
• The Services have to be the same in T0/T1
• Because of the change of the status of the
  systems, over time
• For user transparency of the different systems
• (Only visible Some services could have
  different flavors for T0 and T1)

5
Vision and Strategy
DEISA The Virtual European HPC Center Operation
- Technology - Applications

Enhancing the existing distributed European HPC
environment (DEISA) to a turnkey operational
infrastructure
Advancing the computational sciences in Europe by
supporting user communities and extreme computing
projectsEnhancing the service provision by
offering a complete variety of options of
interaction with computational resources
Integration of T-1 and T-0 centres The Petascale
Systems need a transparent access from and into
the national data repositories
6
DEISA Partners and Associate Partners
DEISA2 May 1st, 2008 April 30th, 2011
7
DEISA Partners Associate Partners
BSC Barcelona Supercomputing Centre
Spain CINECA Consortio
Interuniversitario per il Calcolo Automatico
Italy CSC Finnish Information
Technology Centre for Science Finland EPCC
University of Edinburgh and CCLRC
UK ECMWF European Centre
for Medium-Range Weather Forecast UK (int) FZJ
Research Centre Juelich Germany H
LRS High Performance Computing Centre
Stuttgart Germany IDRIS Institut du
Développement et des Ressources
France en Informatique Scientifique -
CNRS LRZ Leibniz Rechenzentrum
Munich Germany RZG Rechenzentrum
Garching of the Max Planck Society Germany SARA
Dutch National High Performance Computing
Netherlands CEA-CCRT Centre de Calcul
Recherche et Technologie, CEA France KTH
Kungliga Tekniska Högskolan Sweden CSCS
Swiss National Supercomputing Centre
Switzerland JSCC Joint Supercomputer
Center of the Russian Russia Academy of
Sciences
8
DEISA 2008/2009 Operating the European HPC
Infrastructure
gt1 PetaFlop/s Aggregated peak performance
Most powerful European supercomputers for most
challenging projects
Top-level Europe-wide application enabling
Grand Challenge projects performed on a regular
basis
Community Support
9
Real needs of HPC users of T0/T1
HPC users - T0/T1 - are conservative, standard
access methods are preferred, no interest in
complicated middleware stacks. Comfortable Data
Access among T0/T1/T2 is the key for satisfied
users and for the future European HPC
Infrastructures!

• Global Login
• HPC users prefer a personal Login in each
  system
• Comfortable Data Access
• HPC users need a global, fast and comfortable
  access to their data
• Common Production Environment
• HPC users do not need an identical but an
  equivalent HPC software stack
• Global Help Desk
• HPC users wish one central point of contact and
  as local as possible
• Application Support
• "HPC users need help in scalability and
  adaptation to different architectures"

Community based data repositories are "local"
10
DEISA dedicated high speed network
11
DEISA Global File System(based on MC-GPFS)
IBM P4 ( BlueGene/P)
NEC SX8
IBM P4 ( BlueGene/P)
AIXLL-MC
AIXLL-MC
GridFTP
IBM P5
Cray XT4
AIXLL
UNICOS/lc PBS Pro
UNICOS/lc PBS Pro
Cray XT4
SGI ALTIX
LINUX PBS Pro
AIXLL-MC
AIXLL-MC
IBM P5
LINUX Maui/Slurm
LINUX LL
IBM P6 BlueGene/P
IBM P5 / P6
IBM PPC
12
DEISA operational and system Services

• Pan European User Environment
• Common Production Environment
• Adaptations for new architectures
• Integration of 7 new HPC systems and technologies
  
• in the last 10 months
• Pan-European operational Services
• INCA monitoring
• Help desk
• Operator on Duty
• Maintenance, installation and configuration
  management

12
13
Operational Model
14
DEISA Software Layers
15
Supercomputing Resources
2009
DEISA partners resources Higher than 1 PF
aggregated Peak performance on state-of-the art
supercomputers
Cray XT4, Linux Power5, Power6, AIX / Linux IBM
BlueGene/P, Linux IBM PowerPC, Linux
(MareNostrum) SGI ALTIX 4700 (Itanium2
Montecito), Linux NEC SX8 vector system, Super UX
16
DEISA Extreme Computing Initiative (DECI)

• DECI launched in early 2005 to enhance DEISAs
  impact
• on science and technology
• Identification, enabling, deploying and operation
  of flagship applications
• in selected areas of science and technology
• Multi-national proposals especially encourage
• Proposals reviewed by national evaluation
  committees
• Projects chosen on the basis of innovation
  potential,
• scientific excellence, relevance criteria, and
  national priorities
• Most appropriate supercomputer architecture
  selected for each project
• Mitigation of the rapid performance decay of a
  single national
• supercomputer within its short lifetime cycle
  of typically about 5 years,
• as implied by Moores law

Most powerful and appropriate HPC architectures
in Europe for the most challenging projects

17
DEISA Extreme Computing Initiative
Projects from DECI calls 2005, 2006, 2007, 2008
Involvement of 160 research institutes and
universities from 15 European countries
Austria Finland France Germany
Hungary Italy Netherlands Poland Portugal
Romania Russia Spain Sweden Switzerland UK
with collaborators from
four other continents North America, South
America, Asia, Australia
18
DEISA Extreme Computing Initiative

• DECI call 2005
• 51 proposals, 12 European countries involved,
  co-investigator from US)
• 30 mio cpu-h requested
• 29 proposals accepted, 12 mio cpu-h awarded
  (normalized to IBM P4)
• DECI call 2006
• 41 proposals, 12 European countries involved
• co-investigators from N S America, Asia (US,
  CA, AR, ISRAEL)
• 28 mio cpu-h requested
• 23 proposals accepted, 12 mio cpu-h awarded
  (normalized to IBM P4)
• DECI call 2007
• 63 proposals, 14 European countries involved,
  co-investigators from
• N S America, Asia, Australia
  (US, CA, BR, AR, ISRAEL, AUS)
• 70 mio cpu-h requested
• 45 proposals accepted, 30 mio cpu-h awarded
  (normalized to IBM P4)
• DECI call 2008
• 66 proposals, 15 European countries involved,
  co-investigators from

19
DECI Project POLYRES
Cover story of Nature - May 24, 2007
Curvy membranes make proteins attractive
For almost two decades, physicists have been on
the track of membrane mediated interactions.
Simulations in DEISA have now revealed that curvy
membranes make proteins attractive
Nature 447 (2007), 461-464

• proteins (red) adhere on a membrane
• (blue/yellow) and locally bend it
• this triggers a growing invagination.
• cross-section through an almost
• complete vesicle

B. J. Reynwar et al. Aggregation and
vesiculation of membrane proteins by curvature
mediated interactions, NATURE Vol 44724 May
2007 doi10.1038/nature05840
20
Achievements and Scientific Impact
Brochures can be downloaded from
http//www.deisa.eu/publications/results
21
Evolution of user categories in DEISA
Support of Virtual Communities and EU projects
Single project support DEISA Extreme Computing
Initiative
Early adopters (Joint Research Activities)
FP6 DEISA
FP7 DEISA2
Preparatory phase
22
Virtual Community Support
Fusion energy research
www.efda.org
European Fusion Development Agreement
www.euforia-project.eu
EU Fusion fOR Iter Applications (EUFORIA)
Life Sciences
www.virolab.org
Virtual laboratory for infectious diseases
23
Virtual Community Support
Earth Sciences
www.enes.org
European Network for Earth System Modelling
Astrophysics/Cosmology
http//www.mpa-garching.mpg.de/Virgo/
Virgo Consortium is an international grouping of
scientists carrying out supercomputer
simulations of the formation of galaxies, ...
http//www.sciops.esa.int/projectPLANCK
Planck is a Mission (M3) of ESA's Horizon 2000
Scientific Programme
24
DEISA and IS-ENES Collaboration

• Development of the community specific
  infrastructure flavor in a European HPC
  Infrastructure
• Dedicated User Environment
• For DECI applications as a prototype for the
  upcoming HPC ecosystem
• Prototype for an efficient transparent data
  transfer from/into the specific data repositories

25
LLNL announced the next system for 2011 26 PF
Peak 200K- 800K Cores How many European
applications can scale up to 500K Cores till
2011/2012?
26
Towards a European HPC Infrastructure DEISA
and its contributions

• Stefan Heinzel
• DEISA, Stefan.Heinzel_at_rzg.mpg.de
• IS-ENES - Kick-off-Meeting
• 30th March 2009, Paris

27
DEISA2
National / Local
User community based approach
There are user communities that both rely on
capability and capacity computing and use all
types of compute resources at national, regional
and local level vertically across the resource
pyramid (e.g. the European Fusion
community). Cooperations of European Initiatives
addressing the various levels of the resource
pyramid should be triggered by and focussed on
the needs of such multi-national science
communities. Pilot example is EUFORIA supported
both by DEISA2 and EGEE3.
-gt Support by EU infrastructure projects (DEISA,
EGEE)
28
Supercomputer Hardware Performance Pyramid
Supercomputer Application Enabling Requirements Py
ramid
Capability computing will always need expert
support for application enabling and
optimizations The more resource demanding one
single problem is, the higher are generally the
requirements for application enabling including
enhancing scalability
EU
National
Local
29
JRA in Plasma Physics
DEISA Fusion Simulation Suite application
enabled and optimized for usage on the massively
parallel state-of-the-art DEISA hardware
Simulation Codes TORB / ORB5
GENE, EUTERPE GEM GYGLES High
relevance for the European fusion
community Significant improvements of both the
single processor performance and the scalability
of the codes
ORB5
GENE
After successful enabling, all codes run
efficiently in the range of 512 up to 32,000
processor-cores.
All enabled codes (but the last just finished
one) selected for successful projects in the
DEISA Extreme Computing Initiative
30
TeraGrid
Evolvements of TeraGrid and DEISA
Distributed Terascale Facility, 4 sites
Extended Terascale Facility, 5 sites
TeraGrid, 8 sites
GPFS production
Production state
2001
2002
2003
2004
2005
2006
Production state also with GPFS Expansion to 11
sites
Project Start, 8 sites
Proposal to EU
DEISA
31
Summary

• Evolvement of this European infrastructure
  towards a robust and persistent European HPC
  ecosystem
• Enhancing the existing services, by deploying new
  services including
• support for European Virtual Communities, and by
  cooperating and
• collaborating with new European initiatives,
  especially PRACE
• DEISA2 as the vector for the integration of
  Tier-0 and Tier-1
• systems in Europe
• To provide a lean and reliable turnkey
  operational solution
• for a persistent European HPC ecosystem
• Bridging worldwide HPC projects To facilitate
  the support of international science communities
  with computational needs traversing existing
  political boundaries