Computational Science at Edinburgh

1
Computational Science at Edinburgh

• From Excellence to Enterprise
• Dr Arthur Trew
• Director

2
on the shoulders of giants

• for centuries science has relied on experiment
  and theory, but
• theory is best suited to a reductionist approach
• experiment is limited in its range of
  applicability
• there is a need for a complementary approach
  simulation ? study of emergent phenomena

Experiment - Italy 1,500 AD
3
becoming the best
arthur 52.86 HPCx 1.0 Donofrio .4 Eades 1.68
SRIF 2.3 eDIKT 1.0 SUR 2.3 NeSC/eSI 3 GCP 0.2
GSC 0.35 e-Strom 0.6 GridNet

• in 1990 Edinburgh established EPCC as its focus
  for HPC simulation
• following a decade of research in Physics
• in 1992 EPCC set itself a mission to be Europes
  top HPC centre within 5 years
• we succeeded
• today, that success is benefiting Scottish
  research and industry
• in 2001 Edinburgh Glasgow created NeSC to
  extend that vision
• over the past 18 months EPCC NeSC have brought
  in 65M of research grants/contracts
• of which 2.3M from SHEFC for eDIKT
• Edinburgh Glasgow backed this with 3M
• and foundation departments lead major UK Grid
  projects

arthur UoE 3.19 - 1.68 UoG 1.45 (AKs
figures)
4
whats that to me?

• as computer performance improves, the range of
  applications increases

5
linking to data

• but simulation is only part of the story
• linking computer modelling to experiment is vital
  to the interpretation of many large experiments,
  eg LHC
• and with big experiments set to generate
  Pbyte/year data management is critical

• but we have the web, dont we?

6
EPCC overview

• 70 staff, 3M turnover
• possible through the win-win-win model

7
HPCx rationale

• UK academic research is increasingly dependent
  upon high-end compute facilities
• recent technological and Grid advances
  highlighted the need to upgrade UK resources
• HPCx objectives are thus
• aim to deliver optimum service resulting in
  world-leading science
• address the problems involved in scaling to the
  capability levels required

8
What is HPCx?

• Consortium of leading UK organisations committed
  to creating and managing the new resource for the
  next 6 years
• led by University of Edinburgh
• multi-stage project to deliver a world-class
  academic computing resource, the largest in
  Europe
• 54M/100M budget
• Grid-enabled, a key component in the UK e-Science
  programme

9
HPCx Consortium Members
10

• the University of Edinburgh is one of the top 5
  research universities in the UK
• EPCC is the leading computer centre in Europe,
  bridging the gap between academia and industry
• and provides both HPC and novel computing
  solutions to a wide range of problems and users
• long experience of providing national HPC
  services including
• Meiko Computing Surfaces
• Thinking Machines CM200
• Cray T3D/T3E

11
Daresbury Laboratory

• A multi disciplinary research lab with over 500
  people

• Provides large-scale research facilities both for
  UK academic and industrial research communities

• Runs the UKs Collaborative Computational Projects

• Daresbury hosts the HPCx hardware

12

• IBM provides the technology for HPCx
• Long standing involvement in HPC including the
  development of a number of ASCI machines and 4 of
  the top dozen machines in the 21st TOP500 list
• ASCI White Rmax 7.3 TFlop/s
• SP Power3 (6656 Processors) Rmax 7.3 TFlop/s
• xSeries (1920 Processors) Rmax 6.6 TFlop/s
• HPCx (1280 processors) Rmax 3.2 TFlop/s
• IBM has the long term technology road map
  essential to a 6 year project such as HPCx

13
HPCx in Place
14
HPCx Phase 1

• System will be commissioned in three main stages
  Phase 1 covering 2002-2004 consists of
• 40 Regatta-H SMP nodes, 1280 processors
• Peak performance 6.6 TFlop/s, with 3.4 Tflop/s
  sustained Linpack
• currently 16th in the Top500

• 1.28 TB total memory capacity
• Over 50 TB of storage capacity
• Double plane Colony switch with total peak
  bandwidth of 250 MB/s per processor

15
The e-Science Grid
16
HPCx Phases 2 3

• Phase 2 (2004-2006)
• aiming for 6 TFlop/s sustained on Linpack and
  2.5 TFlop/s on sPPM
• O(48) Regatta-H SMP nodes
• interconnect upgraded to Federation switch
• doubling of I/O and storage
• already built a cluster with 8 Regatta-H frames
  and a pre-release Federation switch
• undertaking a phased upgrade during 1H2004
• Phase 3 (2006-2008)
• target of 12 TFlop/s sustained on Linpack
• may be additional nodes or alternative technology

17
HPCx Science Support

• 18 staff in 5 dual-centre functional support teams

18
HPCx Status Usage
19
HPCx Status Application Areas

• currently 28 active consortia, and over 450 users
  
• Life Sciences outreach activity supported by IBM

20
Atomic and Molecular Physics

• The UK Multiphoton, Electron Collision and Bose
  Einstein Condensates (MECBEC) HPC Consortium
• Two flagship projects model two-electron atoms
  (helium) and molecules (hydrogen) exposed to
  intense, ultra-short laser pulses

• Modelling involves the grid solution of
  multi-dimensional, time-dependent partial
  differential equations)

• Requires large amounts of computing power

• Visualisation techniques crucial in extracting
  information

Simulation of Double Ionization of laser-driven
helium performed at Queens University Belfast
21
Environmental Science POLCOMS

• POLCOMS is a multi-disciplinary model developed
  at the Proudman Oceanographic Laboratory

• a 3-D hydrodynamic model integrating coasts and
  oceans using a wide range of associated models

• POLCOMS is a step towards real time modelling of
  coastal zones, enabling better analysis of
  impacts to, and sustainability of, the marine
  environment

A simulation of Chlorophyll density in UK waters
using the POLCOMS model
22
Material Science Crystal

• CRYSTAL computes electronic structure and
  related properties of periodic systems
• Developed jointly by Daresbury and the University
  of Turin
• A Fortran90 and MPI program that performs
  Hartree-Fock, density functional and other
  approximation calculations
• On HPCx, CRYSTAL used to calculate the structure
  of the Crambin molecule, the largest Hartree-Fock
  calculation ever converged (1284 atoms)
• next, the Rusticyanin molecule (6284 atoms)

23
Engineering UKTC

• UK Turbulence Consortium is developing world
  leading turbulence simulation codes using HPCx
• essential that they can easily facilitate the
  scientific usage of the resulting data
• the calculated data are transferred from HPCx to
  a remote site for analysis
• so, the Grid is becoming increasingly important

24
Chemistry RealityGrid

• RealityGrid is a UK collaboration
• which aims to grid-enable the realistic modelling
  and simulation of complex condensed matter
  structures at the meso and nanoscale levels
• HPCx terascaling team has worked on
  parallelisation optmisation of RealityGrid
  codes, such as LB3D
• RealityGrid also uses the Grid for data transfer,
  computational steering, and remote visualisation
• using Globus on HPCx
• TeraGyroid won award at SC2003

25
The Vision

• we want Edinburgh to lead the e-science
  revolution
• to become the European equivalent of one of the
  big US centres, eg San Diego Supercomputer Center