HighPerformance Computing on German Gigabit WANs

1
High-Performance Computingon German Gigabit WANs

• André Merzky, Alexander Reinefeld
• ZIB Berlin
• ISThmus 2000 Research and Development for the
  Information Society,
• Poznan, 11.-13.04.2000

2
Outline

• High-Performance Computing in Germany
• major HPC centers trends
• Wide Area Networking in Germany
• from B-WIN to G-WIN
• Gigabit Projects _at_ ZIB
• GUETE
• META
• TIKSL

3
High-Performance Computing in Germany
4
The Roots The German Konrad Zuse invented the
Computer in 1937

• 1920 Zuse born in Berlin
• 1938 the first mechanical computer Z1 with
  program control in work destroyed at World War
  II

• Z1 has many features of
• a modern computer
• processing unit
• micro sequencer
• separate memory
• floating point arithmetic
• but no conditional branches

5
Germany A Country Without Computer Manufacturers
?

• Suprenum still leaking the wounds of the good
  old days
• Zuse KG taken over by Siemens
• Nixdorf joined Siemens
• ? Siemens, Siemens-Nixdorf, Fujitsu-Siemens
• Parsytec From Supercomputing to Embedded
  Computing
• Custer computing
• hpcLine (Paderborn/Siemens), Parastation
  (Karlsruhe), Atoll (Mannheim/Siemens),

6
Major Supercomputer Sites
Top Site System PEs GFlop/s GFlop/s
peak sustained 9 Dt. Wetterdienst DWD T3E-1200
812 974 671 (30 of Top1) 20 NIC
Jülich T3E-1200 540 648 447
25 Max-Planck-Ges. MPI/PP T3E-600 812
487 355 27 HWW Stuttgart T3E-900
540 486 341 39 ZIB Berlin T3E-900/1200 404
363 253 in 6/2000 LRZ Munich Hitachi
SR8000 1300 TF DKRZ Hamburg significant MPP in
2001 HLRN significant MPP in 2001 J.
Dongarra, H. Meuer TOP 500, 11/99
7
Konrad-Zuse-Zentrum für Informationstechnik
Berlin (ZIB)
8
ZIB Systems

• SGI Cray T3E, 384 (408) PEs
• 256 Alpha EV5.6 450 MHz
• 128 Alpha EV5.6 600 MHz
• 67 GB memory
• UNICOS/Mk
• F90, C, C, HPF
• MPI, PVM, shmem
• 403 GFlop/s peak

• Cray J932/16
• 16 CPU J90, 100 MHz
• 8 GB shared memory
• UNICOS
• F90, C, C
• MPI, PVM, shmem
• 3 GFlop/s peak

9
Mass Storage _at_ ZIB

• 2 StorageTek Robots for max. 12,0000 cassettes
• cassettes drives
• 1999 3000 Redwood cassettes, 8 x STK SD3 ACS
  drives capacity 50 GBbandwidth 15 MB/s
  mount/dismount 45/45 sØ positioning time 53
  s
• 2000 700 STK 9840 Eagle cassettes, 4 x STK
  9310 ACS drivescapacity 20 GBbandwidth 10
  MB/s mount/dismount 13/11 sØ positioning
  time 11 s
• 2001 x TC60 cassettes (gt 60 GB) gt10
  drives
• migration software DMF (SGI/Cray)
• NetBackup (Veritas)

10
NVV - Norddeutscher Vektorrechnerverbund
Berlin, Niedersachsen, Schleswig-Holstein Goal
co-ordinated investement and co-operative use of
HPC
Konrad-Zuse-Zentrum für Informationstechnik
Berlin CRAY T3E-900 / 1200, 386 nodes CRAY J90,
16 nodes (3 Gflop)
Universität Hannover, Niedersachsen Fujitsu VPP
300, 4 CPUs, 8 GB (8 Gflop peak) CRAY T3E-900 AC
20 (18 Gflop)
Universität Kiel, Schleswig-Holstein CRAY T90, 4
CPUs, 1 GByte (7 Gflop peak) CRAY SV1, 20 CPUs
J90se (4 Gflop peak) Cray T3E-1200 AC 18
11
HLRN High-Performance Computer North

• consortium of 6 North-German states
• Berlin
• Bremen
• Hamburg
• Mecklenburg-Vorpommern
• Niedersachsen
• Schleswig-Holstein
• proposal for establishing a multi teraflops
  MPP-cluster Berlin/Hannover
• HLRN is an extension of the NVV

12
The German Gigabit Research Network(G-WIN)
13
DFN Verein Verein zur Förderung eines Deutschen
Forschungsnetzes e.V

• established 1984 by 11 research institutes
  universities
• BMBF funded
• 400 members (2000)
• GOAL To establish and operate a German Research
  Network (WiN, B-WiN, G-WiN, )

14
Traffic Volume
increase factor 2.2 per year
15

• starting with B-WIN 34, 155 Mb
• provider German Telekom AG
• level 2 nodes 2 links
• level 1 nodes gt 2 links

G-WIN (start topology)
Rostock
Kiel
Hamburg
Oldenburg
Braunschweig
Berlin
Magdeburg
Hannover
Bielefeld
Essen
Göttingen
Leipzig
Dresden
St. Augustin
Ilmenau
Marburg
Aachen
Würzburg
Frankfurt
Erlangen
(Darmstadt)
Heidelberg
Regensburg
Karlsruhe
Kaiserslautern
Stuttgart
Garching
Augsburg
(Freiburg)
16
Gigabit Network Projects _at_ ZIB
17
DFNGigabitProjects
18
BRAIN Berlin Research Area Information Network
(managed by ZIB)
19
Overview Gigabit Süd Projects

• \item \bfKoMaB -- Coordination and
  Management of the Gigabit-Testbed in Berlin (ZIB)
• \item \bfG\"UTE -- Evaluation of
  Gigabit-Communication Technologies (ZIB, TU
  Berlin (EANTC))
• \item \bfDIANA -- Distance Learning
  in the Gigabit-Testbed (HU Berlin, TU M\"unchen)
• \item \bfFIGARO -- Distance Eartch
  Surveyance Automatic Surface Reconstruction (TU
  Berlin, DLR (Berlin), LMU M\"unchen)
• \item \bfGiga-Media --
  Gigabit-Media-Services for Cooperative Post
  Production for Film and Audio (HHI (Berlin), GMD
  Fokus (Berlin), DAS
• WERK (M\"unchen), Institut f\"ur
  Rundfunktechnik
• M\"unchen)
• \item \bfMeta -- Metacomputing with
  Coupled Supercomputers
• GAMESS-UK (RZG Garching/MPG, ZIB,
  LRZ M\"unchen,
• Universit\"at Stuttgart, Institutes
  of the MPG)
• \item \bfTTC -- Tele Teacher Coaching
  (Training and
• Education in Sports) (TU Berlin, TU
• M\"unchen, Universit\"at
  Erlangen-N\"urnberg,

20
GUETE
TU ltgt HU-Adlershof
21
Distributed Quantum Chemistry

• Gigabit Testbed Süd/Berlin, Project META
• GAMESS/UK with layered message passing TCGMSG -
  MPI - PACX
• Running on T3E _at_ ZIB and MPG

22
META
TCGMSG theoretical chemistry group messag
passing toolkit
23
MetaGAMESS Work / Data Flow
Setup (atoms, basis, ...)
Setup (atoms, basis, ...)
Integral batch, Fock matrix elements
Global Op Fock matrix
Eigen solver
Eigen solver
Integral derivatives --gt gradient
CRAY T3E _at_ ZIB
CRAY T3E _at_ RZG
24
MetaGAMESS SCF Scaling

• Morphine, 6-31G(d,p), BLYP
• 410 basis functions
• PACX
• local partitions
• FDDI interface

25
TIKSL