PC clusters in KEK

1
PC clusters in KEK

• A.Manabe KEK(Japan)

2
PC clusters in KEK

• Belle (in KEKB) PC clusters
• Neutron Shielding Simulation cluster
• Some other activity on PC cluster

3
Belle PC cluster

• Utilized for experimental data production. raw
  data to reconstructed data.
• 400 CPUs by 3 clusters
• Cooperation with SUN servers for I/O (B computer
  system)
• Number of Users is rather small. (
• Major PCs are 4 CPU SMP machines.Homemade SW
  basf for SMP processing.(old system was by
  28cpu SMP servers)

4
Belle PC cluster (1)

• since 1999
• CPU nodes - DELL PowerEdge 6300(4CPU_at_Pentium III
  Xeon 500MHz9Gdisk) x 36
• Disk nodes(2CPU800GB (Arena) RAID DISK) x 8
• Switch100BaseT Switch 1000BaseSX uplink to B
  computer system
• Installed by physicists, Rack is homemade

5
Belle PC cluster1,2
Cluster 2
Cluster 1
6
Belle PC cluster (2)

• since 2000 winter
• CPU nodes - Compaq Proliant DL360 (2CPU x
  Pentium III 800-900MHz9Gdisk)x40
• Disk nodes(2CPU 1.2TB RAID disk)x4
• Installed by Compaq (HW and SW)(1week for all
  installation)

7
Belle PC cluster(3)

• since 2001 March
• CPU nodes - Compaq ProLiant D580(4CPU_at_Pentium
  III Xeon 700MHz50GBdisk)x60
• Switch100BaseT to each nodes1000BaseSX to B
  computer system
• 5 years lease
• included in the B computer budget.
• Installation service initial and a few/year.

8
Belle PC cluster3
9
Tape Library
1. TCP data transfer SW copy a RAW data file
in tape to files in Disk server. (9MB/s) 2.
Production jobs running in PC nodes
reading/writing from/to the file using
NFS. 3. write back Reconstructed data files
to a tape or HSM system.
3.
SUN WS
1.
SW
2.
PC nodes
PC Disk Server
10
Some numbers

• 1 event processing takes about 6 sec by 1GHz
  Pentium III
• 1 job 1 exp. run 16GB 32files 1020 hours
  by 4CPU(1PC node)
• Job submission is done manually with help of Perl
  scripts and a DB managing exp. run information.

11
Belle PC cluster summary

• Belle PC cluster
• 464 CPU (500-933MHz), 256MB memory/CPU
• 14TB disk (RAID 10TB,local4TB)
• 100BaseT network
• B computer system (data server and general
  users)
• 40 SUN servers (each has a DTF2 tape drive)
• Tape Library 500TB (Sony DTF2)
• 20 NFS/HSM Disk (10TB RAID) servers

HSMHierarchical Storage Management System
12
Simulation farm for the HIPAF beam line design.

• High Intensity Proton Accelerator Facility50GeV
  15microA
• Simulation for Neutron beam line design and
  neutron radiation shielding.
• 50 x (2 CPU_at_1GHz Pentium III)
• NMTC, MCNP on MPI
• Will be Installed in this year

13
Other Activity

• PC farm I/O RD by the comp. center
• HPSS (Linux HPSS client API driver by IBM)
• Storage Area Network (with Fujitsu)
• GRID Activity for ATLAS Regional Center in Japan.
• Gfarm (http//datafarm.apgrid.org/index.en.html)