HPC Project Case study,

1

• HPC Project Case study,
• University of Bristol
• Robin Stone AUE Chairman

2
Initial need

• Computer Centre driven by Mathematics and Science
  Departments
• Facility then would be used for many other
  applications
• Lots of potential for cash generation from
  outside users

3
Concept

• 18 interconnected racks
• Housing linked multiple processors, and bulk data
  storage devices
• High processor density per rack
• Redundant support system for cooling and power

4
Conventional Cooling Approach

• Raised floor and plenum ceiling void
• Central cooling plant, feeding into floor void
• Air released into room with floor grilles
• Air recovered though ceiling grilles
• Limitation maximum heat density of around 8-10
  KW per rack
• Requires either vertical air flow in the rack or
  strong pull through

5
Alternative cooling approaches

• New rack processors, contain 4 per slot
• When filled, rack loadings can reach 20KW!
• No vertical air paths at all
• APC solution
• In-rack chilled water coolers
• Enclosed Hot aisle arrangement
• Horizontal Air distribution

6
Alternative cooling approaches

• Trox System approach
• Uses Co2 as a refrigerant gas
• Direct expansion cooling unit in front of the
  rack
• Very early days
• Some concerns regarding access to equipment

7
APC System

• The system of choice
• Better track record
• Chilled water systems well tried
• Cooling capacity rating up to 25KW per rack
• New cooling unit half rack wide, compared with
  previous unit, which was 3 racks wide
• System redundancy at several levels
• Rack management system
• Rack UPS options also available

8
Some lessons learned

• APC offer top or bottom entry for cooling
• Use bottom entry!
• Rack system quite expensive, but also offers top
  cable management options
• They are still learning on systems support issues

9
Location

• Off site, two options
• On site, best option, Physics Tank room
• Strip-out costs 400k!
• Special problems not being at ground level
• Adjacent flat roof available for plant
• Calculated maximum electrical load 1.75 MVA,
  requiring a dedicated HV supply

10
Other interesting facts on HPC

• UPS support limited to core only, but you still
  need power conditioners for the rest
• Standby generation not needed, but this limits
  the use of the room to HPC only
• Rack loads can ramp up quickly from initial
  install, so allow plenty of space/infrastructure
• Initial install 18 racks, but with fallow field,
  potential for 48.
• High constant energy use- possible ccp
  candidate, or part of a trigen larger solution
• Has a 100 fallow field
• HPC photos

11
Physics Building
12
Main Machine Room
13
Main Machine Room Fallow Field
14
Hot Aisle
15
Hot Aisle Side to Side Airflow
16
Fire Media - NOVEC
17
In Line fans
18
In Line Fans
19
Communications Racks
20
Roof Top Plant
21
Roof Top Sub-Station 2 MVACast Resin
22
Roof Top Condensers
23
Roof Top Condensers
24
Questions