Power and Cooling of HPC Data Centers Requirements

1
Power and Cooling of HPC Data Centers
Requirements
Roger A Panton Avetec Executive Director
DICE rpanton_at_avetec.org
2
Background and Objective

• Avetec is under contract to evaluate power and
  cooling requirements for HPC data centers
• To survey power and cooling constraints and
  solutions, both current and future
• From representative HPC data centers
• Vendors including HPC systems, equipment and
  facilities
• Avetec contracted with IDC to conduct a survey
• Current power and cooling situation
• Planning in place to address requirements
• Forecasted solutions in the next three to five
  years

3
Survey Sample and Methodology

• Survey includes 41 respondents
• 28 HPC data centers
• 13 vendors of products and services
• Response rate was approximately 20
• Respondents were from US, Europe and Asia
• HPC data centers were selected from the Top500
  list
• Centers selected fell between number 50 and 250
• Interviews were conducted by phone or in person
• Respondents had the option to complete the survey
  on their own

4
Initial General Findings

• HPC data centers averages
• Available floor space about 26,000 ft2
• Used floor space about 17,000 ft2
• Cooling capacity 22.7 million BTUs or 1,839 tons
• Annual power consumption 6.356 MW
• HPC data centers costs
• Annual power cost is 2.9 million or 456 per KW
• Ten sites provided the percentage of their budget
  spent on poweraverage was 23
• Two-thirds of the sites had budget for power and
  cooling upgrades
• Average amount budgeted is 6.87 million

5
Initial Key Findings

• Study revealed wealth of information
• Key findings will be summarized in the following
  four areas
• Current Situation
• Challenges and Expansion Constraints
• Current Approaches
• Future Solutions and Technologies

6
Current Situation

• Over 96 of the centers consider green design
  important
• Majority of sites expect power and cooling to
  impact future HPC center planning
• Majority of respondents have studied or
  implemented greener operations
• Most centers have used software models to analyze
  heat flow and/or power consumption
• Approximately half of the centers paid for power
  and cooling out of their budgets

7
Challenges and Expansion Constraints

• Majority of centers are starting to consider
  power and cooling efficiency equal to or more
  important than HPC computing performance
• Power and cooling issues are becoming the biggest
  barriers to expansion and upgrades
• HPC vendors are starting to see power and cooling
  as a brake on performance

8
Current Approaches

• Power and cooling costs are becoming a key factor
  in upgrade decisions
• Majority of centers have accomplished an air flow
  analysis to improve air cooling efficiency
• Use of chilled water for cooling is increasing
• The power and cooling issues are being discussed
  across the HPC community, i.e. data center, HPC
  systems vendors, and processor vendors

9
Future Solutions and Technologies

• Approximately two-thirds of centers plan to
  expand or build new data centers
• About half of the data centers have or are
  planning to distribute HPC resources
• Liquid computing is being considered as an
  alternative
• HPC centers and vendors differ sharply on the
  likelihood of any game-changing cooling
  technologies emerging in the next 3-5 years

10
Viewpoint

• Current status
• June 2008 marked a new milestone the first Pflop
  HPC system made the Top500
• Top500 lists the sum of the top 500 systems
• 1993 Sum of top 500 equaled 1 ½ Tflops
• 2004 Sum of top 500 equaled 1 Pflop
• If current growth is maintained, sum will be 100
  Pflops by 2012
• The balance between facilities and infrastructure
  to accommodate new systems does not exist
• This imbalance leads to a policy question
• What should the supercomputer community response
  be to restore the balance?

11
Policy Discussion

• Should the community take a proactive position
  through collaborative discussions and then
  recommend a set of Public Policies?
• To start the discussions should
• The Federal Government establish a timeframe and
  fund the following research areas
• Invest to maintain the current performance growth
  in HPC?
• Invest in new cooling technologies to improve
  efficiencies?
• Invest in low power higher performance
  processor(s)?
• Invest in new material research for chips?
• HPC data centers need to become more accountable
  for power and cooling consumption

12
Join Us to Learn Other Findings

• Findings were extensive and cannot be fully
  covered today, such as
• What alternative approaches are you exploring to
  meet future power cooling requirements?
• Describe any new power cooling solutions expect
  to implement in next 2-3 years.
• What special cooling methods do you expect to use
  in the future?
• Future HPC system acquisition power cooling
  requirements.
• And more!
• Final study results will be unveiled at DICE
  Alliance 2009 on May 20 along with a panel
  discussion with some of the study participants

13
Want More Information?

• We Invite You to Attend
• DICE Alliance 2009
• May 18-20
• Wittenberg University
• Barbara Kuss Science Center
• Springfield, Ohio
• Register at
• www.diceprogram.org

14
Other DICE Alliance Topics

• Monday May 18th 600-800 Opening Reception
• Tuesday May 19th
• Keynote Be Careful What You Wish For (Jay
  Boisseau, PhD)
• Americas Need for HPC (Retired, Congressman Dave
  Hobson)
• Panel Power Cooling Efficiency in Data Centers
  (Ed Wahl, lead)
• Panel Public Policy in HPC (Charlie Hayes, lead)
• Wednesday May 20th
• Multicore Architecture Panacea or Nightmare?
  (Earl Joseph, PhD)
• Panel The Integration of Scheduling HPC Resource
  Management Data Lifecycle Management Tools (Jay
  Blair)
• Panel Power Cooling Trends and Technology
  (Earl Joseph, PhD)