Improving Data Center Efficiency

1
Improving Data Center Efficiency

• Andrew Kutz

2
Improving Data Center Efficiency

• Thesis
• The energy efficiency of todays information
  technology (IT) infrastructure is not doubling
  every two years along with performance, leading
  to very high performing, very inefficient data
  centers. The inefficiencies in energy
  consumption must be mitigated so that efficiency
  will catch up with performance.

3
Improving Data Center Efficiency

• Agenda
• Power in the data center today
• Areas for improvement
• Recommendations

4
Improving Data Center Efficiency

• Agenda
• Power in the data center today
• Data center power distribution
• Data center power consumption
• The result
• Areas for improvement
• Recommendations

5
Power in the data center today

• Data center power distribution

6
Power in the data center today

• Data center power consumption
• The breakdown leaves room for plenty of areas of
  improvement.

Data center
40
30
30
IT equipment
Power delivery
Cooling equipment
IT equipment
60
20
20
Compute servers
Storage servers
Comm. misc.
Server components
35
15
20
15
15
CPUs
CPU VR loss
Source Dell, APC, SUN
Memory
HDDs and misc.
PSU loss
7
Power in the data center today

• The result

Source Christian Belady
8
Improving Data Center Efficiency

• Agenda
• Power in the data center today
• Areas for improvement
• Processors
• Power supplies
• Power delivery
• Cooling
• Recommendations

9
Areas For Improvement

• Processors Last gen processors were power
  hungry beasts!
• A first-generation AMD Opteron (856) at 3.0GHz
  used 92.6W
• According to AMD, processors can use between
  50-60 of rated power when in an idle state.
• That means the same processor was consuming 46.3-
  55.56W of energy when doing nothing.

10
Areas For Improvement

• Processors Multi-core CPUs
• Multiple cores on a single chip and processor
  development helps to reduce energy consumption
• An Intel Xeon 3.80 GHz used110W
• An Intel Xeon E5345 2.33 GHz Quad-Core uses 80W.
• 2.45 times the processing power at .72 times the
  energy consumption!

11
Areas For Improvement

• Processors Power stepping CPUs
• CPUs can step down into reduced performance modes
  by adjusting frequency and voltage in
  synchronization with load.

Source AMD
12
Areas For Improvement

• Power supplies Todays problems
• The typical efficiency of todays power supply is
  anywhere between 75-80.
• 1000 servers _at_ 400W _at_ 75 efficiency means 100kW
  is being radiated into heat that cooling systems
  must remove.
• 100kW equals 341,400 BTU.
• At 0.10 USD/ kWh it costs 52,317.07 USDover a 4
  year life cycle to remove the extra heat.

13
Areas For Improvement

• Power supplies Are tomorrows opportunities
• Power supplies being developed with digital
  feedback control can maintain 90-95 efficiency
  over a wide range.
• 1000 servers _at_ 400 W _at_ 95 efficiency means only
  20kW, or 68,280 BTU, is radiated as heat.
• At 0.10 USD / kWh it costs 14,953.32 USD over a
  four year life cycle to remove the extra heat.
• Increasing PSU efficiency by 20 reduces wasted
  cooling costs by four fifths.

14
Areas For Improvement

• Power delivery Uninterruptible Power Supplies
  (UPS)
• UPS devices are long-term investments. However,
  it may be time to upgrade.
• May lose .59 efficiency for every year a UPS
  device is in service. That is a potential
  improvement of 10.03 for upgrading a UPS from
  1990.

Source Eaton Power
15
Areas For Improvement

• Cooling Inefficiencies
• General cooling is a guaranteed way to generate
  hot spots, resulting in wasted energy.
• Over-cooling Cooling systems are often designed
  to run at maximum capacity at all times,
  resulting in wasted energy.

16
Areas For Improvement

• Cooling Improving Efficiency
• Targeted cooling and right-sizing
• The closer the cooling is to the source of the
  heat, the better.
• While it is tempting to buy big, only so much
  cooling is needed. Any more than is needed is
  wasted energy.
• Take off the sweaters!
• Remove the
• The heat the biggest barrier to cooling is
  heat!
• The gaps seal cable cut-outs and install
  blanking panels.
• Investigate liquid cooling
• Or how I learned to stop worrying and learned to
  love the H2O.
• APC, IBM, Liebert are offering liquid cooled rack
  solutions.

17
Areas for Improvement

• Direct Current (DC) Power Delivery
• Pros
• Delivering DC power directly may offer greater
  energy efficiency by reducing the number of
  distinct AC?DC inductors.
• One side effect would be reducing the amount of
  cooling required per server since excess heat in
  the AC?DC conversion is localized to fewer areas.
• Cons
• Only one major vendor (Rackable Systems)
  currently offers full DC support in its x86
  server product line. This could lead to vendor
  lock-in.
• Although used for years by the TelCo industry,
  the number of engineers experienced in DC power
  vs. AC power is small.

18
Areas For Improvement

• Cooling Improving efficiency (continued)
• Variable Frequency Drives (VFDs)
• A subset of adjustable speed drives (ASDs).
• Adjusts speed by altering the frequency and
  voltage output.
• Cooling equipment with VFDs can dynamically
  adjust the speed of the fan based on different
  input factors such as static pressure using a
  pitot tube and manometer.
• Reducing the output voltage when extra cooling is
  not need leads to less energy consumption.

19
Improving Data Center Efficiency

• Agenda
• Power in the data center today
• Areas for improvement
• Recommendations
• Technical
• Business
• Community

20
Recommendations

• Technical
• Remember, CPUs represent almost 10 of a data
  centers power consumption.
• Rethink power and cooling designs and systems.
• Explore consolidation efforts
• Compute
• Virtualization
• Blades
• Storage
• Shared storage SANs, Filers
• Storage designs massive array of idle disks
  (MAID), data deduplication

21
Recommendations

• Technical
• Delivering DC power directly to the servers
  sounds like a good idea, but it should be
  classified under wait and see.

22
Recommendations

• Business
• Make the power and cooling issue a bold line item
  on the CxOs budget.
• Consider creative options
• Rent roof space to local utilities companies for
  solar panels and buy back cheap energy.
• Move the data center to a colder climate.
• Pressure vendors to think green. Purchasing
  power is the greatest bargaining tool that
  customers have.
• Every percent counts. A 400kW data center _at_ 0.10
  USD / kW 245,173.46 USD / year. 1 will result
  in an annual savings of 3,504 USD. 15
  36,775.77 USD. 30 73,551.54 USD.

23
Recommendations

• Community
• Join organizations, such as the green grid, that
  are dedicated to helping to build an energy
  efficient data center.
• Pay attention to existing and emerging standards
  like Green Grids Power Usage Effectiveness (PUE)
  when buying new equipment to ensure energy
  efficiency.
• The Energy Star is expanding to cover servers.
• Encourage vendors who are already thinking green
  to keep it up Fujitsu-Siemens, NEC, IBM, HP,
  Dell, and Sun to name a few.

24
Conclusions

• The opportunity to increase efficiency will
  probably not disappear while electricity prices
  are not prohibitively expensive and compute power
  is cheap.
• The government and industry initiatives are
  developing and will continue to fine tune energy
  efficiency related standards. Efficiency used to
  be Power Out / Power In. It will be transformed
  to Productivity Out / Power In.
• To have the most effect, attack the largest
  culprit first processors. CPUs account for
  almost 10 of a data centers energy consumption.
  Save the power leader, save the world!

25
References

• AMD http//enterprise.amd.com/Downloads/34146D_P
  C_WP.pdf
• APC http//www.apcmedia.com/salestools/NRAN-6CN8
  PK_R0_EN.pdf
• Christian Belady http//electronics-cooling.com/
  articles/2007/feb/a3/
• Dell http//www.dell.com/downloads/global/power/
  ps1q07-20070210-CoverStory.pdf
• Intel http//www.intel.com