Tuesday, November 24, 2009

Server power

Many IT shops are looking to reduce power and the current solution is virtualization and moving toward local clouds. Little effort is put into the servers in the data center. Now certainly virtualizing 10 servers into 2 physical systems is likely to give a good deal of power reduction. However there are still the 2 physical servers, along with other servers that may not be appropriate for virtualization, and possibly a computational GRID system.
Is there much savings in the servers themselves? of course, and possibly quite a bit. There are two things to consider here, internal components (cpu, ram, disk, power supply, etc) and the chassis itself (2 cpu system, 4 cpu system, etc.). For chassis itself this is often dictated by the requirements. Try to avoid purchasing a system with future expansion capabilities. If 2 cpus are required, purchase a system that can handle just two cpus. You will waist a good deal of energy for the expansion capability, and worse by the time you are ready to expand the system, Moore's law generally means that it is cheaper to purchase a new one than to expand.

For the internal components though, you find different situations. It is well known to look at the CPU for power differences. Most CPUs have known power draws, though the draw at any given time is based on load, and modern CPUs have many power savings features for when there is not a heavy load. Server CPUs often vary from 45 to 120 watts, however performance can vary with this power consumption. Sometimes the choice can be pretty easy to judge. Say a server is needed with the performance capabilities of two Nehalem CPUs at 2.26 GHZ. This could be met by either two E5520 CPUs at 160 watts or two L5520 CPUs at 120 watts. The L5520 CPUs will likely cost a little more but with almost 1KW savings per day with the same performance.
On a basic server system today, server processors consume the most power, followed by memory, then disks or PCI slots, the motherboard, and lastly the fan and networking interconnects.

For example say specifications require a dual cpu system with 24G RAM. With a Nehalem architecture the RAM could be configured at least 3 different ways; 12x2, 6x4, and 12x2. Further the RAM can be a mix UDIMMs and RDIMMs as well as single, dual, and quad ranked. These configurations are hard enough to work out just to get a working system but there can be a huge difference in the power consumption based on the RAM configured, worse even the manufacture of the RAM. It is pretty easy to configure two seemingly identical systems of the same make and model with different RAM configurations such that one draws nearly twice the power of the other. Some server manufacturers give some web based tools to give an indication of how much power is required for a given configuration. Generally more ranks on the DIMM the more power efficient, also use higher density DIMMs for overall system.

This savings will be slightly more than doubled at the meter. For every watt saved in the computer, the average data center saves an additional:
  • 0.04W in power distribution
  • 0.14W in UPS
  • 1.07W in cooling
  • 0.10W in building transformer and switch gear
The savings could be much more significant than just the electric cost. It is common to fit 40 1U servers or 64 blade servers to a rack. Even with just modest changes a savings of 100 watts per server is possible, which is easily achieved with combination of CPU and RAM arrangement. With 64 blades per rack this comes to 6.4KW. This could be the difference between the infrastructure being able to support the servers and needing to upgrade the power distribution, UPS, and cooling in the data center.

The hardware used for virtualization, local clouds, and compute intensive environments are taking an increasingly large portion of the data center power budget. The configuration of servers for these systems have to balance initial cost, licensing, support and now power use. The SPECpower_ssj2008 benchmark can be used to assist though the systems are often not optimally configured for virtualization or GRID work. What is needed is a merger of SPECpower and the (currently unfinished) SPECvirtualization and/or SPECmpi2007.
Anandtech has an excellent article on server performance to power.

Monday, November 23, 2009

Data center power

A recent study by Lawrence Berkeley National Lab, Self-benchmarking Guide for Data Center Energy Performance, revealed that in a typical data center installation, an average of 33 percent of total power goes to IT equipment. The rest is consumed by cooling (50%), the power system (9%), and lighting (8%). The most efficient data centers can achieve 80% power utilization for IT equipment. This measurement, however, does not take into account the efficiency of the computer systems at doing the desired work; only the ratio of electric power for computers vs. power for support equipment.

The real goal is to do the same or more electronic work using less power. Many data centers have expanded to the point where there just isn’t enough power or cooling to allow new projects. Expanding data center power and cooling infrastructure can be very costly, and will only result in increased annual costs. Spending the money to make the center more efficient solves the same problem and reduces expenses while setting the data center on the GreenIT path.

The simplest and most direct way to reduce power consumption in a data center is to reduce the power used by the equipment. In the average data center, for every watt reduced on direct electronic equipment (computers, network equipment, and storage), at least one more watt will be saved on the facility side (HVA/C , UPS, power distribution). Further, in the average data center this equipment is already at or near capacity. Therefore the most direct path to savings is in reducing the IT equipment power needs with such methods as consolidation, virtualization, use of larger disks in storage systems, etc. Efforts spend on infrastructure without corresponding IT effort is wasted, as any savings will eventually be re-absorbed by continued wasteful IT side growth.

Successful efforts must include both Facility and IT systems. This is the crux of the problem as these two groups generally have little to do with each other with very different goals and needs. Facilities often has the electric costs and infrastructure maintenance costs, with goals like reduce electric bill, maintain data center temperature and power. IT usually doesn’t have the electric costs for the computers it maintains, and has goals such as maintain up time, reduce equipment and IT maintenance costs. None of these goals are in opposition but the two groups do not likely talk to each other and many of their terms sound like a different language to each other.