Performance of a Rack of Liquid-Cooled Servers
Electronics densification is continuing at an unrelenting pace at the server, rack, and facility level. With increasing facility density levels, air flow management has become a major challenge and concern. In an effort to deal with the resulting thermal management challenges, manufacturers are increasingly turning to liquid-cooling as a practical solution. The majority of manufacturers have turned to liquid-cooled enclosed racks, or rear door heat exchangers, in which chilled water is delivered to the racks. Some manufacturers are now looking to cold plate cooling solutions that take the heat directly off problem components such as the CPUs, and to get it directly out of the facility. The current paper describes work done at the Pacific Northwest National Labs (PNNL) under a Department of Energy funded program entitled “Energy Smart Data Center”. An 8.2 kW rack of HP rx2600 2U servers has been converted from air-cooling to liquid spray cooling (CPUs only). The rack has been integrated into PNNL’s main cluster and subjected to a suite of acceptance tests. Under the testing, the spray cooled CPUs ran an average of 10C cooler than the air-cooled CPUs. Other peripheral devices such as the memory DIMMs ran an average of 8C cooler, and the power pod board was measured at 15C cooler. Since installation in July, 2005, the rack has been undergoing a one year uptime and reliability investigation. As part of the investigation, the rack has been subjected to monthly robustness testing and ongoing performance evaluation while running applications such as High Performance Linpack, parts of the NASA NPB-2 Benchmark Suite, and NWChem. The rack has undergone 3 months’ worth of robustness testing with no major events. Including the robustness testing, the rack uptime is at 95.54% over 299 days. While undergoing application testing, no computational performance differences have been observed between the liquid-cooled and standard air-cooled racks. A miniature Spray Cooled Energy Smart Data Center is now being designed as a final step to demonstrate the feasibility of scaling liquid-cooling at the single rack up to an entire facility.
- Research Organization:
- Pacific Northwest National Lab. (PNNL), Richland, WA (United States)
- Sponsoring Organization:
- USDOE
- DOE Contract Number:
- AC05-76RL01830
- OSTI ID:
- 921368
- Report Number(s):
- PNNL-SA-56301; ASHTAG; DP1501000; TRN: US200804%%717
- Journal Information:
- ASHRAE Transactions, 113(Part 1):101-114, Vol. 113, Issue Part 1; ISSN 0001-2505
- Country of Publication:
- United States
- Language:
- English
Similar Records
Energy Efficient and Sustainable HPC at Pacific Northwest National Laboratory
Real-time Data Center Energy Efficiency At Pacific Northwest National Laboratory