A survey on software methods to improve the energy efficiency of parallel computing

Jin, Chao; de Supinski, Bronis R.; Abramson, David; Poxon, Heidi; DeRose, Luiz; Dinh, Minh Ngoc; Endrei, Mark; Jessup, Elizabeth R.

doi:10.1177/1094342016665471

Title: A survey on software methods to improve the energy efficiency of parallel computing

Journal Article · Fri Sep 09 00:00:00 EDT 2016 · International Journal of High Performance Computing Applications

DOI:https://doi.org/10.1177/1094342016665471· OSTI ID:1812125

Jin, Chao ^[1]; de Supinski, Bronis R. ^[2]; Abramson, David ^[3]; Poxon, Heidi ^[4]; DeRose, Luiz ^[4]; Dinh, Minh Ngoc ^[3]; Endrei, Mark ^[3]; Jessup, Elizabeth R. ^[5]

Univ. of Queensland, Brisbane, QLD (Australia); Monash Univ., Melbourne, VIC (Australia)
Lawrence Livermore National Lab. (LLNL), Livermore, CA (United States)
Univ. of Queensland, Brisbane, QLD (Australia)
Cray Inc., Saint Paul, MN (United States)
Univ. of Colorado, Boulder, CO (United States)

Energy consumption is one of the top challenges for achieving the next generation of supercomputing. Codesign of hardware and software is critical for improving energy efficiency (EE) for future large-scale systems. Many architectural power-saving techniques have been developed, and most hardware components are approaching physical limits. Accordingly, parallel computing software, including both applications and systems, should exploit power-saving hardware innovations and manage efficient energy use. In addition, new power-aware parallel computing methods are essential to decrease energy usage further. This article surveys software-based methods that aim to improve EE for parallel computing. It reviews the methods that exploit the characteristics of parallel scientific applications, including load imbalance and mixed precision of floating-point (FP) calculations, to improve EE. In addition, this article summarizes widely used methods to improve power usage at different granularities, such as the whole system and per application. In particular, it describes the most important techniques to measure and to achieve energy-efficient usage of various parallel computing facilities, including processors, memories, and networks. Overall, this article reviews the state-of-the-art of energy-efficient methods for parallel computing to motivate researchers to achieve optimal parallel computing under a power budget constraint.

View Accepted Manuscript (DOE)

Cite

Export

Save

Research Organization:: Lawrence Livermore National Laboratory (LLNL), Livermore, CA (United States)

Sponsoring Organization:: USDOE National Nuclear Security Administration (NNSA)

Grant/Contract Number:: AC52-07NA27344

OSTI ID:: 1812125

Report Number(s):: LLNL-JRNL-698191; 829335

Journal Information:: International Journal of High Performance Computing Applications, Vol. 31, Issue 6; ISSN 1094-3420

Publisher:: SAGECopyright Statement

Country of Publication:: United States

Language:: English

References (34)

The International Exascale Software Project roadmap Dongarra, Jack; Beckman, Pete; Moore, Terry The International Journal of High Performance Computing Applications, Vol. 25, Issue 1 https://doi.org/10.1177/1094342010391989	journal	January 2011
Prediction-Based Power-Performance Adaptation of Multithreaded Scientific Codes Curtis-Maury, M.; Blagojevic, F.; Antonopoulos, C. D. IEEE Transactions on Parallel and Distributed Systems, Vol. 19, Issue 10 https://doi.org/10.1109/TPDS.2007.70804	journal	October 2008
Energy-Efficient Server Clusters Elnozahy, E. N.; Kistler, Michael; Rajamony, Ramakrishnan Power-Aware Computer Systems https://doi.org/10.1007/3-540-36612-1_12	book	January 2003
Energy-Efficient High Performance Computing Laros III, James H.; Pedretti, Kevin; Kelly, Suzanne M. SpringerBriefs in Computer Science https://doi.org/10.1007/978-1-4471-4492-2	book	January 2013
On the Interplay of Parallelization, Program Performance, and Energy Consumption Cho, Sangyeun; Melhem, Rami G. IEEE Transactions on Parallel and Distributed Systems, Vol. 21, Issue 3 https://doi.org/10.1109/TPDS.2009.41	journal	March 2010
A Survey of Methods for Analyzing and Improving GPU Energy Efficiency Mittal, Sparsh; Vetter, Jeffrey S. ACM Computing Surveys, Vol. 47, Issue 2 https://doi.org/10.1145/2636342	journal	August 2014
Parallel job scheduling for power constrained HPC systems Etinski, M.; Corbalan, J.; Labarta, J. Parallel Computing, Vol. 38, Issue 12 https://doi.org/10.1016/j.parco.2012.08.001	journal	December 2012
TUE, a New Energy-Efficiency Metric Applied at ORNL’s Jaguar Patterson, Michael K.; Poole, Stephen W.; Hsu, Chung-Hsing Lecture Notes in Computer Science https://doi.org/10.1007/978-3-642-38750-0_28	book	January 2013
Minimizing Communication in Numerical Linear Algebra Ballard, Grey; Demmel, James; Holtz, Olga SIAM Journal on Matrix Analysis and Applications, Vol. 32, Issue 3 https://doi.org/10.1137/090769156	journal	July 2011
Active Harmony: Towards Automated Performance Tuning Tapus, C.; Hollingsworth, J. K. ACM/IEEE SC 2002 Conference (SC'02) https://doi.org/10.1109/SC.2002.10062	conference	January 2002
Measuring power consumption on IBM Blue Gene/P Hennecke, Michael; Frings, Wolfgang; Homberg, Willi Computer Science - Research and Development, Vol. 27, Issue 4 https://doi.org/10.1007/s00450-011-0192-y	journal	August 2011
Power-Management Architecture of the Intel Microarchitecture Code-Named Sandy Bridge Rotem, Efraim; Naveh, Alon; Ananthakrishnan, Avinash IEEE Micro, Vol. 32, Issue 2 https://doi.org/10.1109/MM.2012.12	journal	March 2012
Reducing Energy Costs for IBM Blue Gene/P via Power-Aware Job Scheduling Zhou, Zhou; Lan, Zhiling; Tang, Wei Job Scheduling Strategies for Parallel Processing https://doi.org/10.1007/978-3-662-43779-7_6	book	January 2014
Dynamic Voltage and Frequency Scaling for Scientific Applications Hsu, Chung-Hsing; Kremer, Ulrich Languages and Compilers for Parallel Computing https://doi.org/10.1007/3-540-35767-X_6	book	January 2003
PowerPack: Energy Profiling and Analysis of High-Performance Systems and Applications Ge, Rong; Feng, Xizhou; Song, Shuaiwen IEEE Transactions on Parallel and Distributed Systems, Vol. 21, Issue 5 https://doi.org/10.1109/TPDS.2009.76	journal	May 2010
Energy efficiency of mixed precision iterative refinement methods using hybrid hardware platforms: An evaluation of different solver and hardware configurations Anzt, Hartwig; Rocker, Björn; Heuveline, Vincent Computer Science - Research and Development, Vol. 25, Issue 3-4 https://doi.org/10.1007/s00450-010-0124-2	journal	August 2010
Characterization of Fixed and Reconfigurable Multi-Core Devices for Application Acceleration Williams, Jason; Massie, Chris; George, Alan D. ACM Transactions on Reconfigurable Technology and Systems, Vol. 3, Issue 4 https://doi.org/10.1145/1862648.1862649	journal	November 2010
Energy-aware job scheduler for high-performance computing Mämmelä, Olli; Majanen, Mikko; Basmadjian, Robert Computer Science - Research and Development, Vol. 27, Issue 4 https://doi.org/10.1007/s00450-011-0189-6	journal	August 2011
Limit to the Bit-Rate Capacity of Electrical Interconnects from the Aspect Ratio of the System Architecture Miller, D. A. B.; Ozaktas, H. M. Journal of Parallel and Distributed Computing, Vol. 41, Issue 1 https://doi.org/10.1006/jpdc.1996.1285	journal	February 1997
Analyzing the Energy-Time Trade-Off in High-Performance Computing Applications Freeh, Vincent W.; Lowenthal, David K.; Pan, Feng IEEE Transactions on Parallel and Distributed Systems, Vol. 18, Issue 6 https://doi.org/10.1109/TPDS.2007.1026	journal	June 2007
Power Aware Dynamic Provisioning of HPC Networks Groves, Taylor; Grant, Ryan https://doi.org/10.2172/1331496	report	October 2015
Wattch: a framework for architectural-level power analysis and optimizations Brooks, David; Tiwari, Vivek; Martonosi, Margaret Proceedings of the 27th annual international symposium on Computer architecture - ISCA '00 https://doi.org/10.1145/339647.339657	conference	January 2000
Performance estimation of high performance computing systems with Energy Efficient Ethernet technology Miwa, Shinobu; Aita, Sho; Nakamura, Hiroshi Computer Science - Research and Development, Vol. 29, Issue 3-4 https://doi.org/10.1007/s00450-013-0238-4	journal	July 2013
Self-Tuned Software-Managed Energy Reduction in InfiniBand Links Dickov, Branimir; Carpenter, Paul M.; Pericas, Miquel 2015 IEEE 21st International Conference on Parallel and Distributed Systems (ICPADS) https://doi.org/10.1109/ICPADS.2015.87	conference	December 2015
Coping with Parametric Variation at Near-Threshold Voltages Karpuzcu, Ulya R.; Kim, Nam Sung; Torrellas, Josep IEEE Micro, Vol. 33, Issue 4 https://doi.org/10.1109/MM.2013.71	journal	July 2013
CALU: A Communication Optimal LU Factorization Algorithm Grigori, Laura; Demmel, James W.; Xiang, Hua SIAM Journal on Matrix Analysis and Applications, Vol. 32, Issue 4 https://doi.org/10.1137/100788926	journal	October 2011
Extending Amdahl's Law for Energy-Efficient Computing in the Many-Core Era Woo, Dong Hyuk; Lee, Hsien-Hsin S. Computer, Vol. 41, Issue 12 https://doi.org/10.1109/MC.2008.494	journal	December 2008
Computer Architecture Techniques for Power-Efficiency Kaxiras, Stefanos; Martonosi, Margaret Synthesis Lectures on Computer Architecture, Vol. 3, Issue 1 https://doi.org/10.2200/S00119ED1V01Y200805CAC004	journal	January 2008
PowerInsight - A commodity power measurement capability Laros, James H.; Pokorny, Phil; DeBonis, David 2013 International Green Computing Conference (IGCC), 2013 International Green Computing Conference Proceedings https://doi.org/10.1109/IGCC.2013.6604485	conference	June 2013
What is Happening to Power, Performance, and Software? Esmaeilzadeh, H.; Cao, T.; Yang, X. IEEE Micro, Vol. 32, Issue 3 https://doi.org/10.1109/MM.2012.20	journal	May 2012
On the Use of Commodity Ethernet Technology in Exascale HPC Systems Benito, Mariano; Vallejo, Enrique; Beivide, Ramon 2015 IEEE 22nd International Conference on High-Performance Computing (HiPC), 2015 IEEE 22nd International Conference on High Performance Computing (HiPC) https://doi.org/10.1109/HiPC.2015.32	conference	December 2015
Dynamic Voltage and Frequency Scaling for Scientific Applications Hsu, Chung-Hsing; Kremer, Ulrich Rutgers University https://doi.org/10.7282/t30k2d5x	text	January 2021
Wattch: a framework for architectural-level power analysis and optimizations Brooks, David; Tiwari, Vivek; Martonosi, Margaret ACM SIGARCH Computer Architecture News, Vol. 28, Issue 2 https://doi.org/10.1145/342001.339657	journal	May 2000
Energy Efficiency of Mixed Precision Iterative Refinement Methods using Hybrid Hardware Platforms: An Evaluation of different Solver and Hardware Configurations Anzt, Hartwig; Heuveline, Vincent; Rocker, Björn University Library Heidelberg https://doi.org/10.11588/emclpp.2010.03.11678	text	January 2010

Cited By (6)

Energy-Aware High-Performance Computing: Survey of State-of-the-Art Tools, Techniques, and Environments Czarnul, Pawel; Proficz, Jerzy; Krzywaniak, Adam Scientific Programming, Vol. 2019 https://doi.org/10.1155/2019/8348791	journal	April 2019
Using meta-heuristics and machine learning for software optimization of parallel computing systems: a systematic literature review Memeti, Suejb; Pllana, Sabri; Binotto, Alécio Computing, Vol. 101, Issue 8 https://doi.org/10.1007/s00607-018-0614-9	journal	April 2018
A Taxonomy and Future Directions for Sustainable Cloud Computing: 360 Degree View Gill, Sukhpal Singh; Buyya, Rajkumar ACM Computing Surveys, Vol. 51, Issue 5 https://doi.org/10.1145/3241038	journal	January 2019
Statistical and machine learning models for optimizing energy in parallel applications Endrei, Mark; Jin, Chao; Dinh, Minh Ngoc The International Journal of High Performance Computing Applications, Vol. 33, Issue 6 https://doi.org/10.1177/1094342019842915	journal	April 2019
A Taxonomy and Future Directions for Sustainable Cloud Computing: 360 Degree View Gill, Sukhpal Singh; Buyya, Rajkumar arXiv https://doi.org/10.48550/arxiv.1712.02899	preprint	January 2017
Using Meta-heuristics and Machine Learning for Software Optimization of Parallel Computing Systems: A Systematic Literature Review Memeti, Suejb; Pllana, Sabri; Binotto, Alecio arXiv https://doi.org/10.48550/arxiv.1801.09444	text	January 2018

Similar Records

Data Locality Enhancement of Dynamic Simulations for Exascale Computing (Final Report)

Technical Report · Fri Nov 29 00:00:00 EST 2019 · OSTI ID:1812125

Shen, Xipeng

Automated Cache Performance Analysis And Optimization

Technical Report · Mon Dec 23 00:00:00 EST 2013 · OSTI ID:1812125

Mohror, Kathryn

Institute for Sustained Performance, Energy, and Resilience (SuPER)

Technical Report · Wed Nov 30 00:00:00 EST 2016 · OSTI ID:1812125

Jagode, Heike; Bosilca, George; Danalis, Anthony; +2 more

Related Subjects

97 MATHEMATICS AND COMPUTING
Parallel computing
high performance computing
power saving
energy efficiency
auto-tuning

Title: A survey on software methods to improve the energy efficiency of parallel computing

Citation Formats

References (34)

Cited By (6)

Similar Records

Related Subjects