DOE PAGES title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: An automated and portable method for selecting an optimal GPU frequency

Journal Article · · Future Generations Computer Systems

Power consumption poses a significant challenge in current and emerging graphics processing unit (GPU) enabled high-performance computing systems. In modern GPUs, dynamic voltage frequency scaling (DVFS) appears to be a reliable control to regulate power consumption and performance. However, the DVFS design space is large - hence, brute-force approaches are infeasible to select the optimal frequency. Furthermore, no single frequency can be universally optimal for applications with varying computational intensities. Thus, the application's complexity and the availability of a wide range of frequency settings are a challenge in selecting the optimal frequency configuration for a given GPU workload. To that end, this paper proposes a systematic approach that consists of three steps. The feature characterization study identifies the fine-grain GPU utilization metrics that influence the power consumption and execution time of a given workload. To understand the performance, power, and energy consumption behaviors of a workload across GPU's DVFS design space, we derived analytical power and performance models using the identified fine-grain features. Here, it is shown that the same set of GPU utilization metrics can estimate both the power consumption and execution time while being agnostic of changes to frequency and input sizes. Applying a power control with the single objective of reducing power may cause performance degradation, leading to more energy consumption. A multi-objective approach is proposed to select the optimal GPU DVFS configuration for a workload that reduces power consumption with negligible degradation in performance. The evaluation was conducted using SPEC ACCEL benchmarks and three real applications - NAMD LAMMPS, and LSTM on NVIDIA GV100, GA100, and AMD MI210 GPUs. On average, real applications showed 29.6% energy savings with a performance loss of 5.2% on GA100 and 22.6% energy savings with a performance loss of 4.7% on GV100. Moreover, the proposed models are portable to real applications, GPU architectures, and vendors, and require metric collection at only the default frequency rather than all supported DVFS configurations. Additionally, we conducted a comparison between our models and the GPU assembly instructions (PTX)-based static models. The results revealed a significant reduction in the average error rates, with a decrease from 19.7% to 3.1% for power models and from 29.4% to 5.2% for performance models.

Research Organization:
Lawrence Berkeley National Laboratory (LBNL), Berkeley, CA (United States)
Sponsoring Organization:
USDOE Office of Science (SC), Basic Energy Sciences (BES). Scientific User Facilities (SUF); National Science Foundation (NSF); USDOE
Grant/Contract Number:
AC02-05CH11231; CNS-1817094; OAC-1835892; CNS-1939140; DEAC02-05CH11231
OSTI ID:
2475335
Alternate ID(s):
OSTI ID: 1992151
Journal Information:
Future Generations Computer Systems, Vol. 149; ISSN 0167-739X
Publisher:
ElsevierCopyright Statement
Country of Publication:
United States
Language:
English

References (16)

Mutual Information between Discrete and Continuous Data Sets journal February 2014
GPU Static Modeling Using PTX and Deep Structured Learning journal January 2019
Computing in the Dark Silicon Era: Current Trends and Research Challenges journal April 2017
GPU power prediction via ensemble machine learning for DVFS space exploration conference May 2018
GPGPU Performance Estimation With Core and Memory Frequency Scaling journal December 2020
A survey and measurement study of GPU DVFS on energy conservation journal May 2017
Application of the multiple criteria decision-making (MCDM) approach in the identification of Carbon Footprint reduction actions in the Brazilian beef production chain journal September 2018
Predictable GPUs Frequency Scaling for Energy and Performance conference August 2019
Power Modeling for GPU Architectures Using McPAT
  • Lim, Jieun; Lakshminarayana, Nagesh B.; Kim, Hyesoon
  • ACM Transactions on Design Automation of Electronic Systems, Vol. 19, Issue 3 https://doi.org/10.1145/2611758
journal June 2014
DVFS-aware application classification to improve GPGPUs energy efficiency journal April 2019
Estimating mutual information journal June 2004
Scalable molecular dynamics on CPU and GPU architectures with NAMD journal July 2020
Coordinated Batching and DVFS for DNN Inference on GPU Accelerators journal October 2022
Energy dissipation in general purpose microprocessors journal January 1996
Understanding GPU Power journal September 2016
A Multiple-Criteria Decision-Making Method Based on D Numbers and Belief Entropy journal March 2019