HAM: Hotspot-Aware Manager for Improving Communications with 3D-Stacked Memory

Journal Article · Tue Jun 01 00:00:00 EDT 2021 · IEEE Transactions on Computers

DOI:https://doi.org/10.1109/TC.2021.3066982· OSTI ID:1787471

Wang, Xi ^[1]; Tumeo, Antonino ^[2]; Leidel, John D. ^[3]; Li, Jie ^[1]; Chen, Yong ^[1]

---

1. Texas Tech University
2. BATTELLE (PACIFIC NW LAB)
3. Tactical Computing Labs

merging High-Performance Computing (HPC) workloads, such as graph analytics, machine learning, and big data science, are data-intensive. Data-intensive workloads usually present fine-grained memory accesses with limited or no data locality, and thus incur frequent cache misses and low utilization of memory bandwidth. 3D-stacked memory devices such as Hybrid Memory Cube (HMC) and High Bandwidth Memory (HBM) can provide significantly higher bandwidth than conventional memory modules. However, the traditional interfaces and optimization methods for JEDEC DDR devices do not allow to fully exploit the potential performance of 3D-stacked memory with the massive amount of irregular memory accesses of data-intensive applications. In this paper, we propose a novel Hotspot-Aware Manager (HAM) infrastructure for 3D-stacked memory devices capable of optimizing memory access streams via request aggregation, hotspot detection, and in-memory prefetching. %and an associated hotspot-aware page policy. We present the HAM design and implementation, and simulate it on a system using RISC-V embedded cores with attached HMC devices. We extensively evaluate HAM with over 12 benchmarks and applications representing diverse irregular memory access patterns. The results show that, on average, HAM reduces redundant requests by 37.51\% and increases the prefetch buffer hit rate by 4.2 times, compared to a baseline streaming prefetcher. On the selected benchmark set, HAM provides performance gains of 21.81\% in average (up to 34.28\%) and power savings of 35.07\% over a standard 3D-stacked memory.

• Research Organization: Pacific Northwest National Lab. (PNNL), Richland, WA (United States)
• Sponsoring Organization: USDOE
• DOE Contract Number: AC05-76RL01830
• OSTI ID: 1787471
• Report Number(s): PNNL-SA-161294
• Journal Information: IEEE Transactions on Computers, Vol. 70, Issue 6
• Country of Publication: United States
• Language: English