MAPA: Multi-Accelerator Pattern Allocation Policy for Multi-Tenant GPU Servers

Conference · Sun Nov 14 04:00:00 EST 2021

DOI:https://doi.org/10.1145/3458817.3480853· OSTI ID:1836016

Ranganath, Kiran ^[1]; Suetterlein, Joshua D. ^[2]; Manzano Franco, Joseph B. ^[2]; Song, Shuaiwen ^[3]; Wong, Daniel ^[1]

---

1. University of California, Riverside
2. BATTELLE (PACIFIC NW LAB)
3. University of Sydney

Multi-accelerator servers are increasingly being deployed in shared multi-tenant environments (such as in cloud data centers) in order to meet the demands of large-scale compute-intensive workloads. In addition, these accelerators are increasingly being inter-connected in complex topologies and workloads are exhibiting a wider variety of inter-accelerator communication patterns. However, existing al-location policies are ill-suited for these emerging use-cases. Specifically, this work identifies that multi-accelerator workloads are commonly fragmented leading to reduced bandwidth and increased latency for inter-accelerator communication. We propose Multi-Accelerator Pattern Allocation (MAPA), a graph pattern mining approach towards providing generalized allocation support for allocating multi-accelerator workloads on multi-accelerator servers. We demonstrate that MAPA is able to improve the execution time of multi-accelerator workloads and that MAPA is able to provide generalized benefits across various accelerator topologies. Finally, we demonstrate a speedup of 12.4% for75th percentile of jobs with the worst case execution time reduced by up to 35% against baseline policy using MAPA.

• Research Organization: Pacific Northwest National Laboratory (PNNL), Richland, WA (United States)
• Sponsoring Organization: USDOE
• DOE Contract Number: AC05-76RL01830
• OSTI ID: 1836016
• Report Number(s): PNNL-SA-165192
• Country of Publication: United States
• Language: English