MARBLE: A Multi-GPU Aware Job Scheduler for Deep Learning on HPC Systems

Conference · 01 May 2020

DOI:https://doi.org/10.1109/CCGrid49817.2020.00-66· OSTI ID:1649080

Han, Jingoo ^[1]; Rafique, Mustafa ^[2]; Xu, Luna ^[1]; Butt, Ali R. ^[1]; Lim, Seung-Hwan  ^[3]; Vazhkudai, Sudharshan  ^[3]

---

1. Virginia Tech, Blacksburg, VA
2. Rochester Institute of Technology, Rochester, NY
3. ORNL

Deep learning (DL) has become a key tool for solving complex scientific problems. However, managing the multi-dimensional large-scale data associated with DL, especially atop extant multiple graphics processing units (GPUs) in modern supercomputers poses significant challenges. Moreover, the latest high-performance computing (HPC) architectures bring different performance trends in training throughput compared to the existing studies. Existing DL optimizations such as larger batch size and GPU locality-aware scheduling have little effect on improving DL training throughput performance due to fast CPU-to-GPU connections. Additionally, DL training on multiple GPUs scales sublinearly. Thus, simply adding more GPUs to a system is ineffective. To this end, we design MARBLE, a first-of-its-kind job scheduler, which considers the non-linear scalability of GPUs at the intra-node level to schedule an appropriate number of GPUs per node for a job. By sharing the GPU resources on a node with multiple DL jobs, MARBLE avoids low GPU utilization in current multi-GPU DL training on HPC systems. Our comprehensive evaluation in the Summit supercomputer shows that MARBLE is able to improve DL training performance by up to 48.3% compared to the popular Platform Load Sharing Facility (LSF) scheduler. Compared to the state-of-the-art of DL scheduler, Optimus, MARBLE reduces the job completion time by up to 47%.

• Research Organization: Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States)
• Sponsoring Organization: USDOE
• DOE Contract Number: AC05-00OR22725
• OSTI ID: 1649080
• Resource Relation: Conference: The 20th IEEE/ACM International Symposium on Cluster, Cloud and Internet Computing (CCGRID) - Melbourne, , Australia - 5/11/2020 12:00:00 PM-5/14/2020 4:00:00 AM
• Country of Publication: United States
• Language: English