MARS: Malleable Actor-Critic Reinforcement Learning Scheduler

Conference · Sat Nov 12 23:00:00 EST 2022

DOI:https://doi.org/10.1109/IPCCC55026.2022.9894315· OSTI ID:1958175

Baheri, Betis ^[1]; Tronge, Jacob ^[1]; Fang, Bo ^[2]; Li, Ang ^[2]; Chaudhary, Vipin ^[3]; Guan, Qiang ^[1]

---

1. Kent State University
2. BATTELLE (PACIFIC NW LAB)
3. Case Western Reserve University

In this paper, we introduce MARS, a new scheduling system for HPC-cloud infrastructures based on a cost-aware, flexible reinforcement learning approach, which serves as an intermediate layer for next generation HPC-cloud resource manager. MARS ensembles the pre-trained models from heuristic workloads and decides on the most cost-effective strategy for optimization. A whole workflow application would be split into several optimizable dependent sub-tasks, then based on the pre- defined resource management plan, a reward will be generated after executing a scheduled task. Lastly, MARS updates the Deep Neural Network (DNN) model based on the reward. MARS is designed to optimize the existing models through reinforcement mechanisms. MARS adapts to the dynamics of workflow applications, selects the most cost-effective scheduling solution among pre-built scheduling strategies (backfilling, SJF, etc.) and self- learning deep neural network model at run-time. We evaluate MARS with different real-world workflow traces. MARS can achieve 5%-60% increased performance compare to state-of-the- art approaches.

• Research Organization: Pacific Northwest National Laboratory (PNNL), Richland, WA (United States)
• Sponsoring Organization: USDOE
• DOE Contract Number: AC05-76RL01830
• OSTI ID: 1958175
• Report Number(s): PNNL-SA-170367
• Country of Publication: United States
• Language: English