RADICAL-Pilot and PMIx/PRRTE: Executing Heterogeneous Workloads at Large Scale on Partitioned HPC Resources

Conference · Sun Jan 01 00:00:00 EST 2023

DOI:https://doi.org/10.1007/978-3-031-22698-4_5· OSTI ID:1999093

Titov, Mikhail ^[1]; Matteo, Turilli ^[2]; Merzky, Andre ^[2]; Naughton III, Thomas  ^[3]; Elwasif, Wael  ^[3]; Jha, Shantenu ^[2]

---

1. Brookhaven National Laboratory (BNL)
2. Rutgers University
3. ORNL

Execution of heterogeneous workflows on high-performance computing (HPC) platforms present unprecedented resource management and execution coordination challenges for runtime systems. Task heterogeneity increases the complexity of resource and execution management, limiting the scalability and efficiency of workflow execution. Resource partitioning and distribution of tasks execution over portioned resources promises to address those problems but we lack an experimental evaluation of its performance at scale. This paper provides a performance evaluation of the Process Management Interface for Exascale (PMIx) and its reference implementation PRRTE on the leadership-class HPC platform Summit, when integrated into a pilot-based runtime system called RADICAL-Pilot. We partition resources across multiple PRRTE Distributed Virtual Machine (DVM) environments, responsible for launching tasks via the PMIx interface. We experimentally measure the workload execution performance in terms of task scheduling/launching rate and distribution of DVM task placement times, DVM startup and termination overheads on the Summit leadership-class HPC platform. Integrated solution with PMIx/PRRTE enables using an abstracted, standardized set of interfaces for orchestrating the launch process, dynamic process management and monitoring capabilities. It extends scaling capabilities allowing to overcome a limitation of other launching mechanisms (e.g., JSM/LSF). Explored different DVM setup configurations provide insights on DVM performance and a layout to leverage it. Our experimental results show that heterogeneous workload of 65,500 tasks on 2048 nodes, and partitioned across 32 DVMs, runs steady with resource utilization not lower than 52%. While having less concurrently executed tasks resource utilization is able to reach up to 85%, based on results of heterogeneous workload of 8200 tasks on 256 nodes and 2 DVMs.

• Research Organization: Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States)
• Sponsoring Organization: USDOE Office of Science (SC)
• DOE Contract Number: AC05-00OR22725
• OSTI ID: 1999093
• Resource Relation: Journal Volume: 13592; Conference: 25th Workshop on Job Scheduling Strategies for Parallel Processing (JSSPP 2022) in conjunction with IPDPS 2022 - Lyon, , France - 6/3/2022 4:00:00 AM-6/3/2022 4:00:00 AM
• Country of Publication: United States
• Language: English