Design and Performance Characterization of RADICAL-Pilot on Leadership-Class Platforms

Merzky, Andre; Turilli, Matteo; Titov, Mikhail; Al-Saadi, Aymen; Jha, Shantenu

doi:10.1109/tpds.2021.3105994

Title: Design and Performance Characterization of RADICAL-Pilot on Leadership-Class Platforms

Journal Article · Fri Apr 01 00:00:00 EDT 2022 · IEEE Transactions on Parallel and Distributed Systems

DOI:https://doi.org/10.1109/tpds.2021.3105994· OSTI ID:1830194

Merzky, Andre ^[1];

^[1];

^[1]; Al-Saadi, Aymen ^[1]; Jha, Shantenu ^[2]

Rutgers Univ., Piscataway, NJ (United States)
Rutgers Univ., Piscataway, NJ (United States); Brookhaven National Lab. (BNL), Upton, NY (United States)

Many extreme scale scientific applications have workloads comprised of a large number of individual highperformance tasks. The Pilot abstraction decouples workload specification, resource management, and task execution via job placeholders and late-binding. As such, suitable implementations of the Pilot abstraction can support the collective execution of large number of tasks on supercomputers. We introduce RADICAL-Pilot (RP) as a portable, modular and extensible Pilot enabled runtime system. We describe RP's design, architecture and implementation. We characterize its performance and show its ability to scalably execute workloads comprised of tens of thousands heterogeneous tasks on DOE and NSF leadership-class HPC platforms. Specifically, we investigate RP's weak/strong scaling with CPU/GPU, single/multi core, (non)MPI tasks and python functions when using most of ORNL Summit and TACC Frontera. RADICAL-Pilot can be used stand-alone, as well as the runtime for third-party workflow systems.

View Accepted Manuscript (DOE)

Cite

Export

Save

Research Organization:: Brookhaven National Laboratory (BNL), Upton, NY (United States)

Sponsoring Organization:: USDOE Office of Science (SC), Advanced Scientific Computing Research

Grant/Contract Number:: SC0012704

OSTI ID:: 1830194

Report Number(s):: BNL-222357-2021-JAAM

Journal Information:: IEEE Transactions on Parallel and Distributed Systems, Vol. 33, Issue 4; ISSN 1045-9219

Publisher:: IEEECopyright Statement

Country of Publication:: United States

Language:: English

References (36)

Makeflow: a portable abstraction for data intensive computing on clusters, clouds, and grids Albrecht, Michael; Donnelly, Patrick; Bui, Peter Proceedings of the 1st ACM SIGMOD Workshop on Scalable Workflow Execution Engines and Technologies - SWEET '12 https://doi.org/10.1145/2443416.2443417	conference	January 2012
Pegasus: A Framework for Mapping Complex Scientific Workflows onto Distributed Systems Deelman, Ewa; Singh, Gurmeet; Su, Mei-Hui Scientific Programming, Vol. 13, Issue 3 https://doi.org/10.1155/2005/128026	journal	January 2005
Pegasus, a workflow management system for science automation Deelman, Ewa; Vahi, Karan; Juve, Gideon Future Generation Computer Systems, Vol. 46 https://doi.org/10.1016/j.future.2014.10.008	journal	May 2015
Eden: Simplified Management of Atypical High-Performance Computing Jobs Simmerman, Scott; Osborne, James; Huang, Jian Computing in Science & Engineering, Vol. 15, Issue 6 https://doi.org/10.1109/MCSE.2012.92	journal	November 2013
JETS: Language and System Support for Many-Parallel-Task Workflows Wozniak, Justin M.; Wilde, Michael; Katz, Daniel S. Journal of Grid Computing, Vol. 11, Issue 3 https://doi.org/10.1007/s10723-013-9259-2	journal	June 2013
Coasters: Uniform Resource Provisioning and Access for Clouds and Grids Hategan, M.; Wozniak, J.; Maheshwari, K. 2011 IEEE 4th International Conference on Utility and Cloud Computing (UCC 2011), 2011 Fourth IEEE International Conference on Utility and Cloud Computing https://doi.org/10.1109/UCC.2011.25	conference	December 2011
Swift: Fast, Reliable, Loosely Coupled Parallel Computation Zhao, Yong; Hategan, Mihael; Clifford, Ben 2007 IEEE Congress on Services (Services 2007) https://doi.org/10.1109/SERVICES.2007.63	conference	July 2007
High-throughput binding affinity calculations at extreme scales Dakka, Jumana; Turilli, Matteo; Wright, David W. BMC Bioinformatics, Vol. 19, Issue S18 https://doi.org/10.1186/s12859-018-2506-6	journal	December 2018
High-Throughput Computing on High-Performance Platforms: A Case Study Oleynik, Danila; Panitkin, Sergey; Turilli, Matteo 2017 IEEE 13th International Conference on e-Science (e-Science) https://doi.org/10.1109/eScience.2017.43	conference	October 2017
Evaluating Distributed Execution of Workloads Turilli, Matteo; Babuji, Yadu Nand; Merzky, Andre 2017 IEEE 13th International Conference on e-Science (e-Science) https://doi.org/10.1109/eScience.2017.41	conference	October 2017
Middleware Building Blocks for Workflow Systems Turilli, Matteo; Balasubramanian, Vivek; Merzky, Andre Computing in Science & Engineering, Vol. 21, Issue 4 https://doi.org/10.1109/MCSE.2019.2920048	journal	July 2019
Flux: A Next-Generation Resource Management Framework for Large HPC Centers Ahn, Dong H.; Garlick, Jim; Grondona, Mark 2014 43nd International Conference on Parallel Processing Workshops (ICCPW), 2014 43rd International Conference on Parallel Processing Workshops https://doi.org/10.1109/ICPPW.2014.15	conference	September 2014
HPX: A Task Based Programming Model in a Global Address Space Kaiser, Hartmut; Heller, Thomas; Adelstein-Lelbach, Bryce Proceedings of the 8th International Conference on Partitioned Global Address Space Programming Models - PGAS '14 https://doi.org/10.1145/2676870.2676883	conference	January 2014
Cilk: an efficient multithreaded runtime system Blumofe, Robert D.; Joerg, Christopher F.; Kuszmaul, Bradley C. ACM SIGPLAN Notices, Vol. 30, Issue 8 https://doi.org/10.1145/209937.209958	journal	August 1995
GWpilot: Enabling multi-level scheduling in distributed infrastructures with GridWay and pilot jobs Rubio-Montero, A. J.; Huedo, E.; Castejón, F. Future Generation Computer Systems, Vol. 45 https://doi.org/10.1016/j.future.2014.10.003	journal	April 2015
FireWorks: a dynamic workflow system designed for high-throughput applications: FireWorks: A Dynamic Workflow System Designed for High-Throughput Applications Jain, Anubhav; Ong, Shyue Ping; Chen, Wei Concurrency and Computation: Practice and Experience, Vol. 27, Issue 17 https://doi.org/10.1002/cpe.3505	journal	May 2015
AI-driven multiscale simulations illuminate mechanisms of SARS-CoV-2 spike dynamics Casalino, Lorenzo; Dommer, Abigail C.; Gaieb, Zied The International Journal of High Performance Computing Applications https://doi.org/10.1177/10943420211006452	journal	April 2021
A Comprehensive Perspective on Pilot-Job Systems Turilli, Matteo; Santcroos, Mark; Jha, Shantenu ACM Computing Surveys, Vol. 51, Issue 2 https://doi.org/10.1145/3177851	journal	April 2018
Parsl: Pervasive Parallel Programming in Python Babuji, Yadu; Foster, Ian; Wilde, Michael Proceedings of the 28th International Symposium on High-Performance Parallel and Distributed Computing - HPDC '19 https://doi.org/10.1145/3307681.3325400	conference	January 2019
Resource Allocation Policies for Loosely Coupled Applications in Heterogeneous Computing Systems Hwang, Eunji; Kim, Suntae; Yoo, Tae-kyung IEEE Transactions on Parallel and Distributed Systems, Vol. 27, Issue 8 https://doi.org/10.1109/TPDS.2015.2461154	journal	August 2016
The future of scientific workflows Deelman, Ewa; Peterka, Tom; Altintas, Ilkay The International Journal of High Performance Computing Applications, Vol. 32, Issue 1 https://doi.org/10.1177/1094342017704893	journal	April 2017
Pilot factory – a Condor-based system for scalable Pilot Job generation in the Panda WMS framework Chiu, Po-Hsiang; Potekhin, Maxim Journal of Physics: Conference Series, Vol. 219, Issue 6 https://doi.org/10.1088/1742-6596/219/6/062041	journal	April 2010
Characterization and identification of HPC applications at leadership computing facility Liu, Zhengchun; Lewis, Ryan; Kettimuthu, Rajkumar ICS '20: 2020 International Conference on Supercomputing, Proceedings of the 34th ACM International Conference on Supercomputing https://doi.org/10.1145/3392717.3392774	conference	June 2020
The Impact of Heterogeneous Computing on Workflows for Biomolecular Simulation and Analysis Cheatham, Thomas E.; Roe, Daniel R. Computing in Science & Engineering, Vol. 17, Issue 2 https://doi.org/10.1109/MCSE.2015.7	journal	March 2015
Synapse: Synthetic application profiler and emulator Merzky, Andre; Ha, Ming Tai; Turilli, Matteo Journal of Computational Science, Vol. 27 https://doi.org/10.1016/j.jocs.2018.06.012	journal	July 2018
Evolution of the ATLAS PanDA workload management system for exascale computational science Maeno, T.; De, K.; Klimentov, A. Journal of Physics: Conference Series, Vol. 513, Issue 3 https://doi.org/10.1088/1742-6596/513/3/032062	journal	June 2014
Characterizing the Performance of Executing Many-tasks on Summit Turilli, Matteo; Merzky, Andre; Naughton, Thomas 2019 IEEE/ACM Third Annual Workshop on Emerging Parallel and Distributed Runtime Systems and Middleware (IPDRM) https://doi.org/10.1109/IPDRM49579.2019.00007	conference	November 2019
The open science grid Pordes, Ruth; Petravick, Don; Kramer, Bill Journal of Physics: Conference Series, Vol. 78 https://doi.org/10.1088/1742-6596/78/1/012057	journal	July 2007
SAGA: A standardized access layer to heterogeneous Distributed Computing Infrastructure Merzky, Andre; Weidner, Ole; Jha, Shantenu SoftwareX, Vol. 1-2 https://doi.org/10.1016/j.softx.2015.03.001	journal	September 2015
DIRAC pilot framework and the DIRAC Workload Management System Casajus, Adrian; Graciani, Ricardo; Paterson, Stuart Journal of Physics: Conference Series, Vol. 219, Issue 6 https://doi.org/10.1088/1742-6596/219/6/062049	journal	April 2010
glideinWMS—a generic pilot-based workload management system Sfiligoi, I. Journal of Physics: Conference Series, Vol. 119, Issue 6 https://doi.org/10.1088/1742-6596/119/6/062044	journal	July 2008
Falkon: a Fast and Light-weight tasK executiON framework Raicu, Ioan; Zhao, Yong; Dumitrescu, Catalin Proceedings of the 2007 ACM/IEEE conference on Supercomputing - SC '07 https://doi.org/10.1145/1362622.1362680	conference	January 2007
CernVM Co-Pilot: an Extensible Framework for Building Scalable Computing Infrastructures on the Cloud Harutyunyan, A.; Blomer, J.; Buncic, P. Journal of Physics: Conference Series, Vol. 396, Issue 3 https://doi.org/10.1088/1742-6596/396/3/032054	journal	December 2012
Fast recovery of free energy landscapes via diffusion-map-directed molecular dynamics Preto, Jordane; Clementi, Cecilia Phys. Chem. Chem. Phys., Vol. 16, Issue 36 https://doi.org/10.1039/c3cp54520b	journal	January 2014
Supercomputing Pipelines Search for Therapeutics Against COVID-19 Vermaas, Josh Vincent; Sedova, Ada; Baker, Matthew Computing in Science & Engineering https://doi.org/10.1109/mcse.2020.3036540	journal	January 2020
Characterizing the Performance of Executing Many-tasks on Summit Turilli, Matteo; Merzky, Andre; Naughton, Thomas arXiv https://doi.org/10.48550/arxiv.1909.03057	preprint	January 2019

Similar Records

Characterizing the Performance of Executing Many-tasks on Summit

Conference · Wed Jan 01 00:00:00 EST 2020 · OSTI ID:1830194

Matteo, Turilli; Merzky, Andre; Naughton III, Thomas; +2 more

RADICAL-Pilot and PMIx/PRRTE: Executing Heterogeneous Workloads at Large Scale on Partitioned HPC Resources

Conference · Sun Jan 01 00:00:00 EST 2023 · OSTI ID:1830194

Titov, Mikhail; Matteo, Turilli; Merzky, Andre; +3 more

$\mathrm{RADICAL}$-Pilot and $\mathrm{PMIx}$/$\mathrm{PRRTE}$: Executing Heterogeneous Workloads at Large Scale on Partitioned $\mathrm{HPC}$ Resources

Journal Article · Thu Jan 12 00:00:00 EST 2023 · Lecture Notes in Computer Science · OSTI ID:1830194

Titov, Mikhail; Turilli, Matteo; Merzky, Andre; +3 more

Related Subjects

97 MATHEMATICS AND COMPUTING

Title: Design and Performance Characterization of RADICAL-Pilot on Leadership-Class Platforms

Citation Formats

References (36)

Similar Records

Related Subjects