Julia as a unifying end-to-end workflow language on the Frontier exascale system

Godoy, William; Valero Lara, Pedro; Anderson, Caira; Lee, Katrina; Gainaru, Ana; Ferreira Da Silva, Rafael; Vetter, Jeffrey

doi:10.1145/3624062.3624278

Julia as a unifying end-to-end workflow language on the Frontier exascale system

Conference · 01 November 2023

DOI:https://doi.org/10.1145/3624062.3624278· OSTI ID:2217706

^[1]; ^[1]; Anderson, Caira ^[1]; Lee, Katrina ^[1]; Gainaru, Ana ^[1]; ^[1]; ^[1]

ORNL

We evaluate Julia as a single language and ecosystem paradigm powered by LLVM to develop workflow components for high-performance computing. We run a Gray-Scott, 2-variable diffusion-reaction application using a memory-bound, 7-point stencil kernel on Frontier, the US Department of Energy’s first exascale supercomputer. We evaluate the performance, scaling, and trade-offs of (i) the computational kernel on AMD’s MI250x GPUs, (ii) weak scaling up to 4,096 MPI processes/GPUs or 512 nodes, (iii) parallel I/O writes using the ADIOS2 library bindings, and (iv) Jupyter Notebooks for interactive analysis. Results suggest that although Julia generates a reasonable LLVM-IR, a nearly 50% performance difference exists vs. native AMD HIP stencil codes when running on the GPUs. As expected, we observed near-zero overhead when using MPI and parallel I/O bindings for system-wide installed implementations. Consequently, Julia emerges as a compelling high-performance and high-productivity workflow composition language, as measured on the fastest supercomputer in the world.

View Conference

Research Organization:: Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States)

Sponsoring Organization:: USDOE Office of Science (SC)

DOE Contract Number:: AC05-00OR22725

OSTI ID:: 2217706

Resource Relation:: Conference: The International Conference for High Performance Computing, Networking, Storage, and Analysis - Denver, Colorado, United States of America - 11/12/2023 5:00:00 AM-11/17/2023 5:00:00 AM

Country of Publication:: United States

Language:: English

References (30)

RAJA: Portable Performance for Large-Scale Scientific Applications Beckingsale, David A.; Scogland, Thomas RW; Burmark, Jason 2019 IEEE/ACM International Workshop on Performance, Portability and Productivity in HPC (P3HPC) https://doi.org/10.1109/P3HPC49587.2019.00012	conference	November 2019
Workflows are the New Applications: Challenges in Performance, Portability, and Productivity Ben-Nun, Tal; Gamblin, Todd; Hollman, D. S. 2020 IEEE/ACM International Workshop on Performance, Portability and Productivity in HPC (P3HPC) https://doi.org/10.1109/P3HPC51967.2020.00011	conference	November 2020
Julia: A Fresh Approach to Numerical Computing Bezanson, Jeff; Edelman, Alan; Karpinski, Stefan SIAM Review, Vol. 59, Issue 1 https://doi.org/10.1137/141000671	journal	January 2017
Kokkos: Enabling manycore performance portability through polymorphic memory access patterns Carter Edwards, H.; Trott, Christian R.; Sunderland, Daniel Journal of Parallel and Distributed Computing, Vol. 74, Issue 12 https://doi.org/10.1016/j.jpdc.2014.07.003	journal	December 2014
Makie.jl: Flexible high-performance data visualization for Julia Danisch, Simon; Krumbiegel, Julius Journal of Open Source Software, Vol. 6, Issue 65 https://doi.org/10.21105/joss.03349	journal	September 2021
Stencil computation optimization and auto-tuning on state-of-the-art multicore architectures Datta, K.; Murphy, M.; Volkov, V. 2008 SC - International Conference for High Performance Computing, Networking, Storage and Analysis https://doi.org/10.1109/SC.2008.5222004	conference	November 2008
The future of scientific workflows Deelman, Ewa; Peterka, Tom; Altintas, Ilkay The International Journal of High Performance Computing Applications, Vol. 32, Issue 1 https://doi.org/10.1177/1094342017704893	journal	April 2017
Flexible Performant GEMM Kernels on GPUs Faingnaert, Thomas; Besard, Tim; De Sutter, Bjorn IEEE Transactions on Parallel and Distributed Systems, Vol. 33, Issue 9 https://doi.org/10.1109/TPDS.2021.3136457	journal	September 2022
A Community Roadmap for Scientific Workflows Research and Development da Silva, Rafael Ferreira; Casanova, Henri; Chard, Kyle 2021 IEEE Workshop on Workflows in Support of Large-Scale Science (WORKS) https://doi.org/10.1109/WORKS54523.2021.00016	conference	November 2021
A characterization of workflow management systems for extreme-scale applications Ferreira da Silva, Rafael; Filgueira, Rosa; Pietri, Ilia Future Generation Computer Systems, Vol. 75 https://doi.org/10.1016/j.future.2017.02.026	journal	October 2017
Productivity meets Performance: Julia on A64FX Giordano, Mose; Klower, Milan; Churavy, Valentin 2022 IEEE International Conference on Cluster Computing (CLUSTER) https://doi.org/10.1109/CLUSTER51413.2022.00072	conference	September 2022
FAIR Computational Workflows Goble, Carole; Cohen-Boulakia, Sarah; Soiland-Reyes, Stian Data Intelligence, Vol. 2, Issue 1-2 https://doi.org/10.1162/dint_a_00033	journal	January 2020
Introduction of Parallel GPGPU Acceleration Algorithms for the Solution of Radiative Transfer Godoy, William F.; Liu, Xu Numerical Heat Transfer, Part B: Fundamentals, Vol. 59, Issue 1 https://doi.org/10.1080/10407790.2010.541359	journal	January 2011
ADIOS 2: The Adaptable Input Output System. A framework for high-performance data management Godoy, William F.; Podhorszki, Norbert; Wang, Ruonan SoftwareX, Vol. 12 https://doi.org/10.1016/j.softx.2020.100561	journal	July 2020
Evaluating performance and portability of high-level programming models: Julia, Python/Numba, and Kokkos on exascale nodes Godoy, William F.; Valero-Lara, Pedro; Dettling, T. Elise 2023 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW) https://doi.org/10.1109/IPDPSW59300.2023.00068	conference	May 2023
High-performance code generation for stencil computations on GPU architectures Holewinski, Justin; Pouchet, Louis-Noël; Sadayappan, P. Proceedings of the 26th ACM international conference on Supercomputing - ICS '12 https://doi.org/10.1145/2304576.2304619	conference	January 2012
Benchmarking Julia’s Communication Performance: Is Julia HPC ready or Full HPC? Hunold, Sascha; Steiner, Sebastian 2020 IEEE/ACM Performance Modeling, Benchmarking and Simulation of High Performance Computer Systems (PMBS) https://doi.org/10.1109/pmbs51919.2020.00008	conference	November 2020
Efficient 3D stencil computations using CUDA Krotkiewski, Marcin; Dabrowski, Marcin Parallel Computing, Vol. 39, Issue 10 https://doi.org/10.1016/j.parco.2013.08.002	journal	October 2013
Numba: a LLVM-based Python JIT compiler Lam, Siu Kwan; Pitrou, Antoine; Seibert, Stanley Proceedings of the Second Workshop on the LLVM Compiler Infrastructure in HPC - LLVM '15 https://doi.org/10.1145/2833157.2833162	conference	January 2015
LLVM: A compilation framework for lifelong program analysis & transformation Lattner, C.; Adve, V. International Symposium on Code Generation and Optimization, 2004. CGO 2004. https://doi.org/10.1109/CGO.2004.1281665	conference	January 2004
Comparing Julia to Performance Portable Parallel Programming Models for HPC Lin, Wei-Chen; McIntosh-Smith, Simon 2021 International Workshop on Performance Modeling, Benchmarking and Simulation of High Performance Computer Systems (PMBS) https://doi.org/10.1109/PMBS54543.2021.00016	conference	November 2021
Bring the BitCODE-Moving Compute and Data in Distributed Heterogeneous Systems Lu, Wenbin; Pena, Luis E.; Shamis, Pavel 2022 IEEE International Conference on Cluster Computing (CLUSTER) https://doi.org/10.1109/CLUSTER51413.2022.00017	conference	September 2022
Complex Patterns in a Simple System Pearson, J. E. Science, Vol. 261, Issue 5118 https://doi.org/10.1126/science.261.5118.189	journal	July 1993
Fides: A General Purpose Data Model Library for Streaming Data Pugmire, David; Ross, Caitlin; Thompson, Nicholas Lecture Notes in Computer Science https://doi.org/10.1007/978-3-030-90539-2_34	book	January 2021
Cataloging the visible universe through Bayesian inference in Julia at petascale Regier, Jeffrey; Fischer, Keno; Pamnany, Kiran Journal of Parallel and Distributed Computing, Vol. 127 https://doi.org/10.1016/j.jpdc.2018.12.008	journal	May 2019
Large-Scale Simulation of Quantum Computational Chemistry on a New Sunway Supercomputer Shang, Honghui; Shen, Li; Fan, Yi SC22: International Conference for High Performance Computing, Networking, Storage and Analysis https://doi.org/10.1109/SC41404.2022.00019	conference	November 2022
Automatic, Efficient and Scalable Provenance Registration for FAIR HPC Workflows Sirvent, Raul; Conejero, Javier; Lordan, Francesc 2022 IEEE/ACM Workshop on Workflows in Support of Large-Scale Science (WORKS) https://doi.org/10.1109/WORKS56498.2022.00006	conference	November 2022
Python Workflows on HPC Systems Strasel, Dominik; Reusch, Philipp; Keuper, Janis 2020 IEEE/ACM 9th Workshop on Python for High-Performance and Scientific Computing (PyHPC) https://doi.org/10.1109/PyHPC51966.2020.00009	conference	November 2020
Extreme Heterogeneity 2018 - Productive Computational Science in the Era of Extreme Heterogeneity: Report for DOE ASCR Workshop on Extreme Heterogeneity Vetter, Jeffrey S.; Brightwell, Ron; Gokhale, Maya https://doi.org/10.2172/1473756	report	December 2018
GPU-Aware Non-contiguous Data Movement In Open MPI Wu, Wei; Bosilca, George; vandeVaart, Rolf Proceedings of the 25th ACM International Symposium on High-Performance Parallel and Distributed Computing - HPDC '16 https://doi.org/10.1145/2907294.2907317	conference	January 2016

Similar Records

Evaluating performance and portability of high-level programming models: Julia, Python/Numba, and Kokkos on exascale nodes

Conference · 2023 · OSTI ID:1994693

Godoy, William; Valero Lara, Pedro; Dettling, Elise; +7 more

JACC: Leveraging HPC Meta-Programming and Performance Portability with the Just-in-Time and LLVM-based Julia Language

Conference · 2024 · OSTI ID:2561236

Valero-Lara, Pedro; Godoy, William F.; Mankad, Het; +4 more

JACC.shared: Leveraging HPC Metaprogramming and Performance Portability for Computations That Use Shared Memory GPUs

Conference · 2024 · OSTI ID:2561231

Valero-Lara, Pedro; Godoy, William F.; Teranishi, Keita; +1 more

Julia as a unifying end-to-end workflow language on the Frontier exascale system

Citation Formats

References (30)

Similar Records

Related Subjects