The Pele Simulation Suite for Reacting Flows at Exascale

Conference · 01 February 2024

DOI:https://doi.org/10.1137/1.9781611977967.2· OSTI ID:2311310

de Frahan, Marc T. Henry ^[1]; Esclapez, Lucas ^[1]; Rood, Jon ^[1]; Wimer, Nicholas ^[1]; Mullowney, Paul ^[2]; Perry, Bruce A. ^[1]; Owen, Landon D. ^[3]; Sitaraman, Hariswaran ^[1]; Yellapantula, Shashank ^[1]; Hassanaly, Malik ^[1]; Rahimi, Mohammad J. ^[1]; Martin, Michael ^[1]; Doronina, Olga ^[1]; Appukuttan, Sreejith ^[1]; Reith, Martin ^[3]; Ge, Wenjun  ^[4]; Sankaran, Ramanan  ^[4]; Almgren, Ann ^[5]; Zhang, Weiqun ^[5]; Bell, John ^[5] ; Day, Marc  ^[1]; Chen, Jacqueline H. ^[3]

---

1. National Renewable Energy Laboratory (NREL)
2. Advanced Micro Devices (AMD)
3. Sandia National Laboratories (SNL)
4. ORNL
5. Lawrence Berkeley National Laboratory (LBNL)

In this work, we present the Pele suite of software tools for compressible and incompressible reacting flows. The Pele suite leverages several different libraries, notably AMReX and SUNDIALS, to achieve performance portability on heterogeneous computing architectures across the supercomputing landscape. The Pele suite is comprised of PeleC, a compressible reacting flow block-structured adaptive mesh refinement solver, PeleLMeX, a low-Mach number reacting flow block-structured adaptive mesh refinement solver, Pele-Physics, a library for transport, thermodynamics, finite rate chemistry, soot, spray and radiation physics. The objective of this paper is (i) to present the code development efforts necessary to achieve highly effective and scalable applications for exascale machines and (ii) to detail the performance results of the Combustion-Pele project applications on Oak Ridge National Laboratory's Frontier. We show good weak and strong scaling results for both PeleC and PeleLMeX up to more than 50 billion cells on more than 4096 Frontier graphics processing unit nodes. We also present a capability demonstration simulation of a dual-fuel pulse compression ignition engine (six adaptive mesh refinement levels, and 60 billion cells or 2.1 trillion degrees of freedom) on Frontier, to date one of the largest simulations performed on the first exascale-class supercomputer.

• Research Organization: Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States)
• Sponsoring Organization: USDOE Office of Science (SC)
• DOE Contract Number: AC05-00OR22725
• OSTI ID: 2311310
• Resource Relation: Conference: 2024 SIAM Conference on Parallel Processing for Scientific Computing - Baltimore, Maryland, United States of America - 3/5/2024 5:00:00 AM-3/8/2024 5:00:00 AM
• Country of Publication: United States
• Language: English