OpenACC offloading of the MFC compressible multiphase flow solver on AMD and NVIDIA GPUs

Conference · Fri Nov 01 04:00:00 EDT 2024

OSTI ID:2538543

Wilfong, Benjamin ^[1]; Radhakrishnan, Anand ^[1]; Le Berre, Henry ^[2]; Abbott, Steve ^[3]; Budiardja, Reuben ^[4]; Bryngelson, Spencer ^[2]

---

1. Georgia Institute of Technology
2. Georgia Tech
3. Hewlett Packard Enterprise
4. ORNL

GPUs are the heart of the latest generations of supercomputers. We efficiently accelerate a compressible multiphase flow solver via OpenACC on NVIDIA and AMD Instinct GPUs. Optimization is accomplished by specifying the directive clauses gang vector and collapse. Further speedups of six and ten times are achieved by packing user-defined types into coalesced multidimensional arrays and manual inlining via metaprogramming. Additional optimizations yield seven-times speedup of array packing and thirty-times speedup of select kernels on Frontier. Weak scaling efficiencies of 97% and 95% are observed when scaling to 50% of Summit and 87% of Frontier. Strong scaling efficiencies of 84% and 81% are observed when increasing the device count by a factor of 8 and 16 on V100 and MI250X hardware. The strong scaling efficiency of AMD’s MI250X increases to 92% when increasing the device count by a factor of 16 when GPU-aware MPI is used for communication.

• Research Organization: Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States)
• Sponsoring Organization: USDOE; USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR) (SC-21)
• DOE Contract Number: AC05-00OR22725
• OSTI ID: 2538543
• Country of Publication: United States
• Language: English