Performance Portable Graphics Processing Unit Acceleration of a High-Order Finite Element Multiphysics Application

Stitt, Thomas; Belcher, Kristi; Campos, Alejandro; Kolev, Tzanio; Mocz, Philip; Rieben, Robert N.; Skinner, Aaron; Tomov, Vladimir; Vargas, Arturo; Weiss, Kenneth

doi:10.1115/1.4064493

Performance Portable Graphics Processing Unit Acceleration of a High-Order Finite Element Multiphysics Application

Journal Article · Thu Feb 08 23:00:00 EST 2024 · Journal of Fluids Engineering

DOI:https://doi.org/10.1115/1.4064493· OSTI ID:2349615

Stitt, Thomas ^[1]; Belcher, Kristi ^[1]; Campos, Alejandro ^[1]; Kolev, Tzanio ^[1]; Mocz, Philip ^[1]; Rieben, Robert N. ^[1]; Skinner, Aaron ^[1]; Tomov, Vladimir ^[1]; Vargas, Arturo ^[1]; Weiss, Kenneth ^[1]

Lawrence Livermore National Laboratory (LLNL), Livermore, CA (United States)

The Lawrence Livermore National Laboratory (LLNL) will soon have in place the El Capitan exascale supercomputer, based on advanced micro devices (AMD) graphics processing units (GPUs). As part of a multiyear effort under the National Nuclear Security Administration (NNSA) Advanced Simulation and Computing (ASC) program, we have been developing marbl, a next generation, performance portable multiphysics application based on high-order finite elements. In previous years, we successfully ported the Arbitrary Lagrangian–Eulerian (ALE), multimaterial, compressible flow capabilities of marbl to nvidia GPUs as described in Vargas et al. Here, in this paper, we describe our ongoing effort in extending marbl's GPU capabilities with additional physics, including multigroup radiation diffusion and thermonuclear burn for high energy density physics (HEDP) and fusion modeling. We also describe how our portability abstraction approach based on the raja Portability Suite and the mfem finite element discretization library has enabled us to achieve high performance on AMD based GPUs with minimal effort in hardware-specific porting. Throughout this work, we highlight numerical and algorithmic developments that were required to achieve GPU performance.

View Accepted Manuscript (DOE)

Research Organization:: Lawrence Livermore National Laboratory (LLNL), Livermore, CA (United States)

Sponsoring Organization:: USDOE National Nuclear Security Administration (NNSA)

Grant/Contract Number:: AC52-07NA27344

OSTI ID:: 2349615

Report Number(s):: LLNL-JRNL-851314; 1077866

Journal Information:: Journal of Fluids Engineering, Journal Name: Journal of Fluids Engineering Journal Issue: 4 Vol. 146; ISSN 0098-2202

Publisher:: ASMECopyright Statement

Country of Publication:: United States

Language:: English

References (16)

Monotonicity in high-order curvilinear finite element arbitrary Lagrangian-Eulerian remap: MONOTONICITY IN HIGH-ORDER CURVILINEAR FINITE ELEMENT ALE REMAP Anderson, R. W.; Dobrev, V. A.; Kolev, Tz. V. International Journal for Numerical Methods in Fluids, Vol. 77, Issue 5 https://doi.org/10.1002/fld.3965	journal	October 2014
Multi-material closure model for high-order finite element Lagrangian hydrodynamics: CLOSURE MODEL FOR HIGH-ORDER FINITE ELEMENT HYDRODYNAMICS Dobrev, V. A.; Kolev, T. V.; Rieben, R. N. International Journal for Numerical Methods in Fluids, Vol. 82, Issue 10 https://doi.org/10.1002/fld.4236	journal	April 2016
On condition numbers in hp-FEM with Gauss–Lobatto-based shape functions Melenk, J. M. Journal of Computational and Applied Mathematics, Vol. 139, Issue 1 https://doi.org/10.1016/S0377-0427(01)00391-0	journal	February 2002
MFEM: A modular finite element methods library Anderson, Robert; Andrej, Julian; Barker, Andrew Computers & Mathematics with Applications, Vol. 81 https://doi.org/10.1016/j.camwa.2020.06.009	journal	January 2021
Matrix-free subcell residual distribution for Bernstein finite element discretizations of linear advection equations Hajduk, Hennes; Kuzmin, Dmitri; Kolev, Tzanio Computer Methods in Applied Mechanics and Engineering, Vol. 359 https://doi.org/10.1016/j.cma.2019.112658	journal	February 2020
Adaptive moment-of-fluid method Ahn, Hyung Taek; Shashkov, Mikhail Journal of Computational Physics, Vol. 228, Issue 8 https://doi.org/10.1016/j.jcp.2008.12.031	journal	May 2009
High-order local maximum principle preserving (MPP) discontinuous Galerkin finite element method for the transport equation Anderson, R.; Dobrev, V.; Kolev, Tz. Journal of Computational Physics, Vol. 334 https://doi.org/10.1016/j.jcp.2016.12.031	journal	April 2017
A high energy density shock driven Kelvin–Helmholtz shear layer experiment Hurricane, O. A.; Hansen, J. F.; Robey, H. F. Physics of Plasmas, Vol. 16, Issue 5 https://doi.org/10.1063/1.3096790	journal	May 2009
Electron-ion equilibration in a partially degenerate plasma Brysk, H. Plasma Physics, Vol. 16, Issue 10 https://doi.org/10.1088/0032-1028/16/10/005	journal	October 1974
Crash: a Block-Adaptive-Mesh code for Radiative Shock Hydrodynamics—Implementation and Verification van der Holst, B.; Tóth, G.; Sokolov, I. V. The Astrophysical Journal Supplement Series, Vol. 194, Issue 2 https://doi.org/10.1088/0067-0049/194/2/23	journal	May 2011
The RAGE radiation-hydrodynamic code Gittings, Michael; Weaver, Robert; Clover, Michael Computational Science & Discovery, Vol. 1, Issue 1 https://doi.org/10.1088/1749-4699/1/1/015005	journal	October 2008
Parallel Auxiliary Space AMG Solver for $H(div)$ Problems Kolev, Tzanio V.; Vassilevski, Panayot S. SIAM Journal on Scientific Computing, Vol. 34, Issue 6 https://doi.org/10.1137/110859361	journal	January 2012
Algebraic Hybridization and Static Condensation with Application to Scalable $H$(div) Preconditioning Dobrev, V.; Kolev, T.; Lee, C. S. SIAM Journal on Scientific Computing, Vol. 41, Issue 3 https://doi.org/10.1137/17M1132562	journal	January 2019
Low-Order Preconditioning for the High-Order Finite Element de Rham Complex Pazner, Will; Kolev, Tzanio; Dohrmann, Clark R. SIAM Journal on Scientific Computing, Vol. 45, Issue 2 https://doi.org/10.1137/22M1486534	journal	April 2023
Umpire: Application-focused management and coordination of complex hierarchical memory Beckingsale, D. A.; McFadden, M. J.; Dahm, J. P. S. IBM Journal of Research and Development, Vol. 64, Issue 3/4 https://doi.org/10.1147/JRD.2019.2954403	journal	May 2020
Matrix-free approaches for GPU acceleration of a high-order finite element hydrodynamics application using MFEM, Umpire, and RAJA Vargas, Arturo; Stitt, Thomas M.; Weiss, Kenneth The International Journal of High Performance Computing Applications, Vol. 36, Issue 4 https://doi.org/10.1177/10943420221100262	journal	May 2022

Similar Records

Optimization and Portability of a Fusion OpenACC-based FORTRAN HPC Code from NVIDIA to AMD GPUs

Conference · Sat Jul 01 00:00:00 EDT 2023 · OSTI ID:2301616

GPU algorithms for Efficient Exascale Discretizations

Journal Article · Sat Sep 25 00:00:00 EDT 2021 · Parallel Computing · OSTI ID:1845216

Enhancements supporting IC usage of PEM libraries on next-gen platforms

Technical Report · Wed Jun 23 00:00:00 EDT 2021 · OSTI ID:1798430

Related Subjects

70 PLASMA PHYSICS AND FUSION TECHNOLOGY
97 MATHEMATICS AND COMPUTING

Performance Portable Graphics Processing Unit Acceleration of a High-Order Finite Element Multiphysics Application

Citation Formats

References (16)

Similar Records

Related Subjects