Towards performance-portability of the Albany Finite Element analysis code using the Kokkos library of Trilinos

Demeshko, Irina; Salinger, Andrew G.; Spotz, William F.; Tezaur, Irina Kalashnikova; Guba, Oksana; Heroux, Michael A.

doi:10.1177/1094342017749957

Towards performance-portability of the Albany Finite Element analysis code using the Kokkos library of Trilinos

Journal Article · 31 January 2016 · International Journal of HPC Applications

DOI:https://doi.org/10.1177/1094342017749957· OSTI ID:1253127

Demeshko, Irina ^[1]; Salinger, Andrew G. ^[2]; Spotz, William F. ^[2]; Tezaur, Irina Kalashnikova ^[3]; Guba, Oksana ^[2]; Heroux, Michael A. ^[2]

Los Alamos National Lab. (LANL), Los Alamos, NM (United States)
Sandia National Lab. (SNL-NM), Albuquerque, NM (United States)
Sandia National Lab. (SNL-CA), Livermore, CA (United States)

Performance portability on heterogeneous high-performance computing (HPC) systems is a major challenge faced today by code developers: parallel code needs to execute correctly as well as with high performance on machines with different architectures, operating systems, and software libraries. The Finite Element Method (FEM) is a popular and flexible method for discretizing partial differential equations arising in a wide variety of scientific, engineering, and industry applications that require HPC. This paper presents some preliminary results pertaining to our development of a performance portable implementation of the FEM-based Albany code. Performance portability is achieved using the Kokkos library of Trilinos. We present performance results for two different physics simulations modules in Albany: the Aeras global atmosphere dynamical code and the FELIX land-ice solver. As a result, numerical experiments show that our single code implementation gives reasonable performance across two multi-core/many-core architectures: NVIDIA GPUs and multi-core CPUs.

View Accepted Manuscript (DOE)

Research Organization:: Sandia National Laboratories (SNL-NM), Albuquerque, NM (United States)

Sponsoring Organization:: USDOE National Nuclear Security Administration (NNSA)

Grant/Contract Number:: AC04-94AL85000

OSTI ID:: 1253127

Alternate ID(s):: OSTI ID: 1421623

Report Number(s):: SAND--2016-1143J; 619204

Journal Information:: International Journal of HPC Applications, Journal Name: International Journal of HPC Applications

Country of Publication:: United States

Language:: English

References (18)

A stabilized assumed deformation gradient finite element formulation for strongly coupled poromechanical simulations at finite strain: STABILIZED F-BAR FINITE ELEMENT FORMULATION FOR POROMECHANICS Sun, WaiChing; Ostien, Jakob T.; Salinger, Andrew G. International Journal for Numerical and Analytical Methods in Geomechanics https://doi.org/10.1002/nag.2161	journal	January 2013
A standard test set for numerical approximations to the shallow water equations in spherical geometry Williamson, David L.; Drake, John B.; Hack, James J. Journal of Computational Physics, Vol. 102, Issue 1 https://doi.org/10.1016/S0021-9991(05)80016-6	journal	September 1992
Kokkos: Enabling manycore performance portability through polymorphic memory access patterns Carter Edwards, H.; Trott, Christian R.; Sunderland, Daniel Journal of Parallel and Distributed Computing, Vol. 74, Issue 12 https://doi.org/10.1016/j.jpdc.2014.07.003	journal	December 2014
Aeras: A Next Generation Global Atmosphere Model Spotz, William F.; Smith, Thomas M.; Demeshko, Irina P. Procedia Computer Science, Vol. 51 https://doi.org/10.1016/j.procs.2015.05.478	journal	January 2015
Quantum computer aided design simulation and optimization of semiconductor quantum dots Gao, X.; Nielsen, E.; Muller, R. P. Journal of Applied Physics, Vol. 114, Issue 16 https://doi.org/10.1063/1.4825209	journal	October 2013
The OpenCL specification Munshi, Aaftab 2009 IEEE Hot Chips 21 Symposium (HCS) https://doi.org/10.1109/HOTCHIPS.2009.7478342	conference	August 2009
ParalleX An Advanced Parallel Execution Model for Scaling-Impaired Applications Kaiser, Hartmut; Brodowicz, Maciek; Sterling, Thomas 2009 International Conference on Parallel Processing Workshops (ICPPW) https://doi.org/10.1109/ICPPW.2009.14	conference	September 2009
PyOP2: A High-Level Framework for Performance-Portable Simulations on Unstructured Meshes Rathgeber, Florian; Markall, Graham R.; Mitchell, Lawrence 2012 SC Companion: High Performance Computing, Networking, Storage and Analysis (SCC), 2012 SC Companion: High Performance Computing, Networking Storage and Analysis https://doi.org/10.1109/SC.Companion.2012.134	conference	November 2012
Firedrake: Automating the Finite Element Method by Composing Abstractions Rathgeber, Florian; Ham, David A.; Mitchell, Lawrence ACM Transactions on Mathematical Software, Vol. 43, Issue 3 https://doi.org/10.1145/2998441	journal	December 2016
Automating Embedded Analysis Capabilities and Managing Software Complexity in Multiphysics Simulation, Part I: Template-Based Generic Programming Pawlowski, Roger P.; Phipps, Eric T.; Salinger, Andrew G. Scientific Programming, Vol. 20, Issue 2 https://doi.org/10.1155/2012/202071	journal	January 2012
Solving PDEs with Intrepid Bochev, P.; Edwards, H. C.; Kirby, R. C. Scientific Programming, Vol. 20, Issue 2 https://doi.org/10.1155/2012/403902	journal	January 2012
Accuracy Analysis of a Spectral Element Atmospheric Model Using a Fully Implicit Solution Framework Evans, Katherine J.; Taylor, Mark A.; Drake, John B. Monthly Weather Review, Vol. 138, Issue 8 https://doi.org/10.1175/2010MWR3288.1	journal	August 2010
Climate Modeling with Spectral Elements Baer, Ferdinand; Wang, Houjun; Tribbia, Joseph J. Monthly Weather Review, Vol. 134, Issue 12 https://doi.org/10.1175/MWR3360.1	journal	December 2006
CAM-SE: A scalable spectral element dynamical core for the Community Atmosphere Model Dennis, John M.; Edwards, Jim; Evans, Katherine J. The International Journal of High Performance Computing Applications, Vol. 26, Issue 1 https://doi.org/10.1177/1094342011428142	journal	November 2011
A baroclinic instability test case for atmospheric model dynamical cores Jablonowski, Christiane; Williamson, David L. Quarterly Journal of the Royal Meteorological Society, Vol. 132, Issue 621C https://doi.org/10.1256/qj.06.12	journal	October 2006
Albany: Using Component-Based Design to Develop a Flexible, Generic Multiphysics Analysis code Salinger, Andrew G.; Bartlett, Roscoe A.; Bradley, Andrew M. International Journal for Multiscale Computational Engineering, Vol. 14, Issue 4 https://doi.org/10.1615/IntJMultCompEng.2016017040	journal	January 2016
Albany/FELIX : a parallel, scalable and robust, finite element, first-order Stokes approximation ice sheet solver built for advanced analysis Tezaur, I. K.; Perego, M.; Salinger, A. G. Geoscientific Model Development, Vol. 8, Issue 4 https://doi.org/10.5194/gmd-8-1197-2015	journal	January 2015
The spectral element method on variable resolution grids: evaluating grid sensitivity and resolution-aware numerical viscosity Guba, O.; Taylor, M. A.; Ullrich, P. A. Geoscientific Model Development Discussions, Vol. 7, Issue 3 https://doi.org/10.5194/gmdd-7-4081-2014	journal	January 2014

Cited By (4)

Evaluation of performance portability frameworks for the implementation of a particle‐in‐cell code Artigues, Victor; Kormann, Katharina; Rampp, Markus Concurrency and Computation: Practice and Experience, Vol. 32, Issue 11 https://doi.org/10.1002/cpe.5640	journal	December 2019
A Study on the Performance Portability of the Finite Element Assembly Process Within the Albany Land Ice Solver Watkins, Jerry; Tezaur, Irina; Demeshko, Irina Numerical Methods for Flows: FEF 2017 Selected Contributions, p. 177-188 https://doi.org/10.1007/978-3-030-30705-9_16	book	February 2020
MPAS-Albany Land Ice (MALI): a variable-resolution ice sheet model for Earth system modeling using Voronoi grids Hoffman, Matthew J.; Perego, Mauro; Price, Stephen F. Geoscientific Model Development, Vol. 11, Issue 9 https://doi.org/10.5194/gmd-11-3747-2018	journal	January 2018
HOMMEXX 1.0: a performance-portable atmospheric dynamical core for the Energy Exascale Earth System Model Bertagna, Luca; Deakin, Michael; Guba, Oksana Geoscientific Model Development, Vol. 12, Issue 4 https://doi.org/10.5194/gmd-12-1423-2019	journal	January 2019