skip to main content
DOE PAGES title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Towards performance-portability of the Albany Finite Element analysis code using the Kokkos library of Trilinos

Abstract

Performance portability on heterogeneous high-performance computing (HPC) systems is a major challenge faced today by code developers: parallel code needs to execute correctly as well as with high performance on machines with different architectures, operating systems, and software libraries. The Finite Element Method (FEM) is a popular and flexible method for discretizing partial differential equations arising in a wide variety of scientific, engineering, and industry applications that require HPC. This paper presents some preliminary results pertaining to our development of a performance portable implementation of the FEM-based Albany code. Performance portability is achieved using the Kokkos library of Trilinos. We present performance results for two different physics simulations modules in Albany: the Aeras global atmosphere dynamical code and the FELIX land-ice solver. As a result, numerical experiments show that our single code implementation gives reasonable performance across two multi-core/many-core architectures: NVIDIA GPUs and multi-core CPUs.

Authors:
 [1];  [2];  [2];  [3];  [2];  [2]
  1. Los Alamos National Lab. (LANL), Los Alamos, NM (United States)
  2. Sandia National Lab. (SNL-NM), Albuquerque, NM (United States)
  3. Sandia National Lab. (SNL-CA), Livermore, CA (United States)
Publication Date:
Research Org.:
Sandia National Lab. (SNL-NM), Albuquerque, NM (United States); Sandia National Lab. (SNL-CA), Livermore, CA (United States); Los Alamos National Lab. (LANL), Los Alamos, NM (United States)
Sponsoring Org.:
USDOE National Nuclear Security Administration (NNSA); SNL Laboratory Directed Research and Development (LDRD) Program; USDOE Office of Science (SC), Biological and Environmental Research (BER) (SC-23)
OSTI Identifier:
1421623
Alternate Identifier(s):
OSTI ID: 1253127
Report Number(s):
SAND-2016-1143J; SAND-2017-8843J
619204
Grant/Contract Number:  
AC04-94AL85000; NA0003525
Resource Type:
Accepted Manuscript
Journal Name:
International Journal of HPC Applications
Additional Journal Information:
Journal Name: International Journal of HPC Applications
Country of Publication:
United States
Language:
English
Subject:
97 MATHEMATICS AND COMPUTING; performance portability; many-core programming; finite element code; climate simulations; Kokkos Trilinos library; Kokkos library

Citation Formats

Demeshko, Irina, Salinger, Andrew G., Spotz, William F., Tezaur, Irina Kalashnikova, Guba, Oksana, and Heroux, Michael A. Towards performance-portability of the Albany Finite Element analysis code using the Kokkos library of Trilinos. United States: N. p., 2016. Web. doi:10.1177/1094342017749957.
Demeshko, Irina, Salinger, Andrew G., Spotz, William F., Tezaur, Irina Kalashnikova, Guba, Oksana, & Heroux, Michael A. Towards performance-portability of the Albany Finite Element analysis code using the Kokkos library of Trilinos. United States. doi:10.1177/1094342017749957.
Demeshko, Irina, Salinger, Andrew G., Spotz, William F., Tezaur, Irina Kalashnikova, Guba, Oksana, and Heroux, Michael A. Mon . "Towards performance-portability of the Albany Finite Element analysis code using the Kokkos library of Trilinos". United States. doi:10.1177/1094342017749957. https://www.osti.gov/servlets/purl/1421623.
@article{osti_1421623,
title = {Towards performance-portability of the Albany Finite Element analysis code using the Kokkos library of Trilinos},
author = {Demeshko, Irina and Salinger, Andrew G. and Spotz, William F. and Tezaur, Irina Kalashnikova and Guba, Oksana and Heroux, Michael A.},
abstractNote = {Performance portability on heterogeneous high-performance computing (HPC) systems is a major challenge faced today by code developers: parallel code needs to execute correctly as well as with high performance on machines with different architectures, operating systems, and software libraries. The Finite Element Method (FEM) is a popular and flexible method for discretizing partial differential equations arising in a wide variety of scientific, engineering, and industry applications that require HPC. This paper presents some preliminary results pertaining to our development of a performance portable implementation of the FEM-based Albany code. Performance portability is achieved using the Kokkos library of Trilinos. We present performance results for two different physics simulations modules in Albany: the Aeras global atmosphere dynamical code and the FELIX land-ice solver. As a result, numerical experiments show that our single code implementation gives reasonable performance across two multi-core/many-core architectures: NVIDIA GPUs and multi-core CPUs.},
doi = {10.1177/1094342017749957},
journal = {International Journal of HPC Applications},
number = ,
volume = ,
place = {United States},
year = {2016},
month = {2}
}

Journal Article:
Free Publicly Available Full Text
Publisher's Version of Record

Citation Metrics:
Cited by: 2 works
Citation information provided by
Web of Science

Figures / Tables:

Figure 1 Figure 1: Phalanx graph of the residual evaluator for the FELIX land-ice physics set in Albany.

Save / Share:

Works referenced in this record:

Kokkos: Enabling manycore performance portability through polymorphic memory access patterns
journal, December 2014

  • Carter Edwards, H.; Trott, Christian R.; Sunderland, Daniel
  • Journal of Parallel and Distributed Computing, Vol. 74, Issue 12
  • DOI: 10.1016/j.jpdc.2014.07.003

Solving PDEs with Intrepid
journal, January 2012

  • Bochev, P.; Edwards, H. C.; Kirby, R. C.
  • Scientific Programming, Vol. 20, Issue 2
  • DOI: 10.1155/2012/403902

Albany/FELIX : a parallel, scalable and robust, finite element, first-order Stokes approximation ice sheet solver built for advanced analysis
journal, January 2015

  • Tezaur, I. K.; Perego, M.; Salinger, A. G.
  • Geoscientific Model Development, Vol. 8, Issue 4
  • DOI: 10.5194/gmd-8-1197-2015

Accuracy Analysis of a Spectral Element Atmospheric Model Using a Fully Implicit Solution Framework
journal, August 2010

  • Evans, Katherine J.; Taylor, Mark A.; Drake, John B.
  • Monthly Weather Review, Vol. 138, Issue 8
  • DOI: 10.1175/2010MWR3288.1

Quantum computer aided design simulation and optimization of semiconductor quantum dots
journal, October 2013

  • Gao, X.; Nielsen, E.; Muller, R. P.
  • Journal of Applied Physics, Vol. 114, Issue 16
  • DOI: 10.1063/1.4825209

Albany: Using Component-Based Design to Develop a Flexible, Generic Multiphysics Analysis code
journal, January 2016

  • Salinger, Andrew G.; Bartlett, Roscoe A.; Bradley, Andrew M.
  • International Journal for Multiscale Computational Engineering, Vol. 14, Issue 4
  • DOI: 10.1615/IntJMultCompEng.2016017040

Climate Modeling with Spectral Elements
journal, December 2006

  • Baer, Ferdinand; Wang, Houjun; Tribbia, Joseph J.
  • Monthly Weather Review, Vol. 134, Issue 12
  • DOI: 10.1175/MWR3360.1

A baroclinic instability test case for atmospheric model dynamical cores
journal, October 2006

  • Jablonowski, Christiane; Williamson, David L.
  • Quarterly Journal of the Royal Meteorological Society, Vol. 132, Issue 621C
  • DOI: 10.1256/qj.06.12

CAM-SE: A scalable spectral element dynamical core for the Community Atmosphere Model
journal, November 2011

  • Dennis, John M.; Edwards, Jim; Evans, Katherine J.
  • The International Journal of High Performance Computing Applications, Vol. 26, Issue 1
  • DOI: 10.1177/1094342011428142

Aeras: A Next Generation Global Atmosphere Model
journal, January 2015


The spectral element method on variable resolution grids: evaluating grid sensitivity and resolution-aware numerical viscosity
journal, January 2014

  • Guba, O.; Taylor, M. A.; Ullrich, P. A.
  • Geoscientific Model Development Discussions, Vol. 7, Issue 3
  • DOI: 10.5194/gmdd-7-4081-2014

Automating Embedded Analysis Capabilities and Managing Software Complexity in Multiphysics Simulation, Part I: Template-Based Generic Programming
journal, January 2012

  • Pawlowski, Roger P.; Phipps, Eric T.; Salinger, Andrew G.
  • Scientific Programming, Vol. 20, Issue 2
  • DOI: 10.1155/2012/202071

A standard test set for numerical approximations to the shallow water equations in spherical geometry
journal, September 1992