Accelerating Time Integration for the Shallow Water Equations on the Sphere Using GPUs
Abstract
The push towards larger and larger computational platforms has made it possible for climate simulations to resolve climate dynamics across multiple spatial and temporal scales. This direction in climate simulation has created a strong need to develop scalable timestepping methods capable of accelerating throughput on high performance computing. This study details the recent advances in the implementation of implicit time stepping of the spectral element dynamical core within the United States Department of Energy (DOE) Accelerated Climate Model for Energy (ACME) on graphical processing units (GPU) based machines. We demonstrate how solvers in the Trilinos project are interfaced with ACME and GPU kernels to increase computational speed of the residual calculations in the implicit time stepping method for the atmosphere dynamics. We demonstrate the optimization gains and data structure reorganization that facilitates the performance improvements.
- Authors:
-
- Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States)
- Sandia National Lab. (SNL-NM), Albuquerque, NM (United States)
- Publication Date:
- Research Org.:
- Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States). Oak Ridge Leadership Computing Facility (OLCF); Sandia National Laboratories (SNL), Albuquerque, NM, and Livermore, CA (United States)
- Sponsoring Org.:
- USDOE Office of Science (SC), Biological and Environmental Research (BER); USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR)
- OSTI Identifier:
- 1214700
- Alternate Identifier(s):
- OSTI ID: 1265329
- Grant/Contract Number:
- AC04-94AL85000; AC05-00OR22725
- Resource Type:
- Accepted Manuscript
- Journal Name:
- Procedia Computer Science
- Additional Journal Information:
- Journal Volume: 51; Journal Issue: C; Journal ID: ISSN 1877-0509
- Publisher:
- Elsevier
- Country of Publication:
- United States
- Language:
- English
- Subject:
- 54 ENVIRONMENTAL SCIENCES; 97 MATHEMATICS AND COMPUTING; Implicit Timestepping; GPU; Trilinos
Citation Formats
Archibald, R., Evans, K. J., and Salinger, A. Accelerating Time Integration for the Shallow Water Equations on the Sphere Using GPUs. United States: N. p., 2015.
Web. doi:10.1016/j.procs.2015.05.470.
Archibald, R., Evans, K. J., & Salinger, A. Accelerating Time Integration for the Shallow Water Equations on the Sphere Using GPUs. United States. https://doi.org/10.1016/j.procs.2015.05.470
Archibald, R., Evans, K. J., and Salinger, A. Mon .
"Accelerating Time Integration for the Shallow Water Equations on the Sphere Using GPUs". United States. https://doi.org/10.1016/j.procs.2015.05.470. https://www.osti.gov/servlets/purl/1214700.
@article{osti_1214700,
title = {Accelerating Time Integration for the Shallow Water Equations on the Sphere Using GPUs},
author = {Archibald, R. and Evans, K. J. and Salinger, A.},
abstractNote = {The push towards larger and larger computational platforms has made it possible for climate simulations to resolve climate dynamics across multiple spatial and temporal scales. This direction in climate simulation has created a strong need to develop scalable timestepping methods capable of accelerating throughput on high performance computing. This study details the recent advances in the implementation of implicit time stepping of the spectral element dynamical core within the United States Department of Energy (DOE) Accelerated Climate Model for Energy (ACME) on graphical processing units (GPU) based machines. We demonstrate how solvers in the Trilinos project are interfaced with ACME and GPU kernels to increase computational speed of the residual calculations in the implicit time stepping method for the atmosphere dynamics. We demonstrate the optimization gains and data structure reorganization that facilitates the performance improvements.},
doi = {10.1016/j.procs.2015.05.470},
journal = {Procedia Computer Science},
number = C,
volume = 51,
place = {United States},
year = {Mon Jun 01 00:00:00 EDT 2015},
month = {Mon Jun 01 00:00:00 EDT 2015}
}
Free Publicly Available Full Text
Publisher's Version of Record
Other availability
Search WorldCat to find libraries that may hold this journal
Cited by: 2 works
Citation information provided by
Web of Science
Web of Science
Save to My Library
You must Sign In or Create an Account in order to save documents to your library.
Works referenced in this record:
Compiler support for the Fortran 2003 and 2008 standards revision 10
journal, July 2012
- Chivers, Ian D.; Sleightholme, Jane
- ACM SIGPLAN Fortran Forum, Vol. 31, Issue 2
Heterogeneous multiscale methods for stiff ordinary differential equations
journal, May 2005
- Engquist, Bjorn; Tsai, Yen-Hsi
- Mathematics of Computation, Vol. 74, Issue 252
A Scalable and Adaptable Solution Framework within Components of the Community Climate System Model
book, January 2009
- Evans, Katherine J.; Rouson, Damian W. I.; Salinger, Andrew G.
- Lecture Notes in Computer Science
Accuracy Analysis of a Spectral Element Atmospheric Model Using a Fully Implicit Solution Framework
journal, August 2010
- Evans, Katherine J.; Taylor, Mark A.; Drake, John B.
- Monthly Weather Review, Vol. 138, Issue 8
A spectral element shallow water model on spherical geodesic grids
journal, January 2001
- Giraldo, Francis X.
- International Journal for Numerical Methods in Fluids, Vol. 35, Issue 8
An initial-value problem for testing numerical models of the global shallow-water equations
journal, January 2004
- Galewsky, Joseph; Scott, Richard K.; Polvani, Lorenzo M.
- Tellus A: Dynamic Meteorology and Oceanography, Vol. 56, Issue 5
An Asymptotic Parallel-in-Time Method for Highly Oscillatory PDEs
journal, January 2014
- Haut, Terry; Wingate, Beth
- SIAM Journal on Scientific Computing, Vol. 36, Issue 2
An overview of the Trilinos project
journal, September 2005
- Heroux, Michael A.; Phipps, Eric T.; Salinger, Andrew G.
- ACM Transactions on Mathematical Software, Vol. 31, Issue 3
A Spectral Deferred Correction Method Applied to the Shallow Water Equations on a Sphere
journal, October 2013
- Jia, Jun; Hill, Judith C.; Evans, Katherine J.
- Monthly Weather Review, Vol. 141, Issue 10
Jacobian-free Newton–Krylov methods: a survey of approaches and applications
journal, January 2004
- Knoll, D. A.; Keyes, D. E.
- Journal of Computational Physics, Vol. 193, Issue 2
Comparative performance of exponential, implicit, and explicit integrators for stiff systems of ODEs
journal, March 2013
- Loffeld, J.; Tokman, M.
- Journal of Computational and Applied Mathematics, Vol. 241
Scalable parallel programming with CUDA
journal, March 2008
- Nickolls, John; Buck, Ian; Garland, Michael
- Queue, Vol. 6, Issue 2
Algorithmic improvements for schemes using the ADER time discretization
journal, June 2013
- Norman, M. R.
- Journal of Computational Physics, Vol. 243
Multi-moment ADER-Taylor methods for systems of conservation laws with source terms in one dimension
journal, August 2012
- Norman, M. R.; Finkel, H.
- Journal of Computational Physics, Vol. 231, Issue 20
Newton-GMRES Preconditioning for Discontinuous Galerkin Discretizations of the Navier–Stokes Equations
journal, January 2008
- Persson, P. -O.; Peraire, J.
- SIAM Journal on Scientific Computing, Vol. 30, Issue 6
Conservative Finite-Difference Approximations of the Primitive Equations on Quasi-Uniform Spherical Grids
journal, February 1972
- Sadourny, Robert
- Monthly Weather Review, Vol. 100, Issue 2
Multirate Explicit Adams Methods for Time Integration of Conservation Laws
journal, August 2008
- Sandu, Adrian; Constantinescu, Emil M.
- Journal of Scientific Computing, Vol. 38, Issue 2
The Spectral Element Method for the Shallow Water Equations on the Sphere
journal, January 1997
- Taylor, Mark; Tribbia, Joseph; Iskandarani, Mohamed
- Journal of Computational Physics, Vol. 130, Issue 1
A standard test set for numerical approximations to the shallow water equations in spherical geometry
journal, September 1992
- Williamson, David L.; Drake, John B.; Hack, James J.
- Journal of Computational Physics, Vol. 102, Issue 1
Parallel multilevel methods for implicit solution of shallow water equations with nonsmooth topography on the cubed-sphere
journal, April 2011
- Yang, Chao; Cai, Xiao-Chuan
- Journal of Computational Physics, Vol. 230, Issue 7
The Spectral Element Method for the Shallow Water Equations on the Sphere
journal, January 1997
- Taylor, Mark; Tribbia, Joseph; Iskandarani, Mohamed
- Journal of Computational Physics, Vol. 130, Issue 1
Works referencing / citing this record:
Acceleration of the IMplicit–EXplicit nonhydrostatic unified model of the atmosphere on manycore processors
journal, October 2017
- Abdi, Daniel S.; Giraldo, Francis X.; Constantinescu, Emil M.
- The International Journal of High Performance Computing Applications, Vol. 33, Issue 2
Acceleration of the Implicit-Explicit Non-hydrostatic Unified Model of the Atmosphere (NUMA) on Manycore Processors
preprint, January 2017
- Abdi, Daniel S.; Giraldo, Francis X.; Constantinescu, Emil M.
- arXiv