Skip to main content
OSTI.GOVU.S. Department of Energy Office of Scientific and Technical Information
U.S. Department of Energy
Office of Scientific and Technical Information
 

Performance analysis of fully explicit and fully implicit solvers within a spectral element shallow-water atmosphere model

Journal Article · · International Journal of High Performance Computing Applications
 [1];  [1];  [2];  [1];  [3];  [2];  [1]
  1. Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States)
  2. Lawrence Livermore National Lab. (LLNL), Livermore, CA (United States)
  3. Sandia National Lab. (SNL-NM), Albuquerque, NM (United States)
+ Show Author Affiliations
In this study, explicit Runge–Kutta methods and implicit multistep methods utilizing a Newton–Krylov nonlinear solver are evaluated for a range of configurations of the shallow-water dynamical core of the spectral element community atmosphere model to evaluate their computational performance. These configurations are designed to explore the attributes of each method under different but relevant model usage scenarios including varied spectral order within an element, static regional refinement, and scaling to the largest problem sizes. This analysis is performed within the shallow-water dynamical core option of a full climate model code base to enable a wealth of simulations for study, with the aim of informing solver development within the more complete hydrostatic dynamical core used for climate research. The limitations and benefits to using explicit versus implicit methods, with different parameters and settings, are discussed in light of the trade-offs with Message Passing Interface (MPI) communication and memory and their inherent efficiency bottlenecks. Given the performance behavior across the configurations analyzed here, the recommendation for future work using the implicit solvers is conditional based on scale separation and the stiffness of the problem. For the regionally refined configurations, the implicit method has about the same efficiency as the explicit method, without considering efficiency gains from a preconditioner. The potential for improvement using a preconditioner is greatest for higher spectral order configurations, where more work is shifted to the linear solver. Finally, initial simulations with OpenACC directives to utilize a Graphics Processing Unit (GPU) when performing function evaluations show improvements locally, and that overall gains are possible with adjustments to data exchanges.
Research Organization:
Lawrence Livermore National Laboratory (LLNL), Livermore, CA (United States); Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States); Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States). Oak Ridge Leadership Computing Facility (OLCF)
Sponsoring Organization:
USDOE National Nuclear Security Administration (NNSA); USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR); USDOE Office of Science (SC), Biological and Environmental Research (BER)
Grant/Contract Number:
AC05-00OR22725; AC52-07NA27344
OSTI ID:
1490619
Alternate ID(s):
OSTI ID: 1770029
Report Number(s):
LLNL-JRNL--716595
Journal Information:
International Journal of High Performance Computing Applications, Journal Name: International Journal of High Performance Computing Applications Journal Issue: 2 Vol. 33; ISSN 1094-3420
Publisher:
SAGECopyright Statement
Country of Publication:
United States
Language:
English

References (35)

A multidecadal simulation of Atlantic tropical cyclones using a variable-resolution global atmospheric general circulation model journal August 2014
A new synoptic scale resolving global climate simulation using the Community Earth System Model journal December 2014
Short‐term time step convergence in a climate model journal February 2015
The effect of time steps and time-scales on parametrization suites journal August 2012
A global shallow-water model using an expanded spherical cube: Gnomonic versus conformal coordinates journal April 1996
Spectral Transform Solutions to the Shallow Water Test Set journal June 1995
The Spectral Element Method for the Shallow Water Equations on the Sphere journal January 1997
Evaluating Performance Portability of OpenACC book January 2015
A Scalable and Adaptable Solution Framework within Components of the Community Climate System Model book January 2009
On the influence of the orthogonalization scheme on the parallel performance of GMRES book January 1998
Effects of time step size on the simulation of tropical climate in NCAR-CAM3 journal January 2011
Algorithmically scalable block preconditioner for fully implicit shallow-water equations in CAM-SE journal October 2014
On balanced approximations for time integration of multiple time scale systems journal March 2003
A standard test set for numerical approximations to the shallow water equations in spherical geometry journal September 1992
Jacobian-free Newton–Krylov methods: a survey of approaches and applications journal January 2004
A compatible and conservative spectral element method on unstructured grids journal August 2010
A case study of CUDA FORTRAN and OpenACC for an atmospheric climate kernel journal July 2015
On the Use of Finite Difference Matrix-vector Products in Newton-krylov Solvers for Implicit Climate Dynamics with Spectral Elements journal January 2015
Accelerating Time Integration for the Shallow Water Equations on the Sphere Using GPUs journal January 2015
An A Posteriori–A Priori Analysis of Multiscale Operator Splitting journal January 2008
A Fully Implicit Domain Decomposition Algorithm for Shallow Water Equations on the Cubed-Sphere journal January 2010
GMRES: A Generalized Minimal Residual Algorithm for Solving Nonsymmetric Linear Systems journal July 1986
The Parallel Performance of Standard Parabolic Marching Schemes journal March 1991
An overview of the Trilinos project journal September 2005
Amesos2 and Belos: Direct and Iterative Solvers for Large Sparse Linear Systems journal January 2012
Accuracy Analysis of a Spectral Element Atmospheric Model Using a Fully Implicit Solution Framework journal August 2010
High-Resolution Global Climate Simulations with the ECMWF Model in Project Athena: Experimental Design, Model Climate, and Seasonal Forecast Skill journal May 2012
AMIP Simulation with the CAM4 Spectral Element Dynamical Core journal February 2013
Tropical Cyclones in the UPSCALE Ensemble of High-Resolution Global Climate Models journal January 2015
A Spectral Deferred Correction Method Applied to the Shallow Water Equations on a Sphere journal October 2013
A Semi-Implicit Version of the MPAS-Atmosphere Dynamical Core journal September 2015
CAM-SE: A scalable spectral element dynamical core for the Community Atmosphere Model journal November 2011
The Integration of a Low Order Spectral Form of the Primitive Meteorological Equations [Low order spectral形式による primitive 方程式の積分について] journal January 1966
An initial-value problem for testing numerical models of the global shallow-water equations journal January 2004
The spectral element method (SEM) on variable-resolution grids: evaluating grid sensitivity and resolution-aware numerical viscosity journal January 2014

Similar Records

Algorithmically scalable block preconditioner for fully implicit shallow-water equations in CAM-SE
Journal Article · Sat Oct 18 20:00:00 EDT 2014 · Computational Geosciences · OSTI ID:1185403
Implicit–explicit (IMEX) Runge–Kutta methods for non-hydrostatic atmospheric models
Journal Article · Mon Apr 16 20:00:00 EDT 2018 · Geoscientific Model Development (Online) · OSTI ID:1433452

Related Subjects

58 GEOSCIENCES
97 MATHEMATICS AND COMPUTING
GPU acceleration
Global climate modeling
Newton-Krylov
Newton–Krylov
implicit methods
regional refinement