Skip to main content
OSTI.GOVU.S. Department of Energy Office of Scientific and Technical Information
U.S. Department of Energy
Office of Scientific and Technical Information
 

Unprecedented cloud resolution in a GPU-enabled full-physics atmospheric climate simulation on OLCF’s summit supercomputer

Journal Article · · International Journal of High Performance Computing Applications
 [1];  [2];  [3];  [2];  [3];  [4];  [2];  [4];  [1];  [4];  [1];  [3];  [5]
  1. National Center for Computational Sciences,Oak Ridge National Laboratory, Oak Ridge, TN, USA
  2. Lawrence Livermore National Laboratory, Livermore, CA, USA
  3. Sandia National Laboratory, Albuquerque, NM, USA
  4. Atmospheric Sciences and Global Change,Pacific Northwes National Laboratory, Richland, WA, USA
  5. Mathematics and Computer Science Division,Argonne National Laboratory, Lemont, IL, USA
+ Show Author Affiliations

Clouds represent a key uncertainty in future climate projection. While explicit cloud resolution remains beyond our computational grasp for global climate, we can incorporate important cloud effects through a computational middle ground called the Multi-scale Modeling Framework (MMF), also known as Super Parameterization. This algorithmic approach embeds high-resolution Cloud Resolving Models (CRMs) to represent moist convective processes within each grid column in a Global Climate Model (GCM). The MMF code requires no parallel data transfers and provides a self-contained target for acceleration. This study investigates the performance of the Energy Exascale Earth System Model-MMF (E3SM-MMF) code on the OLCF Summit supercomputer at an unprecedented scale of simulation. Hundreds of kernels in the roughly 10K lines of code in the E3SM-MMF CRM were ported to GPUs with OpenACC directives. A high-resolution benchmark using 4600 nodes on Summit demonstrates the computational capability of the GPU-enabled E3SM-MMF code in a full physics climate simulation.

Research Organization:
Lawrence Livermore National Laboratory (LLNL), Livermore, CA (United States); Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States). Oak Ridge Leadership Computing Facility (OLCF); Sandia National Laboratories (SNL-NM), Albuquerque, NM (United States)
Sponsoring Organization:
USDOE; USDOE National Nuclear Security Administration (NNSA); USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR); USDOE Office of Science (SC), Biological and Environmental Research (BER)
Grant/Contract Number:
AC05-00OR22725; AC52-07NA27344; NA0003525
OSTI ID:
1808341
Alternate ID(s):
OSTI ID: 1817417
OSTI ID: 1894990
Report Number(s):
LLNL-JRNL-814494
Journal Information:
International Journal of High Performance Computing Applications, Journal Name: International Journal of High Performance Computing Applications Journal Issue: 1 Vol. 36; ISSN 1094-3420
Publisher:
SAGE PublicationsCopyright Statement
Country of Publication:
United States
Language:
English

References (30)

Mean-state acceleration of cloud-resolving models and large eddy simulations: CRM AND LES MEAN-STATE ACCELERATION journal October 2015
Robustness and sensitivities of central U.S. summer convection in the super-parameterized CAM: Multi-model intercomparison with a new regional EOF index: U.S. CONVECTION IN SP-CAM: A NEW EOF INDEX journal June 2013
The “Cubed Sphere”: A New Method for the Solution of Partial Differential Equations in Spherical Geometry journal March 1996
Simulation of the South American climate by a coupled model with super-parameterized convection journal January 2015
Computational aspects of a scalable high-order discontinuous Galerkin atmospheric dynamical core journal February 2009
Climate simulations and projections with a super-parameterized climate model journal October 2014
A study of spectral element and discontinuous Galerkin methods for the Navier–Stokes equations in nonhydrostatic mesoscale atmospheric modeling: Equation sets and test cases journal April 2008
Mixed finite elements for numerical weather prediction journal August 2012
Exploring an Ensemble-Based Approach to Atmospheric Climate Modeling and Testing at Scale journal January 2017
A cloud resolving model as a cloud parameterization in the NCAR Community Climate System Model: Preliminary results journal September 2001
An ocean-atmosphere climate simulation with an embedded cloud resolving model: A GLOBAL CLIMATE SIMULATION journal January 2010
Dreary state of precipitation in global models: MODEL AND OBSERVED PRECIPITATION journal December 2010
“Super-parameterization”: A better way to simulate regional extreme precipitation?: SUPER PARAMETERIZATION FOR EXTREME PRECIPITATION SIMULATIONS journal February 2012
Uncertainty quantification and parameter tuning in the CAM5 Zhang-McFarlane convection scheme and impact of improved convection on the global circulation and climate: PARAMETERS TUNING IN CAM5 AND ITS IMPACT journal January 2013
Initial Results From the Super‐Parameterized E3SM journal January 2020
Causes of model dry and warm bias over central U.S. and impact on climate projections journal October 2017
10M-Core Scalable Fully-Implicit Solver for Nonhydrostatic Atmospheric Dynamics
  • Yang, Chao; Xue, Wei; Fu, Haohuan
  • SC16: International Conference for High Performance Computing, Networking, Storage and Analysis https://doi.org/10.1109/SC.2016.5
conference November 2016
A Holistic Algorithmic Approach to Improving Accuracy, Robustness, and Computational Efficiency for Atmospheric Dynamics journal January 2020
The Accumulation of Rounding Errors and Port Validation for Global Atmospheric Models journal March 1997
Redesigning CAM-SE for peta-scale climate modeling performance and ultra-high resolution on Sunway TaihuLight
  • Fu, Haohuan; Liu, Weiguo; Wang, Lanning
  • Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis on - SC '17 https://doi.org/10.1145/3126908.3126909
conference January 2017
The Spectral Element Atmosphere Model (SEAM): High-Resolution Parallel Computation and Localized Resolution of Regional Dynamics journal March 2004
Structure of the Madden–Julian Oscillation in the Superparameterized CAM journal November 2009
Breaking the Cloud Parameterization Deadlock journal November 2003
The Role of Cloud Microphysics Parameterization in the Simulation of Mesoscale Convective System Clouds and Precipitation in the Tropical Western Pacific journal April 2013
The Madden–Julian Oscillation’s Impacts on Worldwide Tropical Cyclone Activity journal March 2014
Precipitation Characteristics in Eighteen Coupled Climate Models journal September 2006
A 10-Year Survey of Extreme Rainfall Events in the Central and Eastern United States Using Gridded Multisensor Precipitation Analyses journal September 2014
Storm Morphology and Rainfall Characteristics of TRMM Precipitation Features journal October 2006
CAM-SE: A scalable spectral element dynamical core for the Community Atmosphere Model journal November 2011
Near-global climate simulation at 1 km resolution: establishing a performance baseline on 4888 GPUs with COSMO 5.0 journal January 2018

Similar Records

Load‐Balancing Intense Physics Calculations to Embed Regionalized High‐Resolution Cloud Resolving Models in the E3SM and CESM Climate Models
Journal Article · Tue May 03 20:00:00 EDT 2022 · Journal of Advances in Modeling Earth Systems · OSTI ID:1866293

Related Subjects

58 GEOSCIENCES
79 ASTRONOMY AND ASTROPHYSICS
97 MATHEMATICS AND COMPUTING
GPU
OpenACC
climate
clouds