skip to main content
DOE PAGES title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Initial Results From the Super-Parameterized E3SM

Abstract

Results from the new DOE super-parameterized (SP) Energy Exascale Earth System Model (SP-E3SM) are analyzed and compared to the traditionally parameterized E3SMv1 and previous studies using SP models. SP-E3SM is unique in that it utilizes GPU hardware acceleration, CRM mean-state acceleration, and reduced radiation to dramatically increase the model throughput and allow decadal experiments at 100-km external resolution. It also differs from other SP models by using a spectral element dynamical core on a cubed sphere grid and a finer vertical grid with a higher model top. Despite these differences, SP-E3SM generally reproduces the behavior of other super-parameterized models. Tropical wave variability is improved relative to E3SM, including the emergence of a Madden-Julian Oscillation and a realistic slowdown of Moist Kelvin Waves. However, the distribution of precipitation exhibits an unrealistically large variance, and while the timing of diurnal rainfall shows modest improvements the signal is not as coherent as observations. A notable grid imprinting bias is identified in the precipitation field and attributed to a unique feedback associated with the interactions between explicit convection and the spectral element grid structure. Spurious zonal mean column water tendencies due to grid imprinting are quantified – while negligible for the conventionally parameterized E3SM,more » they become large with super-parameterization, approaching 10% of the physical tendencies. The implication is that finding a remedy to grid imprinting will become especially important as spectral element dynamical cores begin to be combined with explicitly resolved convection.« less

Authors:
 [1];  [2];  [3];  [4];  [1];  [3]; ORCiD logo [2];  [5];  [6]; ORCiD logo [2];  [2];  [1]
  1. Lawrence Livermore National Lab. (LLNL), Livermore, CA (United States)
  2. Pacific Northwest National Lab. (PNNL), Richland, WA (United States)
  3. Sandia National Lab. (SNL-NM), Albuquerque, NM (United States)
  4. Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States)
  5. Univ. of California, Irvine, CA (United States)
  6. Colorado State Univ., Fort Collins, CO (United States)
Publication Date:
Research Org.:
Pacific Northwest National Lab. (PNNL), Richland, WA (United States); Lawrence Livermore National Lab. (LLNL), Livermore, CA (United States); Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States). Oak Ridge Leadership Computing Facility (OLCF); Lawrence Berkeley National Laboratory (LBNL), Berkeley, CA (United States). National Energy Research Scientific Computing Center (NERSC); Sandia National Lab. (SNL-NM), Albuquerque, NM (United States)
Sponsoring Org.:
USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR) (SC-21); USDOE Office of Science (SC), Biological and Environmental Research (BER) (SC-23); USDOE National Nuclear Security Administration (NNSA)
OSTI Identifier:
1599174
Alternate Identifier(s):
OSTI ID: 1601439
Report Number(s):
[PNNL-SA-146506; SAND-2019-9595J]
[Journal ID: ISSN 1942-2466]
Grant/Contract Number:  
[AC04-94AL85000; AC05-76RL01830; AC52-07NA27344; AC05-00OR22725; AC02-05CH11231]
Resource Type:
Accepted Manuscript
Journal Name:
Journal of Advances in Modeling Earth Systems
Additional Journal Information:
[ Journal Volume: 12; Journal Issue: 1]; Journal ID: ISSN 1942-2466
Publisher:
American Geophysical Union (AGU)
Country of Publication:
United States
Language:
English
Subject:
54 ENVIRONMENTAL SCIENCES

Citation Formats

Hannah, Walter M., Jones, Christopher R., Hillman, Benjamin R., Norman, Matthew R., Bader, David C., Taylor, Mark A., Leung, Lai-Yung R., Pritchard, Michael S., Branson, Mark D., Lin, Guangxing, Pressel, Kyle G., and Lee, Jungmin M. Initial Results From the Super-Parameterized E3SM. United States: N. p., 2020. Web. doi:10.1029/2019MS001863.
Hannah, Walter M., Jones, Christopher R., Hillman, Benjamin R., Norman, Matthew R., Bader, David C., Taylor, Mark A., Leung, Lai-Yung R., Pritchard, Michael S., Branson, Mark D., Lin, Guangxing, Pressel, Kyle G., & Lee, Jungmin M. Initial Results From the Super-Parameterized E3SM. United States. doi:10.1029/2019MS001863.
Hannah, Walter M., Jones, Christopher R., Hillman, Benjamin R., Norman, Matthew R., Bader, David C., Taylor, Mark A., Leung, Lai-Yung R., Pritchard, Michael S., Branson, Mark D., Lin, Guangxing, Pressel, Kyle G., and Lee, Jungmin M. Sun . "Initial Results From the Super-Parameterized E3SM". United States. doi:10.1029/2019MS001863. https://www.osti.gov/servlets/purl/1599174.
@article{osti_1599174,
title = {Initial Results From the Super-Parameterized E3SM},
author = {Hannah, Walter M. and Jones, Christopher R. and Hillman, Benjamin R. and Norman, Matthew R. and Bader, David C. and Taylor, Mark A. and Leung, Lai-Yung R. and Pritchard, Michael S. and Branson, Mark D. and Lin, Guangxing and Pressel, Kyle G. and Lee, Jungmin M.},
abstractNote = {Results from the new DOE super-parameterized (SP) Energy Exascale Earth System Model (SP-E3SM) are analyzed and compared to the traditionally parameterized E3SMv1 and previous studies using SP models. SP-E3SM is unique in that it utilizes GPU hardware acceleration, CRM mean-state acceleration, and reduced radiation to dramatically increase the model throughput and allow decadal experiments at 100-km external resolution. It also differs from other SP models by using a spectral element dynamical core on a cubed sphere grid and a finer vertical grid with a higher model top. Despite these differences, SP-E3SM generally reproduces the behavior of other super-parameterized models. Tropical wave variability is improved relative to E3SM, including the emergence of a Madden-Julian Oscillation and a realistic slowdown of Moist Kelvin Waves. However, the distribution of precipitation exhibits an unrealistically large variance, and while the timing of diurnal rainfall shows modest improvements the signal is not as coherent as observations. A notable grid imprinting bias is identified in the precipitation field and attributed to a unique feedback associated with the interactions between explicit convection and the spectral element grid structure. Spurious zonal mean column water tendencies due to grid imprinting are quantified – while negligible for the conventionally parameterized E3SM, they become large with super-parameterization, approaching 10% of the physical tendencies. The implication is that finding a remedy to grid imprinting will become especially important as spectral element dynamical cores begin to be combined with explicitly resolved convection.},
doi = {10.1029/2019MS001863},
journal = {Journal of Advances in Modeling Earth Systems},
number = [1],
volume = [12],
place = {United States},
year = {2020},
month = {1}
}

Journal Article:
Free Publicly Available Full Text
Publisher's Version of Record

Save / Share:

Works referenced in this record:

Structure of the Madden–Julian Oscillation in the Superparameterized CAM
journal, November 2009

  • Benedict, James J.; Randall, David A.
  • Journal of the Atmospheric Sciences, Vol. 66, Issue 11
  • DOI: 10.1175/2009JAS3030.1

Overview of the Coupled Model Intercomparison Project Phase 6 (CMIP6) experimental design and organization
journal, January 2016

  • Eyring, Veronika; Bony, Sandrine; Meehl, Gerald A.
  • Geoscientific Model Development, Vol. 9, Issue 5
  • DOI: 10.5194/gmd-9-1937-2016

Mean-state acceleration of cloud-resolving models and large eddy simulations: CRM AND LES MEAN-STATE ACCELERATION
journal, October 2015

  • Jones, C. R.; Bretherton, C. S.; Pritchard, M. S.
  • Journal of Advances in Modeling Earth Systems, Vol. 7, Issue 4
  • DOI: 10.1002/2015MS000488

Breaking the Cloud Parameterization Deadlock
journal, November 2003

  • Randall, David; Khairoutdinov, Marat; Arakawa, Akio
  • Bulletin of the American Meteorological Society, Vol. 84, Issue 11
  • DOI: 10.1175/BAMS-84-11-1547

The “Cubed Sphere”: A New Method for the Solution of Partial Differential Equations in Spherical Geometry
journal, March 1996

  • Ronchi, C.; Iacono, R.; Paolucci, P. S.
  • Journal of Computational Physics, Vol. 124, Issue 1
  • DOI: 10.1006/jcph.1996.0047

Understanding Cloud and Convective Characteristics in Version 1 of the E3SM Atmosphere Model
journal, October 2018

  • Xie, Shaocheng; Lin, Wuyin; Rasch, Philip J.
  • Journal of Advances in Modeling Earth Systems, Vol. 10, Issue 10
  • DOI: 10.1029/2018MS001350

Ensemble superparameterization versus stochastic parameterization: A comparison of model uncertainty representation in tropical weather prediction
journal, May 2017

  • Subramanian, Aneesh C.; Palmer, Tim N.
  • Journal of Advances in Modeling Earth Systems, Vol. 9, Issue 2
  • DOI: 10.1002/2016MS000857

Restricting 32-128 km horizontal scales hardly affects the MJO in the Superparameterized Community Atmosphere Model v.3.0 but the number of cloud-resolving grid columns constrains vertical mixing
journal, August 2014

  • Pritchard, Michael S.; Bretherton, Christopher S.; DeMott, Charlotte A.
  • Journal of Advances in Modeling Earth Systems, Vol. 6, Issue 3
  • DOI: 10.1002/2014MS000340

CRCP: a Cloud Resolving Convection Parameterization for modeling the tropical convecting atmosphere
journal, September 1999


Two Modes of Change of the Distribution of Rain
journal, November 2014


How Often Does It Rain?
journal, March 2006

  • Sun, Ying; Solomon, Susan; Dai, Aiguo
  • Journal of Climate, Vol. 19, Issue 6
  • DOI: 10.1175/JCLI3672.1

Climate simulations and projections with a super-parameterized climate model
journal, October 2014


A Multiscale Modeling System: Developments, Applications, and Critical Issues
journal, April 2009

  • Tao, Wei-Kuo; Chern, Jiun-Dar; Atlas, Robert
  • Bulletin of the American Meteorological Society, Vol. 90, Issue 4
  • DOI: 10.1175/2008BAMS2542.1

Orogenic Propagating Precipitation Systems over the United States in a Global Climate Model with Embedded Explicit Convection
journal, August 2011

  • Pritchard, Michael S.; Moncrieff, Mitchell W.; Somerville, Richard C. J.
  • Journal of the Atmospheric Sciences, Vol. 68, Issue 8
  • DOI: 10.1175/2011JAS3699.1

Simulation of the Indian Summer Monsoon in the Superparameterized Climate Forecast System Version 2: Preliminary Results
journal, November 2015

  • Goswami, Bidyut B.; Krishna, R. P. M.; Mukhopadhyay, P.
  • Journal of Climate, Vol. 28, Issue 22
  • DOI: 10.1175/JCLI-D-14-00607.1

Towards Global Large Eddy Simulation: Super-Parameterization Revisited
journal, January 2016

  • Grabowski, Wojciech W.
  • Journal of the Meteorological Society of Japan. Ser. II, Vol. 94, Issue 4
  • DOI: 10.2151/jmsj.2016-017

Global Precipitation at One-Degree Daily Resolution from Multisatellite Observations
journal, February 2001


Simulations of the Atmospheric General Circulation Using a Cloud-Resolving Model as a Superparameterization of Physical Processes
journal, July 2005

  • Khairoutdinov, Marat; Randall, David; DeMott, Charlotte
  • Journal of the Atmospheric Sciences, Vol. 62, Issue 7
  • DOI: 10.1175/JAS3453.1

Synoptic Control of Heavy-Rain-Producing Convective Training Episodes
journal, July 2014


Performance of the Goddard multiscale modeling framework with Goddard ice microphysical schemes
journal, January 2016

  • Chern, Jiun‐Dar; Tao, Wei‐Kuo; Lang, Stephen E.
  • Journal of Advances in Modeling Earth Systems, Vol. 8, Issue 1
  • DOI: 10.1002/2015MS000469

Effects of explicit convection on global land-atmosphere coupling in the superparameterized CAM: EXPLICIT CONVECTION ON GLOBAL LAND-ATMOSPHERE COUPLING
journal, August 2016

  • Sun, Jian; Pritchard, Michael S.
  • Journal of Advances in Modeling Earth Systems, Vol. 8, Issue 3
  • DOI: 10.1002/2016MS000689

A compatible and conservative spectral element method on unstructured grids
journal, August 2010


Global consequences of interactions between clouds and radiation at scales unresolved by global climate models
journal, January 2005


The impact of stochastic physics on tropical rainfall variability in global climate models on daily to weekly time scales: STOCHASTIC PHYSICS AND TROPICAL PRECIPITATION
journal, June 2017

  • Watson, Peter A. G.; Berner, Judith; Corti, Susanna
  • Journal of Geophysical Research: Atmospheres, Vol. 122, Issue 11
  • DOI: 10.1002/2016JD026386

A perspective on climate model hierarchies: MODEL HIERARCHIES
journal, August 2017

  • Jeevanjee, Nadir; Hassanzadeh, Pedram; Hill, Spencer
  • Journal of Advances in Modeling Earth Systems, Vol. 9, Issue 4
  • DOI: 10.1002/2017MS001038

North American Climate in CMIP5 Experiments: Part III: Assessment of Twenty-First-Century Projections
journal, March 2014


The moist static energy budget in NCAR CAM5 hindcasts during DYNAMO
journal, May 2014

  • Hannah, Walter M.; Maloney, Eric D.
  • Journal of Advances in Modeling Earth Systems, Vol. 6, Issue 2
  • DOI: 10.1002/2013MS000272

A Lagrangian View of Moisture Dynamics during DYNAMO
journal, May 2016

  • Hannah, Walter M.; Mapes, Brian E.; Elsaesser, Gregory S.
  • Journal of the Atmospheric Sciences, Vol. 73, Issue 5
  • DOI: 10.1175/JAS-D-15-0243.1

From ERA-Interim to ERA5: the considerable impact of ECMWF's next-generation reanalysis on Lagrangian transport simulations
journal, January 2019

  • Hoffmann, Lars; Günther, Gebhard; Li, Dan
  • Atmospheric Chemistry and Physics, Vol. 19, Issue 5
  • DOI: 10.5194/acp-19-3097-2019

Cloud Resolving Modeling of the ARM Summer 1997 IOP: Model Formulation, Results, Uncertainties, and Sensitivities
journal, February 2003


The Community Earth System Model: A Framework for Collaborative Research
journal, September 2013

  • Hurrell, James W.; Holland, M. M.; Gent, P. R.
  • Bulletin of the American Meteorological Society, Vol. 94, Issue 9
  • DOI: 10.1175/BAMS-D-12-00121.1

Insensitivity of the Cloud Response to Surface Warming Under Radical Changes to Boundary Layer Turbulence and Cloud Microphysics: Results From the Ultraparameterized CAM
journal, December 2018

  • Parishani, Hossein; Pritchard, Michael S.; Bretherton, Christopher S.
  • Journal of Advances in Modeling Earth Systems, Vol. 10, Issue 12
  • DOI: 10.1029/2018MS001409

The DOE E3SM Coupled Model Version 1: Overview and Evaluation at Standard Resolution
journal, July 2019

  • Golaz, Jean‐Christophe; Caldwell, Peter M.; Van Roekel, Luke P.
  • Journal of Advances in Modeling Earth Systems, Vol. 11, Issue 7
  • DOI: 10.1029/2018MS001603

Organization and Environmental Properties of Extreme-Rain-Producing Mesoscale Convective Systems
journal, April 2005

  • Schumacher, Russ S.; Johnson, Richard H.
  • Monthly Weather Review, Vol. 133, Issue 4
  • DOI: 10.1175/MWR2899.1

Representing Convective Organization in Prediction Models by a Hybrid Strategy
journal, December 2006

  • Moncrieff, Mitchell W.; Liu, Changhai
  • Journal of the Atmospheric Sciences, Vol. 63, Issue 12
  • DOI: 10.1175/JAS3812.1

Improved Diurnal Cycle of Precipitation in E3SM With a Revised Convective Triggering Function
journal, July 2019

  • Xie, Shaocheng; Wang, Yi‐Chi; Lin, Wuyin
  • Journal of Advances in Modeling Earth Systems, Vol. 11, Issue 7
  • DOI: 10.1029/2019MS001702

An Overview of the Atmospheric Component of the Energy Exascale Earth System Model
journal, August 2019

  • Rasch, P. J.; Xie, S.; Ma, P. ‐L.
  • Journal of Advances in Modeling Earth Systems, Vol. 11, Issue 8
  • DOI: 10.1029/2019MS001629

Impacts of cloud superparameterization on projected daily rainfall intensity climate changes in multiple versions of the Community Earth System Model
journal, October 2016

  • Kooperman, Gabriel J.; Pritchard, Michael S.; Burt, Melissa A.
  • Journal of Advances in Modeling Earth Systems, Vol. 8, Issue 4
  • DOI: 10.1002/2016MS000715

Quasi-Geostrophic Motions in the Equatorial Area [赤道近くでの準地衡風的運動]
journal, January 1966

  • Matsuno, Taroh
  • Journal of the Meteorological Society of Japan. Ser. II, Vol. 44, Issue 1
  • DOI: 10.2151/jmsj1965.44.1_25

Dreary state of precipitation in global models: MODEL AND OBSERVED PRECIPITATION
journal, December 2010

  • Stephens, Graeme L.; L'Ecuyer, Tristan; Forbes, Richard
  • Journal of Geophysical Research: Atmospheres, Vol. 115, Issue D24
  • DOI: 10.1029/2010JD014532

Assessing the diurnal cycle of precipitation in a multi-scale climate model
journal, October 2009

  • Pritchard, Michael S.; Somerville, Richard C. J.
  • Journal of Advances in Modeling Earth Systems, Vol. 2
  • DOI: 10.3894/JAMES.2009.1.12

Nonhydrostatic icosahedral atmospheric model (NICAM) for global cloud resolving simulations
journal, March 2008


Reduced African Easterly Wave Activity with Quadrupled CO 2 in the Superparameterized CESM
journal, October 2017


Convectively Coupled Equatorial Waves: Analysis of Clouds and Temperature in the Wavenumber–Frequency Domain
journal, February 1999


Physics–Dynamics Coupling with Element-Based High-Order Galerkin Methods: Quasi-Equal-Area Physics Grid
journal, January 2019

  • Herrington, Adam R.; Lauritzen, Peter H.; Taylor, Mark A.
  • Monthly Weather Review, Vol. 147, Issue 1
  • DOI: 10.1175/MWR-D-18-0136.1

Orographic effects on precipitating clouds
journal, January 2012


A cloud resolving model as a cloud parameterization in the NCAR Community Climate System Model: Preliminary results
journal, September 2001

  • Khairoutdinov, Marat F.; Randall, David A.
  • Geophysical Research Letters, Vol. 28, Issue 18
  • DOI: 10.1029/2001GL013552

A mass and energy conserving spectral element atmospheric dynamical core on the cubed-sphere grid
journal, July 2007


Robust effects of cloud superparameterization on simulated daily rainfall intensity statistics across multiple versions of the C ommunity E arth S ystem M odel
journal, February 2016

  • Kooperman, Gabriel J.; Pritchard, Michael S.; Burt, Melissa A.
  • Journal of Advances in Modeling Earth Systems, Vol. 8, Issue 1
  • DOI: 10.1002/2015MS000574

Moist Static Energy Budget of MJO-like Disturbances in the Atmosphere of a Zonally Symmetric Aquaplanet
journal, April 2012


The response of US summer rainfall to quadrupled CO 2 climate change in conventional and superparameterized versions of the NCAR community atmosphere model
journal, August 2014

  • Kooperman, Gabriel J.; Pritchard, Michael S.; Somerville, Richard C. J.
  • Journal of Advances in Modeling Earth Systems, Vol. 6, Issue 3
  • DOI: 10.1002/2014MS000306

Exploring a Lower‐Resolution Physics Grid in CAM‐SE‐CSLAM
journal, July 2019

  • Herrington, Adam R.; Lauritzen, Peter H.; Reed, Kevin A.
  • Journal of Advances in Modeling Earth Systems, Vol. 11, Issue 7
  • DOI: 10.1029/2019MS001684

Role of Longwave Cloud–Radiation Feedback in the Simulation of the Madden–Julian Oscillation
journal, September 2015


Clouds and the Earth’s Radiant Energy System (CERES) Energy Balanced and Filled (EBAF) Top-of-Atmosphere (TOA) Edition-4.0 Data Product
journal, January 2018


An Improved Framework for Superparameterization
journal, August 2004


The Explicit-Cloud Parameterized-Pollutant hybrid approach for aerosol–cloud interactions in multiscale modeling framework models: tracer transport results
journal, April 2008

  • Gustafson Jr, William I.; Berg, Larry K.; Easter, Richard C.
  • Environmental Research Letters, Vol. 3, Issue 2
  • DOI: 10.1088/1748-9326/3/2/025005

Simulations of the West African Monsoon with a Superparameterized Climate Model. Part II: African Easterly Waves
journal, November 2014

  • McCrary, Rachel R.; Randall, David A.; Stan, Cristiana
  • Journal of Climate, Vol. 27, Issue 22
  • DOI: 10.1175/JCLI-D-13-00677.1

The TRMM Multisatellite Precipitation Analysis (TMPA): Quasi-Global, Multiyear, Combined-Sensor Precipitation Estimates at Fine Scales
journal, February 2007

  • Huffman, George J.; Bolvin, David T.; Nelkin, Eric J.
  • Journal of Hydrometeorology, Vol. 8, Issue 1
  • DOI: 10.1175/JHM560.1

Toward low-cloud-permitting cloud superparameterization with explicit boundary layer turbulence: LOW-CLOUD ULTRAPARAMETERIZATION
journal, July 2017

  • Parishani, Hossein; Pritchard, Michael S.; Bretherton, Christopher S.
  • Journal of Advances in Modeling Earth Systems, Vol. 9, Issue 3
  • DOI: 10.1002/2017MS000968

Improved simulation of extreme precipitation in a high-resolution atmosphere model: EXTREME PRECIPITATION AT HIGH RESOLUTION
journal, November 2013

  • Kopparla, Pushkar; Fischer, Erich M.; Hannay, Cécile
  • Geophysical Research Letters, Vol. 40, Issue 21
  • DOI: 10.1002/2013GL057866