skip to main content
OSTI.GOV title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Parametric Sensitivity and Uncertainty Quantification in the Version 1 of E3SM Atmosphere Model Based on Short Perturbed Parameter Ensemble Simulations

Journal Article · · Journal of Geophysical Research: Atmospheres
DOI:https://doi.org/10.1029/2018JD028927· OSTI ID:1484410
ORCiD logo [1]; ORCiD logo [1]; ORCiD logo [2]; ORCiD logo [3]; ORCiD logo [1]; ORCiD logo [1]; ORCiD logo [4]; ORCiD logo [1]; ORCiD logo [1];  [5]; ORCiD logo [1]; ORCiD logo [3]; ORCiD logo [1]; ORCiD logo [1]; ORCiD logo [1];  [3]; ORCiD logo [1]
  1. Pacific Northwest National Laboratory Richland WA USA
  2. Pacific Northwest National Laboratory Richland WA USA, School of Atmospheric Sciences Nanjing University Nanjing China
  3. Lawrence Livermore National Laboratory Livermore CA USA
  4. University of Wisconsin‐Milwaukee Milwaukee WI USA
  5. Brookhaven National Laboratory Upton NY USA
+ Show Author Affiliations

The atmospheric component of Energy Exascale Earth System Model (E3SM) version 1 (EAMv1) has included many new features in the physics parameterizations compared to its predecessors. Potential complex nonlinear interactions among the new features create a significant challenge for understanding the model behaviors and parameter tuning. Using the one-at-a-time method, the benefit of tuning one parameter may offset the benefit of tuning another parameter, or improvement in one target variable may lead to degradation in another target variable. To better understand the EAMv1 model behaviors and physics, we conducted a large number of short simulations (3 days) in which 18 parameters carefully selected from parameterizations of deep convection, shallow convection and cloud macrophysics and microphysics were perturbed simultaneously using the Latin Hypercube sampling method. From the Perturbed Parameters Ensemble (PPE) simulations and use of different skill score functions, we identified the most sensitive parameters, quantified how the model responds to changes of the parameters for both global mean and spatial distribution, and estimated the maximum likelihood of model parameter space for a number of important fidelity metrics. Comparison of the parametric sensitivity using simulations of two different lengths suggests that PPE using short simulations has some bearing on understanding parametric sensitivity of longer simulations. Results from this analysis provide a more comprehensive picture of the EAMv1 behavior. The difficulty in reducing biases in multiple variables simultaneously highlights the need of characterizing model structural uncertainty (so-called embedded errors) to inform future development efforts.

Research Organization:
Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States). Oak Ridge Leadership Computing Facility (OLCF); Brookhaven National Laboratory (BNL), Upton, NY (United States); Pacific Northwest National Laboratory (PNNL), Richland, WA (United States); Lawrence Livermore National Laboratory (LLNL), Livermore, CA (United States)
Sponsoring Organization:
USDOE Office of Science (SC), Basic Energy Sciences (BES)
Grant/Contract Number:
DEAC02‐05CH11231; DE‐AC05‐00OR22725; DE‐SC0016287; DE‐AC52‐07NA27344; DE‐AC06‐76RLO 1830; SC0012704; AC05-76RL01830; AC52-07NA27344
OSTI ID:
1484410
Alternate ID(s):
OSTI ID: 1482569; OSTI ID: 1484411; OSTI ID: 1496811; OSTI ID: 1498472
Report Number(s):
BNL-209469-2018-JAAM; PNNL-SA-138521; LLNL-JRNL-765390
Journal Information:
Journal of Geophysical Research: Atmospheres, Journal Name: Journal of Geophysical Research: Atmospheres Vol. 123 Journal Issue: 23; ISSN 2169-897X
Publisher:
American Geophysical Union (AGU)Copyright Statement
Country of Publication:
United States
Language:
English
Citation Metrics:
Cited by: 45 works
Citation information provided by
Web of Science

References (88)

Elucidating Model Inadequacies in a Cloud Parameterization by Use of an Ensemble-Based Calibration Framework journal December 2007
The Transpose-AMIP II Experiment and Its Application to the Understanding of Southern Ocean Cloud Biases in Climate Models journal May 2013
Combining satellite data and models to estimate cloud radiative effect at the surface and in the atmosphere: Cloud radiative effect at the surface and in the atmosphere journal August 2011
Climate projections of future extreme events accounting for modelling uncertainties and historical simulation biases journal March 2014
A Bayesian Examination of Deep Convective Squall-Line Sensitivity to Changes in Cloud Microphysical Parameters journal February 2016
Uncertainty Quantification in Climate Modeling and Projection journal May 2016
Strategies for reducing the climate noise in model simulations: ensemble runs versus a long continuous run journal May 2014
A PDF-Based Model for Boundary Layer Clouds. Part I: Method and Model Description journal December 2002
How well must climate models agree with observations? journal October 2015
Efficient screening of climate model sensitivity to a large number of perturbed input parameters: PERTURBED INPUT-PARAMETER SENSITIVITY journal July 2013
Global Precipitation: A 17-Year Monthly Analysis Based on Gauge Observations, Satellite Estimates, and Numerical Model Outputs journal November 1997
Very fast simulated re-annealing journal January 1989
Understanding Cloud and Convective Characteristics in Version 1 of the E3SM Atmosphere Model journal October 2018
Sensitivity of climate simulations to the parameterization of cumulus convection in the Canadian climate centre general circulation model journal September 1995
Latin hypercube sampling and the propagation of uncertainty in analyses of complex systems journal July 2003
Bayesian analysis of computer code outputs: A tutorial journal October 2006
Quantification of Cloud Microphysical Parameterization Uncertainty Using Radar Reflectivity journal November 2012
Parameter variations in prediction skill optimization at ECMWF journal January 2013
Summarizing multiple aspects of model performance in a single diagram journal April 2001
Toward Optimal Closure of the Earth's Top-of-Atmosphere Radiation Budget journal February 2009
Advanced Two-Moment Bulk Microphysics for Global Models. Part II: Global Model Solutions and Aerosol–Cloud Interactions journal February 2015
Carbon and nitrogen cycle dynamics in the O-CN land surface model: 1. Model description, site-scale evaluation, and sensitivity to parameter estimates: SITE-SCALE EVALUATION OF A C-N MODEL journal February 2010
Short ensembles: an efficient method for discerning climate-relevant sensitivities in atmospheric general circulation models journal January 2014
Transient climate changes in a perturbed parameter ensemble of emissions-driven earth system model simulations journal March 2014
Large Sample Properties of Simulations Using Latin Hypercube Sampling journal May 1987
A sensitivity study of radiative fluxes at the top of atmosphere to cloud-microphysics and aerosol parameters in the community atmosphere model CAM5 journal January 2013
Parameter Tuning and Calibration of RegCM3 with MIT–Emanuel Cumulus Parameterization Scheme over CORDEX East Asia Domain journal October 2014
Parametric behaviors of CLUBB in simulations of low clouds in the Community Atmosphere Model (CAM): PARAMETRIC BEHAVIORS OF CLUBB IN CAM journal July 2015
History matching for exploring and reducing climate model parameter space using observations and a large perturbed physics ensemble journal August 2013
Using Probability Density Functions to Derive Consistent Closure Relationships among Higher-Order Moments journal April 2005
Estimates of the Global Water Budget and Its Annual Cycle Using Observational and Model Data journal August 2007
The impact of structural error on parameter constraint in a climate model journal January 2016
Considerations for parameter optimization and sensitivity in climate models journal November 2010
Some issues in uncertainty quantification and parameter tuning: a case study of convective parameterization scheme in the WRF regional climate model journal January 2012
The ERA-Interim reanalysis: configuration and performance of the data assimilation system journal April 2011
Linearization of Microphysical Parameterization Uncertainty Using Multiplicative Process Perturbation Parameters journal January 2014
Analyzing the Climate Sensitivity of the HadSM3 Climate Model Using Ensembles from Different but Related Experiments journal July 2009
Optimizing Parameters in an Atmospheric General Circulation Model journal September 2005
A methodology for probabilistic predictions of regional climate change from perturbed physics ensembles
  • Murphy, J. M.; Booth, B. B. B.; Collins, M.
  • Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, Vol. 365, Issue 1857 https://doi.org/10.1098/rsta.2007.2077
journal June 2007
NCEP–DOE AMIP-II Reanalysis (R-2) journal November 2002
Diagnostics for Gaussian Process Emulators journal November 2009
Objective calibration of regional climate models: OBJECTIVE CALIBRATION OF RCMS journal December 2012
Uncertainty in predictions of the climate response to rising levels of greenhouse gases journal January 2005
A new approach to modeling aerosol effects on East Asian climate: Parametric uncertainties associated with emissions, cloud microphysics, and their interactions journal September 2015
The GCM-Oriented CALIPSO Cloud Product (CALIPSO-GOCCP) journal January 2010
Tuning without over-tuning: parametric uncertainty quantification for the NEMO ocean model journal January 2017
The Role of Convective Gustiness in Reducing Seasonal Precipitation Biases in the Tropical West Pacific journal April 2018
Higher-Order Turbulence Closure and Its Impact on Climate Simulations in the Community Atmosphere Model journal December 2013
Description and evaluation of a new four-mode version of the Modal Aerosol Module (MAM4) within version 5.3 of the Community Atmosphere Model journal January 2016
Evaluating Parameterizations in General Circulation Models: Climate Simulation Meets Weather Prediction journal December 2004
An Efficient Stochastic Bayesian Approach to Optimal Parameter and Uncertainty Estimation for Climate Model Predictions journal July 2004
Assessing WRF model parameter sensitivity: A case study with 5 day summer precipitation forecasting in the Greater Beijing Area: Assessing WRF parameter sensitivity journal January 2015
A sensitivity analysis of cloud properties to CLUBB parameters in the single-column Community Atmosphere Model (SCAM5) journal August 2014
Earth Radiation Budget Experiment (ERBE): An Overview journal March 1982
PDF Parameterization of Boundary Layer Clouds in Models with Horizontal Grid Spacings from 2 to 16 km journal January 2012
On the Correspondence between Mean Forecast Errors and Climate Errors in CMIP5 Models journal February 2014
Hydrometeor Detection Using Cloudsat —An Earth-Orbiting 94-GHz Cloud Radar journal April 2008
Sensitivity of surface flux simulations to hydrologic parameters based on an uncertainty quantification framework applied to the Community Land Model: UNCERTAINTY QUANTIFICATION FOR CLM4 journal August 2012
Mean seasonal and spatial variability in gauge-corrected, global precipitation journal March 1990
Climate model forecast biases assessed with a perturbed physics ensemble journal October 2016
Identifying and removing structural biases in climate models with history matching journal October 2014
Climate model errors, feedbacks and forcings: a comparison of perturbed physics and multi-model ensembles journal May 2010
Uncertainty quantification and parameter tuning in the CAM5 Zhang-McFarlane convection scheme and impact of improved convection on the global circulation and climate: PARAMETERS TUNING IN CAM5 AND ITS IMPACT journal January 2013
Error Reduction and Convergence in Climate Prediction journal December 2008
Advanced Two-Moment Bulk Microphysics for Global Models. Part I: Off-Line Tests and Comparison with Other Schemes journal February 2015
How Much More Rain Will Global Warming Bring? journal July 2007
Exploring parameter sensitivities of the land surface using a locally coupled land-atmosphere model: EXPLORING PARAMETER SENSITIVITIES journal November 2004
Robust Characterization of Model Physics Uncertainty for Simulations of Deep Moist Convection journal May 2010
Parametric sensitivity and calibration for the Kain‑Fritsch convective parameterization scheme in the WRF model journal March 2014
Inverse modeling of hydrologic parameters using surface flux and runoff observations in the Community Land Model journal January 2013
Assessing parameter importance of the Common Land Model based on qualitative and quantitative sensitivity analysis journal January 2013
Surrogate-based analysis and optimization journal January 2005
Optimization of NWP model closure parameters using total energy norm of forecast error as a target journal January 2014
Surrogate-based optimization of climate model parameters using response correction journal December 2011
The Annual Cycle of the Energy Budget. Part I: Global Mean and Land–Ocean Exchanges journal May 2008
Regional assessment of the parameter-dependent performance of CAM4 in simulating tropical clouds: REGIONAL TROPICAL CLOUDS IN CAM4 journal July 2012
An evaluation of adaptive surrogate modeling based optimization with two benchmark problems journal October 2014
On minimum relative entropy concepts and prior compatibility issues in vadose zone inverse and forward modeling: ON MRE CONCEPT AND PRIOR COMPATIBILITY ISSUES journal December 2005
Parametric sensitivity analysis of precipitation at global and local scales in the Community Atmosphere Model CAM5 journal April 2015
Multi-objective parameter optimization of common land model using adaptive surrogate modeling journal January 2015
The Art and Science of Climate Model Tuning journal March 2017
The Effect of Orographically Excited Gravity Wave Drag on the General Circulation of the Lower Stratosphere and Troposphere journal July 1987
The hydrological cycle and its influence on climate journal October 1992
Annealing evolutionary stochastic approximation Monte Carlo for global optimization journal April 2010
The Impact of Parametric Uncertainties on Biogeochemistry in the E3SM Land Model journal February 2018
A Review of Global Precipitation Data Sets: Data Sources, Estimation, and Intercomparisons journal January 2018
Bayesian Calibration of the Community Land Model Using Surrogates journal January 2015
The Version-2 Global Precipitation Climatology Project (GPCP) Monthly Precipitation Analysis (1979–Present) journal December 2003

Figures / Tables (16)

Table 1(p. 50)table Table 1
Figure 1(p. 51)figure Figure 1
Figure 2(p. 52)figure Figure 2
Figure 3(p. 53)figure Figure 3
Figure 4(p. 54)figure Figure 4
Figure 5(p. 55)figure Figure 5
Figure 6(p. 56)figure Figure 6
Figure 7(p. 57)figure Figure 7
Figure 8(p. 58)figure Figure 8
Figure 9(p. 59)figure Figure 9
Figure 10(p. 60)figure Figure 10
Figure 11(p. 61)figure Figure 11
Figure 12(p. 62)figure Figure 12
Figure 13(p. 63)figure Figure 13
Figure 14(p. 64)figure Figure 14
Figure 15(p. 65)figure Figure 15

Similar Records

Region and cloud regime dependence of parametric sensitivity in E3SM atmosphere model
Journal Article · Wed Oct 18 00:00:00 EDT 2023 · Climate Dynamics · OSTI ID:1484410
+11 more
Understanding changes in cloud simulations from E3SM version 1 to version 2
Journal Article · Wed Jan 10 00:00:00 EST 2024 · Geoscientific Model Development (Online) · OSTI ID:1484410
+8 more
Understanding Changes in Cloud Simulations from E3SM Version 1 to Version 2
Journal Article · Fri Jul 21 00:00:00 EDT 2023 · EGUsphere · OSTI ID:1484410
+8 more

Related Subjects

54 ENVIRONMENTAL SCIENCES
Parametric sensitivity
uncertainty quantification
E3SM
PPE
short simulations
sensitivity analysis
Perturbed Parameters Ensemble
Energy Exascale Earth System Model