skip to main content
DOE PAGES title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Efficient Screening of Climate Model Sensitivity to a Large Number of Perturbed Input Parameters [plus supporting information]

Abstract

Modern climate models contain numerous input parameters, each with a range of possible values. Since the volume of parameter space increases exponentially with the number of parameters N, it is generally impossible to directly evaluate a model throughout this space even if just 2-3 values are chosen for each parameter. Sensitivity screening algorithms, however, can identify input parameters having relatively little effect on a variety of output fields, either individually or in nonlinear combination.This can aid both model development and the uncertainty quantification (UQ) process. Here we report results from a parameter sensitivity screening algorithm hitherto untested in climate modeling, the Morris one-at-a-time (MOAT) method. This algorithm drastically reduces the computational cost of estimating sensitivities in a high dimensional parameter space because the sample size grows linearly rather than exponentially with N. It nevertheless samples over much of the N-dimensional volume and allows assessment of parameter interactions, unlike traditional elementary one-at-a-time (EOAT) parameter variation. We applied both EOAT and MOAT to the Community Atmosphere Model (CAM), assessing CAM’s behavior as a function of 27 uncertain input parameters related to the boundary layer, clouds, and other subgrid scale processes. For radiation balance at the top of the atmosphere, EOAT and MOATmore » rank most input parameters similarly, but MOAT identifies a sensitivity that EOAT underplays for two convection parameters that operate nonlinearly in the model. MOAT’s ranking of input parameters is robust to modest algorithmic variations, and it is qualitatively consistent with model development experience. Supporting information is also provided at the end of the full text of the article.« less

Authors:
 [1];  [1];  [1];  [1];  [2]
  1. Lawrence Livermore National Lab. (LLNL), Livermore, CA (United States)
  2. Lawrence Livermore National Lab. (LLNL), Livermore, CA (United States); Univ. of California, Berkeley, CA (United States). Dept. of Astronomy
Publication Date:
Research Org.:
Lawrence Livermore National Lab. (LLNL), Livermore, CA (United States)
Sponsoring Org.:
USDOE
OSTI Identifier:
1240071
Report Number(s):
LLNL-JRNL-626954
Journal ID: ISSN 1942-2466
Grant/Contract Number:  
AC52-07NA27344
Resource Type:
Accepted Manuscript
Journal Name:
Journal of Advances in Modeling Earth Systems
Additional Journal Information:
Journal Volume: 5; Journal Issue: 3; Journal ID: ISSN 1942-2466
Publisher:
American Geophysical Union (AGU)
Country of Publication:
United States
Language:
English
Subject:
54 ENVIRONMENTAL SCIENCES; 58 GEOSCIENCES; 97 MATHEMATICS, COMPUTING, AND INFORMATION SCIENCE

Citation Formats

Covey, Curt, Lucas, Donald D., Tannahill, John, Garaizar, Xabier, and Klein, Richard. Efficient Screening of Climate Model Sensitivity to a Large Number of Perturbed Input Parameters [plus supporting information]. United States: N. p., 2013. Web. doi:10.1002/jame.20040.
Covey, Curt, Lucas, Donald D., Tannahill, John, Garaizar, Xabier, & Klein, Richard. Efficient Screening of Climate Model Sensitivity to a Large Number of Perturbed Input Parameters [plus supporting information]. United States. doi:10.1002/jame.20040.
Covey, Curt, Lucas, Donald D., Tannahill, John, Garaizar, Xabier, and Klein, Richard. Mon . "Efficient Screening of Climate Model Sensitivity to a Large Number of Perturbed Input Parameters [plus supporting information]". United States. doi:10.1002/jame.20040. https://www.osti.gov/servlets/purl/1240071.
@article{osti_1240071,
title = {Efficient Screening of Climate Model Sensitivity to a Large Number of Perturbed Input Parameters [plus supporting information]},
author = {Covey, Curt and Lucas, Donald D. and Tannahill, John and Garaizar, Xabier and Klein, Richard},
abstractNote = {Modern climate models contain numerous input parameters, each with a range of possible values. Since the volume of parameter space increases exponentially with the number of parameters N, it is generally impossible to directly evaluate a model throughout this space even if just 2-3 values are chosen for each parameter. Sensitivity screening algorithms, however, can identify input parameters having relatively little effect on a variety of output fields, either individually or in nonlinear combination.This can aid both model development and the uncertainty quantification (UQ) process. Here we report results from a parameter sensitivity screening algorithm hitherto untested in climate modeling, the Morris one-at-a-time (MOAT) method. This algorithm drastically reduces the computational cost of estimating sensitivities in a high dimensional parameter space because the sample size grows linearly rather than exponentially with N. It nevertheless samples over much of the N-dimensional volume and allows assessment of parameter interactions, unlike traditional elementary one-at-a-time (EOAT) parameter variation. We applied both EOAT and MOAT to the Community Atmosphere Model (CAM), assessing CAM’s behavior as a function of 27 uncertain input parameters related to the boundary layer, clouds, and other subgrid scale processes. For radiation balance at the top of the atmosphere, EOAT and MOAT rank most input parameters similarly, but MOAT identifies a sensitivity that EOAT underplays for two convection parameters that operate nonlinearly in the model. MOAT’s ranking of input parameters is robust to modest algorithmic variations, and it is qualitatively consistent with model development experience. Supporting information is also provided at the end of the full text of the article.},
doi = {10.1002/jame.20040},
journal = {Journal of Advances in Modeling Earth Systems},
number = 3,
volume = 5,
place = {United States},
year = {2013},
month = {7}
}

Journal Article:
Free Publicly Available Full Text
Publisher's Version of Record

Citation Metrics:
Cited by: 2 works
Citation information provided by
Web of Science

Save / Share:

Works referenced in this record:

A probabilistic approach for analysis of uncertainty in the evaluation of watershed management practices
journal, February 2007


Future research challenges for incorporation of uncertainty in environmental and ecological decision-making
journal, December 2008


An effective screening design for sensitivity analysis of large models
journal, October 2007

  • Campolongo, Francesca; Cariboni, Jessica; Saltelli, Andrea
  • Environmental Modelling & Software, Vol. 22, Issue 10
  • DOI: 10.1016/j.envsoft.2006.10.004

The role of sensitivity analysis in ecological modelling
journal, April 2007


Modeling Northern Hemisphere Summer Heat Extreme Changes and Their Uncertainties Using a Physics Ensemble of Climate Sensitivity Experiments
journal, September 2006

  • Clark, Robin T.; Brown, Simon J.; Murphy, James M.
  • Journal of Climate, Vol. 19, Issue 17
  • DOI: 10.1175/JCLI3877.1

Climate model errors, feedbacks and forcings: a comparison of perturbed physics and multi-model ensembles
journal, May 2010


The Formulation and Atmospheric Simulation of the Community Atmosphere Model Version 3 (CAM3)
journal, June 2006

  • Collins, William D.; Rasch, Philip J.; Boville, Byron A.
  • Journal of Climate, Vol. 19, Issue 11
  • DOI: 10.1175/JCLI3760.1

Analysis of uncertainty in building design evaluations and its implications
journal, October 2002


Efficient calculation of sensitivity coefficients for complex atmospheric models
journal, January 1981


An Overview of the Results of the Atmospheric Model Intercomparison Project (AMIP I)
journal, January 1999


Improvements in a half degree atmosphere/land version of the CCSM
journal, July 2009


Performance metrics for climate models
journal, January 2008

  • Gleckler, P. J.; Taylor, K. E.; Doutriaux, C.
  • Journal of Geophysical Research, Vol. 113, Issue D6
  • DOI: 10.1029/2007JD008972

Parameterization of moist convection in the National Center for Atmospheric Research community climate model (CCM2)
journal, January 1994

  • Hack, James J.
  • Journal of Geophysical Research, Vol. 99, Issue D3
  • DOI: 10.1029/93JD03478

Local Versus Nonlocal Boundary-Layer Diffusion in a Global Climate Model
journal, October 1993


Sensitivity analysis for hydrology and pesticide supply towards the river in SWAT
journal, January 2005

  • Holvoet, K.; van Griensven, A.; Seuntjens, P.
  • Physics and Chemistry of the Earth, Parts A/B/C, Vol. 30, Issue 8-10
  • DOI: 10.1016/j.pce.2005.07.006

The Dynamical Simulation of the Community Atmosphere Model Version 3 (CAM3)
journal, June 2006

  • Hurrell, James W.; Hack, James J.; Phillips, Adam S.
  • Journal of Climate, Vol. 19, Issue 11
  • DOI: 10.1175/JCLI3762.1

Error Reduction and Convergence in Climate Prediction
journal, December 2008

  • Jackson, Charles S.; Sen, Mrinal K.; Huerta, Gabriel
  • Journal of Climate, Vol. 21, Issue 24
  • DOI: 10.1175/2008JCLI2112.1

Global Sensitivity Analysis Challenges in Biological Systems Modeling
journal, August 2009

  • Kiparissides, A.; Kucherenko, S. S.; Mantalaris, A.
  • Industrial & Engineering Chemistry Research, Vol. 48, Issue 15
  • DOI: 10.1021/ie900139x

On Constraining Estimates of Climate Sensitivity with Present-Day Observations through Model Weighting
journal, December 2011

  • Klocke, Daniel; Pincus, Robert; Quaas, Johannes
  • Journal of Climate, Vol. 24, Issue 23
  • DOI: 10.1175/2011JCLI4193.1

Mapping the uncertainty in global CCN using emulation
journal, January 2012

  • Lee, L. A.; Carslaw, K. S.; Pringle, K. J.
  • Atmospheric Chemistry and Physics, Vol. 12, Issue 20
  • DOI: 10.5194/acp-12-9739-2012

Assessing parameter importance of the Common Land Model based on qualitative and quantitative sensitivity analysis
journal, January 2013

  • Li, J. D.; Duan, Q. Y.; Gong, W.
  • Hydrology and Earth System Sciences Discussions, Vol. 10, Issue 2
  • DOI: 10.5194/hessd-10-2243-2013

Failure analysis of parameter-induced simulation crashes in climate models
journal, January 2013

  • Lucas, D. D.; Klein, R.; Tannahill, J.
  • Geoscientific Model Development Discussions, Vol. 6, Issue 1
  • DOI: 10.5194/gmdd-6-585-2013

THE WCRP CMIP3 Multimodel Dataset: A New Era in Climate Change Research
journal, September 2007

  • Meehl, Gerald A.; Covey, Curt; Delworth, Thomas
  • Bulletin of the American Meteorological Society, Vol. 88, Issue 9
  • DOI: 10.1175/BAMS-88-9-1383

Factorial Sampling Plans for Preliminary Computational Experiments
journal, May 1991


Quantification of modelling uncertainties in a large ensemble of climate change simulations
journal, August 2004

  • Murphy, James M.; Sexton, David M. H.; Barnett, David N.
  • Nature, Vol. 430, Issue 7001
  • DOI: 10.1038/nature02771

Considerations for parameter optimization and sensitivity in climate models
journal, November 2010

  • Neelin, J. D.; Bracco, A.; Luo, H.
  • Proceedings of the National Academy of Sciences, Vol. 107, Issue 50
  • DOI: 10.1073/pnas.1015473107

Simplex-based screening designs for estimating metamodels
journal, July 2009


Sensitivity analysis: Could better methods be used?
journal, February 1999

  • Saltelli, Andrea
  • Journal of Geophysical Research: Atmospheres, Vol. 104, Issue D3
  • DOI: 10.1029/1998JD100042

Uncertainty in predictions of the climate response to rising levels of greenhouse gases
journal, January 2005

  • Stainforth, D. A.; Aina, T.; Christensen, C.
  • Nature, Vol. 433, Issue 7024
  • DOI: 10.1038/nature03301

Global sensitivity analysis using polynomial chaos expansions
journal, July 2008


Poster: Data intensive uncertainty quantification: applications to climate modeling
conference, January 2011

  • Tannahill, John; Lucas, Donald D.; Domyancic, David
  • Proceedings of the 2011 companion on High Performance Computing Networking, Storage and Analysis Companion - SC '11 Companion
  • DOI: 10.1145/2148600.2148610

Combining Quantitative and Qualitative Measures of Uncertainty in Model‐Based Environmental Assessment: The NUSAP System
journal, April 2005


Structural Similarities and Differences in Climate Responses to CO 2 Increase between Two Perturbed Physics Ensembles
journal, March 2010

  • Yokohata, Tokuta; Webb, Mark J.; Collins, Matthew
  • Journal of Climate, Vol. 23, Issue 6
  • DOI: 10.1175/2009JCLI2917.1

Sensitivity analysis and robust experimental design of a signal transduction pathway system
journal, November 2008

  • Yue, Hong; Brown, Martin; He, Fei
  • International Journal of Chemical Kinetics, Vol. 40, Issue 11
  • DOI: 10.1002/kin.20369

Regional assessment of the parameter-dependent performance of CAM4 in simulating tropical clouds: REGIONAL TROPICAL CLOUDS IN CAM4
journal, July 2012

  • Zhang, Yuying; Xie, Shaocheng; Covey, Curt
  • Geophysical Research Letters, Vol. 39, Issue 14
  • DOI: 10.1029/2012GL052184

    Works referencing / citing this record:

    Finding plausible and diverse variants of a climate model. Part II: development and validation of methodology
    journal, February 2019

    • Karmalkar, Ambarish V.; Sexton, David M. H.; Murphy, James M.
    • Climate Dynamics, Vol. 53, Issue 1-2
    • DOI: 10.1007/s00382-019-04617-3

    Finding plausible and diverse variants of a climate model. Part 1: establishing the relationship between errors at weather and climate time scales
    journal, February 2019


    Finding plausible and diverse variants of a climate model. Part II: development and validation of methodology
    journal, February 2019

    • Karmalkar, Ambarish V.; Sexton, David M. H.; Murphy, James M.
    • Climate Dynamics, Vol. 53, Issue 1-2
    • DOI: 10.1007/s00382-019-04617-3

    Finding plausible and diverse variants of a climate model. Part 1: establishing the relationship between errors at weather and climate time scales
    journal, February 2019