A Representative Democracy to Reduce Interdependency in a Multimodel Ensemble

Sanderson, Benjamin M.; Knutti, Reto; Caldwell, Peter

doi:10.1175/jcli-d-14-00362.1

A Representative Democracy to Reduce Interdependency in a Multimodel Ensemble

Journal Article · Wed Jul 01 00:00:00 EDT 2015 · Journal of Climate

DOI:https://doi.org/10.1175/jcli-d-14-00362.1· OSTI ID:1840132

Sanderson, Benjamin M. ^[1]; Knutti, Reto ^[2]; Caldwell, Peter ^[3]

National Center for Atmospheric Research (NCAR), Boulder, CO (United States)
Eidgenoessische Technische Hochschule (ETH), Zurich (Switzerland)
Lawrence Livermore National Lab. (LLNL), Livermore, CA (United States)

The collection of Earth system models available in the archive of phase 5 of CMIP (CMIP5) represents, at least to some degree, a sample of uncertainty of future climate evolution. The presence of duplicated code as well as shared forcing and validation data in the multiple models in the archive raises at least three potential problems: biases in the mean and variance, the overestimation of sample size, and the potential for spurious correlations to emerge in the archive because of model replication. Analytical evidence is presented to demonstrate that the distribution of models in the CMIP5 archive is not consistent with a random sample, and a weighting scheme is proposed to reduce some aspects of model codependency in the ensemble. A method is proposed for selecting diverse and skillful subsets of models in the archive, which could be used for impact studies in cases where physically consistent joint projections of multiple variables (and their temporal and spatial characteristics) are required.

View Accepted Manuscript (DOE)

Research Organization:: Lawrence Livermore National Laboratory (LLNL), Livermore, CA (United States)

Sponsoring Organization:: USDOE National Nuclear Security Administration (NNSA); USDOE Office of Science (SC), Biological and Environmental Research (BER)

Grant/Contract Number:: AC52-07NA27344; FC02-97ER62402

OSTI ID:: 1840132

Report Number(s):: LLNL-JRNL--668949; 790708

Journal Information:: Journal of Climate, Journal Name: Journal of Climate Journal Issue: 13 Vol. 28; ISSN 0894-8755

Publisher:: American Meteorological SocietyCopyright Statement

Country of Publication:: United States

Language:: English

References (35)

Statistical significance of climate sensitivity predictors obtained by data mining Caldwell, Peter M.; Bretherton, Christopher S.; Zelinka, Mark D. Geophysical Research Letters, Vol. 41, Issue 5 https://doi.org/10.1002/2014GL059205	journal	March 2014
Climate model genealogy: Generation CMIP5 and how we got there: CLIMATE MODEL GENEALOGY Knutti, Reto; Masson, David; Gettelman, Andrew Geophysical Research Letters, Vol. 40, Issue 6 https://doi.org/10.1002/grl.50256	journal	March 2013
The flexible global ocean-atmosphere-land system model, Grid-point Version 2: FGOALS-g2 Li, Lijuan; Lin, Pengfei; Yu, Yongqiang Advances in Atmospheric Sciences, Vol. 30, Issue 3 https://doi.org/10.1007/s00376-012-2140-6	journal	April 2013
Climate model dependence and the replicate Earth paradigm Bishop, Craig H.; Abramowitz, Gab Climate Dynamics, Vol. 41, Issue 3-4 https://doi.org/10.1007/s00382-012-1610-y	journal	December 2012
Reliability and importance of structural diversity of climate model ensembles Yokohata, Tokuta; Annan, James D.; Collins, Matthew Climate Dynamics, Vol. 41, Issue 9-10 https://doi.org/10.1007/s00382-013-1733-9	journal	April 2013
On the persistent spread in snow-albedo feedback Qu, Xin; Hall, Alex Climate Dynamics, Vol. 42, Issue 1-2 https://doi.org/10.1007/s00382-013-1774-0	journal	May 2013
The end of model democracy?: An editorial comment Knutti, Reto Climatic Change, Vol. 102, Issue 3-4 https://doi.org/10.1007/s10584-010-9800-2	journal	January 2010
An overview of BCC climate system model development and application for climate change studies Wu, Tongwen; Song, Lianchun; Li, Weiping Acta Meteorologica Sinica, Vol. 28, Issue 1 https://doi.org/10.1007/s13351-014-3041-7	journal	February 2014
Using the current seasonal cycle to constrain snow albedo feedback in future climate change Hall, Alex; Qu, Xin Geophysical Research Letters, Vol. 33, Issue 3 https://doi.org/10.1029/2005GL025127	journal	January 2006
Uncertainty estimates in regional and global observed temperature changes: A new data set from 1850 Brohan, P.; Kennedy, J. J.; Harris, I. Journal of Geophysical Research, Vol. 111, Issue D12 https://doi.org/10.1029/2005JD006548	journal	January 2006
Performance metrics for climate models Gleckler, P. J.; Taylor, K. E.; Doutriaux, C. Journal of Geophysical Research, Vol. 113, Issue D6 https://doi.org/10.1029/2007JD008972	journal	January 2008
Climate model genealogy: CLIMATE MODEL GENEALOGY Masson, D.; Knutti, R. Geophysical Research Letters, Vol. 38, Issue 8 https://doi.org/10.1029/2011GL046864	journal	April 2011
Mapping model agreement on future climate projections: MAPPING MODEL AGREEMENT Tebaldi, Claudia; Arblaster, Julie M.; Knutti, Reto Geophysical Research Letters, Vol. 38, Issue 23 https://doi.org/10.1029/2011GL049863	journal	December 2011
On the interpretation of constrained climate model ensembles: ENSEMBLE INTERPRETATION Sanderson, Benjamin M.; Knutti, Reto Geophysical Research Letters, Vol. 39, Issue 16 https://doi.org/10.1029/2012GL052665	journal	August 2012
Tuning the climate of a global model: TUNING THE CLIMATE OF A GLOBAL MODEL Mauritsen, Thorsten; Stevens, Bjorn; Roeckner, Erich Journal of Advances in Modeling Earth Systems, Vol. 4, Issue 3 https://doi.org/10.1029/2012MS000154	journal	March 2012
Uncertainty in predictions of the climate response to rising levels of greenhouse gases Stainforth, D. A.; Aina, T.; Christensen, C. Nature, Vol. 433, Issue 7024 https://doi.org/10.1038/nature03301	journal	January 2005
Robustness and uncertainties in the new CMIP5 climate model projections Knutti, Reto; Sedláček, Jan Nature Climate Change, Vol. 3, Issue 4 https://doi.org/10.1038/nclimate1716	journal	October 2012
Incorporating model quality information in climate change detection and attribution studies Santer, B. D.; Taylor, K. E.; Gleckler, P. J. Proceedings of the National Academy of Sciences, Vol. 106, Issue 35 https://doi.org/10.1073/pnas.0901736106	journal	August 2009
Second-Order Exchangeability Analysis for Multimodel Ensembles Rougier, Jonathan; Goldstein, Michael; House, Leanna Journal of the American Statistical Association, Vol. 108, Issue 503 https://doi.org/10.1080/01621459.2013.802963	journal	September 2013
Does climate adaptation policy need probabilities? Dessai, Suraje; Hulme, Mike Climate Policy, Vol. 4, Issue 2 https://doi.org/10.1080/14693062.2004.9685515	journal	January 2004
Optimally choosing small ensemble members to produce robust climate simulations Evans, Jason P.; Ji, Fei; Abramowitz, Gab Environmental Research Letters, Vol. 8, Issue 4 https://doi.org/10.1088/1748-9326/8/4/044050	journal	December 2013
The use of the multi-model ensemble in probabilistic climate projections Tebaldi, Claudia; Knutti, Reto Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, Vol. 365, Issue 1857 https://doi.org/10.1098/rsta.2007.2076	journal	June 2007
A methodology for probabilistic predictions of regional climate change from perturbed physics ensembles Murphy, J. M.; Booth, B. B. B.; Collins, M. Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, Vol. 365, Issue 1857 https://doi.org/10.1098/rsta.2007.2077	journal	June 2007
AIRS/AMSU/HSB on the aqua mission: design, science objectives, data products, and processing systems Aumann, H. H.; Chahine, M. T.; Gautier, C. IEEE Transactions on Geoscience and Remote Sensing, Vol. 41, Issue 2 https://doi.org/10.1109/TGRS.2002.808356	journal	February 2003
Local eigenvalue analysis of CMIP3 climate model errors Jun, Mikyoung; Knutti, Reto; Nychka, Douglas W. Tellus A: Dynamic Meteorology and Oceanography, Vol. 60, Issue 5 https://doi.org/10.1111/j.1600-0870.2008.00356.x	journal	January 2008
A Less Cloudy Future: The Role of Subtropical Subsidence in Climate Sensitivity Fasullo, J. T.; Trenberth, K. E. Science, Vol. 338, Issue 6108 https://doi.org/10.1126/science.1227465	journal	November 2012
Simulating present-day climate with the INMCM4.0 coupled model of the atmospheric and oceanic general circulations Volodin, E. M.; Dianskii, N. A.; Gusev, A. V. Izvestiya, Atmospheric and Oceanic Physics, Vol. 46, Issue 4 https://doi.org/10.1134/S000143381004002X	journal	August 2010
Improved Climate Simulation by MIROC5: Mean States, Variability, and Climate Sensitivity Watanabe, Masahiro; Suzuki, Tatsuo; O’ishi, Ryouta Journal of Climate, Vol. 23, Issue 23 https://doi.org/10.1175/2010JCLI3679.1	journal	December 2010
On the Effective Number of Climate Models Pennell, Christopher; Reichler, Thomas Journal of Climate, Vol. 24, Issue 9 https://doi.org/10.1175/2010JCLI3814.1	journal	May 2011
Understanding the CMIP3 Multimodel Ensemble Annan, J. D.; Hargreaves, J. C. Journal of Climate, Vol. 24, Issue 16 https://doi.org/10.1175/2011JCLI3873.1	journal	August 2011
An Overview of CMIP5 and the Experiment Design Taylor, Karl E.; Stouffer, Ronald J.; Meehl, Gerald A. Bulletin of the American Meteorological Society, Vol. 93, Issue 4 https://doi.org/10.1175/BAMS-D-11-00094.1	journal	April 2012
Predictor Screening, Calibration, and Observational Constraints in Climate Model Ensembles: An Illustration Using Climate Sensitivity Masson, David; Knutti, Reto Journal of Climate, Vol. 26, Issue 3 https://doi.org/10.1175/JCLI-D-11-00540.1	journal	February 2013
Ocean Heat Transport and Its Projected Change in CanESM2 Yang, Duo; Saenko, Oleg A. Journal of Climate, Vol. 25, Issue 23 https://doi.org/10.1175/JCLI-D-11-00715.1	journal	December 2012
Addressing Interdependency in a Multimodel Ensemble by Interpolation of Model Properties Sanderson, Benjamin M.; Knutti, Reto; Caldwell, Peter Journal of Climate, Vol. 28, Issue 13 https://doi.org/10.1175/JCLI-D-14-00361.1	journal	July 2015
Australia's CMIP5 submission using the CSIRO-Mk3.6 model Jeffrey, S.; Rotstayn, L.; Collier, M. Australian Meteorological and Oceanographic Journal, Vol. 63, Issue 1 https://doi.org/10.22499/2.6301.001	journal	March 2013

Cited By (45)

Interdependency in Multimodel Climate Projections: Component Replication and Result Similarity Boé, Julien Geophysical Research Letters, Vol. 45, Issue 6 https://doi.org/10.1002/2017gl076829	journal	March 2018
Non-linear effect of sea ice: Spectacled Eider survival declines at both extremes of the ice spectrum Christie, Katherine S.; Hollmen, Tuula E.; Flint, Paul Ecology and Evolution, Vol. 8, Issue 23 https://doi.org/10.1002/ece3.4637	journal	November 2018
Using multi-model ensembles of CMIP5 global climate models to reproduce observed monthly rainfall and temperature with machine learning methods in Australia Wang, Bin; Zheng, Lihong; Liu, De Li International Journal of Climatology, Vol. 38, Issue 13 https://doi.org/10.1002/joc.5705	journal	July 2018
Reducing Uncertainties in Climate Projections with Emergent Constraints: Concepts, Examples and Prospects Brient, Florent Advances in Atmospheric Sciences, Vol. 37, Issue 1 https://doi.org/10.1007/s00376-019-9140-8	journal	December 2019
Projections of southern hemisphere tropical cyclone track density using CMIP5 models Bell, Samuel S.; Chand, Savin S.; Tory, Kevin J. Climate Dynamics, Vol. 52, Issue 9-10 https://doi.org/10.1007/s00382-018-4497-4	journal	October 2018
Ensemble optimisation, multiple constraints and overconfidence: a case study with future Australian precipitation change Herger, Nadja; Abramowitz, Gab; Sherwood, Steven Climate Dynamics, Vol. 53, Issue 3-4 https://doi.org/10.1007/s00382-019-04690-8	journal	April 2019
Asian water tower evinced in total column water vapor: a comparison among multiple satellite and reanalysis data sets Zhao, Yin; Zhou, Tianjun Climate Dynamics, Vol. 54, Issue 1-2 https://doi.org/10.1007/s00382-019-04999-4	journal	October 2019
Projected changes in ENSO-driven regional tropical cyclone tracks Bell, Samuel S.; Chand, Savin S.; Turville, Christopher Climate Dynamics, Vol. 54, Issue 3-4 https://doi.org/10.1007/s00382-020-05129-1	journal	January 2020
Exploring the future change space for fire weather in southeast Australia Clarke, Hamish; Evans, Jason P. Theoretical and Applied Climatology, Vol. 136, Issue 1-2 https://doi.org/10.1007/s00704-018-2507-4	journal	May 2018
Identifying credible and diverse GCMs for regional climate change studies—case study: Northeastern United States Karmalkar, Ambarish V.; Thibeault, Jeanne M.; Bryan, Alexander M. Climatic Change, Vol. 154, Issue 3-4 https://doi.org/10.1007/s10584-019-02411-y	journal	April 2019
Evaluation of methods for selecting climate models to simulate future hydrological change Ross, Andrew C.; Najjar, Raymond G. Climatic Change, Vol. 157, Issue 3-4 https://doi.org/10.1007/s10584-019-02512-8	journal	November 2019
Sensitivity of future climate change and uncertainty over India to performance-based model weighting Singh, Ram; AchutaRao, K. Climatic Change, Vol. 160, Issue 3 https://doi.org/10.1007/s10584-019-02643-y	journal	January 2020
Prospects and Caveats of Weighting Climate Models for Summer Maximum Temperature Projections Over North America Lorenz, Ruth; Herger, Nadja; Sedláček, Jan Journal of Geophysical Research: Atmospheres, Vol. 123, Issue 9 https://doi.org/10.1029/2017jd027992	journal	May 2018
Ranking CMIP5 GCMs for Model Ensemble Selection on Regional Scale: Case Study of the Indochina Region Chhin, Rattana; Yoden, Shigeo Journal of Geophysical Research: Atmospheres, Vol. 123, Issue 17 https://doi.org/10.1029/2017jd028026	journal	September 2018
Usable Science for Managing the Risks of Sea‐Level Rise Kopp, Robert E.; Gilmore, Elisabeth A.; Little, Christopher M. Earth's Future, Vol. 7, Issue 12 https://doi.org/10.1029/2018ef001145	journal	December 2019
Parametric Sensitivity of Vegetation Dynamics in the TRIFFID Model and the Associated Uncertainty in Projected Climate Change Impacts on Western U.S. Forests Hawkins, Linnia R.; Rupp, David E.; McNeall, Doug J. Journal of Advances in Modeling Earth Systems, Vol. 11, Issue 8 https://doi.org/10.1029/2018ms001577	journal	August 2019
Evaluation of Satellite and Reanalysis Precipitable Water Vapor Data Sets Against Radiosonde Observations in Central Asia Jiang, Jie; Zhou, Tianjun; Zhang, Wenxia Earth and Space Science, Vol. 6, Issue 7 https://doi.org/10.1029/2019ea000654	journal	July 2019
The End of the Wait for Climate Sensitivity? Sanderson, Benjamin Geophysical Research Letters, Vol. 46, Issue 21 https://doi.org/10.1029/2019gl084685	journal	November 2019
Beyond equilibrium climate sensitivity Knutti, Reto; Rugenstein, Maria A. A.; Hegerl, Gabriele C. Nature Geoscience, Vol. 10, Issue 10 https://doi.org/10.1038/ngeo3017	journal	September 2017
Learning algorithms allow for improved reliability and accuracy of global mean surface temperature projections Strobach, Ehud; Bel, Golan Nature Communications, Vol. 11, Issue 1 https://doi.org/10.1038/s41467-020-14342-9	journal	January 2020
Taking climate model evaluation to the next level Eyring, Veronika; Cox, Peter M.; Flato, Gregory M. Nature Climate Change, Vol. 9, Issue 2 https://doi.org/10.1038/s41558-018-0355-y	journal	January 2019
Progressing emergent constraints on future climate change Hall, Alex; Cox, Peter; Huntingford, Chris Nature Climate Change, Vol. 9, Issue 4 https://doi.org/10.1038/s41558-019-0436-6	journal	March 2019
Increasing impacts of extreme droughts on vegetation productivity under climate change Xu, Chonggang; McDowell, Nate G.; Fisher, Rosie A. Nature Climate Change, Vol. 9, Issue 12 https://doi.org/10.1038/s41558-019-0630-6	journal	November 2019
A surrogate weighted mean ensemble method to reduce the uncertainty at a regional scale for the calculation of potential evapotranspiration Yoo, Byoung Hyun; Kim, Junhwan; Lee, Byun-Woo Scientific Reports, Vol. 10, Issue 1 https://doi.org/10.1038/s41598-020-57466-0	journal	January 2020
Crop and forest pest metawebs shift towards increased linkage and suitability overlap under climate change Grünig, Marc; Mazzi, Dominique; Calanca, Pierluigi Communications Biology, Vol. 3, Issue 1 https://doi.org/10.1038/s42003-020-0962-9	journal	May 2020
Striking stationarity of large-scale climate model bias patterns under strong climate change Krinner, Gerhard; Flanner, Mark G. Proceedings of the National Academy of Sciences, Vol. 115, Issue 38 https://doi.org/10.1073/pnas.1807912115	journal	September 2018
Ranking of CMIP5-based global climate models using standard performance metrics for Telangana region in the southern part of India Sreelatha, K.; Anand Raj, P. ISH Journal of Hydraulic Engineering https://doi.org/10.1080/09715010.2019.1634648	journal	June 2019
Quantifying uncertainty in European climate projections using combined performance-independence weighting Brunner, Lukas; Lorenz, Ruth; Zumwald, Marius Environmental Research Letters, Vol. 14, Issue 12 https://doi.org/10.1088/1748-9326/ab492f	journal	November 2019
Vegetation demographics in Earth System Models: A review of progress and priorities Fisher, Rosie A.; Koven, Charles D.; Anderegg, William R. L. Global Change Biology, Vol. 24, Issue 1 https://doi.org/10.1111/gcb.13910	journal	October 2017
We need more realistic climate change experiments for understanding ecosystems of the future Korell, Lotte; Auge, Harald; Chase, Jonathan M. Global Change Biology https://doi.org/10.1111/gcb.14797	journal	September 2019
Model complexity affects species distribution projections under climate change Brun, Philipp; Thuiller, Wilfried; Chauvier, Yohann Journal of Biogeography, Vol. 47, Issue 1 https://doi.org/10.1111/jbi.13734	journal	September 2019
Demographic trade-offs predict tropical forest dynamics Rüger, Nadja; Condit, Richard; Dent, Daisy H. Science, Vol. 368, Issue 6487 https://doi.org/10.1126/science.aaz4797	journal	April 2020
Skill and independence weighting for multi-model assessments Sanderson, Benjamin M.; Wehner, Michael; Knutti, Reto ETH Zurich https://doi.org/10.3929/ethz-b-000191237	text	January 2017
Beyond equilibrium climate sensitivity Knutti, Reto; Rugenstein, Maria A. A.; Hegerl, Gabriele C. ETH Zurich https://doi.org/10.3929/ethz-b-000197761	text	January 2017
Selecting a climate model subset to optimise key ensemble properties Nadja, Herger,; Gab, Abramowitz,; Reto, Knutti, ETH Zurich https://doi.org/10.3929/ethz-b-000246202	text	January 2018
Prospects and Caveats of Weighting Climate Models for Summer Maximum Temperature Projections Over North America Lorenz, Ruth; Herger, Nadja; Sedláček, Jan ETH Zurich https://doi.org/10.3929/ethz-b-000268868	text	January 2018
ESD Reviews: Model dependence in multi-model climate ensembles: weighting, sub-selection and out-of-sample testing Abramowitz, Gab; Herger, Nadja; Gutmann, Ethan ETH Zurich https://doi.org/10.3929/ethz-b-000325234	text	January 2019
Crop and forest pest metawebs shift towards increased linkage and suitability overlap under climate change Marc, Grünig,; Dominique, Mazzi,; Pierluigi, Calanca, ETH Zurich https://doi.org/10.3929/ethz-b-000415892	text	January 2020
ESMValTool (v1.0) – a community diagnostic and performance metrics tool for routine evaluation of Earth system models in CMIP Eyring, Veronika; Righi, Mattia; Lauer, Axel Copernicus Publications https://doi.org/10.48350/167138	text	January 2016
ESD Reviews: Model dependence in multi-model climate ensembles: weighting, sub-selection and out-of-sample testing Abramowitz, Gab; Herger, Nadja; Gutmann, Ethan Earth System Dynamics, Vol. 10, Issue 1 https://doi.org/10.5194/esd-10-91-2019	journal	January 2019
Selecting a climate model subset to optimise key ensemble properties Herger, Nadja; Abramowitz, Gab; Knutti, Reto Earth System Dynamics, Vol. 9, Issue 1 https://doi.org/10.5194/esd-9-135-2018	journal	January 2018
Skill and independence weighting for multi-model assessments Sanderson, Benjamin M.; Wehner, Michael; Knutti, Reto Geoscientific Model Development, Vol. 10, Issue 6 https://doi.org/10.5194/gmd-10-2379-2017	journal	January 2017
ESMValTool (v1.0) – a community diagnostic and performance metrics tool for routine evaluation of Earth system models in CMIP Eyring, Veronika; Righi, Mattia; Lauer, Axel Geoscientific Model Development, Vol. 9, Issue 5 https://doi.org/10.5194/gmd-9-1747-2016	journal	January 2016
ESMValTool (v1.0) – a community diagnostic and performance metrics tool for routine evaluation of Earth System Models in CMIP Eyring, V.; Righi, M.; Evaldsson, M. Geoscientific Model Development Discussions, Vol. 8, Issue 9 https://doi.org/10.5194/gmdd-8-7541-2015	journal	January 2015
Does the weighting of climate simulations result in a better quantification of hydrological impacts? Wang, Hui-Min; Chen, Jie; Xu, Chong-Yu Hydrology and Earth System Sciences, Vol. 23, Issue 10 https://doi.org/10.5194/hess-23-4033-2019	journal	January 2019

Similar Records

Addressing Interdependency in a Multimodel Ensemble by Interpolation of Model Properties

Journal Article · Tue Jun 30 20:00:00 EDT 2015 · Journal of Climate · OSTI ID:1840116

Rethinking the Default Construction of Multimodel Climate Ensembles

Journal Article · Mon Jul 20 20:00:00 EDT 2015 · Bulletin of the American Meteorological Society · OSTI ID:1408078

CMIP5 Scientific Gaps and Recommendations for CMIP6

Journal Article · Sun Jan 22 19:00:00 EST 2017 · Bulletin of the American Meteorological Society · OSTI ID:1377777

Related Subjects

54 ENVIRONMENTAL SCIENCES
Geosciences

A Representative Democracy to Reduce Interdependency in a Multimodel Ensemble

Citation Formats

References (35)

Cited By (45)

Similar Records

Related Subjects