Skill and independence weighting for multi-model assessments
Abstract
We present a weighting strategy for use with the CMIP5 multi-model archive in the fourth National Climate Assessment, which considers both skill in the climatological performance of models over North America as well as the inter-dependency of models arising from common parameterizations or tuning practices. The method exploits information relating to the climatological mean state of a number of projection-relevant variables as well as metrics representing long-term statistics of weather extremes. The weights, once computed can be used to simply compute weighted means and significance information from an ensemble containing multiple initial condition members from potentially co-dependent models of varying skill. Two parameters in the algorithm determine the degree to which model climatological skill and model uniqueness are rewarded; these parameters are explored and final values are defended for the assessment. The influence of model weighting on projected temperature and precipitation changes is found to be moderate, partly due to a compensating effect between model skill and uniqueness. However, more aggressive skill weighting and weighting by targeted metrics is found to have a more significant effect on inferred ensemble confidence in future patterns of change for a given projection.
- Authors:
-
[3]
- National Center for Atmospheric Research, Boulder, CO (United States)
- Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States)
- Swiss Federal Inst. of Technology (ETH), Zurich (Switzerland); National Center for Atmospheric Research, Boulder, CO (United States)
- Publication Date:
- Research Org.:
- Lawrence Berkeley National Laboratory (LBNL), Berkeley, CA (United States)
- Sponsoring Org.:
- USDOE Office of Science (SC), Biological and Environmental Research (BER)
- OSTI Identifier:
- 1408439
- Grant/Contract Number:
- AC02-05CH11231; FC02-97ER62402
- Resource Type:
- Accepted Manuscript
- Journal Name:
- Geoscientific Model Development (Online)
- Additional Journal Information:
- Journal Name: Geoscientific Model Development (Online); Journal Volume: 10; Journal Issue: 6; Journal ID: ISSN 1991-9603
- Publisher:
- European Geosciences Union
- Country of Publication:
- United States
- Language:
- English
- Subject:
- 58 GEOSCIENCES
Citation Formats
Sanderson, Benjamin M., Wehner, Michael, and Knutti, Reto. Skill and independence weighting for multi-model assessments. United States: N. p., 2017.
Web. doi:10.5194/gmd-10-2379-2017.
Sanderson, Benjamin M., Wehner, Michael, & Knutti, Reto. Skill and independence weighting for multi-model assessments. United States. https://doi.org/10.5194/gmd-10-2379-2017
Sanderson, Benjamin M., Wehner, Michael, and Knutti, Reto. Wed .
"Skill and independence weighting for multi-model assessments". United States. https://doi.org/10.5194/gmd-10-2379-2017. https://www.osti.gov/servlets/purl/1408439.
@article{osti_1408439,
title = {Skill and independence weighting for multi-model assessments},
author = {Sanderson, Benjamin M. and Wehner, Michael and Knutti, Reto},
abstractNote = {We present a weighting strategy for use with the CMIP5 multi-model archive in the fourth National Climate Assessment, which considers both skill in the climatological performance of models over North America as well as the inter-dependency of models arising from common parameterizations or tuning practices. The method exploits information relating to the climatological mean state of a number of projection-relevant variables as well as metrics representing long-term statistics of weather extremes. The weights, once computed can be used to simply compute weighted means and significance information from an ensemble containing multiple initial condition members from potentially co-dependent models of varying skill. Two parameters in the algorithm determine the degree to which model climatological skill and model uniqueness are rewarded; these parameters are explored and final values are defended for the assessment. The influence of model weighting on projected temperature and precipitation changes is found to be moderate, partly due to a compensating effect between model skill and uniqueness. However, more aggressive skill weighting and weighting by targeted metrics is found to have a more significant effect on inferred ensemble confidence in future patterns of change for a given projection.},
doi = {10.5194/gmd-10-2379-2017},
journal = {Geoscientific Model Development (Online)},
number = 6,
volume = 10,
place = {United States},
year = {Wed Jun 28 00:00:00 EDT 2017},
month = {Wed Jun 28 00:00:00 EDT 2017}
}
Free Publicly Available Full Text
Publisher's Version of Record
Other availability
Search WorldCat to find libraries that may hold this journal
Cited by: 97 works
Citation information provided by
Web of Science
Web of Science
Save to My Library
You must Sign In or Create an Account in order to save documents to your library.
Works referenced in this record:
AIRS/AMSU/HSB on the aqua mission: design, science objectives, data products, and processing systems
journal, February 2003
- Aumann, H. H.; Chahine, M. T.; Gautier, C.
- IEEE Transactions on Geoscience and Remote Sensing, Vol. 41, Issue 2
Climate model genealogy: CLIMATE MODEL GENEALOGY
journal, April 2011
- Masson, D.; Knutti, R.
- Geophysical Research Letters, Vol. 38, Issue 8
Climate model dependence and the replicate Earth paradigm
journal, December 2012
- Bishop, Craig H.; Abramowitz, Gab
- Climate Dynamics, Vol. 41, Issue 3-4
The end of model democracy?: An editorial comment
journal, January 2010
- Knutti, Reto
- Climatic Change, Vol. 102, Issue 3-4
Climate Model Dependence and the Ensemble Dependence Transformation of CMIP Projections
journal, March 2015
- Abramowitz, G.; Bishop, C. H.
- Journal of Climate, Vol. 28, Issue 6
II—Wendy S. Parker: Confirmation and adequacy-for-Purpose in Climate Modelling
journal, June 2009
- Parker, Wendy S.
- Aristotelian Society Supplementary Volume, Vol. 83, Issue 1
A climate model projection weighting scheme accounting for performance and interdependence: Model Projection Weighting Scheme
journal, January 2017
- Knutti, Reto; Sedláček, Jan; Sanderson, Benjamin M.
- Geophysical Research Letters
A Representative Democracy to Reduce Interdependency in a Multimodel Ensemble
journal, July 2015
- Sanderson, Benjamin M.; Knutti, Reto; Caldwell, Peter
- Journal of Climate, Vol. 28, Issue 13
On the Effective Number of Climate Models
journal, May 2011
- Pennell, Christopher; Reichler, Thomas
- Journal of Climate, Vol. 24, Issue 9
Climate model genealogy: Generation CMIP5 and how we got there: CLIMATE MODEL GENEALOGY
journal, March 2013
- Knutti, Reto; Masson, David; Gettelman, Andrew
- Geophysical Research Letters, Vol. 40, Issue 6
The ERA-40 re-analysis
journal, October 2005
- Uppala, S. M.; KÅllberg, P. W.; Simmons, A. J.
- Quarterly Journal of the Royal Meteorological Society, Vol. 131, Issue 612
Skill of CMIP5 climate models in reproducing 20th century basic climate features in Central America: SKILL OF CMIP5 CLIMATE MODELS IN CENTRAL AMERICA
journal, December 2014
- Hidalgo, Hugo G.; Alfaro, Eric J.
- International Journal of Climatology, Vol. 35, Issue 12
Understanding the CMIP3 Multimodel Ensemble
journal, August 2011
- Annan, J. D.; Hargreaves, J. C.
- Journal of Climate, Vol. 24, Issue 16
Development and Testing of Canada-Wide Interpolated Spatial Models of Daily Minimum–Maximum Temperature and Precipitation for 1961–2003
journal, April 2009
- Hutchinson, Michael F.; McKenney, Dan W.; Lawrence, Kevin
- Journal of Applied Meteorology and Climatology, Vol. 48, Issue 4
An Overview of CMIP5 and the Experiment Design
journal, April 2012
- Taylor, Karl E.; Stouffer, Ronald J.; Meehl, Gerald A.
- Bulletin of the American Meteorological Society, Vol. 93, Issue 4
On the interpretation of constrained climate model ensembles: ENSEMBLE INTERPRETATION
journal, August 2012
- Sanderson, Benjamin M.; Knutti, Reto
- Geophysical Research Letters, Vol. 39, Issue 16
Challenges in Combining Projections from Multiple Climate Models
journal, May 2010
- Knutti, Reto; Furrer, Reinhard; Tebaldi, Claudia
- Journal of Climate, Vol. 23, Issue 10
Climate extremes indices in the CMIP5 multimodel ensemble: Part 2. Future climate projections: CMIP5 PROJECTIONS OF EXTREMES INDICES
journal, March 2013
- Sillmann, J.; Kharin, V. V.; Zwiers, F. W.
- Journal of Geophysical Research: Atmospheres, Vol. 118, Issue 6
The use of the multi-model ensemble in probabilistic climate projections
journal, June 2007
- Tebaldi, Claudia; Knutti, Reto
- Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, Vol. 365, Issue 1857
A climate model projection weighting scheme accounting for performance and interdependence
text, January 2017
- Knutti, Reto; Sedláček, Jan; Sanderson, Benjamin M.
- ETH Zurich
Challenges in combining projections from multiple climate models
text, January 2010
- Cermak, J.; Furrer, R.; Knutti, R.
- AMS, Boston MA
The end of model democracy?: An editorial comment
journal, January 2010
- Knutti, Reto
- Climatic Change, Vol. 102, Issue 3-4
The use of the multi-model ensemble in probabilistic climate projections
journal, June 2007
- Tebaldi, Claudia; Knutti, Reto
- Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, Vol. 365, Issue 1857
Calculation of Average, Uncertainty Range, and Reliability of Regional Climate Changes from AOGCM Simulations via the “Reliability Ensemble Averaging” (REA) Method
journal, May 2002
- Giorgi, Filippo; Mearns, Linda O.
- Journal of Climate, Vol. 15, Issue 10
David Draper and E. I. George, and a rejoinder by the authors
journal, November 1999
- Volinsky, Chris T.; Raftery, Adrian E.; Madigan, David
- Statistical Science, Vol. 14, Issue 4
Works referencing / citing this record:
Interdependency in Multimodel Climate Projections: Component Replication and Result Similarity
journal, March 2018
- Boé, Julien
- Geophysical Research Letters, Vol. 45, Issue 6
Biased estimates of the isotope ratios of steady‐state evaporation from the assumption of equilibrium between vapour and precipitation
journal, July 2019
- Fiorella, Richard P.; West, Jason B.; Bowen, Gabriel J.
- Hydrological Processes, Vol. 33, Issue 19
Assessment of uncertainty in multi‐model means of downscaled South Florida precipitation for projected (2019–2099) climate
journal, April 2020
- Infanti, Johnna M.; Kirtman, Ben P.; Aumen, Nicholas G.
- International Journal of Climatology, Vol. 40, Issue 5
Ensemble optimisation, multiple constraints and overconfidence: a case study with future Australian precipitation change
journal, April 2019
- Herger, Nadja; Abramowitz, Gab; Sherwood, Steven
- Climate Dynamics, Vol. 53, Issue 3-4
Using indices of atmospheric circulation to refine southern Australian winter rainfall climate projections
journal, June 2019
- Grose, Michael R.; Foster, Scott; Risbey, James S.
- Climate Dynamics, Vol. 53, Issue 9-10
Asian water tower evinced in total column water vapor: a comparison among multiple satellite and reanalysis data sets
journal, October 2019
- Zhao, Yin; Zhou, Tianjun
- Climate Dynamics, Vol. 54, Issue 1-2
Weighted multi-model ensemble projection of extreme precipitation in the Mediterranean region using statistical downscaling
journal, May 2019
- Keupp, Luzia; Hertig, Elke; Kaspar-Ott, Irena
- Theoretical and Applied Climatology, Vol. 138, Issue 3-4
Sensitivity of future climate change and uncertainty over India to performance-based model weighting
journal, January 2020
- Singh, Ram; AchutaRao, K.
- Climatic Change, Vol. 160, Issue 3
Ranking CMIP5 GCMs for Model Ensemble Selection on Regional Scale: Case Study of the Indochina Region
journal, September 2018
- Chhin, Rattana; Yoden, Shigeo
- Journal of Geophysical Research: Atmospheres, Vol. 123, Issue 17
Evaluation of Satellite and Reanalysis Precipitable Water Vapor Data Sets Against Radiosonde Observations in Central Asia
journal, July 2019
- Jiang, Jie; Zhou, Tianjun; Zhang, Wenxia
- Earth and Space Science, Vol. 6, Issue 7
Spatiotemporal Reliability Ensemble Averaging of Multimodel Simulations
journal, November 2019
- Tegegne, Getachew; Kim, Young‐Oh; Lee, Jae‐Kyoung
- Geophysical Research Letters, Vol. 46, Issue 21
Beyond equilibrium climate sensitivity
journal, September 2017
- Knutti, Reto; Rugenstein, Maria A. A.; Hegerl, Gabriele C.
- Nature Geoscience, Vol. 10, Issue 10
Learning algorithms allow for improved reliability and accuracy of global mean surface temperature projections
journal, January 2020
- Strobach, Ehud; Bel, Golan
- Nature Communications, Vol. 11, Issue 1
Taking climate model evaluation to the next level
journal, January 2019
- Eyring, Veronika; Cox, Peter M.; Flato, Gregory M.
- Nature Climate Change, Vol. 9, Issue 2
Precipitation regime change in Western North America: The role of Atmospheric Rivers
journal, July 2019
- Gershunov, Alexander; Shulgina, Tamara; Clemesha, Rachel E. S.
- Scientific Reports, Vol. 9, Issue 1
A surrogate weighted mean ensemble method to reduce the uncertainty at a regional scale for the calculation of potential evapotranspiration
journal, January 2020
- Yoo, Byoung Hyun; Kim, Junhwan; Lee, Byun-Woo
- Scientific Reports, Vol. 10, Issue 1
Ranking of CMIP5-based global climate models using standard performance metrics for Telangana region in the southern part of India
journal, June 2019
- Sreelatha, K.; Anand Raj, P.
- ISH Journal of Hydraulic Engineering
Quantifying uncertainty in European climate projections using combined performance-independence weighting
journal, November 2019
- Brunner, Lukas; Lorenz, Ruth; Zumwald, Marius
- Environmental Research Letters, Vol. 14, Issue 12
A Proposal to Evaluate Drought Characteristics Using Multiple Climate Models for Multiple Timescales
journal, September 2018
- Tien Thanh, Nguyen
- Climate, Vol. 6, Issue 4
ESD Reviews: Model dependence in multi-model climate ensembles: weighting, sub-selection and out-of-sample testing
journal, January 2019
- Abramowitz, Gab; Herger, Nadja; Gutmann, Ethan
- Earth System Dynamics, Vol. 10, Issue 1
Selecting a climate model subset to optimise key ensemble properties
journal, January 2018
- Herger, Nadja; Abramowitz, Gab; Knutti, Reto
- Earth System Dynamics, Vol. 9, Issue 1
Similarities within a multi-model ensemble: functional data analysis framework
journal, January 2019
- Holtanová, Eva; Mendlik, Thomas; Koláček, Jan
- Geoscientific Model Development, Vol. 12, Issue 2
Does the weighting of climate simulations result in a better quantification of hydrological impacts?
journal, January 2019
- Wang, Hui-Min; Chen, Jie; Xu, Chong-Yu
- Hydrology and Earth System Sciences, Vol. 23, Issue 10
A Representative Democracy to Reduce Interdependency in a Multimodel Ensemble
journal, July 2015
- Sanderson, Benjamin M.; Knutti, Reto; Caldwell, Peter
- Journal of Climate, Vol. 28, Issue 13
Prospects and Caveats of Weighting Climate Models for Summer Maximum Temperature Projections Over North America
text, January 2018
- Lorenz, Ruth; Herger, Nadja; Sedláček, Jan
- ETH Zurich
Taking climate model evaluation to the next level
text, January 2019
- Veronika, Eyring,; M., Cox, Peter; M., Flato, Gregory
- ETH Zurich
ESD Reviews: Model dependence in multi-model climate ensembles: weighting, sub-selection and out-of-sample testing
text, January 2019
- Abramowitz, Gab; Herger, Nadja; Gutmann, Ethan
- ETH Zurich
Quantifying uncertainty in European climate projections using combined performance-independence weighting
text, January 2019
- Brunner, Lukas; Lorenz, Ruth; Zumwald, Marius
- ETH Zurich
Beyond equilibrium climate sensitivity
text, January 2017
- Knutti, Reto; Rugenstein, Maria A. A.; Hegerl, Gabriele C.
- ETH Zurich
Selecting a climate model subset to optimise key ensemble properties
text, January 2018
- Nadja, Herger,; Gab, Abramowitz,; Reto, Knutti,
- ETH Zurich
Learning algorithms allow for improved reliability and accuracy of global mean surface temperature projections
journal, January 2020
- Strobach, Ehud; Bel, Golan
- Nature Communications, Vol. 11, Issue 1
A surrogate weighted mean ensemble method to reduce the uncertainty at a regional scale for the calculation of potential evapotranspiration
journal, January 2020
- Yoo, Byoung Hyun; Kim, Junhwan; Lee, Byun-Woo
- Scientific Reports, Vol. 10, Issue 1
Ensembling geophysical models with Bayesian Neural Networks
text, January 2020
- Sengupta, Ushnish; Amos, Matt; Hosking, J. Scott
- arXiv