Skill and independence weighting for multi-model assessments

Sanderson, Benjamin M.; Wehner, Michael; Knutti, Reto

doi:10.5194/gmd-10-2379-2017

Title: Skill and independence weighting for multi-model assessments

Journal Article · Wed Jun 28 00:00:00 EDT 2017 · Geoscientific Model Development (Online)

DOI:https://doi.org/10.5194/gmd-10-2379-2017· OSTI ID:1408439

Sanderson, Benjamin M. ^[1]; Wehner, Michael ^[2];

^[3]

National Center for Atmospheric Research, Boulder, CO (United States)
Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States)
Swiss Federal Inst. of Technology (ETH), Zurich (Switzerland); National Center for Atmospheric Research, Boulder, CO (United States)

We present a weighting strategy for use with the CMIP5 multi-model archive in the fourth National Climate Assessment, which considers both skill in the climatological performance of models over North America as well as the inter-dependency of models arising from common parameterizations or tuning practices. The method exploits information relating to the climatological mean state of a number of projection-relevant variables as well as metrics representing long-term statistics of weather extremes. The weights, once computed can be used to simply compute weighted means and significance information from an ensemble containing multiple initial condition members from potentially co-dependent models of varying skill. Two parameters in the algorithm determine the degree to which model climatological skill and model uniqueness are rewarded; these parameters are explored and final values are defended for the assessment. The influence of model weighting on projected temperature and precipitation changes is found to be moderate, partly due to a compensating effect between model skill and uniqueness. However, more aggressive skill weighting and weighting by targeted metrics is found to have a more significant effect on inferred ensemble confidence in future patterns of change for a given projection.

View Accepted Manuscript (DOE)

Cite

Export

Save

Research Organization:: Lawrence Berkeley National Laboratory (LBNL), Berkeley, CA (United States)

Sponsoring Organization:: USDOE Office of Science (SC), Biological and Environmental Research (BER)

Grant/Contract Number:: AC02-05CH11231; FC02-97ER62402

OSTI ID:: 1408439

Journal Information:: Geoscientific Model Development (Online), Vol. 10, Issue 6; ISSN 1991-9603

Publisher:: European Geosciences UnionCopyright Statement

Country of Publication:: United States

Language:: English

Citation Metrics:

Cited by: 97 works

Citation information provided by
Web of Science

References (24)

AIRS/AMSU/HSB on the aqua mission: design, science objectives, data products, and processing systems Aumann, H. H.; Chahine, M. T.; Gautier, C. IEEE Transactions on Geoscience and Remote Sensing, Vol. 41, Issue 2 https://doi.org/10.1109/TGRS.2002.808356	journal	February 2003
Climate model genealogy: CLIMATE MODEL GENEALOGY Masson, D.; Knutti, R. Geophysical Research Letters, Vol. 38, Issue 8 https://doi.org/10.1029/2011GL046864	journal	April 2011
Climate model dependence and the replicate Earth paradigm Bishop, Craig H.; Abramowitz, Gab Climate Dynamics, Vol. 41, Issue 3-4 https://doi.org/10.1007/s00382-012-1610-y	journal	December 2012
The end of model democracy?: An editorial comment Knutti, Reto Climatic Change, Vol. 102, Issue 3-4 https://doi.org/10.1007/s10584-010-9800-2	journal	January 2010
Climate Model Dependence and the Ensemble Dependence Transformation of CMIP Projections Abramowitz, G.; Bishop, C. H. Journal of Climate, Vol. 28, Issue 6 https://doi.org/10.1175/JCLI-D-14-00364.1	journal	March 2015
II—Wendy S. Parker: Confirmation and adequacy-for-Purpose in Climate Modelling Parker, Wendy S. Aristotelian Society Supplementary Volume, Vol. 83, Issue 1 https://doi.org/10.1111/j.1467-8349.2009.00180.x	journal	June 2009
A climate model projection weighting scheme accounting for performance and interdependence: Model Projection Weighting Scheme Knutti, Reto; Sedláček, Jan; Sanderson, Benjamin M. Geophysical Research Letters https://doi.org/10.1002/2016GL072012	journal	January 2017
A Representative Democracy to Reduce Interdependency in a Multimodel Ensemble Sanderson, Benjamin M.; Knutti, Reto; Caldwell, Peter Journal of Climate, Vol. 28, Issue 13 https://doi.org/10.1175/JCLI-D-14-00362.1	journal	July 2015
On the Effective Number of Climate Models Pennell, Christopher; Reichler, Thomas Journal of Climate, Vol. 24, Issue 9 https://doi.org/10.1175/2010JCLI3814.1	journal	May 2011
Climate model genealogy: Generation CMIP5 and how we got there: CLIMATE MODEL GENEALOGY Knutti, Reto; Masson, David; Gettelman, Andrew Geophysical Research Letters, Vol. 40, Issue 6 https://doi.org/10.1002/grl.50256	journal	March 2013
The ERA-40 re-analysis Uppala, S. M.; KÅllberg, P. W.; Simmons, A. J. Quarterly Journal of the Royal Meteorological Society, Vol. 131, Issue 612 https://doi.org/10.1256/qj.04.176	journal	October 2005
Skill of CMIP5 climate models in reproducing 20th century basic climate features in Central America: SKILL OF CMIP5 CLIMATE MODELS IN CENTRAL AMERICA Hidalgo, Hugo G.; Alfaro, Eric J. International Journal of Climatology, Vol. 35, Issue 12 https://doi.org/10.1002/joc.4216	journal	December 2014
Understanding the CMIP3 Multimodel Ensemble Annan, J. D.; Hargreaves, J. C. Journal of Climate, Vol. 24, Issue 16 https://doi.org/10.1175/2011JCLI3873.1	journal	August 2011
Development and Testing of Canada-Wide Interpolated Spatial Models of Daily Minimum–Maximum Temperature and Precipitation for 1961–2003 Hutchinson, Michael F.; McKenney, Dan W.; Lawrence, Kevin Journal of Applied Meteorology and Climatology, Vol. 48, Issue 4 https://doi.org/10.1175/2008JAMC1979.1	journal	April 2009
An Overview of CMIP5 and the Experiment Design Taylor, Karl E.; Stouffer, Ronald J.; Meehl, Gerald A. Bulletin of the American Meteorological Society, Vol. 93, Issue 4 https://doi.org/10.1175/BAMS-D-11-00094.1	journal	April 2012
On the interpretation of constrained climate model ensembles: ENSEMBLE INTERPRETATION Sanderson, Benjamin M.; Knutti, Reto Geophysical Research Letters, Vol. 39, Issue 16 https://doi.org/10.1029/2012GL052665	journal	August 2012
Challenges in Combining Projections from Multiple Climate Models Knutti, Reto; Furrer, Reinhard; Tebaldi, Claudia Journal of Climate, Vol. 23, Issue 10 https://doi.org/10.1175/2009JCLI3361.1	journal	May 2010
Climate extremes indices in the CMIP5 multimodel ensemble: Part 2. Future climate projections: CMIP5 PROJECTIONS OF EXTREMES INDICES Sillmann, J.; Kharin, V. V.; Zwiers, F. W. Journal of Geophysical Research: Atmospheres, Vol. 118, Issue 6 https://doi.org/10.1002/jgrd.50188	journal	March 2013
The use of the multi-model ensemble in probabilistic climate projections Tebaldi, Claudia; Knutti, Reto Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, Vol. 365, Issue 1857 https://doi.org/10.1098/rsta.2007.2076	journal	June 2007
The end of model democracy? Reto, Knutti, ETH Zurich https://doi.org/10.3929/ethz-b-000023411	text	January 2010
A climate model projection weighting scheme accounting for performance and interdependence Knutti, Reto; Sedláček, Jan; Sanderson, Benjamin M. ETH Zurich https://doi.org/10.3929/ethz-b-000129180	text	January 2017
Challenges in combining projections from multiple climate models Cermak, J.; Furrer, R.; Knutti, R. AMS, Boston MA https://doi.org/10.5167/uzh-35822	text	January 2010
Calculation of Average, Uncertainty Range, and Reliability of Regional Climate Changes from AOGCM Simulations via the “Reliability Ensemble Averaging” (REA) Method Giorgi, Filippo; Mearns, Linda O. Journal of Climate, Vol. 15, Issue 10 https://doi.org/10.1175/1520-0442(2002)015<1141:coaura>2.0.co;2	journal	May 2002
David Draper and E. I. George, and a rejoinder by the authors Volinsky, Chris T.; Raftery, Adrian E.; Madigan, David Statistical Science, Vol. 14, Issue 4 https://doi.org/10.1214/ss/1009212519	journal	November 1999

Cited By (31)

Interdependency in Multimodel Climate Projections: Component Replication and Result Similarity Boé, Julien Geophysical Research Letters, Vol. 45, Issue 6 https://doi.org/10.1002/2017gl076829	journal	March 2018
Biased estimates of the isotope ratios of steady‐state evaporation from the assumption of equilibrium between vapour and precipitation Fiorella, Richard P.; West, Jason B.; Bowen, Gabriel J. Hydrological Processes, Vol. 33, Issue 19 https://doi.org/10.1002/hyp.13531	journal	July 2019
Assessment of uncertainty in multi‐model means of downscaled South Florida precipitation for projected (2019–2099) climate Infanti, Johnna M.; Kirtman, Ben P.; Aumen, Nicholas G. International Journal of Climatology, Vol. 40, Issue 5 https://doi.org/10.1002/joc.6365	journal	April 2020
Ensemble optimisation, multiple constraints and overconfidence: a case study with future Australian precipitation change Herger, Nadja; Abramowitz, Gab; Sherwood, Steven Climate Dynamics, Vol. 53, Issue 3-4 https://doi.org/10.1007/s00382-019-04690-8	journal	April 2019
Using indices of atmospheric circulation to refine southern Australian winter rainfall climate projections Grose, Michael R.; Foster, Scott; Risbey, James S. Climate Dynamics, Vol. 53, Issue 9-10 https://doi.org/10.1007/s00382-019-04880-4	journal	June 2019
Asian water tower evinced in total column water vapor: a comparison among multiple satellite and reanalysis data sets Zhao, Yin; Zhou, Tianjun Climate Dynamics, Vol. 54, Issue 1-2 https://doi.org/10.1007/s00382-019-04999-4	journal	October 2019
Weighted multi-model ensemble projection of extreme precipitation in the Mediterranean region using statistical downscaling Keupp, Luzia; Hertig, Elke; Kaspar-Ott, Irena Theoretical and Applied Climatology, Vol. 138, Issue 3-4 https://doi.org/10.1007/s00704-019-02851-7	journal	May 2019
Sensitivity of future climate change and uncertainty over India to performance-based model weighting Singh, Ram; AchutaRao, K. Climatic Change, Vol. 160, Issue 3 https://doi.org/10.1007/s10584-019-02643-y	journal	January 2020
Ranking CMIP5 GCMs for Model Ensemble Selection on Regional Scale: Case Study of the Indochina Region Chhin, Rattana; Yoden, Shigeo Journal of Geophysical Research: Atmospheres, Vol. 123, Issue 17 https://doi.org/10.1029/2017jd028026	journal	September 2018
Evaluation of Satellite and Reanalysis Precipitable Water Vapor Data Sets Against Radiosonde Observations in Central Asia Jiang, Jie; Zhou, Tianjun; Zhang, Wenxia Earth and Space Science, Vol. 6, Issue 7 https://doi.org/10.1029/2019ea000654	journal	July 2019
Spatiotemporal Reliability Ensemble Averaging of Multimodel Simulations Tegegne, Getachew; Kim, Young‐Oh; Lee, Jae‐Kyoung Geophysical Research Letters, Vol. 46, Issue 21 https://doi.org/10.1029/2019gl083053	journal	November 2019
Beyond equilibrium climate sensitivity Knutti, Reto; Rugenstein, Maria A. A.; Hegerl, Gabriele C. Nature Geoscience, Vol. 10, Issue 10 https://doi.org/10.1038/ngeo3017	journal	September 2017
Learning algorithms allow for improved reliability and accuracy of global mean surface temperature projections Strobach, Ehud; Bel, Golan Nature Communications, Vol. 11, Issue 1 https://doi.org/10.1038/s41467-020-14342-9	journal	January 2020
Taking climate model evaluation to the next level Eyring, Veronika; Cox, Peter M.; Flato, Gregory M. Nature Climate Change, Vol. 9, Issue 2 https://doi.org/10.1038/s41558-018-0355-y	journal	January 2019
Precipitation regime change in Western North America: The role of Atmospheric Rivers Gershunov, Alexander; Shulgina, Tamara; Clemesha, Rachel E. S. Scientific Reports, Vol. 9, Issue 1 https://doi.org/10.1038/s41598-019-46169-w	journal	July 2019
A surrogate weighted mean ensemble method to reduce the uncertainty at a regional scale for the calculation of potential evapotranspiration Yoo, Byoung Hyun; Kim, Junhwan; Lee, Byun-Woo Scientific Reports, Vol. 10, Issue 1 https://doi.org/10.1038/s41598-020-57466-0	journal	January 2020
Ranking of CMIP5-based global climate models using standard performance metrics for Telangana region in the southern part of India Sreelatha, K.; Anand Raj, P. ISH Journal of Hydraulic Engineering https://doi.org/10.1080/09715010.2019.1634648	journal	June 2019
Quantifying uncertainty in European climate projections using combined performance-independence weighting Brunner, Lukas; Lorenz, Ruth; Zumwald, Marius Environmental Research Letters, Vol. 14, Issue 12 https://doi.org/10.1088/1748-9326/ab492f	journal	November 2019
A Proposal to Evaluate Drought Characteristics Using Multiple Climate Models for Multiple Timescales Tien Thanh, Nguyen Climate, Vol. 6, Issue 4 https://doi.org/10.3390/cli6040079	journal	September 2018
ESD Reviews: Model dependence in multi-model climate ensembles: weighting, sub-selection and out-of-sample testing Abramowitz, Gab; Herger, Nadja; Gutmann, Ethan Earth System Dynamics, Vol. 10, Issue 1 https://doi.org/10.5194/esd-10-91-2019	journal	January 2019
Selecting a climate model subset to optimise key ensemble properties Herger, Nadja; Abramowitz, Gab; Knutti, Reto Earth System Dynamics, Vol. 9, Issue 1 https://doi.org/10.5194/esd-9-135-2018	journal	January 2018
Similarities within a multi-model ensemble: functional data analysis framework Holtanová, Eva; Mendlik, Thomas; Koláček, Jan Geoscientific Model Development, Vol. 12, Issue 2 https://doi.org/10.5194/gmd-12-735-2019	journal	January 2019
Does the weighting of climate simulations result in a better quantification of hydrological impacts? Wang, Hui-Min; Chen, Jie; Xu, Chong-Yu Hydrology and Earth System Sciences, Vol. 23, Issue 10 https://doi.org/10.5194/hess-23-4033-2019	journal	January 2019
A Representative Democracy to Reduce Interdependency in a Multimodel Ensemble Sanderson, Benjamin M.; Knutti, Reto; Caldwell, Peter Journal of Climate, Vol. 28, Issue 13 https://doi.org/10.1175/JCLI-D-14-00362.1	journal	July 2015
Prospects and Caveats of Weighting Climate Models for Summer Maximum Temperature Projections Over North America Lorenz, Ruth; Herger, Nadja; Sedláček, Jan ETH Zurich https://doi.org/10.3929/ethz-b-000268868	text	January 2018
Taking climate model evaluation to the next level Veronika, Eyring,; M., Cox, Peter; M., Flato, Gregory ETH Zurich https://doi.org/10.3929/ethz-b-000323491	text	January 2019
ESD Reviews: Model dependence in multi-model climate ensembles: weighting, sub-selection and out-of-sample testing Abramowitz, Gab; Herger, Nadja; Gutmann, Ethan ETH Zurich https://doi.org/10.3929/ethz-b-000325234	text	January 2019
Quantifying uncertainty in European climate projections using combined performance-independence weighting Brunner, Lukas; Lorenz, Ruth; Zumwald, Marius ETH Zurich https://doi.org/10.3929/ethz-b-000384838	text	January 2019
Beyond equilibrium climate sensitivity Knutti, Reto; Rugenstein, Maria A. A.; Hegerl, Gabriele C. ETH Zurich https://doi.org/10.3929/ethz-b-000197761	text	January 2017
Selecting a climate model subset to optimise key ensemble properties Nadja, Herger,; Gab, Abramowitz,; Reto, Knutti, ETH Zurich https://doi.org/10.3929/ethz-b-000246202	text	January 2018
Ensembling geophysical models with Bayesian Neural Networks Sengupta, Ushnish; Amos, Matt; Hosking, J. Scott arXiv https://doi.org/10.48550/arxiv.2010.03561	text	January 2020

Similar Records

A Public-Private-Acadmic Partnership to Advance Solar Power Forecasting

Technical Report · Tue Apr 19 00:00:00 EDT 2016 · OSTI ID:1408439

Haupt, Sue Ellen

Relating CMIP5 Model Biases to Seasonal Forecast Skill in the Tropical Pacific

Journal Article · Tue Mar 10 00:00:00 EDT 2020 · Geophysical Research Letters · OSTI ID:1408439

Ding, Hui; Newman, Matthew; Alexander, Michael A.; +1 more

Addressing Interdependency in a Multimodel Ensemble by Interpolation of Model Properties

Journal Article · Wed Jul 01 00:00:00 EDT 2015 · Journal of Climate · OSTI ID:1408439

Sanderson, Benjamin M.; Knutti, Reto; Caldwell, Peter M.

Related Subjects

58 GEOSCIENCES

Title: Skill and independence weighting for multi-model assessments

Citation Formats

References (24)

Cited By (31)

Similar Records

Related Subjects