skip to main content
OSTI.GOV title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Skill and independence weighting for multi-model assessments

Abstract

We present a weighting strategy for use with the CMIP5 multi-model archive in the fourth National Climate Assessment, which considers both skill in the climatological performance of models over North America as well as the inter-dependency of models arising from common parameterizations or tuning practices. The method exploits information relating to the climatological mean state of a number of projection-relevant variables as well as metrics representing long-term statistics of weather extremes. The weights, once computed can be used to simply compute weighted means and significance information from an ensemble containing multiple initial condition members from potentially co-dependent models of varying skill. Two parameters in the algorithm determine the degree to which model climatological skill and model uniqueness are rewarded; these parameters are explored and final values are defended for the assessment. The influence of model weighting on projected temperature and precipitation changes is found to be moderate, partly due to a compensating effect between model skill and uniqueness. However, more aggressive skill weighting and weighting by targeted metrics is found to have a more significant effect on inferred ensemble confidence in future patterns of change for a given projection.

Authors:
 [1];  [2]; ORCiD logo [3]
  1. National Center for Atmospheric Research, Boulder, CO (United States)
  2. Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States)
  3. Swiss Federal Inst. of Technology (ETH), Zurich (Switzerland); National Center for Atmospheric Research, Boulder, CO (United States)
Publication Date:
Research Org.:
National Center for Atmospheric Research, Boulder, CO (United States); Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States)
Sponsoring Org.:
USDOE Office of Science (SC), Biological and Environmental Research (BER) (SC-23)
OSTI Identifier:
1408439
Grant/Contract Number:
AC02-05CH11231; FC02-97ER62402
Resource Type:
Journal Article: Accepted Manuscript
Journal Name:
Geoscientific Model Development (Online)
Additional Journal Information:
Journal Name: Geoscientific Model Development (Online); Journal Volume: 10; Journal Issue: 6; Journal ID: ISSN 1991-9603
Publisher:
European Geosciences Union
Country of Publication:
United States
Language:
English
Subject:
58 GEOSCIENCES

Citation Formats

Sanderson, Benjamin M., Wehner, Michael, and Knutti, Reto. Skill and independence weighting for multi-model assessments. United States: N. p., 2017. Web. doi:10.5194/gmd-10-2379-2017.
Sanderson, Benjamin M., Wehner, Michael, & Knutti, Reto. Skill and independence weighting for multi-model assessments. United States. doi:10.5194/gmd-10-2379-2017.
Sanderson, Benjamin M., Wehner, Michael, and Knutti, Reto. Wed . "Skill and independence weighting for multi-model assessments". United States. doi:10.5194/gmd-10-2379-2017. https://www.osti.gov/servlets/purl/1408439.
@article{osti_1408439,
title = {Skill and independence weighting for multi-model assessments},
author = {Sanderson, Benjamin M. and Wehner, Michael and Knutti, Reto},
abstractNote = {We present a weighting strategy for use with the CMIP5 multi-model archive in the fourth National Climate Assessment, which considers both skill in the climatological performance of models over North America as well as the inter-dependency of models arising from common parameterizations or tuning practices. The method exploits information relating to the climatological mean state of a number of projection-relevant variables as well as metrics representing long-term statistics of weather extremes. The weights, once computed can be used to simply compute weighted means and significance information from an ensemble containing multiple initial condition members from potentially co-dependent models of varying skill. Two parameters in the algorithm determine the degree to which model climatological skill and model uniqueness are rewarded; these parameters are explored and final values are defended for the assessment. The influence of model weighting on projected temperature and precipitation changes is found to be moderate, partly due to a compensating effect between model skill and uniqueness. However, more aggressive skill weighting and weighting by targeted metrics is found to have a more significant effect on inferred ensemble confidence in future patterns of change for a given projection.},
doi = {10.5194/gmd-10-2379-2017},
journal = {Geoscientific Model Development (Online)},
number = 6,
volume = 10,
place = {United States},
year = {Wed Jun 28 00:00:00 EDT 2017},
month = {Wed Jun 28 00:00:00 EDT 2017}
}

Journal Article:
Free Publicly Available Full Text
Publisher's Version of Record

Citation Metrics:
Cited by: 3works
Citation information provided by
Web of Science

Save / Share:
  • In this paper, the authors discuss the problem of how human skill can be represented as a parametric model using a hidden Markov model (HMM), and how an HMM-based skill model can be used to learn human skill. HMM is feasible to characterize a doubly stochastic process--measurable action and immeasurable mental states--that is involved in the skill learning. The authors formulated the learning problem as a multidimensional HMM and developed a testbed for a variety of skill learning applications. Based on ''the most likely performance'' criterion, the best action sequence can be selected from all previously measured action data bymore » modeling the skill as an HMM. The proposed method has been implemented in the teleoperation control of a space station robot system, and some important implementation issues have been discussed. The method allows a robot to learn human skill certain tasks and to improve motion performance.« less
  • The feasibility of using a two-tier approach to provide guidance to operational long-lead seasonal prediction explored. The approach includes a first forecast of global sea surface temperature (SSTs) using a coupled general circulation model, followed by an atmospheric forecast using an atmospheric general circulation model (AGCM). For this exploration, ensembles of decade-long integrations of the AGCM driven by observed SSTs and ensembles of integrations of selects cases driven by forecast SSTs have been conducted. The ability of the model in these sets of runs to reproduce observed atmospheric conditions has been evaluated with a multiparameter performance analysis. Results have identifiedmore » performance and skill levels in the specified SST runs, for winters and springs over the Pacific/North America region, that are sufficient to impact operational seasonal predictions in years with major El Nino Southern Oscillation (ENSO) episodes. Further, these levels were substantially reproduced in the forecast SST runs for 1-month leads and in many instances for up to one season leads. In fact, overall the 0- and 1-month-lead forecasts of better than corresponding official forecasts. Thus, there is considerable reason to develop a dynamical component for the official seasonal forecast process. 23 refs., 10 figs., 3 tabs.« less
  • Three different mechanisms for the generation of ENSO SST anomalies within a simplified tropical Pacific Ocean model are examined: thermocline depth changes, Ekman-induced upwelling anomalies, and zonal advection changes. The effect of varying the relative influence of these terms on the realism of tropical pacific coupled models is analyzed. The principal tool used to assess such realism is hindcast skill, with forced ocean and oscillatory behavior also being examined. Of the mechanisms considered, thermocline perturbations are shown to be crucially important for high coupled-model hindcast skills. Furthermore, it is concluded that the realism of the model (as measured by hindcastmore » skill) deteriorates markedly when the influence on SST of Ekman upwelling becomes greater than a small fraction of the thermocline influence. This provides strong evidence for the hypothesis that Ekman upwelling anomalies (which are essentially a local response to wind stress anomalies) have only a small influence on the creation of real world SST anomalies. The implications of this latter point for coupled models involving ocean general circulation models is briefly discussed. It is also demonstrated that western boundary reflections provide a vital role by means of a negative feedback in ensuring realistic performance. The hindcast skill (as measured by NINO3 anomaly correlation) demonstrated by a model involving only the thermocline mechanism can be tuned to exceed that of the benchmark Cane and Zebiak model for hindcast lags up to 7 months (from 7 to 12 months the model skills are roughly equal). 47 refs., 32 figs.« less
  • The possibility of using forecast errors originating from the finite-time dominant linear modes for the prediction of forecast skill for a primitive equation regional forecast model is studied. This is similar to the method for skill prediction suggested by several other authors using simplified models. Two main problems associated with a sophisticated forecast model not considered in these other studies are investigated: (1) The number of degrees of freedom is typically too large for the evaluation of the spectrum of dominant modes associated with the linear error evolution equation, and (2) many different, physically meaningful, error measures may be usedmore » for this model and the dominant linear modes may be sensitive to the selection of the error measure. It is shown first that the finite-time dominant linear solutions can be computed with sufficient accuracy for the complex forecast model using standard [open quotes]power[close quotes] method with a small number of iterations for all error measures considered in this study. The forecast skill is then estimated using the nonlinear forecast errors originating from the initial errors that are defined by these measure-dependent dominant solutions. These results show that the estimated forecast skill is very sensitive to the choice of error measure used for the computation of the finite-time dominant modes. 17 refs., 7 figs.« less