DOE PAGES title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Towards improved and more routine Earth system model evaluation in CMIP

Abstract

The Coupled Model Intercomparison Project (CMIP) has successfully provided the climate community with a rich collection of simulation output from Earth system models (ESMs) that can be used to understand past climate changes and make projections and uncertainty estimates of the future. Confidence in ESMs can be gained because the models are based on physical principles and reproduce many important aspects of observed climate. More research is required to identify the processes that are most responsible for systematic biases and the magnitude and uncertainty of future projections so that more relevant performance tests can be developed. At the same time, there are many aspects of ESM evaluation that are well established and considered an essential part of systematic evaluation but have been implemented ad hoc with little community coordination. Given the diversity and complexity of ESM analysis, we argue that the CMIP community has reached a critical juncture at which many baseline aspects of model evaluation need to be performed much more efficiently and consistently. We provide a perspective and viewpoint on how a more systematic, open, and rapid performance assessment of the large and diverse number of models that will participate in current and future phases of CMIP canmore » be achieved, and announce our intention to implement such a system for CMIP6. Accomplishing this could also free up valuable resources as many scientists are frequently "re-inventing the wheel" by re-writing analysis routines for well-established analysis methods. A more systematic approach for the community would be to develop and apply evaluation tools that are based on the latest scientific knowledge and observational reference, are well suited for routine use, and provide a wide range of diagnostics and performance metrics that comprehensively characterize model behaviour as soon as the output is published to the Earth System Grid Federation (ESGF). The CMIP infrastructure enforces data standards and conventions for model output and documentation accessible via the ESGF, additionally publishing observations (obs4MIPs) and reanalyses (ana4MIPs) for model intercomparison projects using the same data structure and organization as the ESM output. This largely facilitates routine evaluation of the ESMs, but to be able to process the data automatically alongside the ESGF, the infrastructure needs to be extended with processing capabilities at the ESGF data nodes where the evaluation tools can be executed on a routine basis. Efforts are already underway to develop community-based evaluation tools, and we encourage experts to provide additional diagnostic codes that would enhance this capability for CMIP. And, at the same time, we encourage the community to contribute observations and reanalyses for model evaluation to the obs4MIPs and ana4MIPs archives. The intention is to produce through the ESGF a widely accepted quasi-operational evaluation framework for CMIP6 that would routinely execute a series of standardized evaluation tasks. Over time, as this capability matures, we expect to produce an increasingly systematic characterization of models which, compared with early phases of CMIP, will more quickly and openly identify the strengths and weaknesses of the simulations. This will also reveal whether long-standing model errors remain evident in newer models and will assist modelling groups in improving their models. Finally, this framework will be designed to readily incorporate updates, including new observations and additional diagnostics and metrics as they become available from the research community.« less

Authors:
 [1];  [2];  [3];  [4];  [2]; ORCiD logo [5];  [6];  [7];  [8]; ORCiD logo [9];  [10]; ORCiD logo [1];  [2]
  1. German Aerospace Center (DLR), Oberpfaffenhofen (Germany). Inst. for Atmospheric Physics
  2. Lawrence Livermore National Lab. (LLNL), Livermore, CA (United States). Program for Climate Model Diagnosis and Intercomparison (PCMDI)
  3. Univ. of Bergen (Norway). Geophysical Inst., Bjerknes Centre for Climate Research; Uni Research Climate, Bergen (Norway)
  4. National Oceanic and Atmospheric Administration (NOAA), Princeton, NJ (United States). Geophysical Fluid Dynamics Lab.
  5. National Oceanic and Atmospheric Administration (NOAA), Princeton, NJ (United States). Geophysical Fluid Dynamics Lab.; Princeton Univ., NJ (United States). Cooperative Inst. for Climate Science
  6. French National Center for Scientific Research (CNRS), Paris (France). Pierre Simon Laplace Inst.; Univ. of Reading (United Kingdom). National Centre for Atmospheric Science
  7. French National Center for Scientific Research (CNRS), Saclay (France). Pierre Simon Laplace Inst.
  8. German Climate Computing Center, Hamburg (Germany)
  9. Univ. of Reading (United Kingdom). National Centre for Atmospheric Science; STFC Rutherford Appleton Lab., Didcot (United Kingdom). Centre for Environmental Data Analysis
  10. National Center for Atmospheric Research, Boulder, CO (United States)
Publication Date:
Research Org.:
Lawrence Livermore National Laboratory (LLNL), Livermore, CA (United States)
Sponsoring Org.:
USDOE Office of Science (SC), Biological and Environmental Research (BER)
OSTI Identifier:
1395317
Grant/Contract Number:  
AC52-07NA27344; FC02-97ER62402; NA08OAR4320752
Resource Type:
Accepted Manuscript
Journal Name:
Earth System Dynamics (Online)
Additional Journal Information:
Journal Name: Earth System Dynamics (Online); Journal Volume: 7; Journal Issue: 4; Journal ID: ISSN 2190-4987
Publisher:
European Geosciences Union
Country of Publication:
United States
Language:
English
Subject:
54 ENVIRONMENTAL SCIENCES

Citation Formats

Eyring, Veronika, Gleckler, Peter J., Heinze, Christoph, Stouffer, Ronald J., Taylor, Karl E., Balaji, V., Guilyardi, Eric, Joussaume, Sylvie, Kindermann, Stephan, Lawrence, Bryan N., Meehl, Gerald A., Righi, Mattia, and Williams, Dean N. Towards improved and more routine Earth system model evaluation in CMIP. United States: N. p., 2016. Web. doi:10.5194/esd-7-813-2016.
Eyring, Veronika, Gleckler, Peter J., Heinze, Christoph, Stouffer, Ronald J., Taylor, Karl E., Balaji, V., Guilyardi, Eric, Joussaume, Sylvie, Kindermann, Stephan, Lawrence, Bryan N., Meehl, Gerald A., Righi, Mattia, & Williams, Dean N. Towards improved and more routine Earth system model evaluation in CMIP. United States. https://doi.org/10.5194/esd-7-813-2016
Eyring, Veronika, Gleckler, Peter J., Heinze, Christoph, Stouffer, Ronald J., Taylor, Karl E., Balaji, V., Guilyardi, Eric, Joussaume, Sylvie, Kindermann, Stephan, Lawrence, Bryan N., Meehl, Gerald A., Righi, Mattia, and Williams, Dean N. Tue . "Towards improved and more routine Earth system model evaluation in CMIP". United States. https://doi.org/10.5194/esd-7-813-2016. https://www.osti.gov/servlets/purl/1395317.
@article{osti_1395317,
title = {Towards improved and more routine Earth system model evaluation in CMIP},
author = {Eyring, Veronika and Gleckler, Peter J. and Heinze, Christoph and Stouffer, Ronald J. and Taylor, Karl E. and Balaji, V. and Guilyardi, Eric and Joussaume, Sylvie and Kindermann, Stephan and Lawrence, Bryan N. and Meehl, Gerald A. and Righi, Mattia and Williams, Dean N.},
abstractNote = {The Coupled Model Intercomparison Project (CMIP) has successfully provided the climate community with a rich collection of simulation output from Earth system models (ESMs) that can be used to understand past climate changes and make projections and uncertainty estimates of the future. Confidence in ESMs can be gained because the models are based on physical principles and reproduce many important aspects of observed climate. More research is required to identify the processes that are most responsible for systematic biases and the magnitude and uncertainty of future projections so that more relevant performance tests can be developed. At the same time, there are many aspects of ESM evaluation that are well established and considered an essential part of systematic evaluation but have been implemented ad hoc with little community coordination. Given the diversity and complexity of ESM analysis, we argue that the CMIP community has reached a critical juncture at which many baseline aspects of model evaluation need to be performed much more efficiently and consistently. We provide a perspective and viewpoint on how a more systematic, open, and rapid performance assessment of the large and diverse number of models that will participate in current and future phases of CMIP can be achieved, and announce our intention to implement such a system for CMIP6. Accomplishing this could also free up valuable resources as many scientists are frequently "re-inventing the wheel" by re-writing analysis routines for well-established analysis methods. A more systematic approach for the community would be to develop and apply evaluation tools that are based on the latest scientific knowledge and observational reference, are well suited for routine use, and provide a wide range of diagnostics and performance metrics that comprehensively characterize model behaviour as soon as the output is published to the Earth System Grid Federation (ESGF). The CMIP infrastructure enforces data standards and conventions for model output and documentation accessible via the ESGF, additionally publishing observations (obs4MIPs) and reanalyses (ana4MIPs) for model intercomparison projects using the same data structure and organization as the ESM output. This largely facilitates routine evaluation of the ESMs, but to be able to process the data automatically alongside the ESGF, the infrastructure needs to be extended with processing capabilities at the ESGF data nodes where the evaluation tools can be executed on a routine basis. Efforts are already underway to develop community-based evaluation tools, and we encourage experts to provide additional diagnostic codes that would enhance this capability for CMIP. And, at the same time, we encourage the community to contribute observations and reanalyses for model evaluation to the obs4MIPs and ana4MIPs archives. The intention is to produce through the ESGF a widely accepted quasi-operational evaluation framework for CMIP6 that would routinely execute a series of standardized evaluation tasks. Over time, as this capability matures, we expect to produce an increasingly systematic characterization of models which, compared with early phases of CMIP, will more quickly and openly identify the strengths and weaknesses of the simulations. This will also reveal whether long-standing model errors remain evident in newer models and will assist modelling groups in improving their models. Finally, this framework will be designed to readily incorporate updates, including new observations and additional diagnostics and metrics as they become available from the research community.},
doi = {10.5194/esd-7-813-2016},
journal = {Earth System Dynamics (Online)},
number = 4,
volume = 7,
place = {United States},
year = {Tue Nov 01 00:00:00 EDT 2016},
month = {Tue Nov 01 00:00:00 EDT 2016}
}

Journal Article:
Free Publicly Available Full Text
Publisher's Version of Record

Citation Metrics:
Cited by: 60 works
Citation information provided by
Web of Science

Save / Share:

Works referenced in this record:

TECA: A Parallel Toolkit for Extreme Climate Analysis
journal, January 2012


Will a perfect model agree with perfect observations? The impact of spatial sampling
journal, January 2016

  • Schutgens, Nick A. J.; Gryspeerdt, Edward; Weigum, Natalie
  • Atmospheric Chemistry and Physics, Vol. 16, Issue 10
  • DOI: 10.5194/acp-16-6335-2016

Improving Antarctic Total Ozone Projections by a Process-Oriented Multiple Diagnostic Ensemble Regression
journal, December 2013

  • Karpechko, Alexey Yu.; Maraun, Douglas; Eyring, Veronika
  • Journal of the Atmospheric Sciences, Vol. 70, Issue 12
  • DOI: 10.1175/JAS-D-13-071.1

A quantitative performance assessment of cloud regimes in climate models
journal, August 2008


Overview of the Coupled Model Intercomparison Project Phase 6 (CMIP6) experimental design and organization
journal, January 2016

  • Eyring, Veronika; Bony, Sandrine; Meehl, Gerald A.
  • Geoscientific Model Development, Vol. 9, Issue 5
  • DOI: 10.5194/gmd-9-1937-2016

The impact of orbital sampling, monthly averaging and vertical resolution on climate chemistry model evaluation with satellite observations
journal, January 2011

  • Aghedo, A. M.; Bowman, K. W.; Shindell, D. T.
  • Atmospheric Chemistry and Physics, Vol. 11, Issue 13
  • DOI: 10.5194/acp-11-6493-2011

Emergent constraints on climate-carbon cycle feedbacks in the CMIP5 Earth system models
journal, May 2014

  • Wenzel, Sabrina; Cox, Peter M.; Eyring, Veronika
  • Journal of Geophysical Research: Biogeosciences, Vol. 119, Issue 5
  • DOI: 10.1002/2013JG002591

Describing Earth system simulations with the Metafor CIM
journal, January 2012

  • Lawrence, B. N.; Balaji, V.; Bentley, P.
  • Geoscientific Model Development, Vol. 5, Issue 6
  • DOI: 10.5194/gmd-5-1493-2012

Consistency of modelled and observed temperature trends in the tropical troposphere
journal, November 2008

  • Santer, B. D.; Thorne, P. W.; Haimberger, L.
  • International Journal of Climatology, Vol. 28, Issue 13
  • DOI: 10.1002/joc.1756

An Overview of CMIP5 and the Experiment Design
journal, April 2012

  • Taylor, Karl E.; Stouffer, Ronald J.; Meehl, Gerald A.
  • Bulletin of the American Meteorological Society, Vol. 93, Issue 4
  • DOI: 10.1175/BAMS-D-11-00094.1

Quantitative performance metrics for stratospheric-resolving chemistry-climate models
journal, January 2008


ESMValTool (v1.0) – a community diagnostic and performance metrics tool for routine evaluation of Earth system models in CMIP
journal, January 2016

  • Eyring, Veronika; Righi, Mattia; Lauer, Axel
  • Geoscientific Model Development, Vol. 9, Issue 5
  • DOI: 10.5194/gmd-9-1747-2016

The Asian summer monsoon: an intercomparison of CMIP5 vs. CMIP3 simulations of the late 20th century
journal, December 2012


CMIP5 Scientific Gaps and Recommendations for CMIP6
journal, January 2017

  • Stouffer, R. J.; Eyring, V.; Meehl, G. A.
  • Bulletin of the American Meteorological Society, Vol. 98, Issue 1
  • DOI: 10.1175/BAMS-D-15-00013.1

Clouds, circulation and climate sensitivity
journal, March 2015

  • Bony, Sandrine; Stevens, Bjorn; Frierson, Dargan M. W.
  • Nature Geoscience, Vol. 8, Issue 4
  • DOI: 10.1038/ngeo2398

The use of the multi-model ensemble in probabilistic climate projections
journal, June 2007

  • Tebaldi, Claudia; Knutti, Reto
  • Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, Vol. 365, Issue 1857
  • DOI: 10.1098/rsta.2007.2076

Statistical significance of climate sensitivity predictors obtained by data mining
journal, March 2014

  • Caldwell, Peter M.; Bretherton, Christopher S.; Zelinka, Mark D.
  • Geophysical Research Letters, Vol. 41, Issue 5
  • DOI: 10.1002/2014GL059205

Emergent Constraints for Cloud Feedbacks
journal, October 2015


ENSO representation in climate models: from CMIP3 to CMIP5
journal, April 2013


Projected land photosynthesis constrained by changes in the seasonal cycle of atmospheric CO2
journal, September 2016

  • Wenzel, Sabrina; Cox, Peter M.; Eyring, Veronika
  • Nature, Vol. 538, Issue 7626
  • DOI: 10.1038/nature19772

Sensitivity of tropical carbon to climate change constrained by carbon dioxide variability
journal, February 2013

  • Cox, Peter M.; Pearson, David; Booth, Ben B.
  • Nature, Vol. 494, Issue 7437
  • DOI: 10.1038/nature11882

Overview of the Coupled Model Intercomparison Project Phase 6 (CMIP6) experimental design and organisation
journal, January 2015

  • Eyring, V.; Bony, S.; Meehl, G. A.
  • Geoscientific Model Development Discussions, Vol. 8, Issue 12
  • DOI: 10.5194/gmdd-8-10539-2015

Evaluating Modes of Variability in Climate Models
journal, December 2014

  • Phillips, Adam S.; Deser, Clara; Fasullo, John
  • Eos, Transactions American Geophysical Union, Vol. 95, Issue 49
  • DOI: 10.1002/2014EO490002

ESMValTool (v1.0) – a community diagnostic and performance metrics tool for routine evaluation of Earth system models in CMIP
text, January 2016

  • Eyring, Veronika; Righi, Mattia; Lauer, Axel
  • Copernicus Publications
  • DOI: 10.48350/167138

Using the current seasonal cycle to constrain snow albedo feedback in future climate change
journal, January 2006


Development cycle 2 of the Modular Earth Submodel System (MESSy2)
journal, January 2010

  • Jöckel, P.; Kerkweg, A.; Pozzer, A.
  • Geoscientific Model Development Discussions, Vol. 3, Issue 3
  • DOI: 10.5194/gmdd-3-1423-2010

Quantitative evaluation of ozone and selected climate parameters in a set of EMAC simulations
journal, January 2015

  • Righi, M.; Eyring, V.; Gottschaldt, K. -D.
  • Geoscientific Model Development, Vol. 8, Issue 3
  • DOI: 10.5194/gmd-8-733-2015

Metrics for the Diurnal Cycle of Precipitation: Toward Routine Benchmarks for Climate Models
journal, June 2016

  • Covey, Curt; Gleckler, Peter J.; Doutriaux, Charles
  • Journal of Climate, Vol. 29, Issue 12
  • DOI: 10.1175/JCLI-D-15-0664.1

Describing Earth System Simulations with the Metafor CIM
journal, January 2012

  • Lawrence, B. N.; Balaji, V.; Bentley, P.
  • Geoscientific Model Development Discussions, Vol. 5, Issue 2
  • DOI: 10.5194/gmdd-5-1669-2012

How Well Do We Understand and Evaluate Climate Change Feedback Processes?
journal, August 2006

  • Bony, Sandrine; Colman, Robert; Kattsov, Vladimir M.
  • Journal of Climate, Vol. 19, Issue 15
  • DOI: 10.1175/JCLI3819.1

COSP: Satellite simulation software for model assessment
journal, August 2011

  • Bodas-Salcedo, A.; Webb, M. J.; Bony, S.
  • Bulletin of the American Meteorological Society, Vol. 92, Issue 8
  • DOI: 10.1175/2011BAMS2856.1

Energy sector vulnerability to climate change: A review
journal, February 2012


Development cycle 2 of the Modular Earth Submodel System (MESSy2)
journal, January 2010

  • Jöckel, P.; Kerkweg, A.; Pozzer, A.
  • Geoscientific Model Development, Vol. 3, Issue 2
  • DOI: 10.5194/gmd-3-717-2010

TECA: A Parallel Toolkit for Extreme Climate Analysis
journal, January 2012


Evaluation of climate models using palaeoclimatic data
journal, March 2012

  • Braconnot, Pascale; Harrison, Sandy P.; Kageyama, Masa
  • Nature Climate Change, Vol. 2, Issue 6
  • DOI: 10.1038/nclimate1456

Addressing Interdependency in a Multimodel Ensemble by Interpolation of Model Properties
journal, July 2015


Visualization and Analysis Tools for Ultrascale Climate Data
journal, October 2014

  • Williams, Dean N.
  • Eos, Transactions American Geophysical Union, Vol. 95, Issue 42
  • DOI: 10.1002/2014EO420002

A comparison of PMIP2 model simulations and the MARGO proxy reconstruction for tropical sea surface temperatures at last glacial maximum
journal, January 2009


Documenting Climate Models and Their Simulations
journal, May 2013

  • Guilyardi, Eric; Balaji, V.; Lawrence, Bryan
  • Bulletin of the American Meteorological Society, Vol. 94, Issue 5
  • DOI: 10.1175/BAMS-D-11-00035.1

Evolving Obs4MIPs to Support Phase 6 of the Coupled Model Intercomparison Project (CMIP6)
journal, August 2015

  • Ferraro, Robert; Waliser, Duane E.; Gleckler, Peter
  • Bulletin of the American Meteorological Society, Vol. 96, Issue 8
  • DOI: 10.1175/BAMS-D-14-00216.1

The use of the multi-model ensemble in probabilistic climate projections
journal, June 2007

  • Tebaldi, Claudia; Knutti, Reto
  • Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, Vol. 365, Issue 1857
  • DOI: 10.1098/rsta.2007.2076

Development cycle 2 of the Modular Earth Submodel System (MESSy2)
journal, January 2010

  • Jöckel, P.; Kerkweg, A.; Pozzer, A.
  • Geoscientific Model Development Discussions, Vol. 3, Issue 3
  • DOI: 10.5194/gmdd-3-1423-2010

How Often Will It Rain?
journal, October 2007

  • Sun, Ying; Solomon, Susan; Dai, Aiguo
  • Journal of Climate, Vol. 20, Issue 19
  • DOI: 10.1175/JCLI4263.1

Europe-wide reduction in primary productivity caused by the heat and drought in 2003
journal, September 2005


Performance metrics for climate models
journal, January 2008

  • Gleckler, P. J.; Taylor, K. E.; Doutriaux, C.
  • Journal of Geophysical Research, Vol. 113, Issue D6
  • DOI: 10.1029/2007jd008972

A More Powerful Reality Test for Climate Models
journal, May 2016


ESMValTool (v1.0) – a community diagnostic and performance metrics tool for routine evaluation of Earth system models in CMIP
journal, January 2016

  • Eyring, Veronika; Righi, Mattia; Lauer, Axel
  • Geoscientific Model Development, Vol. 9, Issue 5
  • DOI: 10.5194/gmd-9-1747-2016

Observational challenges in evaluating climate models
journal, October 2013

  • Collins, Mat; AchutaRao, Krishna; Ashok, Karumuri
  • Nature Climate Change, Vol. 3, Issue 11
  • DOI: 10.1038/nclimate2012

A framework for benchmarking land models
journal, January 2012


Constraining projections of summer Arctic sea ice
journal, January 2012


A Global Repository for Planet-Sized Experiments and Observations
journal, May 2016

  • Williams, Dean N.; Balaji, V.; Cinquini, Luca
  • Bulletin of the American Meteorological Society, Vol. 97, Issue 5
  • DOI: 10.1175/BAMS-D-15-00132.1

ESMValTool (v1.0) – a community diagnostic and performance metrics tool for routine evaluation of Earth System Models in CMIP
journal, January 2015

  • Eyring, V.; Righi, M.; Evaldsson, M.
  • Geoscientific Model Development Discussions, Vol. 8, Issue 9
  • DOI: 10.5194/gmdd-8-7541-2015

The impact of orbital sampling, monthly averaging and vertical resolution on climate chemistry model evaluation with satellite observations
journal, January 2011

  • Aghedo, A. M.; Bowman, K. W.; Shindell, D. T.
  • Atmospheric Chemistry and Physics, Vol. 11, Issue 13
  • DOI: 10.5194/acp-11-6493-2011

Consistency of modelled and observed temperature trends in the tropical troposphere
journal, November 2008

  • Santer, B. D.; Thorne, P. W.; Haimberger, L.
  • International Journal of Climatology, Vol. 28, Issue 13
  • DOI: 10.1002/joc.1756

Recent Progress in Constraining Climate Sensitivity With Model Ensembles
journal, August 2015

  • Fasullo, J. T.; Sanderson, B. M.; Trenberth, K. E.
  • Current Climate Change Reports, Vol. 1, Issue 4
  • DOI: 10.1007/s40641-015-0021-7

A Less Cloudy Future: The Role of Subtropical Subsidence in Climate Sensitivity
journal, November 2012


The impact of orbital sampling, monthly averaging and vertical resolution on climate chemistry model evaluation with satellite observations
journal, January 2011

  • Aghedo, A. M.; Bowman, K. W.; Shindell, D. T.
  • Atmospheric Chemistry and Physics Discussions, Vol. 11, Issue 3
  • DOI: 10.5194/acpd-11-9705-2011

A Strategy for Process-Oriented Validation of Coupled Chemistry–Climate Models
journal, August 2005

  • Eyring, V.; Harris, N. R. P.; Rex, M.
  • Bulletin of the American Meteorological Society, Vol. 86, Issue 8
  • DOI: 10.1175/BAMS-86-8-1117

A community diagnostic tool for chemistry climate model validation
journal, January 2012

  • Gettelman, A.; Eyring, V.; Fischer, C.
  • Geoscientific Model Development, Vol. 5, Issue 5
  • DOI: 10.5194/gmd-5-1061-2012

Describing Earth System Simulations with the Metafor CIM
journal, January 2012

  • Lawrence, B. N.; Balaji, V.; Bentley, P.
  • Geoscientific Model Development Discussions, Vol. 5, Issue 2
  • DOI: 10.5194/gmdd-5-1669-2012

Satellite Observations for CMIP5: The Genesis of Obs4MIPs
journal, September 2014

  • Teixeira, Joao; Waliser, Duane; Ferraro, Robert
  • Bulletin of the American Meteorological Society, Vol. 95, Issue 9
  • DOI: 10.1175/BAMS-D-12-00204.1

ENSO representation in climate models: from CMIP3 to CMIP5
journal, April 2013


Tuning the climate of a global model: TUNING THE CLIMATE OF A GLOBAL MODEL
journal, March 2012

  • Mauritsen, Thorsten; Stevens, Bjorn; Roeckner, Erich
  • Journal of Advances in Modeling Earth Systems, Vol. 4, Issue 3
  • DOI: 10.1029/2012MS000154

Spread in model climate sensitivity traced to atmospheric convective mixing
journal, January 2014

  • Sherwood, Steven C.; Bony, Sandrine; Dufresne, Jean-Louis
  • Nature, Vol. 505, Issue 7481
  • DOI: 10.1038/nature12829

Evaluating the Land and Ocean Components of the Global Carbon Cycle in the CMIP5 Earth System Models
journal, September 2013


Moving beyond the Total Sea Ice Extent in Gauging Model Biases
journal, December 2016

  • Ivanova, Detelina P.; Gleckler, Peter J.; Taylor, Karl E.
  • Journal of Climate, Vol. 29, Issue 24
  • DOI: 10.1175/JCLI-D-16-0026.1

Clouds, circulation and climate sensitivity
journal, March 2015

  • Bony, Sandrine; Stevens, Bjorn; Frierson, Dargan M. W.
  • Nature Geoscience, Vol. 8, Issue 4
  • DOI: 10.1038/ngeo2398

September Arctic sea ice predicted to disappear near 2°C global warming above present: ARCTIC SEA ICE TO DISAPPEAR NEAR 2°C
journal, March 2012

  • Mahlstein, Irina; Knutti, Reto
  • Journal of Geophysical Research: Atmospheres, Vol. 117, Issue D6
  • DOI: 10.1029/2011JD016709

On Constraining Estimates of Climate Sensitivity with Present-Day Observations through Model Weighting
journal, December 2011

  • Klocke, Daniel; Pincus, Robert; Quaas, Johannes
  • Journal of Climate, Vol. 24, Issue 23
  • DOI: 10.1175/2011JCLI4193.1

The Art and Science of Climate Model Tuning
journal, March 2017

  • Hourdin, Frédéric; Mauritsen, Thorsten; Gettelman, Andrew
  • Bulletin of the American Meteorological Society, Vol. 98, Issue 3
  • DOI: 10.1175/BAMS-D-15-00135.1

Understanding El Niño in Ocean–Atmosphere General Circulation Models: Progress and Challenges
journal, March 2009

  • Guilyardi, Eric; Wittenberg, Andrew; Fedorov, Alexey
  • Bulletin of the American Meteorological Society, Vol. 90, Issue 3
  • DOI: 10.1175/2008BAMS2387.1

Evaluation of climate models using palaeoclimatic data
journal, March 2012

  • Braconnot, Pascale; Harrison, Sandy P.; Kageyama, Masa
  • Nature Climate Change, Vol. 2, Issue 6
  • DOI: 10.1038/nclimate1456

Constraining Future Summer Austral Jet Stream Positions in the CMIP5 Ensemble by Process-Oriented Multiple Diagnostic Regression
journal, January 2016

  • Wenzel, Sabrina; Eyring, Veronika; Gerber, Edwin P.
  • Journal of Climate, Vol. 29, Issue 2
  • DOI: 10.1175/JCLI-D-15-0412.1

Long-term Climate Change: Projections, Commitments and Irreversibility Pages 1029 to 1076
book, June 2014


Works referencing / citing this record:

Requirements for a global data infrastructure in support of CMIP6
journal, January 2018

  • Balaji, Venkatramani; Taylor, Karl E.; Juckes, Martin
  • Geoscientific Model Development, Vol. 11, Issue 9
  • DOI: 10.5194/gmd-11-3659-2018

AerChemMIP: quantifying the effects of chemistry and aerosols in CMIP6
journal, January 2017

  • Collins, William J.; Lamarque, Jean-François; Schulz, Michael
  • Geoscientific Model Development, Vol. 10, Issue 2
  • DOI: 10.5194/gmd-10-585-2017

Requirements for a global data infrastructure in support of CMIP6
journal, January 2018

  • Balaji, Venkatramani; Taylor, Karl E.; Juckes, Martin
  • Geoscientific Model Development, Vol. 11, Issue 9
  • DOI: 10.5194/gmd-11-3659-2018

ESD Reviews: Climate feedbacks in the Earth system and prospects for their evaluation
journal, January 2019

  • Heinze, Christoph; Eyring, Veronika; Friedlingstein, Pierre
  • Earth System Dynamics, Vol. 10, Issue 3
  • DOI: 10.5194/esd-10-379-2019

Evaluation of Impact of Potential Extreme Rainfall Events on Mining in Peru
journal, July 2018

  • Gonzalez, Francisco R.; Raval, Simit; Taplin, Ros
  • Natural Resources Research, Vol. 28, Issue 2
  • DOI: 10.1007/s11053-018-9396-1

Multivariable Integrated Evaluation of Model Performance with the Vector Field Evaluation Diagram
posted_content, May 2017

  • Xu, Zhongfeng; Han, Ying; Fu, Congbin
  • Geoscientific Model Development Discussions
  • DOI: 10.5194/gmd-2017-95

ESD Reviews: Climate feedbacks in the Earth system and prospects for their evaluation
journal, January 2019

  • Heinze, Christoph; Eyring, Veronika; Friedlingstein, Pierre
  • Earth System Dynamics, Vol. 10, Issue 3
  • DOI: 10.5194/esd-10-379-2019

Quantifying the agreement between observed and simulated extratropical modes of interannual variability
journal, July 2018


Quantifying climate feedbacks in polar regions
journal, May 2018


Multivariable integrated evaluation of model performance with the vector field evaluation diagram
journal, January 2017

  • Xu, Zhongfeng; Han, Ying; Fu, Congbin
  • Geoscientific Model Development, Vol. 10, Issue 10
  • DOI: 10.5194/gmd-10-3805-2017

An interactive ocean surface albedo scheme (OSAv1.0): formulation and evaluation in ARPEGE-Climat (V6.1) and LMDZ (V5A)
journal, January 2018

  • Séférian, Roland; Baek, Sunghye; Boucher, Olivier
  • Geoscientific Model Development, Vol. 11, Issue 1
  • DOI: 10.5194/gmd-11-321-2018

Quantifying climate feedbacks in polar regions
journal, May 2018


An interactive ocean surface albedo scheme (OSAv1.0): formulation and evaluation in ARPEGE-Climat (V6.1) and LMDZ (V5A)
journal, January 2018

  • Séférian, Roland; Baek, Sunghye; Boucher, Olivier
  • Geoscientific Model Development, Vol. 11, Issue 1
  • DOI: 10.5194/gmd-11-321-2018

Data Distribution Centre Support for the IPCC Sixth Assessment
journal, January 2019

  • Stockhause, Martina; Juckes, Martin; Chen, Robert
  • Data Science Journal, Vol. 18
  • DOI: 10.5334/dsj-2019-020

Taking climate model evaluation to the next level
text, January 2019


ESD Reviews: Climate feedbacks in the Earth system and prospects for their evaluation
text, January 2019


Probabilistic evaluation of competing climate models
journal, October 2017

  • Braverman, Amy; Chatterjee, Snigdhansu; Heyman, Megan
  • Advances in Statistical Climatology, Meteorology and Oceanography, Vol. 3, Issue 2
  • DOI: 10.5194/ascmo-3-93-2017

The use of the Community Earth System Model in human dimensions climate research and applications
journal, January 2019

  • Laidlaw, Emily K.; O'Neill, Brian C.; Harp, Ryan D.
  • Wiley Interdisciplinary Reviews: Climate Change, Vol. 10, Issue 3
  • DOI: 10.1002/wcc.582

Deep learning and process understanding for data-driven Earth system science
journal, February 2019


Taking climate model evaluation to the next level
journal, January 2019


AerChemMIP: quantifying the effects of chemistry and aerosols in CMIP6
journal, January 2017

  • Collins, William J.; Lamarque, Jean-François; Schulz, Michael
  • Geoscientific Model Development, Vol. 10, Issue 2
  • DOI: 10.5194/gmd-10-585-2017

Benchmarking CMIP5 models with a subset of ESA CCI Phase 2 data using the ESMValTool
journal, December 2017