skip to main content
OSTI.GOV title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Quantifying the agreement between observed and simulated extratropical modes of interannual variability

Abstract

Using historical simulations of the Coupled Model Intercomparison Project-5 (CMIP5) and multiple observationally-based datasets, we employ skill metrics to analyze the fidelity of the simulated Northern Annular Mode, the North Atlantic Oscillation, the Pacific North America pattern, the Southern Annular Mode, the Pacific Decadal Oscillation, the North Pacific Oscillation, and the North Pacific Gyre Oscillation. We assess the benefits of a unified approach to evaluate these modes of variability, which we call the common basis function (CBF) approach, based on projecting model anomalies onto observed empirical orthogonal functions (EOFs). The CBF approach circumvents issues with conventional EOF analysis, eliminating, for example, corrections of arbitrarily assigned, but inconsistent, signs of the EOF’s/PC’s being compared. It also avoids the problem that sometimes the first observed EOF is more similar to a higher order model EOF, particularly if the simulated EOFs are not well separated. Compared to conventional EOF analysis of models, the CBF approach indicates that models compare significantly better with observations in terms of pattern correlation and root-mean-squared-error (RMSE) than heretofore suggested. In many cases, models are doing a credible job at capturing the observationally-based estimates of patterns; however, errors in simulated amplitudes can be large and more egregious than patternmore » errors. In the context of the broad distribution of errors in the CMIP5 ensemble, sensitivity tests demonstrate that our results are relatively insensitive to methodological considerations (CBF vs. conventional approach), observational uncertainties in pattern (as determined by using multiple datasets), and internal variability (when multiple realizations from the same model are compared). The skill metrics proposed in this study can provide a useful summary of the ability of models to reproduce the observed EOF patterns and amplitudes. Additionally, the skill metrics can be used as a tool to objectively highlight where potential model improvements might be made. We advocate more systematic and objective testing of simulated extratropical variability, especially during the non-dominant seasons of each mode, when many models are performing relatively poorly.« less

Authors:
ORCiD logo [1];  [1];  [1];  [1];  [1]
  1. Lawrence Livermore National Lab. (LLNL), Livermore, CA (United States). Program for Climate Model Diagnosis and Intercomparison (PCMDI)
Publication Date:
Research Org.:
Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States). Oak Ridge Leadership Computing Facility (OLCF); Lawrence Livermore National Lab. (LLNL), Livermore, CA (United States)
Sponsoring Org.:
USDOE Office of Science (SC)
OSTI Identifier:
1565715
DOE Contract Number:  
AC52-07NA27344
Resource Type:
Journal Article
Journal Name:
Climate Dynamics
Additional Journal Information:
Journal Volume: 52; Journal Issue: 7-8; Journal ID: ISSN 0930-7575
Publisher:
Springer-Verlag
Country of Publication:
United States
Language:
English
Subject:
Meteorology & Atmospheric Sciences

Citation Formats

Lee, Jiwoo, Sperber, Kenneth R., Gleckler, Peter J., Bonfils, Céline J. W., and Taylor, Karl E. Quantifying the agreement between observed and simulated extratropical modes of interannual variability. United States: N. p., 2018. Web. doi:10.1007/s00382-018-4355-4.
Lee, Jiwoo, Sperber, Kenneth R., Gleckler, Peter J., Bonfils, Céline J. W., & Taylor, Karl E. Quantifying the agreement between observed and simulated extratropical modes of interannual variability. United States. doi:10.1007/s00382-018-4355-4.
Lee, Jiwoo, Sperber, Kenneth R., Gleckler, Peter J., Bonfils, Céline J. W., and Taylor, Karl E. Fri . "Quantifying the agreement between observed and simulated extratropical modes of interannual variability". United States. doi:10.1007/s00382-018-4355-4.
@article{osti_1565715,
title = {Quantifying the agreement between observed and simulated extratropical modes of interannual variability},
author = {Lee, Jiwoo and Sperber, Kenneth R. and Gleckler, Peter J. and Bonfils, Céline J. W. and Taylor, Karl E.},
abstractNote = {Using historical simulations of the Coupled Model Intercomparison Project-5 (CMIP5) and multiple observationally-based datasets, we employ skill metrics to analyze the fidelity of the simulated Northern Annular Mode, the North Atlantic Oscillation, the Pacific North America pattern, the Southern Annular Mode, the Pacific Decadal Oscillation, the North Pacific Oscillation, and the North Pacific Gyre Oscillation. We assess the benefits of a unified approach to evaluate these modes of variability, which we call the common basis function (CBF) approach, based on projecting model anomalies onto observed empirical orthogonal functions (EOFs). The CBF approach circumvents issues with conventional EOF analysis, eliminating, for example, corrections of arbitrarily assigned, but inconsistent, signs of the EOF’s/PC’s being compared. It also avoids the problem that sometimes the first observed EOF is more similar to a higher order model EOF, particularly if the simulated EOFs are not well separated. Compared to conventional EOF analysis of models, the CBF approach indicates that models compare significantly better with observations in terms of pattern correlation and root-mean-squared-error (RMSE) than heretofore suggested. In many cases, models are doing a credible job at capturing the observationally-based estimates of patterns; however, errors in simulated amplitudes can be large and more egregious than pattern errors. In the context of the broad distribution of errors in the CMIP5 ensemble, sensitivity tests demonstrate that our results are relatively insensitive to methodological considerations (CBF vs. conventional approach), observational uncertainties in pattern (as determined by using multiple datasets), and internal variability (when multiple realizations from the same model are compared). The skill metrics proposed in this study can provide a useful summary of the ability of models to reproduce the observed EOF patterns and amplitudes. Additionally, the skill metrics can be used as a tool to objectively highlight where potential model improvements might be made. We advocate more systematic and objective testing of simulated extratropical variability, especially during the non-dominant seasons of each mode, when many models are performing relatively poorly.},
doi = {10.1007/s00382-018-4355-4},
journal = {Climate Dynamics},
issn = {0930-7575},
number = 7-8,
volume = 52,
place = {United States},
year = {2018},
month = {7}
}

Works referenced in this record:

A New Globally Complete Monthly Historical Gridded Mean Sea Level Pressure Dataset (HadSLP2): 1850–2004
journal, November 2006


Arctic Oscillation or North Atlantic Oscillation?
journal, August 2001


Classification, Seasonality and Persistence of Low-Frequency Atmospheric Circulation Patterns
journal, June 1987


Atmospheric conservation properties in ERA-Interim
journal, July 2011

  • Berrisford, P.; Kållberg, P.; Kobayashi, S.
  • Quarterly Journal of the Royal Meteorological Society, Vol. 137, Issue 659
  • DOI: 10.1002/qj.864

Investigating the possibility of a human component in various pacific decadal oscillation indices
journal, October 2010


Relative Contributions of Mean-State Shifts and ENSO-Driven Variability to Precipitation Changes in a Warming Climate
journal, December 2015

  • Bonfils, Céline J. W.; Santer, Benjamin D.; Phillips, Thomas J.
  • Journal of Climate, Vol. 28, Issue 24
  • DOI: 10.1175/JCLI-D-15-0341.1

On the lack of stratospheric dynamical variability in low-top versions of the CMIP5 models: STRATOSPHERE IN CMIP5 MODELS
journal, March 2013

  • Charlton-Perez, Andrew J.; Baldwin, Mark P.; Birner, Thomas
  • Journal of Geophysical Research: Atmospheres, Vol. 118, Issue 6
  • DOI: 10.1002/jgrd.50125

Feasibility of a 100-Year Reanalysis Using Only Surface Pressure Data
journal, February 2006

  • Compo, Gilbert P.; Whitaker, Jeffrey S.; Sardeshmukh, Prashant D.
  • Bulletin of the American Meteorological Society, Vol. 87, Issue 2
  • DOI: 10.1175/BAMS-87-2-175

The Twentieth Century Reanalysis Project
journal, January 2011

  • Compo, G. P.; Whitaker, J. S.; Sardeshmukh, P. D.
  • Quarterly Journal of the Royal Meteorological Society, Vol. 137, Issue 654
  • DOI: 10.1002/qj.776

The ERA-Interim reanalysis: configuration and performance of the data assimilation system
journal, April 2011

  • Dee, D. P.; Uppala, S. M.; Simmons, A. J.
  • Quarterly Journal of the Royal Meteorological Society, Vol. 137, Issue 656
  • DOI: 10.1002/qj.828

On the teleconnectivity of the “Arctic Oscillation”
journal, March 2000


Sea Surface Temperature Variability: Patterns and Mechanisms
journal, January 2010


North Pacific Gyre Oscillation links ocean climate and ecosystem change
journal, January 2008

  • Di Lorenzo, E.; Schneider, N.; Cobb, K. M.
  • Geophysical Research Letters, Vol. 35, Issue 8
  • DOI: 10.1029/2007GL032838

Towards improved and more routine Earth system model evaluation in CMIP
journal, January 2016

  • Eyring, Veronika; Gleckler, Peter J.; Heinze, Christoph
  • Earth System Dynamics, Vol. 7, Issue 4
  • DOI: 10.5194/esd-7-813-2016

ESMValTool (v1.0) – a community diagnostic and performance metrics tool for routine evaluation of Earth system models in CMIP
journal, January 2016

  • Eyring, Veronika; Righi, Mattia; Lauer, Axel
  • Geoscientific Model Development, Vol. 9, Issue 5
  • DOI: 10.5194/gmd-9-1747-2016

Storminess over the North Atlantic and northwestern Europe-A review
journal, May 2014

  • Feser, F.; Barcikowska, M.; Krueger, O.
  • Quarterly Journal of the Royal Meteorological Society, Vol. 141, Issue 687
  • DOI: 10.1002/qj.2364

The Summer North Atlantic Oscillation: Past, Present, and Future
journal, March 2009

  • Folland, Chris K.; Knight, Jeff; Linderholm, Hans W.
  • Journal of Climate, Vol. 22, Issue 5
  • DOI: 10.1175/2008JCLI2459.1

North Pacific Decadal Variability and Climate Change in the IPCC AR4 Models
journal, June 2011

  • Furtado, Jason C.; Di Lorenzo, Emanuele; Schneider, Niklas
  • Journal of Climate, Vol. 24, Issue 12
  • DOI: 10.1175/2010JCLI3584.1

Annular mode changes in the CMIP5 simulations: ANNULAR MODE CHANGES IN CMIP5
journal, March 2013

  • Gillett, N. P.; Fyfe, J. C.
  • Geophysical Research Letters, Vol. 40, Issue 6
  • DOI: 10.1002/grl.50249

Performance metrics for climate models
journal, January 2008

  • Gleckler, P. J.; Taylor, K. E.; Doutriaux, C.
  • Journal of Geophysical Research, Vol. 113, Issue D6
  • DOI: 10.1029/2007JD008972

A More Powerful Reality Test for Climate Models
journal, May 2016


A Large-Ensemble Model Study of the Wintertime AO–NAO and the Role of Interannual Snow Perturbations
journal, December 2002


A Framework for Assessing Operational Madden–Julian Oscillation Forecasts: A CLIVAR MJO Working Group Project
journal, September 2010

  • Gottschalck, J.; Wheeler, M.; Weickmann, K.
  • Bulletin of the American Meteorological Society, Vol. 91, Issue 9
  • DOI: 10.1175/2010BAMS2816.1

New Insights into North European and North Atlantic Surface Pressure Variability, Storminess, and Related Climatic Change since 1830
journal, December 2008


Empirical orthogonal functions and related techniques in atmospheric science: A review
journal, January 2007

  • Hannachi, A.; Jolliffe, I. T.; Stephenson, D. B.
  • International Journal of Climatology, Vol. 27, Issue 9
  • DOI: 10.1002/joc.1499

The Potential to Narrow Uncertainty in Regional Climate Predictions
journal, August 2009

  • Hawkins, Ed; Sutton, Rowan
  • Bulletin of the American Meteorological Society, Vol. 90, Issue 8
  • DOI: 10.1175/2009BAMS2607.1

The potential to narrow uncertainty in projections of regional precipitation change
journal, April 2010


North Atlantic climate variability: The role of the North Atlantic Oscillation
journal, August 2009


A New Method for Identifying the Pacific–South American Pattern and Its Influence on Regional Climate Variability
journal, September 2016


The Community Earth System Model (CESM) Large Ensemble Project: A Community Resource for Studying Climate Change in the Presence of Internal Climate Variability
journal, August 2015

  • Kay, J. E.; Deser, C.; Phillips, A.
  • Bulletin of the American Meteorological Society, Vol. 96, Issue 8
  • DOI: 10.1175/BAMS-D-13-00255.1

Evaluation of short-term climate change prediction in multi-model CMIP5 decadal hindcasts: MULTI-MODEL CMIP5 DECADAL PREDICTIONS
journal, May 2012

  • Kim, Hye-Mi; Webster, Peter J.; Curry, Judith A.
  • Geophysical Research Letters, Vol. 39, Issue 10
  • DOI: 10.1029/2012GL051644

Boreal winter low-frequency variability in CMIP5 models: LOW-FREQUENCY VARIABILITY IN CMIP5
journal, July 2013

  • Lee, Yun-Young; Black, Robert X.
  • Journal of Geophysical Research: Atmospheres, Vol. 118, Issue 13
  • DOI: 10.1002/jgrd.50493

The North Pacific Oscillation–West Pacific Teleconnection Pattern: Mature-Phase Structure and Winter Impacts
journal, May 2008


A Pacific Interdecadal Climate Oscillation with Impacts on Salmon Production
journal, June 1997


Initialized decadal prediction for transition to positive phase of the Interdecadal Pacific Oscillation
journal, June 2016

  • Meehl, Gerald A.; Hu, Aixue; Teng, Haiyan
  • Nature Communications, Vol. 7, Issue 1
  • DOI: 10.1038/ncomms11718

Empirical Orthogonal Functions: The Medium is the Message
journal, December 2009

  • Monahan, Adam H.; Fyfe, John C.; Ambaum, Maarten H. P.
  • Journal of Climate, Vol. 22, Issue 24
  • DOI: 10.1175/2009JCLI3062.1

The Pacific Decadal Oscillation, Revisited
journal, June 2016

  • Newman, Matthew; Alexander, Michael A.; Ault, Toby R.
  • Journal of Climate, Vol. 29, Issue 12
  • DOI: 10.1175/JCLI-D-15-0508.1

Sampling Errors in the Estimation of Empirical Orthogonal Functions
journal, July 1982


An Evaluation of Reproducibility of the Pacific Decadal Oscillation in the CMIP3 Simulations
journal, January 2009

  • Oshima, Kazuhiro; Tanimoto, Youichi
  • Journal of the Meteorological Society of Japan, Vol. 87, Issue 4
  • DOI: 10.2151/jmsj.87.755

Impact of stratospheric ozone hole recovery on Antarctic climate
journal, January 2008

  • Perlwitz, Judith; Pawson, Steven; Fogt, Ryan L.
  • Geophysical Research Letters, Vol. 35, Issue 8
  • DOI: 10.1029/2008GL033317

Evaluating Modes of Variability in Climate Models
journal, December 2014

  • Phillips, Adam S.; Deser, Clara; Fasullo, John
  • Eos, Transactions American Geophysical Union, Vol. 95, Issue 49
  • DOI: 10.1002/2014EO490002

Natural climate variability and teleconnections to precipitation over the Pacific-North American region in CMIP3 and CMIP5 models: CLIMATE VARIABILITY IN CMIP5 MODELS
journal, May 2013

  • Polade, Suraj D.; Gershunov, Alexander; Cayan, Daniel R.
  • Geophysical Research Letters, Vol. 40, Issue 10
  • DOI: 10.1002/grl.50491

ERA-20C: An Atmospheric Reanalysis of the Twentieth Century
journal, June 2016


Global analyses of sea surface temperature, sea ice, and night marine air temperature since the late nineteenth century
journal, January 2003


The North Pacific Oscillation
journal, January 1981


Spatial Variability of Sea Level Pressure and 500 mb Height Anomalies over the Southern Hemisphere
journal, October 1982


Climate Variability and Change: Past, Present and Future – An Overview
journal, May 2005


Identification of human-induced changes in atmospheric moisture content
journal, September 2007

  • Santer, B. D.; Mears, C.; Wentz, F. J.
  • Proceedings of the National Academy of Sciences, Vol. 104, Issue 39
  • DOI: 10.1073/pnas.0702872104

North American Climate in CMIP5 Experiments. Part I: Evaluation of Historical Simulations of Continental and Regional Climatology
journal, December 2013


North American Climate in CMIP5 Experiments. Part II: Evaluation of Historical Simulations of Intraseasonal to Decadal Variability
journal, December 2013


Climate extremes indices in the CMIP5 multimodel ensemble: Part 1. Model evaluation in the present climate: CLIMATE EXTREMES INDICES IN CMIP5
journal, February 2013

  • Sillmann, J.; Kharin, V. V.; Zhang, X.
  • Journal of Geophysical Research: Atmospheres, Vol. 118, Issue 4
  • DOI: 10.1002/jgrd.50203

Improvements to NOAA’s Historical Merged Land–Ocean Surface Temperature Analysis (1880–2006)
journal, May 2008

  • Smith, Thomas M.; Reynolds, Richard W.; Peterson, Thomas C.
  • Journal of Climate, Vol. 21, Issue 10
  • DOI: 10.1175/2007JCLI2100.1

The Impact of Stratospheric Ozone Recovery on the Southern Hemisphere Westerly Jet
journal, June 2008


Madden-Julian variability in NCAR CAM2.0 and CCSM2.0
journal, August 2004


The Madden–Julian oscillation in ECHAM4 coupled and uncoupled general circulation models
journal, June 2005

  • Sperber, Kenneth R.; Gualdi, Silvio; Legutke, Stephanie
  • Climate Dynamics, Vol. 25, Issue 2-3
  • DOI: 10.1007/s00382-005-0026-3

The Asian summer monsoon: an intercomparison of CMIP5 vs. CMIP3 simulations of the late 20th century
journal, December 2012


Assessing General Circulation Model Simulations of Atmospheric Teleconnection Patterns
journal, August 2009

  • Stoner, Anne Marie K.; Hayhoe, Katharine; Wuebbles, Donald J.
  • Journal of Climate, Vol. 22, Issue 16
  • DOI: 10.1175/2009JCLI2577.1

Summarizing multiple aspects of model performance in a single diagram
journal, April 2001

  • Taylor, Karl E.
  • Journal of Geophysical Research: Atmospheres, Vol. 106, Issue D7
  • DOI: 10.1029/2000JD900719

An Overview of CMIP5 and the Experiment Design
journal, April 2012

  • Taylor, Karl E.; Stouffer, Ronald J.; Meehl, Gerald A.
  • Bulletin of the American Meteorological Society, Vol. 93, Issue 4
  • DOI: 10.1175/BAMS-D-11-00094.1

The Arctic oscillation signature in the wintertime geopotential height and temperature fields
journal, May 1998

  • Thompson, David W. J.; Wallace, John M.
  • Geophysical Research Letters, Vol. 25, Issue 9
  • DOI: 10.1029/98GL00950

Annular Modes in the Extratropical Circulation. Part I: Month-to-Month Variability*
journal, March 2000


Regional Climate Impacts of the Northern Hemisphere Annular Mode
journal, July 2001


The Met Office Hadley Centre sea ice and sea surface temperature data set, version 2: 1. Sea ice concentrations: HADISST.2.1.0.0 SEA ICE CONCENTRATIONS
journal, March 2014

  • Titchner, Holly A.; Rayner, Nick A.
  • Journal of Geophysical Research: Atmospheres, Vol. 119, Issue 6
  • DOI: 10.1002/2013JD020316

Teleconnections in the Geopotential Height Field during the Northern Hemisphere Winter
journal, April 1981


An All-Season Real-Time Multivariate MJO Index: Development of an Index for Monitoring and Prediction
journal, August 2004


Visualization and Analysis Tools for Ultrascale Climate Data
journal, October 2014

  • Williams, Dean N.
  • Eos, Transactions American Geophysical Union, Vol. 95, Issue 42
  • DOI: 10.1002/2014EO420002

A Global Repository for Planet-Sized Experiments and Observations
journal, May 2016

  • Williams, Dean N.; Balaji, V.; Cinquini, Luca
  • Bulletin of the American Meteorological Society, Vol. 97, Issue 5
  • DOI: 10.1175/BAMS-D-15-00132.1

Are historical records sufficient to constrain ENSO simulations?
journal, January 2009


ENSO-like Interdecadal Variability: 1900–93
journal, May 1997