Search Menu
DOE PAGES title logo U.S. Department of Energy
Office of Scientific and Technical Information
Search Scholarly Publications

Title: On the Effect of Correlations on Rank Histograms: Reliability of Temperature and Wind Speed Forecasts from Finescale Ensemble Reforecasts

Abstract

The rank histogram (RH) is a visual tool for assessing the reliability of ensemble forecasts (i.e., the degree to which the forecasts and the observations have the same distribution). But it is already known that in certain situations it conveys misleading information. Here, it is shown that a temporal correlation can lead to a misleading RH, but such a correlation contributes only to the sampling variability of the RH, and so it is accounted for by producing a RH that explicitly displays sampling variability. A simulation is employed to show that the variance within each ensemble member (i.e., climatological variance), the correlation between ensemble members, and the correlation between the observations and the forecasts, all have a confounding effect on the RH, making it difficult to use the RH for assessing the climatological component of forecast reliability. It is proposed that a “residual” quantile–quantile plot (denoted R-Q-Q plot) is better suited than the RH for assessing the climatological component of forecast reliability. Then, the RH and R-Q-Q plots for temperature and wind speed forecasts at 90 stations across the continental United States are computed. A wide range of forecast reliability is noted. For some stations, the nonreliability of the forecastsmore » can be attributed to bias and/or under-or overclimatological dispersion. For others, the difference between the distributions can be traced to lighter or heavier tails in the distributions, while for other stations the distributions of the forecasts and the observations appear to be completely different. A spatial signature is also noted and discussed briefly.« less

Authors:
 [1];  [2];  [3];  [4]
+ Show Author Affiliations
  1. Univ. of Washington, Seattle, WA (United States). Dept. of Statistics, and Applied Physics Laboratory
  2. Univ. of Washington, Seattle, WA (United States). Dept. of Statistics
  3. Atmospheric Technology Services Company, Norman, OK (United States)
  4. Duke Energy Corporation, Charlotte, NC (United States)
Publication Date:
Research Org.:
Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States). Oak Ridge Leadership Computing Facility (OLCF)
Sponsoring Org.:
USDOE Office of Science (SC)
OSTI Identifier:
1564799
Resource Type:
Accepted Manuscript
Journal Name:
Monthly Weather Review
Additional Journal Information:
Journal Volume: 139; Journal Issue: 1; Journal ID: ISSN 0027-0644
Publisher:
American Meteorological Society
Country of Publication:
United States
Language:
English
Subject:
54 ENVIRONMENTAL SCIENCES; Meteorology & Atmospheric Sciences

Citation Formats

Marzban, Caren, Wang, Ranran, Kong, Fanyou, and Leyton, Stephen. On the Effect of Correlations on Rank Histograms: Reliability of Temperature and Wind Speed Forecasts from Finescale Ensemble Reforecasts. United States: N. p., 2011. Web. doi:10.1175/2010mwr3129.1.
Copy to clipboard
Marzban, Caren, Wang, Ranran, Kong, Fanyou, & Leyton, Stephen. On the Effect of Correlations on Rank Histograms: Reliability of Temperature and Wind Speed Forecasts from Finescale Ensemble Reforecasts. United States. https://doi.org/10.1175/2010mwr3129.1
Copy to clipboard
Marzban, Caren, Wang, Ranran, Kong, Fanyou, and Leyton, Stephen. Sat . "On the Effect of Correlations on Rank Histograms: Reliability of Temperature and Wind Speed Forecasts from Finescale Ensemble Reforecasts". United States. https://doi.org/10.1175/2010mwr3129.1. https://www.osti.gov/servlets/purl/1564799.
Copy to clipboard
@article{osti_1564799,
title = {On the Effect of Correlations on Rank Histograms: Reliability of Temperature and Wind Speed Forecasts from Finescale Ensemble Reforecasts},
author = {Marzban, Caren and Wang, Ranran and Kong, Fanyou and Leyton, Stephen},
abstractNote = {The rank histogram (RH) is a visual tool for assessing the reliability of ensemble forecasts (i.e., the degree to which the forecasts and the observations have the same distribution). But it is already known that in certain situations it conveys misleading information. Here, it is shown that a temporal correlation can lead to a misleading RH, but such a correlation contributes only to the sampling variability of the RH, and so it is accounted for by producing a RH that explicitly displays sampling variability. A simulation is employed to show that the variance within each ensemble member (i.e., climatological variance), the correlation between ensemble members, and the correlation between the observations and the forecasts, all have a confounding effect on the RH, making it difficult to use the RH for assessing the climatological component of forecast reliability. It is proposed that a “residual” quantile–quantile plot (denoted R-Q-Q plot) is better suited than the RH for assessing the climatological component of forecast reliability. Then, the RH and R-Q-Q plots for temperature and wind speed forecasts at 90 stations across the continental United States are computed. A wide range of forecast reliability is noted. For some stations, the nonreliability of the forecasts can be attributed to bias and/or under-or overclimatological dispersion. For others, the difference between the distributions can be traced to lighter or heavier tails in the distributions, while for other stations the distributions of the forecasts and the observations appear to be completely different. A spatial signature is also noted and discussed briefly.},
doi = {10.1175/2010mwr3129.1},
journal = {Monthly Weather Review},
number = 1,
volume = 139,
place = {United States},
year = {Sat Jan 01 00:00:00 EST 2011},
month = {Sat Jan 01 00:00:00 EST 2011}
}
Copy to clipboard
Journal Article:
Free Publicly Available Full Text
Publisher's Version of Record
https://doi.org/10.1175/2010mwr3129.1
Copyright Statement
Other availability
Search WorldCat Search WorldCat to find libraries that may hold this journal
Citation Metrics:
Cited by: 22 works
Citation information provided by
Web of Science
Save / Share:
Export Metadata
Save to My Library
You must Sign In or Create an Account in order to save documents to your library.

Works referenced in this record:

Bootstraps for Time Series
journal, May 2002

An Introduction to the Bootstrap
book, May 1994

  • Efron, Bradley; Tibshirani, R. J.
  • Monographs on Statistics and Applied Probability
  • DOI: 10.1201/9780429246593

Alternatives to the Chi-Square Test for Evaluating Rank Histograms from Ensemble Forecasts
journal, October 2005

  • Elmore, Kimberly L.
  • Weather and Forecasting, Vol. 20, Issue 5
  • DOI: 10.1175/WAF884.1

Probabilistic forecasts, calibration and sharpness
journal, April 2007

  • Gneiting, Tilmann; Balabdaoui, Fadoua; Raftery, Adrian E.
  • Journal of the Royal Statistical Society: Series B (Statistical Methodology), Vol. 69, Issue 2
  • DOI: 10.1111/j.1467-9868.2007.00587.x

Interpretation of Rank Histograms for Verifying Ensemble Forecasts
journal, March 2001

Verification of Eta–RSM Short-Range Ensemble Forecasts
journal, June 1997

Evaluating Rank Histograms Using Decompositions of the Chi-Square Test Statistic
journal, June 2008

Subsampling
book, January 1999

On the Reliability of the Rank Histogram
journal, January 2011

    Sort by:
    [ × clear filter / sort ]

    Works referencing / citing this record:

    Scaling regimes and linear/nonlinear responses of last millennium climate to volcanic and solar forcings
    journal, January 2016

      Sort by:
      [ × clear filter / sort ]

      Similar Records in DOE PAGES and OSTI.GOV collections: