skip to main content
OSTI.GOV title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: PyForecastTools

Abstract

The PyForecastTools package provides Python routines for calculating metrics for model validation, forecast verification and model comparison. For continuous predictands the package provides functions for calculating bias (mean error, mean percentage error, median log accuracy, symmetric signed bias), and for calculating accuracy (mean squared error, mean absolute error, mean absolute scaled error, normalized RMSE, median symmetric accuracy). Convenience routines to calculate the component parts (e.g. forecast error, scaled error) of each metric are also provided. To compare models the package provides: generic skill score; percent better. Robust measures of scale including median absolute deviation, robust standard deviation, robust coefficient of variation and the Sn estimator are all provided by the package. Finally, the package implements Python classes for NxN contingency tables. In the case of a multi-class prediction, accuracy and skill metrics such as proportion correct and the Heidke and Peirce skill scores are provided as object methods. The special case of a 2x2 contingency table inherits from the NxN class and provides many additional metrics for binary classification: probability of detection, probability of false detection, false alarm ration, threat score, equitable threat score, bias. Confidence intervals for many of these quantities can be calculated using either the Wald methodmore » or Agresti-Coull intervals.« less

Authors:
 [1]
  1. LANL
Publication Date:
Research Org.:
Los Alamos National Lab. (LANL), Los Alamos, NM (United States)
Sponsoring Org.:
USDOE
Contributing Org.:
Los Alamos National Laboratory (LANL)
OSTI Identifier:
1401961
Report Number(s):
PyForecastTools; 005499MLTPL00
C17112
DOE Contract Number:
AC52-06NA25396
Resource Type:
Software
Software Revision:
00
Software Package Number:
005499
Software CPU:
MLTPL
Open Source:
Yes
Open source under the BSD license.
Source Code Available:
Yes
Related Software:
Numpy, Scipy, and SpacePy libraries
Country of Publication:
United States

Citation Formats

Morley, Steven. PyForecastTools. Computer software. https://www.osti.gov//servlets/purl/1401961. Vers. 00. USDOE. 22 Sep. 2017. Web.
Morley, Steven. (2017, September 22). PyForecastTools (Version 00) [Computer software]. https://www.osti.gov//servlets/purl/1401961.
Morley, Steven. PyForecastTools. Computer software. Version 00. September 22, 2017. https://www.osti.gov//servlets/purl/1401961.
@misc{osti_1401961,
title = {PyForecastTools, Version 00},
author = {Morley, Steven},
abstractNote = {The PyForecastTools package provides Python routines for calculating metrics for model validation, forecast verification and model comparison. For continuous predictands the package provides functions for calculating bias (mean error, mean percentage error, median log accuracy, symmetric signed bias), and for calculating accuracy (mean squared error, mean absolute error, mean absolute scaled error, normalized RMSE, median symmetric accuracy). Convenience routines to calculate the component parts (e.g. forecast error, scaled error) of each metric are also provided. To compare models the package provides: generic skill score; percent better. Robust measures of scale including median absolute deviation, robust standard deviation, robust coefficient of variation and the Sn estimator are all provided by the package. Finally, the package implements Python classes for NxN contingency tables. In the case of a multi-class prediction, accuracy and skill metrics such as proportion correct and the Heidke and Peirce skill scores are provided as object methods. The special case of a 2x2 contingency table inherits from the NxN class and provides many additional metrics for binary classification: probability of detection, probability of false detection, false alarm ration, threat score, equitable threat score, bias. Confidence intervals for many of these quantities can be calculated using either the Wald method or Agresti-Coull intervals.},
url = {https://www.osti.gov//servlets/purl/1401961},
doi = {},
year = {Fri Sep 22 00:00:00 EDT 2017},
month = {Fri Sep 22 00:00:00 EDT 2017},
note =
}

Software:
To order this software, request consultation services, or receive further information, please fill out the following request.

Save / Share:

To initiate an order for this software, request consultation services, or receive further information, fill out the request form below. You may also reach us by email at: .

OSTI staff will begin to process an order for scientific and technical software once the payment and signed site license agreement are received. If the forms are not in order, OSTI will contact you. No further action will be taken until all required information and/or payment is received. Orders are usually processed within three to five business days.

Software Request

(required)
(required)
(required)
(required)
(required)
(required)
(required)
(required)