skip to main content
DOE PAGES title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: The importance of round-robin validation when assessing machine-learning-based vertical extrapolation of wind speeds

Abstract

The extrapolation of wind speeds measured at a meteorological mast to wind turbine rotor heights is a key component in a bankable wind farm energy assessment and a significant source of uncertainty. Industry-standard methods for extrapolation include the power-law and logarithmic profiles. The emergence of machine-learning applications in wind energy has led to several studies demonstrating substantial improvements in vertical extrapolation accuracy in machine-learning methods over these conventional power-law and logarithmic profile methods. In all cases, these studies assess relative model performance at a measurement site where, critically, the machine-learning algorithm requires knowledge of the rotor-height wind speeds in order to train the model. This prior knowledge provides fundamental advantages to the site-specific machine-learning model over the power-law and log profiles, which, by contrast, are not highly tuned to rotor-height measurements but rather can generalize to any site. Furthermore, there is no practical benefit in applying a machine-learning model at a site where winds at the heights relevant for wind energy production are known; rather, its performance at nearby locations (i.e., across a wind farm site) without rotor-height measurements is of most practical interest. To more fairly and practically compare machine-learning-based extrapolation to standard approaches, we implemented a round-robin extrapolationmore » model comparison, in which a random-forest machine-learning model is trained and evaluated at different sites and then compared against the power-law and logarithmic profiles. We consider 20 months of lidar and sonic anemometer data collected at four sites between 50 and 100 km apart in the central United States. We find that the random forest outperforms the standard extrapolation approaches, especially when incorporating surface measurements as inputs to include the influence of atmospheric stability. When compared at a single site (the traditional comparison approach), the machine-learning improvement in mean absolute error was 28 % and 23 % over the power-law and logarithmic profiles, respectively. Using the round-robin approach proposed here, this improvement drops to 20 % and 14 %, respectively. These latter values better represent practical model performance, and we conclude that round-robin validation should be the standard for machine-learning-based wind speed extrapolation methods.« less

Authors:
 [1]; ORCiD logo [1]
+ Show Author Affiliations
  1. National Renewable Energy Lab. (NREL), Golden, CO (United States)
Publication Date:
Research Org.:
National Renewable Energy Lab. (NREL), Golden, CO (United States)
Sponsoring Org.:
USDOE Office of Energy Efficiency and Renewable Energy (EERE), Renewable Power Office. Wind Energy Technologies Office
OSTI Identifier:
1677497
Report Number(s):
NREL/JA-5000-78109
Journal ID: ISSN 2366-7451; MainId:32018;UUID:9f734336-6146-4cfb-ae2c-7b470e3c05f6;MainAdminID:18706
Grant/Contract Number:  
AC36-08GO28308
Resource Type:
Accepted Manuscript
Journal Name:
Wind Energy Science (Online)
Additional Journal Information:
Journal Name: Wind Energy Science (Online); Journal Volume: 5; Journal Issue: 2; Journal ID: ISSN 2366-7451
Publisher:
European Wind Energy Association - Copernicus
Country of Publication:
United States
Language:
English
Subject:
17 WIND ENERGY; wind speeds; round-robin validation; machine learning; wind energy

Citation Formats

Bodini, Nicola, and Optis, Mike. The importance of round-robin validation when assessing machine-learning-based vertical extrapolation of wind speeds. United States: N. p., 2020. Web. https://doi.org/10.5194/wes-5-489-2020.
Copy to clipboard
Bodini, Nicola, & Optis, Mike. The importance of round-robin validation when assessing machine-learning-based vertical extrapolation of wind speeds. United States. https://doi.org/10.5194/wes-5-489-2020
Copy to clipboard
Bodini, Nicola, and Optis, Mike. Fri . "The importance of round-robin validation when assessing machine-learning-based vertical extrapolation of wind speeds". United States. https://doi.org/10.5194/wes-5-489-2020. https://www.osti.gov/servlets/purl/1677497.
Copy to clipboard
@article{osti_1677497,
title = {The importance of round-robin validation when assessing machine-learning-based vertical extrapolation of wind speeds},
author = {Bodini, Nicola and Optis, Mike},
abstractNote = {The extrapolation of wind speeds measured at a meteorological mast to wind turbine rotor heights is a key component in a bankable wind farm energy assessment and a significant source of uncertainty. Industry-standard methods for extrapolation include the power-law and logarithmic profiles. The emergence of machine-learning applications in wind energy has led to several studies demonstrating substantial improvements in vertical extrapolation accuracy in machine-learning methods over these conventional power-law and logarithmic profile methods. In all cases, these studies assess relative model performance at a measurement site where, critically, the machine-learning algorithm requires knowledge of the rotor-height wind speeds in order to train the model. This prior knowledge provides fundamental advantages to the site-specific machine-learning model over the power-law and log profiles, which, by contrast, are not highly tuned to rotor-height measurements but rather can generalize to any site. Furthermore, there is no practical benefit in applying a machine-learning model at a site where winds at the heights relevant for wind energy production are known; rather, its performance at nearby locations (i.e., across a wind farm site) without rotor-height measurements is of most practical interest. To more fairly and practically compare machine-learning-based extrapolation to standard approaches, we implemented a round-robin extrapolation model comparison, in which a random-forest machine-learning model is trained and evaluated at different sites and then compared against the power-law and logarithmic profiles. We consider 20 months of lidar and sonic anemometer data collected at four sites between 50 and 100 km apart in the central United States. We find that the random forest outperforms the standard extrapolation approaches, especially when incorporating surface measurements as inputs to include the influence of atmospheric stability. When compared at a single site (the traditional comparison approach), the machine-learning improvement in mean absolute error was 28 % and 23 % over the power-law and logarithmic profiles, respectively. Using the round-robin approach proposed here, this improvement drops to 20 % and 14 %, respectively. These latter values better represent practical model performance, and we conclude that round-robin validation should be the standard for machine-learning-based wind speed extrapolation methods.},
doi = {10.5194/wes-5-489-2020},
journal = {Wind Energy Science (Online)},
number = 2,
volume = 5,
place = {United States},
year = {2020},
month = {4}
}
Copy to clipboard
Journal Article:
Free Publicly Available Full Text
Publisher's Version of Record
https://doi.org/10.5194/wes-5-489-2020
Copyright Statement
Other availability
Search WorldCat Search WorldCat to find libraries that may hold this journal

Figures / Tables:

Figure 1 Figure 1: Map of the four sites at the Southern Great Plains atmospheric observatory considered in this study. Contour lines at 50 m intervals are shown in the map. Digital elevation model data courtesy of the U.S. Geological Survey.
All figures and tables (13 total)
Save / Share:
Export Metadata
Save to My Library
You must Sign In or Create an Account in order to save documents to your library.

Works referenced in this record:

Difficulties in using power laws for wind energy assessment
journal, January 1983

Flux Parameterization over Land Surfaces for Atmospheric Models
journal, March 1991

Calculation of Wind Speed Variation with Height over the Sea
journal, May 2006

Flux-gradient relationships in the constant flux layer
journal, October 1970

  • Dyer, A. J.; Hicks, B. B.
  • Quarterly Journal of the Royal Meteorological Society, Vol. 96, Issue 410
  • DOI: 10.1002/qj.49709641012

Power Law Extrapolation of Wind Measurements for Predicting Wind Energy Production
journal, May 2009

Effects of precipitation on sonic anemometer measurements of turbulent fluxes in the atmospheric surface layer
journal, May 2016

  • Zhang, Rongwang; Huang, Jian; Wang, Xin
  • Journal of Ocean University of China, Vol. 15, Issue 3
  • DOI: 10.1007/s11802-016-2804-4

Wind Speed Extrapolation Using Machine Learning Methods and LiDAR Measurements
journal, January 2018

Diurnal Cycles of Surface Winds and Temperatures as Simulated by Five Boundary Layer Parameterizations
journal, January 2004

On the Use of Power Laws for Estimates of Wind Power Potential
journal, March 1978

Moving Beyond Monin–Obukhov Similarity Theory in Modelling Wind-Speed Profiles in the Lower Atmospheric Boundary Layer under Stable Stratification
journal, August 2014

  • Optis, Michael; Monahan, Adam; Bosveld, Fred C.
  • Boundary-Layer Meteorology, Vol. 153, Issue 3
  • DOI: 10.1007/s10546-014-9953-z

Observations and simulations of diurnal cycles of near-surface wind speeds over land and sea
journal, September 1996

  • Barthelmie, R. J.; Grisogono, B.; Pryor, S. C.
  • Journal of Geophysical Research: Atmospheres, Vol. 101, Issue D16
  • DOI: 10.1029/96JD01520

The MATERHORN: Unraveling the Intricacies of Mountain Weather
journal, November 2015

  • Fernando, H. J. S.; Pardyjak, E. R.; Di Sabatino, S.
  • Bulletin of the American Meteorological Society, Vol. 96, Issue 11
  • DOI: 10.1175/BAMS-D-13-00131.1

Measurements of Boundary Layer Profiles in an Urban Environment
journal, June 2006

  • Frehlich, Rod; Meillier, Yannick; Jensen, Michael L.
  • Journal of Applied Meteorology and Climatology, Vol. 45, Issue 6
  • DOI: 10.1175/JAM2368.1

Extrapolating Satellite Winds to Turbine Operating Heights
journal, April 2016

  • Badger, Merete; Peña, Alfredo; Hahmann, Andrea N.
  • Journal of Applied Meteorology and Climatology, Vol. 55, Issue 4
  • DOI: 10.1175/JAMC-D-15-0197.1

The prediction of the wind speed at different heights by machine learning methods
journal, July 2016

  • Türkan, Yusuf S.; Yumurtacı Aydoğmuş, Hacer; Erdal, Hamit
  • An International Journal of Optimization and Control: Theories & Applications (IJOCTA), Vol. 6, Issue 2
  • DOI: 10.11121/ijocta.01.2016.00315

machine.
journal, October 2001

The elements of statistical learning: data mining, inference and prediction
journal, March 2005

  • Franklin, James
  • The Mathematical Intelligencer, Vol. 27, Issue 2
  • DOI: 10.1007/BF02985802

A Comparison of Equilibrium and Time-Evolving Approaches to Modeling the Wind Profile under Stable Stratification
journal, May 2017

  • Optis, Michael; Monahan, Adam
  • Journal of Applied Meteorology and Climatology, Vol. 56, Issue 5
  • DOI: 10.1175/JAMC-D-16-0324.1

The Determination of Kinematic Properties of a Wind Field Using Doppler Radar
journal, February 1968

    [ × clear filter / sort ]

    Figures / Tables found in this record:

    Figure 1 (p. 3)figure
    Table 1 (p. 3)table
    Table 2 (p. 5)table
    Table 3 (p. 5)table
    Figure 2 (p. 6)figure
    Figure 3 (p. 6)figure
    Table 4 (p. 7)table
    Figure 4 (p. 8)figure
    Figure 5 (p. 8)figure
    Figure 6 (p. 9)figure
    Figure 7 (p. 9)figure
    Table 5 (p. 10)table
    Table A1 (p. 11)table
      [ × clear filter / sort ]
      Figures/Tables have been extracted from DOE-funded journal article accepted manuscripts.

      Similar Records in DOE PAGES and OSTI.GOV collections: