skip to main content
OSTI.GOV title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: The Importance of Round-Robin Validation When Assessing Machine-Learning-Based Vertical Extrapolation of Wind Speeds.

Abstract

The extrapolation of wind speeds measured at a meteorological mast to wind turbine hub heights is a key component in a bankable wind farm energy assessment and a significant source of uncertainty. Industry-standard methods for extrapolation include the power law and logarithmic profile. The emergence of machine-learning applications in wind energy has led to several studies demonstrating substantial improvements in vertical extrapolation accuracy in machine-learning methods over these conventional power law and logarithmic profile methods. In all cases, these studies assess relative model performance at a measurement site where, critically, the machine-learning algorithm requires knowledge of the hub-height wind speeds in order to train the model. This prior knowledge provides fundamental advantages to the site-specific machine-learning model over the power law and log profile, which, by contrast, are not highly tuned to hub-height measurements but rather can generalize to any site. Furthermore, there is no practical benefit in applying a machine-learning model at a site where hub-height winds are known; rather, its performance at nearby locations (i.e., across a wind farm site) without hub-height measurements is of most practical interest. To more fairly and practically compare machine-learning-based extrapolation to standard approaches, we implemented a round-robin extrapolation model comparison, in whichmore » a random forest machine-learning model is trained and evaluated at different sites and then compared against the power law and logarithmic profile. We consider 20 months of lidar and sonic anemometer data collected at four sites between 50-100 kilometers apart in the central United States. We find that the random forest outperforms the standard extrapolation approaches, especially when incorporating surface measurements as inputs to include the influence of atmospheric stability. When compared at a single site (the traditional comparison approach), the machine-learning improvement in mean absolute error was 28% and 23% over the power law and logarithmic profile, respectively. Using the round-robin approach proposed here, this improvement drops to 19% and 14%, respectively. These latter values better represent practical model performance, and we conclude that round-robin validation should be the standard for machine-learning-based, wind-speed extrapolation methods.« less

Authors:
 [1]; ORCiD logo [1]
  1. National Renewable Energy Laboratory (NREL), Golden, CO (United States)
Publication Date:
Research Org.:
National Renewable Energy Lab. (NREL), Golden, CO (United States)
Sponsoring Org.:
USDOE Office of Energy Efficiency and Renewable Energy (EERE), Wind and Water Technologies Office (EE-4W)
OSTI Identifier:
1606120
Report Number(s):
NREL/JA-5000-75500
DOE Contract Number:  
AC36-08GO28308
Resource Type:
Journal Article
Journal Name:
Wind Energy Science
Additional Journal Information:
Journal Name: Wind Energy Science
Country of Publication:
United States
Language:
English
Subject:
17 WIND ENERGY; wind speeds; round-robin validation; machine learning; wind energy

Citation Formats

Bodini, Nicola, and Optis, Michael. The Importance of Round-Robin Validation When Assessing Machine-Learning-Based Vertical Extrapolation of Wind Speeds.. United States: N. p., 2020. Web. doi:10.5194/wes-2020-2.
Bodini, Nicola, & Optis, Michael. The Importance of Round-Robin Validation When Assessing Machine-Learning-Based Vertical Extrapolation of Wind Speeds.. United States. doi:10.5194/wes-2020-2.
Bodini, Nicola, and Optis, Michael. Tue . "The Importance of Round-Robin Validation When Assessing Machine-Learning-Based Vertical Extrapolation of Wind Speeds.". United States. doi:10.5194/wes-2020-2.
@article{osti_1606120,
title = {The Importance of Round-Robin Validation When Assessing Machine-Learning-Based Vertical Extrapolation of Wind Speeds.},
author = {Bodini, Nicola and Optis, Michael},
abstractNote = {The extrapolation of wind speeds measured at a meteorological mast to wind turbine hub heights is a key component in a bankable wind farm energy assessment and a significant source of uncertainty. Industry-standard methods for extrapolation include the power law and logarithmic profile. The emergence of machine-learning applications in wind energy has led to several studies demonstrating substantial improvements in vertical extrapolation accuracy in machine-learning methods over these conventional power law and logarithmic profile methods. In all cases, these studies assess relative model performance at a measurement site where, critically, the machine-learning algorithm requires knowledge of the hub-height wind speeds in order to train the model. This prior knowledge provides fundamental advantages to the site-specific machine-learning model over the power law and log profile, which, by contrast, are not highly tuned to hub-height measurements but rather can generalize to any site. Furthermore, there is no practical benefit in applying a machine-learning model at a site where hub-height winds are known; rather, its performance at nearby locations (i.e., across a wind farm site) without hub-height measurements is of most practical interest. To more fairly and practically compare machine-learning-based extrapolation to standard approaches, we implemented a round-robin extrapolation model comparison, in which a random forest machine-learning model is trained and evaluated at different sites and then compared against the power law and logarithmic profile. We consider 20 months of lidar and sonic anemometer data collected at four sites between 50-100 kilometers apart in the central United States. We find that the random forest outperforms the standard extrapolation approaches, especially when incorporating surface measurements as inputs to include the influence of atmospheric stability. When compared at a single site (the traditional comparison approach), the machine-learning improvement in mean absolute error was 28% and 23% over the power law and logarithmic profile, respectively. Using the round-robin approach proposed here, this improvement drops to 19% and 14%, respectively. These latter values better represent practical model performance, and we conclude that round-robin validation should be the standard for machine-learning-based, wind-speed extrapolation methods.},
doi = {10.5194/wes-2020-2},
journal = {Wind Energy Science},
number = ,
volume = ,
place = {United States},
year = {2020},
month = {1}
}