Resampling and data augmentation for short-term PV output prediction based on an imbalanced sky images dataset using convolutional neural networks

Nie, Yuhao; Zamzam, Ahmed S.; Brandt, Adam

doi:10.1016/j.solener.2021.05.095

Resampling and data augmentation for short-term PV output prediction based on an imbalanced sky images dataset using convolutional neural networks

Journal Article · Wed Jun 16 00:00:00 EDT 2021 · Solar Energy

DOI:https://doi.org/10.1016/j.solener.2021.05.095· OSTI ID:1805186

^[1]; ^[2]; ^[1]

Stanford Univ., CA (United States)
National Renewable Energy Lab. (NREL), Golden, CO (United States)

Integrating photovoltaics (PV) into electricity grids is challenged by potentially large fluctuations in power generation. In recent years, sky image-based PV output prediction using convolutional neural networks (CNNs) has emerged as a promising approach to forecasting fluctuations. A key challenge is imbalanced sky image datasets: because of the geography of solar PV system installations, sky image datasets are often rich in sunny condition data but deficient in cloudy condition data. This imbalance contrasts with the fact that model errors are dominated by cloudy condition performance. In this study, we attempt to remedy this by exploring the enrichment and augmentation of an imbalanced sky images dataset for two PV output prediction tasks: nowcasting (predicting concurrent PV output) and forecasting (predicting 15-minute-ahead future PV output). We empirically examine the efficacy of using different resampling and data augmentation approaches to create a rebalanced dataset for model development. A three-stage greedy search is used to determine the optimal resampling approach, data augmentation techniques and over-sampling rate. The results show that for the nowcast problem, resampling and data augmentation can effectively enhance the model performance, reducing overall root mean squared error (RMSE) by an average of 4%, or a 15 std. (standard deviation) of improvement compared to the variability of the baseline model. In contrast, the treatment RMSE for the forecast problem nearly always overlaps the baseline performance at the ± 2 std. level. The optimal resampling approach expands on the original dataset by over-sampling the minority cloudy data, with the best results from large over-sampling rate (e.g., 4 ~ 6 times over-sampling of cloudy images).

View Accepted Manuscript (DOE)

Research Organization:: National Renewable Energy Laboratory (NREL), Golden, CO (United States)

Sponsoring Organization:: USDOE National Renewable Energy Laboratory (NREL), Laboratory Directed Research and Development (LDRD) Program

Grant/Contract Number:: AC36-08GO28308

OSTI ID:: 1805186

Report Number(s):: NREL/JA--5D00-80184; MainId:42387; UUID:2c9c3c5c-19b9-4736-8c87-457bb9442e2a; MainAdminID:25654

Journal Information:: Solar Energy, Journal Name: Solar Energy Vol. 224; ISSN 0038-092X

Publisher:: ElsevierCopyright Statement

Country of Publication:: United States

Language:: English

References (27)

Empirical Risk Minimization Zhang, Xinhua Encyclopedia of Machine Learning and Data Mining https://doi.org/10.1007/978-1-4899-7687-1_79	book	January 2017
GAN-based synthetic medical image augmentation for increased CNN performance in liver lesion classification Frid-Adar, Maayan; Diamant, Idit; Klang, Eyal Neurocomputing, Vol. 321 https://doi.org/10.1016/j.neucom.2018.09.013	journal	December 2018
A systematic study of the class imbalance problem in convolutional neural networks Buda, Mateusz; Maki, Atsuto; Mazurowski, Maciej A. Neural Networks, Vol. 106 https://doi.org/10.1016/j.neunet.2018.07.011	journal	October 2018
Real-time prediction intervals for intra-hour DNI forecasts Chu, Yinghao; Li, Mengying; Pedro, Hugo T. C. Renewable Energy, Vol. 83 https://doi.org/10.1016/j.renene.2015.04.022	journal	November 2015
Hybrid intra-hour DNI forecasts with sky image processing enhanced by stochastic learning Chu, Yinghao; Pedro, Hugo T. C.; Coimbra, Carlos F. M. Solar Energy, Vol. 98 https://doi.org/10.1016/j.solener.2013.10.020	journal	December 2013
Short-term reforecasting of power output from a 48 MWe solar PV plant Chu, Yinghao; Urquhart, Bryan; Gohari, Seyyed M. I. Solar Energy, Vol. 112 https://doi.org/10.1016/j.solener.2014.11.017	journal	February 2015
Deep photovoltaic nowcasting Zhang, Jinsong; Verschae, Rodrigo; Nobuhara, Shohei Solar Energy, Vol. 176 https://doi.org/10.1016/j.solener.2018.10.024	journal	December 2018
Short-term solar power forecast with deep learning: Exploring optimal input and output configuration Sun, Yuchi; Venugopal, Vignesh; Brandt, Adam R. Solar Energy, Vol. 188 https://doi.org/10.1016/j.solener.2019.06.041	journal	August 2019
Data augmentation using generative adversarial networks (CycleGAN) to improve generalizability in CT segmentation tasks Sandfort, Veit; Yan, Ke; Pickhardt, Perry J. Scientific Reports, Vol. 9, Issue 1 https://doi.org/10.1038/s41598-019-52737-x	journal	November 2019
Solar PV output prediction from video streams using convolutional neural networks Sun, Yuchi; Szűcs, Gergely; Brandt, Adam R. Energy & Environmental Science, Vol. 11, Issue 7 https://doi.org/10.1039/C7EE03420B	journal	January 2018
A guideline to solar forecasting research practice: Reproducible, operational, probabilistic or physically-based, ensemble, and skill (ROPES) Yang, Dazhi Journal of Renewable and Sustainable Energy, Vol. 11, Issue 2 https://doi.org/10.1063/1.5087462	journal	March 2019
PV power output prediction from sky images using convolutional neural network: The comparison of sky-condition-specific sub-models and an end-to-end model Nie, Yuhao; Sun, Yuchi; Chen, Yuanlei Journal of Renewable and Sustainable Energy, Vol. 12, Issue 4 https://doi.org/10.1063/5.0014016	journal	July 2020
Solar Irradiance Capturing in Cloudy Sky Days–A Convolutional Neural Network Based Image Regression Approach Jiang, Huaiguang; Gu, Yi; Xie, Yu IEEE Access, Vol. 8 https://doi.org/10.1109/ACCESS.2020.2969549	journal	January 2020
Image Style Transfer Using Convolutional Neural Networks Gatys, Leon A.; Ecker, Alexander S.; Bethge, Matthias 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) https://doi.org/10.1109/CVPR.2016.265	conference	June 2016
Learning Deep Representation for Imbalanced Classification Huang, Chen; Li, Yining; Loy, Chen Change 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) https://doi.org/10.1109/CVPR.2016.580	conference	June 2016
Image-to-Image Translation with Conditional Adversarial Networks Isola, Phillip; Zhu, Jun-Yan; Zhou, Tinghui 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) https://doi.org/10.1109/CVPR.2017.632	conference	July 2017
High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs Wang, Ting-Chun; Liu, Ming-Yu; Zhu, Jun-Yan 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) https://doi.org/10.1109/CVPR.2018.00917	conference	June 2018
Unpaired Image-to-Image Translation Using Cycle-Consistent Adversarial Networks Zhu, Jun-Yan; Park, Taesung; Isola, Phillip 2017 IEEE International Conference on Computer Vision (ICCV) https://doi.org/10.1109/ICCV.2017.244	conference	October 2017
Plankton classification on imbalanced large scale database via convolutional neural networks with transfer learning Lee, Hansang; Park, Minseok; Kim, Junmo 2016 IEEE International Conference on Image Processing (ICIP) https://doi.org/10.1109/ICIP.2016.7533053	conference	September 2016
Training cost-sensitive Deep Belief Networks on imbalance data problems Zhang, Chong; Tan, Kay Chen; Ren, Ruoxu 2016 International Joint Conference on Neural Networks (IJCNN) https://doi.org/10.1109/IJCNN.2016.7727769	conference	July 2016
Training deep neural networks on imbalanced data sets Wang, Shoujin; Liu, Wei; Wu, Jia 2016 International Joint Conference on Neural Networks (IJCNN) https://doi.org/10.1109/IJCNN.2016.7727770	conference	July 2016
Domain randomization for transferring deep neural networks from simulation to the real world Tobin, Josh; Fong, Rachel; Ray, Alex 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) https://doi.org/10.1109/IROS.2017.8202133	conference	September 2017
Predicting Hospital Readmission via Cost-Sensitive Deep Learning Wang, Haishuai; Cui, Zhicheng; Chen, Yixin IEEE/ACM Transactions on Computational Biology and Bioinformatics, Vol. 15, Issue 6 https://doi.org/10.1109/TCBB.2018.2827029	journal	November 2018
Imbalanced Deep Learning by Minority Class Incremental Rectification Dong, Qi; Gong, Shaogang; Zhu, Xiatian IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 41, Issue 6 https://doi.org/10.1109/TPAMI.2018.2832629	journal	June 2019
Resampling strategies for regression Torgo, Luís; Branco, Paula; Ribeiro, Rita P. Expert Systems, Vol. 32, Issue 3 https://doi.org/10.1111/exsy.12081	journal	August 2014
Survey on deep learning with class imbalance Johnson, Justin M.; Khoshgoftaar, Taghi M. Journal of Big Data, Vol. 6, Issue 1 https://doi.org/10.1186/s40537-019-0192-5	journal	March 2019
A survey on Image Data Augmentation for Deep Learning Shorten, Connor; Khoshgoftaar, Taghi M. Journal of Big Data, Vol. 6, Issue 1 https://doi.org/10.1186/s40537-019-0197-0	journal	July 2019

Similar Records

Benchmarking of Solar Irradiance Nowcast Performance Derived from All-Sky Imagers

Journal Article · Sat Sep 03 00:00:00 EDT 2022 · Renewable Energy · OSTI ID:1890144

Even a good influenza forecasting model can benefit from internet-based nowcasts, but those benefits are limited

Journal Article · Thu Jan 31 19:00:00 EST 2019 · PLoS Computational Biology (Online) · OSTI ID:1495153

Spatio-Temporal Denoising Graph Autoencoders with Data Augmentation for Missing Photovoltaic Data Imputation

Conference · Fri Jun 23 00:00:00 EDT 2023 · OSTI ID:1959292

Related Subjects

14 SOLAR ENERGY
97 MATHEMATICS AND COMPUTING
convolutional neural networks
imbalanced datasets
resampling
sky images
solar forecasting

Resampling and data augmentation for short-term PV output prediction based on an imbalanced sky images dataset using convolutional neural networks

Citation Formats

References (27)

Similar Records

Related Subjects