Multi-agent voltage control in distribution systems using GAN-DRL-based approach

Hossain, Rakib; Gautam, Mukesh; Olowolaju, Joshua; Livani, Hanif; Benidris, Mohammed

doi:10.1016/j.epsr.2024.110528

Multi-agent voltage control in distribution systems using GAN-DRL-based approach

Journal Article · Fri Jun 07 00:00:00 EDT 2024 · Electric Power Systems Research

DOI:https://doi.org/10.1016/j.epsr.2024.110528· OSTI ID:2403534

Hossain, Rakib ^[1]; ^[2]; ^[1]; Livani, Hanif ^[1]; ^[3]

Univ. of Nevada, Reno, NV (United States)
Pacific Northwest National Laboratory (PNNL), Richland, WA (United States); Idaho National Laboratory (INL), Idaho Falls, ID (United States)
Michigan State Univ., East Lansing, MI (United States)

Active distribution grids can experience voltage fluctuations and violations due to the high penetration of variable distributed energy resources (DERs). These problems might occur because of the uncertain and variable generation natures of these resources, especially solar photovoltaic resources, during panel shadowing scenarios. Volt-VAR control (VVC) is an efficient method that controls the reactive power set-points of the inverters to regulate the voltage of distribution grids. Although several VVC approaches have been proposed recently, the performance of these approaches degrades significantly if behind-the-meter solar generation data are unobservable/missing. Therefore, it is necessary to impute missing/unobservable PV data accurately to be utilized in VVC approaches. Further, this paper proposes a model-free, data-driven, centrally trained, and decentrally executed multi-agent deep reinforcement learning-based VVC architecture to regulate the voltage of distribution networks. A generative adversarial network (GAN) is incorporated to impute the unobservable PV data accurately, which improves the performance of the proposed control architecture. The proposed multi-agent-soft-actor–critic algorithm (MASAC)-based VVC technique utilizes the actual PV dataset as well as the imputed dataset from the GAN framework to learn the optimal coordinated control policy for controlling the optimal reactive power set-points of PV inverters. The effectiveness of the proposed approach is analyzed on a modified IEEE 34-bus test case with added PV inverters. The results are compared and analyzed with a base case model with no VVC and VVC with a local droop control approach, genetic algorithm optimization, and a centralized soft actor–critic-based approach. Moreover, the performance of the proposed approach is compared with that of a multi-agent VVC framework without using the PV generation data and load information as the system state. The results illustrate that the proposed method with more state input improves the voltage profile and reduces the power loss of the network across various loading and PV generation scenarios.

View Accepted Manuscript (DOE)

Research Organization:: Idaho National Laboratory (INL), Idaho Falls, ID (United States); Pacific Northwest National Laboratory (PNNL), Richland, WA (United States)

Sponsoring Organization:: USDOE Office of Energy Efficiency and Renewable Energy (EERE)

Grant/Contract Number:: AC05-76RL01830; AC07-05ID14517; EE0009022

OSTI ID:: 2403534

Alternate ID(s):: OSTI ID: 2475152

Report Number(s):: INL/JOU--23-74649-Rev000; PNNL-SA--195602

Journal Information:: Electric Power Systems Research, Journal Name: Electric Power Systems Research Vol. 234; ISSN 0378-7796

Publisher:: ElsevierCopyright Statement

Country of Publication:: United States

Language:: English

References (31)

Voltage behaviour in radial distribution systems under the uncertainties of photovoltaic systems and electric vehicle charging loads Ruiz-Rodriguez, Francisco Javier; Hernández, Jesus C.; Jurado, Francisco International Transactions on Electrical Energy Systems, Vol. 28, Issue 2 https://doi.org/10.1002/etep.2490	journal	December 2017
A centralized-based method to determine the local voltage control strategies of distributed generator operation in active distribution networks Ji, Haoran; Wang, Chengshan; Li, Peng Applied Energy, Vol. 228 https://doi.org/10.1016/j.apenergy.2018.07.065	journal	October 2018
Online reconfiguration scheme of self-sufficient distribution network based on a reinforcement learning approach Oh, Seok Hwa; Yoon, Yong Tae; Kim, Seung Wan Applied Energy, Vol. 280 https://doi.org/10.1016/j.apenergy.2020.115900	journal	December 2020
An unsupervised data-driven approach for behind-the-meter photovoltaic power generation disaggregation Pan, Keda; Chen, Zhaohua; Lai, Chun Sing Applied Energy, Vol. 309 https://doi.org/10.1016/j.apenergy.2021.118450	journal	March 2022
State-based Volt/VAR control strategies for active distribution networks Yilmaz, Mehmet; El-Shatshat, Ramadan International Journal of Electrical Power & Energy Systems, Vol. 100 https://doi.org/10.1016/j.ijepes.2018.02.040	journal	September 2018
Deep reinforcement learning assisted co-optimization of Volt-VAR grid service in distribution networks Hossain, Rakib; Gautam, Mukesh; Thapa, Jitendra Sustainable Energy, Grids and Networks, Vol. 35 https://doi.org/10.1016/j.segan.2023.101086	journal	September 2023
Efficient Control of Active Transformers for Increasing the PV Hosting Capacity of LV Grids Hashemi, Seyedmostafa; Ostergaard, Jacob; Degner, Thomas IEEE Transactions on Industrial Informatics, Vol. 13, Issue 1 https://doi.org/10.1109/TII.2016.2619065	journal	February 2017
An Adaptive Machine Learning Framework for Behind-the-Meter Load/PV Disaggregation Saeedi, Ramyar; Sadanandan, Sajan K.; Srivastava, Anurag K. IEEE Transactions on Industrial Informatics, Vol. 17, Issue 10 https://doi.org/10.1109/TII.2021.3060898	journal	October 2021
Two-Level Distributed Volt/Var Control Using Aggregated PV Inverters in Distribution Networks Wang, Yu; Zhao, Tianyang; Ju, Chengquan IEEE Transactions on Power Delivery, Vol. 35, Issue 4 https://doi.org/10.1109/TPWRD.2019.2955506	journal	August 2020
An Optimal and Distributed Method for Voltage Regulation in Power Distribution Systems Zhang, Baosen; Lam, Albert Y. S.; Dominguez-Garcia, Alejandro D. IEEE Transactions on Power Systems, Vol. 30, Issue 4 https://doi.org/10.1109/TPWRS.2014.2347281	journal	July 2015
Multi-Timescale Coordinated Voltage/Var Control of High Renewable-Penetrated Distribution Systems Xu, Yan; Dong, Zhao Yang; Zhang, Rui IEEE Transactions on Power Systems, Vol. 32, Issue 6 https://doi.org/10.1109/TPWRS.2017.2669343	journal	November 2017
Local and Distributed Voltage Control Algorithms in Distribution Networks Cavraro, Guido; Carli, Ruggero IEEE Transactions on Power Systems, Vol. 33, Issue 2 https://doi.org/10.1109/TPWRS.2017.2731682	journal	March 2018
Real-Time Coordinated Voltage Control of PV Inverters and Energy Storage for Weak Networks With High PV Penetration Wang, Licheng; Bai, Feifei; Yan, Ruifeng IEEE Transactions on Power Systems, Vol. 33, Issue 3 https://doi.org/10.1109/TPWRS.2018.2789897	journal	May 2018
State Estimation in Power Distribution Systems Based on Ensemble Kalman Filtering Carquex, Come; Rosenberg, Catherine; Bhattacharya, Kankar IEEE Transactions on Power Systems, Vol. 33, Issue 6 https://doi.org/10.1109/TPWRS.2018.2847289	journal	November 2018
A Multi-Agent Deep Reinforcement Learning Based Voltage Regulation Using Coordinated PV Inverters Cao, Di; Hu, Weihao; Zhao, Junbo IEEE Transactions on Power Systems, Vol. 35, Issue 5 https://doi.org/10.1109/TPWRS.2020.3000652	journal	September 2020
Grid Influences From Reactive Power Flow of Photovoltaic Inverters With a Power Factor Specification of One Spring, Andreas; Wirth, Georg; Becker, Gerd IEEE Transactions on Smart Grid, Vol. 7, Issue 3 https://doi.org/10.1109/TSG.2015.2413949	journal	May 2016
Distributed Control of Battery Energy Storage Systems for Voltage Regulation in Distribution Networks With High PV Penetration Zeraati, Mehdi; Hamedani Golshan, Mohamad Esmail; Guerrero, Josep M. IEEE Transactions on Smart Grid, Vol. 9, Issue 4 https://doi.org/10.1109/TSG.2016.2636217	journal	July 2018
Network Partition-Based Zonal Voltage Control for Distribution Networks With Distributed PV Systems Zhao, Bo; Xu, Zhicheng; Xu, Chen IEEE Transactions on Smart Grid, Vol. 9, Issue 5 https://doi.org/10.1109/TSG.2017.2648779	journal	September 2018
Data-Driven Local Control Design for Active Distribution Grids Using Off-Line Optimal Power Flow and Machine Learning Techniques Karagiannopoulos, Stavros; Aristidou, Petros; Hug, Gabriela IEEE Transactions on Smart Grid, Vol. 10, Issue 6 https://doi.org/10.1109/TSG.2019.2905348	journal	November 2019
Dynamic Distribution State Estimation Using Synchrophasor Data Song, Jianhan; Dall'Anese, Emiliano; Simonetto, Andrea IEEE Transactions on Smart Grid, Vol. 11, Issue 1 https://doi.org/10.1109/TSG.2019.2943540	journal	January 2020
Distributed Adaptive Robust Voltage/VAR Control With Network Partition in Active Distribution Networks Li, Peishuai; Zhang, Cuo; Wu, Zaijun IEEE Transactions on Smart Grid, Vol. 11, Issue 3 https://doi.org/10.1109/TSG.2019.2950120	journal	May 2020
Safe Off-Policy Deep Reinforcement Learning Algorithm for Volt-VAR Control in Power Distribution Systems Wang, Wei; Yu, Nanpeng; Gao, Yuanqi IEEE Transactions on Smart Grid, Vol. 11, Issue 4 https://doi.org/10.1109/TSG.2019.2962625	journal	July 2020
Deep Reinforcement Learning Based Volt-VAR Optimization in Smart Distribution Systems Zhang, Ying; Wang, Xinan; Wang, Jianhui IEEE Transactions on Smart Grid, Vol. 12, Issue 1 https://doi.org/10.1109/TSG.2020.3010130	journal	January 2021
Consensus Multi-Agent Reinforcement Learning for Volt-VAR Control in Power Distribution Networks Gao, Yuanqi; Wang, Wei; Yu, Nanpeng IEEE Transactions on Smart Grid, Vol. 12, Issue 4 https://doi.org/10.1109/TSG.2021.3058996	journal	July 2021
Data-Driven Multi-Agent Deep Reinforcement Learning for Distribution System Decentralized Voltage Control With High Penetration of PVs Cao, Di; Zhao, Junbo; Hu, Weihao IEEE Transactions on Smart Grid, Vol. 12, Issue 5 https://doi.org/10.1109/TSG.2021.3072251	journal	September 2021
Optimal Local Reactive Power Control by PV Inverters Weckx, Sam; Driesen, Johan IEEE Transactions on Sustainable Energy, Vol. 7, Issue 4 https://doi.org/10.1109/TSTE.2016.2572162	journal	October 2016
Distribution Voltage Regulation Through Active Power Curtailment With PV Inverters and Solar Generation Forecasts Ghosh, Shibani; Rahman, Saifur; Pipattanasomporn, Manisa IEEE Transactions on Sustainable Energy, Vol. 8, Issue 1 https://doi.org/10.1109/TSTE.2016.2577559	journal	January 2017
Double-Time-Scale Coordinated Voltage Control in Active Distribution Networks Based on MPC Guo, Yifei; Wu, Qiuwei; Gao, Houlei IEEE Transactions on Sustainable Energy, Vol. 11, Issue 1 https://doi.org/10.1109/TSTE.2018.2890621	journal	January 2020
Attention Enabled Multi-Agent DRL for Decentralized Volt-VAR Control of Active Distribution System Using PV Inverters and SVCs Cao, Di; Zhao, Junbo; Hu, Weihao IEEE Transactions on Sustainable Energy, Vol. 12, Issue 3 https://doi.org/10.1109/TSTE.2021.3057090	journal	July 2021
Decentralized Communication Based Two-Tier Volt-Var Control Strategy for Large-Scale Centralized Photovoltaic Power Plant Li, Haixiao; Guo, Ke; Hao, Gaofeng IEEE Transactions on Sustainable Energy, Vol. 13, Issue 1 https://doi.org/10.1109/TSTE.2021.3121536	journal	January 2022
Optimal Reactive Power Compensation in Distribution Networks with Radial and Meshed Structures Using D-STATCOMs: A Mixed-Integer Convex Approach Garrido, Víctor Manuel; Montoya, Oscar Danilo; Medina-Quesada, Ángeles Sensors, Vol. 22, Issue 22 https://doi.org/10.3390/s22228676	journal	November 2022

Similar Records

Deep Reinforcement Learning Enabled Physical-Model-Free Two-Timescale Voltage Control Method for Active Distribution Systems

Journal Article · Wed Sep 15 20:00:00 EDT 2021 · IEEE Transactions on Smart Grid · OSTI ID:1823582

Deep reinforcement learning assisted co-optimization of Volt-VAR grid service in distribution networks

Journal Article · Sun Jun 11 20:00:00 EDT 2023 · Sustainable Energy, Grids and Networks · OSTI ID:2418735

Related Subjects

14 SOLAR ENERGY
24 POWER TRANSMISSION AND DISTRIBUTION
42 ENGINEERING
Distribution Grid
Generative adversarial network
PV inverters
Power loss minimization
Volt-VAR control
Voltage control
multi-agent-soft-actor-critic

Multi-agent voltage control in distribution systems using GAN-DRL-based approach

Citation Formats

References (31)

Similar Records

Related Subjects