Skip to main content
OSTI.GOVU.S. Department of Energy Office of Scientific and Technical Information
U.S. Department of Energy
Office of Scientific and Technical Information
 

Visibility-enhanced model-free deep reinforcement learning algorithm for voltage control in realistic distribution systems using smart inverters

Journal Article · · Applied Energy
 [1];  [1];  [1];  [2];  [1];  [2]
  1. Univ. of Connecticut, Storrs, CT (United States)
  2. National Renewable Energy Laboratory (NREL), Golden, CO (United States)
+ Show Author Affiliations
Increasing integration of distributed solar photovoltaic (PV) into distribution networks could result in adverse effects on grid operation. Traditional model-based control algorithms require accurate model information that is difficult to acquire and thus are challenging to implement in practice. Here, this paper proposes a surrogate model-enabled grid visibility scheme to empower deep reinforcement learning (DRL) approach for distribution network voltage regulation using PV inverters with minimal system knowledge. In contrast to existing DRL methods, this paper presents and corroborates the adverse impact of missing load information on DRL performance and, based on this finding, proposes a surrogate model methodology to impute load information utilizing observable data. Additionally, a multi-fidelity neural network is utilized to construct the DRL training environment, chosen for its efficient data utilization and enhanced robustness to data uncertainty. The feasibility and effectiveness of the proposed algorithm are assessed by considering DRL testing across varying degrees of observable load information and diverse training environments on a realistic power system.
Research Organization:
National Renewable Energy Laboratory (NREL), Golden, CO (United States)
Sponsoring Organization:
USDOE; USDOE Office of Energy Efficiency and Renewable Energy (EERE), Renewable Power Office. Solar Energy Technologies Office
Grant/Contract Number:
AC36-08GO28308
OSTI ID:
2426909
Alternate ID(s):
OSTI ID: 2396040
Report Number(s):
NREL/JA--5D00-85937; MainId:86710; UUID:1118bbf9-9b57-4940-b40f-662d3caf6662; MainAdminId:73295
Journal Information:
Applied Energy, Journal Name: Applied Energy Vol. 372; ISSN 0306-2619
Publisher:
ElsevierCopyright Statement
Country of Publication:
United States
Language:
English

References (23)

Model-free voltage control of active distribution system with PVs using surrogate model-based deep reinforcement learning journal January 2022
Optimal Volt–Var Curve Setting of a Smart Inverter for Improving Its Performance in a Distribution System journal January 2020
Voltage-Sensitivity-Based Volt-VAR-Watt Settings of Smart Inverters for Mitigating Voltage Rise in Distribution Systems journal January 2021
Improving distribution network PV hosting capacity via smart inverter reactive power support conference July 2015
Technologies to increase PV hosting capacity in distribution feeders conference July 2016
Multi-Fidelity Learning for Distribution System Voltage Probabilistic Analysis with High Penetration of PVs conference July 2022
Locational sensitivity investigation on PV hosting capacity and fast track PV screening conference May 2016
DC Power Flow Revisited journal August 2009
Impact of increased penetration of photovoltaic generation on power systems journal May 2013
Multi-Timescale Coordinated Voltage/Var Control of High Renewable-Penetrated Distribution Systems journal November 2017
Linear Decision Rules for Control of Reactive Power by Distributed Photovoltaic Generators journal March 2018
Network Partition and Voltage Coordination Control for Distribution Networks With High Penetration of Distributed PV Units journal May 2018
Deep-Reinforcement-Learning-Based Autonomous Voltage Control for Power Grid Operations journal January 2020
Graph Convolutional Network-Based Topology Embedded Deep Reinforcement Learning for Voltage Stability Control journal September 2021
Designing Reactive Power Control Rules for Smart Inverters Using Support Vector Machines journal March 2020
Two-Timescale Voltage Control in Distribution Grids Using Deep Reinforcement Learning journal May 2020
Safe Off-Policy Deep Reinforcement Learning Algorithm for Volt-VAR Control in Power Distribution Systems journal July 2020
Deep Reinforcement Learning Based Volt-VAR Optimization in Smart Distribution Systems journal January 2021
Data-Driven Multi-Agent Deep Reinforcement Learning for Distribution System Decentralized Voltage Control With High Penetration of PVs journal September 2021
Attention Enabled Multi-Agent DRL for Decentralized Volt-VAR Control of Active Distribution System Using PV Inverters and SVCs journal July 2021
Data-Driven Distribution System Coordinated PV Inverter Control Using Deep Reinforcement Learning conference December 2021
Survey of Multifidelity Methods in Uncertainty Propagation, Inference, and Optimization journal January 2018
Issues in Deciding Whether to Use Multifidelity Surrogates journal May 2019

Figures / Tables (15)

Fig. 1(p. 2)figure Fig. 1
Fig. 2(p. 3)figure Fig. 2
Fig. 3(p. 5)figure Fig. 3
Fig. 4(p. 7)figure Fig. 4
Fig. 5(p. 7)figure Fig. 5
TABLE II(p. 7)table TABLE II
Table I(p. 7)table Table I
Fig. 6(p. 8)figure Fig. 6
Fig. 7(p. 8)figure Fig. 7
TABLE III(p. 8)table TABLE III
Table IV(p. 8)table Table IV
Table V(p. 9)table Table V
Fig. 8(p. 10)figure Fig. 8
Fig. 9(p. 10)figure Fig. 9
TABLE VI(p. 10)table TABLE VI

Similar Records

Efficient learning of power grid voltage control strategies via model-based deep reinforcement learning
Journal Article · Sun Nov 05 19:00:00 EST 2023 · Machine Learning · OSTI ID:2345752
Multi-agent voltage control in distribution systems using GAN-DRL-based approach
Journal Article · Thu Jun 06 20:00:00 EDT 2024 · Electric Power Systems Research · OSTI ID:2403534

Related Subjects

14 SOLAR ENERGY
97 MATHEMATICS AND COMPUTING
PV inverter
active distribution systems
grid visibility
reinforcement learning
surrogate model
voltage regulation