Reinforcement Learning for Volt- Var Control: A Novel Two-stage Progressive Training Strategy
- North Carolina State University,Department of Electrical and Computer Engineering,Raleigh,NC; North Carolina State University
- North Carolina State University,Department of Electrical and Computer Engineering,Raleigh,NC
This paper develops a reinforcement learning (RL) approach to solve a cooperative, multi-agent Volt-Var Control (VVC) problem for high solar penetration distribution systems. The ingenuity of our RL method lies in a novel two-stage progressive training strategy that can effectively improve training speed and convergence of the machine learning algorithm. In Stage 1 (individual training), while holding all the other agents inactive, we separately train each agent to obtain its own optimal VVC actions in the action space: fconsume, generate, do-nothingg. In Stage 2 (cooperative training), all agents are trained again coordinatively to share VVC responsibility. Rewards and costs in our RL scheme include (i) a system-level reward (for taking an action), (ii) an agent-level reward (for doing-nothing), and (iii) an agent-level action cost function. This new framework allows rewards to be dynamically allocated to each agent based on their contribution while accounting for the trade-off between control effectiveness and action cost. The proposed methodology is tested and validated in a modified IEEE 123-bus system using realistic PV and load profiles. Simulation results confirm that the proposed approach is robust and computationally efficient; and it achieves desirable volt-var control performance under a wide range of operation conditions.
- Research Organization:
- North Carolina State University
- Sponsoring Organization:
- USDOE Advanced Research Projects Agency - Energy (ARPA-E)
- Contributing Organization:
- North Carolina State University
- DOE Contract Number:
- EE0008770
- OSTI ID:
- 2329462
- Journal Information:
- 2022 IEEE Power & Energy Society General Meeting (PESGM), Journal Name: 2022 IEEE Power & Energy Society General Meeting (PESGM)
- Country of Publication:
- United States
- Language:
- English
A Data-Driven Multi-Agent Autonomous Voltage Control Framework Using Deep Reinforcement Learning
|
journal | November 2020 |
Deep Reinforcement Learning Based Volt-VAR Optimization in Smart Distribution Systems
|
journal | January 2021 |
Two-Stage Volt/Var Control in Active Distribution Networks With Multi-Agent Deep Reinforcement Learning Method
|
journal | July 2021 |
Similar Records
Volt-VAR Optimization in Distribution Networks Using Twin Delayed Deep Reinforcement Learning
Deep Reinforcement Learning Based Volt-VAR Optimization in Smart Distribution Systems