Data-Driven Multi-agent Deep Reinforcement Learning for Distribution System Decentralized Voltage Control with High Penetration of PVs

Journal Article · Fri Apr 09 00:00:00 EDT 2021 · IEEE Transactions on Smart Grid

DOI:https://doi.org/10.1109/tsg.2021.3072251· OSTI ID:1781623

Cao, Di ^[1]; Zhao, Junbo ^[2]; Hu, Weihao ^[1]; Ding, Fei ^[3]; Huang, Qi ^[4]; Chen, Zhe ^[5]; Blaabjerg, Frede ^[5]

---

1. Univ. of Electronic Science and Technology of China, Chengdu (China)
2. Mississippi State Univ., Starkville, MS (United States)
3. National Renewable Energy Lab. (NREL), Golden, CO (United States)
4. Univ. of Electronic Science and Technology of China, Chengdu (China); Chengdu Univ. (China)
5. Aalborg Univ., Aalborg (Denmark)

This paper proposes a novel model-free/data-driven centralized training and decentralized execution multi-agent deep reinforcement learning (MADRL) framework for distribution system voltage control with high penetration of PVs. The proposed MADRL can coordinate both the real and reactive power control of PVs with existing static var compensators and battery storage systems. Unlike the existing DRL-based voltage control methods, our proposed method does not rely on a system model during both the training and execution stages. This is achieved by developing a new interaction scheme between the surrogate modeling of the original system and the multi-agent soft actor critic (MASAC) MADRL algorithm. In particular, the sparse pseudo-Gaussian process with a few-shots of measurements is utilized to construct the surrogate model of the original environment, i.e., power flow model. This is a data-driven process and no model parameters are needed. Furthermore, the MASAC enabled MADRL allows to achieve better scalability by dividing the original system into different voltage control regions with the aid of real and reactive power sensitivities to voltage, where each region is treated as an agent. This also serves as the foundation for the centralized training and decentralized execution, thus significantly reducing the communication requirements as only local measurements are required for control. Comparative results with other alternatives on the IEEE 123-nodes and 342-nodes systems demonstrate the superiority of the proposed method.

• Research Organization: National Renewable Energy Laboratory (NREL), Golden, CO (United States)
• Sponsoring Organization: USDOE Office of Energy Efficiency and Renewable Energy (EERE)
• Grant/Contract Number: AC36-08GO28308
• OSTI ID: 1781623
• Report Number(s): NREL/JA-5D00-79880; MainId:39098; UUID:6d533b6b-55af-4ece-8bb8-b824c2d1146c; MainAdminID:22350
• Journal Information: IEEE Transactions on Smart Grid, Journal Name: IEEE Transactions on Smart Grid Journal Issue: 5 Vol. 12; ISSN 1949-3053
• Publisher: IEEE
• Country of Publication: United States
• Language: English

Related Subjects

14 SOLAR ENERGY
Gaussian process regression
PVs
distribution network
multi-agent deep reinforcement learning
network partition
voltage regulation