Learning Sequential Distribution System Restoration via Graph-Reinforcement Learning

Zhao, Tianqiao; Wang, Jianhui

doi:10.1109/tpwrs.2021.3102870

Title: Learning Sequential Distribution System Restoration via Graph-Reinforcement Learning

Journal Article · 05 August 2021 · IEEE Transactions on Power Systems

DOI: https://doi.org/10.1109/tpwrs.2021.3102870 · OSTI ID:1868518

^[1];

^[2]

Brookhaven National Laboratory (BNL), Upton, NY (United States)
Southern Methodist University, Dallas, TX (United States)

We report a distribution service restoration algorithm as a fundamental resilient paradigm for system operators provides an optimally coordinated, resilient solution to enhance the restoration performance. The restoration problem is formulated to coordinate distribution generators and controllable switches optimally. A model-based control scheme is usually designed to solve this problem, relying on a precise model and resulting in low scalability. To tackle these limitations, this work proposes a graph-reinforcement learning framework for the restoration problem. We link the power system topology with a graph convolutional network, which captures the complex mechanism of network restoration in power networks and understands the mutual interactions among controllable devices. Latent features over graphical power networks produced by graph convolutional layers are exploited to learn the control policy for network restoration using deep reinforcement learning. The solution scalability is guaranteed by modeling distributed generators as agents in a multi-agent environment and a proper pre-training paradigm. Comparative studies on IEEE 123-node and 8500-node test systems demonstrate the performance of the proposed solution.

View Accepted Manuscript (DOE)

Cite

Export

Save

Research Organization:: Brookhaven National Laboratory (BNL), Upton, NY (United States)

Sponsoring Organization:: USDOE Office of Energy Efficiency and Renewable Energy (EERE), Renewable Power Office. Solar Energy Technologies Office

Grant/Contract Number:: SC0012704

OSTI ID:: 1868518

Report Number(s):: BNL-222934-2022-JAAM

Journal Information:: IEEE Transactions on Power Systems, Journal Name: IEEE Transactions on Power Systems Journal Issue: 2 Vol. 37; ISSN 0885-8950

Publisher:: IEEECopyright Statement

Country of Publication:: United States

Language:: English

References (29)

Neural Fitted Q Iteration – First Experiences with a Data Efficient Neural Reinforcement Learning Method Riedmiller, Martin Machine Learning: ECML 2005 https://doi.org/10.1007/11564096_32	book	January 2005
Target transfer Q-learning and its convergence analysis Wang, Yue; Liu, Yuting; Chen, Wei Neurocomputing, Vol. 392 https://doi.org/10.1016/j.neucom.2020.02.117	journal	June 2020
Mastering the game of Go with deep neural networks and tree search Silver, David; Huang, Aja; Maddison, Chris J. Nature, Vol. 529, Issue 7587 https://doi.org/10.1038/nature16961	journal	January 2016
Comparative study of modern heuristic algorithms to service restoration in distribution systems Toune, S.; Fudo, H.; Genji, T. IEEE Transactions on Power Delivery, Vol. 17, Issue 1 https://doi.org/10.1109/61.974205	journal	January 2002
Heuristic Search Approach to Distribution System Restoration Morelato, A. L.; Monticelli, A. IEEE Power Engineering Review, Vol. 9, Issue 10 https://doi.org/10.1109/MPER.1989.4310339	journal	January 1989
Resilient Load Restoration in Microgrids Considering Mobile Energy Storage Fleets: A Deep Reinforcement Learning Approach Yao, Shuhan; Gu, Jiuxiang; Zhang, Huajun 2020 IEEE Power & Energy Society General Meeting (PESGM) https://doi.org/10.1109/PESGM41954.2020.9282132	conference	August 2020
A Resilience-Based Architecture for Joint Distributed Energy Resources Allocation and Hourly Network Reconfiguration Kianmehr, Ehsan; Nikkhah, Saman; Vahidinasab, Vahid IEEE Transactions on Industrial Informatics, Vol. 15, Issue 10 https://doi.org/10.1109/TII.2019.2901538	journal	October 2019
A Comprehensive Survey on Graph Neural Networks Wu, Zonghan; Pan, Shirui; Chen, Fengwen IEEE Transactions on Neural Networks and Learning Systems, Vol. 32, Issue 1 https://doi.org/10.1109/TNNLS.2020.2978386	journal	January 2021
Toward Optimal Multiperiod Network Reconfiguration for Increasing the Hosting Capacity of Distribution Networks Fu, Yang-Yang; Chiang, Hsiao-Dong IEEE Transactions on Power Delivery, Vol. 33, Issue 5 https://doi.org/10.1109/TPWRD.2018.2801332	journal	October 2018
Optimized Restoration of Unbalanced Distribution Systems Khushalani, Sarika; Solanki, Jignesh M.; Schulz, Noel N. IEEE Transactions on Power Systems, Vol. 22, Issue 2 https://doi.org/10.1109/TPWRS.2007.894866	journal	May 2007
Distribution System Restoration With Microgrids Using Spanning Tree Search Li, Juan; Ma, Xi-Yuan; Liu, Chen-Ching IEEE Transactions on Power Systems, Vol. 29, Issue 6 https://doi.org/10.1109/TPWRS.2014.2312424	journal	November 2014
Self-Healing Resilient Distribution Systems Based on Sectionalization Into Microgrids Wang, Zhaoyu; Wang, Jianhui IEEE Transactions on Power Systems, Vol. 30, Issue 6 https://doi.org/10.1109/TPWRS.2015.2389753	journal	November 2015
Value of Distribution Network Reconfiguration in Presence of Renewable Energy Resources Dorostkar-Ghamsari, Mohammad Reza; Fotuhi-Firuzabad, Mahmud; Lehtonen, Matti IEEE Transactions on Power Systems, Vol. 31, Issue 3 https://doi.org/10.1109/TPWRS.2015.2457954	journal	May 2016
Sequential Service Restoration for Unbalanced Distribution Systems and Microgrids Chen, Bo; Chen, Chen; Wang, Jianhui IEEE Transactions on Power Systems, Vol. 33, Issue 2 https://doi.org/10.1109/TPWRS.2017.2720122	journal	March 2018
Reinforcement Learning Approach for Optimal Distributed Energy Management in a Microgrid Foruzan, Elham; Soh, Leen-Kiat; Asgarpoor, Sohrab IEEE Transactions on Power Systems, Vol. 33, Issue 5 https://doi.org/10.1109/TPWRS.2018.2823641	journal	September 2018
Pandapower—An Open-Source Python Tool for Convenient Modeling, Analysis, and Optimization of Electric Power Systems Thurner, Leon; Scheidler, Alexander; Schafer, Florian IEEE Transactions on Power Systems, Vol. 33, Issue 6 https://doi.org/10.1109/TPWRS.2018.2829021	journal	November 2018
Critical Load Restoration Using Distributed Energy Resources for Resilient Power Distribution System Poudel, Shiva; Dubey, Anamika IEEE Transactions on Power Systems, Vol. 34, Issue 1 https://doi.org/10.1109/TPWRS.2018.2860256	journal	January 2019
Toward a MILP Modeling Framework for Distribution System Restoration Chen, Bo; Ye, Zhigang; Chen, Chen IEEE Transactions on Power Systems, Vol. 34, Issue 3 https://doi.org/10.1109/TPWRS.2018.2885322	journal	May 2019
Toward a Synthetic Model for Distribution System Restoration and Crew Dispatch Chen, Bo; Ye, Zhigang; Chen, Chen IEEE Transactions on Power Systems, Vol. 34, Issue 3 https://doi.org/10.1109/TPWRS.2018.2885763	journal	May 2019
Coordinated Restoration of Transmission and Distribution System Using Decentralized Scheme Zhao, Jin; Wang, Hongtao; Liu, Yutian IEEE Transactions on Power Systems, Vol. 34, Issue 5 https://doi.org/10.1109/TPWRS.2019.2908449	journal	September 2019
From Optimization-Based Machine Learning to Interpretable Security Rules for Operation Cremer, Jochen L.; Konstantelos, Ioannis; Strbac, Goran IEEE Transactions on Power Systems, Vol. 34, Issue 5 https://doi.org/10.1109/TPWRS.2019.2911598	journal	September 2019
Resilient Distribution System by Microgrids Formation After Natural Disasters Chen, Chen; Wang, Jianhui; Qiu, Feng IEEE Transactions on Smart Grid, Vol. 7, Issue 2 https://doi.org/10.1109/TSG.2015.2429653	journal	March 2016
Fault Detection, Isolation, and Service Restoration in Distribution Systems: State-of-the-Art and Future Trends Zidan, Aboelsood; Khairalla, Mutaz; Abdrabou, Ahmed M. IEEE Transactions on Smart Grid, Vol. 8, Issue 5 https://doi.org/10.1109/TSG.2016.2517620	journal	September 2017
Multi-Time Step Service Restoration for Advanced Distribution Systems and Microgrids Chen, Bo; Chen, Chen; Wang, Jianhui IEEE Transactions on Smart Grid, Vol. 9, Issue 6 https://doi.org/10.1109/TSG.2017.2723798	journal	November 2018
DGs for Service Restoration to Critical Loads in a Secondary Network Xu, Yin; Liu, Chen-Ching; Wang, Zhiwen IEEE Transactions on Smart Grid, Vol. 10, Issue 1 https://doi.org/10.1109/TSG.2017.2743158	journal	January 2019
Deep Reinforcement Learning for Joint Bidding and Pricing of Load Serving Entity Xu, Hanchen; Sun, Hongbo; Nikovski, Daniel IEEE Transactions on Smart Grid, Vol. 10, Issue 6 https://doi.org/10.1109/TSG.2019.2903756	journal	November 2019
Batch-Constrained Reinforcement Learning for Dynamic Distribution Network Reconfiguration Gao, Yuanqi; Wang, Wei; Shi, Jie IEEE Transactions on Smart Grid, Vol. 11, Issue 6 https://doi.org/10.1109/TSG.2020.3005270	journal	November 2020
Multi-Agent Safe Policy Learning for Power Management of Networked Microgrids Zhang, Qianzhi; Dehghanpour, Kaveh; Wang, Zhaoyu IEEE Transactions on Smart Grid, Vol. 12, Issue 2 https://doi.org/10.1109/TSG.2020.3034827	journal	March 2021
Curriculum learning Bengio, Yoshua; Louradour, Jérôme; Collobert, Ronan Proceedings of the 26th Annual International Conference on Machine Learning - ICML '09 https://doi.org/10.1145/1553374.1553380	conference	January 2009

Similar Records

Multi-Agent Reinforcement Learning for Distribution System Critical Load Restoration: Preprint

Conference · 2023 · OSTI ID:1992817

Yao, Yiyun; Zhang, Xiangyu; Wang, Jiyu; +1 more

Multi-Agent Reinforcement Learning for Distribution System Critical Load Restoration

Conference · 2023 · OSTI ID:2229753

Yao, Yiyun; Zhang, Xiangyu; Wang, Jiyu; +1 more

Hybrid Imitation Learning for Real-Time Service Restoration in Resilient Distribution Systems

Journal Article · 2021 · IEEE Transactions on Industrial Informatics · OSTI ID:1960294

Zhang, Yichen; Qiu, Feng; Hong, Tianqi; +2 more

Related Subjects

24 POWER TRANSMISSION AND DISTRIBUTION
distributed generation
distribution system restoration
graph convolutional networks
graph reinforcement learning

Title: Learning Sequential Distribution System Restoration via Graph-Reinforcement Learning

Citation Formats

References (29)

Similar Records

Related Subjects