Multi-Agent Graph-Attention Deep Reinforcement Learning for Post-Contingency Grid Emergency Voltage Control

Zhang, Ying; Yue, Meng; Wang, Jianhui; Yoo, Shinjae

doi:10.1109/tnnls.2023.3341334

Multi-Agent Graph-Attention Deep Reinforcement Learning for Post-Contingency Grid Emergency Voltage Control

Journal Article · Thu Jan 25 00:00:00 EST 2024 · IEEE Transactions on Neural Networks and Learning Systems

DOI:https://doi.org/10.1109/tnnls.2023.3341334· OSTI ID:2433976

^[1]; ^[2]; ^[3]; ^[2]

Oklahoma State Univ., Stillwater, OK (United States)
Brookhaven National Laboratory (BNL), Upton, NY (United States)
Southern Methodist Univ., Dallas, TX (United States)

Grid emergency voltage control (GEVC) is paramount in electric power systems to improve voltage stability and prevent cascading outages and blackouts in case of contingencies. While most deep reinforcement learning (DRL)-based paradigms perform single agents in a static environment, real-world agents for GEVC are expected to cooperate in a dynamically shifting grid. Moreover, due to high uncertainties from combinatory natures of various contingencies and load consumption, along with the complexity of dynamic grid operation, the data efficiency and control performance of the existing DRL-based methods are challenged. To address these limitations, we propose a multi-agent graph-attention (GATT)-based DRL algorithm for GEVC in multi-area power systems. Here, we develop graph convolutional network (GCN)-based agents for feature representation of the graph-structured voltages to improve the decision accuracy in a data-efficient manner. Furthermore, a cutting-edge attention mechanism concentrates on effective information sharing among multiple agents, synergizing different-sized subnetworks in the grid for cooperative learning. We address several key challenges in the existing DRL-based GEVC approaches, including low scalability and poor stability against high uncertainties. Test results in the IEEE benchmark system verify the advantages of the proposed method over several recent multi-agent DRL-based algorithms.

View Accepted Manuscript (DOE)

Research Organization:: Brookhaven National Laboratory (BNL), Upton, NY (United States)

Sponsoring Organization:: USDOE Office of Electricity (OE), Advanced Grid Research & Development. Power Systems Engineering Research

Grant/Contract Number:: SC0012704; 36291

OSTI ID:: 2433976

Report Number(s):: BNL-225967-2024-JAAM

Journal Information:: IEEE Transactions on Neural Networks and Learning Systems, Vol. 35, Issue 3; ISSN 2162-237X

Publisher:: IEEE Computational Intelligence SocietyCopyright Statement

Country of Publication:: United States

Language:: English

References (10)

Emergency Voltage Stability Controls: an Overview Van Cutsem, T.; Vournas, C. D. 2007 IEEE Power Engineering Society General Meeting https://doi.org/10.1109/pes.2007.386089	conference	June 2007
Toward a Reinforcement Learning Environment Toolbox for Intelligent Electric Motor Control Traue, Arne; Book, Gerrit; Kirchgassner, Wilhelm IEEE Transactions on Neural Networks and Learning Systems, Vol. 33, Issue 3 https://doi.org/10.1109/tnnls.2020.3029573	journal	March 2022
A critical review of the state‐of‐art schemes for under voltage load shedding Larik, Raja Masood; Mustafa, Mohd Wazir; Aman, Muhammad Naveed International Transactions on Electrical Energy Systems, Vol. 29, Issue 5 https://doi.org/10.1002/2050-7038.2828	journal	January 2019
Voltage instability in interconnected power systems: a simulation approach Lachs, W. R.; Sutanto, D. IEEE Transactions on Power Systems, Vol. 7, Issue 2 https://doi.org/10.1109/59.141782	journal	May 1992
Load Shedding Scheme with Deep Reinforcement Learning to Improve Short-term Voltage Stability Zhang, Jingyi; Lu, Chao; Fang, Chen 2018 IEEE Innovative Smart Grid Technologies - Asia (ISGT Asia) https://doi.org/10.1109/ISGT-Asia.2018.8467877	conference	May 2018
Adaptive Load Shedding for Grid Emergency Control via Deep Reinforcement Learning Zhang, Ying; Yue, Meng; Wang, Jianhui 2021 IEEE Power & Energy Society General Meeting (PESGM) https://doi.org/10.1109/PESGM46819.2021.9638058	conference	July 2021
Leader-to-Formation Stability Tanner, H. G.; Pappas, G. J.; Kumar, V. IEEE Transactions on Robotics and Automation, Vol. 20, Issue 3 https://doi.org/10.1109/TRA.2004.825275	journal	June 2004
A Multi-agent approach to coordination of different emergency control devices against voltage collapse Panasetsky, D. A.; Voropai, N. I. 2009 IEEE Bucharest PowerTech https://doi.org/10.1109/ptc.2009.5281995	conference	June 2009
Leader–follower formation control of nonholonomic mobile robots with input constraints Consolini, Luca; Morbidi, Fabio; Prattichizzo, Domenico Automatica, Vol. 44, Issue 5 https://doi.org/10.1016/j.automatica.2007.09.019	journal	May 2008
Adaptive Coordinated Voltage Control—Part I: Basic Scheme Ma, Haomin; Hill, David J. IEEE Transactions on Power Systems, Vol. 29, Issue 4 https://doi.org/10.1109/tpwrs.2013.2293577	journal	July 2014

Similar Records

Off-policy deep reinforcement learning with automatic entropy adjustment for adaptive online grid emergency control

Journal Article · Wed Jan 25 00:00:00 EST 2023 · Electric Power Systems Research · OSTI ID:1992867

Zhang, Ying; Yue, Meng; Wang, Jianhui

Efficient learning of power grid voltage control strategies via model-based deep reinforcement learning

Journal Article · Mon Nov 06 00:00:00 EST 2023 · Machine Learning · OSTI ID:2345752

Hossain, Ramij Raja; Yin, Tianzhixi; Du, Yan; +5 more

Graph Convolutional Network-Based Topology Embedded Deep Reinforcement Learning for Voltage Stability Control

Journal Article · Wed Sep 01 00:00:00 EDT 2021 · IEEE Transactions on Power Systems · OSTI ID:1824163

Hossain, Ramij Raja; Huang, Qiuhua; Huang, Renke

Related Subjects

24 POWER TRANSMISSION AND DISTRIBUTION

Multi-Agent Graph-Attention Deep Reinforcement Learning for Post-Contingency Grid Emergency Voltage Control

Citation Formats

References (10)

Similar Records

Related Subjects