Topology-Aware Reinforcement Learning for Voltage Control: Centralized and Decentralized Strategies

Journal Article · Fri Feb 28 00:00:00 EST 2025 · IEEE Transactions on Industry Applications

DOI:https://doi.org/10.1109/TIA.2025.3546598· OSTI ID:2574179

Hossain, Rakib ^[1]; Gautam, Mukesh ^[2]; MansourLakouraj, Mohammad ^[3]; Livani, Hanif ^[3]; Benidris, Mohammed ^[4]

---

1. Univ. of Nevada, Reno, NV (United States); Pacific Northwest National Laboratory (PNNL), Richland, WA (United States)
2. Pacific Northwest National Laboratory (PNNL), Richland, WA (United States)
3. Univ. of Nevada, Reno, NV (United States)
4. Michigan State Univ., East Lansing, MI (United States)

Volt-VAR control (VVC) methods based on deep reinforcement learning (DRL) can effectively control distribution grid voltage and minimize power loss by implementing corrective and preventive control measures on the reactive power output of inverter-based distributed energy resources (DERs). However, model-free DRL-based VVC approaches usually cannot capture the important topological feature of the power system since they use a fully-connected network (FCN) to deliver the action. Therefore, this paper proposes a graph convolutional network (GCN)-based DRL approach that can employ the topological information of the network to take better control action for regulating the voltage. Our implementation allows for both centralized and decentralized configurations, utilizing a single agent and multiple agents respectively. Although the centralized GCN-based DRL approach has its advantages of minimizing voltage fluctuation and power loss, it is not suitable for large scale power systems due to its challenges in terms of scalability, computation speed and potential single points of failure. Therefore, these problems can be resolved using the decentralized GCN-based DRL approach. Moreover, to ensure the safe operation of the model, our proposed approach incorporates an exponential barrier function while formulating the reward function for each agent. To validate performance of the proposed approaches, the proposed model is tested on modified IEEE test systems and the performances are measured in terms on voltage fluctuation reduction, minimization of power loss and computational speed. Finally, the results show that the proposed topology-aware approach outperforms the FCN-based DRL approach in terms of reducing voltage fluctuation and minimizing power loss of the network. Moreover, it is shown that the decentralized GCN-based DRL has faster computational speed than other approaches.

• Research Organization: Pacific Northwest National Laboratory (PNNL), Richland, WA (United States)
• Sponsoring Organization: USDOE Office of Energy Efficiency and Renewable Energy (EERE), Renewable Power Office. Solar Energy Technologies Office
• Grant/Contract Number: EE0009022
• OSTI ID: 2574179
• Report Number(s): PNNL-SA--195187
• Journal Information: IEEE Transactions on Industry Applications, Journal Name: IEEE Transactions on Industry Applications Journal Issue: 4 Vol. 61; ISSN 0093-9994; ISSN 1939-9367
• Publisher: IEEE
• Country of Publication: United States
• Language: English

Related Subjects

Volt-VAR control
centralized and decentralized regulations
distribution grids
graph convolutional network
inverters
multi-agent-soft actor-critic
soft actor-critic