Quantum Reinforcement Learning for Volt-VAR Control in Power Distribution Systems

Lin, Ding; Wang, Jianhui; Tseng, Huan-Hsin; Zhao, Tianqiao; Yue, Meng; Yoo, Shinjae

doi:10.1109/oajpe.2025.3634993

Quantum Reinforcement Learning for Volt-VAR Control in Power Distribution Systems

Journal Article · Wed Nov 19 00:00:00 EST 2025 · IEEE Open Access Journal of Power and Energy

DOI:https://doi.org/10.1109/oajpe.2025.3634993· OSTI ID:3011674

^[1]; ^[1]; ^[2]; ^[2]; ^[2]; ^[2]

Southern Methodist Univ., Dallas, TX (United States)
Brookhaven National Laboratory (BNL), Upton, NY (United States)

Volt-VAR control (VVC) is crucial in active distribution networks for optimizing voltage profiles and minimizing network losses. While traditional deep reinforcement learning (DRL) algorithms exhibit promise for VVC, they often require extensive computational resources to handle such a high-dimensional problem. As a potential solution, quantum reinforcement learning (QRL) algorithms integrate the computational capabilities of quantum computing into the DRL framework. However, existing QRL algorithms struggle with complex VVC problems due to the limitations of current quantum hardware. To bridge this gap, this paper proposes an innovative QRL algorithm featuring an end-to-end architecture that integrates a classical autoencoder, variational quantum circuits (VQCs), and classical post-processing layers. This design efficiently compresses high-dimensional grid states, enabling VQCs to leverage quantum advantages while producing multiple control device outputs tailored for VVC tasks. Numerical studies on three representative distribution systems verify the effectiveness and scalability of the proposed QRL algorithm, and demonstrate its enhanced performance over classical approaches with only approximately 1% of the parameters. Additionally, the robustness of our developed algorithm is validated through noisy quantum environments.

View Accepted Manuscript (DOE)

Research Organization:: Brookhaven National Laboratory (BNL), Upton, NY (United States)

Sponsoring Organization:: USDOE Office of Electricity (OE), Advanced Grid Research & Development

Grant/Contract Number:: SC0012704

OSTI ID:: 3011674

Report Number(s):: BNL--229291-2025-JAAM

Journal Information:: IEEE Open Access Journal of Power and Energy, Journal Name: IEEE Open Access Journal of Power and Energy Vol. 12; ISSN 2687-7910

Publisher:: IEEECopyright Statement

Country of Publication:: United States

Language:: English

References (27)

Principal component analysis: Principal component analysis Abdi, Hervé; Williams, Lynne J. Wiley Interdisciplinary Reviews: Computational Statistics, Vol. 2, Issue 4 https://doi.org/10.1002/wics.101	journal	June 2010
An efficient and scalable variational quantum circuits approach for deep reinforcement learning Bar, Niyazi Furkan; Yetis, Hasan; Karakose, Mehmet Quantum Information Processing, Vol. 22, Issue 8 https://doi.org/10.1007/s11128-023-04051-9	journal	August 2023
Hybrid quantum-classical reinforcement learning in latent observation spaces Nagy, Dániel T. R.; Czabán, Csaba; Bakó, Bence Quantum Machine Intelligence, Vol. 7, Issue 2 https://doi.org/10.1007/s42484-025-00306-z	journal	September 2025
Variational quantum circuit learning-enabled robust optimization for AI data center energy control and decarbonization Ajagekar, Akshay; You, Fengqi Advances in Applied Energy, Vol. 14 https://doi.org/10.1016/j.adapen.2024.100179	journal	July 2024
Safe deep reinforcement learning-based constrained optimal control scheme for active distribution networks Kou, Peng; Liang, Deliang; Wang, Chen Applied Energy, Vol. 264 https://doi.org/10.1016/j.apenergy.2020.114772	journal	April 2020
A graph policy network approach for Volt-Var Control in power distribution systems Lee, Xian Yeow; Sarkar, Soumik; Wang, Yubo Applied Energy, Vol. 323 https://doi.org/10.1016/j.apenergy.2022.119530	journal	October 2022
Variational quantum circuit based demand response in buildings leveraging a hybrid quantum-classical strategy Ajagekar, Akshay; You, Fengqi Applied Energy, Vol. 364 https://doi.org/10.1016/j.apenergy.2024.123244	journal	June 2024
Quantum computation in power systems: An overview of recent advances Golestan, S.; Habibi, M. R.; Mousazadeh Mousavi, S. Y. Energy Reports, Vol. 9 https://doi.org/10.1016/j.egyr.2022.11.185	journal	December 2023
Quantum Computation and Quantum Information Nielsen, Michael A.; Chuang, Isaac L. Cambridge University Press https://doi.org/10.1017/cbo9780511976667	book	January 2011
Variational quantum algorithms Cerezo, M.; Arrasmith, Andrew; Babbush, Ryan Nature Reviews Physics, Vol. 3, Issue 9 https://doi.org/10.1038/s42254-021-00348-9	journal	August 2021
Quantum circuit learning Mitarai, K.; Negoro, M.; Kitagawa, M. Physical Review A, Vol. 98, Issue 3 https://doi.org/10.1103/PhysRevA.98.032309	journal	September 2018
Structural vulnerability of the North American power grid Albert, Réka; Albert, István; Nakarado, Gary L. Physical Review E, Vol. 69, Issue 2 https://doi.org/10.1103/PhysRevE.69.025103	journal	February 2004
Variational Quantum Circuits for Deep Reinforcement Learning Chen, Samuel Yen-Chi; Yang, Chao-Han Huck; Qi, Jun IEEE Access, Vol. 8 https://doi.org/10.1109/ACCESS.2020.3010470	journal	January 2020
A Multiagent Quantum Deep Reinforcement Learning Method for Distributed Frequency Control of Islanded Microgrids Yan, Rudai; Wang, Yu; Xu, Yan IEEE Transactions on Control of Network Systems, Vol. 9, Issue 4 https://doi.org/10.1109/TCNS.2022.3140702	journal	December 2022
Coordination Between OLTC and SVC for Voltage Regulation in Unbalanced Distribution System Distributed Generation Daratha, Novalio; Das, Biswarup; Sharma, Jaydev IEEE Transactions on Power Systems, Vol. 29, Issue 1 https://doi.org/10.1109/TPWRS.2013.2280022	journal	January 2014
Model-Free Volt-Var Control Based on Measurement Data Analytics Bagheri, Pooya; Xu, Wilsun IEEE Transactions on Power Systems, Vol. 34, Issue 2 https://doi.org/10.1109/TPWRS.2018.2874543	journal	March 2019
A Fast Quantum Algorithm for Searching the Quasi-Optimal Solutions of Unit Commitment Zheng, Xiaodong; Wang, Jianhui; Yue, Meng IEEE Transactions on Power Systems, Vol. 39, Issue 2 https://doi.org/10.1109/TPWRS.2024.3350382	journal	March 2024
A Framework for Volt-VAR Optimization in Distribution Systems Ahmadi, Hamed; Marti, Jose R.; Dommel, Hermann W. IEEE Transactions on Smart Grid, Vol. 6, Issue 3 https://doi.org/10.1109/TSG.2014.2374613	journal	May 2015
A Decentralized Adaptive Model-Based Real-Time Control for Active Distribution Networks Using Battery Energy Storage Systems Bahramipanah, Maryam; Torregrossa, Dimitri; Cherkaoui, Rachid IEEE Transactions on Smart Grid, Vol. 9, Issue 4 https://doi.org/10.1109/TSG.2016.2631569	journal	July 2018
Designing Reactive Power Control Rules for Smart Inverters Using Support Vector Machines Jalali, Mana; Kekatos, Vassilis; Gatsis, Nikolaos IEEE Transactions on Smart Grid, Vol. 11, Issue 2 https://doi.org/10.1109/TSG.2019.2942850	journal	March 2020
Safe Off-Policy Deep Reinforcement Learning Algorithm for Volt-VAR Control in Power Distribution Systems Wang, Wei; Yu, Nanpeng; Gao, Yuanqi IEEE Transactions on Smart Grid, Vol. 11, Issue 4 https://doi.org/10.1109/TSG.2019.2962625	journal	July 2020
Two-Stage Deep Reinforcement Learning for Inverter-Based Volt-VAR Control in Active Distribution Networks Liu, Haotian; Wu, Wenchuan IEEE Transactions on Smart Grid, Vol. 12, Issue 3 https://doi.org/10.1109/TSG.2020.3041620	journal	May 2021
Consensus Multi-Agent Reinforcement Learning for Volt-VAR Control in Power Distribution Networks Gao, Yuanqi; Wang, Wei; Yu, Nanpeng IEEE Transactions on Smart Grid, Vol. 12, Issue 4 https://doi.org/10.1109/TSG.2021.3058996	journal	July 2021
Reinforcement Learning for Selective Key Applications in Power Systems: Recent Advances and Future Challenges Chen, Xin; Qu, Guannan; Tang, Yujie IEEE Transactions on Smart Grid, Vol. 13, Issue 4 https://doi.org/10.1109/TSG.2022.3154718	journal	July 2022
Three-Stage Inverter-Based Peak Shaving and Volt-VAR Control in Active Distribution Networks Using Online Safe Deep Reinforcement Learning Nguyen, Hoang Tien; Choi, Dae-Hyun IEEE Transactions on Smart Grid, Vol. 13, Issue 4 https://doi.org/10.1109/TSG.2022.3166192	journal	July 2022
Auto-Encoding Variational Bayes Kingma, Diederik P.; Welling, Max arXiv https://doi.org/10.48550/arXiv.1312.6114	preprint	January 2013
Soft Actor-Critic Algorithms and Applications Haarnoja, Tuomas; Zhou, Aurick; Hartikainen, Kristian arXiv https://doi.org/10.48550/arXiv.1812.05905	preprint	January 2018

Similar Records

Reinforcement Learning for Volt- Var Control: A Novel Two-stage Progressive Training Strategy

Conference · Sun Jul 17 00:00:00 EDT 2022 · 2022 IEEE Power & Energy Society General Meeting (PESGM) · OSTI ID:2329462

Deep Reinforcement Learning Based Volt-VAR Optimization in Smart Distribution Systems

Journal Article · Fri Jul 17 00:00:00 EDT 2020 · IEEE Transactions on Smart Grid · OSTI ID:1762475

Deep reinforcement learning assisted co-optimization of Volt-VAR grid service in distribution networks

Journal Article · Sun Jun 11 20:00:00 EDT 2023 · Sustainable Energy, Grids and Networks · OSTI ID:2418735

Related Subjects

97 MATHEMATICS AND COMPUTING

Quantum Reinforcement Learning for Volt-VAR Control in Power Distribution Systems

Citation Formats

References (27)

Similar Records

Related Subjects