A Reinforcement Learning Approach to Parameter Selection for Distributed Optimal Power Flow

Conference · Tue Nov 01 04:00:00 EDT 2022

DOI:https://doi.org/10.1016/j.epsr.2022.108546· OSTI ID:2324947

Zeng, Sihan; Molzahn, Daniel K.; Kody, Alyssa A; Kim, Youngdae; Kim, Kibaek

With the increasing penetration of distributed energy resources, distributed optimization algorithms have attracted significant attention for power systems applications due to their potential for superior scalability, privacy, and robustness to a single point-of-failure. The Alternating Direction Method of Multipliers (ADMM) is a popular distributed optimization algorithm; however, its convergence performance is highly dependent on the selection of penalty parameters, which are usually chosen heuristically. In this work, we use reinforcement learning (RL) to develop an adaptive penalty parameter selection policy for alternating current optimal power flow (ACOPF) problem solved via ADMM with the goal of minimizing the number of iterations until convergence. We train our RL policy using deep Q-learning and show that this policy can result in significantly accelerated convergence (up to a 59% reduction in the number of iterations compared to existing, curvatureinformed penalty parameter selection methods). Furthermore, we show that our RL policy demonstrates promise for generalizability, performing well under unseen loading schemes as well as under unseen losses of lines and generators (up to a 50% reduction in iterations). This work thus provides a proof-of-concept for using RL for parameter selection in ADMM for power systems applications.

• Research Organization: Argonne National Laboratory (ANL)
• Sponsoring Organization: Argonne National Laboratory - Laboratory Directed Research and Development (LDRD)
• DOE Contract Number: AC02-06CH11357
• OSTI ID: 2324947
• Country of Publication: United States
• Language: English

Related Subjects

alternating current optimal power flow
alternating direction method of multipliers
deep Q-learning
distributed optimization
reinforcement learning