PowerGridworld: A Framework for Multi-Agent Reinforcement Learning in Power Systems
We present the PowerGridworld open source software package to provide users with a lightweight, modular, and customizable framework for creating power-systems-focused, multi-agent Gym environments that readily integrate with existing training frameworks for reinforcement learning (RL). Although many frameworks exist for training multi-agent RL (MARL) policies, none can rapidly prototype and develop the environments themselves, especially in the context of heterogeneous (composite, multi-device) power systems where power flow solutions are required to define grid-level variables and costs. PowerGridworld helps to fill this gap. To highlight PowerGridworld's key features, we present two case studies and demonstrate learning MARL policies using both OpenAI's multi-agent deep deterministic policy gradient (MADDPG) and RL-Lib's proximal policy optimization (PPO) algorithms. In both cases, at least some subset of agents incorporates elements of the power flow solution at each time step as part of their reward (negative cost) structures.
- Research Organization:
- National Renewable Energy Lab. (NREL), Golden, CO (United States)
- Sponsoring Organization:
- USDOE National Renewable Energy Laboratory (NREL), Laboratory Directed Research and Development (LDRD) Program
- DOE Contract Number:
- AC36-08GO28308
- OSTI ID:
- 1881415
- Report Number(s):
- NREL/CP-2C00-83699; MainId:84472; UUID:a7cadee3-1dc8-41e3-8eef-85264f27f971; MainAdminID:65091
- Resource Relation:
- Conference: Presented at the Thirteenth ACM International Conference on Future Energy Systems, 28 June - 1 July 2022
- Country of Publication:
- United States
- Language:
- English
Similar Records
PowerGridworld: A Framework for Multi-Agent Reinforcement Learning in Power Systems [SWR-22-07]
Hybrid-RL-MPC4CLR (Hybird-Reinforcement-Learning-Model-Predictive-Control-for-Reserve-Policy-Assisted-Critical-Load-Restoration-in-Distribution-Grids)