Deep Reinforcement Learning for Microgrid Cost Optimization Considering Load Flexibility

Pei, Yansong; Yao, Yiyun; Zhao, Junbo; Ding, Fei; Wang, Jiyu

doi:10.1109/PESGM51994.2024.10688837

Deep Reinforcement Learning for Microgrid Cost Optimization Considering Load Flexibility

Conference · Fri Oct 04 04:00:00 EDT 2024

DOI:https://doi.org/10.1109/PESGM51994.2024.10688837· OSTI ID:2477396

Pei, Yansong; ; Zhao, Junbo; Ding, Fei; Wang, Jiyu

This paper proposes a novel Soft-Actor-Critic (SAC) based Deep Reinforcement Learning (DRL) method for optimizing the cost of microgrid operation by leveraging load flexibility. The proposed SAC-DRL method is designed to coordinate the control of distributed energy resources (DERs) and flexible load, addressing practical energy billing formation by power distribution utilities. Key contributions include an innovative reward function to mitigate sparse reward challenges and a mixed control strategy for discrete and continuous variables, ensuring radial network topology and minimizing power loss. We evaluate the proposed method on the model of a real microgrid located in Southern California, U.S.. The SAC-DRL model is tested to demonstrate its efficacy in reducing grid dependence, optimizing resource use, and minimizing costs. The results highlight the potential of DRL in modern energy systems, offering a sustainable and economically efficient solution for energy management in microgrids.

Research Organization:: National Renewable Energy Laboratory (NREL), Golden, CO (United States)

Sponsoring Organization:: USDOE National Renewable Energy Laboratory (NREL)

DOE Contract Number:: AC36-08GO28308

OSTI ID:: 2477396

Report Number(s):: NREL/CP-5D00-92053; MainId:93831; UUID:443fa518-197c-48d3-a530-8ff4b0b5d392; MainAdminId:74239

Country of Publication:: United States

Language:: English

References (10)

Improving Distribution System Resiliency using Distributed Energy Resources Santoni, Ricardo; Chandwani, Ashwin; Shah, Apoorva K. 2020 52nd North American Power Symposium (NAPS) https://doi.org/10.1109/NAPS50074.2021.9449759	conference	April 2021
Market-Oriented Energy Management of a Hybrid Wind-Battery Energy Storage System Via Model Predictive Control With Constraint Optimizer Abdeltawab, Hussein Hassan; Mohamed, Yasser Abdel-Rady I. IEEE Transactions on Industrial Electronics, Vol. 62, Issue 11 https://doi.org/10.1109/TIE.2015.2435694	journal	November 2015
Multi-Agent Deep Reinforcement Learning for HVAC Control in Commercial Buildings Yu, Liang; Sun, Yi; Xu, Zhanbo IEEE Transactions on Smart Grid, Vol. 12, Issue 1 https://doi.org/10.1109/TSG.2020.3011739	journal	January 2021
On-Line Building Energy Optimization Using Deep Reinforcement Learning Mocanu, Elena; Mocanu, Decebal Constantin; Nguyen, Phuong H. IEEE Transactions on Smart Grid, Vol. 10, Issue 4 https://doi.org/10.1109/TSG.2018.2834219	journal	July 2019
Optimal Demand Response Using Device-Based Reinforcement Learning Wen, Zheng; O'Neill, Daniel; Maei, Hamid IEEE Transactions on Smart Grid, Vol. 6, Issue 5 https://doi.org/10.1109/TSG.2015.2396993	journal	September 2015
Fuzzy Prediction Interval Models for Forecasting Renewable Resources and Loads in Microgrids Saez, Doris; Avila, Fernand; Olivares, Daniel IEEE Transactions on Smart Grid, Vol. 6, Issue 2 https://doi.org/10.1109/TSG.2014.2377178	journal	March 2015
Unleash Values From Grid-Edge Flexibility: An overview, experience, and vision for leveraging grid-edge distributed energy resources to improve grid operations Ding, Fei; Liu, Weijia; Kumar, Utkarsh IEEE Electrification Magazine, Vol. 10, Issue 4 https://doi.org/10.1109/MELE.2022.3211017	journal	December 2022
Privacy Preserving Load Control of Residential Microgrid via Deep Reinforcement Learning Qin, Zhaoming; Liu, Di; Hua, Haochen IEEE Transactions on Smart Grid, Vol. 12, Issue 5 https://doi.org/10.1109/TSG.2021.3088290	journal	September 2021
A Microgrid Energy Management System Based on the Rolling Horizon Strategy Palma-Behnke, Rodrigo; Benavides, Carlos; Lanas, Fernando IEEE Transactions on Smart Grid, Vol. 4, Issue 2 https://doi.org/10.1109/TSG.2012.2231440	journal	June 2013
Model-Free Real-Time Autonomous Control for a Residential Multi-Energy System Using Deep Reinforcement Learning Ye, Yujian; Qiu, Dawei; Wu, Xiaodong IEEE Transactions on Smart Grid, Vol. 11, Issue 4 https://doi.org/10.1109/TSG.2020.2976771	journal	July 2020

Similar Records

Multi-Agent Hierarchical Deep Reinforcement Learning for HVAC Control With Flexible DERs

Journal Article · Tue Aug 12 00:00:00 EDT 2025 · IEEE Transactions on Smart Grid · OSTI ID:2587057

Flexible dynamic boundary microgrid operation considering network and load unbalances

Journal Article · Sat Jun 15 00:00:00 EDT 2024 · Applied Energy · OSTI ID:2376386

Federated Deep Reinforcement Learning for Decentralized VVO of BTM DERs

Conference · Tue Oct 01 00:00:00 EDT 2024 · OSTI ID:2477510

Related Subjects

MATHEMATICS AND COMPUTING
deep reinforcement learning
microgrid
peak load management
voltage regulation

Deep Reinforcement Learning for Microgrid Cost Optimization Considering Load Flexibility

Citation Formats

References (10)

Similar Records

Related Subjects