Approximating Nash Equilibrium in Day-ahead Electricity Market Bidding with Multi-agent Deep Reinforcement Learning

Journal Article · Mon Apr 19 00:00:00 EDT 2021 · Journal of Modern Power Systems and Clean Energy

DOI:https://doi.org/10.35833/mpce.2020.000502· OSTI ID:1782068

Du, Yan ^[1]; Li, Fangxing ^[1]; Zandi, Helia ^[2]; Xue, Yaosuo ^[2]

---

1. Univ. of Tennessee, Knoxville, TN (United States)
2. Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States)

In this paper, a day-ahead electricity market bidding problem with multiple strategic generation company (GEN-CO) bidders is studied. The problem is formulated as a Markov game model, where GENCO bidders interact with each other todevelop their optimal day-ahead bidding strategies. Considering unobservable information in the problem, a model-free and data-driven approach, known as multi-agent deep deterministic policy gradient (MADDPG), is applied for approximating the Nash equilibrium (NE) in the above Markov game. The MADDPG algorithm has the advantage of generalization due to the automatic feature extraction ability of the deep neural networks. The algorithm is tested on an IEEE 30-bus system with three competitive GENCO bidders in both an uncongested caseand a congested case. Comparisons with a truthful bidding strategy and state-of-the-art deep reinforcement learning methods including deep Q network and deep deterministic policy gradient (DDPG) demonstrate that the applied MADDPG algorithm can find a superior bidding strategy for all the market participants with increased profit gains. In addition, the comparison with a conventional model-based method shows that the MADDPG algorithm has higher computational efficiency, which is feasible for real-world applications.

• Research Organization: Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States)
• Sponsoring Organization: USDOE Office of Electricity Delivery and Energy Reliability (OE)
• Grant/Contract Number: AC05-00OR22725
• OSTI ID: 1782068
• Journal Information: Journal of Modern Power Systems and Clean Energy, Journal Name: Journal of Modern Power Systems and Clean Energy Journal Issue: 3 Vol. 9; ISSN 2196-5625
• Publisher: Springer
• Country of Publication: United States
• Language: English

Related Subjects

24 POWER TRANSMISSION AND DISTRIBUTION
Games
Markov game
Markov processes
Nash equilibrium
bidding strategy
computational modeling
day-ahead electricity market
deep reinforcement learning
electricity supply industry
mathematical model
multi-agent deterministic policy gradient (MADDPG)
training