Skip to main content
OSTI.GOVU.S. Department of Energy Office of Scientific and Technical Information
U.S. Department of Energy
Office of Scientific and Technical Information
 

Power System Frequency Dynamics Modeling, State Estimation, and Control using Neural Ordinary Differential Equations (NODEs) and Soft Actor-Critic (SAC) Machine Learning Approaches

Journal Article · · Applied Computing Review
 [1];  [2];  [3];  [3];  [2];  [2];  [4];  [5];  [2]
  1. South Dakota State University, Brookings, SD (United States); South Dakota State University
  2. South Dakota State University, Brookings, SD (United States)
  3. University of Maine, Orono, ME (United States)
  4. Technical University of Munich (Germany)
  5. University of Connecticut, Storrs, CT (United States)
+ Show Author Affiliations

With the global energy transition of the electric power system, grid control, supervision, and protection is becoming more challenging. With the increasing integration of renewable energy sources (RES), the system dynamics are changing, causing traditional power system dynamic modeling with swing equation-based modeling approaches to fail. Additionally, the converter-dominated power grid is decreasing the system inertia, making the power system more fragile to the frequency swings. This paper first investigates and compares the application of a model-based Kalman filter state estimation approach with (i) a model-free machine learning approach --- neural ordinary differential equations (NODEs) --- and (ii) a data-driven system identification (SysId) approach to model and infer critical state values of the power system frequency dynamics. Then a model predictive control (MPC) framework is compared to a model-free Soft Actor-Critic (SAC) reinforcement learning (RL) control algorithm in providing efficient fast frequency response (FFR) to the power system frequency dynamics. The approaches are compared in terms of their performance goals as well as their per-timestep computational efficiency. Furthermore, the comparative study for state estimation shows that for the model-free requirement, both NODEs and SysId can provide accurate state estimates; however, with increasing model complexity, NODEs can be a better choice for model identification. Similarly, the results from the FFR comparative study show that the SAC RL-based FFR, once trained, outperforms MPC with better control signals and faster computation time, making the SAC RL-based FFR better option for providing FFR to the power system.

Research Organization:
South Dakota State University, Brookings, SD (United States)
Sponsoring Organization:
USDOE Office of Science (SC); USDOE Office of Electricity (OE); USDOE Office of Energy Efficiency and Renewable Energy (EERE), Renewable Power Office. Solar Energy Technologies Office; National Science Foundation (NSF)
Grant/Contract Number:
SC0020281
OSTI ID:
2404598
Journal Information:
Applied Computing Review, Journal Name: Applied Computing Review Journal Issue: 1 Vol. 24; ISSN 1559-6915
Publisher:
Association for Computing MachineryCopyright Statement
Country of Publication:
United States
Language:
English

References (20)

A new autocovariance least-squares method for estimating noise covariances journal February 2006
What does the GB power outage on 9 August 2019 tell us about the current state of decarbonised power systems? journal November 2020
Synthetic inertia versus fast frequency response: a definition journal April 2018
Real-Time Estimation of Microgrid Inertia and Damping Constant journal January 2021
Data-Driven Model Predictive Control for Fast-Frequency Support conference October 2023
Implementation of Model Predictive Control for Frequency Support in a Real-time Digital Simulator conference November 2022
Virtual synchronous machine conference October 2007
Kalman Filter and Its Application conference November 2015
Power system state estimation comparison of Kalman filters with a new approach conference November 2016
A Review and Performance Comparison of Power System State Estimation Techniques conference May 2018
Fast Frequency Control Scheme through Adaptive Virtual Inertia Emulation conference May 2018
Multiarea Inertia Estimation Using Convolutional Neural Networks and Federated Learning journal December 2022
Evaluation of Probing Signals for Implementing Moving Horizon Inertia Estimation in Microgrids conference April 2021
Application of Neural Ordinary Differential Equations to Power System Frequency Dynamics conference October 2023
A Soft Actor-Critic Approach for Power System Fast Frequency Response conference October 2023
Optimization-Based Fast-Frequency Estimation and Control of Low-Inertia Microgrids journal June 2021
Roles of Dynamic State Estimation in Power System Modeling, Monitoring and Operation journal May 2021
Dynamic State Estimation of a Synchronous Machine Using PMU Data: A Comparative Study journal January 2015
An Asynchronous Advantage Actor-Critic Reinforcement Learning Method for Stock Selection and Portfolio Management conference October 2018
Robustness and Consistency in Linear Quadratic Control with Untrusted Predictions
  • Li, Tongxin; Yang, Ruixiao; Qu, Guannan
  • Proceedings of the ACM on Measurement and Analysis of Computing Systems, Vol. 6, Issue 1 https://doi.org/10.1145/3508038
journal February 2022

Figures / Tables (16)

Figure 1(p. 5)figure Figure 1
Figure 2(p. 5)figure Figure 2
Figure 3(p. 6)figure Figure 3
Figure 4(p. 6)figure Figure 4
Table 1(p. 7)table Table 1
Figure 5(p. 8)figure Figure 5
Table 2(p. 8)table Table 2
Figure 6(p. 9)figure Figure 6
Table 3(p. 9)table Table 3
Table 4(p. 9)table Table 4
Figure 7(p. 11)figure Figure 7
Table 5(p. 11)table Table 5
Figure 10(p. 12)figure Figure 10
Figure 8(p. 12)figure Figure 8
Table 6(p. 12)table Table 6
Figure 9(p. 13)figure Figure 9

Similar Records

Comparative analysis of model-free and model-based HVAC control for residential demand response
Conference · Mon Nov 01 00:00:00 EDT 2021 · OSTI ID:1837403

Related Subjects

97 MATHEMATICS AND COMPUTING
Dynamic modeling
neural ordinary differential equations
power system frequency dynamics
reinforcement learning
soft actor critic