Search Menu
DOE PAGES title logo U.S. Department of Energy
Office of Scientific and Technical Information
Search Scholarly Publications

Title: Adaptive critic design-based reinforcement learning approach in controlling virtual inertia-based grid-connected inverters

Abstract

In this report, an adaptive critic design (ACD) approach is proposed to control the phase and voltage of a grid-connected virtual synchronous generator (VSG). The penetration of fast responding inertia-less power converters significantly affect the stability of the power system, especially weak systems such as micro grids. The concept of virtual inertia addresses this concern by virtually emulating the behavior of a synchronous generator. However, the conventional VSG is designed based on two conditions: (i) fixed operating point and (ii) inductive grid connections. The performance of VSGs in low-voltage semi-resistive microgrids is far from optimal. To overcome the aforementioned concerns, a heuristic dynamic programing (HDP) approach is proposed to optimally control grid-connected VSGs. The neural-network-based inherence of the HDP enables the proposed technique to adapt to any impedance angle. The HDP controller includes two subnetworks: (i) the action network that controls the system optimally and (ii) the critic network, which evaluates the effectiveness of the action network. The simulation and experimental results are provided to evaluate the effectiveness of the proposed technique. As shown, the HDP-based approach illustrates a better performance in comparison with the conventional PI-based VSG in various operating conditions.

Authors:
 [1];  [1];  [1]
+ Show Author Affiliations
  1. Missouri Univ. of Science and Technology, Rolla, MO (United States)
Publication Date:
Research Org.:
Univ. of Missouri, Columbia, MO (United States)
Sponsoring Org.:
USDOE Office of Energy Efficiency and Renewable Energy (EERE)
OSTI Identifier:
1848798
Alternate Identifier(s):
OSTI ID: 1809499
Grant/Contract Number:  
EE0008449
Resource Type:
Accepted Manuscript
Journal Name:
International Journal of Electrical Power and Energy Systems
Additional Journal Information:
Journal Volume: 127; Journal Issue: C; Journal ID: ISSN 0142-0615
Publisher:
Elsevier
Country of Publication:
United States
Language:
English
Subject:
24 POWER TRANSMISSION AND DISTRIBUTION; Engineering; Adaptive critic design; Approximate dynamic programming; DC/AC converters; Heuristic dynamic programming; Machine learning; Reinforcement learning; Virtual inertia; Synchronverter

Citation Formats

Saadatmand, Sepehr, Shamsi, Pourya, and Ferdowsi, Mehdi. Adaptive critic design-based reinforcement learning approach in controlling virtual inertia-based grid-connected inverters. United States: N. p., 2021. Web. doi:10.1016/j.ijepes.2020.106657.
Copy to clipboard
Saadatmand, Sepehr, Shamsi, Pourya, & Ferdowsi, Mehdi. Adaptive critic design-based reinforcement learning approach in controlling virtual inertia-based grid-connected inverters. United States. https://doi.org/10.1016/j.ijepes.2020.106657
Copy to clipboard
Saadatmand, Sepehr, Shamsi, Pourya, and Ferdowsi, Mehdi. Sat . "Adaptive critic design-based reinforcement learning approach in controlling virtual inertia-based grid-connected inverters". United States. https://doi.org/10.1016/j.ijepes.2020.106657. https://www.osti.gov/servlets/purl/1848798.
Copy to clipboard
@article{osti_1848798,
title = {Adaptive critic design-based reinforcement learning approach in controlling virtual inertia-based grid-connected inverters},
author = {Saadatmand, Sepehr and Shamsi, Pourya and Ferdowsi, Mehdi},
abstractNote = {In this report, an adaptive critic design (ACD) approach is proposed to control the phase and voltage of a grid-connected virtual synchronous generator (VSG). The penetration of fast responding inertia-less power converters significantly affect the stability of the power system, especially weak systems such as micro grids. The concept of virtual inertia addresses this concern by virtually emulating the behavior of a synchronous generator. However, the conventional VSG is designed based on two conditions: (i) fixed operating point and (ii) inductive grid connections. The performance of VSGs in low-voltage semi-resistive microgrids is far from optimal. To overcome the aforementioned concerns, a heuristic dynamic programing (HDP) approach is proposed to optimally control grid-connected VSGs. The neural-network-based inherence of the HDP enables the proposed technique to adapt to any impedance angle. The HDP controller includes two subnetworks: (i) the action network that controls the system optimally and (ii) the critic network, which evaluates the effectiveness of the action network. The simulation and experimental results are provided to evaluate the effectiveness of the proposed technique. As shown, the HDP-based approach illustrates a better performance in comparison with the conventional PI-based VSG in various operating conditions.},
doi = {10.1016/j.ijepes.2020.106657},
journal = {International Journal of Electrical Power and Energy Systems},
number = C,
volume = 127,
place = {United States},
year = {Sat May 01 00:00:00 EDT 2021},
month = {Sat May 01 00:00:00 EDT 2021}
}
Copy to clipboard
Journal Article:
Free Publicly Available Full Text
Publisher's Version of Record
https://doi.org/10.1016/j.ijepes.2020.106657
Copyright Statement
Other availability
Search WorldCat Search WorldCat to find libraries that may hold this journal
Save / Share:
Export Metadata
Save to My Library
You must Sign In or Create an Account in order to save documents to your library.

Works referenced in this record:

Overview of Control and Grid Synchronization for Distributed Power Generation Systems
journal, October 2006

  • Blaabjerg, F.; Teodorescu, R.; Liserre, M.
  • IEEE Transactions on Industrial Electronics, Vol. 53, Issue 5
  • DOI: 10.1109/TIE.2006.881997

Wind Farms With HVdc Delivery in Inertial Response and Primary Frequency Control
journal, December 2010

  • Miao, Zhixin; Fan, Lingling; Osborn, Dale
  • IEEE Transactions on Energy Conversion, Vol. 25, Issue 4
  • DOI: 10.1109/TEC.2010.2060202

Learning-Based Energy-Efficient Resource Management by Heterogeneous RF/VLC for Ultra-Reliable Low-Latency Industrial IoT Networks
journal, August 2020

  • Yang, Helin; Alphones, Arokiaswami; Zhong, Wen-De
  • IEEE Transactions on Industrial Informatics, Vol. 16, Issue 8
  • DOI: 10.1109/TII.2019.2933867

Provision of Inertial and Primary Frequency Control Services Using Offshore Multiterminal HVDC Networks
journal, October 2012

  • Silva, B.; Moreira, C. L.; Seca, L.
  • IEEE Transactions on Sustainable Energy, Vol. 3, Issue 4
  • DOI: 10.1109/TSTE.2012.2199774

Virtual synchronous generators: A survey and new perspectives
journal, January 2014

  • Bevrani, Hassan; Ise, Toshifumi; Miura, Yushi
  • International Journal of Electrical Power & Energy Systems, Vol. 54
  • DOI: 10.1016/j.ijepes.2013.07.009

Virtual Inertia Control Strategy in Microgrid Based on Virtual Synchronous Generator Technology
journal, January 2018

Virtual synchronous generators
conference, July 2008

  • Driesen, J.; Visscher, K.
  • 2008 IEEE Power and Energy Society General Meeting - Conversion and Delivery of Electrical Energy in the 21st Century
  • DOI: 10.1109/PES.2008.4596800

Novel Comprehensive Control Framework for Incorporating VSCs to Smart Power Grids Using Bidirectional Synchronous-VSC
journal, March 2014

  • Ashabani, Mahdi; Mohamed, Yasser Abdel-Rady I.
  • IEEE Transactions on Power Systems, Vol. 29, Issue 2
  • DOI: 10.1109/TPWRS.2013.2287291

Synchronverters: Inverters That Mimic Synchronous Generators
journal, April 2011

  • Zhong, Qing-Chang; Weiss, George
  • IEEE Transactions on Industrial Electronics, Vol. 58, Issue 4
  • DOI: 10.1109/TIE.2010.2048839

Impedance Analysis of Virtual Synchronous Generator-Based Vector Controlled Converters for Weak AC Grid Integration
journal, July 2019

  • Asrari, Arash; Mustafa, Mehran; Ansari, Meisam
  • IEEE Transactions on Sustainable Energy, Vol. 10, Issue 3
  • DOI: 10.1109/TSTE.2019.2892670

Inertia Response Improvement in AC Microgrids: A Fuzzy-Based Virtual Synchronous Generator Control
journal, April 2020

  • Karimi, Amin; Khayat, Yousef; Naderi, Mobin
  • IEEE Transactions on Power Electronics, Vol. 35, Issue 4
  • DOI: 10.1109/TPEL.2019.2937397

Adaptive critic designs
journal, September 1997

  • Prokhorov, D. V.; Wunsch, D. C.
  • IEEE Transactions on Neural Networks, Vol. 8, Issue 5
  • DOI: 10.1109/72.623201

Power System Small-Signal Angular Stability Affected by Virtual Synchronous Generators
journal, July 2019

Robust Frequency Control of Microgrids Using an Extended Virtual Synchronous Generator
journal, November 2018

  • Fathi, Abdolwahhab; Shafiee, Qobad; Bevrani, Hassan
  • IEEE Transactions on Power Systems, Vol. 33, Issue 6
  • DOI: 10.1109/TPWRS.2018.2850880

Intelligent Resource Management Based on Reinforcement Learning for Ultra-Reliable and Low-Latency IoV Communication Networks
journal, May 2019

  • Yang, Helin; Xie, Xianzhong; Kadoch, Michel
  • IEEE Transactions on Vehicular Technology, Vol. 68, Issue 5
  • DOI: 10.1109/TVT.2018.2890686

Dual heuristic dynamic programing control of grid-connected synchronverters
conference, October 2019

A Generalized Droop Control for Grid-Supporting Inverter Based on Comparison Between Traditional Droop Control and Virtual Synchronous Generator Control
journal, June 2019

An Adaptive Control Strategy for Virtual Synchronous Generator
journal, September 2018

  • Wang, Fei; Zhang, Lijun; Feng, Xiayun
  • IEEE Transactions on Industry Applications, Vol. 54, Issue 5
  • DOI: 10.1109/TIA.2018.2859384

New Transient Stability and LVRT Improvement of Multi-VSG Grids Using the Frequency of the Center of Inertia
journal, January 2020

  • Choopani, Mahdi; Hosseinian, Seyed Hossein; Vahidi, Behrooz
  • IEEE Transactions on Power Systems, Vol. 35, Issue 1
  • DOI: 10.1109/TPWRS.2019.2928319

Self-tuning virtual synchronous generator control for improving frequency stability in autonomous photovoltaic-diesel microgrids
journal, December 2017

  • Shi, Rongliang; Zhang, Xing; Hu, Chao
  • Journal of Modern Power Systems and Clean Energy, Vol. 6, Issue 3
  • DOI: 10.1007/s40565-017-0347-3

Power-Electronics-Enabled Autonomous Power Systems: Architecture and Technical Routes
journal, July 2017

Virtual synchronous machine
conference, October 2007

  • Beck, Hans-Peter; Hesse, Ralf
  • 2007 9th International Conference on Electrical Power Quality and Utilisation
  • DOI: 10.1109/EPQU.2007.4424220

Parameter Constraints for Virtual Synchronous Generator Considering Stability
journal, May 2019

Functional Approximations and Dynamic Programming
journal, October 1959

  • Bellman, Richard; Dreyfus, Stuart
  • Mathematical Tables and Other Aids to Computation, Vol. 13, Issue 68
  • DOI: 10.2307/2002797

A copula-based Bayesian method for probabilistic solar power forecasting
journal, January 2020

    Sort by:
    [ × clear filter / sort ]

    Similar Records in DOE PAGES and OSTI.GOV collections: