Deep Reinforcement Learning Enabled Physical-Model-Free Two-Timescale Voltage Control Method for Active Distribution Systems
- Univ. of Electronic Science and Technology of China, Chengdu (China)
- Mississippi State Univ., Mississippi State, MS (United States)
- Univ. of California, Riverside, CA (United States)
- National Renewable Energy Lab. (NREL), Golden, CO (United States)
- Univ. of Electronic Science and Technology of China, Chengdu (China); Chengdu Univ. of Technology (China)
- Aalborg Univ. (Denmark)
Active distribution networks are being challenged by frequent and rapid voltage violations due to renewable energy integration. Conventional model-based voltage control methods rely on accurate parameters of the distribution networks, which are difficult to achieve in practice. This paper proposes a novel physical-model-free two-timescale voltage control framework for active distribution systems. To achieve fast control of PV inverters, the whole network is first partitioned into several subnetworks using voltage-reactive power sensitivity. Then, the scheduling of PV inverters in the multiple sub-networks is formulated as Markov games and solved by a multi-agent soft actor-critic (MASAC) algorithm, where each subnetwork is modeled as an intelligent agent. All agents are trained in a centralized manner to learn a coordinated strategy while being executed based on only local information for fast response. For the slower time-scale control, OLTCs and switched capacitors are coordinated by a single agent-based SAC algorithm using the global information with considering control behaviors of the inverters. Particularly, the two-level agents are trained concurrently with information exchange according to the reward signal calculated from the data-driven surrogate model. Comparative tests with different benchmark methods on IEEE 33-and 123-bus systems and 342-node low voltage distribution system demonstrate that the proposed method can effectively mitigate the fast voltage violations and achieve systematical coordination of different voltage regulation assets without the knowledge of accurate system model.
- Research Organization:
- National Renewable Energy Laboratory (NREL), Golden, CO (United States)
- Sponsoring Organization:
- USDOE Office of Energy Efficiency and Renewable Energy (EERE)
- Grant/Contract Number:
- AC36-08GO28308
- OSTI ID:
- 1823582
- Report Number(s):
- NREL/JA-5D00-81108; MainId:79884; UUID:27e99170-5db3-4021-9ab3-7cf5f44562a2; MainAdminID:63062
- Journal Information:
- IEEE Transactions on Smart Grid, Journal Name: IEEE Transactions on Smart Grid Journal Issue: 1 Vol. 13; ISSN 1949-3053
- Publisher:
- IEEECopyright Statement
- Country of Publication:
- United States
- Language:
- English
Similar Records
Data-Driven Multi-agent Deep Reinforcement Learning for Distribution System Decentralized Voltage Control with High Penetration of PVs
Multi-Agent Deep Reinforcement Learning for Realistic Distribution System Voltage Control Using PV Inverters