57 Search Results

Grid emergency voltage control (GEVC) is paramount in electric power systems to improve voltage stability and prevent cascading outages and blackouts in case of contingencies. While most deep reinforcement learning (DRL)-based paradigms perform single agents in a static environment, real-world agents for GEVC are expected to cooperate in a dynamically shifting grid. Moreover, due to high uncertainties from combinatory natures of various contingencies and load consumption, along with the complexity of dynamic grid operation, the data efficiency and control performance of the existing DRL-based methods are challenged. To address these limitations, we propose a multi-agent graph-attention (GATT)-based DRL algorithm formore » GEVC in multi-area power systems. Here, we develop graph convolutional network (GCN)-based agents for feature representation of the graph-structured voltages to improve the decision accuracy in a data-efficient manner. Furthermore, a cutting-edge attention mechanism concentrates on effective information sharing among multiple agents, synergizing different-sized subnetworks in the grid for cooperative learning. We address several key challenges in the existing DRL-based GEVC approaches, including low scalability and poor stability against high uncertainties. Test results in the IEEE benchmark system verify the advantages of the proposed method over several recent multi-agent DRL-based algorithms.« less

Battery energy storage (BES) is a versatile resource for the secure and economic operation of microgrids (MGs). Prevailing stochastic optimization-based approaches for BES expansion planning for MGs are computationally complicated. This work proposes a data-driven bi-level multi-period BES expansion planning framework to determine the siting, sizing, and timing of BES installations. The proposed planning framework unifies deep reinforcement learning (DRL) and linear programming, thereby decoupling the determinations for the integer and continuous decision variables in two time scales, respectively. In the upper level, a rainbow DRL agent with quantile regression is trained to provide dynamic planning policies to accommodate stochasticmore » renewable energy resources (RESs), load, and battery price changes efficiently. Further, the lower level computes the optimal operation of MGs with frequency constraints to hedge the islanding contingency. The two levels communicate with one another by exchanging storage configuration and operating expenses in order to accomplish the shared goal of minimizing investment and operation costs. Comparative case studies on an MG are carried out to demonstrate the superiority of the proposed DRL-based solution to the mixed-integer linear programming counterpart on efficiency, scalability, and adaptability.« less

Electric overloading conditions and contingencies put modern power systems at risk of voltage collapse and blackouts. Load shedding is crucial to maintain voltage stability for grid emergency control. However, the rule- or model-based schemes rely on accurate dynamic system models and face considerable challenges in adapting to various operating conditions and uncertain event occurrences. Here, to address these issues, this paper proposes a novel deep reinforcement learning (DRL)-based voltage stability control algorithm with automatic entropy adjustment (AEA) for grid emergency control. Various dynamic network components for complex system operations are modeled to construct the DRL environment. An off-policy soft actor-criticmore » architecture is developed to maximize the expected reward and policy entropy simultaneously. The AEA mechanism is proposed to facilitate the policy maximum entropy procedure, and the proposed method can automatically provide effective discrete and continuous actions against various fault scenarios. Our approach accomplishes high sampling efficiency, scalability, and auto-adaptivity of the control policies under high uncertainties. Comparative studies with the existing DRL-based control methods in IEEE benchmarks indicate salient performance improvement of the proposed method for dynamic system emergency control.« less

Power system stability and control have become more challenging due to the increasing uncertainty associated with renewable generation. Here, the performance of conventional control is highly driven by the physics-based offline-developed dynamic models that can deviate from the actual system characteristics under different operating conditions and/or configurations. Data-driven approaches based on online measurements can be a better solution to addressing these issues by capturing real-time operation conditions. This article describes a novel fully data-driven probabilistic framework to derive a linear representation of postcontingency grid dynamics and online prescribe control based on the derived model to enhance transient stability. The complexmore » nonlinear power system dynamics is approximated by a linear model by using multiple neural network modules that infer distributions of the observations and introducing a Koopman layer to sample possible Koopman linear models from the inferred distributions. The trained model features linearity that can be easily incorporated into the existing linear control design paradigm and ease the controller design process. The effectiveness of Koopman-based control designs is validated through comparative case studies, which demonstrate increased prediction accuracy and control performance when applied to a power system with heterogeneous generator dynamics.« less

Increasing integration of renewable resources brings more flexibility and poses new challenges to modern power systems, leading to highly nonlinear and complex dynamics. Here, this paper aims to provide a general solution framework to traditional control problems, such as frequency control and voltage control, which attempt to maintain the stability of either synchronous generators-governed or inverter-governed systems when subjected to a disturbance and simultaneously guarantee operational constraints, providing a complete complement to existing works on control design. Building on reinforcement learning (RL) and control barrier functions, the framework includes two subsystems, i.e., a model-free controller and a barrier-certification system, whichmore » discover RL-based control actions and sequentially filter them using a barrier certificate to satisfy operational constraints. Calculating a barrier function is generally challenging for a complex power system. This is addressed by representing the barrier function using neural networks (NNs) and data-based approaches. An adaptive method is introduced to certify the neural barrier function that perseveres barrier conditions, which is more compatible with online implementation. The proposed framework synthesizes a stabilizing controller that satisfies predefined safety regions. The effectiveness of the proposed framework is demonstrated via several comparative case studies.« less

The increasing complexity associated with renewable generation brings more challenges to power system stability assessment (SA). Data-driven approaches based on machine learning (ML) techniques for stability assessment have received significant research interest and shown their promising performance. However, ML-based models are recognized to be vulnerable to adversarial disturbances, where a slight perturbation to power system measurements could lead to unacceptable errors. To address this issue, this paper develops a novel lightweight mitigation strategy, i.e., robust online stability assessment (ROSA), to enhance the ML-based assessment model against both white-box and the black-box adversarial disturbances (i.e., purification) in the online implementation. Themore » ROSA involves a supervised learning-based module for the primary stability assessment and a self-supervised learning-based module. Further, the two modules are trained jointly with different objective (loss) functions and implemented in sequence. A suitable purification objective and various time-series data augmentation methods are designed for SA applications to tackle adversarial disturbances adaptively. Case studies are performed, and the comparative results have clearly illustrated the competitive, robust accuracy against various adversarial scenarios and verified the effectiveness of the proposed online purification strategy.« less

Power system resilience has become a critical topic in recent years because of the increasing trend of extreme events and the growing integration of intermittent renewable energy sources. To enhance grid resilience against high-impact, low-frequency events, two questions should be answered: how to quantify the resilience of a given grid and how to incorporate the quantification into power system planning, operation, and restoration. Here this paper develops a new set of quantitative metrics with clear physical interpretation to comprehensively evaluate power system resilience. Using microgrids as an example, an event-based corrective scheduling (ECS) model and an online model predictive controlmore » (OMPC) model are developed to integrate the proposed quantitative resilience metrics into power system optimization models for resilience enhancement. The ECS model employs extreme event data to investigate the optimal restoration solution and to help microgrid operators prepare to respond to similar events. The OMPC model provides online decision-making support for operators to handle ongoing outages in the most resilient fashion. The effectiveness and superiority of the proposed quantitative resilience metrics and the resilience enhancement models are demonstrated through simulations and comparative studies on an IEEE test feeder and a real distribution feeder in Southern California.« less

Online transient analysis plays an increasingly important role in dynamic power grids as the renewable generation continues growing. Traditional numerical methods for transient analysis not only are computationally intensive but also require precise contingency information as input, and therefore, are not suitable for online applications. Existing online transient assessment studies focus on the determination of post-contingency system stability or stability margin. Here, this paper develops a novel graph-learning framework, Deep-learning Neural Representation or DNR, for online prediction, of the time-series trajectories of the system states using initial system responses that can be measured by phasor measurement units (PMUs). The proposedmore » DNR framework consists of two sequential modules: a Network Constructor that captures network dependencies among generators, and a Dynamics Predictor that predicts the system trajectories. The key to improved prediction performance is the introduction of the spatio-temporal message-passing operations into graph neural networks with structural knowledge. Its effectiveness and scalability are validated through comparative studies, demonstrating the prediction performance under different contingency scenarios for systems of different sizes. This framework provides a solution to online predicting post-fault system dynamics based on real-time PMU measurements. Additionally, it can also be applied to facilitate the offline transient simulation without simulating the entire trajectories.« less

We report a distribution service restoration algorithm as a fundamental resilient paradigm for system operators provides an optimally coordinated, resilient solution to enhance the restoration performance. The restoration problem is formulated to coordinate distribution generators and controllable switches optimally. A model-based control scheme is usually designed to solve this problem, relying on a precise model and resulting in low scalability. To tackle these limitations, this work proposes a graph-reinforcement learning framework for the restoration problem. We link the power system topology with a graph convolutional network, which captures the complex mechanism of network restoration in power networks and understands themore » mutual interactions among controllable devices. Latent features over graphical power networks produced by graph convolutional layers are exploited to learn the control policy for network restoration using deep reinforcement learning. The solution scalability is guaranteed by modeling distributed generators as agents in a multi-agent environment and a proper pre-training paradigm. Comparative studies on IEEE 123-node and 8500-node test systems demonstrate the performance of the proposed solution.« less

Emergency control, typically such as under-voltage load shedding (UVLS), is broadly used to grapple with low voltage and voltage instability issues in real-world power systems under contingencies. However, existing emergency control schemes are rule-based and cannot be adaptively applied to uncertain and floating operating conditions. Here, we propose an adaptive UVLS algorithm for emergency control via deep reinforcement learning (DRL) and expert systems. We first construct dynamic components for picturing the power system operation as the environment. The transient voltage recovery criteria, which poses time-varying requirements to UVLS, is integrated into the states and reward function to advise the learningmore » of deep neural networks. The proposed method has no tuning issue of coefficients in reward functions, and this issue was regarded as a deficiency in the existing DRL-based algorithms. Case studies illustrate that the proposed method outperforms the traditional UVLS relay in both the timeliness and efficacy for emergency control.« less