Skip to main content
OSTI.GOVU.S. Department of Energy Office of Scientific and Technical Information
U.S. Department of Energy
Office of Scientific and Technical Information
 

Learning Constrained Parametric Differentiable Predictive Control Policies With Guarantees

Journal Article · · IEEE Transactions on Systems, Man, and Cybernetics: Systems
 [1];  [1];  [1]
  1. Pacific Northwest National Laboratory (PNNL), Richland, WA (United States)
+ Show Author Affiliations
We present differentiable predictive control (DPC), a method for offline learning of constrained neural control policies for nonlinear dynamical systems with performance guarantees. We show that the sensitivities of the parametric optimal control problem can be used to obtain direct policy gradients. Specifically, we employ automatic differentiation (AD) to efficiently compute the sensitivities of the model predictive control (MPC) objective function and constraints penalties. To guarantee safety upon deployment, we derive probabilistic guarantees on closed-loop stability and constraint satisfaction based on indicator functions and Hoeffding’s inequality. We empirically demonstrate that the proposed method can learn neural control policies for various parametric optimal control tasks. In particular, we show that the proposed DPC method can stabilize systems with unstable dynamics, track time-varying references, and satisfy nonlinear state and input constraints. Our DPC method has practical time savings compared to alternative approaches for fast and memory-efficient controller design. Specifically, DPC does not depend on a supervisory controller as opposed to approximate MPC based on imitation learning. We demonstrate that, without losing performance, DPC is scalable with greatly reduced demands on memory and computation compared to implicit and explicit MPC while being more sample efficient than model-free reinforcement learning (RL) algorithms.
Research Organization:
Pacific Northwest National Laboratory (PNNL), Richland, WA (United States)
Sponsoring Organization:
USDOE Laboratory Directed Research and Development (LDRD) Program
Grant/Contract Number:
AC05-76RL01830
OSTI ID:
2438488
Report Number(s):
PNNL-SA--162577
Journal Information:
IEEE Transactions on Systems, Man, and Cybernetics: Systems, Journal Name: IEEE Transactions on Systems, Man, and Cybernetics: Systems Journal Issue: 6 Vol. 54; ISSN 2168-2216
Publisher:
IEEECopyright Statement
Country of Publication:
United States
Language:
English

References (40)

Learning an approximate model predictive controller with guarantees collection January 2018
On the implementation of an interior-point filter line-search algorithm for large-scale nonlinear programming journal April 2005
Challenges of real-world reinforcement learning: definitions, benchmarks and analysis journal April 2021
CasADi: a software framework for nonlinear optimization and optimal control journal July 2018
OSQP: an operator splitting solver for quadratic programs journal February 2020
A Quasi-Infinite Horizon Nonlinear Model Predictive Control Scheme with Guaranteed Stability∗∗This paper was not presented at any IFAC meeting. This paper was accepted for publication in revised form by Associate Editor W. Bequette under the direction of Editor Prof. S. Skogestad. journal October 1998
Constrained model predictive control: Stability and optimality journal June 2000
Approximate model predictive building control via machine learning journal May 2018
Provably safe and robust learning-based model predictive control journal May 2013
Large scale model predictive control with neural networks and primal active sets journal January 2022
Approximate moving horizon estimation and robust nonlinear model predictive control via deep learning journal May 2021
Learning-Based Robust Model Predictive Control with State-Dependent Uncertainty journal January 2018
A Neural Network Architecture to Learn Explicit MPC Controllers from Data journal January 2020
Move blocking strategies in receding horizon control journal July 2007
On the performance of economic model predictive control with self-tuning terminal cost journal August 2014
Differentiable predictive control: Deep learning alternative to explicit model predictive control for unknown nonlinear systems journal June 2022
Convex Optimization book January 2004
An apologia for stabilising terminal conditions in model predictive control journal November 2013
Probability Inequalities for Sums of Bounded Random Variables journal March 1963
The explicit solution of model predictive control via multiparametric quadratic programming conference January 2000
Model Predictive Control Design: New Trends and Tools conference January 2006
Learning a feasible and stabilizing explicit model predictive control law by robust optimization conference December 2011
A fast condensing method for solution of linear-quadratic control problems conference December 2013
Adaptive MPC for Iterative Tasks conference December 2018
Neural Lyapunov Differentiable Predictive Control conference December 2022
Reachability Analysis of Discrete-Time Systems With Disturbances journal April 2006
Clipping-Based Complexity Reduction in Explicit MPC journal July 2012
A General Safety Framework for Learning-Based Control in Uncertain Robotic Systems journal July 2019
Cautious Model Predictive Control Using Gaussian Process Regression journal November 2020
Efficient Representation and Approximation of Model Predictive Control Laws via Deep Learning journal September 2020
Deep Learning-Based Model Predictive Control for Resonant Power Converters journal January 2021
Robust Model Predictive Control of Nonlinear Systems With Unmodeled Dynamics and Bounded Uncertainties Based on Neural Networks journal March 2014
Learning-Based Model Predictive Control: Toward Safe Learning in Control journal May 2020
DeepMPC: Learning Deep Latent Features for Model Predictive Control conference July 2015
Approximating Explicit Model Predictive Control Using Constrained Neural Networks conference June 2018
Safe and Near-Optimal Policy Learning for Model Predictive Control using Primal-Dual Neural Networks conference July 2019
Reinforcement Learning based on MPC and the Stochastic Policy Gradient Method conference May 2021
Multi-Parametric Toolbox 3.0 conference July 2013
Practical Reinforcement Learning of Stabilizing Economic MPC conference June 2019
Model Predictive Control with generalized terminal state constraint journal January 2012

Similar Records

Learning Stochastic Parametric Differentiable Predictive Control Policies
Conference · Mon Oct 10 00:00:00 EDT 2022 · OSTI ID:1907869

Related Subjects

97 MATHEMATICS AND COMPUTING
deep learning
dynamical systems
neural networks
optimal control
optimal control theory