Lax-Oleinik-Type Formulas and Efficient Algorithms for Certain High-Dimensional Optimal Control Problems

Chen, Paula; Darbon, Jérôme; Meng, Tingwei

doi:10.1007/s42967-024-00371-4

Title: Lax-Oleinik-Type Formulas and Efficient Algorithms for Certain High-Dimensional Optimal Control Problems

Journal Article · 29 April 2024 · Communications on Applied Mathematics and Computation

DOI: https://doi.org/10.1007/s42967-024-00371-4 · OSTI ID:2441158

Chen, Paula ^[1];

^[2]; Meng, Tingwei ^[3]

Brown University, Providence, RI (United States); Brown University
Brown University, Providence, RI (United States)
University of California, Los Angeles, CA (United States)

Two of the main challenges in optimal control are solving problems with state-dependent running costs and developing efficient numerical solvers that are computationally tractable in high dimension. In this paper, we provide analytical solutions to certain optimal control problems whose running cost depends on the state variable and with constraints on the control. We also provide Lax-Oleinik-type representation formulas for the corresponding Hamilton-Jacobi partial differential equations with state-dependent Hamiltonians. Additionally, we present an efficient, grid-free numerical solver based on our representation formulas, which is shown to scale linearly with the state dimension, and thus, to overcome the curse of dimensionality. Using existing optimization methods and the min-plus technique, we extend our numerical solvers to address more general classes of convex and nonconvex initial costs. We demonstrate the capabilities of our numerical solvers using implementations on a central processing unit (CPU) and a field-programmable gate array (FPGA). In several cases, our FPGA implementation obtains over a 10 times speedup compared to the CPU, which demonstrates the promising performance boosts FPGAs can achieve. Furthermore, our numerical results show that our solvers have the potential to serve as a building block for solving broader classes of high-dimensional optimal control problems in real-time.

View Accepted Manuscript (DOE)

Cite

Export

Save

Research Organization:: Brown University, Providence, RI (United States)

Sponsoring Organization:: USDOE Office of Science (SC); Air Force Office of Scientific Research (AFOSR)

Grant/Contract Number:: SC0023191

OSTI ID:: 2441158

Journal Information:: Communications on Applied Mathematics and Computation, Journal Name: Communications on Applied Mathematics and Computation Journal Issue: 2 Vol. 6; ISSN 2096-6385

Publisher:: Springer NatureCopyright Statement

Country of Publication:: United States

Language:: English

References (71)

An Architecture for Online Affordance-based Perception and Whole-body Planning: An Architecture for Online Affordance-based Perception Fallon, Maurice; Kuindersma, Scott; Karumanchi, Sisir Journal of Field Robotics, Vol. 32, Issue 2 https://doi.org/10.1002/rob.21546	journal	October 2014
Optimal Control and Viscosity Solutions of Hamilton-Jacobi-Bellman Equations Bardi, Martino; Capuzzo-Dolcetta, Italo https://doi.org/10.1007/978-0-8176-4755-1	book	January 1997
Variational Analysis Rockafellar, R. Tyrrell; Wets, Roger J. B. Grundlehren der mathematischen Wissenschaften https://doi.org/10.1007/978-3-642-02431-3	book	January 1998
Perspectives on Characteristics Based Curse-of-Dimensionality-Free Numerical Approaches for Solving Hamilton–Jacobi Equations Yegorov, Ivan; Dower, Peter M. Applied Mathematics & Optimization, Vol. 83, Issue 1 https://doi.org/10.1007/s00245-018-9509-6	journal	July 2018
Optimization-based locomotion planning, estimation, and control design for the atlas humanoid robot Kuindersma, Scott; Deits, Robin; Fallon, Maurice Autonomous Robots, Vol. 40, Issue 3 https://doi.org/10.1007/s10514-015-9479-3	journal	July 2015
Mitigating the curse of dimensionality: sparse grid characteristics method for optimal feedback control and HJB equations Kang, Wei; Wilcox, Lucas C. Computational Optimization and Applications, Vol. 68, Issue 2 https://doi.org/10.1007/s10589-017-9910-0	journal	April 2017
On Hamilton–Jacobi PDEs and Image Denoising Models with Certain Nonadditive Noise Darbon, Jérôme; Meng, Tingwei; Resmerita, Elena Journal of Mathematical Imaging and Vision, Vol. 64, Issue 4 https://doi.org/10.1007/s10851-022-01073-3	journal	March 2022
An Adaptive Sparse Grid Semi-Lagrangian Scheme for First Order Hamilton-Jacobi Bellman Equations Bokanowski, Olivier; Garcke, Jochen; Griebel, Michael Journal of Scientific Computing, Vol. 55, Issue 3 https://doi.org/10.1007/s10915-012-9648-x	journal	October 2012
On the Global and Linear Convergence of the Generalized Alternating Direction Method of Multipliers Deng, Wei; Yin, Wotao Journal of Scientific Computing, Vol. 66, Issue 3 https://doi.org/10.1007/s10915-015-0048-x	journal	May 2015
Suboptimal Feedback Control of PDEs by Solving HJB Equations on Adaptive Sparse Grids Garcke, Jochen; Kröner, Axel Journal of Scientific Computing, Vol. 70, Issue 1 https://doi.org/10.1007/s10915-016-0240-7	journal	June 2016
Deep Neural Networks Algorithms for Stochastic Control Problems on Finite Horizon: Numerical Applications Bachouch, Achref; Huré, Côme; Langrené, Nicolas Methodology and Computing in Applied Probability, Vol. 24, Issue 1 https://doi.org/10.1007/s11009-019-09767-9	journal	January 2021
Overcoming the curse of dimensionality for some Hamilton–Jacobi partial differential equations via neural network architectures Darbon, Jérôme; Langlois, Gabriel P.; Meng, Tingwei Research in the Mathematical Sciences, Vol. 7, Issue 3 https://doi.org/10.1007/s40687-020-00215-6	journal	July 2020
Intelligent optimal control of robotic manipulators using neural networks Kim, Young H.; Lewis, Frank L.; Dawson, Darren M. Automatica, Vol. 36, Issue 9 https://doi.org/10.1016/S0005-1098(00)00045-5	journal	September 2000
Spectral method for constrained linear–quadratic optimal control Jaddu, Hussein Mathematics and Computers in Simulation, Vol. 58, Issue 2 https://doi.org/10.1016/S0378-4754(01)00359-7	journal	January 2002
DGM: A deep learning algorithm for solving partial differential equations Sirignano, Justin; Spiliopoulos, Konstantinos Journal of Computational Physics, Vol. 375 https://doi.org/10.1016/j.jcp.2018.08.029	journal	December 2018
A new scalable algorithm for computational optimal control under uncertainty Lambrianides, Panos; Gong, Qi; Venturi, Daniele Journal of Computational Physics, Vol. 420 https://doi.org/10.1016/j.jcp.2020.109710	journal	November 2020
On some neural network architectures that can represent viscosity solutions of certain high dimensional Hamilton–Jacobi partial differential equations Darbon, Jérôme; Meng, Tingwei Journal of Computational Physics, Vol. 425 https://doi.org/10.1016/j.jcp.2020.109907	journal	January 2021
Robot manipulator control using neural networks: A survey Jin, Long; Li, Shuai; Yu, Jiguo Neurocomputing, Vol. 285 https://doi.org/10.1016/j.neucom.2018.01.002	journal	April 2018
SympNets: Intrinsic structure-preserving symplectic networks for identifying Hamiltonian systems Jin, Pengzhan; Zhang, Zhen; Zhu, Aiqing Neural Networks, Vol. 132 https://doi.org/10.1016/j.neunet.2020.08.017	journal	December 2020
LQ tracking controls with fixed terminal states and their application to receding horizon controls Park, Jung Hun; Han, Soohee; Kwon, Wook Hyun Systems & Control Letters, Vol. 57, Issue 9 https://doi.org/10.1016/j.sysconle.2008.03.006	journal	September 2008
Efficient computation of optimal actions Todorov, Emanuel Proceedings of the National Academy of Sciences, Vol. 106, Issue 28 https://doi.org/10.1073/pnas.0710743106	journal	July 2009
Solving high-dimensional partial differential equations using deep learning Han, Jiequn; Jentzen, Arnulf; E., Weinan Proceedings of the National Academy of Sciences, Vol. 115, Issue 34 https://doi.org/10.1073/pnas.1718942115	journal	August 2018
Deep backward schemes for high-dimensional nonlinear PDEs Huré, Côme; Pham, Huyên; Warin, Xavier Mathematics of Computation, Vol. 89, Issue 324 https://doi.org/10.1090/mcom/3514	journal	January 2020
An optimal control approach to robust control of robot manipulators IEEE Transactions on Robotics and Automation, Vol. 14, Issue 1 https://doi.org/10.1109/70.660845	journal	January 1998
Curse-of-complexity attenuation in the curse-of-dimensionality-free method for HJB PDEs McEneaney, William M.; Deshpande, Ameet; Gaubert, Stephane 2008 American Control Conference https://doi.org/10.1109/ACC.2008.4587234	conference	June 2008
Neural approximation of PDE solutions: An application to reachability computations Djeridane, Badis; Lygeros, John Proceedings of the 45th IEEE Conference on Decision and Control https://doi.org/10.1109/CDC.2006.377184	conference	January 2006
A Neural Approximation to Continuous Time Reachability Computations Niarchos, K. N.; Lygeros, J. Proceedings of the 45th IEEE Conference on Decision and Control https://doi.org/10.1109/CDC.2006.377358	conference	January 2006
Efficient MPC Optimization using Pontryagin's Minimum Principle Cannon, Mark; Liao, Weiheng; Kouvaritakis, Basil Proceedings of the 45th IEEE Conference on Decision and Control https://doi.org/10.1109/CDC.2006.377753	conference	January 2006
Curse of dimensionality reduction in max-plus based approximation methods: Theoretical estimates and improved pruning algorithms Gaubert, Stephane; McEneaney, William; Qu, Zheng IEEE Conference on Decision and Control and European Control Conference https://doi.org/10.1109/CDC.2011.6161386	conference	December 2011
Linear Hamilton Jacobi Bellman Equations in high dimensions Horowitz, Matanya B.; Damle, Anil; Burdick, Joel W. 2014 IEEE 53rd Annual Conference on Decision and Control (CDC), 53rd IEEE Conference on Decision and Control https://doi.org/10.1109/CDC.2014.7040310	conference	December 2014
Managing HBM Bandwidth on Multi-Die FPGAs with FPGA Overlay NoCs Prakash, Srinirdheeshwar Kuttuva; Patel, Hiren; Kapre, Nachiket 2022 IEEE 30th Annual International Symposium on Field-Programmable Custom Computing Machines (FCCM) https://doi.org/10.1109/FCCM53951.2022.9786203	conference	May 2022
Optimization based full body control for the atlas robot Feng, Siyuan; Whitman, Eric; Xinjilefu, X. 2014 IEEE-RAS International Conference on Humanoid Robots https://doi.org/10.1109/HUMANOIDS.2014.7041347	conference	November 2014
Optimal motion planning for humanoid robots El Khoury, Antonio; Lamiraux, Florent; Taix, Michel 2013 IEEE International Conference on Robotics and Automation https://doi.org/10.1109/ICRA.2013.6631013	conference	May 2013
Application of an approximate model predictive control scheme on an unmanned aerial vehicle Hofer, Matthias; Muehlebach, Michael; D'Andrea, Raffaello 2016 IEEE International Conference on Robotics and Automation (ICRA) https://doi.org/10.1109/ICRA.2016.7487459	conference	May 2016
DeepReach: A Deep Learning Approach to High-Dimensional Reachability Bansal, Somil; Tomlin, Claire J. 2021 IEEE International Conference on Robotics and Automation (ICRA) https://doi.org/10.1109/ICRA48506.2021.9561949	conference	May 2021
Optimal Trajectories of a UAV Base Station Using Lagrangian Mechanics Coupechoux, Marceau; Darbon, Jérôme; Kélif, Jean-Marc IEEE INFOCOM 2019 - IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS) https://doi.org/10.1109/INFCOMW.2019.8845287	conference	April 2019
An optimal planning of falling motions of a humanoid robot Fujiwara, Kiyoshi; Kajita, Shuuji; Harada, Kensuke 2007 IEEE/RSJ International Conference on Intelligent Robots and Systems https://doi.org/10.1109/IROS.2007.4399327	conference	October 2007
Constrained iterative LQR for on-road autonomous driving motion planning Chen, Jianyu; Zhan, Wei; Tomizuka, Masayoshi 2017 IEEE 20th International Conference on Intelligent Transportation Systems (ITSC) https://doi.org/10.1109/ITSC.2017.8317745	conference	October 2017
A Hopf-Lax Formula in Hamilton–Jacobi Analysis of Reach-Avoid Problems Lee, Donggun; Tomlin, Claire J. IEEE Control Systems Letters, Vol. 5, Issue 3 https://doi.org/10.1109/LCSYS.2020.3009933	journal	July 2021
QRnet: Optimal Regulator Design With LQR-Augmented Neural Networks Nakamura-Zimmerer, Tenavi; Gong, Qi; Kang, Wei IEEE Control Systems Letters, Vol. 5, Issue 4 https://doi.org/10.1109/LCSYS.2020.3034415	journal	October 2021
Optimal Rendezvous Trajectory for Unmanned Aerial-Ground Vehicles Rucco, Alessandro; Sujit, P. B.; Aguiar, A. Pedro IEEE Transactions on Aerospace and Electronic Systems, Vol. 54, Issue 2 https://doi.org/10.1109/TAES.2017.2767958	journal	April 2018
Alternating Direction Method of Multipliers for Constrained Iterative LQR in Autonomous Driving Ma, Jun; Cheng, Zilong; Zhang, Xiaoxue IEEE Transactions on Intelligent Transportation Systems, Vol. 23, Issue 12 https://doi.org/10.1109/TITS.2022.3194571	journal	December 2022
Autonomous Driving Motion Planning With Constrained Iterative LQR Chen, Jianyu; Zhan, Wei; Tomizuka, Masayoshi IEEE Transactions on Intelligent Vehicles, Vol. 4, Issue 2 https://doi.org/10.1109/TIV.2019.2904385	journal	June 2019
Analytical solution of a linear quadratic optimal control problem with control value constraints Aipanov, Sh. A.; Murzabekov, Z. N. Journal of Computer and Systems Sciences International, Vol. 53, Issue 1 https://doi.org/10.1134/s1064230713060026	journal	January 2014
HJB-POD-Based Feedback Design for the Optimal Control of Evolution Problems Kunisch, K.; Volkwein, S.; Xie, L. SIAM Journal on Applied Dynamical Systems, Vol. 3, Issue 4 https://doi.org/10.1137/030600485	journal	January 2004
A Curse-of-Dimensionality-Free Numerical Method for Solution of Certain HJB PDEs McEneaney, William M. SIAM Journal on Control and Optimization, Vol. 46, Issue 4 https://doi.org/10.1137/040610830	journal	January 2007
The Max-Plus Finite Element Method for Solving Deterministic Optimal Control Problems: Basic Properties and Convergence Analysis Akian, Marianne; Gaubert, Stéphane; Lakhoua, Asma SIAM Journal on Control and Optimization, Vol. 47, Issue 2 https://doi.org/10.1137/060655286	journal	January 2008
Convergence Rate for a Curse-of-Dimensionality-Free Method for a Class of HJB PDEs McEneaney, William M.; Kluberg, L. Jonathan SIAM Journal on Control and Optimization, Vol. 48, Issue 5 https://doi.org/10.1137/070681934	journal	January 2010
High-Order Essentially Nonoscillatory Schemes for Hamilton–Jacobi Equations Osher, Stanley; Shu, Chi-Wang SIAM Journal on Numerical Analysis, Vol. 28, Issue 4 https://doi.org/10.1137/0728049	journal	August 1991
A Duality Approach for Solving Control-Constrained Linear-Quadratic Optimal Control Problems Burachik, R. S.; Kaya, C. Y.; Majeed, S. N. SIAM Journal on Control and Optimization, Vol. 52, Issue 3 https://doi.org/10.1137/130910221	journal	January 2014
The Principle of Least Action and Fundamental Solutions of Mass-Spring and N-Body Two-Point Boundary Value Problems McEneaney, William M.; Dower, Peter M. SIAM Journal on Control and Optimization, Vol. 53, Issue 5 https://doi.org/10.1137/130921908	journal	January 2015
On Convex Finite-Dimensional Variational Methods in Imaging Sciences and Hamilton--Jacobi Equations Darbon, Jérôme SIAM Journal on Imaging Sciences, Vol. 8, Issue 4 https://doi.org/10.1137/130944163	journal	January 2015
Error Analysis for POD Approximations of Infinite Horizon Problems via the Dynamic Programming Approach Alla, A.; Falcone, M.; Volkwein, S. SIAM Journal on Control and Optimization, Vol. 55, Issue 5 https://doi.org/10.1137/15M1039596	journal	January 2017
Polynomial Approximation of High-Dimensional Hamilton--Jacobi--Bellman Equations and Applications to Feedback Control of Semilinear Parabolic PDEs Kalise, Dante; Kunisch, Karl SIAM Journal on Scientific Computing, Vol. 40, Issue 2 https://doi.org/10.1137/17M1116635	journal	January 2018
An Efficient DP Algorithm on a Tree-Structure for Finite Horizon Optimal Control Problems Alla, Alessandro; Falcone, Maurizio; Saluzzi, Luca SIAM Journal on Scientific Computing, Vol. 41, Issue 4 https://doi.org/10.1137/18M1203900	journal	January 2019
Robust Feedback Control of Nonlinear PDEs by Numerical Approximation of High-Dimensional Hamilton--Jacobi--Isaacs Equations Kalise, Dante; Kundu, Sudeep; Kunisch, Karl SIAM Journal on Applied Dynamical Systems, Vol. 19, Issue 2 https://doi.org/10.1137/19M1262139	journal	January 2020
On Decomposition Models in Imaging Sciences and Multi-time Hamilton--Jacobi Partial Differential Equations Darbon, Jérôme; Meng, Tingwei SIAM Journal on Imaging Sciences, Vol. 13, Issue 2 https://doi.org/10.1137/19M1266332	journal	January 2020
Adaptive Deep Learning for High-Dimensional Hamilton--Jacobi--Bellman Equations Nakamura-Zimmerer, Tenavi; Gong, Qi; Kang, Wei SIAM Journal on Scientific Computing, Vol. 43, Issue 2 https://doi.org/10.1137/19M1288802	journal	January 2021
Tensor Decomposition Methods for High-dimensional Hamilton--Jacobi--Bellman Equations Dolgov, Sergey; Kalise, Dante; Kunisch, Karl K. SIAM Journal on Scientific Computing, Vol. 43, Issue 3 https://doi.org/10.1137/19M1305136	journal	January 2021
Deep Neural Networks Algorithms for Stochastic Control Problems on Finite Horizon: Convergence Analysis Huré, Côme; Pham, Huyên; Bachouch, Achref SIAM Journal on Numerical Analysis, Vol. 59, Issue 1 https://doi.org/10.1137/20M1316640	journal	January 2021
Actor-Critic Method for High Dimensional Static Hamilton--Jacobi--Bellman Partial Differential Equations based on Neural Networks Zhou, Mo; Han, Jiequn; Lu, Jianfeng SIAM Journal on Scientific Computing, Vol. 43, Issue 6 https://doi.org/10.1137/21M1402303	journal	January 2021
A Max-Plus-Based Algorithm for a Hamilton--Jacobi--Bellman Equation of Nonlinear Filtering Fleming, Wendell H.; McEneaney, William M. SIAM Journal on Control and Optimization, Vol. 38, Issue 3 https://doi.org/10.1137/S0363012998332433	journal	January 2000
Weighted ENO Schemes for Hamilton--Jacobi Equations Jiang, Guang-Shan; Peng, Danping SIAM Journal on Scientific Computing, Vol. 21, Issue 6 https://doi.org/10.1137/S106482759732455X	journal	January 2000
A Discontinuous Galerkin Finite Element Method for Hamilton--Jacobi Equations Hu, Changqing; Shu, Chi-Wang SIAM Journal on Scientific Computing, Vol. 21, Issue 2 https://doi.org/10.1137/S1064827598337282	journal	January 1999
Rectified deep neural networks overcome the curse of dimensionality for nonsmooth value functions in zero-sum games of nonlinear stiff systems Reisinger, Christoph; Zhang, Yufei Analysis and Applications, Vol. 18, Issue 06 https://doi.org/10.1142/S0219530520500116	journal	July 2020
Algorithms for overcoming the curse of dimensionality for certain Hamilton–Jacobi equations arising in control theory and elsewhere Darbon, Jérôme; Osher, Stanley Research in the Mathematical Sciences, Vol. 3, Issue 1 https://doi.org/10.1186/s40687-016-0068-7	journal	September 2016
Faster Convergence Rates of Relaxed Peaceman-Rachford and ADMM Under Regularity Assumptions Davis, Damek; Yin, Wotao Mathematics of Operations Research, Vol. 42, Issue 3 https://doi.org/10.1287/moor.2016.0827	journal	August 2017
Adaptive Control Processes Bellman, Richard E. Princeton University Press https://doi.org/10.1515/9781400874668	book	December 1961
Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers Boyd, Stephen; Parikh, Neal; Chu, Eric https://doi.org/10.1561/2200000016	book	January 2011
Hamilton-Jacobi-Bellman equations for fuzzy-dual optimization Zhong, Lunlong; Mora-Camino, Felix; Bouadi, Hakim 2017 36th Chinese Control Conference (CCC) https://doi.org/10.23919/ChiCC.2017.8027369	conference	July 2017
Reachability-Based Safety and Goal Satisfaction of Unmanned Aerial Platoons on Air Highways Chen, Mo; Hu, Qie; Fisac, Jaime F. Journal of Guidance, Control, and Dynamics, Vol. 40, Issue 6 https://doi.org/10.2514/1.G000774	journal	June 2017

Similar Records

Leveraging Multitime Hamilton–Jacobi PDEs for Certain Scientific Machine Learning Problems

Journal Article · 2024 · SIAM Journal on Scientific Computing · OSTI ID:2441157

Chen, Paula; Meng, Tingwei; Zou, Zongren; +2 more

Security-Constrained Unit Commitment with Linearized AC Optimal Power Flow.

Journal Article · 2015 · OSTI ID:1427190

Watson, Jean-Paul; Silva Monroy, Cesar Augusto; Castillo, Anya; +2 more

Reduced-complexity numerical method for optimal gate synthesis

Journal Article · 2010 · Physical Review. A · OSTI ID:21450737

Sridharan, Srinivas; James, Matthew R; Gu, Mile; +1 more

Related Subjects

97 MATHEMATICS AND COMPUTING
Field-programmable gate arrays (FPGAs)
Grid-free numerical methods
Hamilton-Jacobi partial differential equations
High dimensions
Optimal control

Title: Lax-Oleinik-Type Formulas and Efficient Algorithms for Certain High-Dimensional Optimal Control Problems

Citation Formats

References (71)

Similar Records

Related Subjects