Search Menu
DOE PAGES title logo U.S. Department of Energy
Office of Scientific and Technical Information
Search Scholarly Publications

Title: High-dimensional stochastic optimal control using continuous tensor decompositions

Abstract

Motion planning and control problems are embedded and essential in almost all robotics applications. These problems are often formulated as stochastic optimal control problems and solved using dynamic programming algorithms. Unfortunately, most existing algorithms that guarantee convergence to optimal solutions suffer from the curse of dimensionality: the run time of the algorithm grows exponentially with the dimension of the state space of the system. We propose novel dynamic programming algorithms that alleviate the curse of dimensionality in problems that exhibit certain low-rank structure. The proposed algorithms are based on continuous tensor decompositions recently developed by the authors. Essentially, the algorithms represent high-dimensional functions (e.g. the value function) in a compressed format, and directly perform dynamic programming computations (e.g. value iteration, policy iteration) in this format. Under certain technical assumptions, the new algorithms guarantee convergence towards optimal solutions with arbitrary precision. Furthermore, the run times of the new algorithms scale polynomially with the state dimension and polynomially with the ranks of the value function. This approach realizes substantial computational savings in “compressible” problem instances, where value functions admit low-rank approximations. We demonstrate the new algorithms in a wide range of problems, including a simulated six-dimensional agile quadcopter maneuvering example and amore » seven-dimensional aircraft perching example. In some of these examples, we estimate computational savings of up to 10 orders of magnitude over standard value iteration algorithms. Finally, we further demonstrate the algorithms running in real time on board a quadcopter during a flight experiment under motion capture.« less

Authors:
ORCiD logo [1];  [1];  [1]
+ Show Author Affiliations
  1. Department of Aeronautics and Astronautics, Massachusetts Institute of Technology, Cambridge, MA, USA
Publication Date:
Research Org.:
Massachusetts Inst. of Technology (MIT), Cambridge, MA (United States)
Sponsoring Org.:
USDOE Office of Science (SC)
OSTI Identifier:
1427309
Alternate Identifier(s):
OSTI ID: 1541871
Grant/Contract Number:  
SC0007099
Resource Type:
Published Article
Journal Name:
International Journal of Robotics Research
Additional Journal Information:
Journal Name: International Journal of Robotics Research Journal Volume: 37 Journal Issue: 2-3; Journal ID: ISSN 0278-3649
Publisher:
SAGE
Country of Publication:
United States
Language:
English
Subject:
42 ENGINEERING; robotics; stochastic optimal control; motion planning; dynamic programming; tensor decompositions

Citation Formats

Gorodetsky, Alex, Karaman, Sertac, and Marzouk, Youssef. High-dimensional stochastic optimal control using continuous tensor decompositions. United States: N. p., 2018. Web. doi:10.1177/0278364917753994.
Copy to clipboard
Gorodetsky, Alex, Karaman, Sertac, & Marzouk, Youssef. High-dimensional stochastic optimal control using continuous tensor decompositions. United States. https://doi.org/10.1177/0278364917753994
Copy to clipboard
Gorodetsky, Alex, Karaman, Sertac, and Marzouk, Youssef. Mon . "High-dimensional stochastic optimal control using continuous tensor decompositions". United States. https://doi.org/10.1177/0278364917753994.
Copy to clipboard
@article{osti_1427309,
title = {High-dimensional stochastic optimal control using continuous tensor decompositions},
author = {Gorodetsky, Alex and Karaman, Sertac and Marzouk, Youssef},
abstractNote = {Motion planning and control problems are embedded and essential in almost all robotics applications. These problems are often formulated as stochastic optimal control problems and solved using dynamic programming algorithms. Unfortunately, most existing algorithms that guarantee convergence to optimal solutions suffer from the curse of dimensionality: the run time of the algorithm grows exponentially with the dimension of the state space of the system. We propose novel dynamic programming algorithms that alleviate the curse of dimensionality in problems that exhibit certain low-rank structure. The proposed algorithms are based on continuous tensor decompositions recently developed by the authors. Essentially, the algorithms represent high-dimensional functions (e.g. the value function) in a compressed format, and directly perform dynamic programming computations (e.g. value iteration, policy iteration) in this format. Under certain technical assumptions, the new algorithms guarantee convergence towards optimal solutions with arbitrary precision. Furthermore, the run times of the new algorithms scale polynomially with the state dimension and polynomially with the ranks of the value function. This approach realizes substantial computational savings in “compressible” problem instances, where value functions admit low-rank approximations. We demonstrate the new algorithms in a wide range of problems, including a simulated six-dimensional agile quadcopter maneuvering example and a seven-dimensional aircraft perching example. In some of these examples, we estimate computational savings of up to 10 orders of magnitude over standard value iteration algorithms. Finally, we further demonstrate the algorithms running in real time on board a quadcopter during a flight experiment under motion capture.},
doi = {10.1177/0278364917753994},
journal = {International Journal of Robotics Research},
number = 2-3,
volume = 37,
place = {United States},
year = {Mon Mar 19 00:00:00 EDT 2018},
month = {Mon Mar 19 00:00:00 EDT 2018}
}
Copy to clipboard
Journal Article:
Free Publicly Available Full Text
Publisher's Version of Record
https://doi.org/10.1177/0278364917753994
Copyright Statement
Other availability
Search WorldCat Search WorldCat to find libraries that may hold this journal
Citation Metrics:
Cited by: 22 works
Citation information provided by
Web of Science
Save / Share:
Export Metadata
Save to My Library
You must Sign In or Create an Account in order to save documents to your library.

Works referenced in this record:

Chebfun: A New Kind of Numerical Computing
book, January 2010

A Multigrid Tutorial, Second Edition
book, January 2000

  • Briggs, William L.; Henson, Van Emden; McCormick, Steve F.
  • Other Titles in Applied Mathematics
  • DOI: 10.1137/1.9780898719505

Trajectory generation and control for precise aggressive maneuvers with quadrotors
journal, January 2012

  • Mellinger, Daniel; Michael, Nathan; Kumar, Vijay
  • The International Journal of Robotics Research, Vol. 31, Issue 5
  • DOI: 10.1177/0278364911434236

Sampling-based algorithms for optimal motion planning
journal, June 2011

  • Karaman, Sertac; Frazzoli, Emilio
  • The International Journal of Robotics Research, Vol. 30, Issue 7
  • DOI: 10.1177/0278364911406761

Randomized Kinodynamic Planning
journal, May 2001

  • LaValle, Steven M.; Kuffner, James J.
  • The International Journal of Robotics Research, Vol. 20, Issue 5
  • DOI: 10.1177/02783640122067453

SARSOP: Efficient Point-Based POMDP Planning by Approximating Optimally Reachable Belief Spaces
conference, June 2008

  • Kurniawati, Hanna; Hsu, David; Sun Lee, Wee
  • Robotics: Science and Systems 2008, Robotics: Science and Systems IV
  • DOI: 10.15607/RSS.2008.IV.009

Path Integral Formulation of Stochastic Optimal Control with Generalized Costs
journal, January 2014

CUR matrix decompositions for improved data analysis
journal, January 2009

  • Mahoney, Michael W.; Drineas, Petros
  • Proceedings of the National Academy of Sciences, Vol. 106, Issue 3
  • DOI: 10.1073/pnas.0803205106

Policy Iteration Adaptive Dynamic Programming Algorithm for Discrete-Time Nonlinear Systems
journal, March 2014

  • Liu, Derong; Wei, Qinglai
  • IEEE Transactions on Neural Networks and Learning Systems, Vol. 25, Issue 3
  • DOI: 10.1109/TNNLS.2013.2281663

An optimal one-way multigrid algorithm for discrete-time stochastic control
journal, January 1991

  • Chow, C. -S.; Tsitsiklis, J. N.
  • IEEE Transactions on Automatic Control, Vol. 36, Issue 8
  • DOI: 10.1109/9.133184

Finite-Approximation-Error-Based Discrete-Time Iterative Adaptive Dynamic Programming
journal, December 2014

  • Wei, Qinglai; Wang, Fei-Yue; Liu, Derong
  • IEEE Transactions on Cybernetics, Vol. 44, Issue 12
  • DOI: 10.1109/TCYB.2014.2354377

Solution of Linear Systems and Matrix Inversion in the TT-Format
journal, January 2012

  • Oseledets, I. V.; Dolgov, S. V.
  • SIAM Journal on Scientific Computing, Vol. 34, Issue 5
  • DOI: 10.1137/110833142

Analysis of individual differences in multidimensional scaling via an n-way generalization of “Eckart-Young” decomposition
journal, September 1970

  • Carroll, J. Douglas; Chang, Jih-Jie
  • Psychometrika, Vol. 35, Issue 3
  • DOI: 10.1007/BF02310791

Linear Hamilton Jacobi Bellman Equations in high dimensions
conference, December 2014

  • Horowitz, Matanya B.; Damle, Anil; Burdick, Joel W.
  • 2014 IEEE 53rd Annual Conference on Decision and Control (CDC), 53rd IEEE Conference on Decision and Control
  • DOI: 10.1109/CDC.2014.7040310

On the controllability of fixed-wing perching
conference, June 2009

Spectral Tensor-Train Decomposition
journal, January 2016

  • Bigoni, Daniele; Engsig-Karup, Allan P.; Marzouk, Youssef M.
  • SIAM Journal on Scientific Computing, Vol. 38, Issue 4
  • DOI: 10.1137/15M1036919

Tensor Spaces and Numerical Tensor Calculus
book, February 2012

Numerical Methods for Stochastic Control Problems in Continuous Time
journal, September 1990

  • Kushner, Harold J.
  • SIAM Journal on Control and Optimization, Vol. 28, Issue 5
  • DOI: 10.1137/0328056

Approximate Dynamic Programming: Solving the Curses of Dimensionality
book, August 2011

QTT approximation of elliptic solution operators in higher dimensions
journal, January 2011

  • Khoromskij, B. N.; Oseledets, I. V.
  • Russian Journal of Numerical Analysis and Mathematical Modelling, Vol. 26, Issue 3
  • DOI: 10.1515/rjnamm.2011.017

Nonlinear Control Synthesis by Convex Optimization
journal, February 2004

  • Prajna, S.; Parrilo, P. A.; Rantzer, A.
  • IEEE Transactions on Automatic Control, Vol. 49, Issue 2
  • DOI: 10.1109/TAC.2003.823000

TT-GMRES: solution to a linear system in the structured tensor format
journal, January 2013

  • Dolgov, S. V.
  • Russian Journal of Numerical Analysis and Mathematical Modelling, Vol. 28, Issue 2
  • DOI: 10.1515/rnam-2013-0009

CHOMP: Gradient optimization techniques for efficient motion planning
conference, May 2009

  • Ratliff, Nathan; Zucker, Matt; Bagnell, J. Andrew
  • 2009 IEEE International Conference on Robotics and Automation (ICRA)
  • DOI: 10.1109/ROBOT.2009.5152817

Probabilistic roadmaps for path planning in high-dimensional configuration spaces
journal, January 1996

  • Kavraki, L. E.; Svestka, P.; Latombe, J. -C.
  • IEEE Transactions on Robotics and Automation, Vol. 12, Issue 4
  • DOI: 10.1109/70.508439

TT-cross approximation for multidimensional arrays
journal, January 2010

  • Oseledets, Ivan; Tyrtyshnikov, Eugene
  • Linear Algebra and its Applications, Vol. 432, Issue 1
  • DOI: 10.1016/j.laa.2009.07.024

Planning Algorithms
book, January 2006

On the Probabilistic Foundations of Probabilistic Roadmap Planning
journal, July 2006

  • Hsu, David; Latombe, Jean-Claude; Kurniawati, Hanna
  • The International Journal of Robotics Research, Vol. 25, Issue 7
  • DOI: 10.1177/0278364906067174

Tensor rank is NP-complete
journal, December 1990

CHOMP: Covariant Hamiltonian optimization for motion planning
journal, August 2013

  • Zucker, Matt; Ratliff, Nathan; Dragan, Anca D.
  • The International Journal of Robotics Research, Vol. 32, Issue 9-10
  • DOI: 10.1177/0278364913488805

High-dimensional additive modeling
journal, December 2009

  • Meier, Lukas; van de Geer, Sara; Bühlmann, Peter
  • The Annals of Statistics, Vol. 37, Issue 6B
  • DOI: 10.1214/09-AOS692

Analysis of probabilistic roadmaps for path planning
journal, February 1998

  • Kavraki, L. E.; Kolountzakis, M. N.; Latombe, J. -C.
  • IEEE Transactions on Robotics and Automation, Vol. 14, Issue 1
  • DOI: 10.1109/70.660866

An Extension of Chebfun to Two Dimensions
journal, January 2013

  • Townsend, Alex; Trefethen, Lloyd N.
  • SIAM Journal on Scientific Computing, Vol. 35, Issue 6
  • DOI: 10.1137/130908002

Incomplete Cross Approximation in the Mosaic-Skeleton Method
journal, June 2000

Experiments in Fixed-Wing UAV Perching
conference, June 2012

  • Cory, Rick; Tedrake, Russ
  • AIAA Guidance, Navigation and Control Conference and Exhibit
  • DOI: 10.2514/6.2008-7256

An Extension of MATLAB to Continuous Functions and Operators
journal, January 2004

  • Battles, Zachary; Trefethen, Lloyd N.
  • SIAM Journal on Scientific Computing, Vol. 25, Issue 5
  • DOI: 10.1137/S1064827503430126

Efficient High-Dimensional Stochastic Optimal Motion Control using Tensor-Train Decomposition
conference, July 2015

  • Gorodetsky, Alex; Karaman, Sertac; Marzouk, Youssef
  • Robotics: Science and Systems 2015, Robotics: Science and Systems XI
  • DOI: 10.15607/RSS.2015.XI.015

Optimization-based Full Body Control for the DARPA Robotics Challenge: Optimization-Based Full Body Control For The DARPA Robotics Challenge
journal, January 2015

  • Feng, Siyuan; Whitman, Eric; Xinjilefu, X.
  • Journal of Field Robotics, Vol. 32, Issue 2
  • DOI: 10.1002/rob.21559

A theory of pseudoskeleton approximations
journal, August 1997

  • Goreinov, S. A.; Tyrtyshnikov, E. E.; Zamarashkin, N. L.
  • Linear Algebra and its Applications, Vol. 261, Issue 1-3
  • DOI: 10.1016/S0024-3795(96)00301-1

LQR-trees: Feedback Motion Planning via Sums-of-Squares Verification
journal, April 2010

  • Tedrake, Russ; Manchester, Ian R.; Tobenkin, Mark
  • The International Journal of Robotics Research, Vol. 29, Issue 8
  • DOI: 10.1177/0278364910369189

Tensor Networks and Hierarchical Tensors for the Solution of High-Dimensional Partial Differential Equations
journal, April 2016

  • Bachmayr, Markus; Schneider, Reinhold; Uschmajew, André
  • Foundations of Computational Mathematics, Vol. 16, Issue 6
  • DOI: 10.1007/s10208-016-9317-9

A New Scheme for the Tensor Representation
journal, October 2009

Path planning in expansive configuration spaces
conference, January 1997

  • Hsu, D.; Latombe, J. -C.; Motwani, R.
  • Proceedings of International Conference on Robotics and Automation
  • DOI: 10.1109/ROBOT.1997.619371

Tensor-Train Decomposition
journal, January 2011

  • Oseledets, I. V.
  • SIAM Journal on Scientific Computing, Vol. 33, Issue 5
  • DOI: 10.1137/090752286

Efficient algorithms for globally optimal trajectories
journal, January 1995

  • Tsitsiklis, J. N.
  • IEEE Transactions on Automatic Control, Vol. 40, Issue 9
  • DOI: 10.1109/9.412624

Cubature, Approximation, and Isotropy in the Hypercube
journal, January 2017

An Architecture for Online Affordance-based Perception and Whole-body Planning: An Architecture for Online Affordance-based Perception
journal, October 2014

  • Fallon, Maurice; Kuindersma, Scott; Karumanchi, Sisir
  • Journal of Field Robotics, Vol. 32, Issue 2
  • DOI: 10.1002/rob.21546

Tensor Rank and the Ill-Posedness of the Best Low-Rank Approximation Problem
journal, January 2008

  • de Silva, Vin; Lim, Lek-Heng
  • SIAM Journal on Matrix Analysis and Applications, Vol. 30, Issue 3
  • DOI: 10.1137/06066518X

Control synthesis and verification for a perching UAV using LQR-Trees
conference, December 2012

  • Moore, Joseph; Tedrake, Russ
  • 2012 IEEE 51st Annual Conference on Decision and Control (CDC), 2012 IEEE 51st IEEE Conference on Decision and Control (CDC)
  • DOI: 10.1109/CDC.2012.6425852

Minimum snap trajectory generation and control for quadrotors
conference, May 2011

  • Mellinger, Daniel; Kumar, Vijay
  • 2011 IEEE International Conference on Robotics and Automation (ICRA)
  • DOI: 10.1109/ICRA.2011.5980409
    Sort by:
    [ × clear filter / sort ]

    Similar Records in DOE PAGES and OSTI.GOV collections: