A continuous analogue of the tensor-train decomposition

Gorodetsky, Alex; Karaman, Sertac; Marzouk, Youssef

doi:10.1016/j.cma.2018.12.015

A continuous analogue of the tensor-train decomposition

Journal Article · Wed Dec 19 00:00:00 EST 2018 · Computer Methods in Applied Mechanics and Engineering

DOI:https://doi.org/10.1016/j.cma.2018.12.015· OSTI ID:1610934

^[1]; Karaman, Sertac ^[2]; Marzouk, Youssef ^[2]

Univ. of Michigan, Ann Arbor, MI (United States); DOE/OSTI
Massachusetts Inst. of Technology (MIT), Cambridge, MA (United States)

We develop new approximation algorithms and data structures for representing and computing with multivariate functions using the functional tensor-train (FT), a continuous extension of the tensor-train (TT) decomposition. The FT represents functions using a tensor-train ansatz by replacing the three-dimensional TT cores with univariate matrix-valued functions. The main contribution of this paper is a framework to compute the FT that employs adaptive approximations of univariate fibers, and that is not connected to any tensorized discretization. The algorithm can be coupled with any univariate linear or nonlinear approximation procedure. We demonstrate that this approach can generate multivariate function approximations that are several orders of magnitude more accurate, for the same cost, than those based on the conventional approach of compressing the coefficient tensor of a tensor-product basis. Our approach is in the spirit of other continuous computation packages such as Chebfun, and yields an algorithm which requires the computation of “continuous” matrix factorizations such as the LU and QR decompositions of vector-valued functions. To support these developments, we describe continuous versions of an approximate maximum-volume cross approximation algorithm and of a rounding algorithm that re-approximates an FT by one of lower ranks. We demonstrate that our technique improves accuracy and robustness, compared to TT and quantics-TT approaches with fixed parameterizations, of high-dimensional integration, differentiation, and approximation of functions with local features such as discontinuities and other nonlinearities.

View Accepted Manuscript (DOE)

Research Organization:: Massachusetts Inst. of Technology (MIT), Cambridge, MA (United States)

Sponsoring Organization:: USDOE; USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR)

Grant/Contract Number:: SC0007099

OSTI ID:: 1610934

Alternate ID(s):: OSTI ID: 1636194

Journal Information:: Computer Methods in Applied Mechanics and Engineering, Journal Name: Computer Methods in Applied Mechanics and Engineering Vol. 347; ISSN 0045-7825

Publisher:: ElsevierCopyright Statement

Country of Publication:: United States

Language:: English

References (39)

Approximation of boundary element matrices Bebendorf, Mario Numerische Mathematik, Vol. 86, Issue 4 https://doi.org/10.1007/pl00005410	journal	October 2000
Quasioptimality of maximum-volume cross interpolation of tensors Savostyanov, Dmitry V. Linear Algebra and its Applications, Vol. 458 https://doi.org/10.1016/j.laa.2014.06.006	journal	October 2014
CUR matrix decompositions for improved data analysis Mahoney, Michael W.; Drineas, Petros Proceedings of the National Academy of Sciences, Vol. 106, Issue 3 https://doi.org/10.1073/pnas.0803205106	journal	January 2009
The maximal-volume concept in approximation by low-rank matrices Goreinov, S. A.; Tyrtyshnikov, E. E. Structured Matrices in Mathematics, Computer Science, and Engineering I https://doi.org/10.1090/conm/280/4620	book	January 2001
Fast adaptive interpolation of multi-dimensional arrays in tensor train format Savostyanov, Dmitry; Oseledets, Ivan The 2011 International Workshop on Multidimensional (nD) Systems https://doi.org/10.1109/nds.2011.6076873	conference	September 2011
Optimal CUR Matrix Decompositions Boutsidis, Christos; Woodruff, David P. SIAM Journal on Computing, Vol. 46, Issue 2 https://doi.org/10.1137/140977898	journal	January 2017
Fast Monte Carlo Algorithms for Matrices II: Computing a Low-Rank Approximation to a Matrix Drineas, Petros; Kannan, Ravi; Mahoney, Michael W. SIAM Journal on Computing, Vol. 36, Issue 1 https://doi.org/10.1137/s0097539704442696	journal	January 2006
QTT approximation of elliptic solution operators in higher dimensions Khoromskij, B. N.; Oseledets, I. V. Russian Journal of Numerical Analysis and Mathematical Modelling, Vol. 26, Issue 3 https://doi.org/10.1515/rjnamm.2011.017	journal	January 2011
Efficient Localization of Discontinuities in Complex Computational Simulations Gorodetsky, Alex A.; Marzouk, Youssef M. arXiv https://doi.org/10.48550/arxiv.1402.2845	preprint	January 2014
Pseudo-skeleton approximations by matrices of maximal volume Goreinov, S. A.; Zamarashkin, N. L.; Tyrtyshnikov, E. E. Mathematical Notes, Vol. 62, Issue 4 https://doi.org/10.1007/BF02358985	journal	October 1997
Approximation of boundary element matrices Bebendorf, Mario Numerische Mathematik, Vol. 86, Issue 4 https://doi.org/10.1007/PL00005410	journal	October 2000
O(dlog N)-Quantics Approximation of N-d Tensors in High-Dimensional Numerical Modeling Khoromskij, Boris N. Constructive Approximation, Vol. 34, Issue 2 https://doi.org/10.1007/s00365-011-9131-1	journal	April 2011
Constructive Representation of Functions in Low-Rank Tensor Formats Oseledets, I. V. Constructive Approximation, Vol. 37, Issue 1 https://doi.org/10.1007/s00365-012-9175-x	journal	December 2012
A theory of pseudoskeleton approximations Goreinov, S. A.; Tyrtyshnikov, E. E.; Zamarashkin, N. L. Linear Algebra and its Applications, Vol. 261, Issue 1-3 https://doi.org/10.1016/S0024-3795(96)00301-1	journal	August 1997
Non-intrusive low-rank separated approximation of high-dimensional stochastic models Doostan, Alireza; Validi, AbdoulAhad; Iaccarino, Gianluca Computer Methods in Applied Mechanics and Engineering, Vol. 263 https://doi.org/10.1016/j.cma.2013.04.003	journal	August 2013
Discontinuity detection in multivariate space for stochastic simulations Archibald, Rick; Gelb, Anne; Saxena, Rishu Journal of Computational Physics, Vol. 228, Issue 7 https://doi.org/10.1016/j.jcp.2009.01.001	journal	April 2009
Gradient-based optimization for regression in the functional tensor-train format Gorodetsky, Alex A.; Jakeman, John D. Journal of Computational Physics, Vol. 374 https://doi.org/10.1016/j.jcp.2018.08.010	journal	December 2018
TT-cross approximation for multidimensional arrays Oseledets, Ivan; Tyrtyshnikov, Eugene Linear Algebra and its Applications, Vol. 432, Issue 1 https://doi.org/10.1016/j.laa.2009.07.024	journal	January 2010
Function space requirements for the single-electron functions within the multiparticle Schrödinger equation Mohlenkamp, Martin J. Journal of Mathematical Physics, Vol. 54, Issue 6 https://doi.org/10.1063/1.4811396	journal	June 2013
Accelerating Asymptotically Exact MCMC for Computationally Intensive Models via Local Approximations Conrad, Patrick R.; Marzouk, Youssef M.; Pillai, Natesh S. Journal of the American Statistical Association, Vol. 111, Issue 516 https://doi.org/10.1080/01621459.2015.1096787	journal	October 2016
Low-rank tensor integration for Gaussian filtering of continuous time nonlinear systems Gorodetsky, Alex A.; Karaman, Sertac; Marzouk, Youssef M. 2017 IEEE 56th Annual Conference on Decision and Control (CDC) https://doi.org/10.1109/CDC.2017.8264064	conference	December 2017
Roots of Polynomials Expressed in Terms of Orthogonal Polynomials Day, David; Romero, Louis SIAM Journal on Numerical Analysis, Vol. 43, Issue 5 https://doi.org/10.1137/040609847	journal	January 2005
Multivariate Regression and Machine Learning with Sums of Separable Functions Beylkin, Gregory; Garcke, Jochen; Mohlenkamp, Martin J. SIAM Journal on Scientific Computing, Vol. 31, Issue 3 https://doi.org/10.1137/070710524	journal	January 2009
Tensor-Train Decomposition Oseledets, I. V. SIAM Journal on Scientific Computing, Vol. 33, Issue 5 https://doi.org/10.1137/090752286	journal	January 2011
An Extension of Chebfun to Two Dimensions Townsend, Alex; Trefethen, Lloyd N. SIAM Journal on Scientific Computing, Vol. 35, Issue 6 https://doi.org/10.1137/130908002	journal	January 2013
A Least-Squares Method for Sparse Low Rank Approximation of Multivariate Functions Chevreuil, M.; Lebrun, R.; Nouy, A. SIAM/ASA Journal on Uncertainty Quantification, Vol. 3, Issue 1 https://doi.org/10.1137/13091899X	journal	January 2015
Efficient Localization of Discontinuities in Complex Computational Simulations Gorodetsky, Alex; Marzouk, Youssef SIAM Journal on Scientific Computing, Vol. 36, Issue 6 https://doi.org/10.1137/140953137	journal	January 2014
Regularized Computation of Approximate Pseudoinverse of Large Matrices Using Low-Rank Tensor Train Decompositions Lee, Namgil; Cichocki, Andrzej SIAM Journal on Matrix Analysis and Applications, Vol. 37, Issue 2 https://doi.org/10.1137/15M1028479	journal	January 2016
Spectral Tensor-Train Decomposition Bigoni, Daniele; Engsig-Karup, Allan P.; Marzouk, Youssef M. SIAM Journal on Scientific Computing, Vol. 38, Issue 4 https://doi.org/10.1137/15M1036919	journal	January 2016
Chebfun in Three Dimensions Hashemi, Behnam; Trefethen, Lloyd N. SIAM Journal on Scientific Computing, Vol. 39, Issue 5 https://doi.org/10.1137/16M1083803	journal	January 2017
Polynomial Fitting for Edge Detection in Irregularly Sampled Signals and Images Archibald, Rick; Gelb, Anne; Yoon, Jungho SIAM Journal on Numerical Analysis, Vol. 43, Issue 1 https://doi.org/10.1137/S0036142903435259	journal	January 2005
An Extension of MATLAB to Continuous Functions and Operators Battles, Zachary; Trefethen, Lloyd N. SIAM Journal on Scientific Computing, Vol. 25, Issue 5 https://doi.org/10.1137/S1064827503430126	journal	January 2004
High-dimensional stochastic optimal control using continuous tensor decompositions Gorodetsky, Alex; Karaman, Sertac; Marzouk, Youssef The International Journal of Robotics Research, Vol. 37, Issue 2-3 https://doi.org/10.1177/0278364917753994	journal	February 2018
High-dimensional additive modeling Meier, Lukas; van de Geer, Sara; Bühlmann, Peter The Annals of Statistics, Vol. 37, Issue 6B https://doi.org/10.1214/09-AOS692	journal	December 2009
TT-GMRES: solution to a linear system in the structured tensor format Dolgov, S. V. Russian Journal of Numerical Analysis and Mathematical Modelling, Vol. 28, Issue 2 https://doi.org/10.1515/rnam-2013-0009	journal	January 2013
Efficient High-Dimensional Stochastic Optimal Motion Control using Tensor-Train Decomposition Gorodetsky, Alex; Karaman, Sertac; Marzouk, Youssef Robotics: Science and Systems 2015, Robotics: Science and Systems XI https://doi.org/10.15607/RSS.2015.XI.015	conference	July 2015
Quantification of Uncertainty from High-Dimensional Scattered data via Polynomial Approximation Mathelin, Lionel International Journal for Uncertainty Quantification, Vol. 4, Issue 3 https://doi.org/10.1615/Int.J.UncertaintyQuantification.2014008084	journal	January 2014
Quantics-TT Collocation Approximation of Parameter-Dependent and Stochastic Elliptic PDEs Khoromskij, B. N.; Oseledets, I. Computational Methods in Applied Mathematics, Vol. 10, Issue 4 https://doi.org/10.2478/cmam-2010-0023	journal	January 2010
Multilevel-Multidelity Approaches for Forward UQ in the DARPA SEQUOIA project Eldred, Michael S.; Geraci, Gianluca; Gorodetsky, Alex 2018 AIAA Non-Deterministic Approaches Conference https://doi.org/10.2514/6.2018-1179	conference	January 2018

Cited By (3)

High-Dimensional Stochastic Optimal Control using Continuous Tensor Decompositions Gorodetsky, Alex A.; Karaman, Sertac; Marzouk, Youssef M. arXiv https://doi.org/10.48550/arxiv.1611.04706	preprint	January 2016
On the compressibility of tensors Shi, Tianyi; Townsend, Alex arXiv https://doi.org/10.48550/arxiv.1812.09576	preprint	January 2018
Machine learning and the physical sciences Carleo, Giuseppe; Cirac, Ignacio; Cranmer, Kyle Reviews of Modern Physics, Vol. 91, Issue 4 https://doi.org/10.1103/revmodphys.91.045002	journal	December 2019

Similar Records

Gradient-based Optimization for Regression in the Functional Tensor-Train Format

Technical Report · Tue Dec 05 23:00:00 EST 2017 · OSTI ID:1733296

An Incremental Tensor Train Decomposition Algorithm

Journal Article · Mon Mar 25 20:00:00 EDT 2024 · SIAM Journal on Scientific Computing · OSTI ID:2283547

Tensor-Train Split-Operator Fourier Transform (TT-SOFT) Method: Multidimensional Nonadiabatic Quantum Dynamics

Journal Article · Mon Jul 31 20:00:00 EDT 2017 · Journal of Chemical Theory and Computation · OSTI ID:1480241

Related Subjects

42 ENGINEERING
chebfun
high-dimensional approximation
tensor decompositions
tensor-train

A continuous analogue of the tensor-train decomposition

Citation Formats

References (39)

Cited By (3)

Similar Records

Related Subjects