Numerical optimization for symmetric tensor decomposition

Kolda, Tamara G.

doi:10.1007/s10107-015-0895-0

Numerical optimization for symmetric tensor decomposition

Journal Article · Sat Apr 11 00:00:00 EDT 2015 · Mathematical Programming

DOI:https://doi.org/10.1007/s10107-015-0895-0· OSTI ID:1497638

Kolda, Tamara G. ^[1]

Sandia National Lab. (SNL-CA), Livermore, CA (United States)

We consider the problem of decomposing a real-valued symmetric tensor as the sum of outer products of real-valued vectors. Algebraic methods exist for computing complex-valued decompositions of symmetric tensors, but here we focus on real-valued decompositions, both unconstrained and nonnegative. We discuss when solutions exist and how to formulate the mathematical program. Numerical results show the properties of the proposed formulations (including one that ignores symmetry) on a set of test problems and illustrate that these straightforward formulations can be effective even though the problem is nonconvex.

View Accepted Manuscript (DOE)

Research Organization:: Sandia National Lab. (SNL-CA), Livermore, CA (United States)

Sponsoring Organization:: USDOE National Nuclear Security Administration (NNSA)

Grant/Contract Number:: AC04-94AL85000

OSTI ID:: 1497638

Report Number(s):: SAND2014--18405J; 672377

Journal Information:: Mathematical Programming, Journal Name: Mathematical Programming Journal Issue: 1 Vol. 151; ISSN 0025-5610

Publisher:: SpringerCopyright Statement

Country of Publication:: United States

Language:: English

References (41)

Towards a standardized notation and terminology in multiway analysis Kiers, Henk A. L. Journal of Chemometrics, Vol. 14, Issue 3 https://doi.org/10.1002/1099-128x(200005/06)14:3<105::aid-cem582>3.0.co;2-i	journal	January 2000
Scalable tensor factorizations for incomplete data Acar, Evrim; Dunlavy, Daniel M.; Kolda, Tamara G. Chemometrics and Intelligent Laboratory Systems, Vol. 106, Issue 1 https://doi.org/10.1016/j.chemolab.2010.08.004	journal	March 2011
Monotonically convergent algorithms for symmetric tensor approximation Regalia, Phillip A. Linear Algebra and its Applications, Vol. 438, Issue 2 https://doi.org/10.1016/j.laa.2011.10.033	journal	January 2013
Scalable Tensor Factorizations with Missing Data Acar, Evrim; Dunlavy, Daniel M.; Kolda, Tamara G. Proceedings of the 2010 SIAM International Conference on Data Mining https://doi.org/10.1137/1.9781611972801.61	conference	December 2013
Jacobi Algorithm for the Best Low Multilinear Rank Approximation of Symmetric Tensors Ishteva, Mariya; Absil, P. -A.; Van Dooren, Paul SIAM Journal on Matrix Analysis and Applications, Vol. 34, Issue 2 https://doi.org/10.1137/11085743x	journal	January 2013
Symmetric tensors and symmetric tensor rank Comon, Pierre; Golub, Gene; Lim, Lek-Heng arXiv https://doi.org/10.48550/arxiv.0802.1681	preprint	January 2008
Symmetric tensor decomposition Brachat, Jerome; Comon, Pierre; Mourrain, Bernard arXiv https://doi.org/10.48550/arxiv.0901.3706	text	January 2009
Eigenvectors of tensors and algorithms for Waring decomposition Oeding, Luke; Ottaviani, Giorgio arXiv https://doi.org/10.48550/arxiv.1103.0203	text	January 2011
An Adaptive Shifted Power Method for Computing Generalized Tensor Eigenpairs Kolda, Tamara G.; Mayo, Jackson R. arXiv https://doi.org/10.48550/arxiv.1401.1183	text	January 2014
A three-step algorithm for CANDECOMP/PARAFAC analysis of large data sets with multicollinearity Kiers, Henk A. L. Journal of Chemometrics, Vol. 12, Issue 3 https://doi.org/10.1002/(SICI)1099-128X(199805/06)12:3<155::AID-CEM502>3.0.CO;2-5	journal	May 1998
Towards a standardized notation and terminology in multiway analysis Kiers, Henk A. L. Journal of Chemometrics, Vol. 14, Issue 3 https://doi.org/10.1002/1099-128X(200005/06)14:3<105::AID-CEM582>3.0.CO;2-I	journal	January 2000
On the uniqueness of multilinear decomposition ofN-way arrays Sidiropoulos, Nicholas D.; Bro, Rasmus Journal of Chemometrics, Vol. 14, Issue 3 https://doi.org/10.1002/1099-128X(200005/06)14:3<229::AID-CEM587>3.0.CO;2-N	journal	January 2000
A scalable optimization approach for fitting canonical tensor decompositions Acar, Evrim; Dunlavy, Daniel M.; Kolda, Tamara G. Journal of Chemometrics, Vol. 25, Issue 2 https://doi.org/10.1002/cem.1335	journal	January 2011
The Expression of a Tensor or a Polyadic as a Sum of Products Hitchcock, Frank L. Journal of Mathematics and Physics, Vol. 6, Issue 1-4 https://doi.org/10.1002/sapm192761164	journal	April 1927
Multiple Invariants and Generalized Rank of a P-Way Matrix or Tensor Hitchcock, Frank L. Journal of Mathematics and Physics, Vol. 7, Issue 1-4 https://doi.org/10.1002/sapm19287139	journal	April 1928
Analysis of individual differences in multidimensional scaling via an n-way generalization of “Eckart-Young” decomposition Carroll, J. Douglas; Chang, Jih-Jie Psychometrika, Vol. 35, Issue 3 https://doi.org/10.1007/BF02310791	journal	September 1970
The Assumption of Proportional Components when Candecomp is Applied to Symmetric Matrices in the Context of Indscal Dosse, Mohammed Bennani; Berge, Jos M. F. Psychometrika, Vol. 73, Issue 2 https://doi.org/10.1007/s11336-007-9044-x	journal	January 2008
Recent developments in CANDECOMP/PARAFAC algorithms: a critical review Faber, Nicolaas (Klaas) M.; Bro, Rasmus; Hopke, Philip K. Chemometrics and Intelligent Laboratory Systems, Vol. 65, Issue 1 https://doi.org/10.1016/S0169-7439(02)00089-8	journal	January 2003
A comparison of algorithms for fitting the PARAFAC model Tomasi, Giorgio; Bro, Rasmus Computational Statistics & Data Analysis, Vol. 50, Issue 7 https://doi.org/10.1016/j.csda.2004.11.013	journal	April 2006
Eigenvalues of a real supersymmetric tensor Qi, Liqun Journal of Symbolic Computation, Vol. 40, Issue 6 https://doi.org/10.1016/j.jsc.2005.05.007	journal	December 2005
Computing symmetric rank for symmetric tensors Bernardi, Alessandra; Gimigliano, Alessandro; Idà, Monica Journal of Symbolic Computation, Vol. 46, Issue 1 https://doi.org/10.1016/j.jsc.2010.08.001	journal	January 2011
Eigenvectors of tensors and algorithms for Waring decomposition Oeding, Luke; Ottaviani, Giorgio Journal of Symbolic Computation, Vol. 54 https://doi.org/10.1016/j.jsc.2012.11.005	journal	July 2013
Symmetric tensor decomposition Brachat, Jerome; Comon, Pierre; Mourrain, Bernard Linear Algebra and its Applications, Vol. 433, Issue 11-12 https://doi.org/10.1016/j.laa.2010.06.046	journal	December 2010
Singular Values and Eigenvalues of Tensors: A Variational Approach No authors listed 1st IEEE International Workshop on Computational Advances in Multi-Sensor Adaptive Processing, 2005. https://doi.org/10.1109/CAMAP.2005.1574201	conference	January 2005
Best rank (R, R, R) super-symmetric tensor approximation-a continuous-time approach Cambre, J.; De Lathauwer, L.; De Moor, B. Proceedings of the IEEE Signal Processing Workshop on Higher-Order Statistics. SPW-HOS '99 https://doi.org/10.1109/HOST.1999.778734	conference	January 1999
Efficiently Computing Tensor Eigenvalues on a GPU Ballard, Grey; Kolda, Tamara; Plantenga, Todd Distributed Processing, Workshops and Phd Forum (IPDPSW), 2011 IEEE International Symposium on Parallel and Distributed Processing Workshops and Phd Forum https://doi.org/10.1109/IPDPS.2011.287	conference	May 2011
Monotonic convergence of fixed-point algorithms for ICA Regalia, P. A.; Kofidis, E. IEEE Transactions on Neural Networks, Vol. 14, Issue 4 https://doi.org/10.1109/TNN.2003.813843	journal	July 2003
Symmetric Tensors and Symmetric Tensor Rank Comon, Pierre; Golub, Gene; Lim, Lek-Heng SIAM Journal on Matrix Analysis and Applications, Vol. 30, Issue 3 https://doi.org/10.1137/060661569	journal	January 2008
Efficient MATLAB Computations with Sparse and Factored Tensors Bader, Brett W.; Kolda, Tamara G. SIAM Journal on Scientific Computing, Vol. 30, Issue 1 https://doi.org/10.1137/060676489	journal	January 2008
Tensor Decompositions and Applications Kolda, Tamara G.; Bader, Brett W. SIAM Review, Vol. 51, Issue 3 https://doi.org/10.1137/07070111X	journal	August 2009
Shifted Power Method for Computing Tensor Eigenpairs Kolda, Tamara G.; Mayo, Jackson R. SIAM Journal on Matrix Analysis and Applications, Vol. 32, Issue 4 https://doi.org/10.1137/100801482	journal	October 2011
Jacobi Algorithm for the Best Low Multilinear Rank Approximation of Symmetric Tensors Ishteva, Mariya; Absil, P. -A.; Van Dooren, Paul SIAM Journal on Matrix Analysis and Applications, Vol. 34, Issue 2 https://doi.org/10.1137/11085743X	journal	January 2013
Nonnegative Tensor Factorization, Completely Positive Tensors, and a Hierarchical Elimination Algorithm Qi, Liqun; Xu, Changqing; Xu, Yi SIAM Journal on Matrix Analysis and Applications, Vol. 35, Issue 4 https://doi.org/10.1137/13092232X	journal	January 2014
Semidefinite Relaxations for Best Rank-1 Tensor Approximations Nie, Jiawang; Wang, Li SIAM Journal on Matrix Analysis and Applications, Vol. 35, Issue 3 https://doi.org/10.1137/130935112	journal	January 2014
An Adaptive Shifted Power Method for Computing Generalized Tensor Eigenpairs Kolda, Tamara G.; Mayo, Jackson R. SIAM Journal on Matrix Analysis and Applications, Vol. 35, Issue 4 https://doi.org/10.1137/140951758	journal	January 2014
All Real Eigenvalues of Symmetric Tensors Cui, Chun-Feng; Dai, Yu-Hong; Nie, Jiawang SIAM Journal on Matrix Analysis and Applications, Vol. 35, Issue 4 https://doi.org/10.1137/140962292	journal	January 2014
SNOPT: An SQP Algorithm for Large-Scale Constrained Optimization Gill, Philip E.; Murray, Walter; Saunders, Michael A. SIAM Review, Vol. 47, Issue 1 https://doi.org/10.1137/S0036144504446096	journal	January 2005
On the Best Rank-1 Approximation of Higher-Order Supersymmetric Tensors Kofidis, Eleftherios; Regalia, Phillip A. SIAM Journal on Matrix Analysis and Applications, Vol. 23, Issue 3 https://doi.org/10.1137/S0895479801387413	journal	January 2002
On the Best Rank-1 and Rank-( R ₁ , R ₂ ,. . ., R _N ) Approximation of Higher-Order Tensors De Lathauwer, Lieven; De Moor, Bart; Vandewalle, Joos SIAM Journal on Matrix Analysis and Applications, Vol. 21, Issue 4 https://doi.org/10.1137/S0895479898346995	journal	January 2000
Algorithm 862: MATLAB tensor classes for fast algorithm prototyping Bader, Brett W.; Kolda, Tamara G. ACM Transactions on Mathematical Software, Vol. 32, Issue 4 https://doi.org/10.1145/1186785.1186794	journal	December 2006
An unconstrained optimization approach for finding real eigenvalues of even order symmetric tensors Han, Lixing Numerical Algebra, Control and Optimization, Vol. 3, Issue 3 https://doi.org/10.3934/naco.2013.3.583	journal	July 2013

Cited By (13)

Joint Embedding of Graphs Wang, Shangsi; Arroyo, Jesus; Vogelstein, Joshua T. IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 43, Issue 4 https://doi.org/10.1109/tpami.2019.2948619	journal	April 2021
Low Rank Symmetric Tensor Approximations Nie, Jiawang SIAM Journal on Matrix Analysis and Applications, Vol. 38, Issue 4 https://doi.org/10.1137/16m1107528	journal	January 2017
Completely Positive Tensors and Multi-Hypergraphs Xu, Changqing; Luo, Ziyan; Qi, Liqun arXiv https://doi.org/10.48550/arxiv.1508.04204	preprint	January 2015
On {0,1} CP Tensors and CP Multi-hypergraphs Xu, Changqing; Chen, Zhibing; Qi, Liqun arXiv https://doi.org/10.48550/arxiv.1611.09206	preprint	January 2016
Validating Quantum-Classical Programming Models with Tensor Network Simulations McCaskey, Alexander; Dumitrescu, Eugene; Chen, Mengsu arXiv https://doi.org/10.48550/arxiv.1807.07914	text	January 2018
Subspace power method for symmetric tensor decomposition and generalized PCA Kileel, Joe; Pereira, João M. arXiv https://doi.org/10.48550/arxiv.1912.04007	preprint	January 2019
Symmetry Breaking in Symmetric Tensor Decomposition Arjevani, Yossi; Bruna, Joan; Field, Michael arXiv https://doi.org/10.48550/arxiv.2103.06234	preprint	January 2021
Separable symmetric tensors and separable anti-symmetric tensors Xu, Changqing arXiv https://doi.org/10.48550/arxiv.2202.12792	preprint	January 2022
Copositive tensor detection and its applications in physics and hypergraphs Chen, Haibin; Huang, Zheng-Hai; Qi, Liqun Computational Optimization and Applications, Vol. 69, Issue 1 https://doi.org/10.1007/s10589-017-9938-1	journal	August 2017
Completely positive tensor recovery with minimal nuclear value Zhou, Anwa; Fan, Jinyan Computational Optimization and Applications, Vol. 70, Issue 2 https://doi.org/10.1007/s10589-018-0003-5	journal	April 2018
The sparsest solutions to Z-tensor complementarity problems Luo, Ziyan; Qi, Liqun; Xiu, Naihua Optimization Letters, Vol. 11, Issue 3 https://doi.org/10.1007/s11590-016-1013-9	journal	February 2016
Validating quantum-classical programming models with tensor network simulations McCaskey, Alexander; Dumitrescu, Eugene; Chen, Mengsu PLOS ONE, Vol. 13, Issue 12 https://doi.org/10.1371/journal.pone.0206704	journal	December 2018
The Sparsest Solutions to $Z$-Tensor Complementarity Problems Luo, Ziyan; Qi, Liqun; Xiu, Naihua arXiv https://doi.org/10.48550/arxiv.1505.00993	preprint	January 2015