Estimating Higher-Order Moments Using Symmetric Tensor Decomposition

Sherman, Samantha; Kolda, Tamara G.

doi:10.1137/19M1299633

Estimating Higher-Order Moments Using Symmetric Tensor Decomposition

Journal Article · Wed Sep 16 00:00:00 EDT 2020 · SIAM Journal on Matrix Analysis and Applications

DOI:https://doi.org/10.1137/19M1299633· OSTI ID:1725841

Sherman, Samantha ^[1]; ^[2]

Univ. of Notre Dame, IN (United States)
Sandia National Lab. (SNL-CA), Livermore, CA (United States)

In this paper, we consider the problem of decomposing higher-order moment tensors, i.e., the sum of symmetric outer products of data vectors. Such a decomposition can be used to estimate the means in a Gaussian mixture model and for other applications in machine learning. The dth-order empirical moment tensor of a set of p observations of n variables is a symmetric d-way tensor. Our goal is to nd a low-rank tensor approximation comprising r $$\ll$$ p symmetric outer products. The challenge is that forming the empirical moment tensor costs O(pn^d) operations and O(n^d) storage, which may be prohibitively expensive; additionally, the algorithm to compute the low-rank approximation costs O(n^d) per iteration. Our contribution is avoiding formation of the moment tensor, computing the low-rank tensor approximation of the moment tensor implicitly using O(pnr) operations per iteration and no extra memory. This advance opens the door to more applications of higher-order moments since they can now be efficiently computed. We present numerical evidence of the computational savings and show an example of estimating the means for higher-order moments.

View Accepted Manuscript (DOE)

Research Organization:: Sandia National Laboratories (SNL-CA), Livermore, CA (United States)

Sponsoring Organization:: USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR) (SC-21); USDOE National Nuclear Security Administration (NNSA)

Grant/Contract Number:: AC04-94AL85000; NA0003525

OSTI ID:: 1725841

Report Number(s):: SAND--2020-11568J; 692160

Journal Information:: SIAM Journal on Matrix Analysis and Applications, Journal Name: SIAM Journal on Matrix Analysis and Applications Journal Issue: 3 Vol. 41; ISSN 0895-4798

Publisher:: SIAMCopyright Statement

Country of Publication:: United States

Language:: English

References (19)

Numerical optimization for symmetric tensor decomposition Kolda, Tamara G. Mathematical Programming, Vol. 151, Issue 1 https://doi.org/10.1007/s10107-015-0895-0	journal	April 2015
Generating Polynomials and Symmetric Tensor Decompositions Nie, Jiawang Foundations of Computational Mathematics, Vol. 17, Issue 2 https://doi.org/10.1007/s10208-015-9291-7	journal	November 2015
A comparison of algorithms for fitting the PARAFAC model Tomasi, Giorgio; Bro, Rasmus Computational Statistics & Data Analysis, Vol. 50, Issue 7 https://doi.org/10.1016/j.csda.2004.11.013	journal	April 2006
Anomaly detection in scientific data using joint statistical moments Aditya, Konduri; Kolla, Hemanth; Kegelmeyer, W. Philip Journal of Computational Physics, Vol. 387 https://doi.org/10.1016/j.jcp.2019.03.003	journal	June 2019
Multivariate skewness and kurtosis measures with an application in ICA Kollo, Tõnu Journal of Multivariate Analysis, Vol. 99, Issue 10 https://doi.org/10.1016/j.jmva.2008.02.033	journal	November 2008
Computing symmetric rank for symmetric tensors Bernardi, Alessandra; Gimigliano, Alessandro; Idà, Monica Journal of Symbolic Computation, Vol. 46, Issue 1 https://doi.org/10.1016/j.jsc.2010.08.001	journal	January 2011
Tensor decomposition in post-Hartree–Fock methods. I. Two-electron integrals and MP2 Benedikt, Udo; Auer, Alexander A.; Espig, Mike The Journal of Chemical Physics, Vol. 134, Issue 5 https://doi.org/10.1063/1.3514201	journal	February 2011
Estimation and Testing of Parameters in Multivariate Laplace Distribution Kollo, Tõnu; Srivastava, Muni S. Communications in Statistics - Theory and Methods, Vol. 33, Issue 10 https://doi.org/10.1081/STA-200031408	journal	January 2005
Independent component analysis and (simultaneous) third-order tensor diagonalization de Lathauwer, L.; de Moor, B.; Vandewalle, J. IEEE Transactions on Signal Processing, Vol. 49, Issue 10 https://doi.org/10.1109/78.950782	journal	January 2001
NPSA: Nonorthogonal Principal Skewness Analysis Geng, Xiurui; Wang, Lei IEEE Transactions on Image Processing, Vol. 29 https://doi.org/10.1109/TIP.2020.2984849	journal	January 2020
Symmetric Tensors and Symmetric Tensor Rank Comon, Pierre; Golub, Gene; Lim, Lek-Heng SIAM Journal on Matrix Analysis and Applications, Vol. 30, Issue 3 https://doi.org/10.1137/060661569	journal	January 2008
Efficient MATLAB Computations with Sparse and Factored Tensors Bader, Brett W.; Kolda, Tamara G. SIAM Journal on Scientific Computing, Vol. 30, Issue 1 https://doi.org/10.1137/060676489	journal	January 2008
Finding Structure with Randomness: Probabilistic Algorithms for Constructing Approximate Matrix Decompositions Halko, N.; Martinsson, P. G.; Tropp, J. A. SIAM Review, Vol. 53, Issue 2 https://doi.org/10.1137/090771806	journal	January 2011
Shifted Power Method for Computing Tensor Eigenpairs Kolda, Tamara G.; Mayo, Jackson R. SIAM Journal on Matrix Analysis and Applications, Vol. 32, Issue 4 https://doi.org/10.1137/100801482	journal	October 2011
Efficient Computation of Higher-Order Cumulant Tensors Domino, Krzysztof; Gawron, Piotr; Pawela, Łukasz SIAM Journal on Scientific Computing, Vol. 40, Issue 3 https://doi.org/10.1137/17M1149365	journal	January 2018
The Sample Complexity of Multireference Alignment Perry, Amelia; Weed, Jonathan; Bandeira, Afonso S. SIAM Journal on Mathematics of Data Science, Vol. 1, Issue 3 https://doi.org/10.1137/18M1214317	journal	January 2019
Most Tensor Problems Are NP-Hard Hillar, Christopher J.; Lim, Lek-Heng Journal of the ACM, Vol. 60, Issue 6 https://doi.org/10.1145/2512329	journal	November 2013
Algorithm 778: L-BFGS-B: Fortran subroutines for large-scale bound-constrained optimization Zhu, Ciyou; Byrd, Richard H.; Lu, Peihuang ACM Transactions on Mathematical Software, Vol. 23, Issue 4 https://doi.org/10.1145/279232.279236	journal	December 1997
Cumulants McCullagh, Peter; Kolassa, John Scholarpedia, Vol. 4, Issue 3 https://doi.org/10.4249/scholarpedia.4699	journal	January 2009

Similar Records

SymProp: Scaling Sparse Symmetric Tucker Decomposition via Symmetry Propagation

Conference · Sun Jun 01 00:00:00 EDT 2025 · OSTI ID:3002148

Shifted power method for computing tensor eigenvalues.

Conference · Thu Jul 01 00:00:00 EDT 2010 · OSTI ID:1021585

Shifted power method for computing tensor eigenpairs.

Technical Report · Fri Oct 01 00:00:00 EDT 2010 · OSTI ID:1005408

Related Subjects

97 MATHEMATICS AND COMPUTING
Gaussian mixture models
higher-order cumulants
higher-order moments
implicit tensor formation
symmetric tensor decomposition

Estimating Higher-Order Moments Using Symmetric Tensor Decomposition

Citation Formats

References (19)

Similar Records

Related Subjects