skip to main content
OSTI.GOV title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Newton-based optimization for Kullback-Leibler nonnegative tensor factorizations

Journal Article · · Optimization Methods and Software
 [1];  [1];  [2]
  1. Sandia National Lab. (SNL-CA), Livermore, CA (United States)
  2. Northwestern Univ., Evanston, IL (United States)
+ Show Author Affiliations

Tensor factorizations with nonnegativity constraints have found application in analysing data from cyber traffic, social networks, and other areas. We consider application data best described as being generated by a Poisson process (e.g. count data), which leads to sparse tensors that can be modelled by sparse factor matrices. In this paper, we investigate efficient techniques for computing an appropriate canonical polyadic tensor factorization based on the Kullback–Leibler divergence function. We propose novel subproblem solvers within the standard alternating block variable approach. Our new methods exploit structure and reformulate the optimization problem as small independent subproblems. We employ bound-constrained Newton and quasi-Newton methods. Finally, we compare our algorithms against other codes, demonstrating superior speed for high accuracy results and the ability to quickly find sparse solutions.

Research Organization:
Sandia National Lab. (SNL-CA), Livermore, CA (United States)
Sponsoring Organization:
Work for Others (WFO)
Grant/Contract Number:
AC04-94AL85000
OSTI ID:
1182987
Report Number(s):
SAND-2014-16243J; 533741
Journal Information:
Optimization Methods and Software, Vol. 30, Issue 5; ISSN 1055-6788
Publisher:
Taylor & FrancisCopyright Statement
Country of Publication:
United States
Language:
English
Citation Metrics:
Cited by: 35 works
Citation information provided by
Web of Science

References (26)

A scalable optimization approach for fitting canonical tensor decompositions journal January 2011
Projected Newton Methods for Optimization Problems with Simple Constraints journal March 1982
A fast non-negativity-constrained least squares algorithm journal September 1997
A Limited Memory Algorithm for Bound Constrained Optimization journal September 1995
On Tensors, Sparsity, and Nonnegative Factorizations journal January 2012
Fast Local Algorithms for Large Scale Nonnegative Matrix and Tensor Factorizations journal January 2009
Global Convergence of a Class of Trust Region Algorithms for Optimization with Simple Bounds journal April 1988
Temporal Link Prediction Using Matrix and Tensor Factorizations journal February 2011
Nonnegative Matrix Factorization Based on Alternating Nonnegativity Constrained Least Squares and Active Set Method journal January 2008
Fast Projection-Based Methods for the Least Squares Nonnegative Matrix Approximation Problem journal January 2008
Tackling Box-Constrained Optimization via a New Projected Quasi-Newton Approach journal January 2010
Learning the parts of objects by non-negative matrix factorization journal October 1999
Projected Gradient Methods for Nonnegative Matrix Factorization journal October 2007
Sparse non-negative tensor factorization using columnwise coordinate descent journal January 2012
Positive matrix factorization: A non-negative factor model with optimal utilization of error estimates of data values journal June 1994
A comparison of algorithms for fitting the PARAFAC model journal April 2006
On the Complexity of Nonnegative Matrix Factorization journal January 2010
Positive tensor factorization journal October 2001
Nonnegative tensor factorization as an alternative Csiszar–Tusnady procedure: algorithms, convergence, probabilistic interpretations and novel probabilistic tensor latent variable analysis algorithms journal August 2010
Nonnegative matrix factorization with constrained second-order optimization journal August 2007
Damped Newton based Iterative Non-negative Matrix Factorization for Intelligent Wood Defects Detection journal August 2010
A weighted non-negative least squares algorithm for three-way ‘PARAFAC’ factor analysis journal October 1997
Updating quasi-Newton matrices with limited storage journal September 1980
Projected Newton methods for optimization problems with simple constraints conference December 1981
On the Goldstein-Levitin-Polyak gradient projection method journal April 1976
All-at-once Optimization for Coupled Matrix and Tensor Factorizations preprint January 2011

Cited By (4)

A unified global convergence analysis of multiplicative update rules for nonnegative matrix factorization journal March 2018
PASTA: a parallel sparse tensor algorithm benchmark suite journal August 2019
Tensors for Data Mining and Data Fusion: Models, Applications, and Scalable Algorithms
  • Papalexakis, Evangelos E.; Faloutsos, Christos; Sidiropoulos, Nicholas D.
  • ACM Transactions on Intelligent Systems and Technology, Vol. 8, Issue 2 https://doi.org/10.1145/2915921
journal January 2017
A Tensor CP Decomposition Method for Clustering Heterogeneous Information Networks via Stochastic Gradient Descent Algorithms journal January 2017

Similar Records

Nonnegative canonical tensor decomposition with linear constraints: nnCANDELINC
Journal Article · Fri Apr 22 00:00:00 EDT 2022 · Numerical Linear Algebra with Applications · OSTI ID:1182987
+1 more
Parameter Sensitivity Analysis of the SparTen High Performance Sparse Tensor Decomposition Software (Extended Analysis)
Technical Report · Fri Nov 06 00:00:00 EST 2020 · OSTI ID:1182987
+1 more
Generalized Canonical Polyadic Tensor Decomposition
Journal Article · Tue Feb 11 00:00:00 EST 2020 · SIAM Review · OSTI ID:1182987

Related Subjects

97 MATHEMATICS AND COMPUTING
tensor factorization
multilinear algebra
nonlinear optimization
Poisson
Kullback–Leibler