Skip to main content
OSTI.GOVU.S. Department of Energy Office of Scientific and Technical Information
U.S. Department of Energy
Office of Scientific and Technical Information
 

Newton-based optimization for Kullback-Leibler nonnegative tensor factorizations

Journal Article · · Optimization Methods and Software
 [1];  [1];  [2]
  1. Sandia National Lab. (SNL-CA), Livermore, CA (United States)
  2. Northwestern Univ., Evanston, IL (United States)
+ Show Author Affiliations
Tensor factorizations with nonnegativity constraints have found application in analysing data from cyber traffic, social networks, and other areas. We consider application data best described as being generated by a Poisson process (e.g. count data), which leads to sparse tensors that can be modelled by sparse factor matrices. In this paper, we investigate efficient techniques for computing an appropriate canonical polyadic tensor factorization based on the Kullback–Leibler divergence function. We propose novel subproblem solvers within the standard alternating block variable approach. Our new methods exploit structure and reformulate the optimization problem as small independent subproblems. We employ bound-constrained Newton and quasi-Newton methods. Finally, we compare our algorithms against other codes, demonstrating superior speed for high accuracy results and the ability to quickly find sparse solutions.
Research Organization:
Sandia National Laboratories (SNL-CA), Livermore, CA (United States)
Sponsoring Organization:
WFO
Grant/Contract Number:
AC04-94AL85000
OSTI ID:
1182987
Report Number(s):
SAND--2014-16243J; 533741
Journal Information:
Optimization Methods and Software, Journal Name: Optimization Methods and Software Journal Issue: 5 Vol. 30; ISSN 1055-6788
Publisher:
Taylor & FrancisCopyright Statement
Country of Publication:
United States
Language:
English

References (31)

A scalable optimization approach for fitting canonical tensor decompositions journal January 2011
A weighted non-negative least squares algorithm for three-way ‘PARAFAC’ factor analysis journal October 1997
Updating quasi-Newton matrices with limited storage journal September 1980
Projected Newton methods for optimization problems with simple constraints conference December 1981
On the Goldstein-Levitin-Polyak gradient projection method journal April 1976
Global Convergence of a Class of Trust Region Algorithms for Optimization with Simple Bounds journal April 1988
On Tensors, Sparsity, and Nonnegative Factorizations journal January 2012
All-at-once Optimization for Coupled Matrix and Tensor Factorizations preprint January 2011
A fast non-negativity-constrained least squares algorithm journal September 1997
Positive matrix factorization: A non-negative factor model with optimal utilization of error estimates of data values journal June 1994
Fast Projection-Based Methods for the Least Squares Nonnegative Matrix Approximation Problem journal January 2008
Analysis of individual differences in multidimensional scaling via an n-way generalization of “Eckart-Young” decomposition journal September 1970
Nonnegative tensor factorization as an alternative Csiszar–Tusnady procedure: algorithms, convergence, probabilistic interpretations and novel probabilistic tensor latent variable analysis algorithms journal August 2010
Positive tensor factorization journal October 2001
A weighted non-negative least squares algorithm for three-way ‘PARAFAC’ factor analysis journal October 1997
A comparison of algorithms for fitting the PARAFAC model journal April 2006
Sparse non-negative tensor factorization using columnwise coordinate descent journal January 2012
Nonnegative matrix factorization with constrained second-order optimization journal August 2007
Learning the parts of objects by non-negative matrix factorization journal October 1999
Updating quasi-Newton matrices with limited storage journal September 1980
On the Goldstein-Levitin-Polyak gradient projection method journal April 1976
Projected Newton Methods for Optimization Problems with Simple Constraints journal March 1982
Nonnegative Matrix Factorization Based on Alternating Nonnegativity Constrained Least Squares and Active Set Method journal January 2008
Tensor Decompositions and Applications journal August 2009
On the Complexity of Nonnegative Matrix Factorization journal January 2010
Tackling Box-Constrained Optimization via a New Projected Quasi-Newton Approach journal January 2010
A Limited Memory Algorithm for Bound Constrained Optimization journal September 1995
Temporal Link Prediction Using Matrix and Tensor Factorizations journal February 2011
Projected Gradient Methods for Nonnegative Matrix Factorization journal October 2007
Fast Local Algorithms for Large Scale Nonnegative Matrix and Tensor Factorizations journal January 2009
Damped Newton based Iterative Non-negative Matrix Factorization for Intelligent Wood Defects Detection journal August 2010

Cited By (4)

A Tensor CP Decomposition Method for Clustering Heterogeneous Information Networks via Stochastic Gradient Descent Algorithms journal January 2017
A unified global convergence analysis of multiplicative update rules for nonnegative matrix factorization journal March 2018
PASTA: a parallel sparse tensor algorithm benchmark suite journal August 2019
Tensors for Data Mining and Data Fusion: Models, Applications, and Scalable Algorithms
  • Papalexakis, Evangelos E.; Faloutsos, Christos; Sidiropoulos, Nicholas D.
  • ACM Transactions on Intelligent Systems and Technology, Vol. 8, Issue 2 https://doi.org/10.1145/2915921
journal January 2017

Similar Records

Nonnegative canonical tensor decomposition with linear constraints: nnCANDELINC
Journal Article · Thu Apr 21 20:00:00 EDT 2022 · Numerical Linear Algebra with Applications · OSTI ID:1864617

Related Subjects

97 MATHEMATICS AND COMPUTING
Kullback–Leibler
Poisson
multilinear algebra
nonlinear optimization
tensor factorization