Newton-based optimization for Kullback-Leibler nonnegative tensor factorizations
Abstract
Tensor factorizations with nonnegativity constraints have found application in analysing data from cyber traffic, social networks, and other areas. We consider application data best described as being generated by a Poisson process (e.g. count data), which leads to sparse tensors that can be modelled by sparse factor matrices. In this paper, we investigate efficient techniques for computing an appropriate canonical polyadic tensor factorization based on the Kullback–Leibler divergence function. We propose novel subproblem solvers within the standard alternating block variable approach. Our new methods exploit structure and reformulate the optimization problem as small independent subproblems. We employ bound-constrained Newton and quasi-Newton methods. Finally, we compare our algorithms against other codes, demonstrating superior speed for high accuracy results and the ability to quickly find sparse solutions.
- Authors:
-
- Sandia National Lab. (SNL-CA), Livermore, CA (United States)
- Northwestern Univ., Evanston, IL (United States)
- Publication Date:
- Research Org.:
- Sandia National Lab. (SNL-CA), Livermore, CA (United States)
- Sponsoring Org.:
- Work for Others (WFO)
- OSTI Identifier:
- 1182987
- Report Number(s):
- SAND-2014-16243J
Journal ID: ISSN 1055-6788; 533741
- Grant/Contract Number:
- AC04-94AL85000
- Resource Type:
- Accepted Manuscript
- Journal Name:
- Optimization Methods and Software
- Additional Journal Information:
- Journal Volume: 30; Journal Issue: 5; Journal ID: ISSN 1055-6788
- Publisher:
- Taylor & Francis
- Country of Publication:
- United States
- Language:
- English
- Subject:
- 97 MATHEMATICS AND COMPUTING; tensor factorization; multilinear algebra; nonlinear optimization; Poisson; Kullback–Leibler
Citation Formats
Plantenga, Todd, Kolda, Tamara G., and Hansen, Samantha. Newton-based optimization for Kullback-Leibler nonnegative tensor factorizations. United States: N. p., 2015.
Web. doi:10.1080/10556788.2015.1009977.
Plantenga, Todd, Kolda, Tamara G., & Hansen, Samantha. Newton-based optimization for Kullback-Leibler nonnegative tensor factorizations. United States. https://doi.org/10.1080/10556788.2015.1009977
Plantenga, Todd, Kolda, Tamara G., and Hansen, Samantha. Thu .
"Newton-based optimization for Kullback-Leibler nonnegative tensor factorizations". United States. https://doi.org/10.1080/10556788.2015.1009977. https://www.osti.gov/servlets/purl/1182987.
@article{osti_1182987,
title = {Newton-based optimization for Kullback-Leibler nonnegative tensor factorizations},
author = {Plantenga, Todd and Kolda, Tamara G. and Hansen, Samantha},
abstractNote = {Tensor factorizations with nonnegativity constraints have found application in analysing data from cyber traffic, social networks, and other areas. We consider application data best described as being generated by a Poisson process (e.g. count data), which leads to sparse tensors that can be modelled by sparse factor matrices. In this paper, we investigate efficient techniques for computing an appropriate canonical polyadic tensor factorization based on the Kullback–Leibler divergence function. We propose novel subproblem solvers within the standard alternating block variable approach. Our new methods exploit structure and reformulate the optimization problem as small independent subproblems. We employ bound-constrained Newton and quasi-Newton methods. Finally, we compare our algorithms against other codes, demonstrating superior speed for high accuracy results and the ability to quickly find sparse solutions.},
doi = {10.1080/10556788.2015.1009977},
journal = {Optimization Methods and Software},
number = 5,
volume = 30,
place = {United States},
year = {Thu Apr 30 00:00:00 EDT 2015},
month = {Thu Apr 30 00:00:00 EDT 2015}
}
Free Publicly Available Full Text
Publisher's Version of Record
Other availability
Search WorldCat to find libraries that may hold this journal
Cited by: 35 works
Citation information provided by
Web of Science
Web of Science
Save to My Library
You must Sign In or Create an Account in order to save documents to your library.
Works referenced in this record:
A scalable optimization approach for fitting canonical tensor decompositions
journal, January 2011
- Acar, Evrim; Dunlavy, Daniel M.; Kolda, Tamara G.
- Journal of Chemometrics, Vol. 25, Issue 2
Projected Newton Methods for Optimization Problems with Simple Constraints
journal, March 1982
- Bertsekas, Dimitri P.
- SIAM Journal on Control and Optimization, Vol. 20, Issue 2
A fast non-negativity-constrained least squares algorithm
journal, September 1997
- Bro, Rasmus; De Jong, Sijmen
- Journal of Chemometrics, Vol. 11, Issue 5
A Limited Memory Algorithm for Bound Constrained Optimization
journal, September 1995
- Byrd, Richard H.; Lu, Peihuang; Nocedal, Jorge
- SIAM Journal on Scientific Computing, Vol. 16, Issue 5
On Tensors, Sparsity, and Nonnegative Factorizations
journal, January 2012
- Chi, Eric C.; Kolda, Tamara G.
- SIAM Journal on Matrix Analysis and Applications, Vol. 33, Issue 4
Fast Local Algorithms for Large Scale Nonnegative Matrix and Tensor Factorizations
journal, January 2009
- Cichocki, Andrzej; Phan, Anh-Huy
- IEICE Transactions on Fundamentals of Electronics, Communications and Computer Sciences, Vol. E92-A, Issue 3
Global Convergence of a Class of Trust Region Algorithms for Optimization with Simple Bounds
journal, April 1988
- Conn, A. R.; Gould, N. I. M.; Toint, Ph. L.
- SIAM Journal on Numerical Analysis, Vol. 25, Issue 2
Temporal Link Prediction Using Matrix and Tensor Factorizations
journal, February 2011
- Dunlavy, Daniel M.; Kolda, Tamara G.; Acar, Evrim
- ACM Transactions on Knowledge Discovery from Data, Vol. 5, Issue 2
Nonnegative Matrix Factorization Based on Alternating Nonnegativity Constrained Least Squares and Active Set Method
journal, January 2008
- Kim, Hyunsoo; Park, Haesun
- SIAM Journal on Matrix Analysis and Applications, Vol. 30, Issue 2
Fast Projection-Based Methods for the Least Squares Nonnegative Matrix Approximation Problem
journal, January 2008
- Kim, Dongmin; Sra, Suvrit; Dhillon, Inderjit S.
- Statistical Analysis and Data Mining, Vol. 1, Issue 1
Tackling Box-Constrained Optimization via a New Projected Quasi-Newton Approach
journal, January 2010
- Kim, Dongmin; Sra, Suvrit; Dhillon, Inderjit S.
- SIAM Journal on Scientific Computing, Vol. 32, Issue 6
Learning the parts of objects by non-negative matrix factorization
journal, October 1999
- Lee, Daniel D.; Seung, H. Sebastian
- Nature, Vol. 401, Issue 6755
Projected Gradient Methods for Nonnegative Matrix Factorization
journal, October 2007
- Lin, Chih-Jen
- Neural Computation, Vol. 19, Issue 10
Sparse non-negative tensor factorization using columnwise coordinate descent
journal, January 2012
- Liu, Ji; Liu, Jun; Wonka, Peter
- Pattern Recognition, Vol. 45, Issue 1
Positive matrix factorization: A non-negative factor model with optimal utilization of error estimates of data values
journal, June 1994
- Paatero, Pentti; Tapper, Unto
- Environmetrics, Vol. 5, Issue 2
A comparison of algorithms for fitting the PARAFAC model
journal, April 2006
- Tomasi, Giorgio; Bro, Rasmus
- Computational Statistics & Data Analysis, Vol. 50, Issue 7
On the Complexity of Nonnegative Matrix Factorization
journal, January 2010
- Vavasis, Stephen A.
- SIAM Journal on Optimization, Vol. 20, Issue 3
Positive tensor factorization
journal, October 2001
- Welling, Max; Weber, Markus
- Pattern Recognition Letters, Vol. 22, Issue 12
Nonnegative tensor factorization as an alternative Csiszar–Tusnady procedure: algorithms, convergence, probabilistic interpretations and novel probabilistic tensor latent variable analysis algorithms
journal, August 2010
- Zafeiriou, Stefanos; Petrou, Maria
- Data Mining and Knowledge Discovery, Vol. 22, Issue 3
Nonnegative matrix factorization with constrained second-order optimization
journal, August 2007
- Zdunek, Rafal; Cichocki, Andrzej
- Signal Processing, Vol. 87, Issue 8
Damped Newton based Iterative Non-negative Matrix Factorization for Intelligent Wood Defects Detection
journal, August 2010
- Zheng, Yafeng; Zhang, Qiaorong; Zhang, Zhao
- Journal of Software, Vol. 5, Issue 8
A scalable optimization approach for fitting canonical tensor decompositions
journal, January 2011
- Acar, Evrim; Dunlavy, Daniel M.; Kolda, Tamara G.
- Journal of Chemometrics, Vol. 25, Issue 2
A weighted non-negative least squares algorithm for three-way ‘PARAFAC’ factor analysis
journal, October 1997
- Paatero, Pentti
- Chemometrics and Intelligent Laboratory Systems, Vol. 38, Issue 2
Updating quasi-Newton matrices with limited storage
journal, September 1980
- Nocedal, Jorge
- Mathematics of Computation, Vol. 35, Issue 151
Projected Newton methods for optimization problems with simple constraints
conference, December 1981
- Bertsekas, Dimitri P.
- 1981 20th IEEE Conference on Decision and Control including the Symposium on Adaptive Processes
On the Goldstein-Levitin-Polyak gradient projection method
journal, April 1976
- Bertsekas, Dimitri P.
- IEEE Transactions on Automatic Control, Vol. 21, Issue 2
Global Convergence of a Class of Trust Region Algorithms for Optimization with Simple Bounds
journal, April 1988
- Conn, A. R.; Gould, N. I. M.; Toint, Ph. L.
- SIAM Journal on Numerical Analysis, Vol. 25, Issue 2
On Tensors, Sparsity, and Nonnegative Factorizations
journal, January 2012
- Chi, Eric C.; Kolda, Tamara G.
- SIAM Journal on Matrix Analysis and Applications, Vol. 33, Issue 4
All-at-once Optimization for Coupled Matrix and Tensor Factorizations
preprint, January 2011
- Acar, Evrim; Kolda, Tamara G.; Dunlavy, Daniel M.
- arXiv
Works referencing / citing this record:
A unified global convergence analysis of multiplicative update rules for nonnegative matrix factorization
journal, March 2018
- Takahashi, Norikazu; Katayama, Jiro; Seki, Masato
- Computational Optimization and Applications, Vol. 71, Issue 1
PASTA: a parallel sparse tensor algorithm benchmark suite
journal, August 2019
- Li, Jiajia; Ma, Yuchen; Wu, Xiaolong
- CCF Transactions on High Performance Computing, Vol. 1, Issue 2
Tensors for Data Mining and Data Fusion: Models, Applications, and Scalable Algorithms
journal, January 2017
- Papalexakis, Evangelos E.; Faloutsos, Christos; Sidiropoulos, Nicholas D.
- ACM Transactions on Intelligent Systems and Technology, Vol. 8, Issue 2
A Tensor CP Decomposition Method for Clustering Heterogeneous Information Networks via Stochastic Gradient Descent Algorithms
journal, January 2017
- Wu, Jibing; Wang, Zhifei; Wu, Yahui
- Scientific Programming, Vol. 2017