Parallel Nonnegative CP Decomposition of Dense Tensors

Ballard, Grey; Hayashi, Koby; Kannan, Ramakrishnan {ramki}

doi:10.1109/HiPC.2018.00012

Title: Parallel Nonnegative CP Decomposition of Dense Tensors

Conference · Sat Dec 01 00:00:00 EST 2018

DOI:https://doi.org/10.1109/HiPC.2018.00012· OSTI ID:1511940

Ballard, Grey ^[1]; Hayashi, Koby ^[2];

^[3]

Wake Forest University, Winston-Salem
Wake Forest University, Winston-Salem, NC
ORNL

The CP tensor decomposition is a low-rank approximation of a tensor. We present a distributed-memory parallel algorithm and implementation of an alternating optimization method for computing a CP decomposition of dense tensors that can enforce nonnegativity of the computed low-rank factors. The principal task is to parallelize the Matricized-Tensor Times Khatri-Rao Product (MTTKRP) bottleneck subcomputation. The algorithm is computation efficient, using dimension trees to avoid redundant computation across MTTKRPs within the alternating method. Our approach is also communication efficient, using a data distribution and parallel algorithm across a multidimensional processor grid that can be tuned to minimize communication. We benchmark our software on synthetic as well as hyperspectral image and neuroscience dynamic functional connectivity data, demonstrating that our algorithm scales well to 100s of nodes (up to 4096 cores) and is faster and more general than the currently available parallel software.

View Conference

Cite

Export

Save

Research Organization:: Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States)

Sponsoring Organization:: USDOE

DOE Contract Number:: AC05-00OR22725

OSTI ID:: 1511940

Resource Relation:: Conference: IEEE 25th International Conference on High Performance Computing (HiPC 2018) - Bangalore, , India - 12/17/2018 3:00:00 PM-12/20/2018 3:00:00 PM

Country of Publication:: United States

Language:: English

References (23)

Constrained Tensor Factorization with Accelerated AO-ADMM Smith, Shaden; Beri, Alec; Karypis, George 2017 46th International Conference on Parallel Processing (ICPP) https://doi.org/10.1109/ICPP.2017.20	conference	August 2017
Time-lapse ratios of cone excitations in natural scenes Foster, David H.; Amano, Kinjiro; Nascimento, Sérgio M. C. Vision Research, Vol. 120 https://doi.org/10.1016/j.visres.2015.03.012	journal	March 2016
Tensor Decomposition for Signal Processing and Machine Learning Sidiropoulos, Nicholas D.; De Lathauwer, Lieven; Fu, Xiao IEEE Transactions on Signal Processing, Vol. 65, Issue 13 https://doi.org/10.1109/TSP.2017.2690524	journal	July 2017
SPLATT: Efficient and Parallel Sparse Tensor-Matrix Multiplication Smith, Shaden; Ravindran, Niranjay; Sidiropoulos, Nicholas D. 2015 IEEE International Parallel and Distributed Processing Symposium (IPDPS) https://doi.org/10.1109/IPDPS.2015.27	conference	May 2015
Fast Local Algorithms for Large Scale Nonnegative Matrix and Tensor Factorizations Cichocki, Andrzej; Phan, Anh-Huy IEICE Transactions on Fundamentals of Electronics, Communications and Computer Sciences, Vol. E92-A, Issue 3 https://doi.org/10.1587/transfun.E92.A.708	journal	January 2009
The Human Connectome Project: A data acquisition perspective Van Essen, D. C.; Ugurbil, K.; Auerbach, E. NeuroImage, Vol. 62, Issue 4 https://doi.org/10.1016/j.neuroimage.2012.02.018	journal	October 2012
Tensor Decompositions and Applications Kolda, Tamara G.; Bader, Brett W. SIAM Review, Vol. 51, Issue 3 https://doi.org/10.1137/07070111X	journal	August 2009
Fast Nonnegative Matrix Factorization: An Active-Set-Like Method and Comparisons Kim, Jingu; Park, Haesun SIAM Journal on Scientific Computing, Vol. 33, Issue 6 https://doi.org/10.1137/110821172	journal	January 2011
Algorithms for nonnegative matrix and tensor factorizations: a unified view based on block coordinate descent framework Kim, Jingu; He, Yunlong; Park, Haesun Journal of Global Optimization, Vol. 58, Issue 2 https://doi.org/10.1007/s10898-013-0035-4	journal	March 2013
Model-Driven Sparse CP Decomposition for Higher-Order Tensors Li, Jiajia; Choi, Jee; Perros, Ioakeim 2017 IEEE International Parallel and Distributed Processing Symposium (IPDPS) https://doi.org/10.1109/IPDPS.2017.80	conference	May 2017
Efficient algorithms for ‘universally’ constrained matrix and tensor factorization Huang, Kejun; Sidiropoulos, Nicholas D.; Liavas, Athanasios P. 2015 23rd European Signal Processing Conference (EUSIPCO) https://doi.org/10.1109/EUSIPCO.2015.7362839	conference	August 2015
Using Multivariate Analysis of Scanning-Rochigram Data to Reveal Material Functionality Jesse, Stephen; Chi, Miaofang; Borisevich, Albina Y. Microscopy and Microanalysis, Vol. 22, Issue S3 https://doi.org/10.1017/S1431927616002312	journal	July 2016
Parallel Candecomp/Parafac Decomposition of Sparse Tensors Using Dimension Trees Kaya, Oguz; Uçar, Bora SIAM Journal on Scientific Computing, Vol. 40, Issue 1 https://doi.org/10.1137/16M1102744	journal	January 2018
Dynamic functional connectivity and individual differences in emotions during social stress Tobia, Michael J.; Hayashi, Koby; Ballard, Grey Human Brain Mapping, Vol. 38, Issue 12 https://doi.org/10.1002/hbm.23821	journal	September 2017
Collective communication: theory, practice, and experience Chan, Ernie; Heimlich, Marcel; Purkayastha, Avi Concurrency and Computation: Practice and Experience, Vol. 19, Issue 13 https://doi.org/10.1002/cpe.1206	journal	January 2007
MPI-FAUN: An MPI-Based Framework for Alternating-Updating Nonnegative Matrix Factorization Kannan, Ramakrishnan; Ballard, Grey; Park, Haesun IEEE Transactions on Knowledge and Data Engineering, Vol. 30, Issue 3 https://doi.org/10.1109/TKDE.2017.2767592	journal	March 2018
Learning the parts of objects by non-negative matrix factorization Lee, Daniel D.; Seung, H. Sebastian Nature, Vol. 401, Issue 6755 https://doi.org/10.1038/44565	journal	October 1999
Positive tensor factorization Welling, Max; Weber, Markus Pattern Recognition Letters, Vol. 22, Issue 12 https://doi.org/10.1016/S0167-8655(01)00070-8	journal	October 2001
High Performance Parallel Algorithms for the Tucker Decomposition of Sparse Tensors Kaya, Oguz; Ucar, Bora 2016 45th International Conference on Parallel Processing (ICPP) https://doi.org/10.1109/ICPP.2016.19	conference	August 2016
Optimization of Collective Communication Operations in MPICH Thakur, Rajeev; Rabenseifner, Rolf; Gropp, William The International Journal of High Performance Computing Applications, Vol. 19, Issue 1 https://doi.org/10.1177/1094342005051521	journal	February 2005
PARAFAC algorithms for large-scale problems Huy Phan, Anh; Cichocki, Andrzej Neurocomputing, Vol. 74, Issue 11 https://doi.org/10.1016/j.neucom.2010.06.030	journal	May 2011
Fast Alternating LS Algorithms for High Order CANDECOMP/PARAFAC Tensor Factorizations Phan, Anh-Huy; Tichavsky, Petr; Cichocki, Andrzej IEEE Transactions on Signal Processing, Vol. 61, Issue 19 https://doi.org/10.1109/TSP.2013.2269903	journal	October 2013
Shared-memory parallelization of MTTKRP for dense tensors Hayashi, Koby; Ballard, Grey; Jiang, Yujie Proceedings of the 23rd ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming https://doi.org/10.1145/3178487.3178522	conference	February 2018

Similar Records

Software for Sparse Tensor Decomposition on Emerging Computing Architectures

Journal Article · Thu Jun 20 00:00:00 EDT 2019 · SIAM Journal on Scientific Computing · OSTI ID:1511940

Phipps, Eric T.; Kolda, Tamara G.

PLANC: Parallel Low-rank Approximation with Nonnegativity Constraints

Journal Article · Fri Jun 25 00:00:00 EDT 2021 · ACM Transactions on Mathematical Software · OSTI ID:1511940

Eswar, Srinivas; Hayashi, Koby; Ballard, Grey; +3 more

True Load Balancing for Matricized Tensor Times Khatri-Rao Product

Journal Article · Fri Jan 22 00:00:00 EST 2021 · IEEE Transactions on Parallel and Distributed Systems · OSTI ID:1511940

Abubaker, Nabil; Acer, Seher; Aykanat, Cevdet

Title: Parallel Nonnegative CP Decomposition of Dense Tensors

Citation Formats

References (23)

Similar Records

Related Subjects