MPIFAUN: An MPIBased Framework for AlternatingUpdating Nonnegative Matrix Factorization
Nonnegative matrix factorization (NMF) is the problem of determining two nonnegative low rank factors W and H, for the given input matrix A, such that A≈WH. NMF is a useful tool for many applications in different domains such as topic modeling in text mining, background separation in video analysis, and community detection in social networks. Despite its popularity in the data mining community, there is a lack of efficient parallel algorithms to solve the problem for big data sets. The main contribution of this work is a new, highperformance parallel computational framework for a broad class of NMF algorithms that iteratively solves alternating nonnegative least squares (NLS) subproblems for W and H. It maintains the data and factor matrices in memory (distributed across processors), uses MPI for interprocessor communication, and, in the dense case, provably minimizes communication costs (under mild assumptions). The framework is flexible and able to leverage a variety of NMF and NLS algorithms, including Multiplicative Update, Hierarchical Alternating Least Squares, and Block Principal Pivoting. Our implementation allows us to benchmark and compare different algorithms on massive dense and sparse data matrices of size that spans from few hundreds of millions to billions. We demonstrate the scalability ofmore »
 Authors:

^{[1]}
;
^{[2]}
;
^{[3]}
 Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States)
 Wake Forest Univ., WinstonSalem, NC (United States)
 Georgia Inst. of Technology, Atlanta, GA (United States)
 Publication Date:
 Grant/Contract Number:
 AC0500OR22725; IIS1348152; ACI1338745; ACI1642385; FA87501220309; FA95501310100
 Type:
 Accepted Manuscript
 Journal Name:
 IEEE Transactions on Knowledge and Data Engineering
 Additional Journal Information:
 Journal Volume: 30; Journal Issue: 3; Journal ID: ISSN 10414347
 Publisher:
 IEEE
 Research Org:
 Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States)
 Sponsoring Org:
 USDOE Office of Science (SC); National Science Foundation (NSF); US Air Force Office of Scientific Research (AFOSR)
 Country of Publication:
 United States
 Language:
 English
 Subject:
 97 MATHEMATICS AND COMPUTING; Program processors; Computational modeling; Algorithm design and analysis; Sparse matrices; Approximation algorithms; Collaboration; Analytical models; HPC; NMF; MPI; 2D
 OSTI Identifier:
 1429224
Kannan, Ramakrishnan, Ballard, Grey, and Park, Haesun. MPIFAUN: An MPIBased Framework for AlternatingUpdating Nonnegative Matrix Factorization. United States: N. p.,
Web. doi:10.1109/TKDE.2017.2767592.
Kannan, Ramakrishnan, Ballard, Grey, & Park, Haesun. MPIFAUN: An MPIBased Framework for AlternatingUpdating Nonnegative Matrix Factorization. United States. doi:10.1109/TKDE.2017.2767592.
Kannan, Ramakrishnan, Ballard, Grey, and Park, Haesun. 2017.
"MPIFAUN: An MPIBased Framework for AlternatingUpdating Nonnegative Matrix Factorization". United States.
doi:10.1109/TKDE.2017.2767592. https://www.osti.gov/servlets/purl/1429224.
@article{osti_1429224,
title = {MPIFAUN: An MPIBased Framework for AlternatingUpdating Nonnegative Matrix Factorization},
author = {Kannan, Ramakrishnan and Ballard, Grey and Park, Haesun},
abstractNote = {Nonnegative matrix factorization (NMF) is the problem of determining two nonnegative low rank factors W and H, for the given input matrix A, such that A≈WH. NMF is a useful tool for many applications in different domains such as topic modeling in text mining, background separation in video analysis, and community detection in social networks. Despite its popularity in the data mining community, there is a lack of efficient parallel algorithms to solve the problem for big data sets. The main contribution of this work is a new, highperformance parallel computational framework for a broad class of NMF algorithms that iteratively solves alternating nonnegative least squares (NLS) subproblems for W and H. It maintains the data and factor matrices in memory (distributed across processors), uses MPI for interprocessor communication, and, in the dense case, provably minimizes communication costs (under mild assumptions). The framework is flexible and able to leverage a variety of NMF and NLS algorithms, including Multiplicative Update, Hierarchical Alternating Least Squares, and Block Principal Pivoting. Our implementation allows us to benchmark and compare different algorithms on massive dense and sparse data matrices of size that spans from few hundreds of millions to billions. We demonstrate the scalability of our algorithm and compare it with baseline implementations, showing significant performance improvements. The code and the datasets used for conducting the experiments are available online.},
doi = {10.1109/TKDE.2017.2767592},
journal = {IEEE Transactions on Knowledge and Data Engineering},
number = 3,
volume = 30,
place = {United States},
year = {2017},
month = {10}
}