Distributed-Memory Parallel Symmetric Nonnegative Matrix Factorization

Eswar, Srinivas; Hayashi, Koby; Ballard, Grey; Kannan, Ramakrishnan {ramki}; Vuduc, Richard; Park, Haesun

doi:10.1109/SC41405.2020.00078

Title: Distributed-Memory Parallel Symmetric Nonnegative Matrix Factorization

Full Record
Other Related Research

Abstract

We develop the first distributed-memory parallel implementation of Symmetric Nonnegative Matrix Factorization (SymNMF), a key data analytics kernel for clustering and dimensionality reduction. Our implementation includes two different algorithms for SymNMF, which give comparable results in terms of time and accuracy. The first algorithm is a parallelization of an existing sequential approach that uses solvers for non symmetric NMF. The second algorithm is a novel approach based on the Gauss-Newton method. It exploits second-order information without incurring large computational and memory costs. We evaluate the scalability of our algorithms on the Summit system at Oak Ridge National Laboratory, scaling up to 128 nodes (4,096 cores) with 70% efficiency. Additionally, we demonstrate our software on an image segmentation task.

Authors:

Eswar, Srinivas ^[1]; Hayashi, Koby ^[1]; Ballard, Grey ^[2];

^[3]; Vuduc, Richard ^[4]; Park, Haesun ^[4]

Georgia Institute of Technology
Wake Forest University, Winston-Salem
ORNL
Georgia Institute of Technology, Atlanta

Publication Date:: 2020-11-01

Research Org.:: Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States)

Sponsoring Org.:: USDOE

OSTI Identifier:: 1798617

DOE Contract Number:: AC05-00OR22725

Resource Type:: Conference

Resource Relation:: Conference: The International Conference for High Performance Computing, Networking, Storage and Analysis - Atlanta, Georgia, United States of America - 11/16/2020 10:00:00 AM-11/19/2020 10:00:00 AM

Country of Publication:: United States

Language:: English

Citation Formats


                    Eswar, Srinivas, Hayashi, Koby, Ballard, Grey, Kannan, Ramakrishnan, Vuduc, Richard, and Park, Haesun. Distributed-Memory Parallel Symmetric Nonnegative Matrix Factorization.  United States: N. p., 2020. 
        Web.  doi:10.1109/SC41405.2020.00078.

Copy to clipboard


                    Eswar, Srinivas, Hayashi, Koby, Ballard, Grey, Kannan, Ramakrishnan, Vuduc, Richard, & Park, Haesun. Distributed-Memory Parallel Symmetric Nonnegative Matrix Factorization.  United States.  https://doi.org/10.1109/SC41405.2020.00078

Copy to clipboard


                    Eswar, Srinivas, Hayashi, Koby, Ballard, Grey, Kannan, Ramakrishnan, Vuduc, Richard, and Park, Haesun. 2020.  
        "Distributed-Memory Parallel Symmetric Nonnegative Matrix Factorization".  United States.  https://doi.org/10.1109/SC41405.2020.00078.  https://www.osti.gov/servlets/purl/1798617.

Copy to clipboard


                    
@article{osti_1798617,

  title        = {Distributed-Memory Parallel Symmetric Nonnegative Matrix Factorization},

  author       = {Eswar, Srinivas and Hayashi, Koby and Ballard, Grey and Kannan, Ramakrishnan and Vuduc, Richard and Park, Haesun},

  abstractNote = {We develop the first distributed-memory parallel implementation of Symmetric Nonnegative Matrix Factorization (SymNMF), a key data analytics kernel for clustering and dimensionality reduction. Our implementation includes two different algorithms for SymNMF, which give comparable results in terms of time and accuracy. The first algorithm is a parallelization of an existing sequential approach that uses solvers for non symmetric NMF. The second algorithm is a novel approach based on the Gauss-Newton method. It exploits second-order information without incurring large computational and memory costs. We evaluate the scalability of our algorithms on the Summit system at Oak Ridge National Laboratory, scaling up to 128 nodes (4,096 cores) with 70% efficiency. Additionally, we demonstrate our software on an image segmentation task.},

  doi          = {10.1109/SC41405.2020.00078},

  url          = {https://www.osti.gov/biblio/1798617},
  journal      = {},
number       = ,

  volume       = ,

  place        = {United States},

  year         = {2020},

  month        = {11}

}

Copy to clipboard

Conference:

View Conference (1.07 MB)

https://doi.org/10.1109/SC41405.2020.00078

Other availability

Please see Document Availability for additional information on obtaining the full-text document. Library patrons may search WorldCat to identify libraries that hold this conference proceeding.

Save / Share:

Export Metadata

Save to My Library

Similar records in OSTI.GOV collections:

Distributed-Memory Parallel JointNMF

Conference Eswar, Srinivas; Cobb, Benjamin; Hayashi, Koby; ...

Joint Nonnegative Matrix Factorization (JointNMF) is a hybrid method for mining information from datasets that contain both feature and connection information. We propose distributed-memory parallelizations of three algorithms for solving the JointNMF problem based on Alternating Nonnegative Least Squares, Projected Gradient Descent, and Projected Gauss-Newton. We extend well-known communication-avoiding algorithms using a single processor grid case to our coupled case on two processor grids. We demonstrate the scalability of the algorithms on up to 960 cores (40 nodes) with 60% parallel efficiency. The more sophisticated Alternating Nonnegative Least Squares (ANLS) and Gauss-Newton variants outperform the first-order gradient descent method inmore »« less
https://doi.org/10.1145/3577193.3593733

Full Text Available
A high-performance parallel algorithm for nonnegative matrix factorization

Journal Article Kannan, Ramakrishnan; Ballard, Grey; Park, Haesun

Non-negative matrix factorization (NMF) is the problem of determining two non-negative low rank factors W and H, for the given input matrix A, such that A ≈ WH. NMF is a useful tool for many applications in different domains such as topic modeling in text mining, background separation in video analysis, and community detection in social networks. Despite its popularity in the data mining community, there is a lack of efficient distributed algorithms to solve the problem for big data sets. We propose a high-performance distributed-memory parallel algorithm that computes the factorization by iteratively solving alternating non-negative least squares (NLS)more »« less
https://doi.org/10.1145/2851141.2851152
A parallel WLS state estimator on shared memory computers

Conference Nieplocha, Jarek; Chavarría-Miranda, Daniel; Tipparaju, Vinod; ...

This paper describes our experience with developing a parallel weighted- least-square (WLS) state estimation (SE) program for shared-memory parallel computers. Since the key computational kernel of the WLS algorithm based on the Newton-Raphson approach is a solver of sparse linear equations, a significant part of our effort was focused on selecting, implementing and evaluating this algorithm. An optimized shared memory version of the conjugate gradient (CG) algorithm was found to be competitive to state-of-the-art implementation of LU solvers for the SE problem on the SGI Altix, a shared memory architecture. We also ported the full SE algorithm including CG tomore »« less
PLANC: Parallel Low-rank Approximation with Nonnegativity Constraints

Journal Article Eswar, Srinivas; Hayashi, Koby; Ballard, Grey; ... - ACM Transactions on Mathematical Software

In this work, we consider the problem of low-rank approximation of massive dense nonnegative tensor data, for example, to discover latent patterns in video and imaging applications. As the size of data sets grows, single workstations are hitting bottlenecks in both computation time and available memory. We propose a distributed-memory parallel computing solution to handle massive data sets, loading the input data across the memories of multiple nodes, and performing efficient and scalable parallel algorithms to compute the low-rank approximation. We present a software package called Parallel Low-rank Approximation with Nonnegativity Constraints, which implements our solution and allows for extensionmore »« less
https://doi.org/10.1145/3432185

Full Text Available
MPI-FAUN: An MPI-Based Framework for Alternating-Updating Nonnegative Matrix Factorization

Journal Article Kannan, Ramakrishnan; Ballard, Grey; Park, Haesun - IEEE Transactions on Knowledge and Data Engineering

Non-negative matrix factorization (NMF) is the problem of determining two non-negative low rank factors W and H, for the given input matrix A, such that A≈WH. NMF is a useful tool for many applications in different domains such as topic modeling in text mining, background separation in video analysis, and community detection in social networks. Despite its popularity in the data mining community, there is a lack of efficient parallel algorithms to solve the problem for big data sets. The main contribution of this work is a new, high-performance parallel computational framework for a broad class of NMF algorithms thatmore »« less
Cited by 17
https://doi.org/10.1109/TKDE.2017.2767592

Full Text Available

Similar Records