Distributed-Memory Parallel JointNMF

Conference · 01 June 2023

DOI:https://doi.org/10.1145/3577193.3593733· OSTI ID:1997627

Eswar, Srinivas ^[1]; Cobb, Benjamin ^[2]; Hayashi, Koby ^[2]; Kannan, Ramakrishnan {ramki}  ^[3]; Ballard, Grey ^[4]; Vuduc, Richard ^[5]; Park, Haesun ^[5]

---

1. Argonne National Laboratory
2. Georgia Institute of Technology
3. ORNL
4. Wake Forest University, Winston-Salem
5. Georgia Institute of Technology, Atlanta

Joint Nonnegative Matrix Factorization (JointNMF) is a hybrid method for mining information from datasets that contain both feature and connection information. We propose distributed-memory parallelizations of three algorithms for solving the JointNMF problem based on Alternating Nonnegative Least Squares, Projected Gradient Descent, and Projected Gauss-Newton. We extend well-known communication-avoiding algorithms using a single processor grid case to our coupled case on two processor grids. We demonstrate the scalability of the algorithms on up to 960 cores (40 nodes) with 60% parallel efficiency. The more sophisticated Alternating Nonnegative Least Squares (ANLS) and Gauss-Newton variants outperform the first-order gradient descent method in reducing the objective on large-scale problems. We perform a topic modelling task on a large corpus of academic papers that consists of over 37 million paper abstracts and nearly a billion citation relationships, demonstrating the utility and scalability of the methods.

• Research Organization: Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States)
• Sponsoring Organization: USDOE
• DOE Contract Number: AC05-00OR22725
• OSTI ID: 1997627
• Resource Relation: Conference: International Conference on Supercomputing (ICS) - Orlando, Florida, United States of America - 6/21/2023 8:00:00 AM-6/23/2023 8:00:00 AM
• Country of Publication: United States
• Language: English