Graph coarsening: from scientific computing to machine learning

Chen, Jie; Saad, Yousef; Zhang, Zechen

doi:10.1007/s40324-021-00282-x

Title: Graph coarsening: from scientific computing to machine learning

Journal Article · 09 January 2022 · SeMA Journal

DOI: https://doi.org/10.1007/s40324-021-00282-x · OSTI ID:1839168

Chen, Jie; Saad, Yousef; Zhang, Zechen

Abstract The general method of graph coarsening or graph reduction has been a remarkably useful and ubiquitous tool in scientific computing and it is now just starting to have a similar impact in machine learning. The goal of this paper is to take a broad look into coarsening techniques that have been successfully deployed in scientific computing and see how similar principles are finding their way in more recent applications related to machine learning. In scientific computing, coarsening plays a central role in algebraic multigrid methods as well as the related class of multilevel incomplete LU factorizations. In machine learning, graph coarsening goes under various names, e.g., graph downsampling or graph reduction. Its goal in most cases is to replace some original graph by one which has fewer nodes, but whose structure and characteristics are similar to those of the original graph. As will be seen, a common strategy in these methods is to rely on spectral properties to define the coarse graph.

View Journal Article

Cite

Export

Save

Sponsoring Organization:: USDOE

Grant/Contract Number:: OE0000910

OSTI ID:: 1839168

Journal Information:: SeMA Journal, Journal Name: SeMA Journal Journal Issue: 1 Vol. 79; ISSN 2254-3902

Publisher:: Springer Science + Business MediaCopyright Statement

Country of Publication:: Country unknown/Code not available

Language:: English

References (80)

Fast multilevel implementation of recursive spectral bisection for partitioning unstructured problems Barnard, Stephen T.; Simon, Horst D. Concurrency: Practice and Experience, Vol. 6, Issue 2 https://doi.org/10.1002/cpe.4330060203	journal	April 1994
An Efficient Heuristic Procedure for Partitioning Graphs Kernighan, B. W.; Lin, S. Bell System Technical Journal, Vol. 49, Issue 2 https://doi.org/10.1002/j.1538-7305.1970.tb01770.x	journal	February 1970
Algebraic multilevel iteration method for Stieltjes matrices Axelsson, O.; Neytcheva, M. Numerical Linear Algebra with Applications, Vol. 1, Issue 3 https://doi.org/10.1002/nla.1680010302	journal	May 1994
Multilevel graph embedding Quiring, Benjamin; Vassilevski, Panayot S. Numerical Linear Algebra with Applications, Vol. 28, Issue 3 https://doi.org/10.1002/nla.2326	journal	September 2020
ARMS: an algebraic recursive multilevel solver for general sparse linear systems Saad, Y.; Suchomel, B. Numerical Linear Algebra with Applications, Vol. 9, Issue 5 https://doi.org/10.1002/nla.279	journal	January 2002
A multilevel Crout ILU preconditioner with pivoting and row permutation: A MULTILEVEL ILU PRECONDITIONER WITH PIVOTING AND ROW PERMUTATION Mayer, Jan Numerical Linear Algebra with Applications, Vol. 14, Issue 10 https://doi.org/10.1002/nla.554	journal	September 2007
The electrical resistance of a graph captures its commute and cover times Chandra, Ashok K.; Raghavan, Prabhakar; Ruzzo, Walter L. Computational Complexity, Vol. 6, Issue 4 https://doi.org/10.1007/BF01270385	journal	December 1996
Algebraic multilevel preconditioning methods. I Axclsson, O.; Vassilevski, P. S. Numerische Mathematik, Vol. 56, Issue 2-3 https://doi.org/10.1007/BF01409783	journal	February 1989
Weighted graph based ordering techniques for preconditioned conjugate gradient methods Clift, Simon S.; Tang, Wei-Pai BIT Numerical Mathematics, Vol. 35, Issue 1 https://doi.org/10.1007/BF01732977	journal	March 1995
A survey of multilevel preconditioned iterative methods Axelsson, O.; Vassilevski, P. S. BIT, Vol. 29, Issue 4 https://doi.org/10.1007/BF01932745	journal	December 1989
On sparse spanners of weighted graphs Althöfer, Ingo; Das, Gautam; Dobkin, David Discrete & Computational Geometry, Vol. 9, Issue 1 https://doi.org/10.1007/BF02189308	journal	January 1993
Algebraic multigrid by smoothed aggregation for second and fourth order elliptic problems Vaněk, P.; Mandel, J.; Brezina, M. Computing, Vol. 56, Issue 3 https://doi.org/10.1007/BF02238511	journal	September 1996
Multilevel ILU decomposition Bank, Randolph E.; Wagner, Christian Numerische Mathematik, Vol. 82, Issue 4 https://doi.org/10.1007/s002110050430	journal	June 1999
Convergence of algebraic multigrid based on smoothed aggregation Van\vek, Petr; Brezina, Marian; Mandel, Jan Numerische Mathematik, Vol. 88, Issue 3 https://doi.org/10.1007/s211-001-8015-y	journal	May 2001
On the eigenvectors belonging to the minimum eigenvalue of an essentially nonnegative symmetric matrix with bipartite graph Roth, Robert Linear Algebra and its Applications, Vol. 118 https://doi.org/10.1016/0024-3795(89)90569-7	journal	June 1989
On the convergence of a relaxation method with natural constraints on the elliptic operator Bakhvalov, N. S. USSR Computational Mathematics and Mathematical Physics, Vol. 6, Issue 5 https://doi.org/10.1016/0041-5553(66)90118-2	journal	January 1966
Distinguishing Enzyme Structures from Non-enzymes Without Alignments Dobson, Paul D.; Doig, Andrew J. Journal of Molecular Biology, Vol. 330, Issue 4 https://doi.org/10.1016/S0022-2836(03)00628-4	journal	July 2003
A robust ILU with pivoting based on monitoring the growth of the inverse factors Bollhöfer, Matthias Linear Algebra and its Applications, Vol. 338, Issue 1-3 https://doi.org/10.1016/S0024-3795(01)00385-8	journal	November 2001
Experimental study of ILU preconditioners for indefinite matrices Chow, Edmond; Saad, Yousef Journal of Computational and Applied Mathematics, Vol. 86, Issue 2 https://doi.org/10.1016/S0377-0427(97)00171-4	journal	December 1997
An algebraic multilevel iteration method for finite element matrices Axelsson, O.; Larin, M. Journal of Computational and Applied Mathematics, Vol. 89, Issue 1 https://doi.org/10.1016/S0377-0427(97)00241-0	journal	March 1998
Graph embedding techniques, applications, and performance: A survey Goyal, Palash; Ferrara, Emilio Knowledge-Based Systems, Vol. 151 https://doi.org/10.1016/j.knosys.2018.03.022	journal	July 2018
Effective graph resistance Ellens, W.; Spieksma, F. M.; Van Mieghem, P. Linear Algebra and its Applications, Vol. 435, Issue 10 https://doi.org/10.1016/j.laa.2011.02.024	journal	November 2011
Geometry of complex networks and topological centrality Ranjan, Gyan; Zhang, Zhi-Li Physica A: Statistical Mechanics and its Applications, Vol. 392, Issue 17 https://doi.org/10.1016/j.physa.2013.04.013	journal	September 2013
A measure of betweenness centrality based on random walks Newman, M. E. J. Social Networks, Vol. 27, Issue 1 https://doi.org/10.1016/j.socnet.2004.11.009	journal	January 2005
Electronic Structure Martin, Richard M. https://doi.org/10.1017/CBO9780511805769	book	January 2004
Modification methods for inverting matrices and solving systems of linear algebraic equations Goldfarb, D. Mathematics of Computation, Vol. 26, Issue 120 https://doi.org/10.1090/S0025-5718-1972-0317527-4	journal	January 1972
Multi-level adaptive solutions to boundary-value problems Brandt, Achi Mathematics of Computation, Vol. 31, Issue 138 https://doi.org/10.1090/S0025-5718-1977-0431719-X	journal	May 1977
Pseudoinverse of the Laplacian and best spreader node in a network Van Mieghem, P.; Devriendt, K.; Cetinay, H. Physical Review E, Vol. 96, Issue 3 https://doi.org/10.1103/PhysRevE.96.032311	journal	September 2017
Dynamic Edge-Conditioned Filters in Convolutional Neural Networks on Graphs Simonovsky, Martin; Komodakis, Nikos 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) https://doi.org/10.1109/CVPR.2017.11	conference	July 2017
Geometric Deep Learning on Graphs and Manifolds Using Mixture Model CNNs Monti, Federico; Boscaini, Davide; Masci, Jonathan 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) https://doi.org/10.1109/CVPR.2017.576	conference	July 2017
Single Pass Spectral Sparsification in Dynamic Streams Kapralov, Michael; Lee, Yin Tat; Musco, Cameron 2014 IEEE 55th Annual Symposium on Foundations of Computer Science (FOCS) https://doi.org/10.1109/FOCS.2014.66	conference	October 2014
Comparison of Descriptor Spaces for Chemical Compound Retrieval and Classification Wale, Nikil; Karypis, George Sixth International Conference on Data Mining (ICDM'06) https://doi.org/10.1109/ICDM.2006.39	conference	December 2006
Graph-Based Multilevel Dimensionality Reduction with Applications to Eigenfaces and Latent Semantic Indexing Sakellaridi, Sophia; Fang, Haw-ren; Saad, Yousef 2008 Seventh International Conference on Machine Learning and Applications https://doi.org/10.1109/ICMLA.2008.140	conference	December 2008
Cascading effects in interdependent networks Shin, Dong-Hoon; Qian, Dajun; Zhang, Junshan IEEE Network, Vol. 28, Issue 4 https://doi.org/10.1109/MNET.2014.6863136	journal	July 2014
The emerging field of signal processing on graphs: Extending high-dimensional data analysis to networks and other irregular domains Shuman, D. I.; Narang, S. K.; Frossard, P. IEEE Signal Processing Magazine, Vol. 30, Issue 3 https://doi.org/10.1109/MSP.2012.2235192	journal	May 2013
Geometric Deep Learning: Going beyond Euclidean data Bronstein, Michael M.; Bruna, Joan; LeCun, Yann IEEE Signal Processing Magazine, Vol. 34, Issue 4 https://doi.org/10.1109/MSP.2017.2693418	journal	July 2017
Kron Reduction of Graphs With Applications to Electrical Networks Dorfler, Florian; Bullo, Francesco IEEE Transactions on Circuits and Systems I: Regular Papers, Vol. 60, Issue 1 https://doi.org/10.1109/TCSI.2012.2215780	journal	January 2013
Weighted Graph Cuts without Eigenvectors A Multilevel Approach Dhillon, Inderjit S.; Guan, Yuqiang; Kulis, Brian IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 29, Issue 11 https://doi.org/10.1109/TPAMI.2007.1115	journal	November 2007
A Multiscale Pyramid Transform for Graph Signals Shuman, David I.; Faraji, Mohammad Javad; Vandergheynst, Pierre IEEE Transactions on Signal Processing, Vol. 64, Issue 8 https://doi.org/10.1109/TSP.2015.2512529	journal	April 2016
Nonlinear Dimensionality Reduction by Locally Linear Embedding Roweis, S. T. Science, Vol. 290, Issue 5500 https://doi.org/10.1126/science.290.5500.2323	journal	December 2000
Multilevel ILU With Reorderings for Diagonal Dominance Saad, Yousef SIAM Journal on Scientific Computing, Vol. 27, Issue 3 https://doi.org/10.1137/030602733	journal	January 2005
Graph Coloring Using Eigenvalue Decomposition Aspvall, Bengt; Gilbert, John R. SIAM Journal on Algebraic Discrete Methods, Vol. 5, Issue 4 https://doi.org/10.1137/0605051	journal	December 1984
A Greedy Strategy for Coarse-Grid Selection MacLachlan, S.; Saad, Yousef SIAM Journal on Scientific Computing, Vol. 29, Issue 5 https://doi.org/10.1137/060654062	journal	January 2007
Greedy Coarsening Strategies for Nonsymmetric Problems MacLachlan, Scott; Saad, Yousef SIAM Journal on Scientific Computing, Vol. 29, Issue 5 https://doi.org/10.1137/060660928	journal	January 2007
Analysis of Aggregation-Based Multigrid Muresan, Adrian C.; Notay, Yvan SIAM Journal on Scientific Computing, Vol. 30, Issue 2 https://doi.org/10.1137/060678397	journal	January 2008
Partitioning Sparse Matrices with Eigenvectors of Graphs Pothen, Alex; Simon, Horst D.; Liou, Kang-Pu SIAM Journal on Matrix Analysis and Applications, Vol. 11, Issue 3 https://doi.org/10.1137/0611030	journal	July 1990
Ordering Methods for Preconditioned Conjugate Gradient Methods Applied to Unstructured Grid Problems D’Azevedo, E. F.; Forsyth, P. A.; Tang, Wei-Pai SIAM Journal on Matrix Analysis and Applications, Vol. 13, Issue 3 https://doi.org/10.1137/0613057	journal	July 1992
Algebraic Multilevel Preconditioning Methods, II Axelsson, O.; Vassilevski, P. S. SIAM Journal on Numerical Analysis, Vol. 27, Issue 6 https://doi.org/10.1137/0727092	journal	December 1990
Algebraic Multilevel Preconditioner for the Helmholtz Equation in Heterogeneous Media Bollhöfer, Matthias; Grote, Marcus J.; Schenk, Olaf SIAM Journal on Scientific Computing, Vol. 31, Issue 5 https://doi.org/10.1137/080725702	journal	January 2009
Graph Sparsification by Effective Resistances Spielman, Daniel A.; Srivastava, Nikhil SIAM Journal on Computing, Vol. 40, Issue 6 https://doi.org/10.1137/080734029	journal	January 2011
Spectral Sparsification of Graphs Spielman, Daniel A.; Teng, Shang-Hua SIAM Journal on Computing, Vol. 40, Issue 4 https://doi.org/10.1137/08074489X	journal	January 2011
Algebraic Distance on Graphs Chen, Jie; Safro, Ilya SIAM Journal on Scientific Computing, Vol. 33, Issue 6 https://doi.org/10.1137/090775087	journal	January 2011
Iterative Methods for Sparse Linear Systems Saad, Yousef https://doi.org/10.1137/1.9780898718003	book	January 2003
A Multigrid Tutorial, Second Edition Briggs, William L.; Henson, Van Emden; McCormick, Steve F. Other Titles in Applied Mathematics https://doi.org/10.1137/1.9780898719505	book	January 2000
Numerical Methods for Large Eigenvalue Problems Saad, Yousef https://doi.org/10.1137/1.9781611970739	book	January 2011
Relaxation-Based Coarsening and Multiscale Graph Organization Ron, Dorit; Safro, Ilya; Brandt, Achi Multiscale Modeling & Simulation, Vol. 9, Issue 1 https://doi.org/10.1137/100791142	journal	January 2011
IMF: An Incomplete Multifrontal $LU$-Factorization for Element-Structured Sparse Linear Systems Vannieuwenhoven, Nick; Meerbergen, Karl SIAM Journal on Scientific Computing, Vol. 35, Issue 1 https://doi.org/10.1137/100818996	journal	January 2013
Modification and Compensation Strategies for Threshold-based Incomplete Factorizations MacLachlan, S.; Osei-Kuffuor, D.; Saad, Yousef SIAM Journal on Scientific Computing, Vol. 34, Issue 1 https://doi.org/10.1137/110834986	journal	January 2012
Matrix Reordering Using Multilevel Graph Coarsening for ILU Preconditioning Osei-Kuffuor, Daniel; Li, Ruipeng; Saad, Yousef SIAM Journal on Scientific Computing, Vol. 37, Issue 1 https://doi.org/10.1137/130936610	journal	January 2015
Matrix Renumbering ILU: An Effective Algebraic Multilevel ILU Preconditioner for Sparse Matrices Botta, E. F. F.; Wubs, F. W. SIAM Journal on Matrix Analysis and Applications, Vol. 20, Issue 4 https://doi.org/10.1137/S0895479897319301	journal	January 1999
Orderings for Incomplete Factorization Preconditioning of Nonsymmetric Problems Benzi, Michele; Szyld, Daniel B.; van Duin, Arno SIAM Journal on Scientific Computing, Vol. 20, Issue 5 https://doi.org/10.1137/S1064827597326845	journal	January 1999
Ordering, Anisotropy, and Factored Sparse Approximate Inverses Bridson, Robert; Tang, Wei-Pai SIAM Journal on Scientific Computing, Vol. 21, Issue 3 https://doi.org/10.1137/S1064827598335842	journal	January 1999
Orderings for Factorized Sparse Approximate Inverse Preconditioners Benzi, Michele; Tuma, Miroslav SIAM Journal on Scientific Computing, Vol. 21, Issue 5 https://doi.org/10.1137/S1064827598339372	journal	January 2000
A Structural Diagnosis of Some IC Orderings Bridson, Robert; Tang, Wei-Pai SIAM Journal on Scientific Computing, Vol. 22, Issue 5 https://doi.org/10.1137/S1064827599353841	journal	January 2001
Graph summarization with bounded error Navlakha, Saket; Rastogi, Rajeev; Shrivastava, Nisheeth Proceedings of the 2008 ACM SIGMOD international conference on Management of data - SIGMOD '08 https://doi.org/10.1145/1376616.1376661	conference	January 2008
Multilevel manifold learning with application to spectral clustering Fang, Haw-ren; Sakellaridi, Sophia; Saad, Yousef Proceedings of the 19th ACM international conference on Information and knowledge management - CIKM '10 https://doi.org/10.1145/1871437.1871493	conference	January 2010
Hypergraph-based multilevel matrix approximation for text information retrieval Fang, Haw-ren; Saad, Yousef Proceedings of the 19th ACM international conference on Information and knowledge management - CIKM '10 https://doi.org/10.1145/1871437.1871681	conference	January 2010
Random sampling in cut, flow, and network design problems Karger, David R. Proceedings of the twenty-sixth annual ACM symposium on Theory of computing - STOC '94 https://doi.org/10.1145/195058.195422	conference	January 1994
Query preserving graph compression Fan, Wenfei; Li, Jianzhong; Wang, Xin Proceedings of the 2012 international conference on Management of Data - SIGMOD '12 https://doi.org/10.1145/2213836.2213855	conference	January 2012
Approximating s-t minimum cuts in Õ ( n ² ) time Benczúr, András A.; Karger, David R. Proceedings of the twenty-eighth annual ACM symposium on Theory of computing - STOC '96 https://doi.org/10.1145/237814.237827	conference	January 1996
Distributed large-scale natural graph factorization Ahmed, Amr; Shervashidze, Nino; Narayanamurthy, Shravan Proceedings of the 22nd international conference on World Wide Web - WWW '13 https://doi.org/10.1145/2488388.2488393	conference	January 2013
Spectral sparsification of graphs: theory and algorithms Batson, Joshua; Spielman, Daniel A.; Srivastava, Nikhil Communications of the ACM, Vol. 56, Issue 8 https://doi.org/10.1145/2492007.2492029	journal	August 2013
DeepWalk: online learning of social representations Perozzi, Bryan; Al-Rfou, Rami; Skiena, Steven Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining - KDD '14 https://doi.org/10.1145/2623330.2623732	conference	January 2014
LINE: Large-scale Information Network Embedding Tang, Jian; Qu, Meng; Wang, Mingzhe WWW '15: 24th International World Wide Web Conference, Proceedings of the 24th International Conference on World Wide Web https://doi.org/10.1145/2736277.2741093	conference	May 2015
Deep Graph Kernels Yanardag, Pinar; Vishwanathan, S. V. N. KDD '15: The 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining https://doi.org/10.1145/2783258.2783417	conference	August 2015
GraRep: Learning Graph Representations with Global Structural Information Cao, Shaosheng; Lu, Wei; Xu, Qiongkai CIKM'15: 24th ACM International Conference on Information and Knowledge Management, Proceedings of the 24th ACM International on Conference on Information and Knowledge Management https://doi.org/10.1145/2806416.2806512	conference	October 2015
node2vec: Scalable Feature Learning for Networks Grover, Aditya; Leskovec, Jure Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining - KDD '16 https://doi.org/10.1145/2939672.2939754	conference	January 2016
Laplacian Eigenmaps for Dimensionality Reduction and Data Representation Belkin, Mikhail; Niyogi, Partha Neural Computation, Vol. 15, Issue 6 https://doi.org/10.1162/089976603321780317	journal	June 2003
Weisfeiler and Leman Go Neural: Higher-Order Graph Neural Networks Morris, Christopher; Ritzert, Martin; Fey, Matthias Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 33 https://doi.org/10.1609/aaai.v33i01.33014602	journal	July 2019
Algebraic connectivity of graphs [Algebraic connectivity of graphs] Fiedler, Miroslav Czechoslovak Mathematical Journal, Vol. 23, Issue 2 https://doi.org/10.21136/CMJ.1973.101168	journal	January 1973

Similar Records

Dr.L: Distributed Recursive (Graph) Layout

Software · 2007 · OSTI ID:code-54626

Martin, Shawn; Brown, W. Michael; Wylie, Brian N.

A Novel Coarsening Method for Scalable and Efficient Mesh Generation

Technical Report · 2010 · OSTI ID:1018444

Yoo, A; Hysom, D; Gunney, B

Accelerate microstructure evolution simulation using graph neural networks with adaptive spatiotemporal resolution

Journal Article · 2024 · Machine Learning: Science and Technology · OSTI ID:2345857

Fan, Shaoxun; Hitt, Andrew L.; Tang, Ming; +2 more

Title: Graph coarsening: from scientific computing to machine learning

Citation Formats

References (80)

Similar Records

Related Subjects