Simple, direct and efficient multi-way spectral clustering

Damle, Anil; Minden, Victor; Ying, Lexing

doi:10.1093/imaiai/iay008

Title: Simple, direct and efficient multi-way spectral clustering

Journal Article · 27 June 2018 · Information and Inference (Online)

DOI: https://doi.org/10.1093/imaiai/iay008 · OSTI ID:1457488

Damle, Anil ^[1]; Minden, Victor ^[2]; Ying, Lexing ^[3]

Department of Computer Science, Cornell University, Gates Hall, Ithaca, NY
Center for Computational Biology, Flatiron Institute, Fifth Avenue, New York, NY
Department of Mathematics and Institute for Computational & Mathematical Engineering, Stanford University, Serra Mall, Bldg, Stanford, CA

Abstract We present a new algorithm for spectral clustering based on a column-pivoted QR factorization that may be directly used for cluster assignment or to provide an initial guess for k-means. Our algorithm is simple to implement, direct and requires no initial guess. Furthermore, it scales linearly in the number of nodes of the graph and a randomized variant provides significant computational gains. Provided the subspace spanned by the eigenvectors used for clustering contains a basis that resembles the set of indicator vectors on the clusters, we prove that both our deterministic and randomized algorithms recover a basis close to the indicators in Frobenius norm. We also experimentally demonstrate that the performance of our algorithm tracks recent information theoretic bounds for exact recovery in the stochastic block model. Finally, we explore the performance of our algorithm when applied to a real-world graph.

View Journal Article

Cite

Export

Save

Sponsoring Organization:: USDOE

Grant/Contract Number:: FG02-97ER25308; SC0009409

OSTI ID:: 1457488

Journal Information:: Information and Inference (Online), Journal Name: Information and Inference (Online) Journal Issue: 1 Vol. 8; ISSN 2049-8772

Publisher:: Oxford University PressCopyright Statement

Country of Publication:: United Kingdom

Language:: English

References (31)

Linear least squares solutions by householder transformations Businger, Peter; Golub, Gene H. Numerische Mathematik, Vol. 7, Issue 3 https://doi.org/10.1007/BF01436084	journal	June 1965
A tutorial on spectral clustering von Luxburg, Ulrike Statistics and Computing, Vol. 17, Issue 4 https://doi.org/10.1007/s11222-007-9033-z	journal	August 2007
Stochastic blockmodels: First steps Holland, Paul W.; Laskey, Kathryn Blackmond; Leinhardt, Samuel Social Networks, Vol. 5, Issue 2 https://doi.org/10.1016/0378-8733(83)90021-7	journal	June 1983
A comparative study of efficient initialization methods for the k-means clustering algorithm Celebi, M. Emre; Kingravi, Hassan A.; Vela, Patricio A. Expert Systems with Applications, Vol. 40, Issue 1 https://doi.org/10.1016/j.eswa.2012.07.021	journal	January 2013
Compressed Representation of Kohn–Sham Orbitals via Selected Columns of the Density Matrix Damle, Anil; Lin, Lin; Ying, Lexing Journal of Chemical Theory and Computation, Vol. 11, Issue 4 https://doi.org/10.1021/ct500985f	journal	March 2015
CUR matrix decompositions for improved data analysis Mahoney, Michael W.; Drineas, Petros Proceedings of the National Academy of Sciences, Vol. 106, Issue 3 https://doi.org/10.1073/pnas.0803205106	journal	January 2009
Spectral redemption in clustering sparse networks Krzakala, F.; Moore, C.; Mossel, E. Proceedings of the National Academy of Sciences, Vol. 110, Issue 52 https://doi.org/10.1073/pnas.1312486110	journal	November 2013
Some metric inequalities in the space of matrices Fan, Ky; Hoffman, A. J. Proceedings of the American Mathematical Society, Vol. 6, Issue 1 https://doi.org/10.1090/S0002-9939-1955-0067841-7	journal	January 1955
Community Detection in General Stochastic Block models: Fundamental Limits and Efficient Algorithms for Recovery Abbe, Emmanuel; Sandon, Colin 2015 IEEE 56th Annual Symposium on Foundations of Computer Science (FOCS) https://doi.org/10.1109/FOCS.2015.47	conference	October 2015
Achieving exact cluster recovery threshold via semidefinite programming Hajek, Bruce; Wu, Yihong; Xu, Jiaming 2015 IEEE International Symposium on Information Theory (ISIT) https://doi.org/10.1109/ISIT.2015.7282694	conference	June 2015
Least squares quantization in PCM Lloyd, S. IEEE Transactions on Information Theory, Vol. 28, Issue 2 https://doi.org/10.1109/TIT.1982.1056489	journal	March 1982
Exact Recovery in the Stochastic Block Model Abbe, Emmanuel; Bandeira, Afonso S.; Hall, Georgina IEEE Transactions on Information Theory, Vol. 62, Issue 1 https://doi.org/10.1109/TIT.2015.2490670	journal	January 2016
Achieving Exact Cluster Recovery Threshold via Semidefinite Programming Hajek, Bruce; Wu, Yihong; Xu, Jiaming IEEE Transactions on Information Theory, Vol. 62, Issue 5 https://doi.org/10.1109/TIT.2016.2546280	journal	May 2016
The Rotation of Eigenvectors by a Perturbation. III Davis, Chandler; Kahan, W. M. SIAM Journal on Numerical Analysis, Vol. 7, Issue 1 https://doi.org/10.1137/0707001	journal	March 1970
Computing the Polar Decomposition—with Applications Higham, Nicholas J. SIAM Journal on Scientific and Statistical Computing, Vol. 7, Issue 4 https://doi.org/10.1137/0907079	journal	October 1986
Efficient Algorithms for Computing a Strong Rank-Revealing QR Factorization Gu, Ming; Eisenstat, Stanley C. SIAM Journal on Scientific Computing, Vol. 17, Issue 4 https://doi.org/10.1137/0917055	journal	July 1996
LAPACK Users' Guide Anderson, E.; Bai, Z.; Bischof, C. https://doi.org/10.1137/1.9780898719604	software	January 1999
Partitioning into Expanders Gharan, Shayan Oveis; Trevisan, Luca Proceedings of the Twenty-Fifth Annual ACM-SIAM Symposium on Discrete Algorithms https://doi.org/10.1137/1.9781611973402.93	conference	January 2014
Decay Properties of Spectral Projectors with Applications to Electronic Structure Benzi, Michele; Boito, Paola; Razouk, Nader SIAM Review, Vol. 55, Issue 1 https://doi.org/10.1137/100814019	journal	January 2013
Computing Localized Representations of the Kohn--Sham Subspace Via Randomization and Refinement Damle, Anil; Lin, Lin; Ying, Lexing SIAM Journal on Scientific Computing, Vol. 39, Issue 6 https://doi.org/10.1137/16M1098589	journal	January 2017
On Rank-Revealing Factorisations Chandrasekaran, Shivkumar; Ipsen, Ilse C. F. SIAM Journal on Matrix Analysis and Applications, Vol. 15, Issue 2 https://doi.org/10.1137/S0895479891223781	journal	April 1994
New Perturbation Bounds for the Unitary Polar Factor Li, Ren-Cang SIAM Journal on Matrix Analysis and Applications, Vol. 16, Issue 1 https://doi.org/10.1137/S0895479893256359	journal	January 1995
A BLAS-3 Version of the QR Factorization with Column Pivoting Quintana-Ortí, Gregorio; Sun, Xiaobai; Bischof, Christian H. SIAM Journal on Scientific Computing, Vol. 19, Issue 5 https://doi.org/10.1137/S1064827595296732	journal	September 1998
Fast monte-carlo algorithms for finding low-rank approximations Frieze, Alan; Kannan, Ravi; Vempala, Santosh Journal of the ACM, Vol. 51, Issue 6 https://doi.org/10.1145/1039488.1039494	journal	November 2004
Semidefinite programs on sparse random graphs and their application to community detection Montanari, Andrea; Sen, Subhabrata Proceedings of the 48th Annual ACM SIGACT Symposium on Theory of Computing - STOC 2016 https://doi.org/10.1145/2897518.2897548	conference	January 2016
Lower Bounds for the Partitioning of Graphs Donath, W. E.; Hoffman, A. J. IBM Journal of Research and Development, Vol. 17, Issue 5 https://doi.org/10.1147/rd.175.0420	journal	September 1973
Spectral clustering and the high-dimensional stochastic blockmodel Rohe, Karl; Chatterjee, Sourav; Yu, Bin The Annals of Statistics, Vol. 39, Issue 4 https://doi.org/10.1214/11-AOS887	journal	August 2011
The geometry of kernelized spectral clustering Schiebinger, Geoffrey; Wainwright, Martin J.; Yu, Bin The Annals of Statistics, Vol. 43, Issue 2 https://doi.org/10.1214/14-AOS1283	journal	April 2015
Sharp nonasymptotic bounds on the norm of random matrices with independent entries Bandeira, Afonso S.; van Handel, Ramon The Annals of Probability, Vol. 44, Issue 4 https://doi.org/10.1214/15-AOP1025	journal	July 2016
An Introduction to Matrix Concentration Inequalities Tropp, Joel A. Foundations and Trends® in Machine Learning, Vol. 8, Issue 1-2 https://doi.org/10.1561/2200000048	journal	January 2015
Algebraic connectivity of graphs [Algebraic connectivity of graphs] Fiedler, Miroslav Czechoslovak Mathematical Journal, Vol. 23, Issue 2 https://doi.org/10.21136/CMJ.1973.101168	journal	January 1973

Similar Records

Numerical aspects of spectral segmentation on polygonal grids

Conference · 2010 · OSTI ID:1000927

Matsekh, Anna; Skurikhin, Alexei; Prasad, Lakshman; +1 more

Simulated Half-Precision Implementation of Blocked QR Factorization and Graph Clustering Applications

Technical Report · 2018 · OSTI ID:1466174

Yang, Lucia Minah; Sanders, Geoffrey D.

Inverse Subspace Iteration for Spectral Stochastic Finite Element Methods

Journal Article · 2016 · SIAM/ASA Journal on Uncertainty Quantification · OSTI ID:1418636

Sousedík, Bedřich; Elman, Howard C.

Title: Simple, direct and efficient multi-way spectral clustering

Citation Formats

References (31)

Similar Records

Related Subjects