Monte Carlo goodness-of-fit tests for degree corrected and related stochastic blockmodels

Karwa, Vishesh; Pati, Debdeep; Petrović, Sonja; Solus, Liam; Alexeev, Nikita; Raič, Mateja; Wilburne, Dane; Williams, Robert; Yan, Bowei

doi:10.1093/jrsssb/qkad084

Monte Carlo goodness-of-fit tests for degree corrected and related stochastic blockmodels

Journal Article · Fri Sep 15 00:00:00 EDT 2023 · Journal of the Royal Statistical Society: Series B (Statistical Methodology)

DOI:https://doi.org/10.1093/jrsssb/qkad084· OSTI ID:2000588

Karwa, Vishesh; Pati, Debdeep; Petrović, Sonja; Solus, Liam; ; Raič, Mateja; Wilburne, Dane; Williams, Robert; Yan, Bowei

Abstract

We construct Bayesian and frequentist finite-sample goodness-of-fit tests for three different variants of the stochastic blockmodel for network data. Since all of the stochastic blockmodel variants are log-linear in form when block assignments are known, the tests for the latent block model versions combine a block membership estimator with the algebraic statistics machinery for testing goodness-of-fit in log-linear models. We describe Markov bases and marginal polytopes of the variants of the stochastic blockmodel and discuss how both facilitate the development of goodness-of-fit tests and understanding of model behaviour. The general testing methodology developed here extends to any finite mixture of log-linear models on discrete data, and as such is the first application of the algebraic statistics machinery for latent-variable models.

View Journal Article

Sponsoring Organization:: USDOE

OSTI ID:: 2000588

Journal Information:: Journal of the Royal Statistical Society: Series B (Statistical Methodology), Journal Name: Journal of the Royal Statistical Society: Series B (Statistical Methodology) Journal Issue: 1 Vol. 86; ISSN 1369-7412

Publisher:: Oxford University PressCopyright Statement

Country of Publication:: United Kingdom

Language:: English

References (40)

Goodness of fit for log-linear network models: dynamic Markov bases using hypergraphs Gross, Elizabeth; Petrović, Sonja; Stasi, Despina Annals of the Institute of Statistical Mathematics, Vol. 69, Issue 3 https://doi.org/10.1007/s10463-016-0560-2	journal	April 2016
Posterior Contraction Rates for Stochastic Block Models Ghosh, Prasenjit; Pati, Debdeep; Bhattacharya, Anirban Sankhya A, Vol. 82, Issue 2 https://doi.org/10.1007/s13171-019-00180-5	journal	October 2019
Stochastic blockmodels: First steps Holland, Paul W.; Laskey, Kathryn Blackmond; Leinhardt, Samuel Social Networks, Vol. 5, Issue 2 https://doi.org/10.1016/0378-8733(83)90021-7	journal	June 1983
The minimal preprocessing pipelines for the Human Connectome Project Glasser, Matthew F.; Sotiropoulos, Stamatios N.; Wilson, J. Anthony NeuroImage, Vol. 80 https://doi.org/10.1016/j.neuroimage.2013.04.127	journal	October 2013
Tensor network factorizations: Relationships between brain structural connectomes and traits Zhang, Zhengwu; Allen, Genevera I.; Zhu, Hongtu NeuroImage, Vol. 197 https://doi.org/10.1016/j.neuroimage.2019.04.027	journal	August 2019
An introduction to exponential random graph (p*) models for social networks Robins, Garry; Pattison, Pip; Kalish, Yuval Social Networks, Vol. 29, Issue 2 https://doi.org/10.1016/j.socnet.2006.08.002	journal	May 2007
A nonparametric view of network models and Newman–Girvan and other modularities Bickel, Peter J.; Chen, Aiyou Proceedings of the National Academy of Sciences, Vol. 106, Issue 50 https://doi.org/10.1073/pnas.0907096106	journal	November 2009
Community structure in social and biological networks Girvan, M.; Newman, M. E. J. Proceedings of the National Academy of Sciences, Vol. 99, Issue 12 https://doi.org/10.1073/pnas.122653799	journal	June 2002
An Exponential Family of Probability Distributions for Directed Graphs Holland, Paul W.; Leinhardt, Samuel Journal of the American Statistical Association, Vol. 76, Issue 373 https://doi.org/10.1080/01621459.1981.10477598	journal	March 1981
Statistical Analysis of Multiple Sociometric Relations Fienberg, Stephen E.; Meyer, Michael M.; Wasserman, Stanley S. Journal of the American Statistical Association, Vol. 80, Issue 389 https://doi.org/10.1080/01621459.1985.10477129	journal	March 1985
Markov Graphs Frank, Ove; Strauss, David Journal of the American Statistical Association, Vol. 81, Issue 395 https://doi.org/10.1080/01621459.1986.10478342	journal	September 1986
A Warning on the Use of Chi-Squared Statistics with Frequency Tables with Small Expected Cell Counts Haberman, Shelby J. Journal of the American Statistical Association, Vol. 83, Issue 402 https://doi.org/10.1080/01621459.1988.10478632	journal	June 1988
Probabilistic Community Detection With Unknown Number of Communities Geng, Junxian; Bhattacharya, Anirban; Pati, Debdeep Journal of the American Statistical Association, Vol. 114, Issue 526 https://doi.org/10.1080/01621459.2018.1458618	journal	July 2018
Covariate Regularized Community Detection in Sparse Graphs Yan, Bowei; Sarkar, Purnamrita Journal of the American Statistical Association, Vol. 116, Issue 534 https://doi.org/10.1080/01621459.2019.1706541	journal	February 2020
A Brief History of Statistical Models for Network Analysis and Open Challenges Fienberg, Stephen E. Journal of Computational and Graphical Statistics, Vol. 21, Issue 4 https://doi.org/10.1080/10618600.2012.738106	journal	October 2012
An Approximation Method for Improving Dynamic Network Model Fitting Carnegie, Nicole Bohme; Krivitsky, Pavel N.; Hunter, David R. Journal of Computational and Graphical Statistics, Vol. 24, Issue 2 https://doi.org/10.1080/10618600.2014.903087	journal	April 2015
An Information Flow Model for Conflict and Fission in Small Groups Zachary, Wayne W. Journal of Anthropological Research, Vol. 33, Issue 4 https://doi.org/10.1086/jar.33.4.3629752	journal	December 1977
What is... a Markov Basis? Petrović, Sonja Notices of the American Mathematical Society, Vol. 66, Issue 07 https://doi.org/10.1090/noti1904	journal	August 2019
Model choice: a minimum posterior predictive loss approach Gelfand, A. Biometrika, Vol. 85, Issue 1 https://doi.org/10.1093/biomet/85.1.1	journal	March 1998
Stochastic blockmodels and community structure in networks Karrer, Brian; Newman, M. E. J. Physical Review E, Vol. 83, Issue 1 https://doi.org/10.1103/PhysRevE.83.016107	journal	January 2011
Estimating the Number of Communities in a Network Newman, M. E. J.; Reinert, Gesine Physical Review Letters, Vol. 117, Issue 7 https://doi.org/10.1103/PhysRevLett.117.078301	journal	August 2016
Statistical clustering of temporal networks through a dynamic stochastic block model Matias, Catherine; Miele, Vincent Journal of the Royal Statistical Society: Series B (Statistical Methodology), Vol. 79, Issue 4 https://doi.org/10.1111/rssb.12200	journal	August 2016
Algebraic Statistics in Practice: Applications to Networks Casanellas, Marta; Petrović, Sonja; Uhler, Caroline Annual Review of Statistics and Its Application, Vol. 7, Issue 1 https://doi.org/10.1146/annurev-statistics-031017-100053	journal	March 2020
Estimation and Prediction for Stochastic Blockstructures Nowicki, Krzysztof; Snijders, Tom A. B. Journal of the American Statistical Association, Vol. 96, Issue 455 https://doi.org/10.1198/016214501753208735	journal	September 2001
Latent Space Approaches to Social Network Analysis Hoff, Peter D.; Raftery, Adrian E.; Handcock, Mark S. Journal of the American Statistical Association, Vol. 97, Issue 460 https://doi.org/10.1198/016214502388618906	journal	December 2002
Goodness of Fit of Social Network Models Hunter, David R.; Goodreau, Steven M.; Handcock, Mark S. Journal of the American Statistical Association, Vol. 103, Issue 481 https://doi.org/10.1198/016214507000000446	journal	March 2008
Likelihood inference in exponential families and directions of recession Geyer, Charles J. Electronic Journal of Statistics, Vol. 3, Issue none https://doi.org/10.1214/08-EJS349	journal	January 2009
Random graphs with a given degree sequence Chatterjee, Sourav; Diaconis, Persi; Sly, Allan The Annals of Applied Probability, Vol. 21, Issue 4 https://doi.org/10.1214/10-AAP728	journal	August 2011
Maximum lilkelihood estimation in the $\beta$-model Rinaldo, Alessandro; Petrović, Sonja; Fienberg, Stephen E. The Annals of Statistics, Vol. 41, Issue 3 https://doi.org/10.1214/12-AOS1078	journal	June 2013
Asymptotic normality of maximum likelihood and its variational approximation for stochastic blockmodels Bickel, Peter; Choi, David; Chang, Xiangyu The Annals of Statistics, Vol. 41, Issue 4 https://doi.org/10.1214/13-AOS1124	journal	August 2013
Pseudo-likelihood methods for community detection in large sparse networks Amini, Arash A.; Chen, Aiyou; Bickel, Peter J. The Annals of Statistics, Vol. 41, Issue 4 https://doi.org/10.1214/13-AOS1138	journal	August 2013
Inference using noisy degrees: Differentially private $\beta$-model and synthetic graphs Karwa, Vishesh; Slavković, Aleksandra The Annals of Statistics, Vol. 44, Issue 1 https://doi.org/10.1214/15-AOS1358	journal	February 2016
A goodness-of-fit test for stochastic block models Lei, Jing The Annals of Statistics, Vol. 44, Issue 1 https://doi.org/10.1214/15-AOS1370	journal	February 2016
Bayesian degree-corrected stochastic blockmodels for community detection Peng, Lijun; Carvalho, Luis Electronic Journal of Statistics, Vol. 10, Issue 2 https://doi.org/10.1214/16-EJS1163	journal	January 2016
Random Graphs Gilbert, E. N. The Annals of Mathematical Statistics, Vol. 30, Issue 4 https://doi.org/10.1214/aoms/1177706098	journal	December 1959
Algebraic algorithms for sampling from conditional distributions Diaconis, Persi; Sturmfels, Bernd The Annals of Statistics, Vol. 26, Issue 1 https://doi.org/10.1214/aos/1030563990	journal	February 1998
Posterior Predictive $p$-Values Meng, Xiao-Li The Annals of Statistics, Vol. 22, Issue 3 https://doi.org/10.1214/aos/1176325622	journal	September 1994
A Survey of Statistical Network Models Goldenberg, Anna Foundations and Trends® in Machine Learning, Vol. 2, Issue 2 https://doi.org/10.1561/2200000005	journal	January 2009
ergm: A Package to Fit, Simulate and Diagnose Exponential-Family Models for Networks Hunter, David R.; Handcock, Mark S.; Butts, Carter T. Journal of Statistical Software, Vol. 24, Issue 3 https://doi.org/10.18637/jss.v024.i03	journal	January 2008
Categorical Data Analysis of Single Sociometric Relations Fienberg, Stephen E.; Wasserman, Stanley S. Sociological Methodology, Vol. 12 https://doi.org/10.2307/270741	journal	January 1981

Similar Records

Generalized blockmodeling of multiple valued networks.

Journal Article · Wed Aug 31 20:00:00 EDT 2011 · Social Networks · OSTI ID:1426919

Standard and goodness-of-fit parameter estimation methods for the three-parameter lognormal distribution

Journal Article · Thu Dec 31 23:00:00 EST 1981 · Commun. Stat., (Part A), Theory Methods; (United States) · OSTI ID:5435067

A measure of the goodness of fit in unbinned likelihood fits

Journal Article · Mon Jul 01 00:00:00 EDT 2002 · Phys.Rev. · OSTI ID:1155672

Monte Carlo goodness-of-fit tests for degree corrected and related stochastic blockmodels

Citation Formats

References (40)

Similar Records

Related Subjects