Context-aided analysis of community evolution in networks
Abstract
Here, we are interested in detecting and analyzing global changes in dynamic networks (networks that evolve with time). More precisely, we consider changes in the activity distribution within the network, in terms of density (ie, edge existence) and intensity (ie, edge weight). Detecting change in local properties, as well as individual measurements or metrics, has been well studied and often reduces to traditional statistical process control. In contrast, detecting change in larger scale structure of the network is more challenging and not as well understood. We address this problem by proposing a framework for detecting change in network structure based on separate pieces: a probabilistic model for partitioning nodes by their behavior, a label-unswitching heuristic, and an approach to change detection for sequences of complex objects. We examine the performance of one instantiation of such a framework using mostly previously available pieces. The dataset we use for these investigations is the publicly available New York City Taxi and Limousine Commission dataset covering all taxi trips in New York City since 2009. Using it, we investigate the evolution of an ensemble of networks under different spatiotemporal resolutions. We identify the community structure by fitting a weighted stochastic block model. In conclusion,more »
- Authors:
-
[1];
- Lawrence Livermore National Lab. (LLNL), Livermore, CA (United States). Computational Engineering Division
- Virginia Polytechnic Institute, Blacksburg, VA (United States). Department of Computer Science, Biocomplexity Institute
- Northwestern Univ., Evanston, IL (United States). Department of Engineering Sciences and Applied Mathematics
- Publication Date:
- Research Org.:
- Lawrence Livermore National Lab. (LLNL), Livermore, CA (United States)
- Sponsoring Org.:
- USDOE
- OSTI Identifier:
- 1393360
- Report Number(s):
- LLNL-JRNL-695017
Journal ID: ISSN 1932-1864
- Grant/Contract Number:
- AC52-07NA27344
- Resource Type:
- Accepted Manuscript
- Journal Name:
- Statistical Analysis and Data Mining
- Additional Journal Information:
- Journal Volume: 10; Journal Issue: 5; Journal ID: ISSN 1932-1864
- Publisher:
- Wiley
- Country of Publication:
- United States
- Language:
- English
- Subject:
- 97 MATHEMATICS AND COMPUTING; network analysis; community discovery; change detection; evolving networks
Citation Formats
Pallotta, Giuliana, Konjevod, Goran, Cadena, Jose, and Nguyen, Phan. Context-aided analysis of community evolution in networks. United States: N. p., 2017.
Web. doi:10.1002/sam.11354.
Pallotta, Giuliana, Konjevod, Goran, Cadena, Jose, & Nguyen, Phan. Context-aided analysis of community evolution in networks. United States. https://doi.org/10.1002/sam.11354
Pallotta, Giuliana, Konjevod, Goran, Cadena, Jose, and Nguyen, Phan. Fri .
"Context-aided analysis of community evolution in networks". United States. https://doi.org/10.1002/sam.11354. https://www.osti.gov/servlets/purl/1393360.
@article{osti_1393360,
title = {Context-aided analysis of community evolution in networks},
author = {Pallotta, Giuliana and Konjevod, Goran and Cadena, Jose and Nguyen, Phan},
abstractNote = {Here, we are interested in detecting and analyzing global changes in dynamic networks (networks that evolve with time). More precisely, we consider changes in the activity distribution within the network, in terms of density (ie, edge existence) and intensity (ie, edge weight). Detecting change in local properties, as well as individual measurements or metrics, has been well studied and often reduces to traditional statistical process control. In contrast, detecting change in larger scale structure of the network is more challenging and not as well understood. We address this problem by proposing a framework for detecting change in network structure based on separate pieces: a probabilistic model for partitioning nodes by their behavior, a label-unswitching heuristic, and an approach to change detection for sequences of complex objects. We examine the performance of one instantiation of such a framework using mostly previously available pieces. The dataset we use for these investigations is the publicly available New York City Taxi and Limousine Commission dataset covering all taxi trips in New York City since 2009. Using it, we investigate the evolution of an ensemble of networks under different spatiotemporal resolutions. We identify the community structure by fitting a weighted stochastic block model. In conclusion, we offer insights on different node ranking and clustering methods, their ability to capture the rhythm of life in the Big Apple, and their potential usefulness in highlighting changes in the underlying network structure.},
doi = {10.1002/sam.11354},
journal = {Statistical Analysis and Data Mining},
number = 5,
volume = 10,
place = {United States},
year = {2017},
month = {9}
}
Free Publicly Available Full Text
Publisher's Version of Record
Other availability
Search WorldCat to find libraries that may hold this journal
Save to My Library
You must Sign In or Create an Account in order to save documents to your library.
Works referenced in this record:
An overview and perspective on social network monitoring
journal, August 2016
- Woodall, William H.; Zhao, Meng J.; Paynabar, Kamran
- IISE Transactions, Vol. 49, Issue 3
Mapping Change in Large Networks
journal, January 2010
- Rosvall, Martin; Bergstrom, Carl T.
- PLoS ONE, Vol. 5, Issue 1
Visual Exploration of Big Spatio-Temporal Urban Data: A Study of New York City Taxi Trips
journal, December 2013
- Ferreira, Nivan; Poco, Jorge; Vo, Huy T.
- IEEE Transactions on Visualization and Computer Graphics, Vol. 19, Issue 12
Learning latent block structure in weighted networks
journal, June 2014
- Aicher, C.; Jacobs, A. Z.; Clauset, A.
- Journal of Complex Networks, Vol. 3, Issue 2
Comparing clusterings—an information based distance
journal, May 2007
- Meilă, Marina
- Journal of Multivariate Analysis, Vol. 98, Issue 5
Model selection for degree-corrected block models
journal, May 2014
- Yan, Xiaoran; Shalizi, Cosma; Jensen, Jacob E.
- Journal of Statistical Mechanics: Theory and Experiment, Vol. 2014, Issue 5
Hypothesis testing for automated community detection in networks
journal, May 2015
- Bickel, Peter J.; Sarkar, Purnamrita
- Journal of the Royal Statistical Society: Series B (Statistical Methodology), Vol. 78, Issue 1
Graph-based change-point detection
journal, February 2015
- Chen, Hao; Zhang, Nancy
- The Annals of Statistics, Vol. 43, Issue 1
Goodness of Fit of Social Network Models
journal, March 2008
- Hunter, David R.; Goodreau, Steven M.; Handcock, Mark S.
- Journal of the American Statistical Association, Vol. 103, Issue 481
Finding and evaluating community structure in networks
journal, February 2004
- Newman, M. E. J.; Girvan, M.
- Physical Review E, Vol. 69, Issue 2
The Accuracy of the Plating Method of Estimating the Density of Bacterial Populations.: with Particular Reference to the use of Thornton'S agar Medium with soil Samples
journal, November 1922
- Fisher, R. A.; Thornton, H. G.; Mackenzie, W. A.
- Annals of Applied Biology, Vol. 9, Issue 3-4
Stochastic blockmodels and community structure in networks
journal, January 2011
- Karrer, Brian; Newman, M. E. J.
- Physical Review E, Vol. 83, Issue 1
Big data computation of taxi movement in New York City
conference, December 2016
- Deri, Joya A.; Franchetti, Franz; Moura, Jose M. F.
- 2016 IEEE International Conference on Big Data (Big Data)
Improved Bayesian inference for the stochastic block model with application to large networks
journal, April 2013
- McDaid, Aaron F.; Murphy, Thomas Brendan; Friel, Nial
- Computational Statistics & Data Analysis, Vol. 60
A Consistent Adjacency Spectral Embedding for Stochastic Blockmodel Graphs
journal, September 2012
- Sussman, Daniel L.; Tang, Minh; Fishkind, Donniell E.
- Journal of the American Statistical Association, Vol. 107, Issue 499
Stochastic blockmodels: First steps
journal, June 1983
- Holland, Paul W.; Laskey, Kathryn Blackmond; Leinhardt, Samuel
- Social Networks, Vol. 5, Issue 2
Aggregating inconsistent information: Ranking and clustering
journal, October 2008
- Ailon, Nir; Charikar, Moses; Newman, Alantha
- Journal of the ACM, Vol. 55, Issue 5
Social Structure from Multiple Networks. I. Blockmodels of Roles and Positions
journal, January 1976
- White, Harrison C.; Boorman, Scott A.; Breiger, Ronald L.
- American Journal of Sociology, Vol. 81, Issue 4
Bayesian finite mixtures with an unknown number of components: The allocation sampler
journal, February 2007
- Nobile, Agostino; Fearnside, Alastair T.
- Statistics and Computing, Vol. 17, Issue 2
Control Chart Tests Based on Geometric Moving Averages
journal, August 1959
- Roberts, S. W.
- Technometrics, Vol. 1, Issue 3
Taxi data in New York city: A network perspective
conference, November 2015
- Deri, Joya A.; Moura, Jose M. F.
- 2015 49th Asilomar Conference on Signals, Systems and Computers
Mining social networks for anomalies: Methods and challenges
journal, June 2016
- Bindu, P. V.; Thilagam, P. Santhi
- Journal of Network and Computer Applications, Vol. 68
Missing and spurious interactions and the reconstruction of complex networks
journal, December 2009
- Guimerà, Roger; Sales-Pardo, Marta
- Proceedings of the National Academy of Sciences, Vol. 106, Issue 52
Community detection in graphs
journal, February 2010
- Fortunato, Santo
- Physics Reports, Vol. 486, Issue 3-5
Sequential detection of temporal communities by estrangement confinement
journal, November 2012
- Kawadia, Vikas; Sreenivasan, Sameet
- Scientific Reports, Vol. 2, Issue 1
Benchmark graphs for testing community detection algorithms
journal, October 2008
- Lancichinetti, Andrea; Fortunato, Santo; Radicchi, Filippo
- Physical Review E, Vol. 78, Issue 4
A nonparametric view of network models and Newman–Girvan and other modularities
journal, November 2009
- Bickel, Peter J.; Chen, Aiyou
- Proceedings of the National Academy of Sciences, Vol. 106, Issue 50
Anomaly detection in online social networks
journal, October 2014
- Savage, David; Zhang, Xiuzhen; Yu, Xinghuo
- Social Networks, Vol. 39