skip to main content
DOE PAGES title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Context-aided analysis of community evolution in networks

Abstract

Here, we are interested in detecting and analyzing global changes in dynamic networks (networks that evolve with time). More precisely, we consider changes in the activity distribution within the network, in terms of density (ie, edge existence) and intensity (ie, edge weight). Detecting change in local properties, as well as individual measurements or metrics, has been well studied and often reduces to traditional statistical process control. In contrast, detecting change in larger scale structure of the network is more challenging and not as well understood. We address this problem by proposing a framework for detecting change in network structure based on separate pieces: a probabilistic model for partitioning nodes by their behavior, a label-unswitching heuristic, and an approach to change detection for sequences of complex objects. We examine the performance of one instantiation of such a framework using mostly previously available pieces. The dataset we use for these investigations is the publicly available New York City Taxi and Limousine Commission dataset covering all taxi trips in New York City since 2009. Using it, we investigate the evolution of an ensemble of networks under different spatiotemporal resolutions. We identify the community structure by fitting a weighted stochastic block model. In conclusion,more » we offer insights on different node ranking and clustering methods, their ability to capture the rhythm of life in the Big Apple, and their potential usefulness in highlighting changes in the underlying network structure.« less

Authors:
 [1]; ORCiD logo [1];  [2];  [3]
  1. Lawrence Livermore National Lab. (LLNL), Livermore, CA (United States). Computational Engineering Division
  2. Virginia Polytechnic Institute, Blacksburg, VA (United States). Department of Computer Science, Biocomplexity Institute
  3. Northwestern Univ., Evanston, IL (United States). Department of Engineering Sciences and Applied Mathematics
Publication Date:
Research Org.:
Lawrence Livermore National Lab. (LLNL), Livermore, CA (United States)
Sponsoring Org.:
USDOE
OSTI Identifier:
1393360
Report Number(s):
LLNL-JRNL-695017
Journal ID: ISSN 1932-1864
Grant/Contract Number:  
AC52-07NA27344
Resource Type:
Accepted Manuscript
Journal Name:
Statistical Analysis and Data Mining
Additional Journal Information:
Journal Volume: 10; Journal Issue: 5; Journal ID: ISSN 1932-1864
Publisher:
Wiley
Country of Publication:
United States
Language:
English
Subject:
97 MATHEMATICS AND COMPUTING; network analysis; community discovery; change detection; evolving networks

Citation Formats

Pallotta, Giuliana, Konjevod, Goran, Cadena, Jose, and Nguyen, Phan. Context-aided analysis of community evolution in networks. United States: N. p., 2017. Web. doi:10.1002/sam.11354.
Pallotta, Giuliana, Konjevod, Goran, Cadena, Jose, & Nguyen, Phan. Context-aided analysis of community evolution in networks. United States. doi:10.1002/sam.11354.
Pallotta, Giuliana, Konjevod, Goran, Cadena, Jose, and Nguyen, Phan. Fri . "Context-aided analysis of community evolution in networks". United States. doi:10.1002/sam.11354. https://www.osti.gov/servlets/purl/1393360.
@article{osti_1393360,
title = {Context-aided analysis of community evolution in networks},
author = {Pallotta, Giuliana and Konjevod, Goran and Cadena, Jose and Nguyen, Phan},
abstractNote = {Here, we are interested in detecting and analyzing global changes in dynamic networks (networks that evolve with time). More precisely, we consider changes in the activity distribution within the network, in terms of density (ie, edge existence) and intensity (ie, edge weight). Detecting change in local properties, as well as individual measurements or metrics, has been well studied and often reduces to traditional statistical process control. In contrast, detecting change in larger scale structure of the network is more challenging and not as well understood. We address this problem by proposing a framework for detecting change in network structure based on separate pieces: a probabilistic model for partitioning nodes by their behavior, a label-unswitching heuristic, and an approach to change detection for sequences of complex objects. We examine the performance of one instantiation of such a framework using mostly previously available pieces. The dataset we use for these investigations is the publicly available New York City Taxi and Limousine Commission dataset covering all taxi trips in New York City since 2009. Using it, we investigate the evolution of an ensemble of networks under different spatiotemporal resolutions. We identify the community structure by fitting a weighted stochastic block model. In conclusion, we offer insights on different node ranking and clustering methods, their ability to capture the rhythm of life in the Big Apple, and their potential usefulness in highlighting changes in the underlying network structure.},
doi = {10.1002/sam.11354},
journal = {Statistical Analysis and Data Mining},
number = 5,
volume = 10,
place = {United States},
year = {2017},
month = {9}
}

Journal Article:
Free Publicly Available Full Text
Publisher's Version of Record

Save / Share:

Works referenced in this record:

An overview and perspective on social network monitoring
journal, August 2016


Mapping Change in Large Networks
journal, January 2010


Visual Exploration of Big Spatio-Temporal Urban Data: A Study of New York City Taxi Trips
journal, December 2013

  • Ferreira, Nivan; Poco, Jorge; Vo, Huy T.
  • IEEE Transactions on Visualization and Computer Graphics, Vol. 19, Issue 12
  • DOI: 10.1109/TVCG.2013.226

Learning latent block structure in weighted networks
journal, June 2014

  • Aicher, C.; Jacobs, A. Z.; Clauset, A.
  • Journal of Complex Networks, Vol. 3, Issue 2
  • DOI: 10.1093/comnet/cnu026

Comparing clusterings—an information based distance
journal, May 2007


Model selection for degree-corrected block models
journal, May 2014

  • Yan, Xiaoran; Shalizi, Cosma; Jensen, Jacob E.
  • Journal of Statistical Mechanics: Theory and Experiment, Vol. 2014, Issue 5
  • DOI: 10.1088/1742-5468/2014/05/P05007

Hypothesis testing for automated community detection in networks
journal, May 2015

  • Bickel, Peter J.; Sarkar, Purnamrita
  • Journal of the Royal Statistical Society: Series B (Statistical Methodology), Vol. 78, Issue 1
  • DOI: 10.1111/rssb.12117

Graph-based change-point detection
journal, February 2015

  • Chen, Hao; Zhang, Nancy
  • The Annals of Statistics, Vol. 43, Issue 1
  • DOI: 10.1214/14-AOS1269

Goodness of Fit of Social Network Models
journal, March 2008

  • Hunter, David R.; Goodreau, Steven M.; Handcock, Mark S.
  • Journal of the American Statistical Association, Vol. 103, Issue 481
  • DOI: 10.1198/016214507000000446

Finding and evaluating community structure in networks
journal, February 2004


Stochastic blockmodels and community structure in networks
journal, January 2011


Big data computation of taxi movement in New York City
conference, December 2016

  • Deri, Joya A.; Franchetti, Franz; Moura, Jose M. F.
  • 2016 IEEE International Conference on Big Data (Big Data)
  • DOI: 10.1109/BigData.2016.7840904

Improved Bayesian inference for the stochastic block model with application to large networks
journal, April 2013

  • McDaid, Aaron F.; Murphy, Thomas Brendan; Friel, Nial
  • Computational Statistics & Data Analysis, Vol. 60
  • DOI: 10.1016/j.csda.2012.10.021

A Consistent Adjacency Spectral Embedding for Stochastic Blockmodel Graphs
journal, September 2012

  • Sussman, Daniel L.; Tang, Minh; Fishkind, Donniell E.
  • Journal of the American Statistical Association, Vol. 107, Issue 499
  • DOI: 10.1080/01621459.2012.699795

Stochastic blockmodels: First steps
journal, June 1983


Aggregating inconsistent information: Ranking and clustering
journal, October 2008


Social Structure from Multiple Networks. I. Blockmodels of Roles and Positions
journal, January 1976

  • White, Harrison C.; Boorman, Scott A.; Breiger, Ronald L.
  • American Journal of Sociology, Vol. 81, Issue 4
  • DOI: 10.1086/226141

Bayesian finite mixtures with an unknown number of components: The allocation sampler
journal, February 2007


Control Chart Tests Based on Geometric Moving Averages
journal, August 1959


Taxi data in New York city: A network perspective
conference, November 2015

  • Deri, Joya A.; Moura, Jose M. F.
  • 2015 49th Asilomar Conference on Signals, Systems and Computers
  • DOI: 10.1109/ACSSC.2015.7421468

Cumulative Sum Charts
journal, February 1961


Mining social networks for anomalies: Methods and challenges
journal, June 2016


Missing and spurious interactions and the reconstruction of complex networks
journal, December 2009

  • Guimerà, Roger; Sales-Pardo, Marta
  • Proceedings of the National Academy of Sciences, Vol. 106, Issue 52
  • DOI: 10.1073/pnas.0908366106

Community detection in graphs
journal, February 2010


Sequential detection of temporal communities by estrangement confinement
journal, November 2012

  • Kawadia, Vikas; Sreenivasan, Sameet
  • Scientific Reports, Vol. 2, Issue 1
  • DOI: 10.1038/srep00794

Benchmark graphs for testing community detection algorithms
journal, October 2008


A nonparametric view of network models and Newman–Girvan and other modularities
journal, November 2009

  • Bickel, Peter J.; Chen, Aiyou
  • Proceedings of the National Academy of Sciences, Vol. 106, Issue 50
  • DOI: 10.1073/pnas.0907096106

Anomaly detection in online social networks
journal, October 2014