A multi-level anomaly detection algorithm for time-varying graph data with interactive visualization

Bridges, Robert A.; Collins, John P.; Ferragut, Erik M.; Laska, Jason A.; Sullivan, Blair D.

doi:10.1007/s13278-016-0409-y

Title: A multi-level anomaly detection algorithm for time-varying graph data with interactive visualization

Full Record
Other Related Research

Abstract

This work presents a novel modeling and analysis framework for graph sequences which addresses the challenge of detecting and contextualizing anomalies in labelled, streaming graph data. We introduce a generalization of the BTER model of Seshadhri et al. by adding flexibility to community structure, and use this model to perform multi-scale graph anomaly detection. Specifically, probability models describing coarse subgraphs are built by aggregating node probabilities, and these related hierarchical models simultaneously detect deviations from expectation. This technique provides insight into a graph's structure and internal context that may shed light on a detected event. Additionally, this multi-scale analysis facilitates intuitive visualizations by allowing users to narrow focus from an anomalous graph to particular subgraphs or nodes causing the anomaly. For evaluation, two hierarchical anomaly detectors are tested against a baseline Gaussian method on a series of sampled graphs. We demonstrate that our graph statistics-based approach outperforms both a distribution-based detector and the baseline in a labeled setting with community structure, and it accurately detects anomalies in synthetic and real-world datasets at the node, subgraph, and graph levels. Furthermore, to illustrate the accessibility of information made possible via this technique, the anomaly detector and an associated interactive visualization tool aremore »« less

Authors:

Bridges, Robert A. ^[1]; Collins, John P. ^[1]; Ferragut, Erik M. ^[1]; Laska, Jason A. ^[1]; Sullivan, Blair D. ^[2]

Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States)
North Carolina State Univ., Raleigh, NC (United States)

Publication Date:: Fri Jan 01 00:00:00 EST 2016

Research Org.:: Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States)

Sponsoring Org.:: USDOE Laboratory Directed Research and Development (LDRD) Program; Work for Others (WFO)

OSTI Identifier:: 1330521

Grant/Contract Number:: AC05-00OR22725

Resource Type:: Accepted Manuscript

Journal Name:: Social Network Analysis and Mining

Additional Journal Information:: Journal Volume: 6; Journal Issue: 1; Journal ID: ISSN 1869-5450

Publisher:: Springer

Country of Publication:: United States

Language:: English

Subject:: 97 MATHEMATICS AND COMPUTING; anomaly detection; graph sequence; visualization

Citation Formats


                    Bridges, Robert A., Collins, John P., Ferragut, Erik M., Laska, Jason A., and Sullivan, Blair D. A multi-level anomaly detection algorithm for time-varying graph data with interactive visualization.  United States: N. p., 2016. 
Web.  doi:10.1007/s13278-016-0409-y.

Copy to clipboard


                    Bridges, Robert A., Collins, John P., Ferragut, Erik M., Laska, Jason A., & Sullivan, Blair D. A multi-level anomaly detection algorithm for time-varying graph data with interactive visualization.  United States.  https://doi.org/10.1007/s13278-016-0409-y

Copy to clipboard


                    Bridges, Robert A., Collins, John P., Ferragut, Erik M., Laska, Jason A., and Sullivan, Blair D. Fri .  
"A multi-level anomaly detection algorithm for time-varying graph data with interactive visualization".  United States.  https://doi.org/10.1007/s13278-016-0409-y.  https://www.osti.gov/servlets/purl/1330521.

Copy to clipboard


                    
@article{osti_1330521,

  title        = {A multi-level anomaly detection algorithm for time-varying graph data with interactive visualization},

  author       = {Bridges, Robert A. and Collins, John P. and Ferragut, Erik M. and Laska, Jason A. and Sullivan, Blair D.},

  abstractNote = {This work presents a novel modeling and analysis framework for graph sequences which addresses the challenge of detecting and contextualizing anomalies in labelled, streaming graph data. We introduce a generalization of the BTER model of Seshadhri et al. by adding flexibility to community structure, and use this model to perform multi-scale graph anomaly detection. Specifically, probability models describing coarse subgraphs are built by aggregating node probabilities, and these related hierarchical models simultaneously detect deviations from expectation. This technique provides insight into a graph's structure and internal context that may shed light on a detected event. Additionally, this multi-scale analysis facilitates intuitive visualizations by allowing users to narrow focus from an anomalous graph to particular subgraphs or nodes causing the anomaly. For evaluation, two hierarchical anomaly detectors are tested against a baseline Gaussian method on a series of sampled graphs. We demonstrate that our graph statistics-based approach outperforms both a distribution-based detector and the baseline in a labeled setting with community structure, and it accurately detects anomalies in synthetic and real-world datasets at the node, subgraph, and graph levels. Furthermore, to illustrate the accessibility of information made possible via this technique, the anomaly detector and an associated interactive visualization tool are tested on NCAA football data, where teams and conferences that moved within the league are identified with perfect recall, and precision greater than 0.786.},

  doi          = {10.1007/s13278-016-0409-y},

  journal      = {Social Network Analysis and Mining},

  number       = 1,

  volume       = 6,

  place        = {United States},

  year         = {Fri Jan 01 00:00:00 EST 2016},

  month        = {Fri Jan 01 00:00:00 EST 2016}

}

Copy to clipboard

Journal Article:

Free Publicly Available Full Text

Accepted Manuscript (DOE)

Publisher's Version of Record

https://doi.org/10.1007/s13278-016-0409-y

Other availability

Search WorldCat to find libraries that may hold this journal

Save / Share:

Export Metadata

Save to My Library

Similar Records in DOE PAGES and OSTI.GOV collections:

Multi-Level Anomaly Detection on Time-Varying Graph Data

Conference Bridges, Robert A ; Collins, John P ; Ferragut, Erik M ; ...

This work presents a novel modeling and analysis framework for graph sequences which addresses the challenge of detecting and contextualizing anomalies in labelled, streaming graph data. We introduce a generalization of the BTER model of Seshadhri et al. by adding flexibility to community structure, and use this model to perform multi-scale graph anomaly detection. Specifically, probability models describing coarse subgraphs are built by aggregating probabilities at finer levels, and these closely related hierarchical models simultaneously detect deviations from expectation. This technique provides insight into a graph's structure and internal context that may shed light on a detected event. Additionally, thismore »« less
Full Text Available
EGBTER: Capturing Degree Distribution, Clustering Coefficients, and Community Structure in a Single Random Graph Model

Conference El-Daghar, Omar ; Lundberg, Erik ; Bridges, Robert

Random graph models are important constructs for data analytic applications as well as pure mathematical developments, as they provide capabilities for network synthesis and principled analysis. Several models have been developed with the aim of faithfully preserving important graph metrics and substructures. With the goal of capturing degree distribution, clustering coefficient, and communities in a single random graph model, we propose a new model to address shortcomings in a progression of network modeling capabilities. The Block Two-Level Erdos-Rényi (BTER) model of Seshadhri et aI., designed to allow prescription of expected degree and clustering coefficient distributions, neglects community modeling, while themore »« less
https://doi.org/10.1109/ASONAM.2018.8508598

Full Text Available
Understanding the Hierarchy of Dense Subgraphs in Stationary and Temporally Varying Setting

Technical Report Sariyuce, Ahmet Erdem ; Pinar, Ali

Graphs are widely used to model relationships in a wide variety of domains such as sociology, bioinformatics, infrastructure, the WWW, to name a few. One of the key observations is that while real-world graphs are often globally sparse, they are locally dense. In other words, the average degree is often quite small (say at most 10 in a million vertex graph), but vertex neighborhoods are often dense. Finding dense subgraphs is a critical aspect of graph mining It has been used for finding communities and spam link farms in web graphs, graph visualization, real-time story identification, DNA motif detection inmore »« less
https://doi.org/10.2172/1527314

Full Text Available
Cyber Graph Queries for Geographically Distributed Data Centers

Technical Report Berry, Jonathan W. ; Collins, Michael ; Kearns, Aaron ; ...

We present new algorithms for a distributed model for graph computations motivated by limited information sharing we first discussed in [20]. Two or more independent entities have collected large social graphs. They wish to compute the result of running graph algorithms on the entire set of relationships. Because the information is sensitive or economically valuable, they do not wish to simply combine the information in a single location. We consider two models for computing the solution to graph algorithms in this setting: 1) limited-sharing: the two entities can share only a polylogarithmic size subgraph; 2) low-trust: the entities must notmore »« less
https://doi.org/10.2172/1183059

Full Text Available
A generative graph model for electrical infrastructure networks

Journal Article Aksoy, Sinan G. ; Purvine, Emilie AH ; Cotilla Sanchez, Eduardo ; ... - Journal of Complex Networks

We propose a generative graph model for electrical infrastructure networks that accounts for heterogeneity in both node and edge type. To inform the design of this model, we analyze the properties of power grid graphs derived from the U.S. Eastern Interconnection, Texas Interconnection, and Poland transmission system power grids. Across these datasets, we find subgraphs induced by nodes of the same voltage level exhibit shared structural properties atypical to small-world networks, including low local clustering, large diameter and large average distance. On the other hand, we find subgraphs induced by transformer edges linking nodes of different voltage types contain amore »« less
https://doi.org/10.1093/comnet/cny016

Full Text Available

Similar Records