skip to main content
OSTI.GOV title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Anomaly Detection in Partially Observed Traffic Networks

Journal Article · · IEEE Transactions on Signal Processing
ORCiD logo [1];  [1];  [1]
  1. Univ. of Michigan, Ann Arbor, MI (United States). EECS Dept.
+ Show Author Affiliations

This paper addresses the problem of detecting anomalous activity in traffic networks where the network is not directly observed. Given knowledge of what the node-tonode traffic in a network should be, any activity that differs significantly from this baseline would be considered anomalous. We propose a Bayesian hierarchical model for estimating the traffic rates and detecting anomalous changes in the network. The probabilistic nature of the model allows us to perform statistical goodness-of-fit tests to detect significant deviations from a baseline network. We show that due to the more defined structure of the hierarchical Bayesian model, such tests perform well even when the empirical models estimated by the EM algorithm are misspecified. We apply our model to both simulated and real datasets to demonstrate its superior performance over existing alternatives.

Research Organization:
Univ. of Michigan, Ann Arbor, MI (United States)
Sponsoring Organization:
USDOE National Nuclear Security Administration (NNSA); National Science Foundation (NSF)
Grant/Contract Number:
NA0002534; CNS-1737598
OSTI ID:
1524444
Alternate ID(s):
OSTI ID: 1798641; OSTI ID: 1798647
Journal Information:
IEEE Transactions on Signal Processing, Vol. 67, Issue 6; ISSN 1053-587X
Publisher:
IEEECopyright Statement
Country of Publication:
United States
Language:
English
Citation Metrics:
Cited by: 13 works
Citation information provided by
Web of Science

References (34)

Regularized EM Algorithms: A Unified Framework and Statistical Guarantees preprint January 2015
Multicast-based inference of network-internal loss characteristics journal November 1999
Diagnosing network-wide traffic anomalies
  • Lakhina, Anukool; Crovella, Mark; Diot, Christophe
  • Proceedings of the 2004 conference on Applications, technologies, architectures, and protocols for computer communications - SIGCOMM '04 https://doi.org/10.1145/1015467.1015492
conference January 2004
Detecting anomalies in network traffic using maximum entropy estimation conference January 2005
Graph based anomaly detection and description: a survey journal July 2014
Anomaly detection in dynamic networks: a survey journal March 2015
Anomaly detection in IP networks journal August 2003
Anomaly Detection via Online Oversampling Principal Component Analysis journal July 2013
Sparse Laplacian Component Analysis for Internet Traffic Anomalies Detection journal December 2018
Spectral anomaly detection using graph-based filtering for wireless sensor networks conference May 2014
Detection and identification of network anomalies using sketch subspaces conference January 2006
Network Volume Anomaly Detection and Identification in Large-Scale Networks Based on Online Time-Structured Traffic Tensor Tracking journal September 2016
Sketch-based change detection: methods, evaluation, and applications conference January 2003
Internet tomography journal May 2002
Spatio-Temporal Compressive Sensing and Internet Traffic Matrices (Extended Version) journal June 2012
Estimating Traffic and Anomaly Maps via Network Tomography journal June 2016
Recovery of Low-Rank Plus Compressed Sparse Matrices With Application to Unveiling Traffic Anomalies journal August 2013
Diagnosing network-wide traffic anomalies journal August 2004
Characterization of network-wide anomalies in traffic flows conference October 2004
Sensitivity of PCA for traffic anomaly detection
  • Ringberg, Haakon; Soule, Augustin; Rexford, Jennifer
  • Proceedings of the 2007 ACM SIGMETRICS international conference on Measurement and modeling of computer systems https://doi.org/10.1145/1254882.1254895
conference June 2007
Network anomography conference January 2005
The Large-Sample Distribution of the Likelihood Ratio for Testing Composite Hypotheses journal March 1938
Unicast-based inference of network link delay distributions with finite mixture models journal August 2003
Hierarchical Inference of Unicast Network Topologies Based on End-to-End Measurements journal May 2007
Network Tomography of Binary Network Performance Characteristics journal December 2006
Statistical Anomaly Detection via Composite Hypothesis Testing for Markov Models journal February 2018
Fast accurate computation of large-scale IP traffic matrices from link loads
  • Zhang, Yin; Roughan, Matthew; Duffield, Nick
  • Proceedings of the 2003 ACM SIGMETRICS international conference on Measurement and modeling of computer systems https://doi.org/10.1145/781027.781053
conference June 2003
An information-theoretic approach to traffic matrix estimation
  • Zhang, Yin; Roughan, Matthew; Lund, Carsten
  • Proceedings of the 2003 conference on Applications, technologies, architectures, and protocols for computer communications https://doi.org/10.1145/863955.863990
conference August 2003
Network Tomography: Recent Developments journal August 2004
Traffic matrix estimation
  • Medina, A.; Taft, N.; Salamatian, K.
  • Proceedings of the 2002 conference on Applications, technologies, architectures, and protocols for computer communications https://doi.org/10.1145/633025.633041
conference August 2002
Maximum likelihood network topology identification from edge-based unicast measurements journal June 2002
Network delay tomography journal August 2003
An empirical comparison of botnet detection methods journal September 2014
Bayesian hypotheses testing using posterior density ratios journal September 1996

Similar Records

Related Subjects

97 MATHEMATICS AND COMPUTING