Relative Hausdorff distance for network analysis
Abstract
Similarity measures are used extensively in machine learning and data science algorithms. The newly proposed graph Relative Hausdorff (RH) distance is a lightweight yet nuanced similarity measure for quantifying the closeness of two graphs. In this work we study the effectiveness of RH distance as a tool for detecting anomalies in time-evolving graph sequences. We apply RH to cyber data with given red team events, as well to synthetically generated sequences of graphs with planted attacks. In our experiments, the performance of RH distance is at times comparable, and sometimes superior, to graph edit distance in detecting anomalous phenomena. Furthermore, our results suggest that in appropriate contexts, RH distance has advantages over more computationally intensive similarity measures.
- Authors:
-
- Pacific Northwest National Lab. (PNNL), Richland, WA (United States)
- Pacific Northwest National Lab. (PNNL), Seattle, WA (United States)
- Publication Date:
- Research Org.:
- Pacific Northwest National Laboratory (PNNL), Richland, WA (United States)
- Sponsoring Org.:
- USDOE
- OSTI Identifier:
- 1576960
- Report Number(s):
- PNNL-SA-141621
Journal ID: ISSN 2364-8228
- Grant/Contract Number:
- AC05-76RL01830
- Resource Type:
- Accepted Manuscript
- Journal Name:
- Applied Network Science
- Additional Journal Information:
- Journal Volume: 4; Journal Issue: 1; Journal ID: ISSN 2364-8228
- Country of Publication:
- United States
- Language:
- English
- Subject:
- 97 MATHEMATICS AND COMPUTING; graph similarity measure; cyber anomaly detection; machine learning; temporal graphs; Relative Hausdorff distance
Citation Formats
Aksoy, Sinan G., Nowak, Kathleen E., Purvine, Emilie, and Young, Stephen J. Relative Hausdorff distance for network analysis. United States: N. p., 2019.
Web. doi:10.1007/s41109-019-0198-0.
Aksoy, Sinan G., Nowak, Kathleen E., Purvine, Emilie, & Young, Stephen J. Relative Hausdorff distance for network analysis. United States. https://doi.org/10.1007/s41109-019-0198-0
Aksoy, Sinan G., Nowak, Kathleen E., Purvine, Emilie, and Young, Stephen J. Thu .
"Relative Hausdorff distance for network analysis". United States. https://doi.org/10.1007/s41109-019-0198-0. https://www.osti.gov/servlets/purl/1576960.
@article{osti_1576960,
title = {Relative Hausdorff distance for network analysis},
author = {Aksoy, Sinan G. and Nowak, Kathleen E. and Purvine, Emilie and Young, Stephen J.},
abstractNote = {Similarity measures are used extensively in machine learning and data science algorithms. The newly proposed graph Relative Hausdorff (RH) distance is a lightweight yet nuanced similarity measure for quantifying the closeness of two graphs. In this work we study the effectiveness of RH distance as a tool for detecting anomalies in time-evolving graph sequences. We apply RH to cyber data with given red team events, as well to synthetically generated sequences of graphs with planted attacks. In our experiments, the performance of RH distance is at times comparable, and sometimes superior, to graph edit distance in detecting anomalous phenomena. Furthermore, our results suggest that in appropriate contexts, RH distance has advantages over more computationally intensive similarity measures.},
doi = {10.1007/s41109-019-0198-0},
journal = {Applied Network Science},
number = 1,
volume = 4,
place = {United States},
year = {Thu Oct 17 00:00:00 EDT 2019},
month = {Thu Oct 17 00:00:00 EDT 2019}
}
Works referenced in this record:
Directed Random Dot Product Graphs
journal, January 2008
- Young, Stephen J.; Scheinerman, Edward
- Internet Mathematics, Vol. 5, Issue 1-2
Graph based anomaly detection and description: a survey
journal, July 2014
- Akoglu, Leman; Tong, Hanghang; Koutra, Danai
- Data Mining and Knowledge Discovery, Vol. 29, Issue 3
Authoritative sources in a hyperlinked environment
journal, September 1999
- Kleinberg, Jon M.
- Journal of the ACM, Vol. 46, Issue 5
HyperHeadTail: a Streaming Algorithm for Estimating the Degree Distribution of Dynamic Multigraphs
conference, January 2017
- Stolman, Andrew; Matulef, Kevin
- Proceedings of the 2017 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining 2017 - ASONAM '17
Multi-centrality graph spectral decompositions and their application to cyber intrusion detection
conference, March 2016
- Chen, Pin-Yu; Choudhury, Sutanay; Hero, Alfred O.
- 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
Information spreading in dynamic graphs
journal, June 2014
- Clementi, Andrea; Silvestri, Riccardo; Trevisan, Luca
- Distributed Computing, Vol. 28, Issue 1
Flooding Time of Edge-Markovian Evolving Graphs
journal, January 2010
- Clementi, Andrea E. F.; Macci, Claudio; Monti, Angelo
- SIAM Journal on Discrete Mathematics, Vol. 24, Issue 4
Temporal networks
journal, October 2012
- Holme, Petter; Saramäki, Jari
- Physics Reports, Vol. 519, Issue 3
The phase transition in inhomogeneous random graphs
journal, January 2007
- Bollobás, Béla; Janson, Svante; Riordan, Oliver
- Random Structures and Algorithms, Vol. 31, Issue 1
Outlier detection in graph streams
conference, April 2011
- Aggarwal, Charu C.; Zhao, Yuchen; Yu, Philip S.
- 2011 IEEE International Conference on Data Engineering (ICDE 2011), 2011 IEEE 27th International Conference on Data Engineering
Catching the Head, Tail, and Everything in Between: A Streaming Algorithm for the Degree Distribution
conference, November 2015
- Simpson, Olivia; Seshadhri, C.; McGregor, Andrew
- 2015 IEEE International Conference on Data Mining (ICDM)
Anomaly detection in dynamic networks: a survey
journal, March 2015
- Ranshous, Stephen; Shen, Shitian; Koutra, Danai
- Wiley Interdisciplinary Reviews: Computational Statistics, Vol. 7, Issue 3
A Measure of Similarity between Graph Vertices: Applications to Synonym Extraction and Web Searching
journal, January 2004
- Blondel, Vincent D.; Gajardo, Anahí; Heymans, Maureen
- SIAM Review, Vol. 46, Issue 4
Quantification and comparison of degree distributions in complex networks
conference, September 2014
- Aliakbary, Sadegh; Habibi, Jafar; Movaghar, Ali
- 2014 7th International Symposium on Telecommunications (IST), 7'th International Symposium on Telecommunications (IST'2014)
General formalism for inhomogeneous random graphs
journal, December 2002
- Söderberg, Bo
- Physical Review E, Vol. 66, Issue 6
Non-Negative Residual Matrix Factorization with Application to Graph Anomaly Detection
conference, December 2013
- Tong, Hanghang; Lin, Ching-Yung
- Proceedings of the 2011 SIAM International Conference on Data Mining
Detecting Periodic Subsequences in Cyber Security Data
conference, September 2017
- Price-Williams, Matthew; Heard, Nick; Turcotte, Melissa
- 2017 European Intelligence and Security Informatics Conference (EISIC)
Measuring Closeness of Graphs—The Hausdorff Distance
journal, November 2015
- Banič, Iztok; Taranenko, Andrej
- Bulletin of the Malaysian Mathematical Sciences Society, Vol. 40, Issue 1
The Average Distance in a Random Graph with Given Expected Degrees
journal, January 2004
- Chung, Fan; Lu, Linyuan
- Internet Mathematics, Vol. 1, Issue 1
Botnet Detection Based on Anomaly and Community Detection
journal, June 2017
- Wang, Jing; Paschalidis, Ioannis Ch.
- IEEE Transactions on Control of Network Systems, Vol. 4, Issue 2
Comparing stars: on approximating graph edit distance
journal, August 2009
- Zeng, Zhiping; Tung, Anthony K. H.; Wang, Jianyong
- Proceedings of the VLDB Endowment, Vol. 2, Issue 1
A distance measure between attributed relational graphs for pattern recognition
journal, May 1983
- Sanfeliu, Alberto; Fu, King-Sun
- IEEE Transactions on Systems, Man, and Cybernetics, Vol. SMC-13, Issue 3
A tutorial on spectral clustering
journal, August 2007
- von Luxburg, Ulrike
- Statistics and Computing, Vol. 17, Issue 4
A survey of graph edit distance
journal, January 2009
- Gao, Xinbo; Xiao, Bing; Tao, Dacheng
- Pattern Analysis and Applications, Vol. 13, Issue 1
Real-Time Dynamic Network Anomaly Detection
journal, March 2018
- Noble, Jordan; Adams, Niall
- IEEE Intelligent Systems, Vol. 33, Issue 2
Computing the Two-Sided Kolmogorov-Smirnov Distribution
journal, January 2011
- Simard, Richard; L'Ecuyer, Pierre
- Journal of Statistical Software, Vol. 39, Issue 11
The average distances in random graphs with given expected degrees
journal, December 2002
- Chung, F.; Lu, L.
- Proceedings of the National Academy of Sciences, Vol. 99, Issue 25
A Survey on Different Graph Based Anomaly Detection Techniques
journal, November 2015
- Sensarma, Debajit; Sen Sarma, Samar
- Indian Journal of Science and Technology, Vol. 8, Issue 31
A survey on some inequalities for expectation and variance
journal, January 2005
- Agarwal, R. P.; Barnett, N. S.; Cerone, P.
- Computers & Mathematics with Applications, Vol. 49, Issue 2-3
A graph distance metric combining maximum common subgraph and minimum common supergraph
journal, May 2001
- Fernández, Mirtha-Lina; Valiente, Gabriel
- Pattern Recognition Letters, Vol. 22, Issue 6-7
Proof without prejudice: use of the Kolmogorov-Smirnov test for the analysis of histograms from flow systems and other sources.
journal, July 1977
- Young, I. T.
- Journal of Histochemistry & Cytochemistry, Vol. 25, Issue 7
Stochastic kronecker graphs
journal, July 2010
- Mahdian, Mohammad; Xu, Ying
- Random Structures & Algorithms, Vol. 38, Issue 4
Visualizing Automatically Detected Periodic Network Activity
conference, October 2018
- Gove, Robert; Deason, Lauren
- 2018 IEEE Symposium on Visualization for Cyber Security (VizSec)
Nonparametric Statistical Inference: Book Reviews
journal, March 2011
- Shalabh,
- Journal of the Royal Statistical Society: Series A (Statistics in Society), Vol. 174, Issue 2
Information spreading in dynamic graphs
conference, January 2012
- Clementi, Andrea; Silvestri, Riccardo; Trevisan, Luca
- Proceedings of the 2012 ACM symposium on Principles of distributed computing - PODC '12
A Survey on Different Graph Based Anomaly Detection Techniques
journal, January 2015
- Sensarma, Debajit
- Indian Journal of Science and Technology, Vol. 8, Issue 1
Anonymized User-Computer Authentication Associations in Time
dataset, January 2014
- Kent, Alex
- Los Alamos National Laboratory (LANL), Los Alamos, NM (United States)
Nonparametric Statistical Inference
journal, August 1986
- Randles, Ronald H.; Gibbons, Jean Dickinson
- Technometrics, Vol. 28, Issue 3
Quantification and Comparison of Degree Distributions in Complex Networks
preprint, January 2013
- Aliakbary, Sadegh; Habibi, Jafar; Movaghar, Ali
- arXiv
Detecting periodic subsequences in cyber security data
preprint, January 2017
- Price-Williams, Matthew; Heard, Nick; Turcotte, Melissa
- arXiv
Visualizing Automatically Detected Periodic Network Activity
conference, October 2018
- Gove, Robert; Deason, Lauren
- 2018 IEEE Symposium on Visualization for Cyber Security (VizSec)
Flooding Time of Edge-Markovian Evolving Graphs
journal, January 2010
- Clementi, Andrea E. F.; Macci, Claudio; Monti, Angelo
- SIAM Journal on Discrete Mathematics, Vol. 24, Issue 4
Graph-based Anomaly Detection and Description: A Survey
preprint, January 2014
- Akoglu, Leman; Tong, Hanghang; Koutra, Danai
- arXiv
Feature Extraction from Degree Distribution for Comparison and Analysis of Complex Networks
preprint, January 2014
- Aliakbary, Sadegh; Habibi, Jafar; Movaghar, Ali
- arXiv
Catching the head, tail, and everything in between: a streaming algorithm for the degree distribution
preprint, January 2015
- Simpson, Olivia; Seshadhri, C.; McGregor, Andrew
- arXiv
A measure of similarity between graph vertices
preprint, January 2004
- Blondel, Vincent; Gajardo, Anahi; Heymans, Maureen
- arXiv