DOE PAGES title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Relative Hausdorff distance for network analysis

Abstract

Similarity measures are used extensively in machine learning and data science algorithms. The newly proposed graph Relative Hausdorff (RH) distance is a lightweight yet nuanced similarity measure for quantifying the closeness of two graphs. In this work we study the effectiveness of RH distance as a tool for detecting anomalies in time-evolving graph sequences. We apply RH to cyber data with given red team events, as well to synthetically generated sequences of graphs with planted attacks. In our experiments, the performance of RH distance is at times comparable, and sometimes superior, to graph edit distance in detecting anomalous phenomena. Furthermore, our results suggest that in appropriate contexts, RH distance has advantages over more computationally intensive similarity measures.

Authors:
ORCiD logo [1];  [1]; ORCiD logo [2]; ORCiD logo [1]
  1. Pacific Northwest National Lab. (PNNL), Richland, WA (United States)
  2. Pacific Northwest National Lab. (PNNL), Seattle, WA (United States)
Publication Date:
Research Org.:
Pacific Northwest National Laboratory (PNNL), Richland, WA (United States)
Sponsoring Org.:
USDOE
OSTI Identifier:
1576960
Report Number(s):
PNNL-SA-141621
Journal ID: ISSN 2364-8228
Grant/Contract Number:  
AC05-76RL01830
Resource Type:
Accepted Manuscript
Journal Name:
Applied Network Science
Additional Journal Information:
Journal Volume: 4; Journal Issue: 1; Journal ID: ISSN 2364-8228
Country of Publication:
United States
Language:
English
Subject:
97 MATHEMATICS AND COMPUTING; graph similarity measure; cyber anomaly detection; machine learning; temporal graphs; Relative Hausdorff distance

Citation Formats

Aksoy, Sinan G., Nowak, Kathleen E., Purvine, Emilie, and Young, Stephen J. Relative Hausdorff distance for network analysis. United States: N. p., 2019. Web. doi:10.1007/s41109-019-0198-0.
Aksoy, Sinan G., Nowak, Kathleen E., Purvine, Emilie, & Young, Stephen J. Relative Hausdorff distance for network analysis. United States. https://doi.org/10.1007/s41109-019-0198-0
Aksoy, Sinan G., Nowak, Kathleen E., Purvine, Emilie, and Young, Stephen J. Thu . "Relative Hausdorff distance for network analysis". United States. https://doi.org/10.1007/s41109-019-0198-0. https://www.osti.gov/servlets/purl/1576960.
@article{osti_1576960,
title = {Relative Hausdorff distance for network analysis},
author = {Aksoy, Sinan G. and Nowak, Kathleen E. and Purvine, Emilie and Young, Stephen J.},
abstractNote = {Similarity measures are used extensively in machine learning and data science algorithms. The newly proposed graph Relative Hausdorff (RH) distance is a lightweight yet nuanced similarity measure for quantifying the closeness of two graphs. In this work we study the effectiveness of RH distance as a tool for detecting anomalies in time-evolving graph sequences. We apply RH to cyber data with given red team events, as well to synthetically generated sequences of graphs with planted attacks. In our experiments, the performance of RH distance is at times comparable, and sometimes superior, to graph edit distance in detecting anomalous phenomena. Furthermore, our results suggest that in appropriate contexts, RH distance has advantages over more computationally intensive similarity measures.},
doi = {10.1007/s41109-019-0198-0},
journal = {Applied Network Science},
number = 1,
volume = 4,
place = {United States},
year = {Thu Oct 17 00:00:00 EDT 2019},
month = {Thu Oct 17 00:00:00 EDT 2019}
}

Works referenced in this record:

Directed Random Dot Product Graphs
journal, January 2008


Graph based anomaly detection and description: a survey
journal, July 2014

  • Akoglu, Leman; Tong, Hanghang; Koutra, Danai
  • Data Mining and Knowledge Discovery, Vol. 29, Issue 3
  • DOI: 10.1007/s10618-014-0365-y

Authoritative sources in a hyperlinked environment
journal, September 1999


HyperHeadTail: a Streaming Algorithm for Estimating the Degree Distribution of Dynamic Multigraphs
conference, January 2017

  • Stolman, Andrew; Matulef, Kevin
  • Proceedings of the 2017 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining 2017 - ASONAM '17
  • DOI: 10.1145/3110025.3119395

Multi-centrality graph spectral decompositions and their application to cyber intrusion detection
conference, March 2016

  • Chen, Pin-Yu; Choudhury, Sutanay; Hero, Alfred O.
  • 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
  • DOI: 10.1109/icassp.2016.7472539

Information spreading in dynamic graphs
journal, June 2014

  • Clementi, Andrea; Silvestri, Riccardo; Trevisan, Luca
  • Distributed Computing, Vol. 28, Issue 1
  • DOI: 10.1007/s00446-014-0219-2

Flooding Time of Edge-Markovian Evolving Graphs
journal, January 2010

  • Clementi, Andrea E. F.; Macci, Claudio; Monti, Angelo
  • SIAM Journal on Discrete Mathematics, Vol. 24, Issue 4
  • DOI: 10.1137/090756053

Temporal networks
journal, October 2012


The phase transition in inhomogeneous random graphs
journal, January 2007

  • Bollobás, Béla; Janson, Svante; Riordan, Oliver
  • Random Structures and Algorithms, Vol. 31, Issue 1
  • DOI: 10.1002/rsa.20168

Outlier detection in graph streams
conference, April 2011

  • Aggarwal, Charu C.; Zhao, Yuchen; Yu, Philip S.
  • 2011 IEEE International Conference on Data Engineering (ICDE 2011), 2011 IEEE 27th International Conference on Data Engineering
  • DOI: 10.1109/icde.2011.5767885

Catching the Head, Tail, and Everything in Between: A Streaming Algorithm for the Degree Distribution
conference, November 2015

  • Simpson, Olivia; Seshadhri, C.; McGregor, Andrew
  • 2015 IEEE International Conference on Data Mining (ICDM)
  • DOI: 10.1109/ICDM.2015.47

Anomaly detection in dynamic networks: a survey
journal, March 2015

  • Ranshous, Stephen; Shen, Shitian; Koutra, Danai
  • Wiley Interdisciplinary Reviews: Computational Statistics, Vol. 7, Issue 3
  • DOI: 10.1002/wics.1347

A Measure of Similarity between Graph Vertices: Applications to Synonym Extraction and Web Searching
journal, January 2004


Quantification and comparison of degree distributions in complex networks
conference, September 2014

  • Aliakbary, Sadegh; Habibi, Jafar; Movaghar, Ali
  • 2014 7th International Symposium on Telecommunications (IST), 7'th International Symposium on Telecommunications (IST'2014)
  • DOI: 10.1109/istel.2014.7000748

General formalism for inhomogeneous random graphs
journal, December 2002


Non-Negative Residual Matrix Factorization with Application to Graph Anomaly Detection
conference, December 2013

  • Tong, Hanghang; Lin, Ching-Yung
  • Proceedings of the 2011 SIAM International Conference on Data Mining
  • DOI: 10.1137/1.9781611972818.13

Detecting Periodic Subsequences in Cyber Security Data
conference, September 2017

  • Price-Williams, Matthew; Heard, Nick; Turcotte, Melissa
  • 2017 European Intelligence and Security Informatics Conference (EISIC)
  • DOI: 10.1109/eisic.2017.40

Measuring Closeness of Graphs—The Hausdorff Distance
journal, November 2015

  • Banič, Iztok; Taranenko, Andrej
  • Bulletin of the Malaysian Mathematical Sciences Society, Vol. 40, Issue 1
  • DOI: 10.1007/s40840-015-0259-1

The Average Distance in a Random Graph with Given Expected Degrees
journal, January 2004


Botnet Detection Based on Anomaly and Community Detection
journal, June 2017

  • Wang, Jing; Paschalidis, Ioannis Ch.
  • IEEE Transactions on Control of Network Systems, Vol. 4, Issue 2
  • DOI: 10.1109/TCNS.2016.2532804

Comparing stars: on approximating graph edit distance
journal, August 2009

  • Zeng, Zhiping; Tung, Anthony K. H.; Wang, Jianyong
  • Proceedings of the VLDB Endowment, Vol. 2, Issue 1
  • DOI: 10.14778/1687627.1687631

A distance measure between attributed relational graphs for pattern recognition
journal, May 1983

  • Sanfeliu, Alberto; Fu, King-Sun
  • IEEE Transactions on Systems, Man, and Cybernetics, Vol. SMC-13, Issue 3
  • DOI: 10.1109/TSMC.1983.6313167

A tutorial on spectral clustering
journal, August 2007


A survey of graph edit distance
journal, January 2009


Real-Time Dynamic Network Anomaly Detection
journal, March 2018


Computing the Two-Sided Kolmogorov-Smirnov Distribution
journal, January 2011

  • Simard, Richard; L'Ecuyer, Pierre
  • Journal of Statistical Software, Vol. 39, Issue 11
  • DOI: 10.18637/jss.v039.i11

The average distances in random graphs with given expected degrees
journal, December 2002

  • Chung, F.; Lu, L.
  • Proceedings of the National Academy of Sciences, Vol. 99, Issue 25
  • DOI: 10.1073/pnas.252631999

A Survey on Different Graph Based Anomaly Detection Techniques
journal, November 2015


A survey on some inequalities for expectation and variance
journal, January 2005

  • Agarwal, R. P.; Barnett, N. S.; Cerone, P.
  • Computers & Mathematics with Applications, Vol. 49, Issue 2-3
  • DOI: 10.1016/j.camwa.2002.11.026

A graph distance metric combining maximum common subgraph and minimum common supergraph
journal, May 2001


Proof without prejudice: use of the Kolmogorov-Smirnov test for the analysis of histograms from flow systems and other sources.
journal, July 1977

  • Young, I. T.
  • Journal of Histochemistry & Cytochemistry, Vol. 25, Issue 7
  • DOI: 10.1177/25.7.894009

Stochastic kronecker graphs
journal, July 2010

  • Mahdian, Mohammad; Xu, Ying
  • Random Structures & Algorithms, Vol. 38, Issue 4
  • DOI: 10.1002/rsa.20335

Visualizing Automatically Detected Periodic Network Activity
conference, October 2018


Nonparametric Statistical Inference: Book Reviews
journal, March 2011


Information spreading in dynamic graphs
conference, January 2012

  • Clementi, Andrea; Silvestri, Riccardo; Trevisan, Luca
  • Proceedings of the 2012 ACM symposium on Principles of distributed computing - PODC '12
  • DOI: 10.1145/2332432.2332439

A Survey on Different Graph Based Anomaly Detection Techniques
journal, January 2015


Anonymized User-Computer Authentication Associations in Time
dataset, January 2014

  • Kent, Alex
  • Los Alamos National Laboratory (LANL), Los Alamos, NM (United States)
  • DOI: 10.11578/1160076

Nonparametric Statistical Inference
journal, August 1986

  • Randles, Ronald H.; Gibbons, Jean Dickinson
  • Technometrics, Vol. 28, Issue 3
  • DOI: 10.2307/1269084

Temporal Networks
text, January 2011


Quantification and Comparison of Degree Distributions in Complex Networks
preprint, January 2013


Detecting periodic subsequences in cyber security data
preprint, January 2017


Visualizing Automatically Detected Periodic Network Activity
conference, October 2018


Flooding Time of Edge-Markovian Evolving Graphs
journal, January 2010

  • Clementi, Andrea E. F.; Macci, Claudio; Monti, Angelo
  • SIAM Journal on Discrete Mathematics, Vol. 24, Issue 4
  • DOI: 10.1137/090756053

Graph-based Anomaly Detection and Description: A Survey
preprint, January 2014


Feature Extraction from Degree Distribution for Comparison and Analysis of Complex Networks
preprint, January 2014


Catching the head, tail, and everything in between: a streaming algorithm for the degree distribution
preprint, January 2015


A measure of similarity between graph vertices
preprint, January 2004