Search Menu
DOE PAGES title logo U.S. Department of Energy
Office of Scientific and Technical Information
Search Scholarly Publications

Title: Multivariate network traffic analysis using clustered patterns

Abstract

Traffic analysis is a core element in network operations and management for various purposes including change detection, traffic prediction, and anomaly detection. In this paper, we introduce a new approach to online traffic analysis based on a pattern-based representation for high-level summarization of the traffic measurement data. Unlike the past online analysis techniques limited to a single variable to summarize (e.g., sketch), the focus of this study is on capturing the network state from the multivariate attributes under consideration. To this end, we employ clustering with its benefit of the aggregation of multidimensional variables. The clustered result represents the state of the network with regard to the monitored variables, which can also be compared with the observed patterns from previous time windows enabling intuitive analysis. Finally, we demonstrate the proposed method with two popular use cases, one for estimating state changes and the other for identifying anomalous states, to confirm its feasibility. Our extensive experimental results with public traces and collected monitoring measurements from ESnet traffic traces show that our pattern-based approach is effective for multivariate analysis of online network traffic with visual and quantitative tools.

Authors:
ORCiD logo [1];  [2];  [3];  [1];  [4]
+ Show Author Affiliations
  1. Texas A & M Univ., Commerce, TX (United States)
  2. Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States)
  3. Energy Sciences Network, Berkeley, CA (United States)
  4. Electronics and Telecommunications Research Inst., Daejon (Korea, Republic of)
Publication Date:
Research Org.:
Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States)
Sponsoring Org.:
USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR)
OSTI Identifier:
1498687
Grant/Contract Number:  
AC02-05CH11231
Resource Type:
Accepted Manuscript
Journal Name:
Computing: Archiv fuer Informatik und Numerik
Additional Journal Information:
Journal Volume: 101; Journal Issue: 4; Journal ID: ISSN 0010-485X
Publisher:
Springer Nature
Country of Publication:
United States
Language:
English
Subject:
97 MATHEMATICS AND COMPUTING; Network traffic analysis; Clustered patterns; Change detection; Anomaly detection; Multivariate analysis

Citation Formats

Kim, Jinoh, Sim, Alex, Tierney, Brian, Suh, Sang, and Kim, Ikkyun. Multivariate network traffic analysis using clustered patterns. United States: N. p., 2018. Web. doi:10.1007/s00607-018-0619-4.
Copy to clipboard
Kim, Jinoh, Sim, Alex, Tierney, Brian, Suh, Sang, & Kim, Ikkyun. Multivariate network traffic analysis using clustered patterns. United States. https://doi.org/10.1007/s00607-018-0619-4
Copy to clipboard
Kim, Jinoh, Sim, Alex, Tierney, Brian, Suh, Sang, and Kim, Ikkyun. Sat . "Multivariate network traffic analysis using clustered patterns". United States. https://doi.org/10.1007/s00607-018-0619-4. https://www.osti.gov/servlets/purl/1498687.
Copy to clipboard
@article{osti_1498687,
title = {Multivariate network traffic analysis using clustered patterns},
author = {Kim, Jinoh and Sim, Alex and Tierney, Brian and Suh, Sang and Kim, Ikkyun},
abstractNote = {Traffic analysis is a core element in network operations and management for various purposes including change detection, traffic prediction, and anomaly detection. In this paper, we introduce a new approach to online traffic analysis based on a pattern-based representation for high-level summarization of the traffic measurement data. Unlike the past online analysis techniques limited to a single variable to summarize (e.g., sketch), the focus of this study is on capturing the network state from the multivariate attributes under consideration. To this end, we employ clustering with its benefit of the aggregation of multidimensional variables. The clustered result represents the state of the network with regard to the monitored variables, which can also be compared with the observed patterns from previous time windows enabling intuitive analysis. Finally, we demonstrate the proposed method with two popular use cases, one for estimating state changes and the other for identifying anomalous states, to confirm its feasibility. Our extensive experimental results with public traces and collected monitoring measurements from ESnet traffic traces show that our pattern-based approach is effective for multivariate analysis of online network traffic with visual and quantitative tools.},
doi = {10.1007/s00607-018-0619-4},
journal = {Computing: Archiv fuer Informatik und Numerik},
number = 4,
volume = 101,
place = {United States},
year = {Sat Apr 28 00:00:00 EDT 2018},
month = {Sat Apr 28 00:00:00 EDT 2018}
}
Copy to clipboard
Journal Article:
Free Publicly Available Full Text
Publisher's Version of Record
https://doi.org/10.1007/s00607-018-0619-4
Copyright Statement
Other availability
Search WorldCat Search WorldCat to find libraries that may hold this journal
Citation Metrics:
Cited by: 7 works
Citation information provided by
Web of Science
Save / Share:
Export Metadata
Save to My Library
You must Sign In or Create an Account in order to save documents to your library.

Works referenced in this record:

Unsupervised Labeling for Supervised Anomaly Detection in Enterprise and Cloud Networks
conference, June 2017

  • Baek, Sunhee; Kwon, Donghwoon; Kim, Jinoh
  • 2017 IEEE 4th International Conference on Cyber Security and Cloud Computing (CSCloud)
  • DOI: 10.1109/CSCloud.2017.26

Measuring IP and TCP behavior on edge nodes with Tstat
journal, January 2005

Network Anomaly Detection Using Co-clustering
conference, August 2012

  • Papalexakis, E. E.; Beutel, A.; Steenkiste, P.
  • 2012 International Conference on Advances in Social Networks Analysis and Mining (ASONAM 2012), 2012 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining
  • DOI: 10.1109/ASONAM.2012.72

Using GMM and SVM-Based Techniques for the Classification of SSH-Encrypted Traffic
conference, June 2009

  • Dusi, M.; Este, A.; Gringoli, F.
  • ICC 2009 - 2009 IEEE International Conference on Communications
  • DOI: 10.1109/ICC.2009.5199557

Observing slow crustal movement in residential user traffic
conference, January 2008

  • Cho, Kenjiro; Fukuda, Kensuke; Esaki, Hiroshi
  • Proceedings of the 2008 ACM CoNEXT Conference on - CONEXT '08
  • DOI: 10.1145/1544012.1544024

A multivariate Kolmogorov-Smirnov test of goodness of fit
journal, October 1997

CoTS: A Scalable Framework for Parallelizing Frequency Counting over Data Streams
conference, March 2009

  • Das, Sudipto; Antony, Shyam; Agrawal, Divyakant
  • 2009 IEEE 25th International Conference on Data Engineering (ICDE)
  • DOI: 10.1109/ICDE.2009.231

A survey of network anomaly detection techniques
journal, January 2016

  • Ahmed, Mohiuddin; Naser Mahmood, Abdun; Hu, Jiankun
  • Journal of Network and Computer Applications, Vol. 60
  • DOI: 10.1016/j.jnca.2015.11.016

Network monitoring using traffic dispersion graphs (tdgs)
conference, January 2007

  • Iliofotou, Marios; Pappu, Prashanth; Faloutsos, Michalis
  • Proceedings of the 7th ACM SIGCOMM conference on Internet measurement - IMC '07
  • DOI: 10.1145/1298306.1298349

Sketch-based change detection: methods, evaluation, and applications
conference, January 2003

  • Krishnamurthy, Balachander; Sen, Subhabrata; Zhang, Yin
  • Proceedings of the 2003 ACM SIGCOMM conference on Internet measurement - IMC '03
  • DOI: 10.1145/948205.948236

Maintaining Stream Statistics over Sliding Windows
journal, January 2002

One Sketch to Rule Them All: Rethinking Network Flow Monitoring with UnivMon
conference, January 2016

  • Liu, Zaoxing; Manousis, Antonis; Vorsanger, Gregory
  • Proceedings of the 2016 conference on ACM SIGCOMM 2016 Conference - SIGCOMM '16
  • DOI: 10.1145/2934872.2934906

A survey of network flow applications
journal, March 2013

  • Li, Bingdong; Springer, Jeff; Bebis, George
  • Journal of Network and Computer Applications, Vol. 36, Issue 2
  • DOI: 10.1016/j.jnca.2012.12.020

Visualizing Traffic Causality for Analyzing Network Anomalies
conference, January 2015

  • Zhang, Hao; Sun, Maoyuan; Yao, Danfeng (Daphne)
  • Proceedings of the 2015 ACM International Workshop on International Workshop on Security and Privacy Analytics - IWSPA '15
  • DOI: 10.1145/2713579.2713583

Heavy-Hitter Detection Entirely in the Data Plane
conference, January 2017

  • Sivaraman, Vibhaalakshmi; Narayana, Srinivas; Rottenstreich, Ori
  • Proceedings of the Symposium on SDN Research - SOSR '17
  • DOI: 10.1145/3050220.3063772

A Cluster-Based Intrusion Detection Framework for Monitoring the Traffic of Cloud Environments
conference, June 2016

  • Li, Bo; Liu, Peng; Lin, Li
  • 2016 IEEE 3rd International Conference on Cyber Security and Cloud Computing (CSCloud)
  • DOI: 10.1109/CSCloud.2016.43

GT: picking up the truth from the ground for internet traffic
journal, October 2009

  • Gringoli, F.; Salgarelli, Luca; Dusi, M.
  • ACM SIGCOMM Computer Communication Review, Vol. 39, Issue 5
  • DOI: 10.1145/1629607.1629610

NeTraMark: a network traffic classification benchmark
journal, January 2011

  • Lee, Suchul; Kim, Hyunchul; Barman, Dhiman
  • ACM SIGCOMM Computer Communication Review, Vol. 41, Issue 1
  • DOI: 10.1145/1925861.1925865

The Science DMZ: a network design pattern for data-intensive science
conference, January 2013

  • Dart, Eli; Rotman, Lauren; Tierney, Brian
  • Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis on - SC '13
  • DOI: 10.1145/2503210.2503245

Data-streams and histograms
conference, January 2001

  • Guha, Sudipto; Koudas, Nick; Shim, Kyuseok
  • Proceedings of the thirty-third annual ACM symposium on Theory of computing - STOC '01
  • DOI: 10.1145/380752.380841

BLINC: multilevel traffic classification in the dark
journal, October 2005

  • Karagiannis, Thomas; Papagiannaki, Konstantina; Faloutsos, Michalis
  • ACM SIGCOMM Computer Communication Review, Vol. 35, Issue 4
  • DOI: 10.1145/1090191.1080119

High Throughput Sketch Based Online Heavy Hitter Detection on FPGA
journal, April 2016

A new intrusion detection system using support vector machines and hierarchical clustering
journal, August 2006

Scalable k-means++
journal, March 2012

  • Bahmani, Bahman; Moseley, Benjamin; Vattani, Andrea
  • Proceedings of the VLDB Endowment, Vol. 5, Issue 7
  • DOI: 10.14778/2180912.2180915

Finite population corrections for the Kolmogorov–Smirnov tests
journal, June 2012

Anomaly-based network intrusion detection: Techniques, systems and challenges
journal, February 2009

  • García-Teodoro, P.; Díaz-Verdejo, J.; Maciá-Fernández, G.
  • Computers & Security, Vol. 28, Issue 1-2
  • DOI: 10.1016/j.cose.2008.08.003

Network anomaly detection using IP flows with Principal Component Analysis and Ant Colony Optimization
journal, April 2016

  • Fernandes, Gilberto; Carvalho, Luiz F.; Rodrigues, Joel J. P. C.
  • Journal of Network and Computer Applications, Vol. 64
  • DOI: 10.1016/j.jnca.2015.11.024

Monitoring abnormal network traffic based on blind source separation approach
journal, September 2011

  • Qin, Tao; Guan, Xiaohong; Li, Wei
  • Journal of Network and Computer Applications, Vol. 34, Issue 5
  • DOI: 10.1016/j.jnca.2011.06.006

Sketch-based change detection: methods, evaluation, and applications
conference, January 2003

  • Krishnamurthy, Balachander; Sen, Subhabrata; Zhang, Yin
  • Proceedings of the conference on Internet measurement conference - IMC '03
  • DOI: 10.1145/948234.948236

Randomized algorithms
journal, March 1996

  • Motwani, Rajeev; Raghavan, Prabhakar
  • ACM Computing Surveys, Vol. 28, Issue 1
  • DOI: 10.1145/234313.234327

BLINC: multilevel traffic classification in the dark
conference, January 2005

  • Karagiannis, Thomas; Papagiannaki, Konstantina; Faloutsos, Michalis
  • Proceedings of the 2005 conference on Applications, technologies, architectures, and protocols for computer communications - SIGCOMM '05
  • DOI: 10.1145/1080091.1080119

Heavy-Hitter Detection Entirely in the Data Plane
text, January 2016

    Sort by:
    [ × clear filter / sort ]

    Similar Records in DOE PAGES and OSTI.GOV collections: