skip to main content
DOE PAGES title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Streaming Solutions for Fine-Grained Network Traffic Measurements and Analysis

Abstract

Online network traffic measurements and analysis is critical for detecting and preventing any real-time anomalies in the network. We propose, implement, and evaluate an online, adaptive measurement platform, which utilizes real-time traffic analysis results to refine subsequent traffic measurements. Central to our solution is the concept of Multi-Resolution Tiling (MRT), a heuristic approach that performs sequential analysis of traffic data to zoom into traffic subregions of interest. However, MRT is sensitive to transient traffic spikes. In this paper, we propose three novel traffic streaming algorithms that overcome the limitations of MRT and can cater to varying degrees of computational and storage budgets, detection latency, and accuracy of query response. We evaluate our streaming algorithms on a highly parallel and programmable hardware as well as a traditional software-based platforms. The algorithms demonstrate significant accuracy improvement over MRT in detecting anomalies consisting of synthetic hard-to-track elephant flows and global icebergs. Our proposed algorithms maintain the worst-case complexities of the MRT while incurring only a moderate increase in average resource utilization.

Authors:
 [1];  [1];  [1];  [1];  [2]
  1. Univ. of California, Davis, CA (United States)
  2. HP-Labs, Palo Alto, CA (United States)
Publication Date:
Research Org.:
Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States)
Sponsoring Org.:
USDOE Office of Science (SC)
OSTI Identifier:
1523891
Grant/Contract Number:  
AC02-05CH11231
Resource Type:
Accepted Manuscript
Journal Name:
IEEE/ACM Transactions on Networking
Additional Journal Information:
Journal Volume: 22; Journal Issue: 2; Journal ID: ISSN 1063-6692
Publisher:
IEEE - ACM
Country of Publication:
United States
Language:
English
Subject:
97 MATHEMATICS AND COMPUTING

Citation Formats

Khan, Faisal, Hosein, Nicholas, Ghiasi, Soheil, Chuah, Chen-Nee, and Sharma, Puneet. Streaming Solutions for Fine-Grained Network Traffic Measurements and Analysis. United States: N. p., 2013. Web. doi:10.1109/TNET.2013.2263228.
Khan, Faisal, Hosein, Nicholas, Ghiasi, Soheil, Chuah, Chen-Nee, & Sharma, Puneet. Streaming Solutions for Fine-Grained Network Traffic Measurements and Analysis. United States. doi:10.1109/TNET.2013.2263228.
Khan, Faisal, Hosein, Nicholas, Ghiasi, Soheil, Chuah, Chen-Nee, and Sharma, Puneet. Mon . "Streaming Solutions for Fine-Grained Network Traffic Measurements and Analysis". United States. doi:10.1109/TNET.2013.2263228. https://www.osti.gov/servlets/purl/1523891.
@article{osti_1523891,
title = {Streaming Solutions for Fine-Grained Network Traffic Measurements and Analysis},
author = {Khan, Faisal and Hosein, Nicholas and Ghiasi, Soheil and Chuah, Chen-Nee and Sharma, Puneet},
abstractNote = {Online network traffic measurements and analysis is critical for detecting and preventing any real-time anomalies in the network. We propose, implement, and evaluate an online, adaptive measurement platform, which utilizes real-time traffic analysis results to refine subsequent traffic measurements. Central to our solution is the concept of Multi-Resolution Tiling (MRT), a heuristic approach that performs sequential analysis of traffic data to zoom into traffic subregions of interest. However, MRT is sensitive to transient traffic spikes. In this paper, we propose three novel traffic streaming algorithms that overcome the limitations of MRT and can cater to varying degrees of computational and storage budgets, detection latency, and accuracy of query response. We evaluate our streaming algorithms on a highly parallel and programmable hardware as well as a traditional software-based platforms. The algorithms demonstrate significant accuracy improvement over MRT in detecting anomalies consisting of synthetic hard-to-track elephant flows and global icebergs. Our proposed algorithms maintain the worst-case complexities of the MRT while incurring only a moderate increase in average resource utilization.},
doi = {10.1109/TNET.2013.2263228},
journal = {IEEE/ACM Transactions on Networking},
number = 2,
volume = 22,
place = {United States},
year = {2013},
month = {7}
}

Journal Article:
Free Publicly Available Full Text
Publisher's Version of Record

Save / Share: