skip to main content

DOE PAGESDOE PAGES

This content will become publicly available on May 28, 2019

Title: Detecting Anomalies from End-to-End Internet Performance Measurements (PingER) Using Cluster Based Local Outlier Factor

PingER (Ping End-to-End Reporting) is a worldwide end-to-end Internet performance measurement framework. It was developed by the SLAC National Accelerator Laboratory, Stanford, USA and running from the last 20 years. It has more than 700 monitoring agents and remote sites which monitor the performance of Internet links around 170 countries of the world. At present, the size of the compressed PingER data set is about 60 GB comprising of 100,000 flat files. The data is publicly available for valuable Internet performance analyses. However, the data sets suffer from missing values and anomalies due to congestion, bottleneck links, queuing overflow, network software misconfiguration, hardware failure, cable cuts, and social upheavals. Therefore, the objective of this paper is to detect such performance drops or spikes labeled as anomalies or outliers for the PingER data set. In the proposed approach, the raw text files of the data set are transformed into a PingER dimensional model. The missing values are imputed using the k-NN algorithm. The data is partitioned into similar instances using the k-means clustering algorithm. Afterward, clustering is integrated with the Local Outlier Factor (LOF) using the Cluster Based Local Outlier Factor (CBLOF) algorithm to detect the anomalies or outliers from themore » PingER data. Lastly, anomalies are further analyzed to identify the time frame and location of the hosts generating the major percentage of the anomalies in the PingER data set ranging from 1998 to 2016.« less
Authors:
 [1] ;  [1] ; ORCiD logo [2] ;  [3]
  1. Guangzhou Univ., Guangzhou (People's Republic of China)
  2. Stanford Linear Accelerator Center, Palo Alto, CA (United States)
  3. Univ. of Agriculture, Faisalabad (Pakistan)
Publication Date:
Grant/Contract Number:
AC02-76SF00515
Type:
Accepted Manuscript
Journal Name:
IEEE
Additional Journal Information:
Journal Name: IEEE
Research Org:
SLAC National Accelerator Lab., Menlo Park, CA (United States)
Sponsoring Org:
USDOE Office of Science (SC)
Country of Publication:
United States
Language:
English
Subject:
97 MATHEMATICS AND COMPUTING; Internet performance measurements; clustering; local outlier factor; anomaly detection
OSTI Identifier:
1440521