skip to main content
OSTI.GOV title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Detecting Anomalies from End-to-End Internet Performance Measurements (PingER) Using Cluster Based Local Outlier Factor

Journal Article · · IEEE International Symposium on Parallel and Distributed Processing, Workshops and Phd Forum
 [1];  [1]; ORCiD logo [2];  [3]
  1. Guangzhou Univ., Guangzhou (People's Republic of China)
  2. Stanford Linear Accelerator Center, Palo Alto, CA (United States)
  3. Univ. of Agriculture, Faisalabad (Pakistan)
+ Show Author Affiliations

PingER (Ping End-to-End Reporting) is a worldwide end-to-end Internet performance measurement framework. It was developed by the SLAC National Accelerator Laboratory, Stanford, USA and running from the last 20 years. It has more than 700 monitoring agents and remote sites which monitor the performance of Internet links around 170 countries of the world. At present, the size of the compressed PingER data set is about 60 GB comprising of 100,000 flat files. The data is publicly available for valuable Internet performance analyses. However, the data sets suffer from missing values and anomalies due to congestion, bottleneck links, queuing overflow, network software misconfiguration, hardware failure, cable cuts, and social upheavals. Therefore, the objective of this paper is to detect such performance drops or spikes labeled as anomalies or outliers for the PingER data set. In the proposed approach, the raw text files of the data set are transformed into a PingER dimensional model. The missing values are imputed using the k-NN algorithm. The data is partitioned into similar instances using the k-means clustering algorithm. Afterward, clustering is integrated with the Local Outlier Factor (LOF) using the Cluster Based Local Outlier Factor (CBLOF) algorithm to detect the anomalies or outliers from the PingER data. Lastly, anomalies are further analyzed to identify the time frame and location of the hosts generating the major percentage of the anomalies in the PingER data set ranging from 1998 to 2016.

Research Organization:
SLAC National Accelerator Lab., Menlo Park, CA (United States)
Sponsoring Organization:
USDOE Office of Science (SC)
Grant/Contract Number:
AC02-76SF00515
OSTI ID:
1440521
Journal Information:
IEEE International Symposium on Parallel and Distributed Processing, Workshops and Phd Forum, Vol. 2017; Conference: 15. IEEE International Symposium on Parallel and Distributed Processing with Applications and 16. IEEE International Conference on Ubiquitous Computing and Communications, Guangzhou (China), 12-15 Dec 2017; ISSN 2164-7062
Publisher:
IEEECopyright Statement
Country of Publication:
United States
Language:
English
Citation Metrics:
Cited by: 4 works
Citation information provided by
Web of Science

Figures / Tables (12)

Figure 1(p. 2)figure Figure 1
Figure 2(p. 3)figure Figure 2
Figure 3(p. 4)figure Figure 3
Figure 4(p. 4)figure Figure 4
Figure 5(p. 5)figure Figure 5
Figure 6(p. 5)figure Figure 6
Figure 7(p. 5)figure Figure 7
TABLE 1(p. 6)table TABLE 1
TABLE 2(p. 6)table TABLE 2
Figure 8(p. 7)figure Figure 8
Figure 9(p. 7)figure Figure 9
TABLE 3(p. 7)table TABLE 3

Similar Records

Internet Performance Analysis of South Asian Countries Using End-to-End Internet Performance Measurements
Journal Article · Mon May 28 00:00:00 EDT 2018 · IEEE International Symposium on Parallel and Distributed Processing, Workshops and Phd Forum · OSTI ID:1440521
+1 more
Internet end-to-end performance monitoring for the High Energy Nuclear and Particle Physics community
Technical Report · Tue Feb 22 00:00:00 EST 2000 · OSTI ID:1440521
PingER: Internet performance monitoring -- How do collisions make better physics
Technical Report · Thu Feb 17 00:00:00 EST 2000 · OSTI ID:1440521

Related Subjects

97 MATHEMATICS AND COMPUTING
Internet performance measurements
clustering
local outlier factor
anomaly detection