skip to main content
OSTI.GOV title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: A lightweight network anomaly detection technique

Abstract

While the network anomaly detection is essential in network operations and management, it becomes further challenging to perform the first line of detection against the exponentially increasing volume of network traffic. In this paper, we develop a technique for the first line of online anomaly detection with two important considerations: (i) availability of traffic attributes during the monitoring time, and (ii) computational scalability for streaming data. The presented learning technique is lightweight and highly scalable with the beauty of approximation based on the grid partitioning of the given dimensional space. With the public traffic traces of KDD Cup 1999 and NSL-KDD, we show that our technique yields 98.5% and 83% of detection accuracy, respectively, only with a couple of readily available traffic attributes that can be obtained without the help of post-processing. Finally, the results are at least comparable with the classical learning methods including decision tree and random forest, with approximately two orders of magnitude faster learning performance.

Authors:
 [1];  [2];  [2];  [1];  [3]
  1. Texas A & M Univ., Commerce, TX (United States)
  2. Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States)
  3. Electronics and Telecommunications Research Inst. (ETRI), Daejeon (Korea, Republic of)
Publication Date:
Research Org.:
Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States); Electronics and Telecommunications Research Inst. (ETRI), Daejeon (Korea, Republic of)
Sponsoring Org.:
USDOE Office of Science (SC), Workforce Development for Teachers and Scientists (WDTS) (SC-27); USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR) (SC-21); Ministry of Science, ICT and Future Planning (MSIP) of Korea
OSTI Identifier:
1379772
Grant/Contract Number:
AC02-05CH11231; B0101-15-1293
Resource Type:
Journal Article: Accepted Manuscript
Journal Name:
2017 International Conference on Computing, Networking and Communications, ICNC 2017
Additional Journal Information:
Journal Name: 2017 International Conference on Computing, Networking and Communications, ICNC 2017
Country of Publication:
United States
Language:
English
Subject:
97 MATHEMATICS AND COMPUTING; complexity theory; testing; conferences; learning systems; computer crime; partitioning algorithms; decision trees; telecommunication traffic; computer network security

Citation Formats

Kim, Jinoh, Yoo, Wucherl, Sim, Alex, Suh, Sang C., and Kim, Ikkyun. A lightweight network anomaly detection technique. United States: N. p., 2017. Web. doi:10.1109/ICCNC.2017.7876251.
Kim, Jinoh, Yoo, Wucherl, Sim, Alex, Suh, Sang C., & Kim, Ikkyun. A lightweight network anomaly detection technique. United States. doi:10.1109/ICCNC.2017.7876251.
Kim, Jinoh, Yoo, Wucherl, Sim, Alex, Suh, Sang C., and Kim, Ikkyun. Mon . "A lightweight network anomaly detection technique". United States. doi:10.1109/ICCNC.2017.7876251. https://www.osti.gov/servlets/purl/1379772.
@article{osti_1379772,
title = {A lightweight network anomaly detection technique},
author = {Kim, Jinoh and Yoo, Wucherl and Sim, Alex and Suh, Sang C. and Kim, Ikkyun},
abstractNote = {While the network anomaly detection is essential in network operations and management, it becomes further challenging to perform the first line of detection against the exponentially increasing volume of network traffic. In this paper, we develop a technique for the first line of online anomaly detection with two important considerations: (i) availability of traffic attributes during the monitoring time, and (ii) computational scalability for streaming data. The presented learning technique is lightweight and highly scalable with the beauty of approximation based on the grid partitioning of the given dimensional space. With the public traffic traces of KDD Cup 1999 and NSL-KDD, we show that our technique yields 98.5% and 83% of detection accuracy, respectively, only with a couple of readily available traffic attributes that can be obtained without the help of post-processing. Finally, the results are at least comparable with the classical learning methods including decision tree and random forest, with approximately two orders of magnitude faster learning performance.},
doi = {10.1109/ICCNC.2017.7876251},
journal = {2017 International Conference on Computing, Networking and Communications, ICNC 2017},
number = ,
volume = ,
place = {United States},
year = {Mon Mar 13 00:00:00 EDT 2017},
month = {Mon Mar 13 00:00:00 EDT 2017}
}

Journal Article:
Free Publicly Available Full Text
Publisher's Version of Record

Save / Share:
  • Purpose: Automated detection of solitary pulmonary nodules using positron emission tomography (PET) and computed tomography (CT) images shows good sensitivity; however, it is difficult to detect nodules in contact with normal organs, and additional efforts are needed so that the number of false positives (FPs) can be further reduced. In this paper, the authors propose an improved FP-reduction method for the detection of pulmonary nodules in PET/CT images by means of convolutional neural networks (CNNs). Methods: The overall scheme detects pulmonary nodules using both CT and PET images. In the CT images, a massive region is first detected using anmore » active contour filter, which is a type of contrast enhancement filter that has a deformable kernel shape. Subsequently, high-uptake regions detected by the PET images are merged with the regions detected by the CT images. FP candidates are eliminated using an ensemble method; it consists of two feature extractions, one by shape/metabolic feature analysis and the other by a CNN, followed by a two-step classifier, one step being rule based and the other being based on support vector machines. Results: The authors evaluated the detection performance using 104 PET/CT images collected by a cancer-screening program. The sensitivity in detecting candidates at an initial stage was 97.2%, with 72.8 FPs/case. After performing the proposed FP-reduction method, the sensitivity of detection was 90.1%, with 4.9 FPs/case; the proposed method eliminated approximately half the FPs existing in the previous study. Conclusions: An improved FP-reduction scheme using CNN technique has been developed for the detection of pulmonary nodules in PET/CT images. The authors’ ensemble FP-reduction method eliminated 93% of the FPs; their proposed method using CNN technique eliminates approximately half the FPs existing in the previous study. These results indicate that their method may be useful in the computer-aided detection of pulmonary nodules using PET/CT images.« less
  • The rapid detection of attackers within firewalls of enterprise computer networks is of paramount importance. Anomaly detectors address this problem by quantifying deviations from baseline statistical models of normal network behavior and signaling an intrusion when the observed data deviates significantly from the baseline model. But, many anomaly detectors do not take into account plausible attacker behavior. As a result, anomaly detectors are prone to a large number of false positives due to unusual but benign activity. Our paper first introduces a stochastic model of attacker behavior which is motivated by real world attacker traversal. Then, we develop a likelihoodmore » ratio detector that compares the probability of observed network behavior under normal conditions against the case when an attacker has possibly compromised a subset of hosts within the network. Since the likelihood ratio detector requires integrating over the time each host becomes compromised, we illustrate how to use Monte Carlo methods to compute the requisite integral. We then present Receiver Operating Characteristic (ROC) curves for various network parameterizations that show for any rate of true positives, the rate of false positives for the likelihood ratio detector is no higher than that of a simple anomaly detector and is often lower. Finally, we demonstrate the superiority of the proposed likelihood ratio detector when the network topologies and parameterizations are extracted from real-world networks.« less
  • The Network Anomaly Detection and Intrusion Reporter (NADIR) is an expert system which is intended to provide real-time security auditing for intrusion and misuse detection at Los Alamos National Laboratory's Integrated Computing Network (ICN). It is based on three basic assumptions: that statistical analysis of computer system and user activities may be used to characterize normal system and user behavior, and that given the resulting statistical profiles, behavior which deviates beyond certain bounds can be detected, that expert system techniques can be applied to security auditing and intrusion detection, and that successful intrusion detection may take place while monitoring amore » limited set of network activities such as user authentication and access control, file movement and storage, and job scheduling. NADIR has been developed to employ these basic concepts while monitoring the audited activities of more than 8000 ICN users.« less
  • The ratios of the hyperfine interaction constants a'' and the nuclear g factors of the stable isotopes of antimony have been measured. From these measurements the hy perfine structure anomaly, defined as nspection. (W.L = (a/ sub 121//a/sub 123/)(g/sub 123/ g/sub 121/)--1, was found to be (--0.352 plus or minus 0.005)%. nspection. (W.L has its origin in the difference in the spatial distribution of the nuclear magnetic dipole for the two isotopes, which is related to the structure of the two nuclei. The experimental result is compared with theoretical values of nspection. (W.L based on a variety of nuclear models.more » The determination of a/sub 121//a/sub 123/ makes use of the electron nuclear double resonance technique (ENPOR) which is discussed in some detail. The sample used in the experiment was silicon dopod with antimony and the microwave resonances were observed at ~9000 Mc/sec at a temperature of 1.2 deg K. The ratio of the nuclear g factors was determined by conventional nuclear magnetic resonance techniques. (auth)« less