A lightweight network anomaly detection technique

Kim, Jinoh; Yoo, Wucherl; Sim, Alex; Suh, Sang C.; Kim, Ikkyun

doi:10.1109/ICCNC.2017.7876251

Title: A lightweight network anomaly detection technique

Full Record
Other Related Research

Abstract

While the network anomaly detection is essential in network operations and management, it becomes further challenging to perform the first line of detection against the exponentially increasing volume of network traffic. In this paper, we develop a technique for the first line of online anomaly detection with two important considerations: (i) availability of traffic attributes during the monitoring time, and (ii) computational scalability for streaming data. The presented learning technique is lightweight and highly scalable with the beauty of approximation based on the grid partitioning of the given dimensional space. With the public traffic traces of KDD Cup 1999 and NSL-KDD, we show that our technique yields 98.5% and 83% of detection accuracy, respectively, only with a couple of readily available traffic attributes that can be obtained without the help of post-processing. Finally, the results are at least comparable with the classical learning methods including decision tree and random forest, with approximately two orders of magnitude faster learning performance.

Authors:

Kim, Jinoh ^[1]; Yoo, Wucherl ^[2]; Sim, Alex ^[2]; Suh, Sang C. ^[1]; Kim, Ikkyun ^[3]

Texas A & M Univ., Commerce, TX (United States)
Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States)
Electronics and Telecommunications Research Inst. (ETRI), Daejeon (Korea, Republic of)

Publication Date:: Mon Mar 13 00:00:00 EDT 2017

Research Org.:: Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States); Electronics and Telecommunications Research Inst. (ETRI), Daejeon (Korea, Republic of)

Sponsoring Org.:: USDOE Office of Science (SC), Workforce Development for Teachers and Scientists (WDTS); USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR); Ministry of Science, ICT and Future Planning (MSIP) of Korea

OSTI Identifier:: 1379772

Grant/Contract Number:: AC02-05CH11231; B0101-15-1293

Resource Type:: Accepted Manuscript

Journal Name:: 2017 International Conference on Computing, Networking and Communications, ICNC 2017

Additional Journal Information:: Journal Name: 2017 International Conference on Computing, Networking and Communications, ICNC 2017

Country of Publication:: United States

Language:: English

Subject:: 97 MATHEMATICS AND COMPUTING; complexity theory; testing; conferences; learning systems; computer crime; partitioning algorithms; decision trees; telecommunication traffic; computer network security

Citation Formats


                    Kim, Jinoh, Yoo, Wucherl, Sim, Alex, Suh, Sang C., and Kim, Ikkyun. A lightweight network anomaly detection technique.  United States: N. p., 2017. 
Web.  doi:10.1109/ICCNC.2017.7876251.

Copy to clipboard


                    Kim, Jinoh, Yoo, Wucherl, Sim, Alex, Suh, Sang C., & Kim, Ikkyun. A lightweight network anomaly detection technique.  United States.  https://doi.org/10.1109/ICCNC.2017.7876251

Copy to clipboard


                    Kim, Jinoh, Yoo, Wucherl, Sim, Alex, Suh, Sang C., and Kim, Ikkyun. Mon .  
"A lightweight network anomaly detection technique".  United States.  https://doi.org/10.1109/ICCNC.2017.7876251.  https://www.osti.gov/servlets/purl/1379772.

Copy to clipboard


                    
@article{osti_1379772,

  title        = {A lightweight network anomaly detection technique},

  author       = {Kim, Jinoh and Yoo, Wucherl and Sim, Alex and Suh, Sang C. and Kim, Ikkyun},

  abstractNote = {While the network anomaly detection is essential in network operations and management, it becomes further challenging to perform the first line of detection against the exponentially increasing volume of network traffic. In this paper, we develop a technique for the first line of online anomaly detection with two important considerations: (i) availability of traffic attributes during the monitoring time, and (ii) computational scalability for streaming data. The presented learning technique is lightweight and highly scalable with the beauty of approximation based on the grid partitioning of the given dimensional space. With the public traffic traces of KDD Cup 1999 and NSL-KDD, we show that our technique yields 98.5% and 83% of detection accuracy, respectively, only with a couple of readily available traffic attributes that can be obtained without the help of post-processing. Finally, the results are at least comparable with the classical learning methods including decision tree and random forest, with approximately two orders of magnitude faster learning performance.},

  doi          = {10.1109/ICCNC.2017.7876251},

  journal      = {2017 International Conference on Computing, Networking and Communications, ICNC 2017},

  number       = ,

  volume       = ,

  place        = {United States},

  year         = {Mon Mar 13 00:00:00 EDT 2017},

  month        = {Mon Mar 13 00:00:00 EDT 2017}

}

Copy to clipboard

Journal Article:

Free Publicly Available Full Text

Accepted Manuscript (DOE)

Publisher's Version of Record

https://doi.org/10.1109/ICCNC.2017.7876251

Other availability

Search WorldCat to find libraries that may hold this journal

Save / Share:

Export Metadata

Save to My Library

Similar Records in DOE PAGES and OSTI.GOV collections:

An approach to online network monitoring using clustered patterns

Journal Article Kim, Jinoh ; Sim, Alex ; Suh, Sang C. ; ... - 2017 International Conference on Computing, Networking and Communications, ICNC 2017

Network traffic monitoring is a core element in network operations and management for various purposes such as anomaly detection, change detection, and fault/failure detection. In this study, we introduce a new approach to online monitoring using a pattern-based representation of the network traffic. Unlike the past online techniques limited to a single variable to summarize (e.g., sketch), the focus of this study is on capturing the network state from the multivariate attributes under consideration. To this end, we employ clustering with its benefit of the aggregation of multidimensional variables. The clustered result represents the state of the network with regardmore »« less
https://doi.org/10.1109/ICCNC.2017.7876207

Full Text Available
HPNAIDM: The High-Performance Network Anomaly/Intrusion Detection and Mitigation System

Technical Report Chen, Yan

Identifying traffic anomalies and attacks rapidly and accurately is critical for large network operators. With the rapid growth of network bandwidth, such as the next generation DOE UltraScience Network, and fast emergence of new attacks/virus/worms, existing network intrusion detection systems (IDS) are insufficient because they: • Are mostly host-based and not scalable to high-performance networks; • Are mostly signature-based and unable to adaptively recognize flow-level unknown attacks; • Cannot differentiate malicious events from the unintentional anomalies. To address these challenges, we proposed and developed a new paradigm called high-performance network anomaly/intrustion detection and mitigation (HPNAIDM) system. The new paradigm ismore »« less
https://doi.org/10.2172/1108982

Full Text Available
Network Anomaly Detection Using Federated Learning

Conference Marfo, William ; Tosh, Deepak K. ; Moore, Shirley V. - MILCOM 2022 - 2022 IEEE Military Communications Conference (MILCOM)

Due to the veracity and heterogeneity in network traffic, detecting anomalous events is challenging. The computational load on global servers is a significant challenge in terms of efficiency, accuracy, and scalability. Our primary motivation is to introduce a robust and scalable framework that enables efficient network anomaly detection. We address the issue of scalability and efficiency for network anomaly detection by leveraging federated learning, in which multiple participants train a global model jointly. Unlike centralized training architectures, federated learning does not require participants to upload their training data to the server, preventing attackers from exploiting the training data. Moreover, mostmore »« less
https://doi.org/10.1109/milcom55135.2022.10017793

Full Text Available
Self-Taught Anomaly Detection With Hybrid Unsupervised/Supervised Machine Learning in Optical Networks

Journal Article Chen, Xiaoliang ; Li, Baojia ; Proietti, Roberto ; ... - Journal of Lightwave Technology

Here this paper proposes a self-taught anomaly detection framework for optical networks. The proposed framework makes use of a hybrid unsupervised and supervised machine learning scheme. First, it employs an unsupervised data clustering module (DCM) to analyze the patterns of monitoring data. The DCM enables a self-learning capability that eliminates the requirement of prior knowledge of abnormal network behaviors and therefore can potentially detect unforeseen anomalies. Second, we introduce a self-taught mechanism that transfers the patterns learned by the DCM to a supervised data regression and classification module (DRCM). The DRCM, whose complexity is mainly related to the scale ofmore »« less
Cited by 55
https://doi.org/10.1109/jlt.2019.2902487

Full Text Available
Signal Processing Based Method for Real-Time Anomaly Detection in High-Performance Computing

Conference Kelly, Christopher ; Dey, Arunavo ; Islam, Tanzima ; ...

Performance anomalies can manifest as irregular execution times or abnormal execution events for many reasons, including network congestion and resource contention. Detecting such anomalies in real-time by analyzing the details of performance traces at scale is impractical due to the sheer volume of data High-Performance Computing (HPC) applications produce. In this paper, we propose formulating HPC performance anomaly detection as a signal-processing problem where anomalies can be treated as noise. We evaluate our proposed method in comparison with two other commonly used anomaly detection techniques of varying complexity based on their detection accuracy and scalability. Since real-time in-situ anomaly detectionmore »« less
https://doi.org/10.1109/compsac57700.2023.00037

Full Text Available

Similar Records