Self-Taught Anomaly Detection With Hybrid Unsupervised/Supervised Machine Learning in Optical Networks
Here this paper proposes a self-taught anomaly detection framework for optical networks. The proposed framework makes use of a hybrid unsupervised and supervised machine learning scheme. First, it employs an unsupervised data clustering module (DCM) to analyze the patterns of monitoring data. The DCM enables a self-learning capability that eliminates the requirement of prior knowledge of abnormal network behaviors and therefore can potentially detect unforeseen anomalies. Second, we introduce a self-taught mechanism that transfers the patterns learned by the DCM to a supervised data regression and classification module (DRCM). The DRCM, whose complexity is mainly related to the scale of the applied supervised learning model, can potentially facilitate more scalable and time-efficient online anomaly detection by avoiding excessively traversing the original dataset. We designed the DCM and DRCM based on the density-based clustering algorithm and the deep neural network structure, respectively. Evaluations with experimental data from two use cases (i.e., single-point detection and end-to-end detection) demonstrate that up to 99% anomaly detection accuracy can be achieved with a false positive rate below 1%.
- Research Organization:
- Univ. of California, Davis, CA (United States)
- Sponsoring Organization:
- USDOE Office of Science (SC); National Science Foundation (NSF)
- Grant/Contract Number:
- SC0016700; ICE-T:RC 1836921
- OSTI ID:
- 1612677
- Journal Information:
- Journal of Lightwave Technology, Vol. 37, Issue 7; ISSN 0733-8724
- Publisher:
- IEEECopyright Statement
- Country of Publication:
- United States
- Language:
- English
Web of Science
Similar Records
Development of Gamma Background Radiation Digital Twin with Machine Learning Algorithms: Application of Unsupervised Machine Learning to Detection of Anomalies and Nuisances in Gamma Background Radiation Environmental Screening Data
MAD: Self-Supervised Masked Anomaly Detection Task for Multivariate Time Series