MAD: Self-Supervised Masked Anomaly Detection Task for Multivariate Time Series

Fu, Yiwei; Xue, Feng

doi:10.1109/ijcnn55064.2022.9892218

MAD: Self-Supervised Masked Anomaly Detection Task for Multivariate Time Series

Conference · Mon Jul 18 04:00:00 EDT 2022 · 2022 International Joint Conference on Neural Networks (IJCNN)

DOI:https://doi.org/10.1109/ijcnn55064.2022.9892218· OSTI ID:1905117

Fu, Yiwei ^[1]; Xue, Feng ^[2]

GE Research,Niskayuna,NY,USA; GE Research
GE Research,Niskayuna,NY,USA

In this paper, we introduce Masked Anomaly Detection (MAD), a general self-supervised learning task for multivariate time series anomaly detection. With the increasing availability of sensor data from industrial systems, being able to detecting anomalies from streams of multivariate time series data is of significant importance. Given the scarcity of anomalies in real-world applications, the majority of literature has been focusing on modeling normality. The learned normal representations can empower anomaly detection as the model has learned to capture certain key underlying data regularities. A typical formulation is to learn a predictive model, i.e., use a window of time series data to predict future data values. In this paper, we propose an alternative self-supervised learning task. By randomly masking a portion of the inputs and training a model to estimate them using the remaining ones, MAD is an improvement over the traditional left-to-right next step prediction (NSP) task. Our experimental results demonstrate that MAD can achieve better anomaly detection rates over traditional NSP approaches when using exactly the same neural network (NN) base models, and can be modified to run as fast as NSP models during test time on the same hardware, thus making it an ideal upgrade for many existing NSP-based NN anomaly detection models.

View Conference

Research Organization:: GE Research

Sponsoring Organization:: USDOE Office of Fossil Energy (FE)

DOE Contract Number:: FE0031763

OSTI ID:: 1905117

Report Number(s):: DOE-GER-FE0031763-4

Conference Information:: Journal Name: 2022 International Joint Conference on Neural Networks (IJCNN)

Country of Publication:: United States

Language:: English

References (19)

Long Short-Term Memory Hochreiter, Sepp; Schmidhuber, Jürgen Neural Computation, Vol. 9, Issue 8 https://doi.org/10.1162/neco.1997.9.8.1735	journal	November 1997
LogBERT: Log Anomaly Detection via BERT Guo, Haixuan; Yuan, Shuhan; Wu, Xintao 2021 International Joint Conference on Neural Networks (IJCNN) https://doi.org/10.1109/IJCNN52387.2021.9534113	conference	July 2021
Fault detection and identification using Bayesian recurrent neural networks Sun, Weike; Paiva, Antonio R. C.; Xu, Peng Computers & Chemical Engineering, Vol. 141 https://doi.org/10.1016/j.compchemeng.2020.106991	journal	October 2020
Context Encoders: Feature Learning by Inpainting Pathak, Deepak; Krahenbuhl, Philipp; Donahue, Jeff 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) https://doi.org/10.1109/CVPR.2016.278	conference	June 2016
Time-series novelty detection using one-class support vector machines Ma, J.; Perkins, S. 2003 International Joint Conference on Neural Networks, Proceedings of the International Joint Conference on Neural Networks, 2003. https://doi.org/10.1109/IJCNN.2003.1223670	conference	January 2003
A comparison study of basic data-driven fault diagnosis and process monitoring methods on the benchmark Tennessee Eastman process Yin, Shen; Ding, Steven X.; Haghani, Adel Journal of Process Control, Vol. 22, Issue 9 https://doi.org/10.1016/j.jprocont.2012.06.009	journal	October 2012
Subspace method aided data-driven design of fault detection and isolation systems Ding, S. X.; Zhang, P.; Naik, A. Journal of Process Control, Vol. 19, Issue 9 https://doi.org/10.1016/j.jprocont.2009.07.005	journal	October 2009
Robust Anomaly Detection for Multivariate Time Series through Stochastic Recurrent Neural Network Su, Ya; Zhao, Youjian; Niu, Chenhao Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining https://doi.org/10.1145/3292500.3330672	conference	July 2019
Anomaly detection: A survey Chandola, Varun; Banerjee, Arindam; Kumar, Vipin ACM Computing Surveys, Vol. 41, Issue 3, p. 1-58 https://doi.org/10.1145/1541880.1541882	journal	July 2009
Discriminative Unsupervised Feature Learning with Exemplar Convolutional Neural Networks Dosovitskiy, Alexey; Fischer, Philipp; Springenberg, Jost Tobias IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 38, Issue 9 https://doi.org/10.1109/TPAMI.2015.2496141	journal	September 2016
Spacecraft Anomaly Detection and Relation Visualization via Masked Time Series Modeling Meng, Hengyu; Li, Yuanxiang; Zhang, Yuxuan 2019 Prognostics and System Health Management Conference (PHM-Qingdao) https://doi.org/10.1109/PHM-Qingdao46334.2019.8943031	conference	October 2019
Momentum Contrast for Unsupervised Visual Representation Learning He, Kaiming; Fan, Haoqi; Wu, Yuxin 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) https://doi.org/10.1109/CVPR42600.2020.00975	conference	June 2020
Unsupervised Learning of Visual Representations Using Videos Wang, Xiaolong; Gupta, Abhinav 2015 IEEE International Conference on Computer Vision (ICCV) https://doi.org/10.1109/ICCV.2015.320	conference	December 2015
Disturbance detection and isolation by dynamic principal component analysis Ku, Wenfu; Storer, Robert H.; Georgakis, Christos Chemometrics and Intelligent Laboratory Systems, Vol. 30, Issue 1 https://doi.org/10.1016/0169-7439(95)00076-3	journal	November 1995
Deep anomaly detection for industrial systems: a case study Xue, Feng; Yan, Weizhong; Wang, Tianyi Annual Conference of the PHM Society, Vol. 12, Issue 1 https://doi.org/10.36001/phmconf.2020.v12i1.1186	journal	November 2020
Detecting Spacecraft Anomalies Using LSTMs and Nonparametric Dynamic Thresholding Hundman, Kyle; Constantinou, Valentino; Laporte, Christopher Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining https://doi.org/10.1145/3219819.3219845	conference	July 2018
A plant-wide industrial process control problem Downs, J. J.; Vogel, E. F. Computers & Chemical Engineering, Vol. 17, Issue 3 https://doi.org/10.1016/0098-1354(93)80018-I	journal	March 1993
Unsupervised Visual Representation Learning by Context Prediction Doersch, Carl; Gupta, Abhinav; Efros, Alexei A. 2015 IEEE International Conference on Computer Vision (ICCV) https://doi.org/10.1109/ICCV.2015.167	conference	December 2015
Shuffle and Learn: Unsupervised Learning Using Temporal Order Verification Misra, Ishan; Zitnick, C. Lawrence; Hebert, Martial Computer Vision – ECCV 2016 https://doi.org/10.1007/978-3-319-46448-0_32	conference	January 2016

Similar Records

Deep anomaly detection for industrial systems: a case study

Conference · Thu Nov 12 23:00:00 EST 2020 · OSTI ID:1717878

Multivariate Time Series Anomaly Detection with Few Positive Samples

Conference · Mon Jul 18 00:00:00 EDT 2022 · 2022 International Joint Conference on Neural Networks (IJCNN) · OSTI ID:1905124

Masked Particle Modeling on Sets: Towards Self-Supervised High Energy Physics Foundation Models

Journal Article · Tue Jul 16 20:00:00 EDT 2024 · Machine Learning: Science and Technology · OSTI ID:2403614

Related Subjects

97 MATHEMATICS AND COMPUTING

MAD: Self-Supervised Masked Anomaly Detection Task for Multivariate Time Series

Citation Formats

References (19)

Similar Records

Related Subjects