skip to main content
OSTI.GOV title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: MAD: Self-Supervised Masked Anomaly Detection Task for Multivariate Time Series

Conference · · 2022 International Joint Conference on Neural Networks (IJCNN)

In this paper, we introduce Masked Anomaly Detection (MAD), a general self-supervised learning task for multivariate time series anomaly detection. With the increasing availability of sensor data from industrial systems, being able to detecting anomalies from streams of multivariate time series data is of significant importance. Given the scarcity of anomalies in real-world applications, the majority of literature has been focusing on modeling normality. The learned normal representations can empower anomaly detection as the model has learned to capture certain key underlying data regularities. A typical formulation is to learn a predictive model, i.e., use a window of time series data to predict future data values. In this paper, we propose an alternative self-supervised learning task. By randomly masking a portion of the inputs and training a model to estimate them using the remaining ones, MAD is an improvement over the traditional left-to-right next step prediction (NSP) task. Our experimental results demonstrate that MAD can achieve better anomaly detection rates over traditional NSP approaches when using exactly the same neural network (NN) base models, and can be modified to run as fast as NSP models during test time on the same hardware, thus making it an ideal upgrade for many existing NSP-based NN anomaly detection models.

Research Organization:
GE Research
Sponsoring Organization:
USDOE Office of Fossil Energy (FE)
DOE Contract Number:
FE0031763
OSTI ID:
1905117
Report Number(s):
DOE-GER-FE0031763-4
Journal Information:
2022 International Joint Conference on Neural Networks (IJCNN), Conference: 2022 International Joint Conference on Neural Networks (IJCNN), Padua, Italy, 18-23 July 2022
Country of Publication:
United States
Language:
English

References (18)

Momentum Contrast for Unsupervised Visual Representation Learning conference June 2020
Unsupervised Learning of Visual Representations Using Videos conference December 2015
Anomaly detection: A survey journal July 2009
Unsupervised Visual Representation Learning by Context Prediction conference December 2015
Context Encoders: Feature Learning by Inpainting conference June 2016
Discriminative Unsupervised Feature Learning with Exemplar Convolutional Neural Networks journal September 2016
Robust Anomaly Detection for Multivariate Time Series through Stochastic Recurrent Neural Network conference July 2019
Deep anomaly detection for industrial systems: a case study journal November 2020
Long Short-Term Memory journal November 1997
Subspace method aided data-driven design of fault detection and isolation systems journal October 2009
Disturbance detection and isolation by dynamic principal component analysis journal November 1995
Detecting Spacecraft Anomalies Using LSTMs and Nonparametric Dynamic Thresholding
  • Hundman, Kyle; Constantinou, Valentino; Laporte, Christopher
  • Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining https://doi.org/10.1145/3219819.3219845
conference July 2018
Time-series novelty detection using one-class support vector machines conference January 2003
Fault detection and identification using Bayesian recurrent neural networks journal October 2020
A comparison study of basic data-driven fault diagnosis and process monitoring methods on the benchmark Tennessee Eastman process journal October 2012
Spacecraft Anomaly Detection and Relation Visualization via Masked Time Series Modeling conference October 2019
LogBERT: Log Anomaly Detection via BERT conference July 2021
A plant-wide industrial process control problem journal March 1993