Monitoring covariance in multivariate time series: Comparing machine learning and statistical approaches

Weix, Derek; Cath, Tzahi Y.; Hering, Amanda S.

doi:10.1002/qre.3551

Title: Monitoring covariance in multivariate time series: Comparing machine learning and statistical approaches

Journal Article · 08 April 2024 · Quality and Reliability Engineering International

DOI: https://doi.org/10.1002/qre.3551 · OSTI ID:2335833

Weix, Derek ^[1];

^[2];

^[1]

Department of Statistical Science Baylor University Waco Texas USA
Department of Civil and Environmental Engineering Colorado School of Mines Golden Colorado USA

Abstract In complex systems with multiple variables monitored at high‐frequency, variables are not only temporally autocorrelated, but they may also be nonlinearly related or exhibit nonstationarity as the inputs or operation changes. One approach to handling such variables is to detrend them prior to monitoring and then apply control charts that assume independence and stationarity to the residuals. Monitoring controlled systems is even more challenging because the control strategy seeks to maintain variables at prespecified mean levels, and to compensate, correlations among variables may change, making monitoring the covariance essential. In this paper, a vector autoregressive model (VAR) is compared with a multivariate random forest (MRF) and a neural network (NN) for detrending multivariate time series prior to monitoring the covariance of the residuals using a multivariate exponentially weighted moving average (MEWMA) control chart. Machine learning models have an advantage when the data's structure is unknown or may change. We design a novel simulation study with nonlinear, nonstationary, and autocorrelated data to compare the different detrending models and subsequent covariance monitoring. The machine learning models have superior performance for nonlinear and strongly autocorrelated data and similar performance for linear data. An illustration with data from a reverse osmosis process is given.

View Journal Article

Cite

Export

Save

Sponsoring Organization:: USDOE

OSTI ID:: 2335833

Alternate ID(s):: OSTI ID: 2335834

Journal Information:: Quality and Reliability Engineering International, Journal Name: Quality and Reliability Engineering International Journal Issue: 5 Vol. 40; ISSN 0748-8017

Publisher:: Wiley Blackwell (John Wiley & Sons)Copyright Statement

Country of Publication:: United Kingdom

Language:: English

References (50)

Monitoring the covariance matrix via penalized likelihood estimation Li, Bo; Wang, Kaibo; Yeh, Arthur B. IIE Transactions, Vol. 45, Issue 2 https://doi.org/10.1080/0740817X.2012.663952	journal	February 2013
Fault detection based on time series modeling and multivariate statistical process control Sánchez-Fernández, A.; Baldán, F. J.; Sainz-Palmero, G. I. Chemometrics and Intelligent Laboratory Systems, Vol. 182 https://doi.org/10.1016/j.chemolab.2018.08.003	journal	November 2018
Fault detection and diagnosis using tree-based ensemble learning methods and multivariate control charts for centrifugal chillers Yao, Wanli; Li, Donghui; Gao, Long Journal of Building Engineering, Vol. 51 https://doi.org/10.1016/j.jobe.2022.104243	journal	July 2022
Run Length Distribution and Percentiles: The Shewhart Chart with Unknown Parameters Chakraborti, S. Quality Engineering, Vol. 19, Issue 2 https://doi.org/10.1080/08982110701276653	journal	April 2007
Robust Anomaly Detection for Multivariate Time Series through Stochastic Recurrent Neural Network Su, Ya; Zhao, Youjian; Niu, Chenhao Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining https://doi.org/10.1145/3292500.3330672	conference	July 2019
Monitoring the cross-covariances of a multivariate time series Śliwa, Przemysław; Schmid, Wolfgang Metrika, Vol. 61, Issue 1 https://doi.org/10.1007/s001840400326	journal	February 2005
Constructing decision trees with multiple response variables Kim, Seong-Jun; Lee, Kang Bae International Journal of Management and Decision Making, Vol. 4, Issue 4 https://doi.org/10.1504/IJMDM.2003.003998	journal	January 2003
An adaptive variable-parameters scheme for the simultaneous monitoring of the mean and variability of an autocorrelated multivariate normal process Sabahno, Hamed; Castagliola, Philippe; Amiri, Amirhossein Journal of Statistical Computation and Simulation, Vol. 90, Issue 8 https://doi.org/10.1080/00949655.2020.1730373	journal	February 2020
Statistical process monitoring of time-dependent data De Ketelaere, Bart; Rato, Tiago; Schmitt, Eric Quality Engineering, Vol. 28, Issue 1 https://doi.org/10.1080/08982112.2015.1100474	journal	January 2016
Continuous Inspection Schemes Page, E. S. Biometrika, Vol. 41, Issue 1-2 https://doi.org/10.1093/biomet/41.1-2.100	journal	January 1954
Multivariate Process Variability Monitoring Djauhari, Maman A.; Mashuri, Muhammad; Herwindiati, Dyah E. Communications in Statistics - Theory and Methods, Vol. 37, Issue 11 https://doi.org/10.1080/03610920701826286	journal	May 2008
A simulation comparison of some distance‐based EWMA control charts for monitoring the covariance matrix with individual observations Ning, Xiu; Li, Pingke Quality and Reliability Engineering International, Vol. 36, Issue 1 https://doi.org/10.1002/qre.2559	journal	October 2019
A unifying framework for detecting outliers and change points from non-stationary time series data Yamanishi, Kenji; Takeuchi, Jun-ichi Proceedings of the eighth ACM SIGKDD international conference on Knowledge discovery and data mining https://doi.org/10.1145/775047.775148	conference	July 2002
Bivariate semiparametric control charts for simultaneous monitoring of process mean and variance Koutras, Markos V.; Sofikitou, Elisavet M. Quality and Reliability Engineering International, Vol. 36, Issue 2 https://doi.org/10.1002/qre.2514	journal	December 2019
CUSUM control schemes for monitoring the covariance matrix of multivariate time series Bodnar, Olha; Schmid, Wolfgang Statistics, Vol. 51, Issue 4 https://doi.org/10.1080/02331888.2016.1268616	journal	December 2016
Regression Adjustment for Variables in Multivariate Quality Control Hawkins, Douglas M. Journal of Quality Technology, Vol. 25, Issue 3 https://doi.org/10.1080/00224065.1993.11979451	journal	July 1993
Multivariate Control Charts for Monitoring the Mean Vector and Covariance Matrix Matrix, Covariance; Reynolds, Marion R.; Cho, Gyo-Young Journal of Quality Technology, Vol. 38, Issue 3 https://doi.org/10.1080/00224065.2006.11918612	journal	July 2006
A unifying framework for detecting outliers and change points from time series Takeuchi, J.; Yamanishi, K. IEEE Transactions on Knowledge and Data Engineering, Vol. 18, Issue 4 https://doi.org/10.1109/TKDE.2006.1599387	journal	April 2006
Multivariate Control Charts for Monitoring Covariance Matrix: A Review Yeh, Arthur B.; Lin, Dennis K. J.; McGrath, Richard N. Quality Technology & Quantitative Management, Vol. 3, Issue 4 https://doi.org/10.1080/16843703.2006.11673124	journal	January 2006
Fault diagnosis in a wastewater treatment plant using dynamic Independent Component Analysis Villegas, T.; Fuente, M. J.; Sainz-Palmero, G. I. 18th Mediterranean Conference on Control and Automation, MED'10 https://doi.org/10.1109/MED.2010.5547760	conference	June 2010
DeepAnT: A Deep Learning Approach for Unsupervised Anomaly Detection in Time Series Munir, Mohsin; Siddiqui, Shoaib Ahmed; Dengel, Andreas IEEE Access, Vol. 7 https://doi.org/10.1109/ACCESS.2018.2886457	journal	January 2019
On the Properties of Neural Machine Translation: Encoder–Decoder Approaches Cho, Kyunghyun; van Merrienboer, Bart; Bahdanau, Dzmitry Proceedings of SSST-8, Eighth Workshop on Syntax, Semantics and Structure in Statistical Translation https://doi.org/10.3115/v1/W14-4012	conference	January 2014
On extended long short-term memory and dependent bidirectional recurrent neural network Su, Yuanhang; Kuo, C. -C. Jay Neurocomputing, Vol. 356 https://doi.org/10.1016/j.neucom.2019.04.044	journal	September 2019
Random Forests Breiman, Leo Machine Learning, Vol. 45, Issue 1, p. 5-32 https://doi.org/10.1023/A:1010933404324	journal	January 2001
A survey of methods for time series change point detection Aminikhanghahi, Samaneh; Cook, Diane J. Knowledge and Information Systems, Vol. 51, Issue 2 https://doi.org/10.1007/s10115-016-0987-z	journal	September 2016
Case studies in real-time fault isolation in a decentralized wastewater treatment facility Klanderman, Molly C.; Newhart, Kathryn B.; Cath, Tzahi Y. Journal of Water Process Engineering, Vol. 38 https://doi.org/10.1016/j.jwpe.2020.101556	journal	December 2020
Long Short-Term Memory Hochreiter, Sepp; Schmidhuber, Jürgen Neural Computation, Vol. 9, Issue 8 https://doi.org/10.1162/neco.1997.9.8.1735	journal	November 1997
Comparison of linear and nonlinear dimension reduction techniques for automated process monitoring of a decentralized wastewater treatment facility Kazor, Karen; Holloway, Ryan W.; Cath, Tzahi Y. Stochastic Environmental Research and Risk Assessment, Vol. 30, Issue 5 https://doi.org/10.1007/s00477-016-1246-2	journal	April 2016
Multivariate random forests Segal, Mark; Xiao, Yuanyuan WIREs Data Mining and Knowledge Discovery, Vol. 1, Issue 1 https://doi.org/10.1002/widm.12	journal	January 2011
A nonparametric CUSUM chart for monitoring multivariate serially correlated processes Xue, Li; Qiu, Peihua Journal of Quality Technology, Vol. 53, Issue 4 https://doi.org/10.1080/00224065.2020.1778430	journal	June 2020
Light Gated Recurrent Units for Speech Recognition Ravanelli, Mirco; Brakel, Philemon; Omologo, Maurizio IEEE Transactions on Emerging Topics in Computational Intelligence, Vol. 2, Issue 2 https://doi.org/10.1109/TETCI.2017.2762739	journal	April 2018
Statistical monitoring of the covariance matrix in multivariate processes: A literature review Ebadi, Mohsen; Chenouri, Shojaeddin; Lin, Dennis K. J. Journal of Quality Technology, Vol. 54, Issue 3 https://doi.org/10.1080/00224065.2021.1889419	journal	May 2021
Two group inspection-based control charts for dispersion matrix Gadre, M. P.; Kakade, V. C. Communications in Statistics - Simulation and Computation, Vol. 47, Issue 6 https://doi.org/10.1080/03610918.2017.1321120	journal	July 2017
Multivariate auto‐correlated process control by a residual‐based mixed CUSUM‐EWMA model Wang, Kung‐Jeng; Asrini, Luh Juni Quality and Reliability Engineering International, Vol. 39, Issue 4 https://doi.org/10.1002/qre.3278	journal	February 2023
A Multivariate Exponentially Weighted Moving Average Control Chart Lowry, Cynthia A.; Woodall, William H.; Champ, Charles W. Technometrics, Vol. 34, Issue 1 https://doi.org/10.2307/1269551	journal	February 1992
A Single Statistic for Monitoring the Covariance Matrix of Bivariate Processes Quinino, Roberto; Costa, A.; Lee Ho, Linda Quality Engineering, Vol. 24, Issue 3 https://doi.org/10.1080/08982112.2012.682046	journal	July 2012
A New Process Control Chart for Monitoring Short-Range Serially Correlated Data Qiu, Peihua; Li, Wendong; Li, Jun Technometrics, Vol. 62, Issue 1 https://doi.org/10.1080/00401706.2018.1562988	journal	May 2019
A Review on Outlier/Anomaly Detection in Time Series Data Blázquez-García, Ane; Conde, Angel; Mori, Usue ACM Computing Surveys, Vol. 54, Issue 3 https://doi.org/10.1145/3444690	journal	April 2021
Variable selection methods in multivariate statistical process control: A systematic literature review Peres, Fernanda Araujo Pimentel; Fogliatto, Flavio Sanson Computers & Industrial Engineering, Vol. 115 https://doi.org/10.1016/j.cie.2017.12.006	journal	January 2018
Control charts for variability monitoring in high-dimensional processes Kim, Jinho; Abdella, Galal M.; Kim, Sangahn Computers & Industrial Engineering, Vol. 130 https://doi.org/10.1016/j.cie.2019.02.012	journal	April 2019
Batch tracking via nonlinear principal component analysis Dong, Dong; McAvoy, Thomas J. AIChE Journal, Vol. 42, Issue 8 https://doi.org/10.1002/aic.690420810	journal	August 1996
Disturbance detection and isolation by dynamic principal component analysis Ku, Wenfu; Storer, Robert H.; Georgakis, Christos Chemometrics and Intelligent Laboratory Systems, Vol. 30, Issue 1 https://doi.org/10.1016/0169-7439(95)00076-3	journal	November 1995
Monitoring Multivariate Process Variability for Individual Observations Huwang, Longcheen; Yeh, Arthur B.; Wu, Chien-Wei Journal of Quality Technology, Vol. 39, Issue 3 https://doi.org/10.1080/00224065.2007.11917692	journal	July 2007
A new multivariate EWMA scheme for monitoring covariance matrices Shen, Xiaobei; Tsung, Fugee; Zou, Changliang International Journal of Production Research, Vol. 52, Issue 10 https://doi.org/10.1080/00207543.2013.842019	journal	October 2013
Fault Isolation for A Complex Decentralized Waste Water Treatment Facility Klanderman, Molly C.; Newhart, Kathryn B.; Cath, Tzahi Y. Journal of the Royal Statistical Society Series C: Applied Statistics, Vol. 69, Issue 4 https://doi.org/10.1111/rssc.12429	journal	July 2020
Elements of Multivariate Time Series Analysis Reinsel, Gregory C. Springer Series in Statistics https://doi.org/10.1007/978-1-4684-0198-1	book	January 1993
The Regression Control Chart Mandel, B. J. Journal of Quality Technology, Vol. 1, Issue 1 https://doi.org/10.1080/00224065.1969.11980341	journal	January 1969
Stacking with Multi-response Model Trees Džeroski, Sašo; Ženko, Bernard Multiple Classifier Systems, p. 201-211 https://doi.org/10.1007/3-540-45428-4_20	book	January 2002
A Least Angle Regression Control Chart for Multidimensional Data Capizzi, Giovanna; Masarotto, Guido Technometrics, Vol. 53, Issue 3 https://doi.org/10.1198/TECH.2011.10027	journal	August 2011
A multivariate control chart for simultaneously monitoring process mean and variability Zhang, Jiujun; Li, Zhonghua; Wang, Zhaojun Computational Statistics & Data Analysis, Vol. 54, Issue 10 https://doi.org/10.1016/j.csda.2010.03.027	journal	October 2010

Similar Records

Field study of dispersion in a heterogeneous aquifer. 3. Geostatistical analysis of hydraulic conductivity

Journal Article · 1992 · Water Resources Research; (United States) · OSTI ID:6793262

Rehfeldt, K R; Gelhar, L W; Boggs, J M

Moving averages for environmental standards

Journal Article · 1982 · Simulation; (United States) · OSTI ID:6660201

Gardenier, T K

Estimates of macrodispersivity based on analyses of hydraulic conductivity variability at the MADE (Microdispersion Experiment) site: Interim report

Technical Report · 1989 · OSTI ID:5724479

Waldrop, W R

Title: Monitoring covariance in multivariate time series: Comparing machine learning and statistical approaches

Citation Formats

References (50)

Similar Records

Related Subjects