Unsupervised Deep Anomaly Detection for Industrial Multivariate Time Series Data
Source abstract
Abstract
With the rapid development of deep learning, researchers are actively exploring its applications in the field of industrial anomaly detection. Deep learning methods differ significantly from traditional mathematical modeling approaches, eliminating the need for intricate mathematical derivations and offering greater flexibility. Deep learning technologies have demonstrated outstanding performance in anomaly detection problems and gained widespread recognition. However, when dealing with multivariate data anomaly detection problems, deep learning faces challenges such as large-scale data annotation and handling relationships between complex data variables. To address these challenges, this study proposes an innovative and lightweight deep learning model—the Attention-Based Deep Convolutional Autoencoding Prediction Network (AT-DCAEP). The model consists of a characterization network based on convolutional autoencoders and a prediction network based on attention mechanisms. The AT-DCAEP exhibits excellent performance in multivariate time series data anomaly detection without the need for pre-labeling large-scale datasets, making it an efficient unsupervised anomaly detection method. We extensively tested the performance of AT-DCAEP on six publicly available datasets, and the results show that compared to current state-of-the-art methods, AT-DCAEP demonstrates superior performance, achieving the optimal balance between anomaly detection performance and computational cost.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Anomaly Detection Techniques and Applications · Time Series Analysis and Forecasting · Network Security and Intrusion Detection
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Faculty of Information Science and Engineering, Ocean University of China, Qingdao 266100, China
Information and Telecommunication Company, State Grid Shandong Electric Power Company, Jinan 250013, China
Funding
National Key Research and Development Program of China · State Grid Shandong Electric Power Company Technology Project
References (46)
Show source-provided references
1. Enabling technologies for Industry 4.0 manufacturing and supply chain: Concepts, current status, and adoption challenges10.1109/EMR.2020.2987884 ↗
2. From raw data to smart manufacturing: AI and semantic web of things for industry 4.010.1109/MIS.2018.043741325 ↗
3. Deep reinforcement learning for resource protection and real-time detection in IoT environment10.1109/JIOT.2020.2974281 ↗
4. Cai, Z., and He, Z. (2019, January 7–9). Trading private range counting over big IoT data. Proceedings of the 2019 IEEE 39th International Conference on Distributed Computing Systems (ICDCS), Dallas, TX, USA.10.1109/ICDCS.2019.00023 ↗
5. Anomaly detection via blockchained deep learning smart contracts in industry 4.010.1007/s00521-020-05189-8 ↗
6. Integrated generative model for industrial anomaly detection via Bidirectional LSTM and attention mechanism10.1109/TII.2021.3078192 ↗
7. Outlier detection for temporal data: A survey10.1109/TKDE.2013.184 ↗
8. Deep learning for anomaly detection in time-series data: Review, analysis, and guidelines10.1109/ACCESS.2021.3107975 ↗
9. Novelty detection: A review—Part 1: Statistical approaches10.1016/j.sigpro.2003.07.018 ↗
10. Kiss, I., Genge, B., Haller, P., and Sebestyén, G. (2014, January 4–6). Data clustering-based anomaly detection in industrial control systems. Proceedings of the 2014 IEEE 10th International Conference on Intelligent Computer Communication and Processing (ICCP), Cluj-Napoca, Romania.10.1109/ICCP.2014.6937009 ↗
11. Ma, J., and Perkins, S. (2003, January 20–24). Time-series novelty detection using one-class support vector machines. Proceedings of the International Joint Conference on Neural Networks, Portland, OR, USA.
12. On the time series k-nearest neighbor classification of abnormal brain activity10.1109/TSMCA.2007.897589 ↗
13. Liu, F.T., Ting, K.M., and Zhou, Z.-H. (2008, January 15–19). Isolation forest. Proceedings of the 2008 Eighth IEEE International Conference on Data Mining, Pisa, Italy.10.1109/ICDM.2008.17 ↗
14. Shipmon, D.T., Gurevitch, J.M., Piselli, P.M., and Edwards, S.T. (2017). Time series anomaly detection; detection of anomalous drops with limited features and sparse examples in noisy highly periodic data. arXiv.
15. Chen, J., Sathe, S., Aggarwal, C., and Turaga, D. (2017, January 27–29). Outlier detection with autoencoder ensembles. Proceedings of the 2017 SIAM International Conference on Data Mining, Houston, TX, USA.10.1137/1.9781611974973.11 ↗
16. Anomaly detection and critical SCADA parameters identification for wind turbines based on LSTM-AE neural network10.1016/j.renene.2021.03.078 ↗
17. A multimodal anomaly detector for robot-assisted feeding using an lstm-based variational autoencoder10.1109/LRA.2018.2801475 ↗
18. Shi, X., Chen, Z., Wang, H., Yeung, D.-Y., Wong, W.-k., and Woo, W.-c. Convolutional LSTM Network: A machine learning approach for precipitation nowcasting. Proceedings of the 28th International Conference on Neural Information Processing Systems—Volume 1, Montreal, QC, Canada.
19. Web traffic anomaly detection using C-LSTM neural networks10.1016/j.eswa.2018.04.004 ↗
20. Kim, Y., Moschitti, A., Pang, B., and Daelemans, W. (2014, January 25–29). Convolutional neural networks for sentence classification. Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), Doha, Qatar.10.3115/v1/D14-1181 ↗
21. Zong, B., Song, Q., Min, M.R., Cheng, W., Lumezanu, C., Cho, D., and Chen, H. (May, January 30). Deep autoencoding gaussian mixture model for unsupervised anomaly detection. Proceedings of the International Conference on Learning Representations, Vancouver, BC, Canada.
22. Li, D., Chen, D., Jin, B., Shi, L., Goh, J., and Ng, S.-K. (2019, January 17–19). MAD-GAN: Multivariate anomaly detection for time series data with generative adversarial networks. Proceedings of the International Conference on Artificial Neural Networks, Munich, Germany.10.1007/978-3-030-30490-4_56 ↗
23. Su, Y., Zhao, Y., Niu, C., Liu, R., Sun, W., and Pei, D. (2019, January 4–8). Robust anomaly detection for multivariate time series through stochastic recurrent neural network. Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, Anchorage, AK, USA.10.1145/3292500.3330672 ↗
24. Unsupervised deep anomaly detection for multi-sensor time-series signals
25. Zhang, C., Song, D., Chen, Y., Feng, X., Lumezanu, C., Cheng, W., Ni, J., Zong, B., Chen, H., and Chawla, N.V. (February, January 27). A deep neural network for unsupervised anomaly detection and diagnosis in multivariate time series data. Proceedings of the AAAI Conference on Artificial Intelligence, Honolulu, HI, USA.
26. Audibert, J., Michiardi, P., Guyard, F., Marti, S., and Zuluaga, M.A. (2020, January 6–10). Usad: Unsupervised anomaly detection on multivariate time series. Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, Online.
27. Deng, A., and Hooi, B. (2021, January 2–9). Graph neural network-based anomaly detection in multivariate time series. Proceedings of the AAAI Conference on Artificial Intelligence, Online.10.1609/aaai.v35i5.16523 ↗
28. Tranad: Deep transformer networks for anomaly detection in multivariate time series data10.14778/3514061.3514067 ↗
29. Siffer, A., Fouque, P.-A., Termier, A., and Largouet, C. (2017, January 13–17). Anomaly detection in streams with extreme value theory. Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Halifax, NS, Canada.10.1145/3097983.3098144 ↗
30. Francois, C. (2023, December 07). Building Autoencoders in Keras. Available online: https://blog.keras.io/building-autoencoders-in-keras.html.
31. Chung, J., Gulcehre, C., Cho, K., and Bengio, Y. (2014). Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv.
32. Long Short-Term Memory10.1162/neco.1997.9.8.1735 ↗
33. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., and Polosukhin, I. (2017, January 4–9). Attention is all you need. Proceedings of the 31st International Conference on Neural Information Processing Systems, Long Beach, CA, USA.
34. Wang, X., Girshick, R., Gupta, A., and He, K. (2018, January 18–23). Non-local neural networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00813 ↗
35. Tan, Z., Wang, M., Xie, J., Chen, Y., and Shi, X. (2018, January 2–7). Deep semantic role labeling with self-attention. Proceedings of the AAAI Conference on Artificial Intelligence, New Orleans, LA, USA.10.1609/aaai.v32i1.11928 ↗
36. Beyond self-attention: External attention using two linear layers for visual tasks
37. Hundman, K., Constantinou, V., Laporte, C., Colwell, I., and Soderstrom, T. (2018, January 19–23). Detecting spacecraft anomalies using lstms and nonparametric dynamic thresholding. Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, London, UK.10.1145/3219819.3219845 ↗
38. Mathur, A.P., and Tippenhauer, N.O. (2016, January 11). SWaT: A water treatment testbed for research and training on ICS security. Proceedings of the 2016 International Workshop on Cyber-Physical Systems for Smart Water Networks (CySWater), Vienna, Austria.10.1109/CySWater.2016.7469060 ↗
39. Ahmed, C.M., Palleti, V.R., and Mathur, A.P. (2017, January 21). WADI: A water distribution testbed for research in the design of secure cyber physical systems. Proceedings of the 3rd International Workshop on Cyber-Physical Systems for Smart Water Networks, Pittsburgh, PA, USA.10.1145/3055366.3055375 ↗
40. Nedelkoski, S., Bogatinovski, J., Mandapati, A.K., Becker, S., Cardoso, J., and Kao, O. (2020, January 28–30). Multi-source distributed system data for ai-powered analytics. Proceedings of the Service-Oriented and Cloud Computing: 8th IFIP WG 2.14 European Conference (ESOCC 2020), Heraklion, Greece.
41. PhysioBank, PhysioToolkit, and PhysioNet: Components of a new research resource for complex physiologic signals10.1161/01.CIR.101.23.e215 ↗
42. The impact of the MIT-BIH arrhythmia database10.1109/51.932724 ↗
43. Boniol, P., Linardi, M., Roncallo, F., and Palpanas, T. (2020, January 20–24). Automated anomaly detection in large sequences. Proceedings of the 2020 IEEE 36th International Conference on Data Engineering (ICDE), Dallas, TX, USA.10.1109/ICDE48307.2020.00182 ↗
44. SAND: Streaming subsequence anomaly detection10.14778/3467861.3467863 ↗
45. Nakamura, T., Imamura, M., Mercer, R., and Keogh, E. (2020, January 17–20). Merlin: Parameter-free discovery of arbitrary length anomalies in massive time series archives. Proceedings of the 2020 IEEE International Conference on Data Mining (ICDM), Sorrento, Italy.10.1109/ICDM50108.2020.00147 ↗
46. Zhao, H., Wang, Y., Duan, J., Huang, C., Cao, D., Tong, Y., Xu, B., Bai, J., Tong, J., and Zhang, Q. (2020, January 17–20). Multivariate time-series anomaly detection via graph attention network. Proceedings of the 2020 IEEE International Conference on Data Mining (ICDM), Sorrento, Italy.10.1109/ICDM50108.2020.00093 ↗