Mutation-Based Multivariate Time-Series Anomaly Generation on Latent Space with an Attention-Based Variational Recurrent Neural Network for Robust Anomaly Detection in an Industrial Control System
Source abstract
Abstract
Anomaly detection involves identifying data that deviates from normal patterns. Two primary strategies are used: one-class classification and binary classification. In Industrial Control Systems (ICS), where anomalies can cause significant damage, timely and accurate detection is essential, often requiring analysis of time-series data. One-class classification is commonly used but tends to have a high false alarm rate. To address this, binary classification is explored, which can better differentiate between normal and anomalous data, though it struggles with class imbalance in ICS datasets. This paper proposes a mutation-based technique for generating ICS time-series anomalies. The method maps ICS time-series data into a latent space using a variational recurrent autoencoder, applies mutation operations, and reconstructs the time-series, introducing plausible anomalies that reflect multivariate correlations. Evaluations of ICS datasets show that these synthetic anomalies are visually and statistically credible. Training a binary classifier on data augmented with these anomalies effectively mitigates the class imbalance problem.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Anomaly Detection Techniques and Applications · Time Series Analysis and Forecasting · Fault Detection and Control Systems
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Department of Computer Engineering (Smart Security), Gachon University, Seongnam-daero 1342, Seongnam-si 13119, Republic of Korea
Electronics and Telecommunications Research Institute (ETRI), Daejeon 34129, Republic of Korea
Funding
Korea Government (MSIT)
References (51)
Show source-provided references
1. CoDetect: Financial Fraud Detection with Anomaly Feature Detection10.1109/ACCESS.2018.2816564 ↗
2. Efficient Cyber Attack Detection in Industrial Control Systems Using Lightweight Neural Networks and PCA10.1109/TDSC.2021.3050101 ↗
3. Elsayed, M.S., Le-Khac, N.A., Dev, S., and Jurcut, A.D. (2020, January 16–20). Network Anomaly Detection Using LSTM Based Autoencoder. In Proceddings of the ACM Symposium on QoS and Security for Wireless and Mobile Networks, Alicante, Spain.
4. Shen, L., Li, Z., and Kwok, J.T. (2020, January 6–12). Timeseries anomaly detection using temporal hierarchical one-class network. Proceedings of the Advances in Neural Information Processing Systems, Online.
5. Xu, H., Wang, Y., Jian, S., Liao, Q., Wang, Y., and Pang, G. (2024). Calibrated One-class Classification for Unsupervised Time Series Anomaly Detection. IEEE Trans. Knowl. Data Eng., 1–14.10.1109/TKDE.2024.3393996 ↗
6. Ghrib, Z., Jaziri, R., and Romdhane, R. (2020, January 19–24). Hybrid approach for Anomaly Detection in Time Series Data. Proceedings of the International Joint Conference on Neural Networks, Glasgow, UK.10.1109/IJCNN48605.2020.9207013 ↗
7. Primus, P., Haunschmid, V., Praher, P., and Widmer, G. (2020). Anomalous Sound Detection as a Simple Binary Classification Problem with Careful Selection of Proxy Outlier Examples. arXiv.
8. One-class classification of point patterns of extremes
9. Lee, J.H., Ji, I.H., Jeon, S.H., and Seo, J.T. (2023). Generating ICS Anomaly Data Reflecting Cyber-Attack Based on Systematic Sampling and Linear Regression. Sensors, 23.10.3390/s23249855 ↗
10. Kingma, D.P., and Welling, M. (2013). Auto-encoding variational bayes. In Proceedings of the 2nd International Conference on Learning Representations. arXiv.
11. Zalewski, M. (2024, August 31). American Fuzzy Lop. Available online: http://lcamtuf.coredump.cx/afl.
12. Fioraldi, A., Maier, D., Eißfeldt, H., and Heuse, M. (2020, January 11). AFL++: Combining incremental steps of fuzzing research. Proceedings of the WOOT 2020—14th USENIX Workshop on Offensive Technologies, Online.
13. A generalized probabilistic monitoring model with both random and sequential data10.1016/j.automatica.2022.110468 ↗
14. An Unsupervised Fault Detection and Diagnosis with Distribution Dissimilarity and Lasso Penalty10.1109/TCST.2023.3330443 ↗
15. MoniNet With Concurrent Analytics of Temporal and Spatial Information for Fault Detection in Industrial Processes10.1109/TCYB.2021.3050398 ↗
16. Dissimilarity-based representations for one-class classification on time series10.1016/j.patcog.2019.107122 ↗
17. Gjorgiev, L., and Gievska, S. (2020, January 8–10). Time Series Anomaly Detection with Variational Autoencoder Using Mahalanobis Distance. Proceedings of the Communications in Computer and Information Science, Virtual Event.10.1007/978-3-030-62098-1_4 ↗
18. Long Short-Term Memory10.1162/neco.1997.9.8.1735 ↗
19. Support-Vector Networks10.1007/BF00994018 ↗
20. He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27–30). Deep residual learning for image recognition. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.10.1109/CVPR.2016.90 ↗
21. Design and Development of RNN Anomaly Detection Model for IoT Networks10.1109/ACCESS.2022.3176317 ↗
22. Cho, K., Merriënboer, B.V., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., and Bengio, Y. (2014, January 25–29). Learning phrase representations using RNN encoder–decoder for statistical machine translation. Proceedings of the EMNLP 2014 Conference on Empirical Methods in Natural Language Processing, Doha, Qatar.10.3115/v1/D14-1179 ↗
23. Han, H., Wang, W.Y., and Mao, B.H. (2005, January 26–28). Borderline-SMOTE: A new over-sampling method in imbalanced data sets learning. Proceedings of the Lecture Notes in Computer Science, Istanbul, Turkey.10.1007/11538059_91 ↗
24. Gundersen, K., Alendal, G., Oleynik, A., and Blaser, N. (2020). Binary time series classification with bayesian convolutional neural networks when monitoring for marine gas discharges. Algorithms, 13.10.3390/a13060145 ↗
25. Gal, Y., and Ghahramani, Z. (2016, January 19–24). Dropout as a Bayesian approximation: Representing model uncertainty in deep learning. Proceedings of the 33rd International Conference on Machine Learning, ICML 2016, New York, NY, USA.
26. Anomaly Detection in Quasi-Periodic Time Series Based on Automatic Data Segmentation and Attentional LSTM-CNN10.1109/TKDE.2020.3014806 ↗
27. Forestier, G., Petitjean, F., Dau, H.A., Webb, G.I., and Keogh, E. (November, January USA). Generating synthetic time series to augment sparse datasets. Proceedings of the IEEE International Conference on Data Mining, ICDM, New Orleans, LA.
28. Goodfellow, I.J., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., and Bengio, Y. (2014, January 8–13). Generative adversarial nets. Proceedings of the Advances in Neural Information Processing Systems, Montreal, QC, Canada.
29. Zhang, C., Kuppannagari, S.R., Kannan, R., and Prasanna, V.K. (2018, January 29–31). Generative Adversarial Network for Synthetic Time Series Data Generation in Smart Grids. Proceedings of the 2018 IEEE International Conference on Communications, Control, and Computing Technologies for Smart Grids, SmartGridComm 2018, Aalborg, Denmark.10.1109/SmartGridComm.2018.8587464 ↗
30. Zhou, L., Poli, M., Xu, W., Massaroli, S., and Ermon, S. (December, January 30). Deep Latent State Space Models for Time-Series Generation. Proceedings of the Machine Learning Research, Seattle, WA, USA.
31. Supervised Anomaly Detection via Conditional Generative Adversarial Network and Ensemble Active Learning10.1109/TPAMI.2022.3225476 ↗
32. Salem, M., Taheri, S., and Yuan, J.S. (2018, January 8–10). Anomaly Generation Using Generative Adversarial Networks in Host-Based Intrusion Detection. Proceedings of the 2018 9th IEEE Annual Ubiquitous Computing, Electronics and Mobile Communication Conference, UEMCON 2018, New York, NY, USA.10.1109/UEMCON.2018.8796769 ↗
33. Zhu, J.Y., Park, T., Isola, P., and Efros, A.A. (2017, January 22–29). Unpaired Image-to-Image Translation Using Cycle-Consistent Adversarial Networks. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.10.1109/ICCV.2017.244 ↗
34. Pourreza, M., Mohammadi, B., Khaki, M., Bouindour, S., Snoussi, H., and Sabokrou, M. (2021, January 5–9). G2D: Generate to detect anomaly. Proceedings of the 2021 IEEE Winter Conference on Applications of Computer Vision, WACV 2021, Waikoloa, HI, USA.10.1109/WACV48630.2021.00205 ↗
35. Unsupervised industrial image ensemble anomaly detection based on object pseudo-anomaly generation and normal image feature combination enhancement10.1016/j.cie.2023.109337 ↗
36. Lin, Y., Deng, H., and Li, X. (2024). FastLogAD: Log Anomaly Detection with Mask-Guided Pseudo Anomaly Generation and Discrimination. arXiv.
37. Hu, T., Zhang, J., Yi, R., Du, Y., Chen, X., Liu, L., Wang, Y., and Wang, C. (2024, January 20–27). AnomalyDiffusion: Few-Shot Anomaly Image Generation with Diffusion Model. Proceedings of the AAAI Conference on Artificial Intelligence, Vancouver, BC, Canada.10.1609/aaai.v38i8.28696 ↗
38. Deep Learning for Anomaly Detection in Time-Series Data: Review, Analysis, and Guidelines10.1109/ACCESS.2021.3107975 ↗
39. Convolutional neural network-based data anomaly detection method using multiple information for structural health monitoring10.1002/stc.2296 ↗
40. Computer vision and deep learning–based data anomaly detection method for structural health monitoring10.1177/1475921718757405 ↗
41. Boniol, P., Paparrizos, J., and Palpanas, T. (2023, January 28–31). New Trends in Time-Series Anomaly Detection. Proceedings of the Advances in Database Technology-EDBT, Ioannina, Greece.
42. Mathur, A.P., and Tippenhauer, N.O. (2016, January 11). SWaT: A water treatment testbed for research and training on ICS security. Proceedings of the 2016 International Workshop on Cyber-physical Systems for Smart Water Networks, CySWater 2016, Vienna, Austria.10.1109/CySWater.2016.7469060 ↗
43. Shin, H.K., Lee, W., Yun, J.H., and Kim, H.C. (2020, January 10). HAI 1.0: HIL-based augmented ICS security dataset. Proceedings of the CSET 2020-13th USENIX Workshop on Cyber Security Experimentation and Test, Online.
44. Jeon, S., and Seo, J.T. (2023). A Synthetic Time-Series Generation Using a Variational Recurrent Autoencoder with an Attention Mechanism in an Industrial Control System. Sensors, 24.10.3390/s24010128 ↗
45. Fabius, O., and van Amersfoort, J.R. (2015, January 7–9). Variational recurrent auto-encoders. Proceedings of the 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA.
46. Chung, J., Kastner, K., Dinh, L., Goel, K., Courville, A., and Bengio, Y. (2015, January 7–12). A recurrent latent variable model for sequential data. Proceedings of the Advances in Neural Information Processing Systems, Montreal, QC, Canada.
47. Kingma, D.P., and Ba, J.L. (2015, January 7–9). Adam: A method for stochastic optimization. Proceedings of the 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA.
48. Adaptive subgradient methods for online learning and stochastic optimization
49. Hinton, G.E., Srivastava, N., and Swersky, K. (2024, August 31). Neural Networks for Machine Learning Lecture 6a Overview of Mini-Batch Gradient Descent. Available online: https://www.cs.toronto.edu/~tijmen/csc321/slides/lecture_slides_lec6.pdf.
50. Bro, R., and Smilde, A.K. (2024, August 31). Principal Component Analysis. Available online: https://doi.org/10.1039/c3ay41907j.10.1039/C3AY41907J ↗
51. Maaten, L.V.D., and Hinton, G. (2008). Visualizing data using t-SNE. J. Mach. Learn. Res., 9, Available online: https://jmlr.org/papers/v9/vandermaaten08a.html.