TGAN-AD: Transformer-Based GAN for Anomaly Detection of Time Series Data
Source abstract
Abstract
Anomaly detection on time series data has been successfully used in power grid operation and maintenance, flow detection, fault diagnosis, and other applications. However, anomalies in time series often lack strict definitions and labels, and existing methods often suffer from the need for rigid hypotheses, the inability to handle high-dimensional data, and highly time-consuming calculation costs. Generative Adversarial Networks (GANs) can learn the distribution pattern of normal data, detecting anomalies by comparing the reconstructed normal data with the original data. However, it is difficult for GANs to extract contextual information from time series data. In this paper, we propose a new method, Transformer-based GAN for Anomaly Detection of Time Series Data (TGAN-AD), The transformer-based generators of TGAN-AD can extract contextual features of time series data to prompt the performance. TGAN-AD’s discriminator can also assist in determining abnormal data. Anomaly scores are calculated through both the generator and the discriminator. We have conducted comprehensive experiments on three public datasets. Experimental results show that our TGAN-AD has better performance in anomaly detection than the state-of-the-art anomaly detection techniques, with the highest Recall and F1 values on all datasets. Our experiments also demonstrate the high efficiency of the model and the optimal choice of hyperparameters.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Anomaly Detection Techniques and Applications · Time Series Analysis and Forecasting · Network Security and Intrusion Detection
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
State Key Laboratory of Smart Grid Protection and Control, Nanjing 211106, China
Nari Group Corporation, Nanjing 211106, China
School of Computer Science, Nanjing University of Posts and Telecommunications, Nanjing 210003, China
Bell Honros School, Nanjing University of Posts and Telecommunications, Nanjing 210003, China
Funding
State Key Laboratory of Smart Grid Protection and Control · Research Foundation for Advanced Talents of Nanjing University of Posts and Telecommunications · Jiangsu Project of Social Development
References (38)
Show source-provided references
1. Anomaly detection for IoT time series data: A survey10.1109/JIOT.2019.2958185 ↗
2. Liu, D., Zhao, Y., Xu, H., Sun, Y., Pei, D., Luo, J., Jing, X., and Feng, M. (2015, January 28–30). Opprentice: Towards practical and automatic anomaly detection through machine learning. Proceedings of the 2015 Internet Measurement Conference, Tokyo, Japan.10.1145/2815675.2815679 ↗
3. Shang, Z., Zhang, Y., Zhang, X., Zhao, Y., Cao, Z., and Wang, X. (2021). Time Series Anomaly Detection for KPIs Based on Correlation Analysis and HMM. Appl. Sci., 11.10.3390/app112311353 ↗
4. Lai, K.H., Zha, D., Xu, J., Zhao, Y., Wang, G., and Hu, X. (2022, August 10). Revisiting Time Series Outlier Detection: Definitions and Benchmarks. Available online: https://github.com/datamllab/tods/tree/benchmark.
5. Karadayı, Y., Aydin, M.N., and Öğrenci, A.S. (2020). A hybrid deep learning framework for unsupervised anomaly detection in multivariate spatio-temporal data. Appl. Sci., 10.10.3390/app10155191 ↗
6. Li, Z., Zhao, Y., Han, J., Su, Y., Jiao, R., Wen, X., and Pei, D. (2021, January 14–18). Multivariate Time Series Anomaly Detection and Interpretation using Hierarchical Inter-Metric and Temporal Embedding. Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining, Singapore.10.1145/3447548.3467075 ↗
7. Wu, J., Lee, P.P., Li, Q., Pan, L., and Zhang, J. (2018, January 14–16). CellPAD: Detecting performance anomalies in cellular networks via regression analysis. Proceedings of the 2018 IFIP Networking Conference (IFIP Networking) and Workshops, Zurich, Switzerland.10.23919/IFIPNetworking.2018.8697027 ↗
8. Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., and Bengio, Y. (2014). Generative adversarial nets. Advances in Neural Information Processing Systems, Curran Associates Inc.
9. Chen, L., Li, Y., Deng, X., Liu, Z., Lv, M., and Zhang, H. (2022). Dual Auto-Encoder GAN-Based Anomaly Detection for Industrial Control System. Appl. Sci., 12.10.3390/app12104986 ↗
10. Fu, R., Zhang, Z., and Li, L. (2016, January 11–13). Using LSTM and GRU neural network methods for traffic flow prediction. Proceedings of the 2016 31st Youth Academic Annual Conference of Chinese Association of Automation (YAC), Wuhan, China.10.1109/YAC.2016.7804912 ↗
11. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., and Polosukhin, I. (2017). Attention is all you need. Advances in Neural Information Processing Systems, Curran Associates Inc.
12. Anomaly detection: A survey10.1145/1541880.1541882 ↗
13. Laurikkala, J., Juhola, M., Kentala, E., Lavrac, N., Miksch, S., and Kavsek, B. (2000, January 22). Informal identification of outliers in medical data. Proceedings of the Fifth International Workshop on Intelligent Data Analysis in Medicine and Pharmacology, Berlin, Germany.
14. Pena, E.H., de Assis, M.V., and Proença, M.L. (2013, January 11–15). Anomaly detection using forecasting methods arima and hwds. Proceedings of the 2013 32nd International Conference of the Chilean Computer Science Society (SCCC), Temuco, Chile.10.1109/SCCC.2013.18 ↗
15. Galeano, P., Peña, D., and Tsay, R.S. (2004). Outlier Detection in Multivariate Time Series via Projection Pursuit, Universidad Carlos III de Madrid.
16. Smith, R., Bivens, A., Embrechts, M., Palagiri, C., and Szymanski, B. (2002). Clustering approaches for anomaly based intrusion detection. Intelligent Engineering Systems through Artificial Neural Networks, ASME.
17. Breunig, M.M., Kriegel, H.P., Ng, R.T., and Sander, J. (2000, January 15–18). LOF: Identifying density-based local outliers. Proceedings of the 2000 ACM SIGMOD International Conference on Management of Data, Dallas, TX, USA.10.1145/342009.335388 ↗
18. Chan, P.K., Mahoney, M.V., and Arshad, M.H. (2003). A Machine Learning Approach to Anomaly Detection, Available online: https://repository.lib.fit.edu/bitstream/handle/11141/114/cs-2003-06.pdf?sequence=1.
19. Discovering cluster-based local outliers10.1016/S0167-8655(03)00003-5 ↗
20. Jiang, J.R., Kao, J.B., and Li, Y.L. (2021). Semi-supervised time series anomaly detection based on statistics and deep learning. Appl. Sci., 11.10.3390/app11156698 ↗
21. Serras, J.L., Vinga, S., and Carvalho, A.M. (2021). Outlier Detection for Multivariate Time Series Using Dynamic Bayesian Networks. Appl. Sci., 11.10.3390/app11041955 ↗
22. Credit card fraud detection using deep learning based on auto-encoder and restricted boltzmann machine
23. Variational autoencoder based anomaly detection using reconstruction probability
24. Li, D., Chen, D., Jin, B., Shi, L., Goh, J., and Ng, S.K. (2019, January 17–19). MAD-GAN: Multivariate anomaly detection for time series data with generative adversarial networks. Proceedings of the International Conference on Artificial Neural Networks, Munich, Germany.10.1007/978-3-030-30490-4_56 ↗
25. Bashar, M.A., and Nayak, R. (2020). TAnoGAN: Time Series Anomaly Detection with Generative Adversarial Networks. arXiv.10.1109/SSCI47803.2020.9308512 ↗
26. Geiger, A., Liu, D., Alnegheimish, S., Cuesta-Infante, A., and Veeramachaneni, K. (2020, January 10–13). TadGAN: Time series anomaly detection using generative adversarial networks. Proceedings of the 2020 IEEE International Conference on Big Data (Big Data), Atlanta, GA, USA.10.1109/BigData50022.2020.9378139 ↗
27. Shaw, P., Uszkoreit, J., and Vaswani, A. (2018). Self-attention with relative position representations. arXiv.10.18653/v1/N18-2074 ↗
28. Dai, Z., Yang, Z., Yang, Y., Carbonell, J., Le, Q.V., and Salakhutdinov, R. (2019). Transformer-xl: Attentive language models beyond a fixed-length context. arXiv.10.18653/v1/P19-1285 ↗
29. Dehghani, M., Gouws, S., Vinyals, O., Uszkoreit, J., and Kaiser, Ł. (2018). Universal transformers. arXiv.
30. Wu, N., Green, B., Ben, X., and O’Banion, S. (2020). Deep transformer models for time series forecasting: The influenza prevalence case. arXiv.
31. Wu, S., Xiao, X., Ding, Q., Zhao, P., Ying, W., and Huang, J. (2020, January 6–12). Adversarial Sparse Transformer for Time Series Forecasting. Proceedings of the 34th Conference on Neural Information Processing Systems (NeurIPS 2020), Vancouver, BC, Canada.
32. Zhou, H., Zhang, S., Peng, J., Zhang, S., Li, J., Xiong, H., and Zhang, W. (2021, January 2–9). Informer: Beyond efficient transformer for long sequence time series forecasting. Proceedings of the AAAI Conference on Artificial Intelligence (AAAI-21), Vancouver, BC, Canada.
33. Enhancing the locality and breaking the memory bottleneck of transformer on time series forecasting
34. Shyu, M.L. (2003, January 19–20). A Novel Anomaly Detection Scheme Based on Principal Component Classifier. Proceedings of the IEEE Foundations and New Directions of Data Mining Workshop, In Conjunction with the Third IEEE International Conference on Data Mining (ICDM03), Melbourne, FL, USA.
35. Xu, Y.X., Pang, M., Feng, J., Ting, K.M., Jiang, Y., and Zhou, Z.H. (May, January 29). Reconstruction-based Anomaly Detection with Completely Random Forest. Proceedings of the 2021 SIAM International Conference on Data Mining (SDM), Virtual Event.10.1137/1.9781611976700.15 ↗
36. Long short term memory networks for anomaly detection in time series
37. Rong, Z., Shandong, D., Xin, N., and Shiguang, X. (2018). Feedforward Neural Network for Time Series Anomaly Detection. arXiv.
38. Graph neural network-based anomaly detection in multivariate time series