Time Series Anomaly Detection Using Signal Processing and Deep Learning
Source abstract
Abstract
In this paper, we propose a two-step approach for time series anomaly detection that combines signal processing techniques with deep learning methods. In the first step, we apply a bandpass filter to the time series data to reduce noise and highlight relevant frequency components, which enhances the signals in them. In the second step, we utilize a Functional Neural Network Autoencoder for anomaly detection, leveraging its ability to capture non-linear temporal relationships in the data. By learning a compact latent representation and remapping the filtered time series, the Autoencoder effectively identifies deviations from normal patterns, allowing us to detect anomalies. Our experiments on several benchmark datasets demonstrate that bandpass filtering consistently improves the performance of deep learning methods, including the Functional Neural Network Autoencoder, by refining the input data. Our proposed approach achieves superior performance of up to 20% in detecting anomalies, particularly in a time series with intricate structures, highlighting its potential for practical applications in multiple domains.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Anomaly Detection Techniques and Applications · Time Series Analysis and Forecasting · Digital Media Forensic Detection
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Industrial AI Lab, Hitachi America, Ltd., R&D, Santa Clara, CA 95054, USA
References (62)
Show source-provided references
1. Darban, Z.Z., Webb, G.I., Pan, S., Aggarwal, C.C., and Salehi, M. (2022). Deep Learning for Time Series Anomaly Detection: A Survey. arXiv.
2. Enhancing Sound-Based Anomaly Detection Using Deep Denoising Autoencoder10.1109/ACCESS.2024.3414435 ↗
3. Anomaly Detection for Industrial Control System Based on Autoencoder Neural Network10.1155/2020/8897926 ↗
4. Lee, X.Y., Kumar, A., Vidyaratne, L., Rao, A.R., Farahat, A., and Gupta, C. (2023, January 5–7). An ensemble of convolution-based methods for fault detection using vibration signals. Proceedings of the 2023 IEEE International Conference on Prognostics and Health Management (ICPHM), Montreal, QC, Canada.10.1109/ICPHM57936.2023.10194112 ↗
5. Accurate Detection of Bearing Faults Using Difference Visibility Graph and Bi-Directional Long Short-Term Memory Network Classifier10.1109/TIA.2022.3167658 ↗
6. Abdallah, M., Joung, B.G., Lee, W.J., Mousoulis, C., Sutherland, J.W., and Bagchi, S. (2022). Anomaly Detection and Inter-Sensor Transfer Learning on Smart Manufacturing Datasets. Sensors, 23.10.3390/s23010486 ↗
7. Kozitsin, V.O., Katser, I.D., and Lakontsev, D. (2021). Online Forecasting and Anomaly Detection Based on the ARIMA Model. Appl. Sci., 11.10.3390/app11073194 ↗
8. Yang, Y., Zhang, C., Zhou, T., Wen, Q., and Sun, L. (2023, January 6–10). DCdetector: Dual Attention Contrastive Representation Learning for Time Series Anomaly Detection. Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, Long Beach, CA, USA.10.1145/3580305.3599295 ↗
9. Xu, J., Wu, H., Wang, J., and Long, M. (2021). Anomaly Transformer: Time Series Anomaly Detection with Association Discrepancy. arXiv.
10. Wagner, D., Michels, T., Schulz, F.C.F., Nair, A., Rudolph, M.R., and Kloft, M. (2023). TimeSeAD: Benchmarking Deep Multivariate Time-Series Anomaly Detection. Trans. Mach. Learn. Res., 2023.
11. Anomaly Detection Based on Convolutional Recurrent Autoencoder for IoT Time Series10.1109/TSMC.2020.2968516 ↗
12. LSTM-Autoencoder-Based Anomaly Detection for Indoor Air Quality Time-Series Data10.1109/JSEN.2022.3230361 ↗
13. TranAD: Deep Transformer Networks for Anomaly Detection in Multivariate Time Series Data10.14778/3514061.3514067 ↗
14. A Survey on Graph Neural Networks for Time Series: Forecasting, Classification, Imputation, and Anomaly Detection10.1109/TPAMI.2024.3443141 ↗
15. A Comprehensive Survey of Deep Transfer Learning for Anomaly Detection in Industrial Time Series: Methods, Applications, and Directions10.1109/ACCESS.2023.3349132 ↗
16. Anomaly Detection in Time Series: A Comprehensive Evaluation10.14778/3538598.3538602 ↗
17. Kim, B., Alawami, M.A., Kim, E., Oh, S., Park, J.H., and Kim, H. (2023). A Comparative Study of Time Series Anomaly Detection Models for Industrial Control Systems. Sensors, 23.10.3390/s23031310 ↗
18. Nonlinear Functional Modeling Using Neural Networks10.1080/10618600.2023.2165498 ↗
19. Rao, A.R., Wang, H., and Gupta, C. (2022, January 17–20). Functional approach for Two Way Dimension Reduction in Time Series. Proceedings of the 2022 IEEE International Conference on Big Data (Big Data), Osaka, Japan.10.1109/BigData55660.2022.10020482 ↗
20. Zhang, C., Zhou, T., Wen, Q., and Sun, L. (2022, January 17–21). TFAD: A Decomposition Time Series Anomaly Detection Architecture with Time-Frequency Analysis. Proceedings of the 31st ACM International Conference on Information & Knowledge Management, Atlanta, GA, USA.10.1145/3511808.3557470 ↗
21. Giovannelli, A., Lippi, M., and Proietti, T. (2023). Band-Pass Filtering with High-Dimensional Time Series. SSRN Electron. J.10.2139/ssrn.4480271 ↗
22. Yaacob, A.H., Tan, I.K., Chien, S.F., and Tan, H.K. (2010, January 26–28). ARIMA Based Network Anomaly Detection. Proceedings of the 2010 Second International Conference on Communication Software and Networks, Singapore.10.1109/ICCSN.2010.55 ↗
23. Pena, E.H.M., de Assis, M.V.O., and Proença, M.L. (2013, January 11–15). Anomaly Detection Using Forecasting Methods ARIMA and HWDS. Proceedings of the 2013 32nd International Conference of the Chilean Computer Science Society (SCCC), Temuco, Chile.10.1109/SCCC.2013.18 ↗
24. Reducing the dimensionality of data with neural networks10.1126/science.1127647 ↗
25. Kingma, D.P. (2013). Auto-encoding variational bayes. arXiv.
26. VAEAT: Variational AutoeEncoder with adversarial training for multivariate time series anomaly detection10.1016/j.ins.2024.120852 ↗
27. Vincent, P., Larochelle, H., Bengio, Y., and Manzagol, P.A. (2008, January 5–9). Extracting and composing robust features with denoising autoencoders. Proceedings of the 25th International Conference on Machine Learning (ICML ’08), Helsinki, Finland.10.1145/1390156.1390294 ↗
28. Temporal convolutional autoencoder for unsupervised anomaly detection in time series10.1016/j.asoc.2021.107751 ↗
29. Malhotra, P., Ramakrishnan, A., Anand, G., Vig, L., Agarwal, P., and Shroff, G. (2016). LSTM-based encoder-decoder for multi-sensor anomaly detection. arXiv.
30. Detecting anomalies in time series data via a deep learning algorithm combining wavelets, neural networks and Hilbert transform10.1016/j.eswa.2017.04.028 ↗
31. Oppenheim, A.V., and Schafer, R.W. (2009). Discrete-Time Signal Processing, Pearson. [3rd ed.].
32. Meyer, Y. (1993). Wavelets: Algorithms and Applications, SIAM.
33. Golgowski, M., and Osowski, S. (2020, January 16–19). Anomaly detection in ECG using wavelet transformation. Proceedings of the 2020 IEEE 21st International Conference on Computational Problems of Electrical Engineering (CPEE), Online Conference, Poland.10.1109/CPEE50798.2020.9238709 ↗
34. Shang, L., Zhang, Z., Tang, F., Cao, Q., Pan, H., and Lin, Z. (2023). CNN-LSTM Hybrid Model to Promote Signal Processing of Ultrasonic Guided Lamb Waves for Damage Detection in Metallic Pipelines. Sensors, 23.10.20944/preprints202307.0929.v1 ↗
35. Lu, Y.X., Jin, X.B., Chen, J., Liu, D.J., and Geng, G.G. (2024). F-SE-LSTM: A Time Series Anomaly Detection Method with Frequency Domain Information. arXiv.
36. Regularizing autoencoders with wavelet transform for sequence anomaly detection10.1016/j.patcog.2022.109084 ↗
37. Kokoszka, P., and Reimherr, M.L. (2017). Introduction to Functional Data Analysis, Chapman and Hall/CRC.10.1201/9781315117416 ↗
38. Nonlinear subspace clustering by functional link neural networks10.1016/j.asoc.2024.112303 ↗
39. Wang, Q., Zheng, S., Farahat, A.K., Serita, S., and Gupta, C. (2019, January 17–20). Remaining Useful Life Estimation Using Functional Data Analysis. Proceedings of the 2019 IEEE International Conference on Prognostics and Health Management (ICPHM), San Francisco, CA, USA.10.1109/ICPHM.2019.8819420 ↗
40. Hsieh, T.Y., Sun, Y., Wang, S., and Honavar, V.G. (May, January 29). Functional Autoencoders for Functional Data Representation Learning. Proceedings of the SDM, Virtual Event.
41. Shi, L., Shen, L., Zakharov, Y.V., and de Lamare, R.C. (2023, January 4–8). Widely Linear Complex-Valued Spline-Based Algorithm for Nonlinear Filtering. Proceedings of the 2023 31st European Signal Processing Conference (EUSIPCO), Helsinki, Finland.10.23919/EUSIPCO58844.2023.10290053 ↗
42. Feng, C., and Tian, P. (2021, January 14–18). Time Series Anomaly Detection for Cyber-physical Systems via Neural System Identification and Bayesian Filtering. Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining, Virtual Event, Singapore.10.1145/3447548.3467137 ↗
43. An Improved Robust Kernel Adaptive Filtering Method for Time-Series Prediction10.1109/JSEN.2023.3300416 ↗
44. Robust kernel adaptive filtering for nonlinear time series prediction10.1016/j.sigpro.2023.109090 ↗
45. Use of Savitzky-Golay Filters to Minimize Multi-temporal Data Anomaly in Land use Land cover mapping
46. STL: A seasonal-trend decomposition
47. A multiscale time-series decomposition learning for crude oil price forecasting10.1016/j.eneco.2024.107733 ↗
48. Vibration signal demodulation and bearing fault detection: A clustering-based segmentation method10.1177/0954406213514960 ↗
49. Selamat, N.A., and Ali, S.H.M. (2021, January 1–3). A Novel Approach of Chewing Detection based on Temporalis Muscle Movement using Proximity Sensor for Diet Monitoring. Proceedings of the 2020 IEEE-EMBS Conference on Biomedical Engineering and Sciences (IECBES), Langkawi Island, Malaysia.10.1109/IECBES48179.2021.9398736 ↗
50. Timeseries anomaly detection using temporal hierarchical one-class network
51. Podder, P., Hasan, M.M., Islam, M.R., and Sayeed, M. (2020). Design and Implementation of Butterworth, Chebyshev-I and Elliptic Filter for Speech Signal Analysis. arXiv.
52. Hochreiter, S. (1997). Long Short-term Memory. Neural Computation, MIT-Press.10.1162/neco.1997.9.8.1735 ↗
53. Malhotra, P., Vig, L., Shroff, G.M., and Agarwal, P. (2015, January 22–24). Long Short Term Memory Networks for Anomaly Detection in Time Series. Proceedings of the European Symposium on Artificial Neural Networks, Bruges, Belgium.
54. Modern non-linear function-on-function regression10.1007/s11222-023-10299-z ↗
55. Xu, H., Feng, Y., Chen, J., Wang, Z., Qiao, H., Chen, W., Zhao, N., Li, Z., Bu, J., and Li, Z. (2018, January 23–27). Unsupervised Anomaly Detection via Variational Auto-Encoder for Seasonal KPIs in Web Applications. Proceedings of the 2018 World Wide Web Conference on World Wide Web—WWW ’18, Lyon, France.10.1145/3178876.3185996 ↗
56. Su, Y., Zhao, Y., Niu, C., Liu, R., Sun, W., and Pei, D. (2019, January 4–8). Robust Anomaly Detection for Multivariate Time Series through Stochastic Recurrent Neural Network. Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, KDD ’19, Anchorage, AK, USA.10.1145/3292500.3330672 ↗
57. Hundman, K., Constantinou, V., Laporte, C., Colwell, I., and Söderström, T. (2018, January 19–23). Detecting Spacecraft Anomalies Using LSTMs and Nonparametric Dynamic Thresholding. Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, London, UK.10.1145/3219819.3219845 ↗
58. Abdulaal, A., Liu, Z., and Lancewicki, T. (2021, January 14–18). Practical Approach to Asynchronous Multivariate Time Series Anomaly Detection and Localization. Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining, KDD ’21, Virtual Event, Singapore.10.1145/3447548.3467174 ↗
59. Multivariate time series dataset for space weather data analytics10.1038/s41597-020-0548-x ↗
60. Mathur, A.P., and Tippenhauer, N.O. (2016, January 11). SWaT: A water treatment testbed for research and training on ICS security. Proceedings of the 2016 International Workshop on Cyber-physical Systems for Smart Water Networks (CySWater), Vienna, Austria.10.1109/CySWater.2016.7469060 ↗
61. Ahmed, C., Palleti, V., and Mathur, A. (2017, January 21). WADI: A water distribution testbed for research in the design of secure cyber physical systems. Proceedings of the 3rd International Workshop on Cyber-Physical Systems for Smart Water Networks, Pittsburgh, PA, USA.10.1145/3055366.3055375 ↗
62. Lai, K.H., Zha, D., Xu, J., Zhao, Y., Wang, G., and Hu, X. (2021, January 6–14). Revisiting Time Series Outlier Detection: Definitions and Benchmarks. Proceedings of the Thirty-Fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 1), Virtual.