Machine Learning-Based Anomaly Prediction for Proactive Monitoring in Data Centers: A Case Study on INFN-CNAF
Source abstract
Abstract
Anomaly prediction in time series is crucial for ensuring the stability and security of data centers, especially in scientific contexts such as INFN-CNAF, the National Center for Research and Development in Information and Communication Technology of the National Institute for Nuclear Physics. At INFN-CNAF, large volumes of heterogeneous data critical to international experiments are managed using dedicated monitoring systems. To ensure continuous availability, artificial intelligence solutions are being explored to detect anomalies and predict potential failures proactively. This work presents a machine learning-based approach for automatic anomaly prediction in the operational metrics of INFN-CNAF’s WebDav service. We evaluate several methods, including Long Short-Term Memory, Random Forest, and various neural networks, assessing their Accuracy and sensitivity in distinguishing normal from anomalous behaviors. The results demonstrate the effectiveness of these methods, not only in predicting anomalies but also in pinpointing critical areas within monitored metrics. This contributes to more proactive IT resource monitoring and enhances data center management efficiency.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Anomaly Detection Techniques and Applications · Network Security and Intrusion Detection · Software System Performance and Reliability
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Department of Informatics-Science and Engineering (DISI), University of Bologna, Via Mura Anteo Zamboni 7, 40126 Bologna, Italy
National Center for Research and Development in Information and Communication Technologies of the Italian National Institute for Nuclear Physics (INFN-CNAF), Viale Berti Pichat 6/2, 40127 Bologna, Italy
Funding
Future AI Research (FAIR) project of the National Recovery and Resilience Plan (NRRP) · European Union—NextGenerationEU
References (55)
Show source-provided references
1. Unsupervised anomaly detection in time-series: An extensive evaluation and analysis of state-of-the-art methods10.1016/j.eswa.2024.124922 ↗
2. Deep Learning for Anomaly Detection in Time-Series Data: Review, Analysis, and Guidelines10.1109/ACCESS.2021.3107975 ↗
3. A review on statistical approaches for anomaly detection in DDoS attacks10.1080/19393555.2020.1717019 ↗
4. Knowledge-based anomaly detection: Survey, challenges, and future directions10.1016/j.engappai.2024.108996 ↗
5. Machine Learning for Anomaly Detection: A Systematic Review10.1109/ACCESS.2021.3083060 ↗
6. Time Series Based Data Explorer and Stream Analysis for Anomaly Prediction
7. Deep Learning for Time Series Forecasting: Tutorial and Literature Survey10.1145/3533382 ↗
8. Deep Learning for Time Series Forecasting: A Survey10.1089/big.2020.0159 ↗
9. Hybrid physics-embedded recurrent neural networks for fault diagnosis under time-varying conditions based on multivariate proprioceptive signals10.1016/j.aei.2024.102851 ↗
10. Hierarchical temporal memory and recurrent neural networks for time series prediction: An empirical validation and reduction to multilayer perceptrons10.1016/j.neucom.2018.09.098 ↗
11. Orozco, B.P. (2020). Recurrent Neural Networks for Time Series Prediction. [Ph.D. Thesis, University of Oxford].
12. Cost sensitive active learning using bidirectional gated recurrent neural networks for imbalanced fault diagnosis10.1016/j.neucom.2020.04.075 ↗
13. Kalinina, I., Bidyuk, P., and Gozhyj, A. (2022, January 10–12). Construction of Forecast Models based on Bayesian Structural Time Series. Proceedings of the 17th IEEE International Conference on Computer Sciences and Information Technologies, CSIT 2022, Lviv, Ukraine.10.1109/CSIT56902.2022.10000484 ↗
14. Fault detection and identification using Bayesian recurrent neural networks10.1016/j.compchemeng.2020.106991 ↗
15. Wastewater treatment monitoring: Fault detection in sensors using transductive learning and improved reinforcement learning10.1016/j.eswa.2024.125805 ↗
16. Multi-label domain adversarial reinforcement learning for unsupervised compound fault recognition10.1016/j.ress.2024.110638 ↗
17. 3-D Seismic Fault Detection Using Recurrent Convolutional Neural Networks With Compound Loss
18. Deep Convolutional and LSTM Recurrent Neural Networks for Rolling Bearing Fault Diagnosis Under Strong Noises and Variable Loads10.1109/ACCESS.2020.2985617 ↗
19. Spatial-Temporal Recurrent Graph Neural Networks for Fault Diagnostics in Power Distribution Systems10.1109/ACCESS.2023.3273292 ↗
20. Compound-Fault Diagnosis of Integrated Energy Systems Based on Graph Embedded Recurrent Neural Networks10.1109/TII.2023.3306935 ↗
21. Recurrent neural network NARX for distributed fault detection in wireless sensor networks10.1504/IJSNET.2021.118488 ↗
22. Hagemann, T., and Katsarou, K. (2020, January 18–20). A Systematic Review on Anomaly Detection for Cloud Computing Environments. Proceedings of the AICCC 2020: 2020 3rd Artificial Intelligence and Cloud Computing Conference, ACM, Kyoto, Japan.10.1145/3442536.3442550 ↗
23. Machine Learning and Deep Learning Techniques for Distributed Denial of Service Anomaly Detection in Software Defined Networks - Current Research Solutions10.1109/ACCESS.2024.3360868 ↗
24. Feng, Y., Yang, Z., Sun, Q., and Liu, Y. (2024). SEDAT: A Stacked Ensemble Learning-Based Detection Model for Multiscale Network Attacks. Electronics, 13.10.3390/electronics13152953 ↗
25. Multivariate time series ensemble model for load prediction on hosts using anomaly detection techniques10.1007/s10586-024-04517-0 ↗
26. An efficient GAN-based predictive framework for multivariate time series anomaly prediction in cloud data centers10.1007/s11227-023-05534-3 ↗
27. GMAT-DU: Traffic Anomaly Prediction With Fine Spatiotemporal Granularity in Sparse Data10.1109/TITS.2023.3249409 ↗
28. Unmanned Aerial Vehicle Flight Data Anomaly Detection and Recovery Prediction Based on Spatio-Temporal Correlation10.1109/TR.2021.3134369 ↗
29. Multi-label classification for simultaneous fault diagnosis of marine machinery: A comparative study10.1016/j.oceaneng.2021.109723 ↗
30. Actual bearing compound fault diagnosis based on active learning and decoupling attentional residual network10.1016/j.measurement.2020.108500 ↗
31. Morganti, L., Cavalli, A., Falabella, A., Fattibene, E., Fornari, F., Prosperini, A., Rendina, A., and Sapunenko, V. (2022, January 6–13). Large Scale Data Handling experience at INFN-CNAF Data Center. Proceedings of the 41st International Conference on High Energy physics—PoS (ICHEP2022), Bologna, Italy.10.22323/1.414.0205 ↗
32. Agostini, F., Bassi, L., Gasparetto, J., Giacomini, F., Miccoli, R., and Vianello, E. (2024, January 24–29). Enhancing StoRM WebDAV data transfer performance with a new deployment architecture behind NGINX reverse proxy. Proceedings of the International Symposium on Grids and Clouds (ISGC2024), Taipei, Taiwan.
33. Wei, Y., Jang-Jaccard, J., Xu, W., Sabrina, F., Camtepe, S., and Boulic, M. (2022). LSTM-Autoencoder based Anomaly Detection for Indoor Air Quality Time Series Data. arXiv.10.1109/JSEN.2022.3230361 ↗
34. Githinji, S., and Maina, C.W. (2023, January 20–22). Anomaly Detection on Time Series Sensor Data Using Deep LSTM-Autoencoder. Proceedings of the IEEE AFRICON 2023, Nairobi, Kenya.10.1109/AFRICON55910.2023.10293676 ↗
35. Ho, M., Tran-Van, N., and Le, K. (2024, January 15–16). A Multi-Input Bi-LSTM Autoencoder Model with Wavelet Transform for Air Quality Prediction. Proceedings of the International Conference on Multimedia Analysis and Pattern Recognition, MAPR 2024, Da Nang, Vietnam.10.1109/MAPR63514.2024.10660818 ↗
36. Detecting Anomalies in Time Series Using Kernel Density Approaches10.1109/ACCESS.2024.3371891 ↗
37. Lindstrom, M.R., Jung, H., and Larocque, D. (2020). Functional Kernel Density Estimation: Point and Fourier Approaches to Time Series Anomaly Detection. Entropy, 22.10.3390/e22121363 ↗
38. Liu, F.T., Ting, K.M., and Zhou, Z. (2008, January 15–19). Isolation Forest. Proceedings of the 8th IEEE International Conference on Data Mining (ICDM 2008), Pisa, Italy.10.1109/ICDM.2008.17 ↗
39. Petkovski, A., and Shehu, V. (2023, January 6–10). Anomaly Detection on Univariate Sensing Time Series Data for Smart Aquaculture Using K-Means, Isolation Forest, and Local Outlier Factor. Proceedings of the 12th Mediterranean Conference on Embedded Computing, MECO 2023, Budva, Montenegro.10.1109/MECO58584.2023.10154991 ↗
40. Deep Isolation Forest for Anomaly Detection10.1109/TKDE.2023.3270293 ↗
41. Fusion-based anomaly detection system using modified isolation forest for internet of things10.1007/s12652-022-04393-9 ↗
42. Time series anomaly detection via clustering-based representation10.1007/s12530-023-09543-8 ↗
43. Subsequence Time Series Clustering-Based Unsupervised Approach for Anomaly Detection of Axial Piston Pumps10.1109/TIM.2023.3264045 ↗
44. Chang, S. (2023, January 1–10). DGNN: Dynamic Graph Neural Networks for Anomaly Detection in Multivariate Time Series. Proceedings of the 35th International Conference on Software Engineering and Knowledge Engineering, SEKE 2023, Pittsburgh, PA, USA.10.18293/SEKE2023-094 ↗
45. EGNN: Energy-efficient anomaly detection for IoT Multivar. Time Ser. Data Using Graph Neural Network10.1016/j.future.2023.09.028 ↗
46. Guo, Z., Kong, W., and Liu, Y. (July, January 30). Grouped Graph Neural Networks for Anomaly Detection in Time Series. Proceedings of the International Joint Conference on Neural Networks, IJCNN 2024, Yokohama, Japan.10.1109/IJCNN60899.2024.10649927 ↗
47. Anomaly Detection in Time Series: A Comprehensive Evaluation10.14778/3538598.3538602 ↗
48. An Experimental Evaluation of Anomaly Detection in Time Series10.14778/3632093.3632110 ↗
49. The perceptron: A probabilistic model for information storage and organization in the brain10.1037/h0042519 ↗
50. Random Forests10.1023/A:1010933404324 ↗
51. Long Short-Term Memory10.1162/neco.1997.9.8.1735 ↗
52. Learning to Forget: Continual Prediction with LSTM10.1162/089976600300015015 ↗
53. Kingma, D.P., and Ba, J. (2017). Adam: A Method for Stochastic Optimization. arXiv.
54. F-Measure as the Error Function to Train Neural Networks
55. Simonyan, K., Vedaldi, A., and Zisserman, A. (2014). Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps. arXiv.