A Real-Time Semi-Supervised Log Anomaly Detection Framework for ALICE O2 Facilities
Source abstract
Abstract
The ALICE (A Large Ion Collider Experiment) detector at the Large Hadron Collider (LHC), operated by the European Organization for Nuclear Research (CERN), is dedicated to heavy-ion collisions. Within ALICE, the application logs of the online computing systems are consolidated through a logging system known as Infologger, which integrates data from various sources. To identify potential anomalies, shifters in the control room manually review logs for anomalies, which require significant expertise and pose challenges due to the frequent onboarding of new personnel. To address this issue, we propose a real-time semi-supervised log anomaly detection framework designed to automatically detect anomalies in ALICE operations. The framework leverages BERTopic, a topic modeling technique, to provide real-time insights for incoming log messages for shifters. This includes an analytical dashboard that represents the anomaly status in log messages, facilitating informative monitoring for shifters. Through evaluation, including Infologger and BGL (BlueGene/L supercomputer), we analyze the effects of word embeddings, clustering algorithms, and HDBSCAN hyperparameters on model performance. The result demonstrates that the BERTopic can enhance the log anomaly detection process over traditional topic models, achieving remarkable performance metrics and attaining F1-scores of 0.957 and 0.958 for the InfoLogger and BGL datasets, respectively, even without the preprocessing technique.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Software System Performance and Reliability · Anomaly Detection Techniques and Applications · Network Security and Intrusion Detection
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Department of Computer Engineering, King Mongkut’s University of Technology Thonburi, Bangkok 10140, Thailand
Experimental Physics Department, European Organization for Nuclear Research (CERN), 1211 Geneva, Switzerland
Funding
NSRF via the Program Management Unit for Human Resources & Institutional Development, Research and Innovation · National Science and Technology Development Agency
References (47)
Show source-provided references
1. The ALICE Collaboration, Aamodt, K., Abrahantes, A., Achenbach, R., Acounis, S., Adamova, D., Adler, C., Aggarwal, M., Agnese, F., and Aglieri Rinella, G. (2008). The ALICE experiment at the CERN LHC. J. Instrum., 3, S08002.10.1088/1748-0221/3/08/S08002 ↗
2. Buncic, P., Krzewicki, M., and Vande Vyvre, P. (2015). Technical Design Report for the Upgrade of the Online-Offline Computing System, CERN. Technical Report CERN-LHCC-2015-006.
3. Liu, F.T., Ting, K.M., and Zhou, Z.H. (2008, January 15–19). Isolation Forest. Proceedings of the 2008 8th IEEE International Conference on Data Mining, Pisa, Italy.10.1109/ICDM.2008.17 ↗
4. Du, M., Li, F., Zheng, G., and Srikumar, V. (November, January 30). DeepLog: Anomaly Detection and Diagnosis from System Logs Through Deep Learning. Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security (CCS ’17), Dallas, TX, USA.10.1145/3133956.3134015 ↗
5. Lu, S., Wei, X., Li, Y., and Wang, L. (2018, January 12–15). Detecting Anomaly in Big Data System Logs Using Convolutional Neural Network. Proceedings of the 2018 IEEE 16th International Conference on Dependable, Autonomic and Secure Computing, 16th International Conference on Pervasive Intelligence and Computing, 4th International Conference on Big Data Intelligence and Computing and Cyber Science and Technology Congress (DASC/PiCom/DataCom/CyberSciTech), Athens, Greece.10.1109/DASC/PiCom/DataCom/CyberSciTec.2018.00037 ↗
6. Le, V.H., and Zhang, H. (2021, January 15–19). Log-Based Anomaly Detection Without Log Parsing. Proceedings of the 2021 36th IEEE/ACM International Conference on Automated Software Engineering (ASE 2021), Melbourne, Australia.10.1109/ASE51524.2021.9678773 ↗
7. Alsini, R., Alsaedi, N., Alarifi, A., and Alsulaiman, M. (2022). Topic Modeling for Anomaly Detection on Twitter. Appl. Sci., 12.
8. Song, Y.J., Nam, K.H., and Yun, I.D. (2023). Anomaly Detection through Grouping of SMD Machine Sounds Using Hierarchical Clustering. Appl. Sci., 13.10.3390/app13137569 ↗
9. Mazarbhuiya, F.A., and Shenify, M. (2023). A mixed clustering approach for real-time anomaly detection. Appl. Sci., 13.10.20944/preprints202303.0031.v1 ↗
10. Grootendorst, M. (2022). BERTopic: Neural topic modeling with a class-based TF-IDF procedure. arXiv.
11. Unsupervised log message anomaly detection10.1016/j.icte.2020.06.003 ↗
12. Rokach, L., Maimon, O., and Shmueli, E. (2023). Autoencoders. Machine Learning for Data Science Handbook: Data Mining and Knowledge Discovery Handbook, Springer International Publishing.10.1007/978-3-031-24628-9 ↗
13. Long Short-Term Memory10.1162/neco.1997.9.8.1735 ↗
14. He, P., Zhu, J., He, S., Li, J., and Lyu, M.R. (July, January 28). An Evaluation Study on Log Parsing and Its Use in Log Mining. Proceedings of the 2016 46th Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN 2016), Toulouse, France.10.1109/DSN.2016.66 ↗
15. Visual Feature Extraction by a Multilayered Network of Analog Threshold Elements10.1109/TSSC.1969.300225 ↗
16. The perceptron: A probabilistic model for information storage and organization in the brain10.1037/h0042519 ↗
17. Zhang, X., Xu, Y., Lin, Q., Qiao, B., Zhang, H., Dang, Y., Xie, C., Yang, X., Cheng, Q., and Li, Z. (2019, January 26–30). Robust Log-Based Anomaly Detection on Unstable Log Data. Proceedings of the 2019 27th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE 2019), Tallinn, Estonia.10.1145/3338906.3338931 ↗
18. Framewise phoneme classification with bidirectional LSTM and other neural network architectures10.1016/j.neunet.2005.06.042 ↗
19. Meng, W., Liu, Y., Zhu, Y., Zhang, S., Pei, D., Liu, Y., Chen, Y., Zhang, R., Tao, S., and Sun, P. (2019, January 10–16). Loganomaly: Unsupervised Detection of Sequential and Quantitative Anomalies in Unstructured Logs. Proceedings of the 28th International Joint Conference on Artificial Intelligence, Macao, China.10.24963/ijcai.2019/658 ↗
20. Devlin, J., Chang, M.W., Lee, K., and Toutanova, K. (2019). BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. arXiv.
21. Latent Dirichlet allocation
22. Learning the parts of objects by non-negative matrix factorization10.1038/44565 ↗
23. Kasliwal, B., Bhatia, S., Saini, S., Thaseen, I.S., and Kumar, C.A. (2014, January 21–22). A Hybrid Anomaly Detection Model Using G-LDA. Proceedings of the 2014 IEEE International Advance Computing Conference (IACC), Gurgaon, India.10.1109/IAdCC.2014.6779336 ↗
24. Elkhadir, Z., Chougdali, K., and Benattou, M. (2017, January 17–19). Combination of R1-PCA and Median LDA for Anomaly Network Detection. Proceedings of the 2017 Intelligent Systems and Computer Vision (ISCV 2017), Fez, Morocco.10.1109/ISACV.2017.8054985 ↗
25. Contextual Anomaly Detection in Text Data10.3390/a5040469 ↗
26. Wu, X., and Li, C. (2019, January 14–19). Short Text Topic Modeling with Flexible Word Patterns. Proceedings of the 2019 International Joint Conference on Neural Networks (IJCNN 2019), Budapest, Hungary.10.1109/IJCNN.2019.8852366 ↗
27. Furnas, G.W., Deerwester, S., Dumais, S.T., Landauer, T.K., Harshman, R.A., Streeter, L.A., and Lochbaum, K.E. (1988, January 13–15). Information Retrieval Using a Singular Value Decomposition Model of Latent Semantic Structure. Proceedings of the 11th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR ’88), Grenoble, France.10.1145/62437.62487 ↗
28. Lefoane, M., Ghafir, I., Kabir, S., and Awan, I.U. (2024, January 19–21). Latent Semantic Analysis for Feature Selection: A Proposed Approach for Anomaly Detection in Network Traffic. Proceedings of the 2024 14th International Conference on Advanced Computer Information Technologies (ACIT 2024), Ceske Budejovice, Czech Republic.10.1109/ACIT62333.2024.10712556 ↗
29. Fawaz, A.M., and Sanders, W.H. (2017, January 22–25). Learning Process Behavioral Baselines for Anomaly Detection. Proceedings of the 2017 IEEE 22nd Pacific Rim International Symposium on Dependable Computing (PRDC 2017), Christchurch, New Zealand.10.1109/PRDC.2017.28 ↗
30. McCulloh, I., Daimler, E., and Carley, K.M. (2008, January 14–17). Using Latent Semantic Analysis of Email to Detect Change in Social Groups. Proceedings of the 2008 International Conference on Data Mining (DMIN 2008), Las Vegas, NV, USA.
31. Hofmann, T. (2013). Probabilistic Latent Semantic Analysis. arXiv.
32. Wang, W., Guan, X., and Zhang, X. (2004, January 14–17). Profiling Program and User Behaviors for Anomaly Intrusion Detection Based on Non-Negative Matrix Factorization. Proceedings of the 2004 43rd IEEE Conference on Decision and Control (CDC 2004), Nassau, Bahamas.
33. Alshammari, H., Ghorbel, O., Aseeri, M., and Abid, M. (2018, January 25–29). Non-Negative Matrix Factorization (NMF) for Outlier Detection in Wireless Sensor Networks. Proceedings of the 2018 14th International Wireless Communications & Mobile Computing Conference (IWCMC 2018), Limassol, Cyprus.10.1109/IWCMC.2018.8450421 ↗
34. Anomaly-aware symmetric non-negative matrix factorization for short text clustering10.1007/s10115-024-02226-z ↗
35. Esposito, F. (2021). A Review on Initialization Methods for Nonnegative Matrix Factorization: Towards Omics Data Experiments. Mathematics, 9.10.3390/math9091006 ↗
36. Nonnegative Matrix Factorization: A Comprehensive Review10.1109/TKDE.2012.51 ↗
37. Jurafsky, D., and Martin, J.H. (2000). Speech and Language Processing: An Introduction to Natural Language Processing, Computational Linguistics, and Speech Recognition, Prentice Hall.
38. McTear, M., Callejas, Z., and Griol, D. (2016). The Conversational Interface: Talking to Smart Devices, Springer.10.1007/978-3-319-32967-3 ↗
39. A Statistical Interpretation of Term Specificity and Its Application in Retrieval10.1108/eb026526 ↗
40. He, P., Zhu, J., Zheng, Z., and Lyu, M.R. (2017, January 25–30). Drain: An Online Log Parsing Approach with Fixed Depth Tree. Proceedings of the 2017 IEEE International Conference on Web Services (ICWS 2017), Honolulu, HI, USA.10.1109/ICWS.2017.13 ↗
41. McInnes, L., Healy, J., and Melville, J. (2020). UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction. arXiv.
42. Pei, J., Tseng, V.S., Cao, L., Motoda, H., and Xu, G. (2013). Density-based clustering based on hierarchical density estimates. Advances in Knowledge Discovery and Data Mining, Springer.
43. Ester, M., Kriegel, H.P., Sander, J., and Xu, X. (1996, January 2–4). A Density-Based Algorithm for Discovering Clusters in Large Spatial Databases with Noise. Proceedings of the 2nd International Conference on Knowledge Discovery and Data Mining (KDD’96), Portland, OR, USA.
44. Least squares quantization in PCM10.1109/TIT.1982.1056489 ↗
45. MacQueen, J. (July, January 21). Some Methods for Classification and Analysis of Multivariate Observations. Proceedings of the 5th Berkeley Symposium on Mathematical Statistics and Probability, Berkeley, CA, USA.
46. The ALICE DAQ infoLogger10.1088/1742-6596/513/1/012005 ↗
47. Zhu, J., He, S., He, P., Liu, J., and Lyu, M.R. (2023, January 9–12). Loghub: A Large Collection of System Log Datasets for AI-Driven Log Analytics. Proceedings of the IEEE International Symposium on Software Reliability Engineering (ISSRE), Florence, Italy.10.1109/ISSRE59848.2023.00071 ↗