Anomaly Detection in Log Files Using Selected Natural Language Processing Methods
Source abstract
Abstract
In this article, we address the problem of detecting anomalies in system log files. Computer systems generate huge numbers of events, which are noted in event log files. While most of them report normal actions, an unusual entry may inform about a failure or malware infection. A human operator may easily miss such an entry; therefore, anomaly detection methods are used for this purpose. In our work, we used an approach known from the natural language processing (NLP) domain, which operates on so-called embeddings, that is vector representations of words or phrases. We describe an improved version of the LogEvent2Vec algorithm, proposed in 2020. In contrast to the original version, we propose a significant shortening of the analysis window, which both increased the accuracy of anomaly detection and made further analysis of suspicious sequences much easier. We experimented with various binary classifiers, such as decision trees or multilayer perceptrons (MLPs), and the Blue Gene/L dataset. We showed that selecting an optimal classifier (in this case, MLP) and a short log sequence gave very good results. The improved version of the algorithm yielded the best F1-score of 0.997, compared to 0.886 in the original version of the algorithm.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Software System Performance and Reliability · Network Security and Intrusion Detection · Anomaly Detection Techniques and Applications
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Faculty of Electronics and Information Technology, Warsaw University of Technology, Nowowiejska 15/19, 00-665 Warsaw, Poland
Funding
SIMARGL Project–Secure Intelligent Methods for Advanced RecoGnition · European Commission
References (47)
Show source-provided references
1. Xu, W., Huang, L., Fox, A., Patterson, D., and Jordan, M.I. (2009). Detecting Large-Scale System Problems by Mining Console Logs. Proceedings of the ACM SIGOPS 22nd Symposium on Operating Systems Principles, Association for Computing Machinery.10.1145/1629575.1629587 ↗
2. Advances and Challenges in Log Analysis10.1145/2076450.2076466 ↗
3. Svacina, J., Raffety, J., Woodahl, C., Stone, B., Cerny, T., Bures, M., Shin, D., Frajtak, K., and Tisnovsky, P. (2020). On Vulnerability and Security Log Analysis: A Systematic Literature Review on Recent Trends. Proceedings of the International Conference on Research in Adaptive and Convergent Systems, Association for Computing Machinery.10.1145/3400286.3418261 ↗
4. A Survey on Automated Log Analysis for Reliability Engineering10.1145/3460345 ↗
5. Müller, A., Miinz, G., and Carle, G. (2011, January 4–7). Collecting router information for error diagnosis and troubleshooting in home networks. Proceedings of the 2011 IEEE 36th Conference on Local Computer Networks, Bonn, Germany.10.1109/LCN.2011.6115547 ↗
6. Brandao, A., and Georgieva, P. (2020, January 28–30). Log Files Analysis For Network Intrusion Detection. Proceedings of the 2020 IEEE 10th International Conference on Intelligent Systems (IS), Varna, Bulgaria.10.1109/IS48319.2020.9199976 ↗
7. He, S., Zhu, J., He, P., and Lyu, M.R. (2016, January 23–27). Experience Report: System Log Analysis for Anomaly Detection. Proceedings of the 2016 IEEE 27th International Symposium on Software Reliability Engineering (ISSRE), Ottawa, ON, Canada.10.1109/ISSRE.2016.21 ↗
8. Mining of Web Server Logs in a Distributed Cluster Using Big Data Technologies
9. Wang, J., Tang, Y., He, S., Zhao, C., Sharma, P., Alfarraj, O., and Tolba, A. (2020). LogEvent2vec: LogEvent-to-Vector Based Anomaly Detection for Large-Scale Logs in Internet of Things. Sensors, 20.10.3390/s20092451 ↗
10. Anomaly Detection: A Survey10.1145/1541880.1541882 ↗
11. Meghanathan, N., Kaushik, B.K., and Nagamalai, D. (2011). Web Log Data Analysis and Mining. Advanced Computing, Springer.10.1007/978-3-642-17881-8 ↗
12. Kim, K.J. (2015). Anomaly Detection from Log Files Using Data Mining Techniques. Information Science and Applications, Springer.10.1007/978-3-662-46578-3 ↗
13. PreFix: Switch Failure Prediction in Datacenter Networks
14. Khatuya, S., Ganguly, N., Basak, J., Bharde, M., and Mitra, B. (2018, January 16–19). ADELE: Anomaly Detection from Event Log Empiricism. Proceedings of the IEEE Conference on Computer Communications (INFOCOM 2018), Honolulu, HI, USA.10.1109/INFOCOM.2018.8486257 ↗
15. Bertero, C., Roy, M., Sauvanaud, C., and Tredan, G. (2017, January 23–26). Experience Report: Log Mining Using Natural Language Processing and Application to Anomaly Detection. Proceedings of the 2017 IEEE 28th International Symposium on Software Reliability Engineering (ISSRE), Toulouse, France.10.1109/ISSRE.2017.43 ↗
16. Mikolov, T., Chen, K., Corrado, G., and Dean, J. (2013). Efficient Estimation of Word Representations in Vector Space. arXiv.
17. Pennington, J., Socher, R., and Manning, C. (2014, January 25–29). GloVe: Global Vectors for Word Representation. Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP 2014), Doha, Qatar.10.3115/v1/D14-1162 ↗
18. Enriching word vectors with subword information10.1162/tacl_a_00051 ↗
19. Meng, W., Liu, Y., Huang, Y., Zhang, S., Zaiter, F., Chen, B., and Pei, D. (2020, January 3–6). A Semantic-aware Representation Framework for Online Log Analysis. Proceedings of the 2020 29th International Conference on Computer Communications and Networks (ICCCN), Honolulu, HI, USA.10.1109/ICCCN49398.2020.9209707 ↗
20. Li, K.L., Huang, H.K., Tian, S.F., and Xu, W. (2003, January 5). Improving one-class SVM for anomaly detection. Proceedings of the International Conference on Machine Learning and Cybernetics (ICLMC), Xi’an, China.
21. Zhang, W., and Chen, L. (2019, January 14–16). Web Log Anomaly Detection Based on Isolated Forest Algorithm. Proceedings of the IEEE 14th International Conference on Intelligent Systems and Knowledge Engineering (ISKE), Dalian, China.10.1109/ISKE47853.2019.9170430 ↗
22. Henriques, J., Caldeira, F., Cruz, T., and Simões, P. (2020). Combining K-Means and XGBoost Models for Anomaly Detection Using Log Datasets. Electronics, 9.10.3390/electronics9071164 ↗
23. An Improved KNN-Based Efficient Log Anomaly Detection Method with Automatically Labeled Samples10.1145/3441448 ↗
24. Du, M., Li, F., Zheng, G., and Srikumar, V. (November, January 30). DeepLog: Anomaly Detection and Diagnosis from System Logs through Deep Learning. Proceedings of the ACM SIGSAC Conference on Computer and Communications Security, Dallas, TX, USA.
25. Chen, Z., Liu, J., Gu, W., Su, Y., and Lyu, M.R. (2021). Experience Report: Deep Learning-based System Log Analysis for Anomaly Detection. arXiv.
26. Guyon, I., Luxburg, U.V., Bengio, S., Wallach, H., Fergus, R., Vishwanathan, S., and Garnett, R. (2017). Attention is All you Need. Advances in Neural Information Processing Systems, Curran Associates, Inc.
27. Huang, Z., Xu, W., and Yu, K. (2015). Bidirectional LSTM-CRF Models for Sequence Tagging. arXiv.
28. Chen, Y., Luktarhan, N., and Lv, D. (2022). LogLS: Research on System Log Anomaly Detection Method Based on Dual LSTM. Symmetry, 14.10.3390/sym14030454 ↗
29. Guo, H., Yuan, S., and Wu, X. (2021, January 18–22). LogBERT: Log Anomaly Detection via BERT. Proceedings of the 2021 International Joint Conference on Neural Networks (IJCNN), Shenzhen, China.10.1109/IJCNN52387.2021.9534113 ↗
30. Devlin, J., Chang, M.W., Lee, K., and Toutanova, K. (2018). BERT: Pre-training of deep bidirectional transformers for language understanding. arXiv.
31. Le, V.H., and Zhang, H. (2021, January 15–19). Log-based Anomaly Detection without Log Parsing. Proceedings of the 36th IEEE/ACM International Conference on Automated Software Engineering (ASE), Melbourne, Australia.10.1109/ASE51524.2021.9678773 ↗
32. QLLog: A log anomaly detection method based on Q-learning algorithm10.1016/j.ipm.2021.102540 ↗
33. Chen, R., Zhang, S., Li, D., Zhang, Y., Guo, F., Meng, W., Pei, D., Zhang, Y., Chen, X., and Liu, Y. (2020, January 12–15). LogTransfer: Cross-System Log Anomaly Detection for Software Systems with Transfer Learning. Proceedings of the IEEE 31st International Symposium on Software Reliability Engineering (ISSRE), Coimbra, Portugal.10.1109/ISSRE5003.2020.00013 ↗
34. Yadav, R.B., Kumar, P.S., and Dhavale, S.V. (2020, January 4–5). A Survey on Log Anomaly Detection using Deep Learning. Proceedings of the 8th International Conference on Reliability, Infocom Technologies and Optimization (Trends and Future Directions) (ICRITO), Noida, India.10.1109/ICRITO48877.2020.9197818 ↗
35. He, P., Zhu, J., Zheng, Z., and Lyu, M.R. (2017, January 25–30). Drain: An Online Log Parsing Approach with Fixed Depth Tree. Proceedings of the IEEE International Conference on Web Services (ICWS), Honolulu, HI, USA.10.1109/ICWS.2017.13 ↗
36. Usenix (2022, February 20). The HPC4 Data. Available online: https://www.usenix.org/cfdr-data#hpc4.
37. IBM (2022, January 10). Drain3. Available online: https://github.com/IBM/Drain3.
38. Kim, E. (2022, February 13). Optimize Computational Efficiency of Skip-Gram with Negative Sampling. Available online: https://aegis4048.github.io/optimize_computational_efficiency_of_skip-gram_with_negative_sampling.
39. Rong, X. (2016). word2vec Parameter Learning Explained. arXiv.
40. Oliner, A., and Stearley, J. (2007, January 25–28). What Supercomputers Say: A Study of Five System Logs. Proceedings of the 37th Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN’07), Edinburgh, UK.10.1109/DSN.2007.103 ↗
41. Python Software Foundation (2022, January 15). Re. Available online: https://docs.python.org/3/library/re.html.
42. Open-Source Python Library (2022, January 15). Numpy. Available online: https://numpy.org/about/.
43. McKinney, W. (2022, January 15). Pandas. Available online: https://pandas.pydata.org/.
44. Facebook (2022, January 19). fastText. Available online: https://fasttext.cc/.
45. Cournapeau, D. (2022, January 20). Scikit-Learn. Available online: https://scikit-learn.org/.
46. Rathore, M. (2022, January 08). Comparison of FastText and Word2Vec. Available online: https://markroxor.github.io/gensim/static/notebooks/Word2Vec_FastText_Comparison.html.
47. He, S., Zhu, J., He, P., and Lyu, M.R. (2020). Loghub: A Large Collection of System Log Datasets towards Automated Log Analytics. arXiv.