Machine Learning Algorithms for Raw and Unbalanced Intrusion Detection Data in a Multi-Class Classification Problem
Source abstract
Abstract
Various machine learning algorithms have been applied to network intrusion classification problems, including both binary and multi-class classifications. Despite the existence of numerous studies involving unbalanced network intrusion datasets, such as CIC-IDS2017, a prevalent approach is to address the issue by either merging the classes to optimize their numbers or retaining only the most dominant ones. However, there is no consistent trend showing that accuracy always decreases as the number of classes increases. Furthermore, it is essential for cybersecurity practitioners to recognize the specific type of attack and comprehend the causal factors that contribute to the resulting outcomes. This study focuses on tackling the challenges associated with evaluating the performance of multi-class classification for network intrusions using highly imbalanced raw data that encompasses the CIC-IDS2017 and CSE-CIC-IDS2018 datasets. The research concentrates on investigating diverse machine learning (ML) models, including Logistic Regression, Random Forest, Decision Trees, CNNs, and Artificial Neural Networks. Additionally, it explores the utilization of explainable AI (XAI) methods to interpret the obtained results. The results obtained indicated that decision trees using the CART algorithm performed best on the 28-class classification task, with an average macro F1-score of 0.96878.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Network Security and Intrusion Detection · Anomaly Detection Techniques and Applications · Advanced Malware Detection Techniques
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Faculty of Informatics, Kaunas University of Technology, Studentu 50, 51368 Kaunas, Lithuania
References (52)
Show source-provided references
1. A Comprehensive Systematic Literature Review on Intrusion Detection Systems10.1109/ACCESS.2021.3129336 ↗
2. A comprehensive review study of cyber-attacks and cyber security; Emerging trends and recent developments10.1016/j.egyr.2021.08.126 ↗
3. Jin, S., Chung, J.-G., and Xu, Y. (2021, January 22–28). Signature-Based Intrusion Detection System (IDS) for In-Vehicle CAN Bus Network. Proceedings of the IEEE International Symposium on Circuits and Systems (ISCAS), Daegu, Republic of Korea.10.1109/ISCAS51556.2021.9401087 ↗
4. Erlacher, F., and Dressler, F. (2018, January 23–27). FIXIDS: A high-speed signature-based flow intrusion detection system. Proceedings of the IEEE/IFIP Network Operations and Management Symposium, Taipei, Taiwan.10.1109/NOMS.2018.8406247 ↗
5. Preuveneers, D., Rimmer, V., Tsingenopoulos, I., Spooren, J., Joosen, W., and Ilie-Zudor, E. (2018). Chained Anomaly Detection Models for Federated Learning: An Intrusion Detection Case Study. Appl. Sci., 8.10.3390/app8122663 ↗
6. A systematic literature review of methods and datasets for anomaly-based network intrusion detection10.1016/j.cose.2022.102675 ↗
7. Lan, Y., Truong-Huu, T., Wu, J., and Teo, S.G. (December, January 28). Cascaded Multi-Class Network Intrusion Detection with Decision Tree and Self-attentive Model. Proceedings of the IEEE International Conference on Data Mining Workshops (ICDMW), Orlando, FL, USA.
8. Performance Analysis of Machine Learning Algorithms in Intrusion Detection System: A Review10.1016/j.procs.2020.04.133 ↗
9. Alsyaibani, O.M.A., Utami, E., and Hartanto, A.D. (2021, January 25–26). An Intrusion Detection System Model Based on Bidirectional LSTM. Proceedings of the 3rd International Conference on Cybernetics and Intelligent System (ICORIS), Makasar, Indonesia.10.1109/ICORIS52787.2021.9649612 ↗
10. Iwendi, C., Khan, S., Anajemba, J.H., Mittal, M., Alenezi, M., and Alazab, M. (2020). The Use of Ensemble Models for Multiple Class and Binary Class Classification for Improving Intrusion Detection Systems. Sensors, 20.10.3390/s20092559 ↗
11. Khan, M.A. (2021). HCRNNIDS: Hybrid Convolutional Recurrent Neural Network-Based Network Intrusion Detection System. Processes, 9.10.3390/pr9050834 ↗
12. The Effect of Sampling Methods on the CICIDS2017 Network Intrusion Data Set. IT Con-vergence and Security10.1007/978-981-16-4118-3_4 ↗
13. Study of Multi-Class Classification Algorithms’ Performance on Highly Imbalanced Network Intrusion Datasets10.15388/21-INFOR457 ↗
14. Tran, T.P., Nguyen, V.C., Vu, L., and Nguyen, Q.U. (2021, January 21–22). DeepInsight-Convolutional Neural Network for Intrusion Detection Systems. Proceedings of the 8th NAFOSTED Conference on Information and Computer Science (NICS), Hanoi, Vietnam.10.1109/NICS54270.2021.9701572 ↗
15. Network intrusion detection using multi-architectural modular deep neural network10.1007/s11227-020-03410-y ↗
16. A Deep Learning Approach for Intrusion Detection Using Recurrent Neural Networks10.1109/ACCESS.2017.2762418 ↗
17. Evaluation of Recurrent Neural Network and its Variants for Intrusion Detection System (IDS)10.4018/IJISMD.2017070103 ↗
18. Deep belief network based intrusion detection techniques: A survey10.1016/j.eswa.2020.114170 ↗
19. Experimental Analysis of Trustworthy In-Vehicle Intrusion Detection System Using eXplainable Artificial Intelligence (XAI)10.1109/ACCESS.2022.3208573 ↗
20. Patil, S., Varadarajan, V., Mazhar, S.-M., Sahibzada, A., Ahmed, N., Sinha, O., Kumar, S., Shaw, K., and Kotecha, K. (2022). Explainable Artificial Intelligence for Intrusion Detection System. Electronics, 11.10.3390/electronics11193079 ↗
21. Explainable Artificial Intelligence (XAI): Concepts, taxonomies, opportunities and challenges toward responsible AI10.1016/j.inffus.2019.12.012 ↗
22. Hartl, A., Bachl, M., Fabini, J., and Zseby, T. (2020, January 3–6). Explainability and Adversarial Robustness for RNNs. Proceedings of the IEEE Sixth International Conference on Big Data Computing Service and Applications (BigDataService), Oxford, UK.10.1109/BigDataService49289.2020.00030 ↗
23. XAI for intrusion detection system: Comparing explanations based on global and local scope10.1007/s11416-022-00441-2 ↗
24. Explainable Artificial Intelligence (XAI) to Enhance Trust Management in Intrusion Detection Systems Using Decision Tree Model10.1155/2021/6634811 ↗
25. Kuppa, A., and Le-Khac, N.-A. (2020, January 19–24). Black Box Attacks on Explainable Artificial Intelligence (XAI) methods in Cyber Security. Proceedings of the International Joint Conference on Neural Networks (IJCNN), Glasgow, UK.10.1109/IJCNN48605.2020.9206780 ↗
26. Adversarial XAI Methods in Cybersecurity10.1109/TIFS.2021.3117075 ↗
27. Evaluating the CIC IDS-2017 Dataset Using Machine Learning Methods and Creating Multiple Predictive Models in the Statistical Computing Language R
28. Toward Constructing a Balanced Intrusion Detection Dataset10.54153/sjpas.2020.v2i3.86 ↗
29. Mbow, M., Koide, H., and Sakurai, K. (2021, January 22–26). An Intrusion Detection System for Imbalanced Dataset Based on Deep Learning. Proceedings of the Ninth International Symposium on Computing and Networking (CANDAR), Matsue, Japan.10.1109/CANDAR53791.2021.00013 ↗
30. An effective convolutional neural network based on SMOTE and Gaussian mixture model for intrusion detection in imbalanced dataset10.1016/j.comnet.2020.107315 ↗
31. Abdulhammed, R., Musafer, H., Alessa, A., Faezipour, M., and Abuzneid, A. (2019). Features Dimensionality Reduction Approaches for Machine Learning Based Network Intrusion Detection. Electronics, 8.10.3390/electronics8030322 ↗
32. Toupas, P., Chamou, D., Giannoutakis, K.M., Drosou, A., and Tzovaras, D. (2019, January 16–19). An Intrusion Detection System for Multi-class Classi-fication Based on Deep Neural Networks. Proceedings of the 18th IEEE International Conference on Machine Learning and Applications (ICMLA), Boca Raton, FL, USA.10.1109/ICMLA.2019.00206 ↗
33. PCCN: Parallel Cross Convolutional Neural Network for Abnormal Network Traffic Flows Detection in Multi-Class Imbalanced Network Traffic Flows10.1109/ACCESS.2019.2933165 ↗
34. Mhawi, D.N., Aldallal, A., and Hassan, S. (2022). Advanced Feature-Selection-Based Hybrid Ensemble Learning Algorithms for Network Intrusion Detection Systems. Symmetry, 14.10.3390/sym14071461 ↗
35. Rosay, R., Cheval, E., Carlier, F., and Leroux, P. (2022, January 9–11). Network Intrusion Detection: A Comprehensive Analysis of CIC-IDS2017. Proceedings of the 8th International Conference on Information Systems Security and Privacy, Online.10.5220/0010774000003120 ↗
36. Lanvin, M., Gimenez, P.-F., Han, Y., Majorczyk, F., Me, L., and Totel, E. (2022, January 7–9). Errors in the CICIDS2017 dataset and the significant differences in detection performances it makes. Proceedings of the 17th International Conference Risks and Security of Internet and Systems, Sousse, Tunisia.10.1007/978-3-031-31108-6_2 ↗
37. CatBoost-Based Network Intrusion Detection on Imbalanced CIC-IDS-2018 Dataset
38. Intrusion Detection of Imbalanced Network Traffic Based on Machine Learning and Deep Learning10.1109/ACCESS.2020.3048198 ↗
39. Detecting cybersecurity attacks across different network features and learners10.1186/s40537-021-00426-w ↗
40. Performance analysis of intrusion detection for deep learning model based on CSE-CIC-IDS2018 dataset
41. A comprehensive intrusion detection framework using boosting algorithms10.1016/j.compeleceng.2022.107869 ↗
42. Alzughaibi, S., and El Khediri, S. (2023). A Cloud Intrusion Detection Systems Based on DNN Using Backpropagation and PSO on the CSE-CIC-IDS2018 Dataset. Appl. Sci., 13.10.3390/app13042276 ↗
43. Deep Learning Algorithms for Intrusion Detection Systems in Internet of Things Using CIC-IDS 2017 Dataset10.11591/ijece.v13i1.pp1134-1141 ↗
44. Wang, Y.-C., Houng, Y.-C., Chen, H.-X., and Tseng, S.-M. (2023). Network Anomaly Intrusion Detection Based on Deep Learning Ap-proach. Sensors, 23.10.3390/s23042171 ↗
45. Ingre, B., Yadav, A., and Soni, A.K. (2017, January 25–26). Decision Tree Based Intrusion Detection System for NSL-KDD Dataset. Proceedings of the Information and Communication Technology for Intelligent Systems (ICTIS 2017), Ahmedabad, India.10.1007/978-3-319-63645-0_23 ↗
46. Brabec, J., and Machlica, L. (2018, January 7–10). Decision-Forest Voting Scheme for Classification of Rare Classes in Network Intrusion Detection. Proceedings of the IEEE International Conference on Systems, Man and Cybernetics, Miyazaki, Japan.10.1109/SMC.2018.00563 ↗
47. Sahani, R., Rout, C., Badajena, J.C., Jena, A.K., and Das, H. (2018). Progress in Computing, Analytics and Networking, Springer.
48. Research on machine learning framework based on random forest algorithm10.1063/1.4977376 ↗
49. Intrusion Detection Model for Imbalanced Dataset Using SMOTE and Random Forest Algorithm
50. Vang-Mata, R. (2020). Multilayer Perceptrons: Theory and Applications, Nova Science Publishers.
51. Huang, G., Liu, Z., Maaten, L., and Weinberger, K. (2017, January 21–26). Densely Connected Convolutional Networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.10.1109/CVPR.2017.243 ↗
52. Kamath, U., and Liu, J. (2021). Explainable Artificial Intelligence: An Introduction to Interpretable Machine Learning, Springer.10.1007/978-3-030-83356-5 ↗