Deep Reinforcement Learning Algorithms for Intrusion Detection: A Bibliometric Analysis and Systematic Review
Source abstract
Abstract
Intrusion detection systems (IDSs) are crucial for safeguarding modern digital infrastructure against the ever-evolving cyber threats. As cyberattacks become increasingly complex, traditional machine learning (ML) algorithms, while remaining effective in classifying known threats, face limitations such as static learning, dependency on labeled data, and susceptibility to adversarial exploits. Deep reinforcement learning (DRL) has recently surfaced as a viable substitute, providing resilience in unanticipated circumstances, dynamic adaptation, and continuous learning. This study conducts a thorough bibliometric analysis and systematic literature review (SLR) of DRL-based intrusion detection systems (DRL-based IDS). The relevant literature from 2020 to 2024 was identified and investigated using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) framework. Emerging research themes, influential works, and structural relationships in the research fields were identified using a bibliometric analysis. SLR was used to synthesize methodological techniques, datasets, and performance analysis. The results indicate that DRL algorithms such as deep Q-network (DQN), double DQNs (DDQN), dueling DQN (D3QN), policy gradient methods, and actor–critic models have been actively utilized for enhancing IDS performance in various applications and datasets. The results highlight the increasing significance of DRL-based solutions for developing intelligent and robust intrusion detection systems and advancing cybersecurity.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Network Security and Intrusion Detection · Information and Cyber Security · Advanced Malware Detection Techniques
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Department of Computer Systems Engineering, Tshwane University of Technology, Private Bag X680, Pretoria 0001, South Africa
Faculty Information and Communication Technology, Tshwane University of Technology, Private Bag X680, Pretoria 0001, South Africa
References (75)
Show source-provided references
1. Goodman, M. (2015). Future Crimes: Inside the Digital Underground and the Battle for Our Connected World, Random House.
2. Achari, A. (2025). Cybersecurity in Cloud Computing, Educohack Press.
3. Pinto, L., Davidson, J., Sukthankar, R., and Gupta, A. (2017). Robust adversarial reinforcement learning. Proceedings of the International Conference on Machine Learning, PMLR.
4. Advanced network intrusion detection with tabtransformer10.53469/jtpes.2024.04(03).18 ↗
5. Revolutionizing perimeter intrusion detection: A machine learning-driven approach with curated dataset generation for enhanced security10.1109/ACCESS.2023.3318600 ↗
6. Navigating the cyber security landscape: A comprehensive review of cyber-attacks, emerging trends, and recent developments
7. Unsupervised anomaly detectors to detect intrusions in the current threat landscape10.1145/3441140 ↗
8. Valencia-Arias, A., González-Ruiz, J.D., Flores, L.V., Vega-Mori, L., Rodríguez-Correa, P., and Santos, G.S. (2024). Machine learning and blockchain: A bibliometric study on security and privacy. Information, 15.10.3390/info15010065 ↗
9. Tang, C., Abbatematteo, B., Hu, J., Chandra, R., Martín-Martín, R., and Stone, P. (March, January 25). Deep reinforcement learning for robotics: A survey of real-world successes. Proceedings of the AAAI Conference on Artificial Intelligence, Philadelphia, PA, USA.
10. Hernández-Carlón, J.J., Pérez-Romero, J., Sallent, O., Vilà, I., and Casadevall, F. (2022). A deep Q-network-based algorithm for multi-connectivity optimization in heterogeneous cellular-networks. Sensors, 22.10.3390/s22166179 ↗
11. Deep Q-network-based intelligent routing protocol for underwater acoustic sensor network10.1109/JSEN.2023.3234112 ↗
12. Dynamic path planning via Dueling Double Deep Q-Network (D3QN) with prioritized experience replay10.1016/j.asoc.2024.111503 ↗
13. On the sample complexity of actor-critic method for reinforcement learning with function approximation10.1007/s10994-023-06303-2 ↗
14. Tiong, T., Saad, I., Teo, K.T.K., and Lago, H.B. (2020). Deep reinforcement learning with robust deep deterministic policy gradient. Proceedings of the 2020 2nd International Conference on Electrical, Control and Instrumentation Engineering (ICECIE), IEEE.10.1109/ICECIE50279.2020.9309539 ↗
15. Powernet: Multi-agent deep reinforcement learning for scalable powergrid control10.1109/TPWRS.2021.3100898 ↗
16. Robust enhancement of intrusion detection systems using deep reinforcement learning and stochastic game10.1109/TVT.2022.3186834 ↗
17. Machine learning for intrusion detection in industrial control systems: Applications, challenges, and recommendations10.1016/j.ijcip.2022.100516 ↗
18. Aldallal, A., and Alisa, F. (2021). Effective intrusion detection system to secure data in cloud using machine learning. Symmetry, 13.10.3390/sym13122306 ↗
19. Van Hasselt, H.P., Guez, A., Hessel, M., Mnih, V., and Silver, D. (2016). Learning values across many orders of magnitude. arXiv.
20. Deep Q-network-based heuristic intrusion detection against edge-based SIoT zero-day attacks10.1016/j.asoc.2023.111080 ↗
21. Suwannalai, E., and Polprasert, C. (2020). Network intrusion detection systems using adversarial reinforcement learning with deep Q-network. Proceedings of the 2020 18th International Conference on ICT and Knowledge Engineering (ICT&KE), IEEE.10.1109/ICTKE50349.2020.9289884 ↗
22. Effective defense strategies in network security using improved double dueling deep Q-network10.1016/j.cose.2023.103578 ↗
23. Intrusion detection in industrial control systems based on deep reinforcement learning10.1109/ACCESS.2024.3477415 ↗
24. Wang, Y., and Zou, S. (2022). Policy gradient method for robust reinforcement learning. Proceedings of the International Conference on Machine Learning, PMLR.
25. Big-IDS: A decentralized multi agent reinforcement learning approach for distributed intrusion detection in big data networks10.1007/s10586-024-04306-9 ↗
26. Machine learning for anomaly detection: A systematic review10.1109/ACCESS.2021.3083060 ↗
27. Deep reinforcement learning for anomaly detection: A systematic review10.1109/ACCESS.2022.3224023 ↗
28. Keele, S. (2025, October 15). Guidelines for Performing Systematic Literature Reviews in Software Engineering. Technical report, ver. 2.3 ebse Technical Report. ebse 2007. Available online: https://legacyfileshare.elsevier.com/promis_misc/525444systematicreviewsguide.pdf.
29. PRISMA 2020 explanation and elaboration: Updated guidance and exemplars for reporting systematic reviews10.1136/bmj.n160 ↗
30. Deep reinforcement learning for cyber security10.1109/TNNLS.2021.3121870 ↗
31. A context-aware robust intrusion detection system: A reinforcement learning-based approach10.1007/s10207-019-00482-7 ↗
32. Priya, S., and PradeepMohankumar, K. (2021). Intelligent outlier detection with optimal deep reinforcement learning model for intrusion detection. Proceedings of the 2021 4th International Conference on Computing and Communications Technologies (ICCCT), IEEE.10.1109/ICCCT53315.2021.9711837 ↗
33. Wang, Z., Wang, Y., Xu, H., and Wang, Y. (2021). Effective anomaly detection based on reinforcement learning in network traffic data. Proceedings of the 2021 IEEE 27th International Conference on Parallel and Distributed Systems (ICPADS), IEEE.10.1109/ICPADS53394.2021.00043 ↗
34. Su, Y.-J., Wu, L.-C., Chen, C.-H., and Chen, T.-Y. (2023). Combining Data Resampling and DRL Algorithm for Intrusion Detection. Proceedings of the 2023 5th International Conference on Computer Communication and the Internet (ICCCI), IEEE.10.1109/ICCCI59363.2023.10210170 ↗
35. Merzouk, M.A., Delas, J., Neal, C., Cuppens, F., Boulahia-Cuppens, N., and Yaich, R. (2022, January 23–26). Evading deep reinforcement learning-based network intrusion detection with adversarial attacks. Proceedings of the 17th International Conference on Availability, Reliability and Security, Vienna, Austria.10.1145/3538969.3539006 ↗
36. Tariq, Z.U.A., Baccour, E., Erbad, A., Guizani, M., and Hamdi, M. (2022). Network intrusion detection for smart infrastructure using multi-armed bandit based reinforcement learning in adversarial environment. Proceedings of the 2022 International Conference on Cyber Warfare and Security (ICCWS), IEEE.10.1109/ICCWS56285.2022.9998440 ↗
37. Reinforcement learning meets network intrusion detection: A transferable and adaptable framework for anomaly behavior identification10.1109/TNSM.2024.3352586 ↗
38. Zolotukhin, M., Kumar, S., and Hämäläinen, T. (2020). Reinforcement learning for attack mitigation in SDN-enabled networks. Proceedings of the 2020 6th IEEE Conference on Network Softwarization (NetSoft), IEEE.10.1109/NetSoft48620.2020.9165383 ↗
39. Sujatha, V., Prasanna, K.L., Niharika, K., Charishma, V., and Sai, K.B. (2023). Network intrusion detection using deep reinforcement learning. Proceedings of the 2023 7th International Conference on Computing Methodologies and Communication (ICCMC), IEEE.10.1109/ICCMC56507.2023.10083673 ↗
40. Dang, Q.-V., and Vo, T.-H. (2021). Studying the Reinforcement Learning techniques for the problem of intrusion detection. Proceedings of the 2021 4th International Conference on Artificial Intelligence and Big Data (ICAIBD), IEEE.10.1109/ICAIBD51990.2021.9459006 ↗
41. Bebortta, S., Tripathy, S.S., Sharma, V., Behera, J.R., and Nayak, A. (2024). A Secure Deep Q-Reinforcement Learning Framework for Network Intrusion Detection in IoT-Fog Systems. Proceedings of the 2024 OPJU International Technology Conference (OTCON) on Smart Computing for Innovation and Advancement in Industry 4.0, IEEE.10.1109/OTCON60325.2024.10687378 ↗
42. Rookard, C., and Khojandi, A. (2023). Applying deep reinforcement learning for detection of Internet-of-Things cyber attacks. Proceedings of the 2023 IEEE 13th Annual Computing and Communication Workshop and Conference (CCWC), IEEE.10.1109/CCWC57344.2023.10099349 ↗
43. Federated reinforcement learning based intrusion detection system using dynamic attention mechanism
44. Hsu, Y.-F., and Matsuoka, M. (2020). A deep reinforcement learning approach for anomaly network intrusion detection system. Proceedings of the 2020 IEEE 9th International Conference on Cloud Networking (CloudNet), IEEE.10.1109/CloudNet51028.2020.9335796 ↗
45. MAFSIDS: A reinforcement learning-based intrusion detection model for multi-agent feature selection networks10.1186/s40537-023-00814-4 ↗
46. A soft actor-critic reinforcement learning algorithm for network intrusion detection10.1016/j.cose.2023.103502 ↗
47. Benaddi, H., Ibrahimi, K., Benslimane, A., and Qadir, J. (2019). A deep reinforcement learning based intrusion detection system (drl-ids) for securing wireless sensor networks and internet of things. Proceedings of the International Wireless Internet Conference, Springer.10.1007/978-3-030-52988-8_7 ↗
48. A novel reinforcement learning-based hybrid intrusion detection system on fog-to-cloud computing10.1007/s11227-024-06417-x ↗
49. Batch reinforcement learning approach using recursive feature elimination for network intrusion detection10.1016/j.engappai.2024.109013 ↗
50. Intelligent defense strategies: Comprehensive attack detection in VANET with deep reinforcement learning10.1016/j.pmcj.2024.101962 ↗
51. Reinforcement learning for intrusion detection: More model longness and fewer updates10.1109/TNSM.2022.3207094 ↗
52. Sethi, K., Kumar, R., Prajapati, N., and Bera, P. (2020). Deep reinforcement learning based intrusion detection system for cloud infrastructure. Proceedings of the 2020 International Conference on COMmunication Systems & NETworkS (COMSNETS), IEEE.10.1109/COMSNETS48256.2020.9027452 ↗
53. Ren, K., Wang, M., Zeng, Y., and Zhang, Y. (2022). An unmanned network intrusion detection model based on deep reinforcement learning. Proceedings of the 2022 IEEE International Conference on Unmanned Systems (ICUS), IEEE.10.1109/ICUS55513.2022.9986601 ↗
54. Raj, N.N., Rajesh, R., Justin, A., and Shihab, F. (2018). Enhancing Network Intrusion Detection Using Deep Reinforcement Learning: An Adaptive Learning Approach. Proceedings of the International Conference on Emerging Trends in Communication, Computing and Electronics, Springer.
55. Alavizadeh, H., Alavizadeh, H., and Jang-Jaccard, J. (2022). Deep Q-learning based reinforcement learning approach for network intrusion detection. Computers, 11.10.3390/computers11030041 ↗
56. Benaddi, H., Jouhari, M., Ibrahimi, K., Othman, J.B., and Amhoud, E.M. (2022). Anomaly detection in industrial IoT using distributional reinforcement learning and generative adversarial networks. Sensors, 22.10.3390/s22218085 ↗
57. Network abnormal traffic detection model based on semi-supervised deep reinforcement learning10.1109/TNSM.2021.3120804 ↗
58. Intrusion detection system for industrial Internet of Things based on deep reinforcement learning10.1155/2022/9023719 ↗
59. ID-RDRL: A deep reinforcement learning-based feature selection intrusion detection model10.1038/s41598-022-19366-3 ↗
60. Leveraging deep reinforcement learning technique for intrusion detection in SCADA infrastructure10.1109/ACCESS.2024.3390722 ↗
61. Baby, R., Pooranian, Z., Shojafar, M., and Tafazolli, R. (2023). A heterogenous IoT attack detection through deep reinforcement learning: A dynamic ML approach. Proceedings of the ICC 2023-IEEE International Conference on Communications, IEEE.10.1109/ICC45041.2023.10278685 ↗
62. Sethi, K., Kumar, R., Mohanty, D., and Bera, P. (2020). Robust adaptive cloud intrusion detection system using advanced deep reinforcement learning. Proceedings of the International Conference on Security, Privacy, and Applied Cryptography Engineering, Springer.10.1007/978-3-030-66626-2_4 ↗
63. Enhanced intrusion detection in wireless sensor networks using deep reinforcement learning with improved feature extraction and selection
64. Policy gradient and actor-critic learning in continuous time and space: Theory and algorithms
65. Bahdanau, D., Brakel, P., Xu, K., Goyal, A., Lowe, R., Pineau, J., Courville, A., and Bengio, Y. (2016). An Actor-Critic Algorithm for Sequence Prediction. arXiv.
66. Distributional soft actor-critic with three refinements10.1109/TPAMI.2025.3537087 ↗
67. Sutton, R.S., and Barto, A.G. (1998). Reinforcement learning: An introduction, MIT Press.10.1109/TNN.1998.712192 ↗
68. Schulman, J., Wolski, F., Dhariwal, P., Radford, A., and Klimov, O. (2017). Proximal Policy Optimization Algorithms. arXiv.
69. Van Hasselt, H., Guez, A., and Silver, D. (2016, January 12–17). Deep reinforcement learning with double q-learning. Proceedings of the AAAI Conference on Artificial Intelligence, Phoenix, AZ, USA.10.1609/aaai.v30i1.10295 ↗
70. Wang, Z., Schaul, T., Hessel, M., Hasselt, H., Lanctot, M., and Freitas, N. (2016). Dueling network architectures for deep reinforcement learning. Proceedings of the International Conference on Machine Learning, PMLR.
71. Application of deep reinforcement learning to intrusion detection for supervised problems10.1016/j.eswa.2019.112963 ↗
72. Multi-agent reinforcement learning: A selective overview of theories and algorithms10.1007/978-3-030-60990-0_12 ↗
73. Albrecht, S.V., Christianos, F., and Schäfer, L. (2024). Multi-Agent Reinforcement Learning: Foundations and Modern Approaches, MIT Press.
74. A deep learning approach for network intrusion detection using a small features vector10.3390/jcp3030023 ↗
75. Robust deep reinforcement learning against adversarial perturbations on state observations