journal article·2024-12-27Open access

A Comparative Study of Convolutional Neural Network and Transformer Architectures for Drone Detection in Thermal Images

Gian Gutierrez, Juan P. Llerena, Luis Usero, Miguel A. Patricio

Applied Sciences· Volume 15· Issue 1

Abstract

The widespread growth of drone technology is generating new security paradigms, especially with regard to the unauthorized activities of UAVs in restricted or sensitive areas, as well as illegal and illicit activities or attacks. Among the various UAV detection technologies, vision systems in different spectra are postulated as outstanding technologies due to their peculiarities compared to other technologies. However, drone detection in thermal imaging is a challenging task due to specific factors such as thermal noise, temperature variability, or cluttered environments. This study addresses these challenges through a comparative evaluation of contemporary neural network architectures—specifically, convolutional neural networks (CNNs) and transformer-based models—for UAV detection in infrared imagery. The research focuses on real-world conditions and examines the performance of YOLOv9, GELAN, DETR, and ViTDet in different scenarios of the Anti-UAV Challenge 2023 dataset. The results show that YOLOv9 stands out for its real-time detection speed, while GELAN provides the highest accuracy in varying conditions and DETR performs reliably in thermally complex environments. The study contributes to the advancement of state-of-the-art UAV detection techniques and highlights the need for the further development of specialized models for specific detection scenarios.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: Video Surveillance and Tracking Methods · Advanced Neural Network Applications · UAV Applications and Optimization

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

Applied Artificial Intelligence Group, Computer Science and Engineering Department, Universidad Carlos III de Madrid (ROR Code 03ths8210), 28270 Colmenarejo, Madrid, Spain

Cognitive Science Research Group, Universidad de Alcalá, 28805 Alcalá de Henares, Madrid, Spain

Funding

Spanish Ministry of Science and Innovation

References (66)

Show source-provided references
1. Investigation on Advances of Unmanned Aerial Vehicle Application Research in Agriculture and Forestry
2. Fleureau, J., Galvane, Q., Tariolle, F.L., and Guillotel, P. (2016). Generic drone control platform for autonomous capture of cinema scenes. DroNet 2016: Proceedings of the 2nd Workshop on Micro Aerial Vehicle Networks, Systems, and Applications for Civilian Use, Association for Computing Machinery.10.1145/2935620.2935622 ↗
3. Özgen, K. (2020). The impact of drones in documentary filmmaking: Renaissance of aerial shot. AVANCA|CINEMA, 559–563.10.37390/ac.v0i0.74 ↗
4. Security Drone for Surveillance in Military10.22214/ijraset.2024.58557 ↗
5. Das, M.S., Kumar, G.R., and Ram Kumar, R.P. (2024). An Insight on Drone Applications in Surveillance Domain. Big Data, Machine Learning, and Applications, Springer. Lecture Notes in Electrical Engineering (LNEE, Volume 1053).10.1007/978-981-99-3481-2_3 ↗
6. Benarbia, T., and Kyamakya, K. (2022). A literature review of drone-based package delivery logistics systems and their implementation feasibility. Sustainability, 14.10.3390/su14010360 ↗
7. UAV-Based Delivery Systems: A Systematic Review, Current Trends, and Research Challenges10.1145/3649224 ↗
8. Drones in last-mile delivery: A systematic review on Efficiency, Accessibility, and Sustainability10.1016/j.trd.2023.103831 ↗
9. The use of drones for the delivery of diagnostic test kits and medical supplies to remote First Nations communities during Covid-1910.1016/j.ajic.2022.03.004 ↗
10. Famili, A., Stavrou, A., Wang, H., Park, J.M., and Gerdes, R. (2024). Securing your airspace: Detection of drones trespassing protected areas. Sensors, 24.10.3390/s24072028 ↗
11. Lykou, G., Moustakas, D., and Gritzalis, D. (2020). Defending airports from uas: A survey on cyber- attacks and counter-drone sensing technologies. Sensors, 20.10.3390/s20123537 ↗
12. A survey on security and privacy issues of UAVs10.1016/j.comnet.2023.109626 ↗
13. Narco drones: Tracing the evolution of cartel aerial tactics in Mexico’s low-intensity conflicts10.1080/09592318.2023.2226382 ↗
14. Seidaliyeva, U., Ilipbayeva, L., Taissariyeva, K., Smailov, N., and Matson, E.T. (2023). Advances and challenges in drone detection and classification techniques: A state-of-the-art review. Sensors, 24.10.3390/s24010125 ↗
15. Machine Learning-Based Drone Detection and Classification: State-of-the-Art in Research10.1109/ACCESS.2019.2942944 ↗
16. Batool, S., Frezza, F., Mangini, F., and Simeoni, P. (2020). Introduction to Radar Scattering Application in Remote Sensing and Diagnostics: Review. Atmosphere, 11.10.3390/atmos11050517 ↗
17. Li, S., Chai, Y., Guo, M., and Liu, Y. (2020, January 27–29). Research on detection method of UAV based on micro-Doppler effect. Proceedings of the 2020 39th Chinese Control Conference (CCC), Shenyang, China.10.23919/CCC50068.2020.9189414 ↗
18. Coluccia, A., Parisi, G., and Fascista, A. (2020). Detection and Classification of Multirotor Drones in Radar Sensor Networks: A Review. Sensors, 20.10.3390/s20154172 ↗
19. Time–Frequency Multiscale Convolutional Neural Network for RF-Based Drone Detection and Identification10.1109/LSENS.2023.3289145 ↗
20. Aouladhadj, D., Kpre, E., Deniau, V., Kharchouf, A., Gransart, C., and Gaquière, C. (2023). Drone Detection and Tracking Using RF Identification Signals. Sensors, 23.10.3390/s23177650 ↗
21. Deep Learning-based drone acoustic event detection system for microphone arrays10.1007/s11042-023-17477-1 ↗
22. Drone Detection and Localization Using Enhanced Fiber-Optic Acoustic Sensor and Distributed Acoustic Sensing Technology10.1109/JLT.2022.3208451 ↗
23. Wang, B., Li, Q., Mao, Q., Wang, J., Chen, C.P., Shangguan, A., and Zhang, H. (2024). A Survey on Vision-Based Anti Unmanned Aerial Vehicles Methods. Drones, 8.10.3390/drones8090518 ↗
24. Drone Detection Using YOLOv510.3390/eng4010025 ↗
25. The Drone-vs-Bird Detection Grand Challenge at ICASSP 2023: A Review of Methods and Results10.1109/OJSP.2024.3379073 ↗
26. Rizzoli, G., Barbato, F., Caligiuri, M., and Zanuttigh, P. (2023, January 2–6). SynDrone—Multi-Modal UAV Dataset for Urban Scenarios. Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) Workshops, Paris, France.10.1109/ICCVW60793.2023.00235 ↗
27. Steininger, D., Widhalm, V., Simon, J., Kriegler, A., and Sulzbachner, C. (2021, January 11–17). The Aircraft Context Dataset: Understanding and Optimizing Data Variability in Aerial Domains. Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) Workshops, Montreal, BC, Canada.10.1109/ICCVW54120.2021.00426 ↗
28. CNN-Based UAV Detection and Classification Using Sensor Fusion10.1109/ACCESS.2023.3293124 ↗
29. Mehta, V., Dadboud, F., Bolic, M., and Mantegh, I. (2023, January 18–20). A Deep Learning Approach for Drone Detection and Classification Using Radar and Camera Sensor Fusion. Proceedings of the 2023 IEEE Sensors Applications Symposium (SAS), Ottawa, ON, Canada.10.1109/SAS58821.2023.10254123 ↗
30. Dudczyk, J., Czyba, R., and Skrzypczyk, K. (2022). Multi-Sensory Data Fusion in Terms of UAV Detection in 3D Space. Sensors, 22.10.3390/s22124323 ↗
31. (2023, January 18–22). CVPR 2023 Anti-UAV Challenge Dataset. Proceedings of the The 3rd Anti-UAV Workshop & Challenge, Vancouver, BC, Canada.
32. Anti-UAV: A large-scale benchmark for vision-based UAV tracking10.1109/TMM.2021.3128047 ↗
33. Zhao, J., Wang, G., Li, J., Jin, L., Fan, N., Wang, M., Wang, X., Yong, T., Deng, Y., and Guo, Y. (2021). The 2nd anti-UAV workshop & challenge: Methods and results. arXiv.
34. Huang, S., Jiang, Y., and Jiang, Y. (2024). Design of Target Detection and Tracking System for Sports Video. IEEE Access.10.1109/ACCESS.2020.3011204 ↗
35. Zhu, H., Wei, H., Li, B., Yuan, X., and Kehtarnavaz, N. (2020). A review of video object detection: Datasets, metrics and methods. Appl. Sci., 10.10.3390/app10217834 ↗
36. New Generation Deep Learning for Video Object Detection: A Survey10.1109/TNNLS.2021.3053249 ↗
37. Mayer, C., Danelljan, M., Bhat, G., Paul, M., Paudel, D.P., Yu, F., and Van Gool, L. (2022, January 18–24). Transforming Model Prediction for Tracking. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.10.1109/CVPR52688.2022.00853 ↗
38. Transformers in Single Object Tracking: An Experimental Survey10.1109/ACCESS.2023.3298440 ↗
39. Deep learning for unmanned aerial vehicles detection: A review10.1016/j.cosrev.2023.100614 ↗
40. High-Performance Transformer Tracking
41. Wang, N., Zhou, W., Wang, J., and Li, H. (2021, January 20–25). Transformer Meets Tracker: Exploiting Temporal Context for Robust Visual Tracking. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA.10.1109/CVPR46437.2021.00162 ↗
42. Drone Detection Through CCTV10.22214/ijraset.2024.57107 ↗
43. Bhagat, P.N., Dasarwar, H.V., Sayyad, M.R., and Shelake, S.D. (2024). Drone Detection and Identification Using Artificial Intelligence. Int. J. Adv. Res. Sci. Commun. Technol.
44. Exploring thermal images for object detection in underexposure regions for autonomous driving10.1016/j.asoc.2022.108793 ↗
45. Thermal Object Detection in Difficult Weather Conditions Using YOLO10.1109/ACCESS.2020.3007481 ↗
46. Deep Learning-Based Thermal Image Reconstruction and Object Detection10.1109/ACCESS.2020.3048437 ↗
47. Enhancing Autonomous Driving By Exploiting Thermal Object Detection Through Feature Fusion
48. Object detection from UAV thermal infrared images and videos using YOLO models
49. Robust UAV detection based on saliency cues and magnified features on thermal images10.1007/s11042-022-14271-3 ↗
50. UAV-Based Human Detection With Visible-Thermal Fused YOLOv5 Network10.1109/TII.2023.3310792 ↗
51. Yu, Q., Ma, Y., He, J., Yang, D., and Zhang, T. (2023, January 17–24). A unified transformer based tracker for anti-uav tracking. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, BC, Canada.10.1109/CVPRW59228.2023.00305 ↗
52. Milosevic, N. (2020). Introduction to Convolutional Neural Networks, Apress.10.1007/978-1-4842-5648-0_12 ↗
53. An Introduction to Convolutional Neural Networks10.22214/ijraset.2022.47789 ↗
54. Wang, C.Y., Yeh, I.H., and Liao, H.Y.M. (2024). Yolov9: Learning what you want to learn using programmable gradient information. arXiv.10.1007/978-3-031-72751-1_1 ↗
55. Xu, W., Zhu, D., Deng, R., Yung, K., and Ip, A.W.H. (2024). Violence-YOLO: Enhanced GELAN Algorithm for Violence Detection. Appl. Sci., 14.10.3390/app14156712 ↗
56. Balakrishnan, T., and Sengar, S.S. (2024). RepVGG-GELAN: Enhanced GELAN with VGG-STYLE ConvNets for Brain Tumour Detection. arXiv.
57. A Survey on Vision Transformer10.1109/TPAMI.2022.3152247 ↗
58. Transformers in Vision: A Survey10.1145/3505244 ↗
59. Carion, N., Massa, F., Synnaeve, G., Usunier, N., Kirillov, A., and Zagoruyko, S. (2020). End-to-End Object Detection with Transformers. Computer Vision—ECCV 2020, Proceedings of the 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part I, Springer.10.1007/978-3-030-58452-8_13 ↗
60. Li, Y., Mao, H., Girshick, R., and He, K. (2022). Exploring Plain Vision Transformer Backbones for Object Detection. Computer Vision—ECCV 2022, Proceedings of the 17th European Conference, Tel Aviv, Israel, October 23–27, 2022, Proceedings, Part IX, Springer.10.1007/978-3-031-20077-9_17 ↗
61. Yang, X., Wang, G., Hu, W., Gao, J., Lin, S., Li, L., Gao, K., and Wang, Y. (2023, January 17–24). Video Tiny-Object Detection Guided by the Spatial-Temporal Motion Information. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, Vancouver, BC, Canada.10.1109/CVPRW59228.2023.00307 ↗
62. Caron, M., Touvron, H., Misra, I., Jegou, H., Mairal, J., Bojanowski, P., and Joulin, A. (2021, January 10–17). Emerging Properties in Self-Supervised Vision Transformers. Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision (ICCV), Montreal, QC, Canada.10.1109/ICCV48922.2021.00951 ↗
63. Tang, Z., Gao, Y., Xun, Z., Peng, F., Sun, Y., Liu, S., and Li, B. (2023, January 17–24). Strong Detector with Simple Tracker. Proceedings of the 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Vancouver, BC, Canada.10.1109/CVPRW59228.2023.00306 ↗
64. Biró, A., Jánosi-Rancz, K.T., Szilágyi, L., Cuesta-Vargas, A.I., Martín-Martín, J., and Szilágyi, S.M. (2022). Visual Object Detection with DETR to Support Video-Diagnosis Using Conference Tools. Appl. Sci., 12.10.3390/app12125977 ↗
65. Hardt, M., Recht, B., and Singer, Y. (2016, January 20–22). Train faster, generalize better: Stability of stochastic gradient descent. Proceedings of the International Conference on Machine Learning (PMLR 48), New York, NY, USA.
66. Choi, D. (2019). On empirical comparisons of optimizers for deep learning. arXiv.