A Semantic Segmentation Dataset and Real-Time Localization Model for Anti-UAV Applications
Source abstract
Abstract
With the rapid development of the unmanned aerial vehicle (UAV) industry and applications, the integration of UAVs into daily life has increased significantly. However, this growing presence raises security concerns, leading to the emergence of anti-UAV technologies. Most existing anti-UAV systems rely on object detection techniques. Yet, these methods often struggle to detect small-sized UAVs accurately. Semantic segmentation, which predicts object locations at the pixel level, offers improved localization for such small targets. Due to the lack of existing datasets for anti-UAV semantic segmentation, we propose a new dataset comprising both infrared (IR) and visible light images. Our dataset includes a total of 605,045 paired UAV images and corresponding segmentation masks. To enhance object diversity and improve model robustness, the dataset integrates multiple existing sources. In addition to the dataset, we evaluate the performance of several baseline models on the semantic segmentation task. We also propose a lightweight model to demonstrate the feasibility of real-time UAV localization using semantic segmentation on VL and IR data.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Video Surveillance and Tracking Methods · Infrared Target Detection Methodologies · Robotics and Sensor-Based Localization
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
School of Computer Science, Kookmin University, 77 Jeongneung-ro, Seongbuk-gu, Seoul 02707, Republic of Korea
Department of Electrical Engineering, Kookmin University, 77 Jeongneung-ro, Seongbuk-gu, Seoul 02707, Republic of Korea
Funding
Korea Research Institute for defense Technology Planning and advancement (KRIT) · Korea government (DAPA (Defense Acquisition Program Administration))
References (46)
Show source-provided references
1. UAV-Based Delivery Systems: A Systematic Review, Current Trends, and Research Challenges10.1145/3649224 ↗
2. Liu, Z., Chen, C., Huang, Z., Chang, Y.C., Liu, L., and Pei, Q. (2024). A Low-Cost and Lightweight Real-Time Object-Detection Method Based on UAV Remote Sensing in Transportation Systems. Remote Sens., 16.10.3390/rs16193712 ↗
3. Urban Air Mobility for Last-Mile Transportation: A Review10.3390/vehicles6030066 ↗
4. Design, Multiperspective Investigations, and Performance Analysis of Multirotor Unmanned Aerial Vehicle for Precision Farming
5. Zhang, Y., Zhao, Q., Mao, P., Bai, Q., Li, F., and Pavlova, S. (2024). Design and Control of an Ultra-Low-Cost Logistic Delivery Fixed-Wing UAV. Appl. Sci., 14.10.3390/app14114358 ↗
6. Aggarwal, V., Kaushik, A.R., Jutla, C., and Ratha, N. (2024, January 25–27). Enhancing Privacy and Security of Autonomous UAV Navigation. Proceedings of the 2024 IEEE Conference on Artificial Intelligence (CAI), Singapore.10.1109/CAI59869.2024.00103 ↗
7. Shah, I.A. (2024). Privacy and Security Challenges in Unmanned Aerial Vehicles (UAVs). Cybersecurity in the Transportation Industry, John Wiley & Sons, Ltd.. Chapter 5.10.1002/9781394204472.ch5 ↗
8. Security and privacy risks in drone-based last mile delivery10.1080/0960085X.2023.2214744 ↗
9. Wang, B., Li, Q., Mao, Q., Wang, J., Chen, C.L.P., Shangguan, A., and Zhang, H. (2024). A Survey on Vision-Based Anti Unmanned Aerial Vehicles Methods. Drones, 8.10.3390/drones8090518 ↗
10. Radar-Based UAV Swarm Surveillance Based on a Two-Stage Wave Path Difference Estimation Method10.1109/JSEN.2022.3141879 ↗
11. RF-Based UAV Detection and Identification Enhanced by Machine Learning Approach10.1109/ACCESS.2024.3502754 ↗
12. AlKhonaini, A., Sheltami, T., Mahmoud, A., and Imam, M. (2024). UAV Detection Using Reinforcement Learning. Sensors, 24.10.3390/s24061870 ↗
13. RF/WiFi-based UAV surveillance systems: A systematic literature review10.1016/j.iot.2024.101201 ↗
14. Deep Learning-based drone acoustic event detection system for microphone arrays10.1007/s11042-023-17477-1 ↗
15. Dewangan, V., Saxena, A., Thakur, R., and Tripathi, S. (2023). Application of Image Processing Techniques for UAV Detection Using Deep Learning and Distance-Wise Analysis. Drones, 7.10.3390/drones7030174 ↗
16. FPGA-SoC implementation of YOLOv4 for flying-object detection10.1007/s11554-024-01440-w ↗
17. Yu, Q., Ma, Y., He, J., Yang, D., and Zhang, T. (2023, January 17–24). A Unified Transformer Based Tracker for Anti-UAV Tracking. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, Vancouver, BC, Canada.10.1109/CVPRW59228.2023.00305 ↗
18. Redmon, J., Divvala, S., Girshick, R., and Farhadi, A. (2016, January 27–30). You Only Look Once: Unified, Real-Time Object Detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.10.1109/CVPR.2016.91 ↗
19. Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C.Y., and Berg, A.C. (2016). SSD: Single Shot MultiBox Detector. Computer Vision—ECCV 2016, Springer International Publishing.10.1007/978-3-319-46448-0_2 ↗
20. Ren, S., He, K., Girshick, R., and Sun, J. (2016). Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks. arXiv.10.1109/TPAMI.2016.2577031 ↗
21. Isaac-Medina, B.K.S., Poyser, M., Organisciak, D., Willcocks, C.G., Breckon, T.P., and Shum, H.P.H. (2021, January 11–17). Unmanned Aerial Vehicle Visual Detection and Tracking using Deep Neural Networks: A Performance Benchmark. Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW), Montreal, BC, Canada.10.1109/ICCVW54120.2021.00142 ↗
22. Anti-UAV: A Large-Scale Benchmark for Vision-Based UAV Tracking10.1109/TMM.2021.3128047 ↗
23. Vision-Based Anti-UAV Detection and Tracking10.1109/TITS.2022.3177627 ↗
24. Coluccia, A., Fascista, A., Schumann, A., Sommer, L., Dimou, A., Zarpalas, D., Akyon, F.C., Eryuksel, O., Ozfuttu, K.A., and Altinuc, S.O. (2021, January 16–19). Drone-vs-Bird Detection Challenge at IEEE AVSS2021. Proceedings of the 2021 17th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS), Washington, DC, USA.10.1109/AVSS52988.2021.9663844 ↗
25. Isaac-Medina, B.K.S., Poyser, M., Organisciak, D., Willcocks, C.G., Breckon, T.P., and Shum, H.P.H. (2021). Unmanned Aerial Vehicle Visual Detection and Tracking using Deep Neural Networks: A Performance Benchmark. arXiv.10.1109/ICCVW54120.2021.00142 ↗
26. Coluccia, A., Fascista, A., Schumann, A., Sommer, L., Dimou, A., Zarpalas, D., Méndez, M., de la Iglesia, D., González, I., and Mercier, J.P. (2021). Drone vs. Bird Detection: Deep Learning Algorithms and Results from a Grand Challenge. Sensors, 21.10.3390/s21082824 ↗
27. Redmon, J., and Farhadi, A. (2018). YOLOv3: An Incremental Improvement. arXiv.
28. Carion, N., Massa, F., Synnaeve, G., Usunier, N., Kirillov, A., and Zagoruyko, S. (2020). End-to-End Object Detection with Transformers. arXiv.10.1007/978-3-030-58452-8_13 ↗
29. Bewley, A., Ge, Z., Ott, L., Ramos, F., and Upcroft, B. (2016, January 25–28). Simple online and realtime tracking. Proceedings of the 2016 IEEE International Conference on Image Processing (ICIP), Phoenix, AZ, USA.10.1109/ICIP.2016.7533003 ↗
30. Wojke, N., Bewley, A., and Paulus, D. (2017, January 17–20). Simple Online and Realtime Tracking with a Deep Association Metric. Proceedings of the 2017 IEEE International Conference on Image Processing (ICIP), Beijing, China.10.1109/ICIP.2017.8296962 ↗
31. Bergmann, P., Meinhardt, T., and Leal-Taixé, L. (November, January 27). Tracking Without Bells and Whistles. Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision (ICCV), Seoul, Republic of Korea.10.1109/ICCV.2019.00103 ↗
32. Huang, B., Chen, J., Xu, T., Wang, Y., Jiang, S., Wang, Y., Wang, L., and Li, J. (2021, January 11–17). SiamSTA: Spatio-Temporal Attention based Siamese Tracker for Tracking UAVs. Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW), Montreal, BC, Canada.10.1109/ICCVW54120.2021.00140 ↗
33. Cheng, F., Liang, Z., Peng, G., Liu, S., Li, S., and Ji, M. (2022). An Anti-UAV Long-Term Tracking Method with Hybrid Attention Mechanism and Hierarchical Discriminator. Sensors, 22.10.3390/s22103701 ↗
34. Fang, A., Feng, S., Liang, B., and Jiang, J. (2024). Real-Time Detection of Unauthorized Unmanned Aerial Vehicles Using SEB-YOLOv8s. Sensors, 24.10.3390/s24123915 ↗
35. Dadboud, F., Patel, V., Mehta, V., Bolic, M., and Mantegh, I. (2021, January 16–19). Single-Stage UAV Detection and Classification with YOLOV5: Mosaic Data Augmentation and PANet. Proceedings of the 2021 17th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS), Washington, DC, USA.10.1109/AVSS52988.2021.9663841 ↗
36. Ge, Z., Liu, S., Wang, F., Li, Z., and Sun, J. (2021). YOLOX: Exceeding YOLO Series in 2021. arXiv.
37. ST-Trans: Spatial-Temporal Transformer for Infrared Small Target Detection in Sequential Images10.1109/TGRS.2024.3355947 ↗
38. Kirillov, A., Mintun, E., Ravi, N., Mao, H., Rolland, C., Gustafson, L., Xiao, T., Whitehead, S., Berg, A.C., and Lo, W.Y. (2023). Segment Anything. arXiv.10.1109/ICCV51070.2023.00371 ↗
39. (2025, January 15). Ultralytics. Home. Available online: https://docs.ultralytics.com/.
40. Ravi, N., Gabeur, V., Hu, Y.T., Hu, R., Ryali, C., Ma, T., Khedr, H., Rädle, R., Rolland, C., and Gustafson, L. (2024). SAM 2: Segment Anything in Images and Videos. arXiv.
41. Navab, N., Hornegger, J., Wells, W.M., and Frangi, A.F. (2015). U-Net: Convolutional Networks for Biomedical Image Segmentation. Medical Image Computing and Computer-Assisted Intervention—MICCAI 2015, Springer.10.1007/978-3-319-24571-3 ↗
42. Mei, Y., Fan, Y., Zhang, Y., Yu, J., Zhou, Y., Liu, D., Fu, Y., Huang, T.S., and Shi, H. (2020). Pyramid Attention Networks for Image Restoration. arXiv.
43. Zhao, H., Shi, J., Qi, X., Wang, X., and Jia, J. (2017). Pyramid Scene Parsing Network. arXiv.10.1109/CVPR.2017.660 ↗
44. Chen, J., Lu, Y., Yu, Q., Luo, X., Adeli, E., Wang, Y., Lu, L., Yuille, A.L., and Zhou, Y. (2021). TransUNet: Transformers Make Strong Encoders for Medical Image Segmentation. arXiv.
45. (2025, May 28). Welcome to Segmentation Models Pytorch Documentation!. Available online: https://segmentation-modelspytorch.readthedocs.io/en/latest/.
46. Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., and Gelly, S. (2021). An Image is Worth 16 × 16 Words: Transformers for Image Recognition at Scale. arXiv.