Improved YOLOv7 Algorithm for Small Object Detection in Unmanned Aerial Vehicle Image Scenarios
Source abstract
Abstract
Object detection in unmanned aerial vehicle (UAV) images has become a popular research topic in recent years. However, UAV images are captured from high altitudes with a large proportion of small objects and dense object regions, posing a significant challenge to small object detection. To solve this issue, we propose an efficient YOLOv7-UAV algorithm in which a low-level prediction head (P2) is added to detect small objects from the shallow feature map, and a deep-level prediction head (P5) is removed to reduce the effect of excessive down-sampling. Furthermore, we modify the bidirectional feature pyramid network (BiFPN) structure with a weighted cross-level connection to enhance the fusion effectiveness of multi-scale feature maps in UAV images. To mitigate the mismatch between the prediction box and ground-truth box, the SCYLLA-IoU (SIoU) function is employed in the regression loss to accelerate the training convergence process. Moreover, the proposed YOLOv7-UAV algorithm has been quantified and compiled in the Vitis-AI development environment and validated in terms of power consumption and hardware resources on the FPGA platform. The experiments show that the resource consumption of YOLOv7-UAV is reduced by 28%, the mAP is improved by 3.9% compared to YOLOv7, and the FPGA implementation improves the energy efficiency by 12 times compared to the GPU.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Advanced Neural Network Applications · Video Surveillance and Tracking Methods · Robotics and Sensor-Based Localization
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
College of Computer Science, Chengdu University, Chengdu 610100, China
Guangdong Provincial Key Laboratory of Future Networks of Intelligence, The Chinese University of Hong Kong, Shenzhen 518172, China
School of Information Engineering, Southwest University of Science and Technology, Mianyang 621000, China
Department of Information and Communication Engineering, Tongji University, Shanghai 201804, China
Funding
National Natural Science Foundation of China · Natural Science Foundation of Sichuan Province · Shanghai Sailing Program · Fundamental Research Funds for the Central Universities · Guangdong Provincial Key Laboratory of Future Networks of Intelligence, the Chinese University of Hong Kong, Shenzhen · Key Laboratory of Medicinal and Edible Plant Resources Development of Sichuan Education Department, the Chengdu University
References (43)
Show source-provided references
1. Object detection in 20 years: A survey10.1109/JPROC.2023.3238524 ↗
2. A survey of modern deep learning-based object detection models10.1016/j.dsp.2022.103514 ↗
3. Object detection in traffic videos: A survey10.1109/TITS.2023.3258683 ↗
4. Visual detection and tracking algorithms for minimally invasive surgical instruments: A comprehensive review of the state-of-the-art10.1016/j.robot.2021.103945 ↗
5. Zhong, L.T., Zhang, X.Q., Ran, L.Y., Han, Y.M., and Chu, H.Y. (2023, January 12–14). Visual SLAM for dynamic environments based on static key-points detection. Proceedings of the International Conference on Virtual Reality (ICVR), Xianyang, China.10.1109/ICVR57957.2023.10169198 ↗
6. Zhou, Z.Y., Zhang, X.Q., Ran, L.Y., Han, Y.M., and Chu, H.Y. (2023, January 12–14). DSC-GraspNet: A lightweight convolutional neural network for robotic grasp detection. Proceedings of the International Conference on Virtual Reality (ICVR), Xianyang, China.10.1109/ICVR57957.2023.10169448 ↗
7. Positioning optimization for sum-rate maximization in UAV-enabled interference channel10.1109/LSP.2019.2934579 ↗
8. Machine learning applications in internet-of-drones: Systematic review, recent deployments, and open issues10.1145/3571728 ↗
9. Deep learning for unmanned aerial vehicle-based object detection and tracking: A survey10.1109/MGRS.2021.3115137 ↗
10. Detection and tracking meet drones challenge10.1109/TPAMI.2021.3119563 ↗
11. Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C.Y., and Berg, A.C. (2016, January 8–16). SSD: Single shot multibox detector. Proceedings of the European Conference on Computer Vision (ECCV), Amsterdam, The Netherlands.10.1007/978-3-319-46448-0_2 ↗
12. Wang, C.-Y., Bochkovskiy, A., and Liao, H.-Y.M. (2021, January 19–25). Scaled-YOLOv4: Scaling cross stage partial network. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Nashville, TN, USA.10.1109/CVPR46437.2021.01283 ↗
13. Xie, X.Z., and Lu, G. (2021, January 24–26). A research of object detection on UAVs aerial images. Proceedings of the International Conference on Big Data & Artificial Intelligence & Software Engineering (ICBASE), Zhuhai, China.10.1109/ICBASE53849.2021.00070 ↗
14. Reinforcement learning-based age of information optimization in UAV-enabled communication system
15. Wang, L., Zhou, H., Bian, C.J., Jiang, K., and Cheng, X.L. (2022). Hardware acceleration and implementation of YOLOX-s for on-orbit FPGA. Electronic, 11.10.3390/electronics11213473 ↗
16. Wei, Y.K., Li, J.H., Duan, W.W., Li, X.M., Zhang, X.Q., and Huang, Y. (2023, January 19–22). YOLOv7-UAV: Improved YOLOv7 algorithm for small object detection in UAV image scenarios. Proceedings of the International Conference on Artificial Intelligence of Things and Systems (AIoTSys), Xi’an, China.10.1109/AIoTSys58602.2023.00031 ↗
17. Statistical analysis of design aspects of various YOLO-based deep learning models for object detection10.1007/s44196-023-00302-w ↗
18. Girshick, R. (2015, January 13–16). Fast R-CNN. Proceedings of the IEEE International Conference on Computer Vision (ICCV), Santiago, Chile.10.1109/ICCV.2015.169 ↗
19. Mask R-CNN10.1109/TPAMI.2018.2844175 ↗
20. Artificial intelligence-based surveillance system for railway crossing traffic10.1109/JSEN.2020.3031861 ↗
21. Redmon, J., Divvala, S., Girshick, R., and Farhadi, A. (2016, January 26–31). You only look once: Unified, real-time object detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, TX, USA.10.1109/CVPR.2016.91 ↗
22. Ganesh, P., Chen, Y., Yang, Y., Chen, D., and Winslett, M. (2022, January 3–8). YOLO-ReT: Towards high accuracy real-time object detection on edge GPUs. Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), Waikoloa, HI, USA.10.1109/WACV51458.2022.00138 ↗
23. Edge YOLO: Real-time intelligent object detection system based on edge-cloud cooperation in autonomous vehicles10.1109/TITS.2022.3158253 ↗
24. Wang, C.-Y., Bochkovskiy, A., and Liao, H.-Y.M. (2023, January 18–22). YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Vancouver, BC, Canada.10.1109/CVPR52729.2023.00721 ↗
25. Zhu, X.K., Lyu, S.C., Wang, X., and Zhao, Q. (2021, January 11–17). TPH-YOLOv5: Improved YOLOv5 based on transformer prediction head for object detection on drone-captured scenarios. Proceedings of the IEEE/CVF International Conference on Computer Vision Workshops (ICCVW), Montreal, ON, Canada.10.1109/ICCVW54120.2021.00312 ↗
26. Cross-layer attention network for small object detection in remote sensing imagery10.1109/JSTARS.2020.3046482 ↗
27. Extended feature pyramid network for small object detection10.1109/TMM.2021.3074273 ↗
28. Chen, Z., Liu, C., Filaretov, V.F., and Yukhimets, D.A. (2023). Multi-scale ship detection algorithm based on YOLOv7 for complex scene SAR images. Remote Sens., 15.10.3390/rs15082071 ↗
29. Redmon, J., and Farhadi, A. (2018). YOLOv3: An incremental improvement. arXiv.
30. Yang, J.L., Yang, H., Wang, F., and Chen, X. (2022, January 15–18). A modified YOLOv5 for object detection in UAV-captured scenarios. Proceedings of the IEEE International Conference on Networking, Sensing and Control (ICNSC), Shanghai, China.10.1109/ICNSC55942.2022.10004160 ↗
31. Few-shot learning for small impurities in tobacco stems with improved YOLOv710.1109/ACCESS.2023.3275023 ↗
32. Arthur, D., and Vassilvitskii, S. (2007, January 7–9). K-means++: The advantages of careful seeding. Proceedings of the Annual ACM-SIAM Symposium on Discrete Algorithms, New Orleans, LA, USA.
33. Du, D., Zhu, P., Wen, L., Bian, X., Lin, H., Hu, Q., Peng, T., Zheng, J., Wang, X., and Zhang, Y. (2019, January 27–28). VisDrone-DET2019: The vision meets drone object detection in image challenge results. Proceedings of the IEEE/CVF International Conference on Computer Vision Workshops (ICCVW), Seoul, Republic of Korea.
34. Lin, T.-Y., Dollar, P., Girshick, R., He, K.M., Hariharan, B., and Belongie, S. (2017, January 21–26). Feature pyramid networks for object detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.10.1109/CVPR.2017.106 ↗
35. Liu, S., Qi, L., Qin, H.F., Shi, J.P., and Jia, J.Y. (2018, January 18–22). Path aggregation network for instance segmentation. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Salt Lake City, UT, USA.10.1109/CVPR.2018.00913 ↗
36. Tan, M.X., Pang, R.M., and Le, Q.V. (2020, January 14–19). EfficientDet: Scalable and efficient object detection. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.10.1109/CVPR42600.2020.01079 ↗
37. Enhancing geometric factors in model learning and inference for object detection and instance Segmentation10.1109/TCYB.2021.3095305 ↗
38. Gevorgyan, Z. (2022). SIoU loss: More powerful learning for bounding box regression. arXiv.
39. Xilinx (2023, December 27). Vitis AI User Guide (UG1414). Available online: https://docs.xilinx.com/r/2.0-English/ug1414-vitis-ai.
40. Xilinx (2023, December 27). DPUCZDX8G for Zynq UltraScale+ MPSoCs Product Guide (PG338). Available online: https://docs.xilinx.com/r/4.0-English/pg338-dpu.
41. Xilinx (2023, December 27). Zynq UltraScale+ MPSoC Data Sheet: Overview (DS891). Available online: https://docs.xilinx.com/v/u/en-US/ds891-zynq-ultrascale-plus-overview.
42. Xilinx (2023, December 27). Vivado Design Suite User Guide: Getting Started (UG910). Available online: https://docs.xilinx.com/r/en-US/ug910-vivado-getting-started.
43. Xilinx (2023, December 27). Petalinux Tools Documentation: Reference Guide (UG1144). Available online: https://docs.xilinx.com/r/en-US/ug1144-petalinux-tools-reference-guide.