YOLO-SE: Improved YOLOv8 for Remote Sensing Object Detection and Recognition
Source abstract
Abstract
Object detection remains a pivotal aspect of remote sensing image analysis, and recent strides in Earth observation technology coupled with convolutional neural networks (CNNs) have propelled the field forward. Despite advancements, challenges persist, especially in detecting objects across diverse scales and pinpointing small-sized targets. This paper introduces YOLO-SE, a novel YOLOv8-based network that innovatively addresses these challenges. First, the introduction of a lightweight convolution SEConv in lieu of standard convolutions reduces the network’s parameter count, thereby expediting the detection process. To tackle multi-scale object detection, the paper proposes the SEF module, an enhancement based on SEConv. Second, an ingenious Efficient Multi-Scale Attention (EMA) mechanism is integrated into the network, forming the SPPFE module. This addition augments the network’s feature extraction capabilities, adeptly handling challenges in multi-scale object detection. Furthermore, a dedicated prediction head for tiny object detection is incorporated, and the original detection head is replaced by a transformer prediction head. To address adverse gradients stemming from low-quality instances in the target detection training dataset, the paper introduces the Wise-IoU bounding box loss function. YOLO-SE showcases remarkable performance, achieving an average precision at IoU threshold 0.5 (AP50) of 86.5% on the optical remote sensing dataset SIMD. This represents a noteworthy 2.1% improvement over YOLOv8 and YOLO-SE outperforms the state-of-the-art model by 0.91%. In further validation, experiments on the NWPU VHR-10 dataset demonstrated YOLO-SE’s superiority with an accuracy of 94.9%, surpassing that of YOLOv8 by 2.6%. The proposed advancements position YOLO-SE as a compelling solution in the realm of deep learning-based remote sensing image object detection.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Advanced Neural Network Applications · Remote-Sensing Image Classification · Advanced Image and Video Retrieval Techniques
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
College of Marine Science and Technology, China University of Geosciences, Wuhan 430074, China
References (48)
Show source-provided references
1. Mao, M., Zhao, H., Tang, G., and Ren, J. (2023). In-Season Crop Type Detection by Combing Sentinel-1A and Sentinel-2 Imagery Based on the CNN Model. Agronomy, 13.10.3390/agronomy13071723 ↗
2. Cardama, F.J., Heras, D.B., and Argüello, F. (2023). Consensus Techniques for Unsupervised Binary Change Detection Using Multi-Scale Segmentation Detectors for Land Cover Vegetation Images. Remote Sens., 15.10.3390/rs15112889 ↗
3. Weakly supervised learning based on coupled convolutional neural networks for aircraft detection10.1109/TGRS.2016.2569141 ↗
4. Tang, T., Zhou, S., Deng, Z., Zou, H., and Lei, L. (2017). Vehicle detection in aerial images based on region convolutional neural networks and hard negative example mining. Sensors, 17.10.3390/s17020336 ↗
5. Zheng, Z., Lei, L., Sun, H., and Kuang, G. (2020, January 10–12). A review of remote sensing image object detection algorithms based on deep learning. Proceedings of the 2020 IEEE 5th International Conference on Image, Vision and Computing (ICIVC), Beijing, China.10.1109/ICIVC50857.2020.9177453 ↗
6. Learning spectral-spatial-temporal features via a recurrent convolutional neural network for change detection in multispectral imagery10.1109/TGRS.2018.2863224 ↗
7. FCD-R2U-net: Forest change detection in bi-temporal satellite images using the recurrent residual-based U-net10.1007/s12145-022-00885-6 ↗
8. Purkait, P., Zhao, C., and Zach, C. (2017). SPP-Net: Deep absolute pose regression with synthetic views. arXiv.
9. Gkioxari, G., Hariharan, B., Girshick, R., and Malik, J. (2014). R-cnns for pose estimation and action detection. arXiv.
10. Ren, S., He, K., Girshick, R., and Sun, J. (2015, January 7–12). Faster r-cnn: Towards real-time object detection with region proposal networks. Proceedings of the Advances in Neural Information Processing Systems 28 (NIPS 2015), Montreal, QC, Canada.
11. Redmon, J., Divvala, S., Girshick, R., and Farhadi, A. (July, January 26). You only look once: Unified, real-time object detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Paradise, NV, USA.
12. Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C.Y., and Berg, A.C. (2016, January 11–14). Ssd: Single shot multibox detector. Proceedings of the Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands. Part I 14.10.1007/978-3-319-46448-0_2 ↗
13. Lin, T.Y., Goyal, P., Girshick, R., He, K., and Dollár, P. (2017, January 22–29). Focal loss for dense object detection. Proceedings of the IEEE International Conference On Computer Vision, Venice, Italy.10.1109/ICCV.2017.324 ↗
14. Law, H., and Deng, J. (2018, January 8–14). Cornernet: Detecting objects as paired keypoints. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.10.1007/978-3-030-01264-9_45 ↗
15. Zhang, S., Wen, L., Bian, X., Lei, Z., and Li, S.Z. (2018, January 18–23). Single-shot refinement neural network for object detection. Proceedings of the 2018 IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00442 ↗
16. Ma, J., Hu, Z., Shao, Q., Wang, Y., Zhou, Y., Liu, J., and Liu, S. (2022). Detection of large herbivores in uav images: A new method for small target recognition in large-scale images. Diversity, 14.10.3390/d14080624 ↗
17. PBNet: Part-based convolutional neural network for complex composite object detection in remote sensing imagery10.1016/j.isprsjprs.2020.12.015 ↗
18. Lai, H., Chen, L., Liu, W., Yan, Z., and Ye, S. (2023). STC-YOLO: Small object detection network for traffic signs in complex environments. Sensors, 23.10.3390/s23115307 ↗
19. Han, K., Wang, Y., Tian, Q., Guo, J., and Xu, C. (2020, January 13–19). Ghostnet: More features from cheap operations. Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.10.1109/CVPR42600.2020.00165 ↗
20. Lin, W., Wu, Z., Chen, J., Huang, J., and Jin, L. (2023). Scale-Aware Modulation Meet Transformer. arXiv.10.1109/ICCV51070.2023.00553 ↗
21. Wan, D., Lu, R., Wang, S., Shen, S., Xu, T., and Lang, X. (2023). YOLO-HR: Improved YOLOv5 for Object Detection in High-Resolution Optical Remote Sensing Images. Remote Sens., 15.10.3390/rs15030614 ↗
22. Xu, D., and Wu, Y. (2020). Improved YOLO-V3 with DenseNet for multi-scale remote sensing target detection. Sensors, 20.10.3390/s20154276 ↗
23. Cao, J., Bao, W., Shang, H., Yuan, M., and Cheng, Q. (2023). GCL-YOLO: A GhostConv-Based Lightweight YOLO Network for UAV Small Object Detection. Remote Sens., 15.10.3390/rs15204932 ↗
24. Bahdanau, D., Cho, K., and Bengio, Y. (2014). Neural machine translation by jointly learning to align and translate. arXiv.
25. Hu, J., Shen, L., and Sun, G. (2018, January 18–23). Squeeze-and-excitation networks. Proceedings of the 2018 IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00745 ↗
26. Dai, J., Qi, H., Xiong, Y., Li, Y., Zhang, G., Hu, H., and Wei, Y. (2017, January 22–29). Deformable convolutional networks. Proceedings of the 2017 IEEE International Conference on Computer Vision, Venice, Italy.10.1109/ICCV.2017.89 ↗
27. Woo, S., Park, J., Lee, J.Y., and Kweon, I.S. (2018, January 8–14). Cbam: Convolutional block attention module. Proceedings of the 2018 European Conference on Computer Vision (ECCV), Munich, Germany.10.1007/978-3-030-01234-2_1 ↗
28. Zhou, T., Wang, S., Zhou, Y., Yao, Y., Li, J., and Shao, L. (2020, January 7–12). Motion-attentive transition for zero-shot video object segmentation. Proceedings of the 2020 AAAI Conference on Artificial Intelligence, New York, NY, USA.
29. Zhou, T., Zhang, M., Zhao, F., and Li, J. (2022, January 18–24). Regional semantic contrast and aggregation for weakly supervised semantic segmentation. Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.10.1109/CVPR52688.2022.00426 ↗
30. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., and Polosukhin, I. (2017, January 10–16). Attention is all you need. Proceedings of the Advances in Neural Information Processing Systems 30 (NIPS 2017), Long Beach, CA, USA.
31. A survey on vision transformer10.1109/TPAMI.2022.3152247 ↗
32. Chen, M., Radford, A., Child, R., Wu, J., Jun, H., Luan, D., and Sutskever, I. (2020, January 13–18). Generative pretraining from pixels. Proceedings of the International Conference on Machine Learning—PMLR 2020, Virtual.
33. Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., and Gelly, S. (2020). An image is worth 16 × 16 words: Transformers for image recognition at scale. arXiv.
34. Han, K., Xiao, A., Wu, E., Guo, J., Xu, C., and Wang, Y. (2021, January 6–14). Transformer in transformer. Proceedings of the Advances in Neural Information Processing Systems 34 (NeurIPS 2021), Virtual.
35. Chu, X., Tian, Z., Wang, Y., Zhang, B., Ren, H., Wei, X., Xia, H., and Shen, C. (2021, January 6–14). Twins: Revisiting the design of spatial attention in vision transformers. Proceedings of the Advances in Neural Information Processing Systems 34 (NeurIPS 2021), Virtual.
36. Lin, H., Cheng, X., Wu, X., Yang, F., Shen, D., Wang, Z., Song, Q., and Yuan, W. (2022, January 18–22). Cat: Cross attention in vision transformer. Proceedings of the 2022 IEEE International Conference on Multimedia and Expo (ICME), Taipei, Taiwan.10.1109/ICME52920.2022.9859720 ↗
37. Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., and Guo, B. (2021, January 1–17). Swin transformer: Hierarchical vision transformer using shifted windows. Proceedings of the 2021 IEEE/CVF International Conference On Computer Vision, Montreal, BC, Canada.10.1109/ICCV48922.2021.00986 ↗
38. Chen, C.F., Panda, R., and Fan, Q. (2021). Regionvit: Regional-to-local attention for vision transformers. arXiv.
39. Zhou, D., Kang, B., Jin, X., Yang, L., Lian, X., Jiang, Z., Hou, Q., and Feng, J. (2021). Deepvit: Towards deeper vision transformer. arXiv.
40. Zhu, X., Lyu, S., Wang, X., and Zhao, Q. (2021, January 1–17). TPH-YOLOv5: Improved YOLOv5 based on transformer prediction head for object detection on drone-captured scenarios. Proceedings of the 2021 IEEE/CVF International Conference ON Computer Vision, Montreal, BC, Canada.10.1109/ICCVW54120.2021.00312 ↗
41. Zhao, Q., Liu, B., Lyu, S., Wang, C., and Zhang, H. (2023). TPH-YOLOv5++: Boosting Object Detection on Drone-Captured Scenarios with Cross-Layer Asymmetric Transformer. Remote Sens., 15.10.3390/rs15061687 ↗
42. Zheng, Z., Wang, P., Liu, W., Li, J., Ye, R., and Ren, D. (2020, January 7–12). Distance-IoU loss: Faster and better learning for bounding box regression. Proceedings of the 2020 AAAI Conference on Artificial Intelligence, New York, NY, USA.
43. Li, X., Wang, W., Wu, L., Chen, S., Hu, X., Li, J., Tang, J., and Yang, J. (2020, January 6–12). Generalized focal loss: Learning qualified and distributed bounding boxes for dense object detection. Proceedings of the Advances in Neural Information Processing Systems 33 (NeurIPS 2020), Virtual.
44. Ouyang, D., He, S., Zhang, G., Luo, M., Guo, H., Zhan, J., and Huang, Z. (2023, January 4–10). Efficient Multi-Scale Attention Module with Cross-Spatial Learning. Proceedings of the ICASSP 2023—2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Rhodes Island, Greece.10.1109/ICASSP49357.2023.10096516 ↗
45. Tong, Z., Chen, Y., Xu, Z., and Yu, R. (2023). Wise-IoU: Bounding Box Regression Loss with Dynamic Focusing Mechanism. arXiv.
46. Wang, C.Y., Bochkovskiy, A., and Liao, H.Y.M. (2023, January 20–22). YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors. Proceedings of the 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, BC, Canada.10.1109/CVPR52729.2023.00721 ↗
47. Ge, Z., Liu, S., Wang, F., Li, Z., and Sun, J. (2021). Yolox: Exceeding yolo series in 2021. arXiv.
48. YOLO-DA: An Efficient YOLO-based Detector for Remote Sensing Object Detection10.1109/LGRS.2023.3303896 ↗