Efficient Small-Object Detection in Underwater Images Using the Enhanced YOLOv8 Network
Source abstract
Abstract
Underwater object detection plays a significant role in marine ecosystem research and marine species conservation. The improvement of related technologies holds practical significance. Although existing object-detection algorithms have achieved an excellent performance on land, they are not satisfactory in underwater scenarios due to two limitations: the underwater objects are often small, densely distributed, and prone to occlusion characteristics, and underwater embedded devices have limited storage and computational capabilities. In this paper, we propose a high-precision, lightweight underwater detector specifically optimizing for underwater scenarios based on the You Only Look Once Version 8 (YOLOv8) model. Firstly, we replace the Darknet-53 backbone of YOLOv8s with FasterNet-T0, reducing model parameters by 22.52%, FLOPS by 23.59%, and model size by 22.73%, achieving model lightweighting. Secondly, we add a Prediction Head for Small Objects, increase the number of channels for high-resolution feature map detection heads, and decrease the number of channels for low-resolution feature map detection heads. This results in a 1.2% improvement in small-object detection accuracy, while the remaining model parameters and memory consumption are nearly unchanged. Thirdly, we use Deformable ConvNets and Coordinate Attention in the neck part to enhance the accuracy in the detection of irregularly shaped and densely occluded small targets. This is achieved by learning convolution offsets from feature maps and emphasizing the regions of interest (RoIs). Our method achieves 52.12% AP on the underwater dataset UTDAC2020, with only 8.5 M parameters, 25.5 B FLOPS, and 17 MB model size. It surpasses the performance of large model YOLOv8l, at 51.69% AP, with 43.6 M parameters, 164.8 B FLOPS, and 84 MB model size. Furthermore, by increasing the input image resolution to 1280 × 1280 pixels, our model achieves 53.18% AP, making it the state-of-the-art (SOTA) model for the UTDAC2020 underwater dataset. Additionally, we achieve 84.4% mAP on the Pascal VOC dataset, with a substantial reduction in model parameters compared to previous, well-established detectors. The experimental results demonstrate that our proposed lightweight method retains effectiveness on underwater datasets and can be generalized to common datasets.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Advanced Neural Network Applications · Underwater Vehicles and Communication Systems · Image Enhancement Techniques
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
College of Information Technology, Shanghai Ocean University, Shanghai 201306, China
Funding
National Natural Science Foundation of China · Young Scientists Fund of the National Natural Science Foundation of China
References (63)
Show source-provided references
1. Active underwater detection with an array of atomic magnetometers10.1364/AO.57.002346 ↗
2. Deep sea habitats in the chemical warfare dumping areas of the Baltic Sea10.1016/j.scitotenv.2017.10.165 ↗
3. Redmon, J., Divvala, S., Girshick, R., and Farhadi, A. (2016, January 27–30). You only look once: Unified, real-time object detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.10.1109/CVPR.2016.91 ↗
4. Spatial pyramid pooling in deep convolutional networks for visual recognition10.1109/TPAMI.2015.2389824 ↗
5. He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27–30). Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.10.1109/CVPR.2016.90 ↗
6. Boosting R-CNN: Reweighting R-CNN samples by RPN’s error for underwater object detection10.1016/j.neucom.2023.01.088 ↗
7. Zhang, M., Xu, S., Song, W., He, Q., and Wei, Q. (2021). Lightweight underwater object detection based on yolo v4 and multi-scale attentional feature fusion. Remote Sens., 13.10.3390/rs13224706 ↗
8. Bochkovskiy, A., Wang, C.-Y., and Liao, H.-Y.M. (2020). Yolov4: Optimal speed and accuracy of object detection. arXiv.
9. Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., and Chen, L.-C. (2018, January 18–23). Mobilenetv2: Inverted residuals and linear bottlenecks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00474 ↗
10. Chollet, F. (2017, January 21–26). Xception: Deep learning with depthwise separable convolutions. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.10.1109/CVPR.2017.195 ↗
11. Chen, J., Kao, S.-h., He, H., Zhuo, W., Wen, S., Lee, C.-H., and Chan, S.-H.G. (2023). Run, Don’t Walk: Chasing Higher FLOPS for Faster Neural Networks. arXiv.10.1109/CVPR52729.2023.01157 ↗
12. Zhu, X., Hu, H., Lin, S., and Dai, J. (2019, January 15–20). Deformable convnets v2: More deformable, better results. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.10.1109/CVPR.2019.00953 ↗
13. Hou, Q., Zhou, D., and Feng, J. (2021, January 20–25). Coordinate attention for efficient mobile network design. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA.10.1109/CVPR46437.2021.01350 ↗
14. Girshick, R. (2015, January 7–13). Fast R-CNN. Proceedings of the IEEE International Conference on Computer Vision, Santiago, Chile.10.1109/ICCV.2015.169 ↗
15. Wang, C.-Y., Bochkovskiy, A., and Liao, H.-Y.M. (2022). YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors. arXiv.10.1109/CVPR52729.2023.00721 ↗
16. Wang, C.-Y., Liao, H.-Y.M., and Yeh, I.-H. (2022). Designing Network Design Strategies Through Gradient Path Analysis. arXiv.
17. Feng, C., Zhong, Y., Gao, Y., Scott, M.R., and Huang, W. (2021, January 10–17). Tood: Task-aligned one-stage object detection. Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), Montreal, QC, Canada.10.1109/ICCV48922.2021.00349 ↗
18. Generalized focal loss: Learning qualified and distributed bounding boxes for dense object detection
19. Imagenet classification with deep convolutional neural networks10.1145/3065386 ↗
20. Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., and Fei-Fei, L. (2009, January 20–25). Imagenet: A large-scale hierarchical image database. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Miami, FL, USA.10.1109/CVPR.2009.5206848 ↗
21. Iandola, F.N., Han, S., Moskewicz, M.W., Ashraf, K., Dally, W.J., and Keutzer, K. (2016). SqueezeNet: AlexNet-level accuracy with 50× fewer parameters and <0.5 MB model size. arXiv.
22. Zhang, X., Zhou, X., Lin, M., and Sun, J. (2018, January 18–23). Shufflenet: An extremely efficient convolutional neural network for mobile devices. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00716 ↗
23. Ma, N., Zhang, X., Zheng, H.-T., and Sun, J. (2018, January 8–14). Shufflenet v2: Practical guidelines for efficient cnn architecture design. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.10.1007/978-3-030-01264-9_8 ↗
24. Howard, A.G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., Andreetto, M., and Adam, H. (2017). Mobilenets: Efficient convolutional neural networks for mobile vision applications. arXiv.
25. Howard, A., Sandler, M., Chu, G., Chen, L.-C., Chen, B., Tan, M., Wang, W., Zhu, Y., Pang, R., and Vasudevan, V. (November, January 27). Searching for MobileNetV3. Proceedings of the IEEE/CVF International Conference on Computer Vision, Seoul, Republic of Korea.
26. Tan, M., and Le, Q. (2019, January 9–15). Efficientnet: Rethinking model scaling for convolutional neural networks. Proceedings of the International Conference on Machine Learning, Long Beach, CA, USA.
27. Han, K., Wang, Y., Tian, Q., Guo, J., Xu, C., and Xu, C. (2020, January 13–19). Ghostnet: More features from cheap operations. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.10.1109/CVPR42600.2020.00165 ↗
28. Mehta, S., and Rastegari, M. (2021). Mobilevit: Light-weight, general-purpose, and mobile-friendly vision transformer. arXiv.
29. Faster R-CNN for marine organisms detection and recognition using data augmentation10.1016/j.neucom.2019.01.084 ↗
30. He, K., Gkioxari, G., Dollár, P., and Girshick, R. (2017, January 22–29). Mask R-CNN. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.10.1109/ICCV.2017.322 ↗
31. A new lightweight deep neural network for surface scratch detection10.1007/s00170-022-10335-8 ↗
32. Lin, T.-Y., Dollár, P., Girshick, R., He, K., Hariharan, B., and Belongie, S. (2017, January 21–26). Feature pyramid networks for object detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.10.1109/CVPR.2017.106 ↗
33. Takano, N., and Alaghband, G. (2019). Srgan: Training dataset matters. arXiv.
34. Ledig, C., Theis, L., Huszár, F., Caballero, J., Cunningham, A., Acosta, A., Aitken, A., Tejani, A., Totz, J., and Wang, Z. (2017, January 21–26). Photo-realistic single image super-resolution using a generative adversarial network. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.10.1109/CVPR.2017.19 ↗
35. Redmon, J., and Farhadi, A. (2017, January 21–26). YOLO9000: Better, faster, stronger. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.10.1109/CVPR.2017.690 ↗
36. Alsubaei, F.S., Al-Wesabi, F.N., and Hilal, A.M. (2022). Deep learning-based small object detection and classification model for garbage waste management in smart cities and iot environment. Appl. Sci., 12.10.3390/app12052281 ↗
37. Dai, J., Qi, H., Xiong, Y., Li, Y., Zhang, G., Hu, H., and Wei, Y. (2017, January 22–29). Deformable convolutional networks. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.10.1109/ICCV.2017.89 ↗
38. Sadeghi, M.A., and Forsyth, D. (2014, January 6–12). 30hz object detection with dpm v5. Proceedings of the Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland. Part I 13.10.1007/978-3-319-10590-1_5 ↗
39. Cai, Z., and Vasconcelos, N. (2018, January 18–23). Cascade r-cnn: Delving into high quality object detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00644 ↗
40. Lin, T.-Y., Goyal, P., Girshick, R., He, K., and Dollár, P. (2017, January 22–29). Focal loss for dense object detection. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.10.1109/ICCV.2017.324 ↗
41. Tian, Z., Shen, C., Chen, H., and He, T. (November, January 27). FCOS: Fully convolutional one-stage object detection. Proceedings of the IEEE/CVF International Conference on Computer Vision, Seoul, Republic of Korea.
42. Zhu, X., Su, W., Lu, L., Li, B., Wang, X., and Dai, J. (2020). Deformable detr: Deformable transformers for end-to-end object detection. arXiv.
43. Pang, J., Chen, K., Shi, J., Feng, H., Ouyang, W., and Lin, D. (2019, January 15–20). Libra R-CNN: Towards balanced learning for object detection. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.10.1109/CVPR.2019.00091 ↗
44. Zhang, H., Chang, H., Ma, B., Wang, N., and Chen, X. (2020, January 23–28). Dynamic R-CNN: Towards high quality object detection via dynamic training. Proceedings of the Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK. Part XV 16.10.1007/978-3-030-58555-6_16 ↗
45. Zhang, S., Chi, C., Yao, Y., Lei, Z., and Li, S.Z. (2020, January 13–19). Bridging the gap between anchor-based and anchor-free detection via adaptive training sample selection. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.10.1109/CVPR42600.2020.00978 ↗
46. Ren, S., He, K., Girshick, R., and Sun, J. (2015, January 7–12). Faster R-CNN: Towards real-time object detection with region proposal networks. Proceedings of the 28th International Conference on Neural Information Processing Systems, Montreal, QC, Canada.
47. Gidaris, S., and Komodakis, N. (2015, January 7–13). Object detection via a multi-region and semantic segmentation-aware cnn model. Proceedings of the IEEE International Conference on Computer Vision, Santiago, Chile.10.1109/ICCV.2015.135 ↗
48. Dai, J., Li, Y., He, K., and Sun, J. (2016, January 5–10). R-FCN: Object detection via region-based fully convolutional networks. Proceedings of the 30th International Conference on Neural Information Processing Systems, Barcelona, Spain.
49. Zhu, Y., Zhao, C., Wang, J., Zhao, X., Wu, Y., and Lu, H. (2017, January 22–29). Couplenet: Coupling global structure with local parts for object detection. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.10.1109/ICCV.2017.444 ↗
50. Shen, Z., Liu, Z., Li, J., Jiang, Y.-G., Chen, Y., and Xue, X. (2017, January 22–29). Dsod: Learning deeply supervised object detectors from scratch. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.10.1109/ICCV.2017.212 ↗
51. Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C.-Y., and Berg, A.C. (2016;, January 11–14). Ssd: Single shot multibox detector. Proceedings of the Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands.10.1007/978-3-319-46448-0_2 ↗
52. Zhou, P., Ni, B., Geng, C., Hu, J., and Xu, Y. (2018, January 18–23). Scale-transferrable object detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00062 ↗
53. Zhang, S., Wen, L., Bian, X., Lei, Z., and Li, S.Z. (2018, January 18–23). Single-shot refinement neural network for object detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00442 ↗
54. Fu, C.-Y., Liu, W., Ranga, A., Tyagi, A., and Berg, A.C. (2017). DSSD: Deconvolutional single shot detector. arXiv.
55. Fan, B., Chen, W., Cong, Y., and Tian, J. (2020, January 23–28). Dual refinement underwater object detection network. Proceedings of the Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK. Part XX 16.10.1007/978-3-030-58565-5_17 ↗
56. Zhang, Z., Qiao, S., Xie, C., Shen, W., Wang, B., and Yuille, A.L. (2018, January 18–23). Single-shot object detection with enriched semantics. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00609 ↗
57. Kong, T., Sun, F., Tan, C., Liu, H., and Huang, W. (2018, January 8–14). Deep feature pyramid reconfiguration for object detection. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.10.1007/978-3-030-01228-1_11 ↗
58. Pang, Y., Wang, T., Anwer, R.M., Khan, F.S., and Shao, L. (2019, January 15–20). Efficient featurized image pyramid network for single shot detector. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.10.1109/CVPR.2019.00751 ↗
59. Liu, S., and Huang, D. (2018, January 8–14). Receptive field block net for accurate and fast object detection. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.10.1007/978-3-030-01252-6_24 ↗
60. Li, Y., Li, J., Lin, W., and Li, J. (2018). Tiny-DSOD: Lightweight object detection for resource-restricted usages. arXiv.
61. Wang, R.J., Li, X., and Ling, C.X. (2018, January 3–8). Pelee: A real-time object detection system on mobile devices. Proceedings of the Proceedings of the 32nd International Conference on Neural Information Processing Systems, Montréal, QC, Canada.
62. Selvaraju, R.R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., and Batra, D. (2017, January 22–29). Grad-CAM: Visual explanations from deep networks via gradient-based localization. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.10.1109/ICCV.2017.74 ↗
63. Jiang, L., Wang, Y., Jia, Q., Xu, S., Liu, Y., Fan, X., Li, H., Liu, R., Xue, X., and Wang, R. (2021, January 20–24). Underwater species detection using channel sharpening attention. Proceedings of the 29th ACM International Conference on Multimedia, Virtual.10.1145/3474085.3475563 ↗