journal article·2024-05-16Open access

A Novel Deep Learning Method for Detecting Strawberry Fruit

Shuo Shen, Famin Duan, Zhiwei Tian, Chunxiao Han

Applied Sciences· Volume 14· Issue 10

Abstract

The recognition and localization of strawberries are crucial for automated harvesting and yield prediction. This article proposes a novel RTF-YOLO (RepVgg-Triplet-FocalLoss-YOLO) network model for real-time strawberry detection. First, an efficient convolution module based on structural reparameterization is proposed. This module was integrated into the backbone and neck networks to improve the detection speed. Then, the triplet attention mechanism was embedded into the last two detection heads to enhance the network’s feature extraction for strawberries and improve the detection accuracy. Lastly, the focal loss function was utilized to enhance the model’s recognition capability for challenging strawberry targets, which thereby improves the model’s recall rate. The experimental results demonstrated that the RTF-YOLO model achieved a detection speed of 145 FPS (frames per second), a precision of 91.92%, a recall rate of 81.43%, and an mAP (mean average precision) of 90.24% on the test dataset. Relative to the baseline of YOLOv5s, it showed improvements of 19%, 2.3%, 4.2%, and 3.6%, respectively. The RTF-YOLO model performed better than other mainstream models and addressed the problems of false positives and false negatives in strawberry detection caused by variations in illumination and occlusion. Furthermore, it significantly enhanced the speed of detection. The proposed model can offer technical assistance for strawberry yield estimation and automated harvesting.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: Smart Agriculture and AI · Spectroscopy and Chemometric Analyses · Currency Recognition and Detection

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

Faculty of Electronic Information and Electrical Engineering, Dalian University of Technology, Dalian 116024, China

Institute of Urban Agriculture, Chinese Academy of Agricultural Sciences, Chengdu 610213, China

SDIC XinJiang Luobupo Potash Co., Ltd., Hami 839000, China

Funding

Chengdu Agricultural Science and Technology Center Local Finance Special Fund Project · Sichuan Provincial Science and Technology Plan Project

References (32)

Show source-provided references
1. Berry antioxidants: Small fruits providing large benefits10.1002/jsfa.6432 ↗
2. A novel greenhouse-based system for the detection and plumpness assessment of strawberry using an improved deep learning technique10.3389/fpls.2020.00559 ↗
3. A review of key techniques of vision-based control for harvesting robot10.1016/j.compag.2016.06.022 ↗
4. Determination of the number of green apples in RGB images recorded in orchards10.1016/j.compag.2011.11.007 ↗
5. Development of an expert system based on wavelet transform and artificial neural networks for the ripe tomato harvesting robot
6. Detecting citrus fruits and occlusion recovery under natural illumination conditions10.1016/j.compag.2014.10.016 ↗
7. Improved image processing-based crop detection using Kalman filtering and the Hungarian algorithm10.1016/j.compag.2018.02.027 ↗
8. DSW-YOLO: A detection method for ground-planted strawberry fruits under different occlusion levels10.1016/j.compag.2023.108304 ↗
9. Deep learning in agriculture: A survey10.1016/j.compag.2018.02.016 ↗
10. Real-time strawberry detection using deep neural networks on embedded system (rtsd-net): An edge AI application10.1016/j.compag.2021.106586 ↗
11. Real-time visual localization of the picking points for a ridge-planting strawberry harvesting robot10.1109/ACCESS.2020.3003034 ↗
12. Strawberry localization in a ridge planting with an autonomous rover10.1016/j.engappai.2022.105810 ↗
13. Fruit detection for strawberry harvesting robot in non-structural environment based on Mask-RCNN10.1016/j.compag.2019.06.001 ↗
14. A fine recognition method of strawberry ripeness combining Mask R-CNN and region segmentation10.3389/fpls.2023.1211830 ↗
15. A fast and accurate deep learning method for strawberry instance segmentation10.1016/j.compag.2020.105736 ↗
16. Chen, Y., Lee, W.S., Gan, H., Peres, N., Fraisse, C., Zhang, Y., and He, Y. (2019). Strawberry yield prediction based on a deep neural network using high-resolution aerial orthoimages. Remote Sens., 11.10.3390/rs11131584 ↗
17. Object-Detection from Multi-View remote sensing Images: A case study of fruit and flower detection and counting on a central Florida strawberry farm
18. Strawberry R-CNN: Recognition and counting model of strawberry based on improved faster R-CNN10.1016/j.ecoinf.2023.102210 ↗
19. DSE-YOLO: Detail semantics enhancement YOLO for multi-stage strawberry detection10.1016/j.compag.2022.107057 ↗
20. Strawberry maturity classification from UAV and near-ground imaging using deep learning10.1016/j.atech.2021.100001 ↗
21. Chai, J.J., Xu, J.L., and O’Sullivan, C. (2023). Real-Time Detection of Strawberry Ripeness Using Augmented Reality and Deep Learning. Sensors, 23.10.2139/ssrn.4455768 ↗
22. Li, Y., Xue, J., Zhang, M., Yin, J., Liu, Y., Qiao, X., Zheng, D., and Li, Z. (2023). YOLOv5-ASFF: A Multistage Strawberry Detection Algorithm Based on Improved YOLOv5. Agronomy, 13.10.3390/agronomy13071901 ↗
23. The effectiveness of data augmentation in image classification using deep learning
24. Lin, T.Y., Dollár, P., Girshick, R., He, K., Hariharan, B., and Belongie, S. (, January 21–26). Feature pyramid networks for object detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.
25. Liu, S., Qi, L., Qin, H., Shi, J., and Jia, J. (2018, January 18–23). Path aggregation network for instance segmentation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00913 ↗
26. Ding, X., Zhang, X., Ma, N., Han, J., Ding, G., and Sun, J. (2021, January 20–25). Repvgg: Making vgg-style convnets great again. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA.10.1109/CVPR46437.2021.01352 ↗
27. Chu, X., Li, L., and Zhang, B. (2022). Make RepVGG Greater Again: A Quantization-aware Approach. arXiv.
28. Chen, J., Kao, S.h., He, H., Zhuo, W., Wen, S., Lee, C.H., and Chan, S.H.G. (2023, January 17–24). Run, Don’t walk: Chasing higher FLOPS for faster neural networks. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, BC, Canada.10.1109/CVPR52729.2023.01157 ↗
29. Misra, D., Nalamada, T., Arasanipalai, A.U., and Hou, Q. (2021, January 5–9). Rotate to attend: Convolutional triplet attention module. Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, Virtual.10.1109/WACV48630.2021.00318 ↗
30. Li, H., Li, J., Wei, H., Liu, Z., Zhan, Z., and Ren, Q. (2022). Slim-neck by GSConv: A better design paradigm of detector architectures for autonomous vehicles. arXiv.
31. Lin, T.Y., Goyal, P., Girshick, R., He, K., and Dollár, P. (2017, January 22–29). Focal loss for dense object detection. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.10.1109/ICCV.2017.324 ↗
32. Woo, S., Park, J., Lee, J.Y., and Kweon, I.S. (2018, January 8–14). Cbam: Convolutional block attention module. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.10.1007/978-3-030-01234-2_1 ↗