journal article·2024-07-19Open access

RT-DETR-Tomato: Tomato Target Detection Algorithm Based on Improved RT-DETR for Agricultural Safety Production

Zhimin Zhao, Shuo Chen, Yuheng Ge, Penghao Yang, Yunkun Wang, Yunsheng Song

Applied Sciences· Volume 14· Issue 14

Abstract

The detection of tomatoes is of vital importance for enhancing production efficiency, with image recognition-based tomato detection methods being the primary approach. However, these methods face challenges such as the difficulty in extracting small targets, low detection accuracy, and slow processing speeds. Therefore, this paper proposes an improved RT-DETR-Tomato model for efficient tomato detection under complex environmental conditions. The model mainly consists of a Swin Transformer block, a BiFormer module, path merging, multi-scale convolutional layers, and fully connected layers. In this proposed model, Swin Transformer is chosen as the new backbone network to replace ResNet50 because of its superior ability to capture broader global dependency relationships and contextual information. Meanwhile, a lightweight BiFormer block is adopted in Swin Transformer to reduce computational complexity through content-aware flexible computation allocation. Experimental results show that the average accuracy of the final RT-DETR-Tomato model is greatly improved compared to the original model, and the model training time is greatly reduced, demonstrating better environmental adaptability. In the future, the RT-DETR-Tomato model can be integrated with intelligent patrol and picking robots, enabling precise identification of crops and ensuring the safety of crops and the smooth progress of agricultural production.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: Smart Agriculture and AI · Spectroscopy and Chemometric Analyses · Digital Imaging for Blood Diseases

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

School of Information Science and Engineering, Shandong Agricultural University, Taian 271018, China

Key Laboratory of Huang-Huai-Hai Smart Agricultural Technology of Ministry of Agriculture and Rural Affars, Shandong Agricultural University, Taian 271018, China

Funding

Shandong Provincial Natural Science Foundation, China · Open Project Foundation of Intelligent Information Processing Key Laboratory of Shanxi Province

References (57)

Show source-provided references
1. Design a robot system for tomato picking based on yolo v510.1016/j.ifacol.2022.05.029 ↗
2. Tomato detection based on modified YOLOv3 framework10.1038/s41598-021-81216-5 ↗
3. Xu, X., Xue, Z., and Zhao, Y. (2022). Research on an algorithm of express parcel sorting based on deeper learning and multi-information recognition. Sensors, 22.10.3390/s22176705 ↗
4. Research on highway vehicle detection based on faster R-CNN and domain adaptation10.1007/s10489-021-02552-7 ↗
5. On plant detection of intact tomato fruits using image analysis and machine learning methods10.3390/s140712191 ↗
6. Detecting tomatoes in greenhouse scenes by combining AdaBoost classifier and colour analysis10.1016/j.biosystemseng.2016.05.001 ↗
7. Luo, L., Tang, Y., Zou, X., Wang, C., Zhang, P., and Feng, W. (2016). Robust grape cluster detection in a vineyard by combining the AdaBoost framework and multiple color components. Sensors, 16.10.3390/s16122098 ↗
8. Liu, G., Mao, S., and Kim, J.H. (2019). A mature-tomato detection algorithm using machine learning and color analysis. Sensors, 19.10.3390/s19092023 ↗
9. Li, C., Li, L., Jiang, H., Weng, K., Geng, Y., Li, L., Ke, Z., Li, Q., Cheng, M., and Nie, W. (2022). YOLOv6: A single-stage object detection framework for industrial applications. arXiv.
10. Revolutionizing Agriculture: Real-Time Ripe Tomato Detection With the Enhanced Tomato-YOLOv7 System10.1109/ACCESS.2023.3336562 ↗
11. Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., and Guo, B. (2021, January 10–17). Swin transformer: Hierarchical vision transformer using shifted windows. Proceedings of the IEEE/CVF International Conference on Computer Vision, Montreal, QC, Canada.10.1109/ICCV48922.2021.00986 ↗
12. Zhu, L., Wang, X., Ke, Z., Zhang, W., and Lau, R.W. (2023, January 17–24). BiFormer: Vision transformer with bi-level routing attention. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, BC, Canada.10.1109/CVPR52729.2023.00995 ↗
13. Scale adaptive small target recognition algorithm in complex agricultural environment-taking bees as research objects
14. Yang, H., Liu, Y., Wang, S., Qu, H., Li, N., Wu, J., Yan, Y., Zhang, H., Wang, J., and Qiu, J. (2023). Improved apple fruit target recognition method based on YOLOv7 model. Agriculture, 13.10.3390/agriculture13071278 ↗
15. Chen, F., Zhang, L., Kang, S., Chen, L., Dong, H., Li, D., and Wu, X. (2023). Soft-NMS-enabled YOLOv5 with SIOU for small water surface floater detection in UAV-captured images. Sustainability, 15.10.3390/su151410751 ↗
16. Guo, Q., Chen, Y., Tang, Y., Zhuang, J., He, Y., Hou, C., Chu, X., Zhong, Z., and Luo, S. (2019). Lychee fruit detection based on monocular machine vision in orchard environment. Sensors, 19.10.3390/s19194091 ↗
17. A robust fruit image segmentation algorithm against varying illumination for vision system of fruit harvesting robot10.1016/j.ijleo.2016.11.177 ↗
18. Xiao, F., Wang, H., Xu, Y., and Zhang, R. (2023). Fruit detection and recognition based on deep learning for automatic harvesting: An overview and review. Agronomy, 13.10.3390/agronomy13061625 ↗
19. Xu, S., Guo, Y., Liang, X., and Lu, H. (2024). Intelligent Rapid Detection Techniques for Low-Content Components in Fruits and Vegetables: A Comprehensive Review. Foods, 13.10.3390/foods13071116 ↗
20. AE—Automation and emerging technologies: A segmentation algorithm for the automatic recognition of Fuji apples at harvest10.1006/bioe.2002.0132 ↗
21. Mao, W., Ji, B., Zhan, J., Zhang, X., and Hu, X. (2009, January 17–19). Apple location method for the apple harvesting robot. Proceedings of the 2009 2nd International Congress on Image and Signal Processing, Tianjin, China.10.1109/CISP.2009.5305224 ↗
22. Yin, H., Chai, Y., Yang, S.X., and Mittal, G.S. (2009, January 11–14). Ripe tomato extraction for a harvesting robotic system. Proceedings of the 2009 IEEE International Conference on Systems, Man and Cybernetics, San Antonio, TX, USA.10.1109/ICSMC.2009.5345994 ↗
23. Automatic method of fruit object extraction under complex agricultural background for vision system of fruit picking robot10.1016/j.ijleo.2014.07.001 ↗
24. Green citrus detection using ‘eigenfruit’, color and circular Gabor texture features under natural outdoor conditions10.1016/j.compag.2011.07.001 ↗
25. Determination of the number of green apples in RGB images recorded in orchards10.1016/j.compag.2011.11.007 ↗
26. Estimating mango crop yield using image analysis using fruit at ‘stone hardening’ stage and night time imaging10.1016/j.compag.2013.11.011 ↗
27. Vision-based localisation of mature apples in tree images using convexity10.1016/j.biosystemseng.2013.11.007 ↗
28. Zhao, Y., Gong, L., Huang, Y., and Liu, C. (2016). Robust tomato recognition for robotic harvesting using feature images fusion. Sensors, 16.10.3390/s16020173 ↗
29. Computer vision for fruit harvesting robots–state of the art and challenges ahead10.1504/IJCVR.2012.046419 ↗
30. Recognition of clustered tomatoes based on binocular stereo vision10.1016/j.compag.2014.05.006 ↗
31. Image fusion of visible and thermal images for fruit detection10.1016/j.biosystemseng.2009.02.009 ↗
32. Hou, G., Chen, H., Jiang, M., and Niu, R. (2023). An Overview of the Application of Machine Vision in Recognition and Localization of Fruit and Vegetable Harvesting Robots. Agriculture, 13.10.3390/agriculture13091814 ↗
33. In-field citrus detection and localisation based on RGB-D image analysis10.1016/j.biosystemseng.2019.06.019 ↗
34. Detection of Red Tomato on Plants Using Image Processing Techniques
35. Combining SUN-based visual attention model and saliency contour detection algorithm for apple image segmentation10.1007/s11042-018-7106-y ↗
36. Detecting fruits in natural scenes by using spatial-frequency based texture analysis and multiview geometry10.1016/j.compag.2011.01.007 ↗
37. Zhang, K., Wang, H., Shen, C., and Chen, X. (2019). Research on the technology used to inspect the visual appearance of tropical fruit, based on machine vision color space. Recent Developments in Intelligent Computing, Communication and Devices, Proceedings of the ICCD 2017, Shenzhen, China, 4–9 December 2017, Springer.10.1007/978-981-10-8944-2_7 ↗
38. Identification of fruit and branch in natural scenes for citrus harvesting robot using machine vision and support vector machine
39. Computer vision-based apple grading for golden delicious apples based on surface features
40. A comprehensive review of fruit and vegetable classification techniques10.1016/j.imavis.2018.09.016 ↗
41. Localisation of litchi in an unstructured environment using binocular stereo vision10.1016/j.biosystemseng.2016.02.004 ↗
42. A vision methodology for harvesting robot to detect cutting points on peduncles of double overlapping grape clusters in a vineyard10.1016/j.compind.2018.03.017 ↗
43. Immature peach detection in colour images acquired in natural illumination conditions using statistical classifiers and neural network10.1007/s11119-013-9323-8 ↗
44. Gulzar, Y. (2023). Fruit image classification model based on MobileNetV2 with deep transfer learning technique. Sustainability, 15.10.3390/su15031906 ↗
45. Afonso, M., Fonteijn, H., Fiorentin, F.S., Lensink, D., Mooij, M., Faber, N., Polder, G., and Wehrens, R. (2020). Tomato fruit detection and counting in greenhouses using deep learning. Front. Plant Sci., 11.10.3389/fpls.2020.571299 ↗
46. Moreira, G., Magalhães, S.A., Pinho, T., dos Santos, F.N., and Cunha, M. (2022). Benchmark of deep learning and a proposed HSV colour space models for the detection and classification of greenhouse tomato. Agronomy, 12.10.3390/agronomy12020356 ↗
47. Mu, Y., Chen, T.S., Ninomiya, S., and Guo, W. (2020). Intact detection of highly occluded immature tomatoes on plants using deep learning techniques. Sensors, 20.10.3390/s20102984 ↗
48. Magalhães, S.A., Castro, L., Moreira, G., Dos Santos, F.N., Cunha, M., Dias, J., and Moreira, A.P. (2021). Evaluating the single-shot multibox detector and YOLO deep learning models for the detection of tomatoes in a greenhouse. Sensors, 21.10.3390/s21103569 ↗
49. Research on tomato detection in natural environment based on RC-YOLOv410.1016/j.compag.2022.107029 ↗
50. Tomato cluster detection and counting using improved YOLOv5 based on RGB-D fusion10.1016/j.compag.2023.107741 ↗
51. He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27–30). Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.10.1109/CVPR.2016.90 ↗
52. Transformer in transformer
53. Xu, H., Li, B., and Zhong, F. (2022). Light-YOLOv5: A lightweight algorithm for improved YOLOv5 in complex fire scenarios. Appl. Sci., 12.10.3390/app122312312 ↗
54. Jiang, B., Luo, R., Mao, J., Xiao, T., and Jiang, Y. (2018, January 8–14). Acquisition of localization confidence for accurate object detection. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.10.1007/978-3-030-01264-9_48 ↗
55. Apple detection during different growth stages in orchards using the improved YOLO-V3 model10.1016/j.compag.2019.01.012 ↗
56. Fast and accurate automated recognition of the dominant cells from fecal images based on Faster R-CNN10.1038/s41598-021-89863-4 ↗
57. Detecting soybean leaf disease from synthetic image using multi-feature fusion faster R-CNN10.1016/j.compag.2021.106064 ↗