Adaptive Active Positioning of Camellia oleifera Fruit Picking Points: Classical Image Processing and YOLOv7 Fusion Algorithm
Source abstract
Abstract
Camellia oleifera fruits are randomly distributed in an orchard, and the fruits are easily blocked or covered by leaves. In addition, the colors of leaves and fruits are alike, and flowers and fruits grow at the same time, presenting many ambiguities. The large shock force will cause flowers to fall and affect the yield. As a result, accurate positioning becomes a difficult problem for robot picking. Therefore, studying target recognition and localization of Camellia oleifera fruits in complex environments has many difficulties. In this paper, a fusion method of deep learning based on visual perception and image processing is proposed to adaptively and actively locate fruit recognition and picking points for Camellia oleifera fruits. First, to adapt to the target classification and recognition of complex scenes in the field, the parameters of the You Only Live Once v7 (YOLOv7) model were optimized and selected to achieve Camellia oleifera fruits’ detection and determine the center point of the fruit recognition frame. Then, image processing and a geometric algorithm are used to process the image, segment, and determine the morphology of the fruit, extract the centroid of the outline of Camellia oleifera fruit, and then analyze the position deviation of its centroid point and the center point in the YOLO recognition frame. The frontlighting, backlight, partial occlusion, and other test conditions for the perceptual recognition processing were validated with several experiments. The results demonstrate that the precision of YOLOv7 is close to that of YOLOv5s, and the mean average precision of YOLOv7 is higher than that of YOLOv5s. For some occluded Camellia oleifera fruits, the YOLOv7 algorithm is better than the YOLOv5s algorithm, which improves the detection accuracy of Camellia oleifera fruits. The contour of Camellia oleifera fruits can be extracted entirely via image processing. The average position deviation between the centroid point of the image extraction and the center point of the YOLO recognition frame is 2.86 pixels; thus, the center point of the YOLO recognition frame is approximately considered to be consistent with the centroid point of the image extraction.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Smart Agriculture and AI
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
College of Urban and Rural Construction, Zhongkai University of Agriculture and Engineering, Guangzhou 510225, China
Foshan-Zhongke Innovation Research Institute of Intelligent Agriculture and Robotics, Foshan 528000, China
College of Mechanical and Electrical Engineering, Hunan Agricultural University, Changsha 410128, China
College of Engineering, South China Agricultural University, Guangzhou 510642, China
Funding
Foshan Science and Technology Innovation Project · Guangdong Dongguan Smart Aquaculture and Unmanned Processing Equipment Technology Innovation Platform Project
References (35)
Show source-provided references
1. Characterizations and application potentials of the hemicel-luloses in waste oil-tea camellia fruit shells from Southern China10.1016/j.indcrop.2022.114551 ↗
2. Jiang, H., Hu, Y., Jiang, X., and Zhou, H. (2022). Maturity Stage Discrimination of Camellia oleifera Fruit Using Visible and Near-Infrared Hyperspectral Imaging. Molecules, 27.10.3390/molecules27196318 ↗
3. A litchi fruit recognition method in a natural environment using RGB-D images10.1016/j.biosystemseng.2021.01.015 ↗
4. Kang, H.W., Zhou, H.Y., Wang, X., and Chen, C. (2020). Real-Time Fruit Recognition and Grasping Estimation for Robotic Apple Har-vesting. Sensors, 20.10.3390/s20195670 ↗
5. Detection of Fruit-Bearing Branches and Localization of Litchi Clusters for Vision-Based Harvesting Robots10.1109/ACCESS.2020.3005386 ↗
6. Three-dimensional perception of orchard banana central stock enhanced by adaptive multi-vision technology10.1016/j.compag.2020.105508 ↗
7. Multi-Target Recognition of Bananas and Automatic Positioning for the Inflorescence Axis Cutting Point10.3389/fpls.2021.705021 ↗
8. Color-, depth-, and shape-based 3D fruit detection10.1007/s11119-019-09654-w ↗
9. Grape detection, segmentation, and tracking using deep neural net-works and three-dimensional association10.1016/j.compag.2020.105247 ↗
10. Xu, D., Chen, L., Mou, X., Wu, Q., and Sun, G. (2021, January 28–30). 3D Reconstruction of Camellia oleifera Fruit Recognition and Fruit Branch based on Kinect Camera. Proceedings of the 2021 2nd International Conference on Artificial Intelligence and Information Systems, Chongqing, China.10.1145/3469213.3470321 ↗
11. Multi-class fruit-on-plant detection for apple in SNAP system using Faster R-CNN10.1016/j.compag.2020.105634 ↗
12. Recognition and Localization Methods for Vi-sion-Based Fruit Picking Robots: A Review10.3389/fpls.2020.00510 ↗
13. Development and field evaluation of a strawberry harvesting robot with a cable-driven gripper10.1016/j.compag.2019.01.009 ↗
14. Complete and accurate holly fruits counting using YOLOX object detection10.1016/j.compag.2022.107062 ↗
15. Ji, W., Pan, Y., Xu, B., and Wang, J.C. (2022). A Real-Time Apple Targets Detection Method for Picking Robot Based on Shuf-flenetV2-YOLOX. Agriculture, 12.10.3390/agriculture12060856 ↗
16. DSE-YOLO: Detail semantics enhancement YOLO for multi-stage strawberry detection10.1016/j.compag.2022.107057 ↗
17. Apple detection during different growth stages in orchards using the improved YOLO-V3 model10.1016/j.compag.2019.01.012 ↗
18. Deep learning for real-time fruit detection and orchard fruit load estimation: Benchmarking of ‘MangoYOLO’10.1007/s11119-019-09642-0 ↗
19. Gao, F., Fang, W., Sun, X., Wu, Z., Zhao, G., Li, G., Li, R., Fu, L., and Zhang, Q. (2022). A novel apple fruit detection and counting methodology based on deep learning and trunk tracking in modern orchard. Comput. Electron. Agric., 197.10.1016/j.compag.2022.107000 ↗
20. Parico, A.I.B., and Ahamed, T. (2021). Real Time Pear Fruit Detection and Counting Using YOLOv4 Models and Deep SORT. Sensors, 21.10.3390/s21144803 ↗
21. Fruit detection and load estimation of an orange orchard using the YOLO models through simple approaches in different imaging and illumination conditions10.1016/j.compag.2021.106533 ↗
22. Detection of fruit maturity stage and yield estimation in wild blueberry using deep learning convolutional neural networks10.1016/j.atech.2022.100099 ↗
23. Fruit detection and positioning technology for a Camellia oleifera C. Abel orchard based on improved YOLOv4-tiny model and binocular stereo vision10.1016/j.eswa.2022.118573 ↗
24. A visual identification method for the apple growth forms in the orchard10.1016/j.compag.2022.106954 ↗
25. Channel pruned YOLO V5s-based deep learning approach for rapid and accurate apple fruitlet detection before fruit thinning10.1016/j.biosystemseng.2021.08.015 ↗
26. Yan, B., Fan, P., Lei, X.Y., Liu, Z.J., and Yang, F.Z. (2021). A Real-Time Apple Targets Detection Method for Picking Robot Based on Im-proved YOLOv5. Remote Sens., 13.10.3390/rs13091619 ↗
27. Rachis detection and three-dimensional localization of cut off point for vision-based banana robot10.1016/j.compag.2022.107079 ↗
28. Wang, C.Y., Bochkovskiy, A., and Liao, H. (2022). YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object de-tectors. arXiv.10.1109/CVPR52729.2023.00721 ↗
29. Novel visual crack width measurement based on backbone double-scale features for improved detection automation10.1016/j.engstruct.2022.115158 ↗
30. Anam, C., Naufal, A., Sutanto, H., Adi, K., and Dougherty, G. (2022). Impact of Iterative Bilateral Filtering on the Noise Power Spectrum of Computed Tomography Images. Algorithms, 15.10.3390/a15100374 ↗
31. Performance Comparison between RGB and HSV Color Segmentations for Road Signs Detection10.4028/www.scientific.net/AMM.393.550 ↗
32. Defect Detection in Fruit and Vegetables by Using Machine Vision Systems and Image Processing
33. A novel image processing algorithm to separate linearly clustered kiwifruits10.1016/j.biosystemseng.2019.04.024 ↗
34. Improved Quantum Image Median Filtering in the Spatial Domain10.1007/s10773-019-04103-w ↗
35. Chen, Z., Wu, R., Lin, Y., Li, C., Chen, S., Yuan, Z., Chen, S., and Zou, X. (2022). Plant Disease Recognition Model Based on Improved YOLOv5. Agronomy, 12.10.3390/agronomy12020365 ↗