A Vision-Based Method Utilizing Deep Convolutional Neural Networks for Fruit Variety Classification in Uncertainty Conditions of Retail Sales
Source abstract
Abstract
This study proposes a double-track method for the classification of fruit varieties for application in retail sales. The method uses two nine-layer Convolutional Neural Networks (CNNs) with the same architecture, but different weight matrices. The first network classifies fruits according to images of fruits with a background, and the second network classifies based on images with the ROI (Region Of Interest, a single fruit). The results are aggregated with the proposed values of weights (importance). Consequently, the method returns the predicted class membership with the Certainty Factor (CF). The use of the certainty factor associated with prediction results from the original images and cropped ROIs is the main contribution of this paper. It has been shown that CFs indicate the correctness of the classification result and represent a more reliable measure compared to the probabilities on the CNN outputs. The method is tested with a dataset containing images of six apple varieties. The overall image classification accuracy for this testing dataset is excellent (99.78%). In conclusion, the proposed method is highly successful at recognizing unambiguous, ambiguous, and uncertain classifications, and it can be used in a vision-based sales systems in uncertain conditions and unplanned situations.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Smart Agriculture and AI · Spectroscopy and Chemometric Analyses · Advanced Chemical Sensor Technologies
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Faculty of Production Engineering and Logistics, Opole University of Technology, 45-758 Opole, Poland
Faculty of Electrical Engineering Automatic Control and Informatics, Opole University of Technology, 45-758 Opole, Poland
References (31)
Show source-provided references
1. Hussain, I., He, Q., and Chen, Z. (2018). Automatic Fruit Recognition Based on DCNN for Commercial Source Trace System. Int. J. Comput. Sci. Appl., 8.10.5121/ijcsa.2018.8301 ↗
2. A comprehensive review of fruit and vegetable classification techniques10.1016/j.imavis.2018.09.016 ↗
3. Fruits, Vegetable and Plants Category Recognition Systems Using Convolutional Neural Networks: A Review
4. Bolle, R.M., Connell, J.H., Haas, N., Mohan, R., and Taubin, G. (1996, January 2–4). Veggievision: A produce recognition system. Proceedings of the Third IEEE Workshop on Applications of Computer Vision, WACV’96, Sarasota, FL, USA.
5. Rachmawati, E., Supriana, I., and Khodra, M.L. (2017, January 19–21). Toward a new approach in fruit recognition using hybrid RGBD features and fruit hierarchy property. Proceedings of the 2017 4th International Conference on Electrical Engineering, Computer Science and Informatics (EECSI), Yogyakarta, Indonesia.10.1109/EECSI.2017.8239110 ↗
6. Design of fruits and vegetables online inspection system based on vision10.1088/1742-6596/1074/1/012160 ↗
7. Fruit Classification by Extracting Color Chromaticity, Shape and Texture Features: Towards an Application for Supermarkets10.1109/TLA.2016.7587652 ↗
8. Yang, M.-M., and Kichida, R. (2017, January 8–10). A study on classification of fruit type and fruit disease, Advances in Engineering Research (AER). Proceedings of the 13rd Annual International Conference on Electronics, Electrical Engineering and Information Science (EEEIS 2017), Guangzhou, Guangdong, China.10.2991/eeeis-17.2017.73 ↗
9. Fruit classification using computer vision and feedforward neural network10.1016/j.jfoodeng.2014.07.001 ↗
10. Fruit Classification by Wavelet-Entropy and Feedforward Neural Network Trained by Fitness-Scaled Chaotic ABC and Biogeography-Based Optimization10.3390/e17085711 ↗
11. Fruit classification by biogeography-based optimization and feedforward neural network10.1111/exsy.12146 ↗
12. Sakai, Y., Oda, T., Ikeda, M., and Barolli, L. (2016, January 6–8). A Vegetable Category Recognition System Using Deep Neural Network. Proceedings of the 2016 10th International Conference on Innovative Mobile and Internet Services in Ubiquitous Computing (IMIS), Fukuoka, Japan.10.1109/IMIS.2016.84 ↗
13. Femling, F., Olsson, A., and Alonso-Fernandez, F. (2018, January 26–29). Fruit and Vegetable Identification Using Machine Learning for Retail Applications. Proceedings of the 14th International Conference on Signal Image Technology and Internet Based Systems, SITIS 2018, Gran Canaria, Spain.10.1109/SITIS.2018.00013 ↗
14. Comparing bags of features, conventional convolutional neural network and AlexNet for fruit recognition
15. Automatic Fruit Classification Using Deep Learning for Industrial Applications10.1109/TII.2018.2875149 ↗
16. Fruit recognition from images using deep learning
17. Image based fruit category classification by 13-layer deep convolutional neural network and data augmentation10.1007/s11042-017-5243-3 ↗
18. Redmon, J., and Farhadi, A. (2018). Yolo V3: An incremental improvement. arXiv.
19. Apple detection during different growth stages in orchards using the improved YOLO-V3 model10.1016/j.compag.2019.01.012 ↗
20. Tian, Y., Yang, G., Wang, Z., Wang, H., Li, E., and Liang, Z. (2019). Detection of Apple Lesions in Orchards Based on Deep Learning Methods of CycleGAN and YOLOV3-Dense. J. Sens., 2019.10.1155/2019/7630926 ↗
21. Ren, S., He, K., Girshick, R.B., and Sun, J. (2016). Faster RCNN: Towards Real-Time Object Detection with Region Proposal Networks. arXiv.10.1109/TPAMI.2016.2577031 ↗
22. Multi Class Fruit Classification Using Efficient Object Detection and Recognition Techniques
23. Hussain, I., Wu, W.L., Hua, H.Q., and Hussain, N. (2019, January 10–12). Intra-Class Recognition of Fruits Using DCNN for Commercial Trace Back-System. Proceedings of the International Conference on Multimedia Systems and Signal Processing (ICMSSP May 2019), Guangzhou, China.10.1145/3330393.3330401 ↗
24. Jarrett, K., Kavukcuoglu, K., Ranzato, M., and LeCun, Y. What is the best multi-stage architecture for object recognition? In Proceedings of the International Conference on Computer Vision (ICCV’09), Kyoto, Japan, 29 September–2 October 2009.10.1109/ICCV.2009.5459469 ↗
25. Qian, R., Yue, Y., Coenen, F., and Zhang, B. (2016, January 13–15). Traffic sign recognition with convolutional neural network based on max pooling positions. Proceedings of the 2th International Conference on Natural Computation, Fuzzy Systems and Knowledge Discovery (ICNC-FSKD), Changsha, China.10.1109/FSKD.2016.7603237 ↗
26. Dropout: A Simple Way to Prevent Neural Networks from Overfitting
27. Kingma, D.P., and Ba, J. (2014). Adam: A Method for Stochastic Optimization. arXiv.
28. Hussain, I., He, Q., Chen, Z., and Xie, W. (2018). Fruit Recognition dataset. Zenodo.
29. Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C.Y., and Berg, A.C. (2016, January 11–14). SSD: Single shot multibox detector. Proceedings of the European Conference on Computer Vision (ECCV), Amsterdam, The Netherlands.10.1007/978-3-319-46448-0_2 ↗
30. Dai, J., Li, Y., He, K., and Sun, J. (2016). R-FCN: Object detection via region based fully convolutional networks. arXiv.
31. Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., and Chen, L.C. (2018, January 18–22). MobileNetV2: Inverted Residuals and Linear Bottlenecks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Salt Lake City, UT, USA.10.1109/CVPR.2018.00474 ↗