journal article·2023-04-03Open access

An Effective Plant Recognition Method with Feature Recalibration of Multiple Pretrained CNN and Layers

Daoxiang Zhou, Xuetao Ma, Shu Feng

Applied Sciences· Volume 13· Issue 7

Abstract

Current existing methods are either not very discriminative or too complex. In this work, an effective and very simple plant recognition method is proposed. The main innovations of our method are threefold. (1) The feature maps of multiple pretrained convolutional neural networks and multiple layers are extracted; the complementary information between different feature maps can be fully explored. (2) Performing spatial and channel feature recalibration on each feature map enables our method to highlight salient visual content and reduce non-salient content; as a result, more informative features can be discerned. (3) In contrast to conventional transfer learning with end-to-end network parameters fine-tuning, in our method one forward process is enough to extract discriminative features. All recalibrated features are concatenated to form the plant leaf representation, which is fed into a linear support vector machine classifier for recognition. Extensive experiments are carried out on eight representative plant databases, yielding outstanding recognition accuracies, which demonstrates the effectiveness and superiority of our method obviously. Moreover, the retrieval experiments show our method can offer higher or competitive mean average precisions compared with state-of-the-art method. The feature visualization shows our learned features have excellent intra-class similarity and inter-class diversity for leaf species from the same genus.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: Smart Agriculture and AI · Remote Sensing in Agriculture · Remote-Sensing Image Classification

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

College of Data Science, Taiyuan University of Technology, Taiyuan 030024, China

Department of Foundation, Shanxi Agricultural University, Jinzhong 030801, China

Funding

National Natural Science Foundation of China · Natural Science Foundation of Shanxi Province of China

References (61)

Show source-provided references
1. Classification of plant leaf images with complicated background10.1016/j.amc.2008.05.108 ↗
2. Global dataset shows geography and life form predict modern plant extinction and rediscovery10.1038/s41559-019-0906-2 ↗
3. MARCH: Multiscale-arch-height description for mobile retrieval of leaf images10.1016/j.ins.2014.07.028 ↗
4. A CNN-based android application for plant leaf classification at remote locations10.1007/s00521-022-07740-1 ↗
5. Plant species recognition methods using leaf image: Overview10.1016/j.neucom.2019.09.113 ↗
6. Survey of feature extraction and classification techniques to identify plant through leaves10.1016/j.eswa.2020.114181 ↗
7. Kritsis, K., Kiourt, C., Stamouli, S., Sevetlidis, V., Solomou, A., Karetsos, G., Katsouros, V., and Pavlidis, G. (2021). GRASP-125: A Dataset for Greek Vascular Plant Recognition in Natural Environment. Sustainability, 13.10.3390/su132111865 ↗
8. Xu, W., Yu, G., Cui, Y., Gloaguen, R., Zare, A., Bonnette, J., Reyes-Cabrera, J., Rajurkar, A., Rowland, D., and Matamala, R. (2022). PRMI: A Dataset of Minirhizotron Images for Diverse Plant Root Study. arXiv.
9. Shape matching and object recognition using shape contexts10.1109/34.993558 ↗
10. Shape Classification Using the Inner-Distance10.1109/TPAMI.2007.41 ↗
11. Multiresolution gray-scale and rotation invariant texture classification with local binary patterns10.1109/TPAMI.2002.1017623 ↗
12. Shape retrieval using triangle-area representation and dynamic space warping10.1016/j.patcog.2006.12.005 ↗
13. Yang, C., Wei, H., and Yu, Q. (September, January 29). Multiscale Triangular Centroid Distance for Shape-Based Plant Leaf Recognition. Proceedings of the Twenty-Second European Conference on Artificial Intelligence (ECAI), The Hague, The Netherlands.
14. Multiscale Fourier descriptor based on triangular features for shape retrieval10.1016/j.image.2018.11.004 ↗
15. Bag of contour fragments for robust shape classification10.1016/j.patcog.2013.12.008 ↗
16. Joint distances by sparse representation and locality-constrained dictionary learning for robust leaf recognition10.1016/j.compag.2017.11.013 ↗
17. Liu, Z., Zhu, L., Zhang, X., Zhou, X., Shang, L., Huang, Z., and Gan, Y. (2015, January 20–23). Hybrid Deep Learning for Plant Leaves Classification. Proceedings of the Intelligent Computing Theories and Methodologies, Fuzhou, China.10.1007/978-3-319-22186-1_11 ↗
18. How deep learning extracts and learns leaf features for plant classification10.1016/j.patcog.2017.05.015 ↗
19. Shah, M.P., Singha, S., and Awate, S.P. (2017, January 17–20). Leaf classification using marginalized shape context and shape+texture dual-path deep convolutional neural network. Proceedings of the IEEE International Conference on Image Processing (ICIP), Beijing, China.10.1109/ICIP.2017.8296403 ↗
20. Multi-Organ Plant Classification Based on Convolutional and Recurrent Neural Networks10.1109/TIP.2018.2836321 ↗
21. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, L., and Polosukhin, I. (2017, January 4–9). Attention is All you Need. Proceedings of the Advances in Neural Information Processing Systems, Long Beach, CA, USA.
22. Squeeze-and-Excitation Networks10.1109/TPAMI.2019.2913372 ↗
23. A multiscale representation method for nonrigid shapes with a single closed contour10.1109/TCSVT.2004.826776 ↗
24. Multiscale Distance Matrix for Fast Plant Leaf Recognition10.1109/TIP.2012.2207391 ↗
25. Fast Plant Leaf Recognition Using Improved Multiscale Triangle Representation and KNN for Optimization10.1109/ACCESS.2020.3037649 ↗
26. Leaf Recognition Based on Elliptical Half Gabor and Maximum Gap Local Line Direction Pattern10.1109/ACCESS.2020.2976117 ↗
27. Plant leaf recognition by integrating shape and texture features10.1016/j.patcog.2020.107809 ↗
28. Research on plant leaf recognition method based on multi-feature fusion in different partition blocks10.1016/j.dsp.2023.103907 ↗
29. M3SPCANet: A simple and effective ConvNets with unsupervised predefined filters for face recognition10.1016/j.engappai.2022.104936 ↗
30. A novel image encryption algorithm with deep neural network10.1016/j.sigpro.2022.108536 ↗
31. Image salient regions encryption for generating visually meaningful ciphertext image10.1007/s00521-016-2490-6 ↗
32. A visually secure image encryption scheme based on semi-tensor product compressed sensing10.1016/j.sigpro.2020.107580 ↗
33. Torsional capacity evaluation of RC beams using an improved bird swarm algorithm optimized 2D convolutional neural network10.1016/j.engstruct.2022.115066 ↗
34. Automated damage diagnosis of concrete jack arch beam using optimized deep stacked autoencoders and multi-sensor fusion10.1016/j.dibe.2023.100128 ↗
35. MaskCOV: A random mask covariance network for ultra-fine-grained visual categorization10.1016/j.patcog.2021.108067 ↗
36. Kernel pooling feature representation of pre-trained convolutional neural networks for leaf recognition10.1007/s11042-021-11769-0 ↗
37. SWP-LeafNET: A novel multistage approach for plant leaf identification based on deep CNN10.1016/j.eswa.2022.117470 ↗
38. Plant leaf identification based on shape and convolutional features10.1016/j.eswa.2023.119626 ↗
39. Plant identification using deep neural networks via optimization of transfer learning parameters10.1016/j.neucom.2017.01.018 ↗
40. Analysis of transfer learning for deep neural network based plant classification models10.1016/j.compag.2019.01.041 ↗
41. Identification of plant leaf diseases using a nine-layer deep convolutional neural network10.1016/j.compeleceng.2019.04.011 ↗
42. Plant leaf disease classification using EfficientNet deep learning model10.1016/j.ecoinf.2020.101182 ↗
43. Soderkvist, O.J.O. (2001). Computer Vision Classifcation of Leaves from Swedish Trees. [Master’s Thesis, Linkoping University].
44. Wu, S.G., Bao, F.S., Xu, E.Y., Wang, Y., Chang, Y., and Xiang, Q. (2007, January 15–18). A Leaf Recognition Algorithm for Plant Classification Using Probabilistic Neural Network. Proceedings of the IEEE International Symposium on Signal Processing and Information Technology, Giza, Egypt.10.1109/ISSPIT.2007.4458016 ↗
45. Leaf recognition of woody species in Central Europe10.1016/j.biosystemseng.2013.04.007 ↗
46. Hierarchical String Cuts: A Translation, Rotation, Scale and Mirror Invariant Descriptor for Fast Shape Retrieval10.1109/TIP.2014.2343457 ↗
47. Kumar, N., Belhumeur, P.N., Biswas, A., Jacobs, D.W., Kress, W.J., Lopez, I.C., and Soares, J.V.B. (2012, January 7–13). Leafsnap: A Computer Vision System for Automatic Plant Species Identification. Proceedings of the European Conference on Computer Vision (ECCV), Florence, Italy.10.1007/978-3-642-33709-3_36 ↗
48. Plant identification using leaf shapes - A pattern counting approach10.1016/j.patcog.2015.04.004 ↗
49. Nilsback, M.E., and Zisserman, A. (2008, January 16–19). Automated Flower Classification over a Large Number of Classes. Proceedings of the Sixth Indian Conference on Computer Vision, Graphics & Image Processing, Bhubaneswar, India.10.1109/ICVGIP.2008.47 ↗
50. Plant species classification using flower images—A comparative study of local feature representations10.1371/journal.pone.0170629 ↗
51. Plant Disease Recognition: A Large-Scale Benchmark Dataset and a Visual Region and Loss Reweighting Approach10.1109/TIP.2021.3049334 ↗
52. PlantDiseaseNet: Convolutional neural network ensemble for plant disease and pest detection10.1007/s11760-021-01909-2 ↗
53. Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., and Fei-Fei, L. (2009, January 20–25). ImageNet: A large-scale hierarchical image database. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Miami, FL, USA.10.1109/CVPR.2009.5206848 ↗
54. Simonyan, K., and Zisserman, A. (2015, January 7–9). Very deep convolutional networks for large-scale image recognition. Proceedings of the International Conference on Learning Representations (ICLR), San Diego, CA, USA.
55. He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27–30). Deep Residual Learning for Image Recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.10.1109/CVPR.2016.90 ↗
56. Woo, S., Park, J., Lee, J.Y., and Kweon, I.S. (2018, January 8–14). CBAM: Convolutional Block Attention Module. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.10.1007/978-3-030-01234-2_1 ↗
57. Kalantidis, Y., Mellina, C., and Osindero, S. (2016, January 8–10). Cross-Dimensional Weighting for Aggregated Deep Convolutional Features. Proceedings of the European Conference on Computer Vision Workshops, Amsterdam, The Netherlands.10.1007/978-3-319-46604-0_48 ↗
58. ImageNet Classification with Deep Convolutional Neural Networks10.1145/3065386 ↗
59. Chord Bunch Walks for Recognizing Naturally Self-Overlapped and Compound Leaves10.1109/TIP.2019.2921526 ↗
60. Wang, B., Gao, Y., Sun, C., Blumenstein, M., and La Salle, J. (2017, January 21–26). Can Walking and Measuring Along Chord Bunches Better Describe Leaf Shapes?. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.10.1109/CVPR.2017.221 ↗
61. Schütze, H., Manning, C.D., and Raghavan, P. (2008). Introduction to Information Retrieval, Cambridge University Press.10.1017/CBO9780511809071 ↗