Local Diversity-Guided Weakly Supervised Fine-Grained Image Classification Method
Source abstract
Abstract
For fine-grained recognition, capturing distinguishable features and effectively utilizing local information play a key role, since the objects of recognition exhibit subtle differences in different subcategories. Finding subtle differences between subclasses is not straightforward. To address this problem, we propose a weakly supervised fine-grained classification network model with Local Diversity Guidance (LDGNet). We designed a Multi-Attention Semantic Fusion Module (MASF) to build multi-layer attention maps and channel–spatial interaction, which can effectively enhance the semantic representation of the attention maps. We also introduce a random selection strategy (RSS) that forces the network to learn more comprehensive and detailed information and more local features from the attention map by designing three feature extraction operations. Finally, both the attention map obtained by RSS and the feature map are employed for prediction through a fully connected layer. At the same time, a dataset of ancient towers is established, and our method is applied to ancient building recognition for practical applications of fine-grained image classification tasks in natural scenes. Extensive experiments conducted on four fine-grained datasets and explainable visualization demonstrate that the LDGNet can effectively enhance discriminative region localization and detailed feature acquisition for fine-grained objects, achieving competitive performance over other state-of-the-art algorithms.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Advanced Neural Network Applications · Advanced Image and Video Retrieval Techniques · Industrial Vision Systems and Defect Detection
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
College of Information and Control Engineering, Xi’an University of Architecture and Technology, Yanta, Xi’an 710055, China
Xi’an Key Laboratory of Intelligent Technology for Building Manufacturing, Xi’an 710311, China
Funding
National Natural Science Foundation of China · Key Research and Development Project of Shaanxi Province
References (53)
Show source-provided references
1. A Survey on Fine-Grained Image Categorization Using Deep Convolutional Features
2. Part-Based R-CNNs for Fine-Grained Category Detection10.1007/978-3-319-10590-1_54 ↗
3. Huang, S., Xu, Z., Tao, D., and Zhang, Y. (2016, January 27–30). Part-Stacked CNN for Fine-Grained Visual Categorization. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.10.1109/CVPR.2016.132 ↗
4. Xiao, T., Xu, Y., Yang, K., Zhang, J., Peng, Y., and Zhang, Z. (2015, January 7–12). The Application of Two-Level Attention Models in Deep Convolutional Neural Network for Fine-Grained Image Classification. Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA.
5. Wang, Z., Wang, S., Yang, S., Li, H., Li, J., and Li, Z. (2020, January 14–19). Weakly Supervised Fine-Grained Image Classification via Guassian Mixture Model Oriented Discriminative Learning. Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.10.1109/CVPR42600.2020.00977 ↗
6. Lin, T.-Y., RoyChowdhury, A., and Maji, S. (2015, January 7–13). Bilinear CNN Models for Fine-Grained Visual Recognition. Proceedings of the 2015 IEEE International Conference on Computer Vision (ICCV), Santiago, Chile.10.1109/ICCV.2015.170 ↗
7. Object-Part Attention Model for Fine-Grained Image Classification10.1109/TIP.2017.2774041 ↗
8. Pay Attention to the Activations: A Modular Attention Mechanism for Fine-Grained Image Recognition10.1109/TMM.2019.2928494 ↗
9. Zheng, H., Fu, J., Zha, Z.-J., and Luo, J. (2019, January 15–20). Looking for the Devil in the Details: Learning Trilinear Attention Sampling Network for Fine-Grained Image Recognition. Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA.10.1109/CVPR.2019.00515 ↗
10. Ji, R., Wen, L., Zhang, L., Du, D., Wu, Y., Zhao, C., Liu, X., and Huang, F. (2020, January 14–19). Attention Convolutional Binary Neural Tree for Fine-Grained Visual Categorization. Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.10.1109/CVPR42600.2020.01048 ↗
11. Khosla, A., Jayadevaprakash, N., Yao, B., and Li, F.-F. (2024, November 07). Novel Dataset for Fine-Grained Image Categorization: Stanford Dogs. Available online: https://people.csail.mit.edu/khosla/papers/fgvc2011.pdf.
12. Krause, J., Stark, M., Deng, J., and Fei-Fei, L. (2013, January 2–8). 3D Object Representations for Fine-Grained Categorization. Proceedings of the 2013 IEEE International Conference on Computer Vision Workshops, Sydney, Australia.10.1109/ICCVW.2013.77 ↗
13. Maji, S., Rahtu, E., Kannala, J., Blaschko, M., and Vedaldi, A. (2024, November 07). Fine-Grained Visual Classification of Aircraft. Available online: http://arxiv.org/abs/1306.5151.
14. Wah, C., Branson, S., Welinder, P., Perona, P., and Belongie, S. (2011). Caltech-UCSD Birds-200-2011, California Institute of Technology.
15. Learning to Navigate for Fine-Grained Classification10.1007/978-3-030-01264-9_26 ↗
16. Fine-Grained Visual Classification via Progressive Multi-Granularity Training of Jigsaw Patches10.1007/978-3-030-58565-5_10 ↗
17. Filtration and Distillation: Enhancing Region Attention for Fine-Grained Visual Categorization
18. Graph-Propagation Based Correlation Learning for Weakly Supervised Fine-Grained Image Classification
19. Zhao, Y., Yan, K., Huang, F., and Li, J. (2021, January 20–25). Graph-Based High-Order Relation Discovery for Fine-Grained Recognition. Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Nashville, TN, USA.10.1109/CVPR46437.2021.01483 ↗
20. Pairwise Confusion for Fine-Grained Visual Classification10.1007/978-3-030-01258-8_5 ↗
21. A Vision Transformer for Fine-Grained Classification by Reducing Noise and Enhancing Discriminative Information10.1016/j.patcog.2023.109979 ↗
22. Luo, W., Yang, X., Mo, X., Lu, Y., Davis, L., Li, J., Yang, J., and Lim, S.-N. (November, January 27). Cross-X Learning for Fine-Grained Visual Categorization. Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision (ICCV), Seoul, Republic of Korea.10.1109/ICCV.2019.00833 ↗
23. Cui, Y., Song, Y., Sun, C., Howard, A., and Belongie, S. (2018, January 18–23). Large Scale Fine-Grained Categorization and Domain-Specific Transfer Learning. Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00432 ↗
24. Carion, N., Massa, F., Synnaeve, G., Usunier, N., Kirillov, A., and Zagoruyko, S. (2020, January 23–28). End-to-end object detection with transformers. Proceedings of the European Conference on Computer Vision, Glasgow, UK.10.1007/978-3-030-58452-8_13 ↗
25. Chen, J., Lu, Y., Yu, Q., Luo, X., Adeli, E., Wang, Y., Lu, L., Yuille, A.L., and Zhou, Y. (2024, November 07). TransUNet: Transformers Make Strong Encoders for Medical Image Segmentation. Available online: http://arxiv.org/abs/2102.04306.
26. Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., and Gelly, S. (2024, November 07). An Image Is Worth 16x16 Words: Transformers for Image Recognition at Scale. Available online: http://arxiv.org/abs/2010.11929.
27. Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., and Guo, B. (2021, January 11–17). Swin Transformer: Hierarchical Vision Transformer Using Shifted Windows. Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision (ICCV), Montreal, QC, Canada.10.1109/ICCV48922.2021.00986 ↗
28. Wang, J., Yu, X., and Gao, Y. (2024, November 07). Feature Fusion Vision Transformer for Fine-Grained Visual Categorization. Available online: http://arxiv.org/abs/2107.02341.
29. Hu, Y., Jin, X., Zhang, Y., Hong, H., Zhang, J., He, Y., and Xue, H. (2021, January 17). RAMS-Trans: Recurrent Attention Multi-Scale Transformer for Fine-Grained Image Recognition. Proceedings of the 29th ACM International Conference on Multimedia, Virtual Event.10.1145/3474085.3475561 ↗
30. Liu, Z., Mao, H., Wu, C.-Y., Feichtenhofer, C., Darrell, T., and Xie, S. (2022, January 18–24). A ConvNet for the 2020s. Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA.10.1109/CVPR52688.2022.01167 ↗
31. Van Horn, G., Mac Aodha, O., Song, Y., Cui, Y., Sun, C., Shepard, A., Adam, H., Perona, P., and Belongie, S. (2018, January 18–23). The iNaturalist Species Classification and Detection Dataset. Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00914 ↗
32. Liu, Z., Luo, P., Qiu, S., Wang, X., and Tang, X. (2016, January 27–30). DeepFashion: Powering Robust Clothes Recognition and Retrieval with Rich Annotations. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.10.1109/CVPR.2016.124 ↗
33. Wei, X.-S., Cui, Q., Yang, L., Wang, P., and Liu, L. (2024, November 07). RPC: A Large-Scale Retail Product Checkout Dataset. Available online: http://arxiv.org/abs/1901.07249.
34. Van Horn, G., Branson, S., Farrell, R., Haber, S., Barry, J., Ipeirotis, P., Perona, P., and Belongie, S. (2015, January 7–12). Building a Bird Recognition App and Large Scale Dataset with Citizen Scientists: The Fine Print in Fine-Grained Dataset Collection. Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA.10.1109/CVPR.2015.7298658 ↗
35. Min, W., Liu, L., Luo, Z., and Jiang, S. (2019, January 15). Ingredient-Guided Cascaded Multi-Attention Network for Food Recognition. Proceedings of the 27th ACM International Conference on Multimedia, Nice, France.10.1145/3343031.3350948 ↗
36. He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27–30). Deep Residual Learning for Image Recognition. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.10.1109/CVPR.2016.90 ↗
37. Chen, T., Lin, L., Chen, R., Wu, Y., and Luo, X. (2018, January 13–19). Knowledge-Embedded Representation Learning for Fine-Grained Image Recognition. Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence, Stockholm, Sweden.10.24963/ijcai.2018/87 ↗
38. Branson, S., Van Horn, G., Perona, P., and Belongie, S. (2014, January 1–5). Improved Bird Species Recognition Using Pose Normalized Deep Convolutional Nets. Proceedings of the British Machine Vision Conference 2014, Nottingham, UK.10.5244/C.28.87 ↗
39. Multi-Attention Multi-Class Constraint for Fine-Grained Image Recognition10.1007/978-3-030-01270-0_49 ↗
40. Zhou, M., Bai, Y., Zhang, W., Zhao, T., and Mei, T. (2020, January 13–19). Look-Into-Object: Self-Supervised Structure Modeling for Object Recognition. Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.10.1109/CVPR42600.2020.01179 ↗
41. Wang, S., Wang, Z., Li, H., and Ouyang, W. (2020, January 12). Category-Specific Semantic Coherency Learning for Fine-Grained Image Recognition. Proceedings of the 28th ACM International Conference on Multimedia, Seattle, WA, USA.10.1145/3394171.3413871 ↗
42. Learning Scale-Consistent Attention Part Network for Fine-Grained Image Recognition10.1109/TMM.2021.3090274 ↗
43. Channel Interaction Networks for Fine-Grained Image Categorization
44. Learning Attentive Pairwise Interaction for Fine-Grained Classification
45. Transformer with Peak Suppression and Knowledge Guidance for Fine-Grained Image Recognition10.1016/j.neucom.2022.04.037 ↗
46. TransFG: A Transformer Architecture for Fine-Grained Recognition
47. Song, J., and Yang, R. (2021, January 18). Feature Boosting, Suppression, and Diversification for Fine-Grained Visual Classification. Proceedings of the 2021 International Joint Conference on Neural Networks (IJCNN), Shenzhen, China.10.1109/IJCNN52387.2021.9534004 ↗
48. Interweaving Insights: High-Order Feature Interaction for Fine-Grained Visual Recognition
49. Fine-Grained Visual Classification via Internal Ensemble Learning Transformer10.1109/TMM.2023.3244340 ↗
50. Delving into Multimodal Prompting for Fine-Grained Visual Classification
51. Bengio, S., Wallach, H.M., Larochelle, H., Grauman, K., Cesa-Bianchi, N., and Garnett, R. (2018, January 3–8). Maximum-Entropy Fine Grained Classification. Proceedings of the Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems 2018, Montréal, QC, Canada.
52. Hierarchical Gate Network for Fine-Grained Visual Recognition10.1016/j.neucom.2021.10.096 ↗
53. Convolution-Enhanced Bi-Branch Adaptive Transformer With Cross-Task Interaction for Food Category and Ingredient Recognition10.1109/TIP.2024.3374211 ↗