Grouping Bilinear Pooling for Fine-Grained Image Classification
Source abstract
Abstract
Fine-grained image classification is a challenging computer visual task due to the small interclass variations and large intra-class variations. Extracting expressive feature representation is an effective way to improve the accuracy of fine-grained image classification. Bilinear pooling is a simple and effective high-order feature interaction method. Compared with common pooling methods, bilinear pooling can obtain better feature representation by capturing complex associations between high-order features. However, the dimensions of bilinear representation are often up to hundreds of thousands or even millions. In order to get compact bilinear representation, we propose grouping bilinear pooling (GBP) for fine-grained image classification in this paper. Firstly, by dividing the feature layers into different groups, and then carrying out intra-group bilinear pooling or inter-group bilinear pooling. The representation captured by GBP can achieve the same accuracy with less than 0.4% parameters compared with full bilinear representation when using the same backbone. This extreme compact representation largely overcomes the high redundancy of the full bilinear representation, the computational cost and storage consumption. Besides, it is because GBP compresses the bilinear representation to the extreme that it can be used with more powerful backbones as a plug-and-play module. The effectiveness of GBP is proved by experiments on the widely used fine-grained recognition datasets CUB and Stanford Cars.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Advanced Neural Network Applications · Advanced Image and Video Retrieval Techniques · Domain Adaptation and Few-Shot Learning
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
School of Microelectronics, University of Science and Technology of China, Hefei 230026, China
References (39)
Show source-provided references
1. Wah, C., Branson, S., Welinder, P., Perona, P., and Belongie, S. (2011). The Caltech-UCSD Birds-200-2011 Dataset, California Institute of Technology. Computation & Neural Systems Technical Report, 2010-001.
2. Krause, J., Stark, M., Deng, J., and Fei-Fei, L. (2013, January 2–8). 3D Object Representations for Fine-Grained Categorization. Proceedings of the 2013 IEEE International Conference on Computer Vision Workshops, Sydney, Australia.10.1109/ICCVW.2013.77 ↗
3. Sohaib, M., and Kim, J.M. (2019). Data Driven Leakage Detection and Classification of a Boiler Tube. Appl. Sci., 9.10.3390/app9122450 ↗
4. Wang, E., Jiang, Y., Li, Y., Yang, J., and Zhang, Q. (2019). MFCSNet: Multi-Scale Deep Features Fusion and Cost-Sensitive Loss Function Based Segmentation Network for Remote Sensing Images. Appl. Sci., 9.10.3390/app9194043 ↗
5. DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs10.1109/TPAMI.2017.2699184 ↗
6. Zeiler, M., and Fergus, R. (2013, January 2–4). Stochastic Pooling for Regularization of Deep Convolutional Neural Networks. Proceedings of the International Conference on Learning Representations (ICLR), Scottsdale, AZ, USA.
7. Yu, D., Wang, H., Chen, P., and Wei, Z. (2014). Mixed Pooling for Convolutional Neural Networks. International Conference On Rough Sets and Knowledge Technology, Springer.10.1007/978-3-319-11740-9_34 ↗
8. Squeeze-and-Excitation Networks10.1109/TPAMI.2019.2913372 ↗
9. Sun, M., Yuan, Y., Zhou, F., and Ding, E. (2018, January 8–14). Multi-Attention Multi-Class Constraint for Fine-grained Image Recognition. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.10.1007/978-3-030-01270-0_49 ↗
10. Lin, T.Y., Dollár, P., Girshick, R., He, K., Hariharan, B., and Belongie, S. (2017, January 21–26). Feature Pyramid Networks for Object Detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.10.1109/CVPR.2017.106 ↗
11. Daniilidis, K., Maragos, P., and Paragios, N. (2010). Improving the Fisher Kernel for Large-Scale Image Classification. European Conference on Computer Vision, Springer.
12. Perronnin, F., and Dance, C. (2007, January 18–23). Fisher Kernels on Visual Vocabularies for Image Categorization. Proceedings of the 2007 IEEE Conference on Computer Vision and Pattern Recognition, Minneapolis, MN, USA.10.1109/CVPR.2007.383266 ↗
13. Jégou, H., Douze, M., Schmid, C., and Pérez, P. (2010, January 13–18). Aggregating local descriptors into a compact image representation. Proceedings of the 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, San Francisco, CA, USA.10.1109/CVPR.2010.5540039 ↗
14. Lazebnik, S., Schmid, C., and Ponce, J. (2006, January 17–22). Beyond Bags of Features: Spatial Pyramid Matching for Recognizing Natural Scene Categories. Proceedings of the 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’06), New York, NY, USA.
15. Lin, T.Y., RoyChowdhury, A., and Maji, S. (2015, January 7–13). Bilinear CNN Models for Fine-Grained Visual Recognition. Proceedings of the 2015 IEEE International Conference on Computer Vision (ICCV), Santiago, Chile.10.1109/ICCV.2015.170 ↗
16. Yu, C., Zhao, X., Zheng, Q., Zhang, P., and You, X. (2018, January 8–14). Hierarchical Bilinear Pooling for Fine-Grained Visual Recognition. Proceedings of the Computer Vision–ECCV 2018, Munich, Germany.10.1007/978-3-030-01270-0_35 ↗
17. Gao, Y., Beijbom, O., Zhang, N., and Darrell, T. (2016, January 27–30). Compact Bilinear Pooling. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.10.1109/CVPR.2016.41 ↗
18. Ni, Z.L., Bian, G.B., Wang, G., Zhou, X.H., Hou, Z.G., Xie, X.L., Chen, H.B., and Li, Z. (2020, January 7–12). Pyramid Attention Aggregation Network for Semantic Segmentation of Surgical Instruments. Proceedings of the AAAI Conference on Artificial Intelligence, New York, NY, USA.
19. Kong, S., and Fowlkes, C. (2017, January 21–26). Low-Rank Bilinear Pooling for Fine-Grained Classification. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.10.1109/CVPR.2017.743 ↗
20. Zheng, H., Fu, J., Zha, Z.J., and Luo, J. (2019). Learning Deep Bilinear Transformation for Fine-grained Image Representation. Advances in Neural Information Processing Systems, Curran Associates, Inc.
21. Random feature maps for dot product kernels
22. Pham, N., and Pagh, R. (2013, January 11–14). Fast and scalable polynomial kernels via explicit feature maps. Proceedings of the 19th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Chicago, IL, USA.10.1145/2487575.2487591 ↗
23. Fukui, A., Park, D., Yang, D., Rohrbach, A., Darrell, T., and Rohrbach, M. (2016). Multimodal Compact Bilinear Pooling for Visual Question Answering and Visual Grounding. arXiv.10.18653/v1/D16-1044 ↗
24. Suh, Y., Wang, J., Tang, S., Mei, T., and Lee, K.M. (2018, January 8–14). Part-aligned bilinear representations for person re-identification. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.10.1007/978-3-030-01264-9_25 ↗
25. Yu, T., Meng, J., and Yuan, J. (2018, January 18–23). Multi-view harmonized bilinear network for 3d object recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00027 ↗
26. Hu, J.F., Zheng, W.S., Pan, J., Lai, J., and Zhang, J. (2018, January 8–14). Deep bilinear learning for rgb-d action recognition. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.10.1007/978-3-030-01234-2_21 ↗
27. Simonyan, K., and Zisserman, A. (2014). Very Deep Convolutional Networks for Large-Scale Image Recognition. arXiv.
28. He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27–30). Deep Residual Learning for Image Recognition. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.10.1109/CVPR.2016.90 ↗
29. Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., and Fei-Fei, L. (2009, January 20–25). ImageNet: A large-scale hierarchical image database. Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition, Miami, FL, USA.10.1109/CVPR.2009.5206848 ↗
30. Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., and Antiga, L. (2019). PyTorch: An Imperative Style, High-Performance Deep Learning Library. Advances in Neural Information Processing Systems, Curran Associates, Inc.
31. Kingma, D., and Ba, J. (2014, January 14–16). Adam: A Method for Stochastic Optimization. Proceedings of the International Conference on Learning Representations, Banff, AB, Canada.
32. Lin, T.Y., and Maji, S. (2017). Improved Bilinear Pooling with CNNs. arXiv.10.5244/C.31.117 ↗
33. Gou, M., Xiong, F., Camps, O., and Sznaier, M. (2018, January 18–22). MoNet: Moments Embedding Network. Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00335 ↗
34. Gao, Z., Wu, Y., Zhang, X., Dai, J., Jia, Y., and Harandi, M. (2020, January 7–12). Revisiting Bilinear Pooling: A Coding Perspective. Proceedings of the AAAI Conference on Artificial Intelligence, New York, NY, USA.10.1609/aaai.v34i04.5811 ↗
35. Liao, Q., Wang, D., Holewa, H., and Xu, M. (2019, January 27–28). Squeezed Bilinear Pooling for Fine-Grained Visual Categorization. Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW), Seoul, Korea.10.1109/ICCVW.2019.00093 ↗
36. Deep CNNs with Spatially Weighted Pooling for Fine-Grained Car Recognition10.1109/TITS.2017.2679114 ↗
37. Fine-Grained Classification via Hierarchical Bilinear Pooling with Aggregated Slack Mask10.1109/ACCESS.2019.2936118 ↗
38. Learning Semantically Enhanced Feature for Fine-Grained Image Classification10.1109/LSP.2020.3020227 ↗
39. The Devil is in the Channels: Mutual-Channel Loss for Fine-Grained Image Classification10.1109/TIP.2020.2973812 ↗