Semantic Multigranularity Feature Learning for High-Resolution Remote Sensing Image Scene Classification
Source abstract
Abstract
High-resolution remote sensing image scene classification is a challenging visual task due to the large intravariance and small intervariance between the categories. To accurately recognize the scene categories, it is essential to learn discriminative features from both global and local critical regions. Recent efforts focus on how to encourage the network to learn multigranularity features with the destruction of the spatial information on the input image at different scales, which leads to meaningless edges that are harmful to training. In this study, we propose a novel method named Semantic Multigranularity Feature Learning Network (SMGFL-Net) for remote sensing image scene classification. The core idea is to learn both global and multigranularity local features from rearranged intermediate feature maps, thus, eliminating the meaningless edges. These features are then fused for the final prediction. Our proposed framework is compared with a collection of state-of-the-art (SOTA) methods on two fine-grained remote sensing image scene datasets, including the NWPU-RESISC45 and Aerial Image Datasets (AID). We justify several design choices, including the branch granularities, fusion strategies, pooling operations, and necessity of feature map rearrangement through a comparative study. Moreover, the overall performance results show that SMGFL-Net consistently outperforms other peer methods in classification accuracy, and the superiority is more apparent with less training data, demonstrating the efficacy of feature learning of our approach.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Remote-Sensing Image Classification · Advanced Image and Video Retrieval Techniques · Remote Sensing and Land Use
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Geo Informatics Lab, School of Civil & Architecture Engineering, Natural Science Campus, Sungkyunkwan University, Suwon 16419, Korea
School of Engineering, Penn State Erie, The Behrend College, Erie, PA 16563, USA
References (41)
Show source-provided references
1. Campbell, J.B., and Wynne, R.H. (2011). Introduction to Remote Sensing, Guilford Press.
2. Imagenet classification with deep convolutional neural networks
3. Simonyan, K., and Zisserman, A. (2014). Very deep convolutional networks for large-scale image recognition. arXiv.
4. He, K., Zhang, X., Ren, S., and Sun, J. (July, January 26). Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.
5. Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., and Wojna, Z. (July, January 26). Rethinking the inception architecture for computer vision. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.
6. Szegedy, C., Ioffe, S., Vanhoucke, V., and Alemi, A.A. (2017, January 4–9). Inception-v4, inception-resnet and the impact of residual connections on learning. Proceedings of the Thirty-first AAAI Conference on Artificial Intelligence, San Francisco, CA, USA.10.1609/aaai.v31i1.11231 ↗
7. Imagenet large scale visual recognition challenge10.1007/s11263-015-0816-y ↗
8. Jaderberg, M., Simonyan, K., Zisserman, A., and Kavukcuoglu, K. (2015). Spatial transformer networks. arXiv.
9. Yang, Z., Luo, T., Wang, D., Hu, Z., Gao, J., and Wang, L. (2018, January 8–14). Learning to navigate for fine-grained classification. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.10.1007/978-3-030-01264-9_26 ↗
10. Zhang, X., Xiong, H., Zhou, W., Lin, W., and Tian, Q. (July, January 26). Picking deep filter responses for fine-grained image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.
11. Fu, J., Zheng, H., and Mei, T. (2018, January 18–22). Look closer to see better: Recurrent attention convolutional neural network for fine-grained image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2017.476 ↗
12. Zhang, F., Li, M., Zhai, G., and Liu, Y. (2021, January 22–24). Multi-branch and multi-scale attention learning for fine-grained visual categorization. Proceedings of the International Conference on Multimedia Modeling, Online.10.1007/978-3-030-67832-6_12 ↗
13. Lin, T.Y., RoyChowdhury, A., and Maji, S. (2015, January 11–18). Bilinear cnn models for fine-grained visual recognition. Proceedings of the IEEE International Conference on Computer Vision, Santiago, Chile.10.1109/ICCV.2015.170 ↗
14. Gao, Y., Beijbom, O., Zhang, N., and Darrell, T. (July, January 26). Compact bilinear pooling. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.
15. Kim, J.H., On, K.W., Lim, W., Kim, J., Ha, J.W., and Zhang, B.T. (2016). Hadamard product for low-rank bilinear pooling. arXiv.
16. Liao, Q., Wang, D., Holewa, H., and Xu, M. (November, January 27). Squeezed bilinear pooling for fine-grained visual categorization. Proceedings of the IEEE/CVF International Conference on Computer Vision Workshops, Seoul, Korea.
17. Chen, Y., Bai, Y., Zhang, W., and Mei, T. (2019, January 15–20). Destruction and construction learning for fine-grained image recognition. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.10.1109/CVPR.2019.00530 ↗
18. Zhuang, P., Wang, Y., and Qiao, Y. (2020, January 7–12). Learning attentive pairwise interaction for fine-grained classification. Proceedings of the AAAI Conference on Artificial Intelligence, New York, NY, USA.10.1609/aaai.v34i07.7016 ↗
19. Wah, C., Branson, S., Welinder, P., Perona, P., and Belongie, S. (2011). The Caltech-Ucsd Birds-200-2011 Dataset, California Institute of Technology.
20. Krause, J., Stark, M., Deng, J., and Fei-Fei, L. (2013, January 1–8). 3d object representations for fine-grained categorization. Proceedings of the IEEE International Conference on Computer Vision Workshops, Sydney, Australia.10.1109/ICCVW.2013.77 ↗
21. Zhang, L., Huang, S., Liu, W., and Tao, D. (November, January 27). Learning a mixture of granularity-specific experts for fine-grained categorization. Proceedings of the IEEE/CVF International Conference on Computer Vision, Seoul, Korea.
22. Du, R., Chang, D., Bhunia, A.K., Xie, J., Ma, Z., Song, Y.Z., and Guo, J. (2020, January 23–28). Fine-grained visual classification via progressive multi-granularity training of jigsaw patches. Proceedings of the European Conference on Computer Vision, Online.10.1007/978-3-030-58565-5_10 ↗
23. Multiple granularity modeling: A coarse-to-fine framework for fine-grained action analysis10.1007/s11263-016-0891-8 ↗
24. Wei, C., Xie, L., Ren, X., Xia, Y., Su, C., Liu, J., Tian, Q., and Yuille, A.L. (2019, January 16–20). Iterative reorganization with weak spatial constraints: Solving arbitrary jigsaw puzzles for unsupervised representation learning. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.10.1109/CVPR.2019.00201 ↗
25. Remote sensing image scene classification: Benchmark and state of the art10.1109/JPROC.2017.2675998 ↗
26. AID: A benchmark data set for performance evaluation of aerial scene classification10.1109/TGRS.2017.2685945 ↗
27. Sun, M., Yuan, Y., Zhou, F., and Ding, E. (2018, January 8–14). Multi-attention multi-class constraint for fine-grained image recognition. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.10.1007/978-3-030-01270-0_49 ↗
28. Bilinear convolutional neural networks for fine-grained visual recognition10.1109/TPAMI.2017.2723400 ↗
29. Kong, S., and Fowlkes, C. (2017, January 21–26). Low-rank bilinear pooling for fine-grained classification. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.10.1109/CVPR.2017.743 ↗
30. Deep learning based feature selection for remote sensing scene classification10.1109/LGRS.2015.2475299 ↗
31. Liu, Q., Hang, R., Song, H., Zhu, F., Plaza, J., and Plaza, A. (2016). Adaptive deep pyramid matching for remote sensing scene classification. arXiv.
32. Yang, Y., and Newsam, S. (2011, January 6–13). Spatial pyramid co-occurrence for image classification. Proceedings of the 2011 International Conference on Computer Vision, Barcelona, Spain.
33. Satellite image classification via two-layer sparse coding with biased image representation10.1109/LGRS.2010.2055033 ↗
34. Remote sensing scene classification using multilayer stacked covariance pooling10.1109/TGRS.2018.2845668 ↗
35. Zhang, W., Tang, P., and Zhao, L. (2019). Remote sensing image scene classification using CNN-CapsNet. Remote Sens., 11.10.3390/rs11050494 ↗
36. Zhao, Q., Lyu, S., Li, Y., Ma, Y., and Chen, L. (2021). MGML: Multigranularity Multilevel Feature Ensemble Network for Remote Sensing Scene Classification. IEEE Trans. Neural Netw. Learn. Syst.
37. Multilayer feature fusion network for scene classification in remote sensing10.1109/LGRS.2019.2960026 ↗
38. Xu, K., Huang, H., and Deng, P. (2021). Remote Sensing Image Scene Classification Based on Global-Local Dual-Branch Structure Model. IEEE Geosci. Remote Sens. Lett.10.1109/LGRS.2021.3075712 ↗
39. Xu, K., Huang, H., Deng, P., and Li, Y. (2021). Deep Feature Aggregation Framework Driven by Graph Convolutional Network for Scene Classification in Remote Sensing. IEEE Trans. Neural Netw. Learn. Syst.10.1109/TNNLS.2021.3071369 ↗
40. Clevert, D.A., Unterthiner, T., and Hochreiter, S. (2015). Fast and accurate deep network learning by exponential linear units (elus). arXiv.
41. Xie, S., Girshick, R., Dollár, P., Tu, Z., and He, K. (2017, January 21–26). Aggregated residual transformations for deep neural networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.10.1109/CVPR.2017.634 ↗