journal article·2025-05-15Open access

Multi-Scale Feature Fusion and Global Context Modeling for Fine-Grained Remote Sensing Image Segmentation

Yifan Li, Gengshen Wu

Applied Sciences· Volume 15· Issue 10

Abstract

High-precision remote sensing image semantic segmentation plays a crucial role in Earth science analysis and urban management, especially in urban remote sensing scenarios with rich details and complex structures. In such cases, the collaborative modeling of global and local contexts is a key challenge for improving segmentation accuracy. Existing methods that rely on single feature extraction architectures, such as convolutional neural networks (i.e., CNNs) and vision transformers, are prone to semantic fragmentation due to their limited feature representation capabilities. To address this issue, we propose a hybrid architecture model called PLGTransformer, which is based on dual-encoder collaborative enhancement and integrates pyramid pooling and graph convolutional network (i.e., GCN) modules. Our model innovatively constructs a parallel encoding architecture combining Swin transformer and CNN: the CNN branch captures fine-grained features such as road and building edges through multi-scale heterogeneous convolutions, while the Swin transformer branch models global dependencies of large-scale land cover using hierarchical window attention. To further strengthen multi-granularity feature fusion, we design a dual-path pyramid pooling module to perform adaptive multi-scale context aggregation for both feature types and dynamically balance local and global contributions using learnable weights. Specifically, we introduce the GCNs to build a topological graph in the feature space, enabling geometric relationship reasoning for multi-scale feature nodes at high resolution. Experiments on the Potsdam and Vaihingen datasets show that our model outperforms contemporary advanced methods and significantly improves segmentation accuracy for small objects such as vehicles and individual buildings, thereby validating the effectiveness of the multi-feature collaborative enhancement mechanism.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: Remote Sensing and Land Use · Remote-Sensing Image Classification

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

Faculty of Data Science, City University of Macau, Macao SAR 999078, China

Funding

Science and Technology Development Fund, Macao SAR

References (57)

Show source-provided references
1. A comprehensive survey and assumption of remote sensing foundation modal
2. Yuan, T., and Hu, B. (2025). REU-Net: A Remote Sensing Image Building Segmentation Network Based on Residual Structure and the Edge Enhancement Attention Module. Appl. Sci., 15.10.3390/app15063206 ↗
3. He, Y., Seng, K.P., Ang, L.M., Peng, B., and Zhao, X. (2025). Hyper-CycleGAN: A New Adversarial Neural Network Architecture for Cross-Domain Hyperspectral Data Generation. Appl. Sci., 15.10.3390/app15084188 ↗
4. Deep Learning for Remote Sensing Data: A Technical Tutorial on the State of the Art10.1109/MGRS.2016.2540798 ↗
5. Kazerouni, A., Karimijafarbigloo, S., Azad, R., Velichko, Y., Bagci, U., and Merhof, D. (2024, January 27–30). FuseNet: Self-Supervised Dual-Path Network For Medical Image Segmentation. Proceedings of the 2024 IEEE International Symposium on Biomedical Imaging (ISBI), Athens, Greece.10.1109/ISBI56570.2024.10635112 ↗
6. Radford, A., Kim, J.W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., and Clark, J. (2021, January 18–24). Learning Transferable Visual Models from Natural Language Supervision. Proceedings of the International Conference on Machine Learning, Salt Lake City, UT, USA.
7. Yuan, L., Chen, D., Chen, Y.L., Codella, N.C.F., Dai, X., Gao, J., Hu, H., Huang, X., Li, B., and Li, C. (2021). Florence: A New Foundation Model for Computer Vision. arXiv.
8. Bao, H., Dong, L., Piao, S., and Wei, F. (2022, January 25–29). BEiT: BERT PreTraining of Image Transformers. Proceedings of the The Tenth International Conference on Learning Representations, Vienna, Austria.
9. Sherrah, J. (2016). Fully convolutional networks for dense semantic labelling of high-resolution aerial imagery. arXiv.
10. Effective sequential classifier training for SVM-based multitemporal remote sensing image classification10.1109/TIP.2018.2808767 ↗
11. Random forest classifier for remote sensing classification10.1080/01431160412331269698 ↗
12. A New Paradigm of Remote Sensing Image Interpretation by Coupling Knowledge Graph and Deep Learning
13. Zhang, S., Wu, G., Gu, J., and Han, J. (2020). Pruning convolutional neural networks with an attention mechanism for remote sensing image classification. Electronics, 9.10.3390/electronics9081209 ↗
14. DenseU-net-based semantic segmentation of small objects in urban remote sensing images10.1109/ACCESS.2019.2917952 ↗
15. Attention-Based Multi-Kernelized and Boundary-Aware Network for image semantic segmentation10.1016/j.neucom.2024.127988 ↗
16. Gaussian dynamic convolution for efficient single-image segmentation10.1109/TCSVT.2021.3096814 ↗
17. Li, Z., Wu, G., and Liu, Y. (2024, January 29–31). Prototype Enhancement for Few-Shot Point Cloud Semantic Segmentation. Proceedings of the International Conference on Algorithms and Architectures for Parallel Processing, Macau, China.10.1007/978-981-96-1528-5_18 ↗
18. Krizhevsky, A., Sutskever, I., and Hinton, G.E. (2012, January 3–6). ImageNet classification with deep convolutional neural networks. Proceedings of the NIPS, Red Hook, NY, USA.
19. An, W., and Wu, G. (2024). Hybrid spatial-channel attention mechanism for cross-age face recognition. Electronics, 13.10.3390/electronics13071257 ↗
20. Parallel complement network for real-time semantic segmentation of road scenes10.1109/TITS.2020.3044672 ↗
21. Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., and Guo, B. (2021, January 10–17). Swin Transformer: Hierarchical vision transformer using shifted windows. Proceedings of the ICCV, Montreal, QC, Canada.10.1109/ICCV48922.2021.00986 ↗
22. Multi-scale medical image segmentation based on pixel encoding and spatial attention mechanism
23. Simonyan, K., and Zisserman, A. (2015, January 7–9). Very deep convolutional networks for large-scale image recognition. Proceedings of the ICLR, San Diego, CA, USA.
24. Long, J., Shelhamer, E., and Darrell, T. (2015, January 7–12). Fully convolutional networks for semantic segmentation. Proceedings of the CVPR, Boston, MA, USA.10.1109/CVPR.2015.7298965 ↗
25. Ronneberger, O., Fischer, P., and Brox, T. (2015, January 5–9). U-Net: Convolutional networks for biomedical image segmentation. Proceedings of the MICCAI, Munich, Germany.10.1007/978-3-319-24574-4_28 ↗
26. Chen, L.C., Zhu, Y., Papandreou, G., Schroff, F., and Adam, H. (2018, January 8–14). Encoder-decoder with atrous separable convolution for semantic image segmentation. Proceedings of the ECCV, Munich, Germany.10.1007/978-3-030-01234-2_49 ↗
27. A self-attention CNN for remote sensing image semantic segmentation10.1109/TGRS.2020.3009143 ↗
28. Scene classification with recurrent attention of VHR remote sensing images10.1109/TGRS.2018.2864987 ↗
29. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., and Polosukhin, I. (2017, January 4–9). Attention is all you need. Proceedings of the NIPS, Long Beach, CA, USA.
30. Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., and Gelly, S. (2021, January 4). An image is worth 16x16 words: Transformers for image recognition at scale. Proceedings of the Proc. ICLR, Vienna, Austria.
31. Swin transformer embedding UNet for remote sensing image semantic segmentation10.1109/TGRS.2022.3230846 ↗
32. Multi-scale spatial context-aware transformer for remote sensing image segmentation
33. Bazi, Y., Bashmal, L., Al Rahhal, M.M., Dayil, R.A., and Ajami, N. (2021). Vision transformers for remote sensing image classification. Remote Sens., 13.10.3390/rs13030516 ↗
34. Liu, C., Wu, H., Li, Y., and Li, X. (2022). SwinFCN: A spatial attention Swin transformer backbone for semantic segmentation of high-resolution aerial images. Remote Sens., 14.
35. Kipf, T.N., and Welling, M. (2017, January 24–26). Semi-supervised classification with graph convolutional networks. Proceedings of the 5th International Conference on Learning Representations (ICLR 2017), Toulon, France.
36. A deep neural network combined CNN and GCN for remote sensing scene classification10.1109/JSTARS.2020.3011333 ↗
37. Chen, Z., Badrinarayanan, V., Lee, C.Y., and Rabinovich, A. (November, January 27). GradNet: Gradient-guided network for visual object tracking. Proceedings of the ICCV, Seoul, Republic of Korea.
38. Dual attention graph convolutional network for semantic segmentation of very high resolution remote sensing imagery10.1109/TGRS.2022.3231215 ↗
39. Multi-scale graph convolutional network for remote sensing image segmentation
40. Wang, X., Girshick, R., Gupta, A., and He, K. (2018, January 18–22). Non-local neural networks. Proceedings of the CVPR, Salt Lake City, UT, USA.10.1109/CVPR.2018.00813 ↗
41. MAT-GCN: A multi-scale attention guided graph convolutional network for remote sensing scene classification
42. Gu, A., and Dao, T. (2023). Mamba: Linear-Time Sequence Modeling with Selective State Spaces. arXiv.
43. Liu, M., Dan, J., Lu, Z., Yu, Y., Li, Y., and Li, X. (2024). CM-UNet: Hybrid CNN-Mamba UNet for Remote Sensing Image Semantic Segmentation. arXiv.
44. Multimodal fusion for remote sensing image segmentation using Mamba model10.3390/rs16010019 ↗
45. Redmon, J., Divvala, S., Girshick, R., and Farhadi, A. (2015, January 7–12). You Only Look Once: Unified, Real-Time Object Detection. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA.10.1109/CVPR.2016.91 ↗
46. Real-time remote sensing image change detection using YOLO
47. Vennerød, C., Kjærran, A., and Bugge, E. (2021). Long Short-term Memory RNN. arXiv.
48. Remote sensing image segmentation based on LSTM for urban change detection
49. Cao, H., Wang, Y., Chen, J., Jiang, D., Zhang, X., Tian, Q., and Wang, M. (October, January 29). Swin-Unet: Unet-like Pure Transformer for Medical Image Segmentation. Proceedings of the ECCV Workshops, Milan, Italy.
50. Chen, Y., Lin, G., Li, S., Bourahla, O.E., Wu, Y., Wang, F., Feng, J., Xu, M., and Li, X. (2020, January 13–19). BANet: Bidirectional Aggregation Network With Occlusion Handling for Panoptic Segmentation. Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.10.1109/CVPR42600.2020.00385 ↗
51. UNetFormer: A UNet-like transformer for efficient semantic segmentation of remote sensing urban scene imagery10.1016/j.isprsjprs.2022.06.008 ↗
52. Xiang, J., Liu, J., Chen, D., Xiong, Q., and Deng, C. (2023). CTFuseNet: A Multi-Scale CNN-Transformer Feature Fused Network for Crop Type Segmentation on UAV Remote Sensing Imagery. Remote Sens., 15.10.3390/rs15041151 ↗
53. Selvaraju, R.R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., and Batra, D. (2017, January 22–29). Grad-cam: Visual explanations from deep networks via gradient-based localization. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.10.1109/ICCV.2017.74 ↗
54. Large-Scale Combined Adjustment of Optical Satellite Imagery and ICESat-2 Data through Terrain Profile Elevation Sequence Similarity Matching10.1109/JSTARS.2024.3481449 ↗
55. LIE-DSM: Leveraging Single Remote Sensing Imagery to Enhance Digital Surface Model Resolution
56. SwinFusion: Cross-domain long-range learning for general image fusion via swin transformer10.1109/JAS.2022.105686 ↗
57. A multilevel multimodal fusion transformer for remote sensing semantic segmentation