Multi-Scale Context-Aware Network Implementation for Efficient Image Semantic Segmentation
Source abstract
Abstract
Image semantic segmentation is essential in autonomous driving, medical imaging, and remote sensing. While convolutional neural networks (CNNs) excel at local feature extraction and spatial structure modeling, their limited receptive fields restrict the capture of long-range dependencies and global semantic consistency. Transformers provide strong global modeling through self-attention but often lack local inductive bias and show weaker generalization on small datasets. To address these limitations, this paper proposes a Multi-Scale Context-aware Network (MSC-Net) for image semantic segmentation. Under an encoder–decoder framework, MSC-Net combines a convolutional backbone with a Multi-Scale Self-Attention module to integrate the complementary strengths of CNNs and attention mechanisms. The backbone extracts local texture and structural information and can adopt architectures such as MobileNet, Xception, DRN, and ResNet, while the attention module captures long-range dependencies and multi-scale contextual information. This design improves cross-layer feature collaboration, multi-scale feature fusion, and boundary quality while maintaining computational efficiency. Experimental results show that MSC-Net achieves 38.8% mIoU and 98.4% ACC under comparable computational settings. Compared with SegFormer and DeepLabV3+, the model improves mIoU by approximately +3.0 and +3.3 percentage points, respectively, while reducing FLOPs and parameter size.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Advanced Neural Network Applications · Generative Adversarial Networks and Image Synthesis · Big Data and Digital Economy
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
SAIC Motor Corporation Limited, Shanghai 201804, China
The College of Automotive Engineering, Jilin University, Changchun 130025, China
References (36)
Show source-provided references
1. Deep learning-based perception systems for autonomous driving: A comprehensive survey10.1016/j.neucom.2021.08.155 ↗
2. Dual-branch aggregation and edge refinement network for few shot semantic segmentation10.1007/s00530-025-01718-4 ↗
3. Deep-learning-based downscaling of precipitation in the middle reaches of the Yellow River using residual-based CNNs10.1002/qj.4759 ↗
4. Inference latency prediction for CNNs on heterogeneous mobile devices and ML frameworks10.1016/j.peva.2024.102429 ↗
5. Lu, Y., Chen, Y., Zhao, D., and Chen, J. (2019). Graph-FCN for image semantic segmentation. International Symposium on Neural Networks, Springer International Publishing.10.1007/978-3-030-22796-8_11 ↗
6. Ronneberger, O., Fischer, P., and Brox, T. (2015). U-net: Convolutional networks for biomedical image segmentation. International Conference on Medical Image Computing and Computer-Assisted Intervention, Springer International Publishing.10.1007/978-3-319-24574-4_28 ↗
7. PSP net-based automatic segmentation network model for prostate magnetic resonance imaging10.1016/j.cmpb.2021.106211 ↗
8. Yin, H., Vahdat, A., Alvarez, J.M., Mallya, A., Kautz, J., and Molchanov, P. (2022). A-vit: Adaptive tokens for efficient vision transformer. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, IEEE Computer Society.10.1109/CVPR52688.2022.01054 ↗
9. Yuan, L., Chen, Y., Wang, T., Yu, W., Shi, Y., Jiang, Z., Tay, F.E.H., Feng, J., and Yan, S. (2021). Tokens-to-token vit: Training vision transformers from scratch on imagenet. Proceedings of the IEEE/CVF International Conference on Computer Vision, IEEE Computer Society.10.1109/ICCV48922.2021.00060 ↗
10. Long, J., Shelhamer, E., and Darrell, T. (2015). Fully convolutional networks for semantic segmentation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, (CVPR), IEEE Computer Society Conference Publishing Services (CPS).10.1109/CVPR.2015.7298965 ↗
11. Chen, L.-C., Zhu, Y., Papandreou, G., Schroff, F., and Adam, H. (2018). Encoder-decoder with atrous separable convolution for semantic image segmentation. Proceedings of the European Conference on Computer Vision (ECCV), Springer.10.1007/978-3-030-01234-2_49 ↗
12. Zheng, S., Lu, J., Zhao, H., Zhu, X., Luo, Z., Wang, Y., Fu, Y., Feng, J., Xiang, T., and Torr, P.H. (2021, January 20–25). Rethinking semantic segmentation from a sequence-to-sequence perspective with transformers. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Nashville, TN, USA.10.1109/CVPR46437.2021.00681 ↗
13. Strudel, R., Garcia, R., Laptev, I., and Schmid, C. (2021, January 10–17). Segmenter: Transformer for semantic segmentation. Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), Montreal, QC, Canada.10.1109/ICCV48922.2021.00717 ↗
14. Xie, E., Wang, W., Yu, Z., Anandkumar, A., Alvarez, J.M., and Luo, P. (2021). SegFormer: Simple and efficient design for semantic segmentation with transformers. Advances in Neural Information Processing Systems (NeurIPS) 34, American Institute of Physics.
15. Pan, H., Hong, Y., Sun, W., and Jia, Y. (2021). Deep dual-resolution networks for real-time and accurate semantic segmentation of road scenes. arXiv.
16. Xu, J., Xiong, Z., and Bhattacharyya, S.P. (2023). PIDNet: A Real-time semantic segmentation network inspired by PID controllers. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), IEEE Computer Society.10.1109/CVPR52729.2023.01871 ↗
17. Ding, E., Feng, H., Gou, C., Han, J., Wang, J., Wang, J., and Wu, Q. (2022). RTFormer: Efficient design for real-time semantic segmentation with transformer. Advances in Neural Information Processing Systems (NeurIPS) 35, American Institute of Physics.10.52202/068431-0539 ↗
18. Xu, G., Li, J., Gao, G., Lu, H., Yang, J., and Yue, D. (2023). Lightweight real-time semantic segmentation network with efficient transformer and CNN. arXiv.10.1109/TITS.2023.3248089 ↗
19. Xu, G., Jia, W., Wu, T., Chen, L., and Gao, G. (2024). HAFormer: Unleashing the power of hierarchy-aware features for lightweight semantic segmentation. arXiv.10.1109/TIP.2024.3425048 ↗
20. Fan, M., Lai, S., Huang, J., Wei, X., Chai, Z., Luo, J., and Wei, X. (2021). Rethinking BiSeNet for real-time semantic segmentation. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), IEEE Computer Society.10.1109/CVPR46437.2021.00959 ↗
21. Gu, J., Kwon, H., Wang, D., Ye, W., Li, M., Chen, Y.-H., Lai, L., Chandra, V., and Pan, D.Z. (2022). Multi-scale high-resolution vision transformer for semantic segmentation. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), IEEE Computer Society.10.1109/CVPR52688.2022.01178 ↗
22. Yuan, Y., Fu, R., Huang, L., Lin, W., Zhang, C., Chen, X., and Wang, J. (2021). HRFormer: High-resolution transformer for dense prediction. Advances in Neural Information Processing Systems (NeurIPS) 34, American Institute of Physics.
23. Cheng, B., Misra, I., Schwing, A.G., Kirillov, A., and Girdhar, R. (2022). Masked-attention mask transformer for universal image segmentation. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), IEEE Computer Society.10.1109/CVPR52688.2022.00135 ↗
24. Chen, L.-C., Papandreou, G., Schroff, F., and Adam, H. (2017). Rethinking atrous convolution for semantic image segmentation. arXiv.
25. Sun, K., Zhao, Y., Jiang, B., Cheng, T., Xiao, B., Liu, D., and Wang, J. (2019). High-resolution representations for labeling pixels and regions. arXiv.
26. Targ, S., Almeida, D., and Lyman, K. (2016). Resnet in resnet: Generalizing residual architectures. arXiv.
27. Chollet, F. (2017). Xception: Deep learning with depthwise separable convolutions. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, IEEE Computer Society Conference Publishing Services (CPS).10.1109/CVPR.2017.195 ↗
28. Qin, Z., Zhang, Z., Chen, X., Wang, C., and Peng, Y. (2018). Fd-Mobilenet: Improved mobilenet with a fast downsampling strategy. Proceedings of the 2018 25th IEEE International Conference on Image Processing (ICIP), IEEE.10.1109/ICIP.2018.8451355 ↗
29. Filter-based deep-compression with global average pooling for convolutional networks10.1016/j.sysarc.2019.02.008 ↗
30. Zhang, Z., and Pfister, T. (2021). Learning fast sample re-weighting without reward data. Proceedings of the IEEE/CVF International Conference on Computer Vision, IEEE Computer Society.10.1109/ICCV48922.2021.00076 ↗
31. Peng, H., Pian, W., Sun, M., and Li, P. (2023). Dynamic re-weighting for long-tailed semi-supervised learning. Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, IEEE Computer Society.10.1109/WACV56688.2023.00640 ↗
32. Increased rates of convergence through learning rate adaptation10.1016/0893-6080(88)90003-2 ↗
33. Lin, T., Goyal, P., Girshick, R., He, K., and Dollár, P. (2017). Focal loss for dense object detection. Proceedings of the IEEE International Conference on Computer Vision, IEEE.10.1109/ICCV.2017.324 ↗
34. Bakogiannis, T., Giannakopoulos, I., Tsoumakos, D., and Koziris, N. (2019). Apollo: A dataset profiling and operator mod-eling system. Proceedings of the 2019 International Conference on Management of Data, Association for Computing Machinery.10.1145/3299869.3320220 ↗
35. Vigo, M., Arrue, M., Brajnik, G., Lomuscio, R., and Abascal, J. (2007). Quantitative metrics for measuring web accessibility. Proceedings of the 2007 International Cross-Disciplinary Conference on Web Accessibility (W4A), Association for Computing Machinery.10.1145/1243441.1243465 ↗
36. Sehestedt, S., Cheng, C.-H., and Bouwers, E. (2014). Towards quantitative metrics for architecture models. Proceedings of the WICSA 2014 Companion Volume, Association for Computing Machinery.10.1145/2578128.2578226 ↗