Detail Guided Multilateral Segmentation Network for Real-Time Semantic Segmentation
Source abstract
Abstract
With the development of unmanned vehicles and other technologies, the technical demand for scene semantic segmentation is more and more intense. Semantic segmentation requires not only rich high-level semantic information, but also rich detail information to ensure the accuracy of the segmentation task. Using a multipath structure to process underlying and semantic information can improve efficiency while ensuring segmentation accuracy. In order to improve the segmentation accuracy and efficiency of some small and thin objects, a detail guided multilateral segmentation network is proposed. Firstly, in order to improve the segmentation accuracy and model efficiency, a trilateral parallel network structure is designed, including the context fusion path (CF-path), the detail information guidance path (DIG-path), and the semantic information supplement path (SIS-path). Secondly, in order to effectively fuse semantic information and detail information, a feature fusion module based on an attention mechanism is designed. Finally, experimental results on CamVid and Cityscapes datasets show that the proposed algorithm can effectively balance segmentation accuracy and inference speed.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Advanced Neural Network Applications · Robotics and Sensor-Based Localization · Visual Attention and Saliency Detection
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Department of Mechanical Engineering, College of Field Engineering, Army Engineering University of PLA, Nanjing 210007, China
Funding
National Natural Science Foundation of China · China National Key Research and Development Program
References (41)
Show source-provided references
1. Han, S., Mao, H., and Dally, W.J. (2015). Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding. arXiv.
2. Bhattacharya, S., and Lane, N.D. (2016, January 14–16). Sparsification and separation of deep learning layers for constrained resource inference on wearables. Proceedings of the 14th ACM Conf. Embedded Network Sensor System (CD-ROM), Stanford, CA, USA.10.1145/2994551.2994564 ↗
3. Liu, Z., Li, J., Shen, Z., Huang, G., Yan, S., and Zhang, C. (2017, January 22–29). Learning Efficient Convolutional Networks through Network Slimming. Proceedings of the 16th IEEE International Conference on Computer Vision (ICCV), Venice, Italy.10.1109/ICCV.2017.298 ↗
4. SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation10.1109/TPAMI.2016.2644615 ↗
5. Zhao, H., Qi, X., Shen, X., Shi, J., and Jia, J. (2018, January 8–14). ICNet for Real-Time Semantic Segmentation on High-Resolution Images. Proceedings of the 15th European Conference on Computer Vision (ECCV), Munich, Germany.10.1007/978-3-030-01219-9_25 ↗
6. Yu, C., Wang, J., Peng, C., Gao, C., Yu, G., and Sang, N. (2018, January 8–14). BiSeNet: Bilateral Segmentation Network for Real-Time Semantic Segmentation. Proceedings of the 15th European Conference on Computer Vision (ECCV), Munich, Germany.10.1007/978-3-030-01261-8_20 ↗
7. BiSeNet V2: Bilateral Network with Guided Aggregation for Real-Time Semantic Segmentation10.1007/s11263-021-01515-2 ↗
8. Fan, M., Lai, S., Huang, J., Wei, X., Chai, Z., Luo, J., and Wei, X. (2021, January 19–25). Rethinking BiSeNet For Real-time Semantic Segmentation. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Electric Network, Nashville, TN, USA.10.1109/CVPR46437.2021.00959 ↗
9. Takikawa, T., Acuna, D., Jampani, V., and Fidler, S. (November, January 27). Gated-SCNN: Gated Shape CNNs for Semantic Segmentation. Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), Seoul, Korea.
10. Cordts, M., Omran, M., Ramos, S., Rehfeld, T., Enzweiler, M., Benenson, R., Franke, U., Roth, S., and Schiele, B. (2016, January 27–30). The Cityscapes Dataset for Semantic Urban Scene Understanding. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.10.1109/CVPR.2016.350 ↗
11. Brostow, G.J., Shotton, J., Fauqueur, J., and Cipolla, R. (2008, January 12–18). Segmentation and Recognition Using Structure from Motion Point Clouds. Proceedings of the 10th European Conference on Computer Vision (ECCV 2008), Marseille, France.10.1007/978-3-540-88682-2_5 ↗
12. Long, J., Shelhamer, E., and Darrell, T. (2015, January 7–12). Fully convolutional networks for semantic segmentation. Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA.10.1109/CVPR.2015.7298965 ↗
13. Zhao, H., Shi, J., Qi, X., Wang, X., and Jia, J. (2017, January 21–26). Pyramid Scene Parsing Network. Proceedings of the 30th IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.10.1109/CVPR.2017.660 ↗
14. Ronneberger, O. (2017). Invited Talk: U-Net Convolutional Networks for Biomedical Image Segmentation, Springer.10.1007/978-3-662-54345-0_3 ↗
15. DeepLab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs
16. Chen, L.-C., Papandreou, G., Schroff, F., and Adam, H.J. (2017). Rethinking Atrous Convolution for Semantic Image Segmentation. arXiv.
17. Chen, L.-C., Zhu, Y., Papandreou, G., Schroff, F., and Adam, H. (2018, January 8–14). Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation. Proceedings of the 15th European Conference on Computer Vision (ECCV), Munich, Germany.10.1007/978-3-030-01234-2_49 ↗
18. Paszke, A., Chaurasia, A., Sangpil, K., and Culurciello, E. (2016). ENet: A Deep Neural Network Architecture for Real-Time Semantic Segmentation. arXiv.
19. Li, H., Xiong, P., Fan, H., Sun, J., and Soc, I.C. (2019, January 16–20). DFANet: Deep Feature Aggregation for Real-Time Semantic Segmentation. Proceedings of the 32nd IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA.10.1109/CVPR.2019.00975 ↗
20. Li, X., Zhou, Y., Pan, Z., Feng, J., and Soc, I.C. (2019, January 16–20). Partial Order Pruning: For Best Speed/Accuracy Trade-off in Neural Architecture Search. Proceedings of the 32nd IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA.10.1109/CVPR.2019.00936 ↗
21. Lin, P., Sun, P., Cheng, G., Xie, S., Li, X., and Shi, J. (2020, January 14–19). Graph-guided Architecture Search for Real-time Semantic Segmentation. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Electric Network, Seattle, DC, USA.10.1109/CVPR42600.2020.00426 ↗
22. Howard, A., Sandler, M., Chu, G., Chen, L.-C., Chen, B., Tan, M., Wang, W., Zhu, Y., Pang, R., and Vasudevan, V. (November, January 27). Searching for MobileNetV3. Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), Seoul, Korea.
23. Zhao, H., Zhang, Y., Liu, S., Shi, J., Loy, C.C., Lin, D., and Jia, J. (2018, January 8–14). PSANet: Point-wise Spatial Attention Network for Scene Parsing. Proceedings of the 15th European Conference on Computer Vision (ECCV), Munich, Germany.10.1007/978-3-030-01240-3_17 ↗
24. He, J., Deng, Z., Zhou, L., Wang, Y., Qiao, Y., and Soc, I.C. (2019, January 16–20). Adaptive Pyramid Context Network for Semantic Segmentation. Proceedings of the 32nd IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA.10.1109/CVPR.2019.00770 ↗
25. Self-attention feature fusion network for semantic segmentation
26. Lin, G., Milan, A., Shen, C., and Reid, I. (2017, January 21–26). RefineNet: Multi-Path Refinement Networks for High-Resolution Semantic Segmentation. Proceedings of the 30th IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.10.1109/CVPR.2017.549 ↗
27. Song, Q., Mei, K., and Huang, R. (2021, January 2–9). AttaNet: Attention-Augmented Network for Fast and Accurate Scene Parsing. Proceedings of the 11th Symposium on Educational Advances in Artificial Intelligence, Electric Network, Virtual.
28. Song, Q., Li, J., Li, C., Guo, H., and Huang, R.J. (2021). Fully Attentional Network for Semantic Segmentation. arXiv.
29. Mehta, S., Rastegari, M., Caspi, A., Shapiro, L., and Hajishirzi, H. (2018, January 8–14). ESPNet: Efficient Spatial Pyramid of Dilated Convolutions for Semantic Segmentation. Proceedings of the 15th European Conference on Computer Vision (ECCV), Munich, Germany.10.1007/978-3-030-01249-6_34 ↗
30. ERFNet: Efficient Residual Factorized ConvNet for Real-Time Semantic Segmentation10.1109/TITS.2017.2750080 ↗
31. AGLNet: Towards real-time semantic segmentation of self-driving images via attention-guided lightweight network10.1016/j.asoc.2020.106682 ↗
32. Li, G., Yun, I., Kim, J., and Kim, J.J. (2019). DABNet: Depth-wise Asymmetric Bottleneck for Real-time Semantic Segmentation. arXiv.
33. Wang, Y., Zhou, Q., Liu, J., Xiong, J., Gao, G., Wu, X., and Latecki, L.J. (2019, January 22–25). LEDNET: A Lightweight Encoder-Decoder Network for Real-Time Semantic Segmentation. Proceedings of the 26th IEEE International Conference on Image Processing (ICIP), Taipei, Taiwan.10.1109/ICIP.2019.8803154 ↗
34. DSANet: Dilated spatial attention for real-time semantic segmentation in urban street scenes10.1016/j.eswa.2021.115090 ↗
35. Liu, J., Zhou, Q., Qiang, Y., Kang, B., Wu, X., and Zheng, B. (2020, January 4–8). FDDWNET: A Lightweight Convolutional Neural Network for Real-Time Semantic Segmentation. Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing, Barcelona, Spain.10.1109/ICASSP40776.2020.9053838 ↗
36. Shi, M., Shen, J., Yi, Q., Weng, J., Huang, Z., Luo, A., and Zhou, Y. (2022). LMFFNet: A Well-Balanced Lightweight Network for Fast and Accurate Semantic Segmentation. IEEE Trans. Neural Netw. Learn. Syst.10.1109/TNNLS.2022.3176493 ↗
37. Orsic, M., Kreso, I., Bevandic, P., Segvic, S., and Soc, I.C. (2019, January 16–20). In Defense of Pre-trained ImageNet Architectures for Real-time Semantic Segmentation of Road-driving Images. Proceedings of the 32nd IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA.10.1109/CVPR.2019.01289 ↗
38. Chen, Z., Duan, Y., Wang, W., He, J., Lu, T., Dai, J., and Qiao, Y.J. (2022). Vision Transformer Adapter for Dense Predictions. arXiv.
39. Yan, H., Zhang, C., and Wu, M.J. (2022). Lawin Transformer: Improving Semantic Segmentation Transformer with Multi-Scale Representations via Large Window Attention. arXiv.
40. Li, P., Dong, X., Yu, X., and Yang, Y. (2020, January 7–10). When Humans Meet Machines: Towards Efficient Segmentation Networks. Proceedings of the 31st British Machine Vision Virtual Conference BMVC, Cardiff, UK.
41. Zhang, Y., Qiu, Z., Liu, J., Yao, T., Liu, D., Mei, T., and Soc, I.C. (2019, January 16–20). Customizable Architecture Search for Semantic Segmentation. Proceedings of the 32nd IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA.10.1109/CVPR.2019.01191 ↗