JAUNet: A U-Shape Network with Jump Attention for Semantic Segmentation of Road Scenes
Source abstract
Abstract
The task of complex scene semantic segmentation is to classify and label the scene image pixel by pixel. For the complex image information in autonomous driving scenes, its characteristics such as many kinds of targets and various scene changes make the segmentation task more difficult, making various kinds of FCN-based networks unable to restore the image information well. In contrast, the encoder–decoder network structure represented by SegNet and UNet uses jump connections and other methods to restore image information. Still, its extraction of shallow details is simple and unfocused. In this paper, we propose a U-shaped convolutional neural network with a jump attention mechanism, which is an improved encoder plus decoder structure to achieve semantic segmentation by four times of convolutional downsampling and four transposed convolutional upsamplings while adding a jump attention module in the upsampling process to realize selective extraction of contextual information from high-dimensional features to guide low-dimensional features, improve the fusion of deep and shallow features, and ensure the consistency of the same type of pixel prediction. The CamVid and Cityscapes datasets are sampled for the experiments, and the model ground mIoU evaluation metrics can reach 66.3% and 69.1%. Compared with other mainstream semantic segmentation algorithms, this method is competitive in terms of segmentation performance and model size.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Advanced Neural Network Applications · Advanced Image and Video Retrieval Techniques · Multimodal Machine Learning Applications
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Collaborative Innovation Center on Atmospheric Environment and Equipment Technology, Nanjing University of Information Science and Technology, Nanjing 210044, China
Wuhan Research Institute of Posts and Telecommunications, Wuhan 430074, China
References (39)
Show source-provided references
1. Gabor filter-based edge detection10.1016/0031-3203(92)90121-X ↗
2. Imagenet classification with deep convolutional neural networks
3. He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27–30). Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.10.1109/CVPR.2016.90 ↗
4. Chen, L.-C., Papandreou, G., Kokkinos, I., Murphy, K., and Yuille, A.L. (2014). Semantic image segmentation with deep convolutional nets and fully connected crfs. arXiv.
5. Noh, H., Hong, S., and Han, B. (2015, January 7–13). Learning deconvolution network for semantic segmentation. Proceedings of the IEEE International Conference on Computer Vision, Santiago, Chile.10.1109/ICCV.2015.178 ↗
6. Zeiler, M.D., Krishnan, D., Taylor, G.W., and Fergus, R. (2010, January 13–18). Deconvolutional networks. Proceedings of the 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, San Francisco, CA, USA.10.1109/CVPR.2010.5539957 ↗
7. Mikolov, T., Kombrink, S., Burget, L., Černocký, J., and Khudanpur, S. (2011, January 22–27). Extensions of recurrent neural network language model. Proceedings of the 2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Prague, Czech Republic.10.1109/ICASSP.2011.5947611 ↗
8. Parameter Identification in Power Transmission Systems Based on Graph Convolution Network10.1109/TPWRD.2021.3124528 ↗
9. Chen, L.-C., Papandreou, G., Schroff, F., and Adam, H. (2017). Rethinking atrous convolution for semantic image segmentation. arXiv.
10. Zhao, H., Shi, J., Qi, X., Wang, X., and Jia, J. (2017, January 21–26). Pyramid scene parsing network. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.10.1109/CVPR.2017.660 ↗
11. MLNet: Multichannel feature fusion lozenge network for land segmentation10.1117/1.JRS.16.016513 ↗
12. MANet: A multi-level aggregation network for semantic segmentation of high-resolution remote sensing images10.1080/01431161.2022.2073795 ↗
13. Axial Cross Attention Meets CNN: Bi-Branch Fusion Network for Change Detection10.1109/JSTARS.2022.3224081 ↗
14. Ronneberger, O., Fischer, P., and Brox, T. (2015, January 5–9). U-net: Convolutional networks for biomedical image segmentation. Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention, Munich, Germany.10.1007/978-3-319-24574-4_28 ↗
15. Segnet: A deep convolutional encoder-decoder architecture for image segmentation10.1109/TPAMI.2016.2644615 ↗
16. Peng, C., Zhang, X., Yu, G., Luo, G., and Sun, J. (2017, January 21–26). Large kernel matters—Improve semantic segmentation by global convolutional network. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.10.1109/CVPR.2017.189 ↗
17. Bilinski, P., and Prisacariu, V. (2018, January 18–23). Dense decoder shortcut connections for single-pass semantic segmentation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00690 ↗
18. Hu, J., Shen, L., and Sun, G. (2018, January 18–23). Squeeze-and-excitation networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00745 ↗
19. Woo, S., Park, J., Lee, J.-Y., and Kweon, I.S. (2018, January 8–14). Cbam: Convolutional block attention module. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.10.1007/978-3-030-01234-2_1 ↗
20. Li, H., Xiong, P., An, J., and Wang, L. (2018). Pyramid attention network for semantic segmentation. arXiv.
21. Shi, X., Chen, Z., Wang, H., Yeung, D.-Y., Wong, W.-K., and Woo, W.-C. (2015, January 7–12). Convolutional LSTM network: A machine learning approach for precipitation nowcasting. Proceedings of the Twenty-ninth Conference on Neural Information Processing Systems, Montréal, QC, Canada.
22. Simonyan, K., and Zisserman, A. (2014). Very deep convolutional networks for large-scale image recognition. arXiv.
23. Liu, W., Rabinovich, A., and Berg, A.C. (2015). Parsenet: Looking wider to see better. arXiv.
24. Hong, S., Noh, H., and Han, B. (2015, January 7–12). Decoupled deep neural network for semi-supervised semantic segmentation. Proceedings of the Twenty-ninth Conference on Neural Information Processing Systems, Montréal, QC, Canada.
25. Multi-scale strip pooling feature aggregation network for cloud and cloud shadow segmentation10.1007/s00521-021-06802-0 ↗
26. Cloud/shadow segmentation based on multi-level feature enhanced network for remote sensing imagery10.1080/01431161.2021.2014077 ↗
27. Strip pooling channel spatial attention network for the segmentation of cloud and cloud shadow10.1016/j.cageo.2021.104940 ↗
28. Semantic object classes in video: A high-definition ground truth database10.1016/j.patrec.2008.04.005 ↗
29. Cordts, M., Omran, M., Ramos, S., Rehfeld, T., Enzweiler, M., Benenson, R., Franke, U., Roth, S., and Schiele, B. (2016, January 27–30). The cityscapes dataset for semantic urban scene understanding. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.10.1109/CVPR.2016.350 ↗
30. Keskar, N.S., and Socher, R. (2017). Improving generalization performance by switching from adam to sgd. arXiv.
31. Sturgess, P., Alahari, K., Ladicky, L., and Torr, P.H. (2009, January 7–10). Combining appearance and structure from motion features for road scene understanding. Proceedings of the BMVC-British Machine Vision Conference, London, UK.10.5244/C.23.62 ↗
32. Tighe, J., and Lazebnik, S. (2010, January 5–11). Superparsing: Scalable nonparametric image parsing with superpixels. Proceedings of the European Conference on Computer Vision, Heraklion, Greece.10.1007/978-3-642-15555-0_26 ↗
33. Paszke, A., Chaurasia, A., Kim, S., and Culurciello, E. (2016). Enet: A deep neural network architecture for real-time semantic segmentation. arXiv.
34. Watanabe, S., Hori, T., Karita, S., Hayashi, T., Nishitoba, J., Unno, Y., Soplin, N.E.Y., Heymann, J., Wiesner, M., and Chen, N. (2018). Espnet: End-to-end speech processing toolkit. arXiv.10.21437/Interspeech.2018-1456 ↗
35. Zhao, H., Qi, X., Shen, X., Shi, J., and Jia, J. (2018, January 8–14). Icnet for real-time semantic segmentation on high-resolution images. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.10.1007/978-3-030-01219-9_25 ↗
36. Li, H., Xiong, P., Fan, H., and Sun, J. (2019, January 15–20). Dfanet: Deep feature aggregation for real-time semantic segmentation. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.10.1109/CVPR.2019.00975 ↗
37. Yu, C., Wang, J., Peng, C., Gao, C., Yu, G., and Sang, N. (2018, January 8–14). Bisenet: Bilateral segmentation network for real-time semantic segmentation. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.10.1007/978-3-030-01261-8_20 ↗
38. Han, W., Zhang, Z., Zhang, Y., Yu, J., Chiu, C.-C., Qin, J., Gulati, A., Pang, R., and Wu, Y. (2020). Contextnet: Improving convolutional neural networks for automatic speech recognition with global context. arXiv.10.21437/Interspeech.2020-2059 ↗
39. Cgnet: A light-weight context guided network for semantic segmentation10.1109/TIP.2020.3042065 ↗