Lightweight Convolutional Neural Networks with Model-Switching Architecture for Multi-Scenario Road Semantic Segmentation
Source abstract
Abstract
A convolutional neural network (CNN) that was trained using datasets for multiple scenarios was proposed to facilitate real-time road semantic segmentation for various scenarios encountered in autonomous driving. However, the CNN inhibited the mutual suppression effect between weights; thus, it did not perform as well as a network that was trained using a single scenario. To address this limitation, we used a model-switching architecture in the network and maintained the optimal weights of each individual model which required considerable space and computation. We, subsequently, incorporated a lightweight process into the model to reduce the model size and computational load. The experimental results indicated that the proposed lightweight CNN with a model-switching architecture outperformed and was faster than the conventional methods across multiple scenarios in road semantic segmentation.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Advanced Neural Network Applications · Infrastructure Maintenance and Monitoring · Adversarial Robustness in Machine Learning
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
College of Mechanical and Electrical Engineering, National Taipei University of Technology, Taipei 10608, Taiwan
Department of Mechanical Engineering, National Taipei University of Technology, Taipei 10608, Taiwan
References (53)
Show source-provided references
1. ImageNet Classification with Deep Convolutional Neural Networks
2. He, K., Zhang, X., Ren, S., and Sun, J. (2015). Deep Residual Learning for Image Recognition. arXiv.10.1109/CVPR.2016.90 ↗
3. Xie, S., Girshick, R., Dollár, P., Tu, Z., and He, K. (2017). Aggregated Residual Transformations for Deep Neural Networks. arXiv.10.1109/CVPR.2017.634 ↗
4. Gao, S., Cheng, M., Zhao, K., Zhang, X., Yang, M., and Philip, T. (2019). Res2Net: A New Multi-scale Backbone Architecture. arXiv.
5. Zoph, B., and Le, Q.V. (2016). Neural Architecture Search with Reinforcement Learning. arXiv.
6. Zoph, B., Vasudevan, V., Shlens, J., and Le, Q.V. (2018). Learning Transferable Architectures for Scalable Image Recognition. arXiv.10.1109/CVPR.2018.00907 ↗
7. Liu, C., Zoph, B., Neumann, M., Shlens, J., Hua, W., Li, L.J., Li, F.F., Yuille, A., Huang, J., and Murphy, K. (2018). Progressive Neural Architecture Search. arXiv.10.1007/978-3-030-01246-5_2 ↗
8. Tan, M., Chen, B., Pang, R., Vasudevan, V., Sandler, M., Howard, A., and Le, Q.V. (2019). MnasNet: Platform-Aware Neural Architecture Search for Mobile. arXiv.10.1109/CVPR.2019.00293 ↗
9. Howard, A.G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., Andreetto, M., and Adam, H. (2017). MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications. arXiv.
10. Mark, S., Andrew, H., Zhu, M., Andrey, Z., and Chen, L.C. (2018). MobileNetV2: Inverted Residuals and Linear Bottlenecks. arXiv.
11. Howard, A., Sandler, M., Chu, G., Chen, L.C., Chen, B., Tan, M., Wang, W., Zhu, Y., Pang, R., and Vasudevan, V. (2019). Searching for MobileNetV3. arXiv.10.1109/ICCV.2019.00140 ↗
12. Zhang, X., Zhou, X., Lin, M., and Sun, J. (2017). ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices. arXiv.10.1109/CVPR.2018.00716 ↗
13. Ma, N., Zhang, X., Zheng, H.T., and Sun, J. (2018). ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Design. arXiv.10.1007/978-3-030-01264-9_8 ↗
14. Tan, M., and Le, Q.V. (2019). EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks. arXiv.
15. Tan, M., and Le, Q.V. (2021). EfficientNetV2: Smaller Models and Faster Training. arXiv.
16. Long, J., Shelhamer, E., and Darrell, T. (2015). Fully Convolutional Networks for Semantic Segmentation, Computing. arXiv.10.1109/CVPR.2015.7298965 ↗
17. Vijay, B., Alex, K., and Roberto, C. (2015). SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation. arXiv.
18. Ronneberger, O., Fischer, P., and Brox, T. (2015). U-Net: Convolutional Networks for Biomedical Image Segmentation. arXiv.10.1007/978-3-319-24574-4_28 ↗
19. Fausto, M., Nassir, N., and Ahmadi, S. (2016). V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation. arXiv.
20. UNet++: Redesigning Skip Connections to Exploit Multiscale Features in Image Segmentation10.1109/TMI.2019.2959609 ↗
21. Zhao, H., Jiang, L., Jia, J., Philip, T., and Vladlen, K. (2020). Point Transformer. arXiv.10.1109/ICCV48922.2021.01595 ↗
22. Guo, M., Cai, J., Liu, Z., Mu, T., Ralph, R.M., and Hu, S. (2021). PCT: Point cloud transformer. arXiv.10.1007/s41095-021-0229-5 ↗
23. Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., and Guo, B. (2021). Swin Transformer: Hierarchical Vision Transformer using Shifted Windows. arXiv.10.1109/ICCV48922.2021.00986 ↗
24. Zhao, H., Shi, J., Qi, X., Wang, X., and Jia, J. (2017). Pyramid Scene Parsing Network. arXiv.10.1109/CVPR.2017.660 ↗
25. Yang, M., Yu, K., Zhang, C., Li, Z., and Yang, K. (2018, January 18–23). DenseASPP for Semantic Segmentation in Street Scenes. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00388 ↗
26. Chen, L.C., Papandreou, G., Kokkinos, I., Murphy, K., and Yuille, A.L. (2016). Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFS. arXiv.
27. Chen, L.C., Papandreou, G., Kokkinos, I., Murphy, K., and Yuille, A.L. (2017). DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs. arXiv.10.1109/TPAMI.2017.2699184 ↗
28. Chen, L.C., Papandreou, G., Schroff, F., and Adam, H. (2017). Rethinking Atrous Convolution for Semantic Image Segmentation. arXiv.
29. Chen, L.C., Zhu, Y., Papandreou, G., Schroff, F., and Adam, H. (2018). Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation. arXiv.10.1007/978-3-030-01234-2_49 ↗
30. Caio, C., Vincent, F., and Denis, F. (2015). Vision-Based Road Detection using Contextual Blocks. arXiv.
31. Rahul, M. (2014). Deep Deconvolutional Networks for Scene Parsing. arXiv.
32. Chen, Z., and Chen, Z. (2017, January 14–18). RBNet: A Deep Neural Network for Unified Road and Road Boundary Detection. Proceedings of the 24th International Conference on Neural Information Processing, Guangzhou, China.10.1007/978-3-319-70087-8_70 ↗
33. Caio, C., Vincent, F., and Denis, F. (2016, January 16–21). Exploiting Fully Convolutional Neural Networks for Fast Road Detection. Proceedings of the 2016 IEEE International Conference on Robotics and Automation, Stockholm, Sweden.
34. Marvin, T., Michael, W., Marius, Z., Roberto, C., and Raquel, U. (2018, January 26–30). MultiNet: Real-time Joint Semantic Reasoning for Autonomous Driving. Proceedings of the 2018 IEEE Intelligent Vehicles Symposium, Changshu, China.
35. Ha, Q., Watanabe, K., Karasawa, T., Ushiku, Y., and Harada, T. (2017, January 24–28). MFNet: Towards Real-Time Semantic Segmentation for Autonomous Vehicles with Multi-Spectral Scenes. Proceedings of the 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems, Vancouver, BC, Canada.10.1109/IROS.2017.8206396 ↗
36. Sun, J.Y., Kim, S.W., Lee, S.W., Kim, Y.W., and Ko, S.J. (2019, January 27–28). Reverse and Boundary Attention Network for Road Segmentation. Proceedings of the IEEE/CVF International Conference on Computer Vision Workshops, Seoul, Korea.10.1109/ICCVW.2019.00116 ↗
37. Shinzato, P.Y., Denis, F.W., and Christoph, S. (2014, January 8–11). Road Terrain Detection: Avoiding Common Obstacle Detection Assumptions Using Sensor Fusion. Proceedings of the IEEE Intelligent Vehicles Symposium, Dearborn, MI, USA.10.1109/IVS.2014.6856454 ↗
38. Xiao, L., Dai, B., Liu, D., Hu, T., and Wu, T. (July, January 28). CRF based Road Detection with Multi-Sensor Fusion. Proceedings of the 2015 IEEE Intelligent Vehicles Symposium, Seoul, Korea.10.1109/IVS.2015.7225685 ↗
39. Gu, S., Zhang, Y., Tang, J., Yang, J., and Kong, H. (2019, January 20–24). Road Detection through CRF based LiDAR-Camera Fusion. Proceedings of the International Conference on Robotics and Automation, Montreal, QC, Canada.10.1109/ICRA.2019.8793585 ↗
40. Luca, C., Mauro, B., Lennart, S., and Mattias, W. (2018). LIDAR-Camera Fusion for Road Detection Using Fully Convolutional Neural Networks. arXiv.
41. Gu, S., Zhang, Y., Yang, J., Jose, M.A., and Kong, H. (2019, January 4–8). Two-View Fusion based Convolutional Neural Network for Urban Road Detection. Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems, The Venetian Macao, Macau, China.10.1109/IROS40897.2019.8968054 ↗
42. RTFNet: RGB-Thermal Fusion Network for Semantic Segmentation of Urban Scenes10.1109/LRA.2019.2904733 ↗
43. Fan, R., Wang, H., Cai, P., and Liu, M. (2020). SNE-RoadSeg: Incorporating Surface Normal Information into Semantic Segmentation for Accurate Freespace Detection. European Conference on Computer Vision, Springer.10.36227/techrxiv.12864287.v1 ↗
44. Progressive LiDAR Adaptation for Road Detection
45. Simonyan, K., and Zisserman, A. (2015). Very Deep Convolutional Networks for Large-Scale Image Recognition. arXiv.
46. Kowsari, K., Heidarysafa, M., Brown, D.E., Meimandi, K.J., and Barnes, L.E. (2018). RMDL: Random Multimodel Deep Learning for Clas-sification. arXiv.10.1109/ICMLA.2017.0-134 ↗
47. Learning Categories From Few Examples With Multi Model Knowledge Transfer10.1109/TPAMI.2013.197 ↗
48. CNN and RNN mixed model for image classification
49. Robust Face Recognition via Multimodal Deep Face Representation10.1109/TMM.2015.2477042 ↗
50. More Diverse Means Better: Multimodal Deep Learning Meets Remote-Sensing Imagery Classi-fication10.1109/TGRS.2020.3016820 ↗
51. Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., and Wojna, Z. (2015). Rethinking the Inception Architecture for Computer Vision. arXiv.10.1109/CVPR.2016.308 ↗
52. Yu, C., Wang, J., Peng, C., Gao, C., Yu, G., and Sang, N. (2018, January 8–14). BiSeNet: Bilateral Segmentation Network for Real-time Semantic Seg-mentation. Proceedings of the European Conference on Computer Vision, Munich, Germany.10.1007/978-3-030-01261-8_20 ↗
53. Yu, C., Gao, C., Wang, J., Yu, G., and Shen, C. (2020). Sang, N. BiSeNet V2: Bilateral Network with Guided Aggregation for Real-time Semantic Segmentation. arXiv.10.1007/s11263-021-01515-2 ↗