A GAN-Based Framework with Dynamic Adaptive Attention for Multi-Class Image Segmentation in Autonomous Driving
Source abstract
Abstract
Image segmentation is a foundation for autonomous driving frameworks that empower vehicles to explore and navigate their surrounding environment. It gives a fundamental setting to the dynamic cycles by dividing the image into significant parts like streets, vehicles, walkers, and traffic signs. Precise segmentation ensures safe navigation and the avoidance of collisions, while following the rules of traffic is very critical for seamless operation in self-driving cars. The most recent deep learning-based image segmentation models have demonstrated impressive performance in structured environments, yet they often fall short when applied to the complex and unpredictable conditions encountered in autonomous driving. This study proposes an Adaptive Ensemble Attention (AEA) mechanism within a Generative Adversarial Network architecture to deal with dynamic and complex driving conditions. The AEA integrates the features of self, spatial, and channel attention adaptively and powerfully changes the amount of each contribution as per input and context-oriented relevance. It does this by allowing the discriminator network in GAN to evaluate the segmentation mask created by the generator. This explains the difference between real and fake masks by considering a concatenated pair of an original image and its mask. The adversarial training will prompt the generator, via the discriminator, to mask out the image in such a way that the output aligns with the expected ground truth and is also very realistic. The exchange of information between the generator and discriminator improves the quality of the segmentation. In order to check the accuracy of the proposed method, the three widely used datasets BDD100K, Cityscapes, and KITTI were selected to calculate average IoU, where the value obtained was 89.46%, 89.02%, and 88.13% respectively. These outcomes emphasize the model’s effectiveness and consistency. Overall, it achieved a remarkable accuracy of 98.94% and AUC of 98.4%, indicating strong enhancements compared to the State-of-the-art (SOTA) models.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Advanced Neural Network Applications · Medical Image Segmentation Techniques · Advanced Image and Video Retrieval Techniques
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Department of Computer Engineering, Necmettin Erbakan University, Konya 42090, Türkiye
References (44)
Show source-provided references
1. Yu, H., Huo, S., Zhu, M., Gong, Y., and Xiang, Y. (2024). Machine learning-based vehicle intention trajectory recognition and prediction for autonomous driving. arXiv.10.1109/ICAACE61206.2024.10548252 ↗
2. Dynamic weighting and boundary-aware active domain adaptation for semantic segmentation in autonomous driving environment10.1109/TITS.2024.3431537 ↗
3. Image Semantic Segmentation for Autonomous Driving Based on Improved U-Net
4. Enhancing semantic segmentation for autonomous vehicle scene understanding in indian context using modified CANet model10.1016/j.mex.2024.103131 ↗
5. Lane detection based on real-time semantic segmentation for end-to-end autonomous driving under low-light conditions10.1016/j.dsp.2024.104752 ↗
6. A Real-Time Semantic Segmentation Method Based on Transformer for Autonomous Driving
7. Towards applying image retrieval approach for finding semantic locations in autonomous vehicles10.1007/s11042-023-16387-6 ↗
8. SegNet network architecture for deep learning image segmentation and its integrated applications and prospects10.54097/rfa5x119 ↗
9. A survey on deep learning for polyp segmentation: Techniques, challenges and future trends10.1007/s44267-024-00071-w ↗
10. Efficient Road Segmentation Techniques with Attention-Enhanced Conditional GANs10.1007/s42979-023-02535-0 ↗
11. Generative adversarial network for semi-supervised image captioning10.1016/j.cviu.2024.104199 ↗
12. Adaptive Attention Module for Image Recognition Systems in Autonomous Driving10.1155/2024/3934270 ↗
13. SegNet4D: Efficient Instance-Aware 4D Semantic Segmentation for LiDAR Point Cloud10.1109/TASE.2025.3568001 ↗
14. Hofmarcher, M., Unterthiner, T., Arjona-Medina, J., Klambauer, G., Hochreiter, S., and Nessler, B. (2019). Visual scene understanding for autonomous driving using semantic segmentation. Explainable AI: Interpreting, Explaining and Visualizing Deep Learning, Springer.10.1007/978-3-030-28954-6_15 ↗
15. Chang, C.C., Lin, W.C., Wang, P.S., Yu, S.F., Lu, Y.C., Lin, K.C., and Wu, K.C. (2023, January 10–14). Q-YOLOP: Quantization-aware you only look once for panoptic driving perception. Proceedings of the 2023 IEEE International Conference on Multimedia and Expo Workshops (ICMEW), Brisbane, Australia.10.1109/ICMEW59549.2023.00015 ↗
16. Ids-model: An efficient multitask model of road scene instance and drivable area segmentation for autonomous driving10.1109/TTE.2023.3293495 ↗
17. Siam, M., Gamal, M., Abdel-Razek, M., Yogamani, S., Jagersand, M., and Zhang, H. (2018, January 18–22). A comparative study of real-time semantic segmentation for autonomous driving. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, Salt Lake City, UT, USA.10.1109/CVPRW.2018.00101 ↗
18. Deep Multi-Modal Object Detection and Semantic Segmentation for Autonomous Driving: Datasets, Methods, and Challenges10.1109/TITS.2020.2972974 ↗
19. HRDLNet: A semantic segmentation network with high resolution representation for urban street view images10.1007/s40747-024-01582-1 ↗
20. Automated Evaluation of Semantic Segmentation Robustness for Autonomous Driving10.1109/TITS.2019.2909066 ↗
21. DLT-Net: Joint detection of drivable areas, lane lines, and traffic objects10.1109/TITS.2019.2943777 ↗
22. Aksoy, E.E., Baci, S., and Cavdar, S. (November, January 19). Salsanet: Fast road and vehicle segmentation in lidar point clouds for autonomous driving. Proceedings of the 2020 IEEE Intelligent Vehicles Symposium (IV), Las Vegas, NV, USA.10.1109/IV47402.2020.9304694 ↗
23. Yolop: You only look once for panoptic driving perception10.1007/s11633-022-1339-y ↗
24. Deep learning-based image recognition for autonomous driving10.1016/j.iatssr.2019.11.008 ↗
25. SegFormer: Simple and efficient design for semantic segmentation with transformers
26. Yang, J., Guo, S., Bocus, M.J., Chen, Q., and Fan, R. (2023). Semantic segmentation for autonomous driving. Autonomous Driving Perception: Fundamentals and Applications, Springer.10.1007/978-981-99-4287-9_4 ↗
27. Scaling up multi-domain semantic segmentation with sentence embeddings10.1007/s11263-024-02060-4 ↗
28. Zhu, Y., Sapra, K., Reda, F.A., Shih, K.J., Newsam, S., Tao, A., and Catanzaro, B. (2019, January 15–20). Improving semantic segmentation via video propagation and label relaxation. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.10.1109/CVPR.2019.00906 ↗
29. A Review of Generative Adversarial Networks for Intrusion Detection Systems: Advances, Challenges, and Future Directions
30. Celik, F., Celik, K., and Celik, A. (2024). Enhancing brain tumor classification through ensemble attention mechanism. Sci. Rep., 14.10.1038/s41598-024-73803-z ↗
31. Semantic segmentation with attention mechanism for remote sensing images
32. Yu, F., Chen, H., Wang, X., Xian, W., Chen, Y., Liu, F., Madhavan, V., and Darrell, T. (2020, January 13–19). Bdd100k: A diverse driving dataset for heterogeneous multitask learning. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.10.1109/CVPR42600.2020.00271 ↗
33. Cordts, M., Omran, M., Ramos, S., Rehfeld, T., Enzweiler, M., Benenson, R., Franke, U., Roth, S., and Schiele, B. (2016, January 27–30). The cityscapes dataset for semantic urban scene understanding. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.10.1109/CVPR.2016.350 ↗
34. Multi-label classifier performance evaluation with confusion matrix
35. Ronneberger, O., Fischer, P., and Brox, T. (2015). U-net: Convolutional networks for biomedical image segmentation. Medical Image Computing and Computer-Assisted Intervention, Springer.10.1007/978-3-319-24574-4_28 ↗
36. Cheng, B., Misra, I., Schwing, A.G., Kirillov, A., and Girdhar, R. (2022, January 18–24). Masked-attention mask transformer for universal image segmentation. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.10.1109/CVPR52688.2022.00135 ↗
37. Yuan, Y., Chen, X., and Wang, J. (2020). Object-contextual representations for semantic segmentation. European Conference on Computer Vision, Springer.10.1007/978-3-030-58539-6_11 ↗
38. Lo, S.Y., Hang, H.M., Chan, S.W., and Lin, J.J. (2019, January 15–18). Efficient dense modules of asymmetric convolution for real-time semantic segmentation. Proceedings of the 1st ACM International Conference on Multimedia in Asia, Beijing, China.10.1145/3338533.3366558 ↗
39. Fan, M., Lai, S., Huang, J., Wei, X., Chai, Z., Luo, J., and Wei, X. (2021, January 20–25). Rethinking bisenet for real-time semantic segmentation. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA.10.1109/CVPR46437.2021.00959 ↗
40. Xu, J., Xiong, Z., and Bhattacharyya, S.P. (2023, January 17–24). PIDNet: A real-time semantic segmentation network inspired by PID controllers. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, BC, Canada.10.1109/CVPR52729.2023.01871 ↗
41. Hong, Y., Pan, H., Sun, W., and Jia, Y. (2021). Deep dual-resolution networks for real-time and accurate semantic segmentation of road scenes. arXiv.
42. Yan, X., Gao, J., Zheng, C., Zheng, C., Zhang, R., Cui, S., and Li, Z. (2022). 2dpass: 2d priors assisted semantic segmentation on lidar point clouds. European Conference on Computer Vision, Springer.10.1007/978-3-031-19815-1_39 ↗
43. Lai, X., Chen, Y., Lu, F., Liu, J., and Jia, J. (2023, January 17–24). Spherical transformer for lidar-based 3d recognition. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, BC, Canada.10.1109/CVPR52729.2023.01683 ↗
44. Wiseman, Y. (2017, January 14–17). Real-time monitoring of traffic congestions. Proceedings of the 2017 IEEE International Conference on Electro Information Technology (EIT), Lincoln, NE, USA.10.1109/EIT.2017.8053413 ↗