Crack Segmentation Using U-Net and Transformer Combined Model
Source abstract
Abstract
Crack detection and analysis are essential for maintaining the stability and longevity of infrastructure; however, traditional manual inspections or simple image processing techniques are inefficient. To address this, automated crack segmentation using deep learning is being actively researched. This study proposes a hybrid model combining U-Net and a Vision Transformer to enhance the accuracy of crack segmentation. The proposed model is based on U-Net’s encoder–decoder architecture and integrates a Convolutional Neural Network (CNN), which is strong in local feature extraction, with a Vision Transformer, which excels at capturing global features and long-range dependencies, to effectively learn complex crack patterns. Experimental results on the CrackSeg9k dataset show that the proposed model achieves a mean Intersection over Union (mIoU) of 0.7184, demonstrating superior segmentation performance compared to other models like the conventional U-Net and Attention U-Net. This indicates that the proposed hybrid approach successfully leverages both local and global features, proving its effectiveness in segmenting complex and irregular crack patterns.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Infrastructure Maintenance and Monitoring · Industrial Vision Systems and Defect Detection · Vehicle License Plate Recognition
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Department of Computer Engineering, Chosun University, Gwangju 61452, Republic of Korea
Funding
MSIT (Ministry of Science and ICT), Korea
References (35)
Show source-provided references
1. Imagenet classification with deep convolutional neural networks
2. Simonyan, K., and Zisserman, A. (2014). Very deep convolutional networks for large-scale image recognition. arXiv.
3. Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., and Rabinovich, A. (2015, January 7–12). Going deeper with convolutions. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA.10.1109/CVPR.2015.7298594 ↗
4. He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27–30). Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.10.1109/CVPR.2016.90 ↗
5. Girshick, R., Donahue, J., Darrell, T., and Malik, J. (2014, January 23–28). Rich feature hierarchies for accurate object detection and semantic segmentation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Columbus, OH, USA.10.1109/CVPR.2014.81 ↗
6. Girshick, R. (2015). Fast R-CNN. arXiv.10.1109/ICCV.2015.169 ↗
7. Faster R-CNN: Towards real-time object detection with region proposal networks10.1109/TPAMI.2016.2577031 ↗
8. Redmon, J., Divvala, S., Girshick, R., and Farhadi, A. (2016, January 27–30). You only look once: Unified, real-time object detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.10.1109/CVPR.2016.91 ↗
9. SSD: Single shot multibox detector
10. Long, J., Shelhamer, E., and Darrell, T. (2015, January 7–12). Fully convolutional networks for semantic segmentation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA.10.1109/CVPR.2015.7298965 ↗
11. U-Net: Convolutional networks for biomedical image segmentation
12. Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs10.1109/TPAMI.2017.2699184 ↗
13. Zhao, H., Shi, J., Qi, X., Wang, X., and Jia, J. (2017, January 21–26). Pyramid scene parsing network. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.10.1109/CVPR.2017.660 ↗
14. He, K., Gkioxari, G., Dollár, P., and Girshick, R. (2017, January 22–29). Mask R-CNN. Proceedings of the IEEE International Conference on Computer Vision (ICCV), Venice, Italy.10.1109/ICCV.2017.322 ↗
15. Attention is all you need
16. Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., and Gelly, S. (2020). An image is worth 16×16 words: Transformers for image recognition at scale. arXiv.
17. Alfarrarjeh, A., Shahabi, C., Luo, Y., and Liu, Y. (2018, January 10–13). A deep learning approach for road damage detection from smartphone images. Proceedings of the 2018 IEEE International Conference on Big Data (Big Data), Seattle, WA, USA.10.1109/BigData.2018.8621899 ↗
18. Deep learning-based crack damage detection using convolutional neural networks10.1111/mice.12263 ↗
19. A real-time bridge crack detection method based on an improved inception-resnet-v2 structure10.1109/ACCESS.2021.3093210 ↗
20. Impact of fully connected layers on performance of convolutional neural networks for image classification10.1016/j.neucom.2019.10.008 ↗
21. DeepCrack: A deep hierarchical feature learning architecture for crack segmentation10.1016/j.neucom.2019.01.036 ↗
22. Feature pyramid and hierarchical boosting network for pavement crack detection10.1109/TITS.2019.2910595 ↗
23. Pixel-level cracking detection on 3D asphalt pavement images through deep-learning-based CrackNet-V10.1109/TITS.2019.2891167 ↗
24. Zheng, S., Lu, J., Zhao, H., Zhu, X., Luo, Z., Wang, Y., Fu, Y., Feng, J., Xiang, T., and Torr, P.H.S. (2021, January 19–25). Rethinking semantic segmentation from a sequence-to-sequence perspective with transformers. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Nashville, TN, USA.10.1109/CVPR46437.2021.00681 ↗
25. Zhang, P., Wang, D., Lu, H., Wang, H., and Ruan, X. (2017, January 22–29). Amulet: Aggregating multi-level convolutional features for salient object detection. Proceedings of the IEEE International Conference on Computer Vision (ICCV), Venice, Italy.10.1109/ICCV.2017.31 ↗
26. Yu, F., and Koltun, V. (2015). Multi-scale context aggregation by dilated convolutions. arXiv.
27. Hassani, A., and Shi, H. (2022). Dilated neighborhood attention transformer. arXiv.10.1109/CVPR52729.2023.00599 ↗
28. Hassani, A., Navon, A., Wolf, L., and Shi, H. (2023, January 18–24). Neighborhood attention transformer. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Vancouver, BC, Canada.10.1109/CVPR52729.2023.00599 ↗
29. Kulkarni, S., Hall, D., Stein, S., and Körner, M. (2022). CrackSeg9k: A collection and benchmark for crack segmentation datasets and frameworks. Computer Vision–ECCV 2022 Workshops, Proceedings of the European Conference on Computer Vision (ECCV), Tel Aviv, Israel, 23–27 October 2022, Springer.10.1007/978-3-031-25082-8_12 ↗
30. Nguyen, N.T.H., Nguyen, D.Q., Tran, N.T., Nguyen, H.T., Nguyen, B.N., and Le, V.H. (2018, January 6–7). Pavement crack detection using convolutional neural network. Proceedings of the 9th International Symposium on Information and Communication Technology (SoICT), Danang, Vietnam.10.1145/3287921.3287949 ↗
31. Di Benedetto, A., Fiani, M., and Gujski, L.M. (2023). U-Net-based CNN architecture for road crack segmentation. Infrastructures, 8.10.3390/infrastructures8050090 ↗
32. Peng, J., Gao, M., Yang, G., Xu, Y., Liu, Y., and Zhou, K. (2022). PP-LiteSeg: A superior real-time semantic segmentation model. arXiv.
33. Yao, Z., Xie, H., Chen, Y., Yang, Y., Yang, W., and Fan, H. (2024). CrackNex: A few-shot low-light crack segmentation model based on Retinex theory for UAV inspections. arXiv.10.1109/ICRA57147.2024.10611660 ↗
34. Fan, M., Zhang, R., Wang, J., Tang, M., and Zhang, Y. (2021, January 19–25). Rethinking BiSeNet for real-time semantic segmentation. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Nashville, TN, USA.10.1109/CVPR46437.2021.00959 ↗
35. Chen, L.C. (2017). Rethinking atrous convolution for semantic image segmentation. arXiv.10.1007/978-3-030-01234-2_49 ↗