Advanced Hybrid Transformer–CNN Vision Framework for Automated Crack Detection to Enhance Structural Condition Assessment of Civil Structures
Source abstract
Abstract
Reliable crack detection is essential for ensuring the safety, serviceability, and long-term performance of civil structures. Conventional manual inspections are labor-intensive and subjective, while existing computer vision models often exhibit reduced accuracy under variable field conditions. This study develops a computer vision-based automated crack detection framework utilizing a hybrid Transformer–CNN architecture to support infrastructure inspection and condition assessment. The proposed model leverages the global context modeling capability of Transformers and the local feature sensitivity of convolutional neural networks (CNNs) to enhance detection robustness. The optimized hybrid model achieved an Intersection over Union (IoU) of 91.8% and an accuracy of 98.7%, outperforming baseline CNN, Transformer-only, and LSTM architectures. Field validation on bridge inspection imagery demonstrated strong resilience to variations in illumination and texture. The developed approach contributes to digital inspection and intelligent lifecycle management of infrastructure assets by enabling reliable, automated, and non-intrusive structural condition evaluation under realistic field conditions.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Infrastructure Maintenance and Monitoring · Advanced Neural Network Applications · Structural Health Monitoring Techniques
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Department of Civil Engineering, Jilin Jianzhu University, Changchun 130118, China
Key Laboratory of Architectural Cold Climate Energy Management, Ministry of Education, Jilin Jianzhu University, Changchun 130118, China
Department of Civil Engineering, University of Texas at Arlington, Arlington, TX 76010, USA
Funding
The Eighth Batch of Jilin Province’s Youth Science and Technology Talent Support Program · Department of Housing and Urban Rural Development of Jilin Province, China · Science Research Project of the Education Department of Jilin Province · USDOTs
References (33)
Show source-provided references
1. Scattering matrix similarity metric optimization for improved defect characterisation based on dynamic graph attention networks10.1016/j.ndteint.2024.103220 ↗
2. Duan, R., Yan, Z., Huang, P., Li, S., and Hu, Q. (2023). A Method for Steel Crack Detection Based on Multispectral Image Fusion. SSRN, preprint.10.2139/ssrn.4556865 ↗
3. A study of the automated eddy current detection of cracks in steel plates10.1007/s10921-019-0647-9 ↗
4. Analysis of acoustic emission signal for crack detection and distance measurement on steel structure10.1007/s40857-020-00208-z ↗
5. Digital image correlation for surface deformation measurement: Historical developments, recent advances and future goals10.1088/1361-6501/aac55b ↗
6. Kim, J.J., Kim, A.-R., and Lee, S.-W. (2020). Artificial neural network-based automated crack detection and analysis for the inspection of concrete structures. Appl. Sci., 10.10.3390/app10228105 ↗
7. Zhang, Z., Pan, H., Wang, X., and Lin, Z. (2022). Deep learning empowered structural health monitoring and damage diagnostics for structures with weldment via decoding ultrasonic guided wave. Sensors, 22.10.3390/s22145390 ↗
8. Zhang, Z., Tang, F., Cao, Q., Pan, H., Wang, X., and Lin, Z. (2022). Deep learning-enriched stress level identification of pretensioned rods via guided wave approaches. Buildings, 12.10.3390/buildings12111772 ↗
9. Ali, R., Zeng, J., and Cha, Y.-J. (2020). Deep learning-based crack detection in a concrete tunnel structure using multispectral dynamic imaging. Proceedings of the Smart Structures and NDE for Industry 4.0, Smart Cities, and Energy Systems, SPIE.10.1117/12.2557900 ↗
10. Lee, Y.-K., and Jo, K.-H. (2020). Faster R-CNN with Attention Feature Map for Robust Object Detection. Proceedings of the Frontiers of Computer Vision: 26th International Workshop, IW-FCV 2020, Ibusuki, Kagoshima, Japan, 20–22 February 2020, Springer. Revised Selected Papers 26, F, 2020.
11. Short-Term Crack in Sewer Forecasting Method Based on CNN-LSTM Hybrid Neural Network Model
12. Pavement crack detection and recognition using the architecture of segNet
13. Fu, Z., Lam, W., Yu, Q., So, A.M.-C., Hu, S., Liu, Z., and Collier, N. (2023). Decoder-only or encoder-decoder? interpreting language model as a regularized encoder-decoder. arXiv.
14. CrackU-net: A novel deep convolutional neural network for pixelwise pavement crack detection10.1002/stc.2551 ↗
15. Liu, H., Miao, X., Mertz, C., Xu, C., and Kong, H. (2021, January 11–17). Crackformer: Transformer network for fine-grained crack detection. Proceedings of the IEEE/CVF International Conference on Computer Vision, Montreal, BC, Canada.10.1109/ICCV48922.2021.00376 ↗
16. Image-based concrete crack detection in tunnels using deep fully convolutional networks10.1016/j.conbuildmat.2019.117367 ↗
17. Hao, F., He, F., Liu, L., Wu, F., Tao, D., and Cheng, J. (2023, January 1–6). Class-aware patch embedding adaptation for few-shot image classification. Proceedings of the IEEE/CVF International Conference on Computer Vision, Paris, France.10.1109/ICCV51070.2023.01733 ↗
18. Narushynska, O., Teslyuk, V., Doroshenko, A., and Arzubov, M. (2024). Data sorting influence on short text manual labeling quality for hierarchical classification. Big Data Cogn. Comput., 8.10.3390/bdcc8040041 ↗
19. Multi-scale fusion and decomposition network for single image deraining10.1109/TIP.2023.3334556 ↗
20. Multi-dimension compression of feed-forward network in vision transformers10.1016/j.patrec.2023.10.014 ↗
21. DM3Loc: Multi-label mRNA subcellular localization prediction and analysis based on multi-head self-attention mechanism10.1093/nar/gkab016 ↗
22. Transformer based named entity recognition for place name extraction from unstructured text10.1080/13658816.2022.2133125 ↗
23. Vision transformer-based autonomous crack detection on asphalt and concrete surfaces10.1016/j.autcon.2022.104316 ↗
24. ReLU-type Hopfield neural network with analog hardware implementation10.1016/j.chaos.2022.113068 ↗
25. Transpose convolution based model for super-resolution image reconstruction10.1007/s10489-022-03745-4 ↗
26. A review of convolutional neural networks in computer vision10.1007/s10462-024-10721-6 ↗
27. UAV based accurate displacement monitoring through automatic filtering out its camera’s translations and rotations10.1016/j.jobe.2021.102992 ↗
28. Bai, X., Yang, M., and Ajmera, B. (2020). An advanced edge-detection method for noncontact structural displacement monitoring. Sensors, 20.10.3390/s20174941 ↗
29. Paszke, A. (2019). Pytorch: An imperative style, high-performance deep learning library. arXiv.
30. Dong, H., Du, Y., Feng, D., Hu, Q., Zhou, M., Xing, J., Zhang, L., Wang, S., and Liu, Y. (2024). CSegNet: A Crack Segmentation Network Combining CNN and Transformer. Res. Sq., preprint.10.21203/rs.3.rs-3925781/v1 ↗
31. Deep learning-based multi-functional therapeutic peptides prediction with a multi-label focal dice loss function10.1093/bioinformatics/btad334 ↗
32. Research on tunnel crack segmentation algorithm based on improved U-net network
33. Rahman, M.A., and Wang, Y. (2016). Optimizing intersection-over-union in deep neural networks for image segmentation. Proceedings of the International Symposium on Visual Computing, Springer.10.1007/978-3-319-50835-1_22 ↗