journal article·2024-02-25Open access

A Novel Hybrid Approach for Concrete Crack Segmentation Based on Deformable Oriented-YOLOv4 and Image Processing Techniques

Zengsheng He, Cheng Su, Yichuan Deng

Applied Sciences· Volume 14· Issue 5

Abstract

Regular crack inspection plays a significant role in the maintenance of concrete structures. However, most deep-learning-based methods suffer from the heavy workload of pixel-level labeling and the poor performance of crack segmentation with the presence of background interferences. To address these problems, the Deformable Oriented YOLOv4 (DO-YOLOv4) is first developed for crack detection based on the traditional YOLOv4, in which crack features can be effectively extracted by deformable convolutional layers, and the crack regions can be tightly enclosed by a series of oriented bounding boxes. Then, the proposed DO-YOLOv4 is further utilized in combination with the image processing techniques (IPTs), leading to a novel hybrid approach, termed DO-YOLOv4-IPTs, for crack segmentation. The experimental results show that, owing to the high precision of DO-YOLOv4 for crack detection under background noise, the present hybrid approach DO-YOLOv4-IPTs outperforms the widely used Convolutional Neural Network (CNN)-based crack segmentation methods with less labeling work and superior segmentation accuracy.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: Infrastructure Maintenance and Monitoring · Geophysical Methods and Applications · Asphalt Pavement Performance Evaluation

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

School of Civil Engineering and Transportation, South China University of Technology, Guangzhou 510641, China

State Key Laboratory of Subtropical Building Science, South China University of Technology, Guangzhou 510641, China

Guangdong Artificial Intelligence and Digital Economy Laboratory, Guangzhou 510641, China

Funding

National Natural Science Foundation of China · Guangdong Provincial Key Laboratory of Modern Civil Engineering Technology · Guangdong Basic and Applied Basic Research Foundation · Guangzhou Science and Technology Program

References (65)

Show source-provided references
1. Machine vision-based surface crack analysis for transportation infrastructure10.1016/j.autcon.2021.103973 ↗
2. Structural crack detection using deep convolutional neural networks10.1016/j.autcon.2021.103989 ↗
3. Zhang, J.M., Lu, C.Q., Wang, J., Wang, L., and Yue, X.G. (2019). Concrete Cracks Detection Based on FCN with Dilated Convolution. Appl. Sci., 9.10.3390/app9132686 ↗
4. Efficient attention-based deep encoder and decoder for automatic crack segmentation10.1177/14759217211053776 ↗
5. A novel U-shaped encoder-decoder network with attention mechanism for detection and evaluation of road cracks at pixel level10.1111/mice.12826 ↗
6. Pavement defect detection with fully convolutional network and an uncertainty framework10.1111/mice.12533 ↗
7. A lightweight crack segmentation network based on knowledge distillation10.1016/j.jobe.2023.107200 ↗
8. Zhu, Y., and Tang, H. (2023). Automatic Damage Detection and Diagnosis for Hydraulic Structures Using Drones and Artificial Intelligence Techniques. Remote Sens., 15.10.3390/rs15030615 ↗
9. Automatic Pixel-Level Crack Detection and Measurement Using Fully Convolutional Network10.1111/mice.12412 ↗
10. Computer vision-based concrete crack detection using U-net fully convolutional networks10.1016/j.autcon.2019.04.005 ↗
11. Estimation of crack width based on shape-sensitive kernels and semantic segmentation10.1002/stc.2504 ↗
12. A Generative adversarial learning strategy for enhanced lightweight crack delineation networks10.1016/j.aei.2022.101575 ↗
13. Automated detection and segmentation of cracks in concrete surfaces using joined segmentation and classification deep neural network10.1016/j.conbuildmat.2023.133582 ↗
14. Cost-effective system for detection and quantification of concrete surface cracks by combination of convolutional neural network and image processing techniques10.1016/j.conbuildmat.2021.123549 ↗
15. Hybrid pixel-level concrete crack segmentation and quantification across complex backgrounds using deep learning10.1016/j.autcon.2020.103291 ↗
16. Pixel-level detection and measurement of concrete crack using faster region-based convolutional neural network and morphological feature extraction10.1088/1361-6501/abb274 ↗
17. Tunnel crack detection using coarse-to-fine region localization and edge detection10.1002/widm.1308 ↗
18. Dai, J., He, K., and Sun, J. (2015, January 7–13). BoxSup: Exploiting Bounding Boxes to Supervise Convolutional Networks for Semantic Segmentation. Proceedings of the International Conference on Computer Vision (ICCV), Santiago, Chile.10.1109/ICCV.2015.191 ↗
19. Ren, S.Q., He, K.M., Girshick, R., and Sun, J. (2015, January 7–12). Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks. Proceedings of the 29th Annual Conference on Neural Information Processing Systems (NIPS), Montreal, Canada.
20. Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C.Y., and Berg, A.C. (2016). Proceedings of the European Conference on Computer Vision, Springer.
21. Redmon, J., Divvala, S., Girshick, R., and Farhadi, A. (2016, January 27–30). You Only Look Once: Unified, real-time object detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, LasVegas, NV, USA.10.1109/CVPR.2016.91 ↗
22. Bochkovskiy, A., Wang, C., and Liao, H.M. (2020). YOLOv4: Optimal Speed and Accuracy of Object Detection. arXiv.
23. Intelligent detection model based on a fully convolutional neural network for pavement cracks
24. Research on crack segmentation method of hydro-junction project based on target detection network10.1007/s12205-020-1896-y ↗
25. Xu, G.Y., Han, X., Zhang, Y.W., and Wu, C.Y. (2023). Dam Crack Image Detection Model on Feature Enhancement and Attention Mechanism. Water, 15.10.3390/w15010064 ↗
26. Road Crack Detection Using Deep Neural Network with Receptive Field Block10.1088/1757-899X/782/4/042033 ↗
27. Automated bridge surface crack detection and segmentation using computer vision-based deep learning model10.1016/j.engappai.2022.105225 ↗
28. Dai, J., Qi, H., Xiong, Y., Li, Y., Zhang, G., Hu, H., and Wei, Y. (2017, January 22–29). Deformable Convolutional Networks. Proceedings of the International Conference on Computer Vision (ICCV), Venice, Italy.10.1109/ICCV.2017.89 ↗
29. Girshick, R., Donahue, J., Darrell, T., and Malik, J. (2014, January 23–28). Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation. Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA.10.1109/CVPR.2014.81 ↗
30. Girshick, R. (2015, January 11–18). Fast R-CNN. Proceedings of the 2015 IEEE International Conference on Computer Vision (ICCV), Santiago, Chile.10.1109/ICCV.2015.169 ↗
31. Autonomous Structural Visual Inspection Using Region-Based Deep Learning for Detecting Multiple Damage Types10.1111/mice.12334 ↗
32. Concrete crack detection with handwriting script interferences using faster region-based convolutional neural network10.1111/mice.12497 ↗
33. Automatic defect detection of metro tunnel surfaces using a vision-based inspection system10.1016/j.aei.2020.101206 ↗
34. Teng, S., Liu, Z.C., Chen, G.F., and Cheng, L. (2021). Concrete Crack Detection Based on Well-Known Feature Extractor Model and the YOLO_v2 Network. Appl. Sci., 11.10.3390/app11020813 ↗
35. Pavement distress detection and classification based on YOLO network10.1080/10298436.2020.1714047 ↗
36. Survey on performance of deep learning models for detecting road damages using multiple dashcam image resources10.1016/j.aei.2020.101182 ↗
37. Redmon, J., and Farhadi, A. (2018). YOLOv3: An incremental improvement. arXiv.
38. He, K.M., Zhang, X.Y., Ren, S.Q., and Sun, J. (2016, January 27–30). Deep Residual Learning for Image Recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.10.1109/CVPR.2016.90 ↗
39. Lin, T.Y., Dollar, P., Girshick, R., He, K.M., Hariharan, B., and Belongie, S. (2017, January 21–26). Feature pyramid networks for object detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.10.1109/CVPR.2017.106 ↗
40. Concrete bridge surface damage detection using a single-stage detector10.1111/mice.12500 ↗
41. A real-time detection approach for bridge cracks based on YOLOv4-FPM10.1016/j.autcon.2020.103514 ↗
42. Automatic detection method of tunnel lining multi-defects via an enhanced You Only Look Once network10.1111/mice.12836 ↗
43. A review on computer vision based defect detection and condition assessment of concrete and asphalt civil infrastructure10.1016/j.aei.2015.01.008 ↗
44. Evaluating Pavement Cracks with Bidimensional Empirical Mode Decomposition10.1155/2008/861701 ↗
45. Analysis of Edge-Detection Techniques for Crack Identification in Bridges10.1061/(ASCE)0887-3801(2003)17:4(255) ↗
46. FoSA: F* Seed-growing Approach for crack-line detection from pavement images10.1016/j.imavis.2011.10.003 ↗
47. Seed-Based Approach for Automated Crack Detection from Pavement Images10.3141/2589-18 ↗
48. Adaptive Road Crack Detection System by Pavement Classification10.3390/s111009628 ↗
49. CrackTree: Automatic crack detection from pavement images10.1016/j.patrec.2011.11.004 ↗
50. Tang, J.S., and Gu, Y.L. (2013, January 13–16). Automatic Crack Detection and Segmetnation Using A Hybrid Algorithm for Road Distress Analysis. Proceedings of the 2013 IEEE International Conference on Systems, Man, and Cybernetics, Manchester, UK.10.1109/SMC.2013.516 ↗
51. Li, Q.Q., and Liu, X.L. (2008, January 27–30). Novel approach to pavement image segmentation based on neighboring difference histogram method. Proceedings of the 1st International Congress on Image and Signal Processing, Sanya, China.10.1109/CISP.2008.13 ↗
52. Oliveira, H., and Correia, P. (2009, January 24–28). Automatic road crack segmentation using entropy and image dynamic thresholding. Proceedings of the 17th European IEEE Signal Processing Conference, Glasgow, Scotland.
53. DUNet: A deformable network for retinal vessel segmentation10.1016/j.knosys.2019.04.025 ↗
54. Wang, Y.H., Ye, S.J., Bai, Y., Gao, G.M., and Gu, Y.F. (2021, January 11–16). Vehicle detection using deep learning with deformable convolution. Proceedings of the 2021 IEEE International Geoscience and Remote Sensing Symposium IGARSS, Brussels, Belgium.10.1109/IGARSS47720.2021.9553028 ↗
55. A lightweight dead fish detection method based on deformable convolution and YOLOV410.1016/j.compag.2022.107098 ↗
56. Structural Crack Detection from Benchmark Data Sets Using Pruned Fully Convolutional Networks10.1061/(ASCE)ST.1943-541X.0003140 ↗
57. (2020, June 29). RoLabelImg. Available online: https://github.com/roLabelImg-master.
58. (2017, January 01). PyTorch. Available online: https://pytorch.org.
59. Kingma, D.P., and Ba, J.L. (2015). Adam: A method for stochastic optimization. arXiv.
60. Distance-IoU Loss: Faster and Better Learning for Bounding Box Regression
61. Focal Loss for Dense Object Detection10.1109/TPAMI.2018.2858826 ↗
62. (2011, July 04). OpenCV. Available online: https://opencv.org.
63. The Pascal visual object classes (VOC) challenge10.1007/s11263-009-0275-4 ↗
64. Image-based concrete crack detection in tunnels using deep fully convolutional networks10.1016/j.conbuildmat.2019.117367 ↗
65. Robust Pixel-Level Crack Detection Using Deep Fully Convolutional Neural Networks10.1061/(ASCE)CP.1943-5487.0000854 ↗