A Comprehensive Review of Deep Learning-Based Crack Detection Approaches
Source abstract
Abstract
The application of deep architectures inspired by the fields of artificial intelligence and computer vision has made a significant impact on the task of crack detection. As the number of studies being published in this field is growing fast, it is important to categorize the studies at deeper levels. In this paper, a comprehensive literature review of deep learning-based crack detection studies and the contributions they have made to the field is presented. The studies are categorised according to the computer vision aspect and at deeper levels to facilitate exploring the studies that utilised similar approaches to address the crack detection problem. Moreover, the authors perform a comparison between the studies which use the same publicly available data sets, in order to find the most promising crack detection approaches. Critical future directions for research are proposed, based on these reviewed studies as well as on trends and developments in areas similar to the crack detection area.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Infrastructure Maintenance and Monitoring · Non-Destructive Testing Techniques · Industrial Vision Systems and Defect Detection
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
School of Information and Communication Technology, Gold Coast Campus, Griffith University, Southport, QLD 4222, Australia
School of Engineering and Built Environment, Gold Coast Campus, Griffith University, Southport, QLD 4222, Australia
References (100+)
Show source-provided references
1. Cagle, R.F. (2003). Infrastructure Asset Management: An Emerging Direction, AACE International Transactions.
2. Feature Pyramid and Hierarchical Boosting Network for Pavement Crack Detection10.1109/TITS.2019.2910595 ↗
3. Autonomous Structural Visual Inspection Using Region-Based Deep Learning for Detecting Multiple Damage Types10.1111/mice.12334 ↗
4. A novel hybrid approach for crack detection10.1016/j.patcog.2020.107474 ↗
5. A survey and evaluation of promising approaches for automatic image-based defect detection of bridge structures10.1080/15732470801945930 ↗
6. Analysis of Edge-Detection Techniques for Crack Identification in Bridges10.1061/(ASCE)0887-3801(2003)17:4(255) ↗
7. Automatic Road Defect Detection by Textural Pattern Recognition Based on AdaBoost10.1111/j.1467-8667.2011.00736.x ↗
8. Oliveira, H., and Correia, P.L. (2009, January 24–28). Automatic road crack segmentation using entropy and image dynamic thresholding. Proceedings of the 2009 17th European Signal Processing Conference, Glasgow, UK.
9. Automated Crack Detection on Concrete Bridges10.1109/TASE.2014.2354314 ↗
10. Automatic Road Crack Detection Using Random Structured Forests10.1109/TITS.2016.2552248 ↗
11. A self organizing map optimization based image recognition and processing model for bridge crack inspection10.1016/j.autcon.2016.08.033 ↗
12. Goodfellow, I., Bengio, Y., Courville, A., and Bengio, Y. (2016). Deep Learning, MIT Press.
13. Machine Learning for Crack Detection: Review and Model Performance Comparison10.1061/(ASCE)CP.1943-5487.0000918 ↗
14. Gopalakrishnan, K. (2018). Deep Learning in Data-Driven Pavement Image Analysis and Automated Distress Detection: A Review. Data, 3.10.3390/data3030028 ↗
15. Review of Pavement Defect Detection Methods10.1109/ACCESS.2020.2966881 ↗
16. Farrar, C.R., and Worden, K. (2012). Structural Health Monitoring: A Machine Learning Perspective, John Wiley & Sons.10.1002/9781118443118 ↗
17. Deep Neural Networks for Acoustic Modeling in Speech Recognition: The Shared Views of Four Research Groups10.1109/MSP.2012.2205597 ↗
18. Proceedings of the NAACL-HLT 2012 Workshop: Will We Ever Really Replace the N-gram Model? On the Future of Language Modeling for HLT
19. Discovering Binary Codes for Documents by Learning Deep Generative Models10.1111/j.1756-8765.2010.01109.x ↗
20. Deep learning for visual understanding: A review10.1016/j.neucom.2015.09.116 ↗
21. A survey on deep learning in medical image analysis10.1016/j.media.2017.07.005 ↗
22. Deep-anomaly: Fully convolutional neural network for fast anomaly detection in crowded scenes10.1016/j.cviu.2018.02.006 ↗
23. Deep Learning-Based Crack Damage Detection Using Convolutional Neural Networks10.1111/mice.12263 ↗
24. Development of an Automatic Detector of Cracks in Concrete Using Machine Learning10.1016/j.proeng.2017.01.418 ↗
25. Pauly, L., Hogg, D., Fuentes, R., and Peel, H. (July, January 28). Deeper networks for pavement crack detection. Proceedings of the 34th International Symposium in Automation and Robotics in Construction, Taipei, Taiwan.
26. Automatic classification of pavement crack using deep convolutional neural network10.1080/10298436.2018.1485917 ↗
27. Automatic recognition of asphalt pavement cracks using metaheuristic optimized edge detection algorithms and convolution neural network10.1016/j.autcon.2018.07.008 ↗
28. Comparison of deep convolutional neural networks and edge detectors for image-based crack detection in concrete10.1016/j.conbuildmat.2018.08.011 ↗
29. Crack and Noncrack Classification from Concrete Surface Images Using Machine Learning10.1177/1475921718768747 ↗
30. Deep Active Learning for Civil Infrastructure Defect Detection and Classification
31. Wang, X., and Hu, Z. (2017, January 8–10). Grid-based pavement crack analysis using deep learning. Proceedings of the 2017 4th International Conference on Transportation Information and Safety (ICTIS), Banff, AB, Canada.10.1109/ICTIS.2017.8047878 ↗
32. Patch-Based Crack Detection in Black Box Images Using Convolutional Neural Networks10.1061/(ASCE)CP.1943-5487.0000831 ↗
33. He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27–30). Deep Residual Learning for Image Recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.10.1109/CVPR.2016.90 ↗
34. Pereira, F., Burges, C.J.C., Bottou, L., and Weinberger, K.Q. (2012). ImageNet Classification with Deep Convolutional Neural Networks. Advances in Neural Information Processing Systems 25, Curran Associates Inc.
35. Gradient-based learning applied to document recognition10.1109/5.726791 ↗
36. Deep Convolutional Neural Networks for Computer-Aided Detection: CNN Architectures, Dataset Characteristics and Transfer Learning10.1109/TMI.2016.2528162 ↗
37. Deep Convolutional Neural Networks with transfer learning for computer vision-based data-driven pavement distress detection10.1016/j.conbuildmat.2017.09.110 ↗
38. Simonyan, K., and Zisserman, A. (2015). Very Deep Convolutional Networks for Large-Scale Image Recognition. arXiv.
39. da Silva, W.R.L., and de Lucena, D.S. (2018). Concrete Cracks Detection Based on Deep Learning Image Classification. Proceedings, 2.10.3390/ICEM18-05387 ↗
40. Kim, B., and Cho, S. (2018). Automated Vision-Based Detection of Cracks on Concrete Surfaces Using a Deep Learning Technique. Sensors, 18.10.3390/s18103452 ↗
41. Dorafshan, S., Thomas, R.J., Coopmans, C., and Maguire, M. (2018, January 12–15). Deep Learning Neural Networks for sUAS-Assisted Structural Inspections: Feasibility and Application. Proceedings of the 2018 International Conference on Unmanned Aircraft Systems (ICUAS), Dallas, TX, USA.10.1109/ICUAS.2018.8453409 ↗
42. Vedaldi, A., and Lenc, K. (2015, January 26–30). MatConvNet. Proceedings of the 23rd ACM international conference on Multimedia, Brisbane, Australia.10.1145/2733373.2807412 ↗
43. Autonomous UAVs for Structural Health Monitoring Using Deep Learning and an Ultrasonic Beacon System with Geo-Tagging10.1111/mice.12375 ↗
44. Wang, K., Zhang, A., Li, J.Q., Fei, Y., Chen, C., and Li, B. (2017, January 27–30). Deep learning for asphalt pavement cracking recognition using convolutional neural network. Proceedings of the 2017 International Conference on Highway Pavements and Airfield Technology, Philadelphia, PA, USA.10.1061/9780784480922.015 ↗
45. Eisenbach, M., Stricker, R., Seichter, D., Amende, K., Debes, K., Sesselmann, M., Ebersbach, D., Stoeckert, U., and Gross, H. (2017, January 14–19). How to get pavement distress detection ready for deep learning? A systematic approach. Proceedings of the 2017 International Joint Conference on Neural Networks (IJCNN), Anchorage, AK, USA.10.1109/IJCNN.2017.7966101 ↗
46. Szeliski, R. (2010). Computer Vision: Algorithms and Applications, Springer. [1st ed.].10.1007/978-1-84882-935-0 ↗
47. Girshick, R., Donahue, J., Darrell, T., and Malik, J. (2013). Rich feature hierarchies for accurate object detection and semantic segmentation. arXiv.10.1109/CVPR.2014.81 ↗
48. Redmon, J., Divvala, S., Girshick, R., and Farhadi, A. (2016). You Only Look Once: Unified, Real-Time Object Detection. arXiv.10.1109/CVPR.2016.91 ↗
49. Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C.Y., and Berg, A.C. (2016). SSD: Single Shot MultiBox Detector. arXiv.10.1007/978-3-319-46448-0_2 ↗
50. Girshick, R. (2015). Fast R-CNN. arXiv.10.1109/ICCV.2015.169 ↗
51. Ren, S., He, K., Girshick, R., and Sun, J. (2015). Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks. arXiv.
52. Kim, I.H., Jeon, H., Baek, S.C., Hong, W.H., and Jung, H.J. (2018). Application of Crack Identification Techniques for an Aging Concrete Bridge Inspection Using an Unmanned Aerial Vehicle. Sensors, 18.10.3390/s18061881 ↗
53. Unified Vision-Based Methodology for Simultaneous Concrete Defect Detection and Geolocalization10.1111/mice.12351 ↗
54. Detection of sealed and unsealed cracks with complex backgrounds using deep convolutional neural network10.1016/j.autcon.2019.102946 ↗
55. Concrete crack detection with handwriting script interferences using faster region-based convolutional neural network10.1111/mice.12497 ↗
56. Ciaparrone, G., Serra, A., Covito, V., Finelli, P., Scarpato, C.A., and Tagliaferri, R. (2018). A Deep Learning Approach for Road Damage Classification. Lecture Notes in Electrical Engineering, Springer.10.1007/978-981-13-1328-8_84 ↗
57. Road Damage Detection and Classification Using Deep Neural Networks with Smartphone Images10.1111/mice.12387 ↗
58. Ioffe, S., and Szegedy, C. (2015). Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift. arXiv.
59. Howard, A.G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., Andreetto, M., and Adam, H. (2017). MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications. arXiv.
60. Redmon, J., and Farhadi, A. (2016). YOLO9000: Better, Faster, Stronger. arXiv.10.1109/CVPR.2017.690 ↗
61. Redmon, J., and Farhadi, A. (2018). YOLOv3: An Incremental Improvement. arXiv.
62. Mandal, V., Uong, L., and Adu-Gyamfi, Y. (2018, January 10–13). Automated Road Crack Detection Using Deep Convolutional Neural Networks. Proceedings of the 2018 IEEE International Conference on Big Data (Big Data), Seattle, WA, USA.10.1109/BigData.2018.8622327 ↗
63. Chen, L.C., Papandreou, G., Kokkinos, I., Murphy, K., and Yuille, A.L. (2016). Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs. arXiv.
64. Pohlen, T., Hermans, A., Mathias, M., and Leibe, B. (2017, January 22–25). Full-Resolution Residual Networks for Semantic Segmentation in Street Scenes. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.10.1109/CVPR.2017.353 ↗
65. Geiger, A., Lenz, P., and Urtasun, R. (2012, January 16–21). Are we ready for autonomous driving? The KITTI vision benchmark suite. Proceedings of the 2012 IEEE Conference on Computer Vision and Pattern Recognition, Washington, DC, USA.10.1109/CVPR.2012.6248074 ↗
66. Zhang, Z., Fidler, S., and Urtasun, R. (2016, January 27–30). Instance-Level Segmentation for Autonomous Driving With Deep Densely Connected MRFs. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.10.1109/CVPR.2016.79 ↗
67. Lee, D.C., Hebert, M., and Kanade, T. (2009, January 20–25). Geometric reasoning for single image structure recovery. Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition, Miami, FL, USA.10.1109/CVPRW.2009.5206872 ↗
68. Kundu, A., Li, Y., Dellaert, F., Li, F., and Rehg, J.M. (2014). Joint Semantic Segmentation and 3D Reconstruction from Monocular Video. Computer Vision—ECCV 2014, Springer International Publishing.10.1007/978-3-319-10599-4_45 ↗
69. Pham, Q., Hua, B., Nguyen, T., and Yeung, S. (2019, January 7–11). Real-Time Progressive 3D Semantic Segmentation for Indoor Scenes. Proceedings of the 2019 IEEE Winter Conference on Applications of Computer Vision (WACV), Waikoloa, HI, USA.10.1109/WACV.2019.00121 ↗
70. Milletari, F., Navab, N., and Ahmadi, S. (2016, January 25–28). V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation. Proceedings of the 2016 Fourth International Conference on 3D Vision (3DV), Stanford, CA, USA.10.1109/3DV.2016.79 ↗
71. Stoyanov, D., Taylor, Z., Carneiro, G., Syeda-Mahmood, T., Martel, A., Maier-Hein, L., Tavares, J.M.R., Bradley, A., Papa, J.P., and Belagiannis, V. (2018). UNet++: A Nested U-Net Architecture for Medical Image Segmentation. Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support, Springer International Publishing.10.1007/978-3-030-00889-5 ↗
72. Alom, M.Z., Hasan, M., Yakopcic, C., Taha, T.M., and Asari, V.K. (2018). Recurrent Residual Convolutional Neural Network based on U-Net (R2U-Net) for Medical Image Segmentation. arXiv.10.1109/NAECON.2018.8556686 ↗
73. Robust Concrete Crack Detection Using Deep Learning-Based Semantic Segmentation10.1007/s42405-018-0120-5 ↗
74. Densely connected deep neural network considering connectivity of pixels for automatic crack detection10.1016/j.autcon.2019.103018 ↗
75. Crack Detection and Segmentation Using Deep Learning with 3D Reality Mesh Model for Quantitative Assessment and Integrated Visualization10.1061/(ASCE)CP.1943-5487.0000890 ↗
76. Hybrid pixel-level concrete crack segmentation and quantification across complex backgrounds using deep learning10.1016/j.autcon.2020.103291 ↗
77. Zernike-moment measurement of thin-crack width in images enabled by dual-scale deep learning10.1111/mice.12421 ↗
78. Unified Approach to Pavement Crack and Sealed Crack Detection Using Preclassification Based on Transfer Learning10.1061/(ASCE)CP.1943-5487.0000736 ↗
79. He, K., Gkioxari, G., Dollar, P., and Girshick, R. (2017, January 22–29). Mask R-CNN. Proceedings of the IEEE International Conference on Computer Vision (ICCV), Venice, Italy.10.1109/ICCV.2017.322 ↗
80. Instance-level recognition and quantification for concrete surface bughole based on deep learning10.1016/j.autcon.2019.102920 ↗
81. Image-based concrete crack assessment using mask and region-based convolutional neural network10.1002/stc.2381 ↗
82. Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., and Rabinovich, A. (2015, January 7–12). Going deeper with convolutions. Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA.10.1109/CVPR.2015.7298594 ↗
83. Fang, F., Li, L., Rice, M., and Lim, J. (2019, January 22–25). Towards Real-Time Crack Detection Using a Deep Neural Network With a Bayesian Fusion Algorithm. Proceedings of the 2019 IEEE International Conference on Image Processing (ICIP), Taipei, Taiwan.10.1109/ICIP.2019.8803357 ↗
84. Tan, C., Uddin, N., and Mohammed, Y.M. (2019, January 4–7). Deep Learning-Based Crack Detection Using Mask R-CNN Technique. Proceedings of the 9th International Conference on Structural Health Monitoring of Intelligent Infrastructure, St. Louis, MI, USA.
85. Pixel-level crack delineation in images with convolutional feature fusion10.1002/stc.2286 ↗
86. Concrete crack detection using context-aware deep semantic segmentation network10.1111/mice.12477 ↗
87. Badrinarayanan, V., Handa, A., and Cipolla, R. (2015). SegNet: A Deep Convolutional Encoder-Decoder Architecture for Robust Semantic Pixel-Wise Labelling. arXiv.
88. Zhang, K., Cheng, H., and Gai, S. (2018, January 4–7). Efficient Dense-Dilation Network for Pavement Cracks Detection with Large Input Image Size. Proceedings of the 2018 21st International Conference on Intelligent Transportation Systems (ITSC), Maui, HI, USA.10.1109/ITSC.2018.8569958 ↗
89. Long, J., Shelhamer, E., and Darrell, T. (2015, January 7–12). Fully Convolutional Networks for Semantic Segmentation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA.10.1109/CVPR.2015.7298965 ↗
90. Takikawa, T., Acuna, D., Jampani, V., and Fidler, S. (November, January 27). Gated-SCNN: Gated Shape CNNs for Semantic Segmentation. Proceedings of the IEEE International Conference on Computer Vision (ICCV), Seoul, Korea.
91. Zhang, L., Yang, F., Daniel Zhang, Y., and Zhu, Y.J. (2016, January 25–28). Road crack detection using deep convolutional neural network. Proceedings of the 2016 IEEE International Conference on Image Processing (ICIP), Phoenix, AZ, USA.10.1109/ICIP.2016.7533052 ↗
92. Fan, Z., Wu, Y., Lu, J., and Li, W. (2018). Automatic Pavement Crack Detection Based on Structured Prediction with the Convolutional Neural Network. arXiv.
93. Fan, Z., Li, C., Chen, Y., Mascio, P.D., Chen, X., Zhu, G., and Loprencipe, G. (2020). Ensemble of Deep Convolutional Neural Networks for Automatic Pavement Crack Detection and Measurement. Coatings, 10.10.3390/coatings10020152 ↗
94. Deep Learning–Based Fully Automated Pavement Crack Detection on 3D Asphalt Surfaces with an Improved CrackNet10.1061/(ASCE)CP.1943-5487.0000775 ↗
95. Pixel-Level Cracking Detection on 3D Asphalt Pavement Images Through Deep-Learning- Based CrackNet-V10.1109/TITS.2019.2891167 ↗
96. Automated Pixel-Level Pavement Crack Detection on 3D Asphalt Surfaces Using a Deep-Learning Network10.1111/mice.12297 ↗
97. Automated Pixel-Level Pavement Crack Detection on 3D Asphalt Surfaces with a Recurrent Neural Network10.1111/mice.12409 ↗
98. Noh, H., Hong, S., and Han, B. (2015, January 7–13). Learning Deconvolution Network for Semantic Segmentation. Proceedings of the IEEE International Conference on Computer Vision (ICCV), Santiago, Chile.10.1109/ICCV.2015.178 ↗
99. Ronneberger, O., Fischer, P., and Brox, T. (2015). U-Net: Convolutional Networks for Biomedical Image Segmentation. Lecture Notes in Computer Science, Springer International Publishing.10.1007/978-3-319-24574-4_28 ↗
100. Jegou, S., Drozdzal, M., Vazquez, D., Romero, A., and Bengio, Y. (2017, January 22–25). The One Hundred Layers Tiramisu: Fully Convolutional DenseNets for Semantic Segmentation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, Honolulu, HI, USA.10.1109/CVPRW.2017.156 ↗