Crack Detection from a Concrete Surface Image Based on Semantic Segmentation Using Deep Learning
Source abstract
Abstract
Due to their wide applicability in inspection of concrete structures, there is considerable interest in the development of automated crack detection method by image processing. However, the accuracy of existing methods tends to be influenced by the existence of traces of tie-rod holes and formworks. In order to reduce these influences, this paper proposes a crack detection method based on semantic segmentation by deep learning. The accuracy of developed method is investigated by the photos of concrete structures with lots of adverse conditions including shadow and dirt, and it is found that not only the crack region could be detected but also the trace of tie-rod holes and formworks could be removed from the detection result with high accuracy. This paper is the English translation from the authors' previous work [Yamane, T. and Chun, P., (2019). “Crack detection from an image of concrete surface based on semantic segmentation by deep learning.” Journal of Structural Engineering, 65A, 130-138. (in Japanese)].
Source: openalex_inverted_index · Quality: baseline
Research context
OpenAlex topics: Infrastructure Maintenance and Monitoring · Concrete Corrosion and Durability · Asphalt Pavement Performance Evaluation
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Department of International Studies, The University of Tokyo, Japan.
Department of Civil Engineering, The University of Tokyo, Japan.
References (63)
Show source-provided references
1. 1) Abdel-Qader, I., Abudayyeh, O. and Kelly, M., (2003). “Analysis of edge-detection techniques for crack identification in bridges.” Journal of Computing in Civil Engineering, 17(4), 255-263.10.1061/(ASCE)0887-3801(2003)17:4(255) ↗
2. 2) Amarasiri, S., Gunaratne, M. and Sarkar, S., (2009). “Modeling of crack depths in digital images of concrete pavements using optical reflection proper-ties.” Journal of Transportation Engineering, 136(6), 489-499.10.1061/(ASCE)TE.1943-5436.0000095 ↗
3. 3) Badrinarayanan, V., Kendall, A. and Cipolla, R., (2017). “Segnet: A deep convolutional encoder-decoder architecture for image segmentation.” IEEE Transac-tions on Pattern Analysis and Machine Intelligence, 39(12), 2481-2495.10.1109/TPAMI.2016.2644615 ↗
4. 4) Bang, S., Park, S., Kim, H. and Kim, H., (2019). “Encoder-decoder network for pixel level road crack detection in black box images.” Computer-Aided Civil and Infrastructure Engineering, 34, 713-727.10.1111/mice.12440 ↗
5. 5) Blake, A., Rother, C., Brown, M., Perez, P. and Torr, P., (2004). “Interactive image segmentation using an adaptive GMMRF model.” In: T. Pajdla and J. Matas Eds. Proceedings of the 8th European Conference on Computer Vision, Part 1, Prague, Czech Republic 11-14 May 2004. Berlin and Heidelberg: Springer, 428-441.10.1007/978-3-540-24670-1_33 ↗
6. 6) Cha, Y. J., Choi, W. and Büyüköztürk, O., (2017). “Deep learning-based crack damage detection using convolutional neural networks.” Computer-Aided Civil and Infrastructure Engineering, 32(5), 361-378.10.1111/mice.12263 ↗
7. 7) Cha, Y. J., Choi, W., Suh, G., Mahmoudkhani, S. and Büyüköztürk, O., (2018). “Autonomous structural visual inspection using region-based deep learning for detecting multiple damage types.” Computer-Aided Civil and Infrastructure Engineering, 33(9), 731-747.10.1111/mice.12334 ↗
8. 8) Chun, P., Lee, S. H., Cho, S. H. and Lim, Y. M., (2013). “Experimental study on blast resistance of SIFCON.” Journal of Advanced Concrete Technology, 11(4), 144-150.10.3151/jact.11.144 ↗
9. 9) Chun, P. and Igo, A., (2015). “Crack detection from image using random forest.” Journal of Japan Society of Civil Engineers, Ser. F3 (Civil Engineering Informatics), 71(2), I_1- I_8. (in Japanese)10.2208/jscejcei.71.I_1 ↗
10. 10) Chun, P., Shimamoto, Y., Okubo, K., Miwa, C. and Ohga, M., (2017). “Deep learning and random forest based crack detection from an image of concrete surface.” Journal of Japan Society of Civil Engineers, Ser. F3 (Civil Engineering Informatics), 73(2), I_297-I_307. (in Japanese)10.2208/jscejcei.73.I_297 ↗
11. 11) Chun, P., Yamane, T., Izumi, S. and Kameda, T., (2019). “Evaluation of tensile performance of steel members by analysis of corroded steel surface using deep learning.” Metals, 9(12), 1259.10.3390/met9121259 ↗
12. 12) Chun, P., Dang, J., Hamasaki, S., Yajima, R., Kameda, T., Wada, H., Yamane, T., Izumi, S. and Nagatani, K., (2020a). “Utilization of unmanned aerial vehicle, artificial intelligence, and remote measurement tech-nology for bridge inspections.” Journal of Robotics and Mechatronics. (in press)10.20965/jrm.2020.p1244 ↗
13. 13) Chun, P., Izumi, S. and Yamane, T., (2020b). “Automatic detection method of cracks from concrete surface imagery using two-step light gradient boosting machine.” Computer-Aided Civil and Infrastructure Engineering. (in press)10.1111/mice.12564 ↗
14. 14) Chun, P., Ujike, I., Mishima K., Kusumoto, M. and Okazaki, S., (2020c). “Random forest-based evaluation technique for internal damage in reinforced concrete featuring multiple nondestructive testing results.” Construction and Building Materials, 253(30), Article ID 119238.10.1016/j.conbuildmat.2020.119238 ↗
15. 15) Chun, P., Yamane, T., Izumi S. and Kuramoto N., (2020d). “Development of a machine learning-based damage identification method using multi-point simultaneous acceleration measurement results.” Sensors, 20(10), 2780.10.3390/s20102780 ↗
16. 16) Dixit, A. and Wagatsuma, H., (2018). “Comparison of effectiveness of dual tree complex wavelet transform and anisotropic diffusion in MCA for concrete crack detection.” In: L. O’Conner, Ed. Proceedings of the 2018 IEEE International Conference on Systems, Man, and Cybernetics (SMC 2018), Miyazaki, Japan 7-10 October 2018. Piscatway, New Jersey, USA: Institute of Electrical and Electronics Engineers, 2681-2686.
17. 17) Dung, C. V., (2019). “Autonomous concrete crack detection using deep fully convolutional neural network.” Automation in Construction, 99, 52-58.10.1016/j.autcon.2018.11.028 ↗
18. 18) Deng, J., Lu, Y. and Lee, V. C. S., (2019). “Concrete crack detection with handwriting script interferences using faster region-based convolutional neural network.” Computer-Aided Civil and Infrastructure Engineering, 35(4), 373-388.10.1111/mice.12497 ↗
19. 19) Fei, Y., Wang, K. C., Zhang, A., Chen, C., Li, J. Q., Liu, Y., Yang, G. and Li, B., (2019). “Pixel-level cracking detection on 3D asphalt pavement images through deep-learning-based CrackNet-V.” IEEE Transactions on Intelligent Transportation Systems, 21(1), 273-284.10.1109/TITS.2019.2891167 ↗
20. 20) Fujita, Y., Mitani, Y. and Hamamoto, Y., (2006). “A method for crack detection on a concrete structure.” In: B. Werner, Ed. Proceedings of the 18th International Conference on Pattern Recognition (ICPR'06), Hong Kong, China 20-24 August 2006. Piscatway, New Jersey, USA: Institute of Electrical and Electronics Engineers, 3, 901-904.10.1109/ICPR.2006.98 ↗
21. 21) Fujita, Y. and Hamamoto, Y., (2009). “A robust method for automatically detecting cracks on noisy concrete surfaces.” In: B. C. Chien, T. P. Hong, S. M. Chen and M. Ali, Eds. Proceedings of the 22nd International Conference on Industrial, Engineering and Other Applications of Applied Intelligent Systems, Tainan, Taiwan 24-27 June 2009. Berlin: Springer, 76-85.10.1007/978-3-642-02568-6_8 ↗
22. 22) Fujita, Y. and Hamamoto, Y., (2011). “A robust automatic crack detection method from noisy concrete surfaces.” Machine Vision and Applications, 22(2), 245-254.10.1007/s00138-009-0244-5 ↗
23. 23) He, K., Zhang, X., Ren, S. and Sun, J., (2016). “Deep residual learning for image recognition.” In: L. O’Conner, Ed. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA 27-30 June 2016. Piscatway, New Jersey, USA: Institute of Electrical and Electronics Engineers, 770-778.10.1109/CVPR.2016.90 ↗
24. 24) He, K., Gkioxari, G., Dollár, P. and Girshick, R., (2017). “Mask R-CNN.” In: L. O’Conner, Ed. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy 22-29 October 2017. Piscatway, New Jersey, USA: Institute of Electrical and Electronics Engineers, 2961-2969.
25. 25) Hutchinson, T. C. and Chen, Z., (2006). “Improved image analysis for evaluating concrete damage.” Journal of Computing in Civil Engineering, 20(3), 210-216.10.1061/(ASCE)0887-3801(2006)20:3(210) ↗
26. 26) Ito, A., Aoki, Y. and Hashimoto, S., (2002). “Accurate extraction and measurement of fine cracks from concrete block surface image.” In: Proceedings of the 28th Annual Conference of the Industrial Electronics Society (IECON 02), Sevilla, Spain 5-8 November 2002. Piscatway, New Jersey, USA: Institute of Electrical and Electronics Engineers, 3, 2202-2207.
27. 27) Iyer, S. and Sinha, S. K., (2006). “Segmentation of pipe images for crack detection in buried sewers.” Computer-Aided Civil and Infrastructure Engineering, 21(6), 395-410.10.1111/j.1467-8667.2006.00445.x ↗
28. 28) Jiang, S. and Zhang, J., (2019). “Real-time crack assess-ment using deep neural networks with wall-climbing unmanned aerial system.” Computer-Aided Civil and Infrastructure Engineering, 35(6), 549-564.10.1111/mice.12519 ↗
29. 29) Kobayashi, T., (2018). “Spiral-Net with F1-based optimization for image-based crack detection.” In: C. V. Jawahar, H. Li, G. Mori and K. Schindler, Eds. Proceedings of the 14th Asian Conference on Computer Vision, Perth, Australia 2-4 December 2018. Berlin: Springer, Part I, 88-104.
30. 30) Koch, C., Georgieva, K., Kasireddy, V., Akinci, B. and Fieguth, P., (2015). “A review on computer vision based defect detection and condition assessment of concrete and asphalt civil infrastructure.” Advanced Engineering Informatics, 29(2), 196-210.10.1016/j.aei.2015.01.008 ↗
31. 31) Lee, B. Y., Kim, Y. Y., Yi, S. T. and Kim, J. K., (2013). “Automated image processing technique for detecting and analysing concrete surface cracks.” Structure and Infrastructure Engineering, 9(6), 567-577.10.1080/15732479.2011.593891 ↗
32. 32) Li, S. and Zhao, X., (2019). “Image-based concrete crack detection using convolutional neural network and exhaustive search technique.” Advances in Civil Engineering, 2019(2), 1-12.10.1155/2019/6520620 ↗
33. 33) Liu, Y., Yao, J., Lu, X., Xie, R. and Li, L., (2019). “DeepCrack: A deep hierarchical feature learning architecture for crack segmentation.” Neurocomputing, 338, 139-153.10.1016/j.neucom.2019.01.036 ↗
34. 34) Long, J., Shelhamer, E. and Darrell, T., (2015). “Fully convolutional networks for semantic segmentation.” In: S. Lakušić, Ed. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA 8-10 June 2015. Piscatway, New Jersey, USA: Institute of Electrical and Electronics Engineers, 3431-3440.
35. 35) Luxmoore, A., (1973). “Holographic detection of cracks in concrete.” Non-Destructive Testing, 6(5), 258-263.10.1016/0029-1021(73)90073-X ↗
36. 36) Minami, T., Urata, W., Fujiu, M., Fukuoka, T., Sagae, M., Suda, S. and Takayama, J., (2019a). “Development of automatic concrete cracks detection system using average shifted mesh.” Journal of the Eastern Asia Society for Transportation Studies, 13, 1560-1570.
37. 37) Minami, T., Fujiu, M., Takayama, J., Suda, S. and Okumura, S., (2019b). “A study on image diagnostic technology for bridge inspection using ultra high resolution camera.” In: Proceedings of the 5th Inter-national Conference on Road and Rail Infrastructure, Zadar, Croatia 17-19 May 2019. Alexandria, VA, USA: International Road Federation, 79-85.
38. 38) MLIT, (2018). “Roads in Japan 2018.” Tokyo: Ministry of Land, Infrastructure, Transport and Tourism, Japan.
39. 39) Mohan, A. and Poobal, S., (2018). “Crack detection using image processing: A critical review and analysis.” Alexandria Engineering Journal, 57(2), 787-798.10.1016/j.aej.2017.01.020 ↗
40. 40) Moon, H. S., Ok, S., Chun, P. J. and Lim, Y. M., (2019). “Artificial neural network for vertical displacement prediction of a bridge from strains (Part 1): Girder bridge under moving vehicles.” Applied Sciences, 9(14), 2881.10.3390/app9142881 ↗
41. 41) Moon, H. S., Chun, P. J., Kim, M. K. and Lim, Y. M., (2020). “Artificial neural network for vertical displacement prediction of a bridge from strains (Part 2): Optimization of strain-measurement points by a genetic algorithm under dynamic loading.” Applied Sciences, 10(3), 777.10.3390/app10030777 ↗
42. 42) Okazaki, Y., Okazaki, S., Asamoto, S. and Chun, P., (2020). “Applicability of machine learning to a crack model in concrete bridges.” Computer-Aided Civil and Infrastructure Engineering, 35(8), 1-18.10.1111/mice.12532 ↗
43. 43) Rafiei, M. H., Khushefati, W. H., Demirboga, R. and Adeli, H., (2017). “Supervised deep restricted Boltzmann machine for estimation of concrete.” ACI Materials Journal, 114(2), 237-244.10.14359/51689560 ↗
44. 44) Rafiei, M. H. and Adeli, H., (2018a). “A novel unsupervised deep learning model for global and local health condition assessment of structures.” Engineering Structures, 156, 598-607.10.1016/j.engstruct.2017.10.070 ↗
45. 45) Rafiei, M. H. and Adeli, H., (2018b). “Novel machine-learning model for estimating construction costs considering economic variables and indexes.” Journal of Construction Engineering and Management, 144(12), Article ID 04018106.10.1061/(ASCE)CO.1943-7862.0001570 ↗
46. 46) Ren, S., He, K., Girshick, R. and Sun, J., (2015). “Faster R-CNN: Towards real-time object detection with region proposal networks.” In: C. Cortes, N. D. Lawrence, D. D. Lee, M. Sugiyama and R. Garnett, Eds. Proceedings of 28th Conference on Advances in Neural Information Processing Systems (NIPS 2015), Montreal, Canada 7-12 December 2015. San Diego, CA, USA: Neural Information Processing Systems Foundation, 91-99.
47. 47) Silva, W. R. L. D. and Lucena, D. S. D., (2018). “Concrete cracks detection based on deep learning image classification.” In: D. V. Hemelrijck, D. Aggelis, N. D. Belie, F. Delaunois, T. Geernaerts, P. Guillaume, A. M. Habraken, P. Hendrick, E. Reynders, A. Simar and S. Vanlanduit, Eds. Proceedings of the 18th International Conference on Experimental Mechanics (ICEM18), Brussels, Belgium 1-5 July 2018. Basel, Switzerland: Multi-Digital Publishing Institute, 2(8), 489.10.3390/ICEM18-05387 ↗
48. 48) Sinha, S. K. and Fieguth, P. W., (2006). “Segmentation of buried concrete pipe images.” Automation in Construction, 15(1), 47-57.10.1016/j.autcon.2005.02.007 ↗
49. 49) Sohn, H. G., Lim, Y. M., Yun, K. H. and Kim, G. H., (2005). “Monitoring crack changes in concrete structures.” Computer-Aided Civil and Infrastructure Engineering, 20(1), 52-61.10.1111/j.1467-8667.2005.00376.x ↗
50. 50) Vezhnevets, V. and Konouchine, V., (2005). “GrowCut: Interactive multi-label ND image segmentation by cellular automata.” In: Proceedings of 15th International Conference on Computer Graphics and Applications (GraphiCon 2005), Novosibirsk Academic Town, Russia 20-24 June 2005. Novosibirsk, Russia: Russian Foundation for Basic Research, 1(4), 150-156.
51. 51) Ward, G. J., (1992). “Measuring and modeling anisotropic reflectance.” Computer Graphics, 26(2), 265-272.10.1145/142920.134078 ↗
52. 52) Xue, Y. and Li, Y., (2018). “A fast detection method via region-based fully convolutional neural networks for shield tunnel lining defects.” Computer-Aided Civil and Infrastructure Engineering, 33(8), 638-654.10.1111/mice.12367 ↗
53. 53) Yamaguchi, T., Nakamura, S., Saegusa, R. and Hashimoto, S., (2008). “Image based crack detection for real concrete surfaces.” IEEJ Transactions on Electrical and Electronic Engineering, 3(1), 128-135.10.1002/tee.20244 ↗
54. 54) Yamaguchi, T. and Hashimoto, S., (2008). “Improved percolation-based method for crack detection in concrete surface images.” In: Proceedings of the 19th International Conference on Pattern Recognition (ICPR’08), Tampa, Florida, USA 8-11 December 2008. Piscatway, New Jersey, USA: Institute of Electrical and Electronics Engineers, 1-4.10.1109/ICPR.2008.4761627 ↗
55. 55) Yamaguchi, T. and Hashimoto, S., (2009). “Practical image measurement of crack width for real concrete structure.” Electronics and Communications in Japan, 92(10), 1-12.10.1002/ecj.10151 ↗
56. 56) Yamaguchi, T. and Hashimoto, S., (2010). “Fast crack detection method for large-size concrete surface images using percolation-based image processing.” Machine Vision and Applications, 21(5), 797-809.10.1007/s00138-009-0189-8 ↗
57. 57) Yamane, T. and Chun, P., (2019). “Crack detection from an image of concrete surface based on semantic segmentation by deep learning.” Journal of Structural Engineering, 65A, 130-138. (in Japanese)
58. 58) Yang, X., Li, H., Yu, Y., Luo, X., Huang, T. and Yang, X., (2018). “Automatic pixel-level crack detection and measurement using fully convolutional network.” Computer-Aided Civil and Infrastructure Engineering, 33(12), 1090-1109.10.1111/mice.12412 ↗
59. 59) Yokoyama, S. and Matsumoto, T., (2017). “Development of an automatic detector of cracks in concrete using machine learning.” Procedia Engineering, 171, 1250-1255.10.1016/j.proeng.2017.01.418 ↗
60. 60) Zakeri, H., Nejad, F. M. and Fahimifar, A., (2017). “Image based techniques for crack detection, classification and quantification in asphalt pavement: A review.” Archives of Computational Methods in Engineering, 24(4), 935-977.10.1007/s11831-016-9194-z ↗
61. 61) Zhang, X., Rajan, D. and Story, B., (2019). “Concrete crack detection using context aware deep semantic segmentation network.” Computer-Aided Civil and Infrastructure Engineering, 34(11), 1-21.10.1111/mice.12477 ↗
62. 62) Zhang, A., Wang, K. C. P., Fei, Y., Liu, Y., Chen, C., Yang, G., Li, J. Q., Yang, E. and Qiu, S., (2018a). “Automated pixel-level pavement crack detection on 3D asphalt surfaces with a recurrent neural network.” Computer-Aided Civil and Infrastructure Engineering, 34(3), 213-229.10.1111/mice.12409 ↗
63. 63) Zhang, A., Wang, K. C. P., Fei, Y., Liu, Y., Tao, S., Chen, C., Li, J. Q. and Li, B., (2018b). “Deep learning-based fully automated pavement crack detection on 3D asphalt surfaces with an improved CrackNet.” Journal of Computing in Civil Engineering, 32(5), Article ID 04018041.10.1061/(ASCE)CP.1943-5487.0000775 ↗