Interpretation and understanding of asphalt crack detection deep learning models using integrated gradient (I.G.) maps
Source abstract
Abstract
Asphalt cracking poses a significant deterioration challenge in road networks. Historically, government agencies employed visual inspection methods to identify asphalt cracking. However, this approach is labour-intensive and time-consuming, prompting most authorities to transition to automated crack identification methods utilising Artificial Intelligence (A.I.). Convolutional neural networks (CNNs) have gained popularity due to their ease of implementation and enhanced accuracy. Nevertheless, CNN models were perceived as opaque entities. In this study, we employed Integrated Gradient (I.G.) maps to elucidate the workings of these models and interpret CNN-based crack image voxels that contributed to the positive (cracked) output of CNN. Our findings revealed that CNN utilised voxels within the cracks to classify the cracked image segments and highlighted the image features that CNN employed to distinguish cracks from their surroundings. These interpretation techniques provide insights into the behaviour of the CNN model, enabling the refinement of Deep Learning (DL) models.
Source: openalex_inverted_index · Quality: baseline
Research context
OpenAlex topics: Infrastructure Maintenance and Monitoring · Asphalt Pavement Performance Evaluation · Non-Destructive Testing Techniques
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
References (63)
Show source-provided references
1. Pothole detection on asphalt pavements from 2D-colour pothole images using fuzzy c-means clustering and morphological reconstruction10.1016/j.autcon.2017.08.017 ↗
2. A real-time automatic pavement crack and pothole recognition system for mobile android-based devices10.1016/j.aei.2016.12.004 ↗
3. Deep convolutional neural networks with transfer learning for computer vision-based data-driven pavement distress detection10.1016/j.conbuildmat.2017.09.110 ↗
4. A Component Decomposition Model for 3D Laser Scanning Pavement Data Based on High-Pass Filtering and Sparse Analysis10.3390/s18072294 ↗
5. Deep learning and infrared thermography for asphalt pavement crack severity classification10.1016/j.autcon.2022.104383 ↗
6. K.T. Chang, J.R. Chang, and J.K. Liu, “Detection of Pavement Distresses Using 3D Laser Scanning Technology,” pp. 1–11, Apr. 2012, doi: 10.1061/40794(179)103.10.1061/40794(179)103 ↗
7. Rapid detection methods for asphalt pavement thicknesses and defects by a vehicle-mounted ground penetrating radar (GPR) system10.3390/s16122067 ↗
8. Detection of concealed cracks from ground penetrating radar images based on deep learning algorithm10.1016/j.conbuildmat.2020.121949 ↗
9. 3D crack skeleton extraction from mobile LiDAR point clouds
10. Pavement distress detection using convolutional neural networks with images captured via UAV10.1016/j.autcon.2021.103991 ↗
11. A review of mobile mapping systems: from sensors to applications10.3390/s22114262 ↗
12. Automatic internal crack detection from a sequence of infrared images with a triple-threshold Canny edge detector10.1088/1361-6501/aa9857 ↗
13. Fast local laplacian-based steerable and sobel filters integrated with adaptive boosting classification tree for automatic recognition of asphalt pavement cracks
14. Road Crack Detection Using Gaussian/prewitt Filter
15. An artificial intelligence method for asphalt pavement pothole detection using least squares support vector machine and neural network with steerable filter-based feature extraction
16. Machine learning for crack detection: review and model performance comparison10.1061/(ASCE)CP.1943-5487.0000918 ↗
17. Cracking classification using minimum rectangular cover–based support vector machine10.1061/(ASCE)CP.1943-5487.0000672 ↗
18. Texture analysis based damage detection of ageing infrastructural elements10.1111/j.1467-8667.2012.00790.x ↗
19. Image processing–based classification of asphalt pavement cracks using support vector machine optimized by artificial bee colony10.1061/(ASCE)CP.1943-5487.0000781 ↗
20. A comprehensive review of deep learning-based crack detection approaches10.3390/app12031374 ↗
21. K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 2016-Decem, pp. 770–778, 2016, doi: 10.1109/CVPR.2016.90.10.1109/CVPR.2016.90 ↗
22. K. Simonyan and A. Zisserman, “Very Deep Convolutional Networks for Large-Scale Image Recognition,” arXiv:1409.1556 [cs], Apr. 2015, doi: https://doi.org/10.48550/arXiv.1409.1556.
23. ImageNet Classification with Deep Convolutional Neural Networks
24. “Deep Convolutional Neural Networks with transfer learning for computer vision-based data-driven pavement distress detection - Google Search.” Accessed: Sep. 06, 2022. [Online]. Available: 〈https://www.google.com/search?q=Deep+Convolutional+Neural+Networks+with+transfer+learning+for+computer+vision-based+data-driven+pavement+distress+detection&rlz= 1C1CHBD_en-GBAU889AU889&sourceid=chrome&ie=UTF-8〉.
25. Automatic recognition of asphalt pavement cracks using metaheuristic optimized edge detection algorithms and convolution neural network10.1016/j.autcon.2018.07.008 ↗
26. Automatic classification of pavement crack using deep convolutional neural network10.1080/10298436.2018.1485917 ↗
27. Automated pixel-level pavement crack detection on 3D asphalt surfaces using a deep-learning network10.1111/mice.12297 ↗
28. Deep learning–based fully automated pavement crack detection on 3D asphalt surfaces with an improved crackNet10.1061/(ASCE)CP.1943-5487.0000775 ↗
29. Pixel-level cracking detection on 3D asphalt pavement images through deep-learning- based crackNet-V10.1109/TITS.2019.2891167 ↗
30. Automatic classification of pavement crack using deep convolutional neural network10.1080/10298436.2018.1485917 ↗
31. Assessment of crack severity of asphalt pavements using deep learning algorithms and geospatial system10.1016/j.conbuildmat.2023.132684 ↗
32. K.S. Kristombu Baduge et al., “Asphalt Cracked and Uncracked Image Dataset,” vol. 1, Aug. 2023, doi: 10.17632/88kdyyc73h.1.
33. Automatic recognition of asphalt pavement cracks using metaheuristic optimized edge detection algorithms and convolution neural network10.1016/j.autcon.2018.07.008 ↗
34. S. Ren, K. He, R. Girshick, and J. Sun, “Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks,” arXiv:1506.01497 [cs], Jan. 2016, Accessed: Jun. 10, 2021. [Online]. Available: 〈http://arxiv.org/abs/1506.01497〉.
35. J. Redmon, S. Divvala, R. Girshick, and A. Farhadi, “You Only Look Once: Unified, Real-Time Object Detection,” May 09, 2016, arXiv: arXiv:1506.02640. doi: 10.48550/arXiv.1506.02640.10.1109/CVPR.2016.91 ↗
36. Road damage detection and classification using deep neural networks with smartphone images10.1111/mice.12387 ↗
37. A deep learning approach for road damage detection from smartphone images10.1109/BigData.2018.8621899 ↗
38. O. Ronneberger, P. Fischer, and T. Brox, “U-Net: Convolutional Networks for Biomedical Image Segmentation,” May 18, 2015, arXiv: arXiv:1505.04597. doi: 10.48550/arXiv.1505.04597.10.1007/978-3-319-24574-4_28 ↗
39. V. Badrinarayanan, A. Kendall, and R. Cipolla, “SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation,” Oct. 10, 2016, arXiv: arXiv:1511.00561. doi: 10.48550/arXiv.1511.00561.
40. S. Jégou, M. Drozdzal, D. Vazquez, A. Romero, and Y. Bengio, “The One Hundred Layers Tiramisu: Fully Convolutional DenseNets for Semantic Segmentation,” Oct. 31, 2017, arXiv: arXiv:1611.09326. doi: 10.48550/arXiv.1611.09326.10.1109/CVPRW.2017.156 ↗
41. S. Tonekaboni, S. Joshi, M.D. McCradden, and A. Goldenberg, “What Clinicians Want: Contextualizing Explainable Machine Learning for Clinical End Use,” Aug. 07, 2019, arXiv: arXiv:1905.05134. doi: 10.48550/arXiv.1905.05134.
42. R. Caruana, Y. Lou, J. Gehrke, P. Koch, M. Sturm, and N. Elhadad, “Intelligible Models for HealthCare: Predicting Pneumonia Risk and Hospital 30-day Readmission,” in Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, in KDD ’15. New York, NY, USA: Association for Computing Machinery, 2015, pp. 1721–1730. doi: 10.1145/2783258.2788613.10.1145/2783258.2788613 ↗
43. From local explanations to global understanding with explainable AI for trees10.1038/s42256-019-0138-9 ↗
44. Axiomatic attribution for deep networks
45. Grad-CAM: visual explanations from deep networks via gradient-based localization10.1007/s11263-019-01228-7 ↗
46. Explainable AI in medical imaging: an overview for clinical practitioners – saliency-based XAI approaches
47. Visualizing and understanding convolutional networks
48. Why should i trust you?’ Explaining the predictions of any classifier10.1145/2939672.2939778 ↗
49. S. Lundberg and S.-I. Lee, “A Unified Approach to Interpreting Model Predictions,” Nov. 25, 2017, arXiv: arXiv:1705.07874. doi: 10.48550/arXiv.1705.07874.
50. V. Petsiuk, “RISE: Randomized Input Sampling for Explanation of Black-box Models”.
51. R.C. Fong and A. Vedaldi, “Interpretable Explanations of Black Boxes by Meaningful Perturbation,” in Proceedings of the IEEE International Conference on Computer Vision (ICCV), Oct. 2017.10.1109/ICCV.2017.371 ↗
52. K. Simonyan, A. Vedaldi, and A. Zisserman, “Deep inside convolutional networks: Visualising image classification models and saliency maps,” 2nd International Conference on Learning Representations, ICLR 2014 - Workshop Track Proceedings, pp. 1–8, 2014.
53. On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation10.1371/journal.pone.0130140 ↗
54. D. Smilkov, N. Thorat, B. Kim, F. Viégas, and M. Wattenberg, “SmoothGrad: removing noise by adding noise,” 2017, [Online]. Available: 〈http://arxiv.org/abs/1706.03825〉.
55. A. Shrikumar, P. Greenside, and A. Kundaje, “Learning Important Features Through Propagating Activation Differences,” Oct. 12, 2019, arXiv: arXiv:1704.02685. doi: 10.48550/arXiv.1704.02685.
56. S. Stalder, N. Perraudin, R. Achanta, F. Perez-Cruz, and M. Volpi, “What You See is What You Classify: Black Box Attributions,” Oct. 07, 2022, arXiv: arXiv:2205.11266. doi: 10.48550/arXiv.2205.11266.
57. F. Özgenel and A.Gönenç Sorguç, “Performance comparison of pretrained convolutional neural networks on crack detection in buildings,” ISARC 2018 - 35th International Symposium on Automation and Robotics in Construction and International AEC/FM Hackathon: The Future of Building Things, 2018, doi: 10.22260/isarc2018/0094.10.22260/ISARC2018/0094 ↗
58. On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation10.1371/journal.pone.0130140 ↗
59. Visualizing and understanding convolutional networks
60. From classification to segmentation with explainable AI: a study on crack detection and growth monitoring10.1016/j.autcon.2024.105497 ↗
61. M. Raghu, B. Poole, J. Kleinberg, S. Ganguli, J. Sohl-Dickstein, On the Expressive Power of Deep Neural Networks,” in Proceedings of the 34th International Conference on Machine Learning, D. Precup and Y.W. Teh, Eds., in Proceedings of Machine Learning Research, vol. 70. PMLR, Aug. 2017, pp. 2847–2854. [Online]. Available: https://proceedings.mlr.press/v70/raghu17a.html.
62. B. Xu, N. Wang, T. Chen, and M. Li, “Empirical Evaluation of Rectified Activations in Convolutional Network,” Nov. 27, 2015, arXiv: arXiv:1505.00853. doi: 10.48550/arXiv.1505.00853.
63. Z.C. Lipton, “The Mythos of Model Interpretability,” Mar. 06, 2017, arXiv: arXiv:1606.03490. doi: 10.48550/arXiv.1606.03490.