An Intelligent Detection and Classification Model Based on Computer Vision for Pavement Cracks in Complicated Scenarios
Source abstract
Abstract
With the extension of road service life, cracks are the most significant type of pavement distress. To monitor road conditions and avoid excessive damage, pavement crack detection is absolutely necessary and an indispensable part of road periodic maintenance and performance assessment. The development and application of computer vision have provided modern methods for crack detection, which are low in cost, less labor-intensive, continuous, and timely. In this paper, an intelligent model based on a target detection algorithm in computer vision was proposed to accurately detect and classify four classes of cracks. Firstly, by vehicle-mounted camera capture, a dataset of pavement cracks with complicated backgrounds that are the most similar to actual scenarios was built, containing 4007 images and 7882 crack samples. Secondly, the YOLOv5 framework was improved from the four aspects of the detection layer, anchor box, neck structure, and cross-layer connection, and thereby the network’s feature extraction capability and small-sized-target detection performance were enhanced. Finally, the experimental results indicated that the proposed model attained an AP of the four classes of 81.75%, 83.81%, 98.20%, and 92.83%, respectively, and a mAP of 89.15%. In addition, the proposed model achieved a 2.20% missed detection rate, representing a 6.75% decrease over the original YOLOv5. These results demonstrated the effectiveness and practicality of our proposed model in addressing the issues of low accuracy and missed detection for small targets in the original network. Overall, the implementation of computer vision-based models in crack detection can promote the intellectualization of road maintenance.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Infrastructure Maintenance and Monitoring · Asphalt Pavement Performance Evaluation · Industrial Vision Systems and Defect Detection
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Emergency Science Research Institute, China Coal Research Institute CCRI, Beijing 100013, China
Funding
Innovation and Entrepreneurship Science and Technology Project of Chinese Institute of Coal Science
References (62)
Show source-provided references
1. A review of research on asphalt pavement design at home and abroad
2. Research on Reliability Evaluation of Asphalt Pavement
3. Causes of Cracks and Deterioration of Pavement on Highways in Jordan from Contractors’ Perspective
4. Influence of temperature on the cracking behavior of asphalt base courses with structural weaknesses10.1016/j.ijtst.2018.04.002 ↗
5. Miller, J., and Bellinger, W. (2014). Distress Identification Manual for the Long-Term Pavement Performance Program.
6. (2012, January 01). Pavement Distresses. Available online: https://pavementinteractive.org/reference-desk/pavement-management/pavement-distresses/.
7. Enhancing sustainability for pavement maintenance decision-making through image processing-based distress detection10.1007/s41062-024-01370-3 ↗
8. Molenaar, A. (1983). Structural Performance and Design of Flexible Road Constructions and Asphalt Concrete Overlays. [Ph.D. Thesis, Delft University of Technology].
9. Improvement of road pavement maintenance models and technologies10.3846/bjrbe.2016.28 ↗
10. Overview of the development of foreign road automatic detection systems
11. Gao, F. (2009). Research and Implementation of Road Comprehensive Information Collection System. [Master’s Thesis, Changan University].
12. Qureshi, W., Hassan, S., McKeever, S., Power, D., Mulry, B., Feighan, K., and O’Sullivan, D. (2022). An Exploration of Recent Intelligent Image Analysis Techniques for Visual Pavement Surface Condition Assessment. Sensors, 22.10.3390/s22229019 ↗
13. Advanced pavement distress recognition and 3D reconstruction by using GA-DenseNet and binocular stereo vision10.1016/j.measurement.2022.111760 ↗
14. The State-of-the-Art Review on Applications of Intrusive Sensing, Image Processing Techniques, and Machine Learning Methods in Pavement Monitoring and Analysis10.1016/j.eng.2020.07.030 ↗
15. Feldman, D., Pyle, T., and Lee, J. (2015). Automated Pavement Condition Survey Manual, California Department of Transportation.
16. Majidifard, H., Jin, P., Adu-Gyamfi, Y., and Buttlar, W. (2020, January 12–16). Pavement Image Datasets: A New Benchmark Dataset to Classify and Densify Pavement Distresses. Proceedings of the TRB 99th Annual Meeting, Washington, DC, USA.10.1177/0361198120907283 ↗
17. Klette, R. (2014). Concise Computer Vision, Springer.10.1007/978-1-4471-6320-6 ↗
18. Morris, T. (2004). Computer Vision and Image Processing, Palgrave Macmillan Ltd.
19. O’Mahony, N., Campbell, S., Carvalho, A., Harapanahalli, S., Hernandez, G., Krpalkova, L., Riordan, D., and Walsh, J. (2020). Advances in Computer Vision, Proceedings of the 2019 Computer Vision Conference (CVC), Las Vegas, CA, USA, 2–3 May 2019, Springer International Publishing.
20. Chen, L., Li, S., Bai, Q., Yang, J., Jiang, S., and Miao, Y. (2021). Review of Image Classification Algorithms Based on Convolutional Neural Networks. Remote Sens., 13.10.3390/rs13224712 ↗
21. Deep learning in computer vision: A critical review of emerging techniques and application scenarios
22. Deep Learning: A Comprehensive Overview on Techniques, Taxonomy, Applications and Research Directions10.1007/s42979-021-00815-1 ↗
23. Valente, J., António, J., Mora, C., and Jardim, S. (2023). Developments in Image Processing Using Deep Learning and Reinforcement Learning. J. Imaging, 9.10.3390/jimaging9100207 ↗
24. Nasser, M., and Yusof, U. (2023). Deep Learning Based Methods for Breast Cancer Diagnosis: A Systematic Review and Future Direction. Diagnostics, 13.10.3390/diagnostics13010161 ↗
25. Systematic review of data-centric approaches in artificial intelligence and machine learning10.1016/j.dsm.2023.06.001 ↗
26. Zhang, L., Yang, F., Zhang, Y., and Zhu, Y. (2016, January 25–28). Road crack detection using deep convolutional neural network. Proceedings of the IEEE International Conference on Image Processing, Phoenix, AZ, USA.10.1109/ICIP.2016.7533052 ↗
27. Yusof, N., Osman, K., Noor, M., Ibrahim, A., Tahir, N., and Yusof, N. (2018, January 23–25). Crack Detection and Classification in Asphalt Pavement Images using Deep Convolution Neural Network. Proceedings of the 2018 8th IEEE International Conference on Control System, Computing and Engineering (ICCSCE), Penang, Malaysia.10.1109/ICCSCE.2018.8685007 ↗
28. Automated Pixel-Level Pavement Crack Detection on 3D Asphalt Surfaces Using a Deep-Learning Network10.1111/mice.12297 ↗
29. Mandal, V., Uong, L., and Adu-Gyamfi, Y. (2018, January 10–13). Automated Road Crack Detection Using Deep Convolutional Neural Networks. Proceedings of the 2018 IEEE International Conference on Big Data (Big Data), Seattle, WA, USA.10.1109/BigData.2018.8622327 ↗
30. Nie, M., and Wang, C. (2019, January 28–30). Pavement Crack Detection based on yolov3. Proceedings of the 2019 2nd International Conference on Safety Produce Informatization (IICSPI), Chongqing, China.10.1109/IICSPI48186.2019.9095956 ↗
31. Cao, W., Zou, Y., Luo, M., Zhang, P., Wang, W., and Huang, W. (2019, January 26–28). Deep Discriminant Learning-based Asphalt Road Cracks Detection via Wireless Camera Network. Proceedings of the 2019 Computing, Communications and IoT Applications (ComComAp), Shenzhen, China.10.1109/ComComAp46287.2019.9018831 ↗
32. Road Damage Detection and Classification Using Deep Neural Networks with Smartphone Images10.1111/mice.12387 ↗
33. Generative adversarial network for road damage detection10.1111/mice.12561 ↗
34. Ruseruka, C., Mwakalonge, J., Comert, G., Siuhi, S., Ngeni, F., and Major, K. (2023). Pavement Distress Identification Based on Computer Vision and Controller Area Network (CAN) Sensor Models. Sustainability, 15.10.3390/su15086438 ↗
35. Wang, Y. (2021). Intelligent Detection System of Pavement Crack Based on Deep Learning. [Master’s Thesis, University of Science and Technology Beijing].
36. Sun, C., Shrivastava, A., Singh, S., and Gupta, A. (2017, January 22–29). Revisiting Unreasonable Effectiveness of Data in Deep Learning Era. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.10.1109/ICCV.2017.97 ↗
37. Temperature distribution and method-experience prediction model of asphalt pavement
38. Prediction model of temperature in different layers of asphalt pavement
39. Gallaway, B. (1972). Fatigue of Compacted Bituminous Aggregate Mixtures, ASTM International.10.1520/STP508-EB ↗
40. Tang, G., Ni, J., Zhao, Y., Gu, Y., and Cao, W. (2024). A Survey of Object Detection for UAVs Based on Deep Learning. Remote Sens., 16.10.3390/rs16010149 ↗
41. Xu, H., Su, X., Wang, Y., Cai, H., Cui, K., and Chen, X. (2019). Automatic Bridge Crack Detection Using a Convolutional Neural Network. Appl. Sci., 9.10.3390/app9142867 ↗
42. Research on detection algorithm for bridge cracks based on deep learning
43. Cui, L. (2017, May 11). CrackForest Dataset. Available online: https://github.com/cuilimeng/CrackForest-dataset.
44. Redmon, J., Divvala, S., Girshick, R., and Farhadi, A. (2016, January 27–30). You Only Look Once: Unified, Real-Time Object Detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.10.1109/CVPR.2016.91 ↗
45. Girshick, R., Donahue, J., Darrell, T., and Malik, J. (2014, January 23–28). Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Columbus, OH, USA.10.1109/CVPR.2014.81 ↗
46. Faster R-CNN: Towards real-time object detection with region proposal networks
47. Ultralytics (2020, June 26). yolov5. Available online: https://github.com/ultralytics/yolov5/.
48. Redmon, J., and Farhadi, A. (2018, January 18–22). YOLOv3: An Incremental Improvement. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.
49. YOLOv4: Optimal Speed and Accuracy of Object Detection
50. A Survey of Modern Deep Learning based Object Detection Models10.1016/j.dsp.2022.103514 ↗
51. Ren, Z., Zhang, H., and Li, Z. (2023). Improved YOLOv5 Network for Real-Time Object Detection in Vehicle-Mounted Camera Capture Scenarios. Sensors, 23.10.3390/s23104589 ↗
52. Tan, M., Pang, R., and Le, Q. (2020, January 13–19). Efficientdet: Scalable and efficient object detection. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.10.1109/CVPR42600.2020.01079 ↗
53. Real time approach for underground objects detection from vehicle-borne ground penetrating radar
54. Road pothole extraction and safety evaluation by integration of point cloud and images derived from mobile mapping sensor10.1016/j.aei.2019.100936 ↗
55. Tan, Y., and Li, Y. (2019). UAV Photogrammetry-Based 3D Road Distress Detection. ISPRS Int. J. Geo-Inf., 8.10.3390/ijgi8090409 ↗
56. Road Structural Defects Detection and Digitalization Based on 3D Ground Penetrating Radar Technology: A State-of-the-art Review
57. Li, Y., Sun, S., Song, W., Zhang, J., and Teng, Q. (2024). CrackYOLO: Rural Pavement Distress Detection Model with Complex Scenarios. Electronics, 13.10.3390/electronics13020312 ↗
58. Xu, X., Zhao, M., Shi, P., Ren, R., He, X., Wei, X., and Yang, H. (2022). Crack Detection and Comparison Study Based on Faster R-CNN and Mask R-CNN. Sensors, 22.10.3390/s22031215 ↗
59. Research on Asphalt Pavement Disease Detection Based on Improved YOLOv5s10.1155/2023/2069044 ↗
60. Yu, G., and Zhou, X. (2023). An Improved YOLOv5 Crack Detection Method Combined with a Bottleneck Transformer. Mathematics, 11.10.3390/math11102377 ↗
61. Ren, J., Zhao, G., Ma, Y., Zhao, D., Liu, T., and Yan, J. (2022). Automatic Pavement Crack Detection Fusing Attention Mechanism. Electronics, 11.10.3390/electronics11213622 ↗
62. A review of top-down cracking in asphalt pavements: Causes, models, experimental tools and future challenges