Comparison of CNN-Based Models for Pothole Detection in Real-World Adverse Conditions: Overview and Evaluation
Source abstract
Abstract
Potholes pose a significant problem for road safety and infrastructure. They can cause damage to vehicles and present a risk to pedestrians and cyclists. The ability to detect potholes in real time and with a high level of accuracy, especially under different lighting conditions, is crucial for the safety of road transport participants and the timely repair of these hazards. With the increasing availability of cameras on vehicles and smartphones, there is a growing interest in using computer vision techniques for this task. Convolutional neural networks (CNNs) have shown great potential for object detection tasks, including pothole detection. This study provides an overview of computer vision algorithms used for pothole detection. Experimental results are then used to evaluate the performance of the latest CNN-based models for pothole detection in different real-world road conditions, including rain, sunset, evening, and night, as well as clean conditions. The models evaluated in this study include both conventional and the newest architectures from the region-based CNN (R-CNN) and You Only Look Once (YOLO) families. The YOLO models demonstrated a faster detection response and higher accuracy in detecting potholes under clear, rain, sunset, and evening conditions. R-CNN models, on the other hand, performed better in the worse-visibility conditions at night. This study provides valuable insights into the performance of different CNN models for pothole detection in real road conditions and may assist in the selection of the most appropriate model for a specific application.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Infrastructure Maintenance and Monitoring · Advanced Neural Network Applications · Geophysical Methods and Applications
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
University Science Park UNIZA, University of Žilina, Univerzitná 8215/1, 010 26 Žilina, Slovakia
Funding
European Regional Development Fund
References (66)
Show source-provided references
1. Novotny, P., and Janosikova, M. (2020). Designating Regional Elements System in a Critical Infrastructure System in the Context of the Czech Republic. Systems, 8.10.3390/systems8020013 ↗
2. Resilience of the Smart Transport System—Risks and Aims10.1016/j.trpro.2021.07.153 ↗
3. (2023, May 02). American Automobile Association; Pothole Damage Costs U.S. Drivers $3 Billion Annually. Available online: https://info.oregon.aaa.com/pothole-damage-costs-u-s-drivers-3-billion-annually/.
4. Drliciak, M., Celko, J., and Cingel, M. (2019, January 9–13). The Economically Active People in the Transport Process. Proceedings of the IOP Conference Series: Materials Science and Engineering, Žilina, Slovakia.10.1088/1757-899X/661/1/012149 ↗
5. Siew, E.F., Ireland-Hay, T., Stephens, G.T., Chen, J.J.J., and Taylor, M.P. (2005, January 13–17). A Study of the Fundamentals of Pothole Formation. Proceedings of the Light Metals 2005: Proceedings of the Technical Sessions Presented by the TMS Aluminum Committee at the 134th TMS Annual Meeting, San Francisco, CA, USA.
6. Morning Modal Split Model of Economically Active People in Zilina Region10.1016/j.trpro.2021.07.077 ↗
7. Heo, D.-H., Choi, J.-Y., Kim, S.-B., Tak, T.-O., and Zhang, S.-P. (2023). Image-Based Pothole Detection Using Multi-Scale Feature Network and Risk Assessment. Electronics, 12.10.3390/electronics12040826 ↗
8. Dynamic Architecture for Analytical Its Services
9. Pena-Caballero, C., Kim, D., Gonzalez, A., Castellanos, O., Cantu, A., and Ho, J. (2020). Real-Time Road Hazard Information System. Infrastructures, 5.10.3390/infrastructures5090075 ↗
10. Ochoa-Ruiz, G., Angulo-Murillo, A.A., Ochoa-Zezzatti, A., Aguilar-Lobo, L.M., Vega-Fernández, J.A., and Natraj, S. (2020). An Asphalt Damage Dataset and Detection System Based on RetinaNet for Road Conditions Assessment. Appl. Sci., 10.10.3390/app10113974 ↗
11. Ahmed, K.R. (2021). Smart Pothole Detection Using Deep Learning Based on Dilated Convolution. Sensors, 21.10.3390/s21248406 ↗
12. Al-Shaghouri, A., Alkhatib, R., and Berjaoui, S. (2021). Real-Time Pothole Detection Using Deep Learning. arXiv.
13. Yu, X., and Marinov, M. (2020). A Study on Recent Developments and Issues with Obstacle Detection Systems for Automated Vehicles. Sustainability, 12.10.3390/su12083281 ↗
14. A Real-Time Pothole Detection Approach for Intelligent Transportation System
15. Vehicle Vibration Signal Processing for Road Surface Monitoring10.1109/JSEN.2017.2719865 ↗
16. Wu, C., Wang, Z., Hu, S., Lepine, J., Na, X., Ainalis, D., and Stettler, M. (2020). An Automated Machine-Learning Approach for Road Pothole Detection Using Smartphone Sensor Data. Sensors, 20.10.3390/s20195564 ↗
17. Chang, K.T., Chang, J.R., and Liu, J.K. (2005, January 12–15). Detection of Pavement Distresses Using 3D Laser Scanning Technology. Proceedings of the International Conference on Computing in Civil Engineering 2005, Cancun, Mexico.10.1061/40794(179)103 ↗
18. Yu, X., and Salari, E. (2011, January 15–17). Pavement Pothole Detection and Severity Measurement Using Laser Imaging. Proceedings of the IEEE International Conference Electro/Information Technology, Mankato, MN, USA.10.1109/EIT.2011.5978573 ↗
19. Stereo Vision Method Application to Road Inspection10.3846/bjrbe.2017.05 ↗
20. Hou, Z., Wang, K.C.P., and Gong, W. (2007, January 22–24). Experimentation of 3D Pavement Imaging through Stereovision. Proceedings of the International Conference on Transportation Engineering 2007, Chengdu, China.10.1061/40932(246)62 ↗
21. Zhang, Z., Ai, X., Chan, C.K., and Dahnoun, N. (2014, January 4–9). An Efficient Algorithm for Pothole Detection Using Stereo Vision. Proceedings of the 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Florence, Italy.10.1109/ICASSP.2014.6853659 ↗
22. Girshick, R., Donahue, J., Darrell, T., and Malik, J. (2014, January 23–28). Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA.10.1109/CVPR.2014.81 ↗
23. Girshick, R. (2015, January 7–13). Fast R-CNN. Proceedings of the IEEE International Conference on Computer Vision, Santiago, Chile.10.1109/ICCV.2015.169 ↗
24. Redmon, J., Divvala, S., Girshick, R., and Farhadi, A. (2016, January 26–27). You Only Look Once: Unified, Real-Time Object Detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.10.1109/CVPR.2016.91 ↗
25. Salaudeen, H., and Çelebi, E. (2022). Pothole Detection Using Image Enhancement GAN and Object Detection Network. Electronics, 11.10.3390/electronics11121882 ↗
26. Leibe, B., Matas, J., Sebe, N., and Welling, M. (2016, January 11–14). SSD: Single Shot MultiBox Detector. Proceedings of the Computer Vision—ECCV 2016, Amsterdam, The Netherlands.10.1007/978-3-319-46454-1 ↗
27. Convolutional Neural Network for Pothole Detection in Asphalt Pavement10.1080/14680629.2019.1615533 ↗
28. Computer Vision for Road Imaging and Pothole Detection: A State-of-the-Art Review of Systems and Algorithms10.1093/tse/tdac026 ↗
29. Pothole Detection Using Location-Aware Convolutional Neural Networks10.1007/s13042-020-01078-7 ↗
30. Park, S.-S., Tran, V.-T., and Lee, D.-E. (2021). Application of Various YOLO Models for Computer Vision-Based Real-Time Pothole Detection. Appl. Sci., 11.10.3390/app112311229 ↗
31. Road Damage Detection and Classification Using Deep Neural Networks with Smartphone Images10.1111/mice.12387 ↗
32. Lin, Y.-C., Chen, W.-H., and Kuo, C.-H. (2021). Implementation of Pavement Defect Detection System on Edge Computing Platform. Appl. Sci., 11.10.3390/app11083725 ↗
33. Ramesh, A., Nikam, D., Balachandran, V.N., Guo, L., Wang, R., Hu, L., Comert, G., and Jia, Y. (2022). Cloud-Based Collaborative Road-Damage Monitoring with Deep Learning and Smartphones. Sustainability, 14.10.3390/su14148682 ↗
34. Salcedo, E., Jaber, M., and Requena Carrión, J. (2022). A Novel Road Maintenance Prioritisation System Based on Computer Vision and Crowdsourced Reporting. J. Sens. Actuator Netw., 11.10.3390/jsan11010015 ↗
35. Ren, S., He, K., Girshick, R., and Sun, J. (2015, January 7–12). Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks. Proceedings of the Advances in Neural Information Processing Systems, Montreal, QC, Canada.
36. He, K., Gkioxari, G., Dollar, P., and Girshick, R. (2017, January 22–29). Mask R-CNN. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.10.1109/ICCV.2017.322 ↗
37. Cascade R-CNN: High Quality Object Detection and Instance Segmentation10.1109/TPAMI.2019.2956516 ↗
38. Sun, P., Zhang, R., Jiang, Y., Kong, T., Xu, C., Zhan, W., Tomizuka, M., Li, L., Yuan, Z., and Wang, C. (2021, January 20–25). Sparse R-CNN: End-to-End Object Detection With Learnable Proposals. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA.10.1109/CVPR46437.2021.01422 ↗
39. Redmon, J., and Farhadi, A. (2018). YOLOv3: An Incremental Improvement. arXiv.
40. Bochkovskiy, A., Wang, C.-Y., and Liao, H.-Y.M. (2020). YOLOv4: Optimal Speed and Accuracy of Object Detection. arXiv.
41. Glenn, J. (2023, May 02). Ultralytics|Revolutionizing the World of Vision AI. Available online: https://ultralytics.com.
42. Li, C., Li, L., Jiang, H., Weng, K., Geng, Y., Li, L., Ke, Z., Li, Q., Cheng, M., and Nie, W. (2022). YOLOv6: A Single-Stage Object Detection Framework for Industrial Applications. arXiv.
43. Wang, C.-Y., Bochkovskiy, A., and Liao, H.-Y.M. (2022). YOLOv7: Trainable Bag-of-Freebies Sets New State-of-the-Art for Real-Time Object Detectors. arXiv.
44. Fleet, D., Pajdla, T., Schiele, B., and Tuytelaars, T. (2014, January 6–12). Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition. Proceedings of the Computer Vision—ECCV 2014, Zurich, Switzerland.10.1007/978-3-319-10578-9 ↗
45. Liu, S., Qi, L., Qin, H., Shi, J., and Jia, J. (2018, January 18–23). Path Aggregation Network for Instance Segmentation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00913 ↗
46. (2018). MMDetection Contributors OpenMMLab Detection Toolbox and Benchmark. arXiv.
47. (2023, May 02). Dataset: Pothole Detection Using Computer Vision in Challenging Conditions. Available online: https://figshare.com/articles/figure/Potholes_dataset_with_YOLO_annotations/21214400/3.
48. The Pascal Visual Object Classes (Voc) Challenge10.1007/s11263-009-0275-4 ↗
49. Adrien, P., Guyon, I., Letournel, A.C., Baró, X., Escalante, H., Escalera, S., Thomas, T., and Xu, Z. (2023, May 02). CodaLab Competitions: An Open Source Platform to Organize Scientific Challenges. Diss. Université Paris-Saclay, FRA. Available online: https://cnrs.hal.science/hal-03629462/.
50. Tao, L., Zhu, C., Xiang, G., Li, Y., Jia, H., and Xie, X. (2017, January 10–13). LLCNN: A Convolutional Neural Network for Low-Light Image Enhancement. Proceedings of the 2017 IEEE Visual Communications and Image Processing (VCIP), St. Petersburg, FL, USA.10.1109/VCIP.2017.8305143 ↗
51. Shen, L., Yue, Z., Feng, F., Chen, Q., Liu, S., and Ma, J. (2017). MSR-Net: Low-Light Image Enhancement Using Deep Convolutional Network. arXiv.
52. Fan, C.-M., Liu, T.-J., and Liu, K.-H. (2022, January 16–19). Half Wavelet Attention on M-Net+ for Low-Light Image Enhancement. Proceedings of the 2022 IEEE International Conference on Image Processing (ICIP), Bordeaux, France.10.1109/ICIP46576.2022.9897503 ↗
53. Attention Guided Low-Light Image Enhancement with a Large Scale Low-Light Simulation Dataset10.1007/s11263-021-01466-8 ↗
54. Selective Wavelet Attention Learning for Single Image Deraining10.1007/s11263-020-01421-z ↗
55. Context-Enhanced Representation Learning for Single Image Deraining10.1007/s11263-020-01425-9 ↗
56. Dong, H., Pan, J., Xiang, L., Hu, Z., Zhang, X., Wang, F., and Yang, M.-H. (2020, January 13–19). Multi-Scale Boosted Dehazing Network With Dense Feature Fusion. Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.10.1109/CVPR42600.2020.00223 ↗
57. Liu, X., Ma, Y., Shi, Z., and Chen, J. (November, January 27). GridDehazeNet: Attention-Based Multi-Scale Network for Image Dehazing. Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision (ICCV), Seoul, Republic of Korea.10.1109/ICCV.2019.00741 ↗
58. Benchmarking Low-Light Image Enhancement and Beyond10.1007/s11263-020-01418-8 ↗
59. Image-Adaptive YOLO for Object Detection in Adverse Weather Conditions
60. Chu, J., Zhang, C., Yan, M., Zhang, H., and Ge, T. (2023). TRD-YOLO: A Real-Time, High-Performance Small Traffic Sign Detection Algorithm. Sensors, 23.10.3390/s23083871 ↗
61. Yao, J., Fan, X., Li, B., and Qin, W. (2022). Adverse Weather Target Detection Algorithm Based on Adaptive Color Levels and Improved YOLOv5. Sensors, 22.10.3390/s22218577 ↗
62. Fu, H., Song, G., and Wang, Y. (2021). Improved YOLOv4 Marine Target Detection Combined with CBAM. Symmetry, 13.10.3390/sym13040623 ↗
63. CEAM-YOLOv7: Improved YOLOv7 Based on Channel Expansion and Attention Mechanism for Driver Distraction Behavior Detection10.1109/ACCESS.2022.3228331 ↗
64. Rain Rendering for Evaluating and Improving Robustness to Bad Weather10.1007/s11263-020-01366-3 ↗
65. Xin, H., Ye, Y., Na, X., Hu, H., Wang, G., Wu, C., and Hu, S. (2023). Sustainable Road Pothole Detection: A Crowdsourcing Based Multi-Sensors Fusion Approach. Sustainability, 15.10.3390/su15086610 ↗
66. Chaturvedi, S.S., Zhang, L., and Yuan, X. (2022, January 21–25). Pay “Attention” to Adverse Weather: Weather-Aware Attention-Based Object Detection. Proceedings of the 26th International Conference on Pattern Recognition (ICPR), Montreal, QC, Canada.10.1109/ICPR56361.2022.9956149 ↗