Architecture for pavement pothole evaluation using deep learning, machine vision, and fuzzy logic
Source abstract
Abstract
Roads are critical for economic growth and trade but are constantly degraded by heavy traffic and adverse weather, leading to potholes that compromise safety. Traditional detection methods, like manual inspections, are labor-intensive, costly, and prone to errors. Existing automated systems also struggle with false positives, particularly in challenging conditions involving shadows, stains, or other environmental interferences. This research presents an architecture for detecting, measuring, and evaluating potholes, as well as generating maintenance recommendations. We integrate 2D images and 3D point clouds captured using the Intel RealSense D435i camera, generating a dataset of 583 images—299 containing potholes and 234 depicting environmental noise on various pavements. Each image is labeled through semantic segmentation and paired with corresponding point clouds. The architecture utilizes transfer learning with a Segformer network, achieving high detection performance with a Recall of 90.87 %, Precision of 90.01 %, Accuracy of 86.8 % F1 Score of 90.433 %, and a loss of 0.0431. The method achieves an IOU of 85.872 %, ensuring accurate diameter estimation, which contrasts with studies using lower IOU values where pothole dimensions are often underestimated due to incomplete detection. Our architecture provides reliable contour detection, facilitating the integration of image data and point clouds to estimate pothole dimensions with a depth estimation error of 5.94 mm. A fuzzy logic system processes these measurements to assess repair urgency and recommend appropriate repair techniques. • A new dataset of 583 semantic images with their point clouds of potholes and noisy environments from rigid/flexible pavement. • Transfer learning for pothole segmentation with a Segformer achieving 90.01% precision, 86.8% accuracy, and 85.872% mean IoU. • Architecture that quantifies potholes via data integration reducing processing load, achieving 5.94 mm depth error. • A fuzzy logic system to assess the pothole damages, determine repair urgency, and recommend maintenance techniques. • To automate pothole severity inspection in urban streets to help recommend maintenance techniques for each detected pothole.
Source: openalex_inverted_index · Quality: baseline
Research context
OpenAlex topics: Infrastructure Maintenance and Monitoring · Asphalt Pavement Performance Evaluation · Water Systems and Optimization
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
References (39)
Show source-provided references
1. Transportation Officials (1993). AASHTO Guide for Design of Pavement Structures, 1993, (Vol. 1). Aashto.
2. Deep learning-based structural health monitoring10.1016/j.autcon.2024.105328 ↗
3. Image-based pothole detection system for ITS service and road management system10.1155/2015/968361 ↗
4. Pothole detection on roads using canny edge detection algorithm
5. A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A.N. Gómez, … and I. Polosukhin (2017)Attention is all you need.Advances in neural information processing systems, 30..
6. Deep learning-based crack damage detection using convolutional neural networks10.1111/mice.12263 ↗
7. Autonomous structural visual inspection using region-based deep learning for detecting multiple damage types10.1111/mice.12334 ↗
8. F.A.L.N. Mouzinho, and H. Fukai (2021). Hierarchical semantic segmentation based approach for road surface damages and markings detection on paved road.In: 2021 8th International Conference on Advanced Informatics: Concepts, Theory and Applications (ICAICTA) 1-5.IEEE..10.1109/ICAICTA53211.2021.9640296 ↗
9. We learn better road pothole detection: from attention aggregation to adversarial domain adaptation
10. V. Pereira, S. Tamura, S. Hayamizu, and H. Fukai (2019). Semantic segmentation of paved road and pothole image using u-net architecture.In: 2019 International Conference of Advanced Informatics: Concepts, Theory and Applications (ICAICTA))1-4.IEEE..10.1109/ICAICTA.2019.8904105 ↗
11. Real time pothole detection using android smartphones with accelerometers
12. Application of vehicle-based sensing technology in monitoring vibration response of pavement conditions10.1061/JPEODX.0000205 ↗
13. R. Ali, Q. Bin-Saeed, O. Buyukozturk, S. Lee, and Y. Cha (2024). Monocular Computer Vision-Based Simultaneous Pothole Segmentation and 3d Volume Prediction Using 3dpredictnet.Available at SSRN 5045587..10.2139/ssrn.5045587 ↗
14. Applications of LiDAR in agriculture and future research directions10.3390/jimaging9030057 ↗
15. Terrestrial laser scanning in forest inventories10.1016/j.isprsjprs.2016.01.006 ↗
16. Dsec: a stereo event camera dataset for driving scenarios10.1109/LRA.2021.3068942 ↗
17. An automatic pothole detection algorithm using pavement 3D data10.1080/10298436.2022.2057978 ↗
18. Road pothole extraction and safety evaluation by integration of point cloud and images derived from mobile mapping sensors
19. Perceptual depth quality in distorted stereoscopic images10.1109/TIP.2016.2642791 ↗
20. Assessing the performance of a structured light scanner. International archives of photogrammetry, remote sensing and spatial
21. Review of recent automated pothole-detection methods10.3390/app12115320 ↗
22. YOLO-LRDD: A lightweight method for road damage detection based on improved YOLOv5s10.1186/s13634-022-00931-x ↗
23. YOLOv5s-M: a deep learning network model for road pavement damage detection from urban street-view imagery
24. Deep learning-based automatic volumetric damage quantification using depth camera10.1016/j.autcon.2018.12.006 ↗
25. SegFormer: simple and efficient design for semantic segmentation with transformers
26. 2023Libro: CSV. CONSERVACIÓN. Capítulo: 008. Determinación de los Deterioros Superficiales de los Pavimentos (DET). (M ⋅ CSV ⋅ CAR ⋅ 1 ⋅ 03 ⋅ 008/23).Secretaría de Infraestructura, Comunicaciones y Transporte. (.
27. Pothole 3D reconstruction with a novel imaging system and structure from motion techniques10.1109/TITS.2021.3054026 ↗
28. Speed bump and pothole detection using deep neural network with images captured through zed camera10.3390/app13148349 ↗
29. I. Katsamenis, A. Sakelliou, N. Bakalos, E. Protopapadakis, C. Klaridopoulos, N. Frangakis, … and D. Kalogeras (2023). Deep transformer networks for precise pothole segmentation tasks.In: Proceedings of the 16th International Conference on PErvasive Technologies Related to Assistive Environments 596-602..10.1145/3594806.3596560 ↗
30. Mobile laser scanning data for the evaluation of pavement surface distress10.3390/rs12060942 ↗
31. Detection of road potholes by applying convolutional neural network method based on road vibration data10.3390/s23229023 ↗
32. An automated machine-learning approach for road pothole detection using smartphone sensor data10.3390/s20195564 ↗
33. Bhatt, U., Mani, S., Xi, E., and Kolter, J.Z. (2017). Intelligent pothole detection and road condition assessment.arXiv preprint arXiv:1710.02595.
34. Assessing severity of road cracks using deep learning-based segmentation and detection10.1007/s11227-022-04560-x ↗
35. J.S. Miller, and W.Y. Bellinger, (2003). Distress identification manual for the long-term pavement performance program (No. FHWA-RD-03-031). United States Federal Highway Administration. Office of Infrastructure Research and Development..
36. B. Dwyer, J. Nelson, (2022), J. Solawetz, , et al. Roboflow (Version 1.0) [Software] Available from 〈https://roboflow.com〉.computer vision..
37. Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography10.1145/358669.358692 ↗
38. Tackling the problem of class imbalance in multi-class sentiment classification: an experimental study10.2478/fcds-2019-0009 ↗
39. Under-sampling class imbalanced datasets by combining clustering analysis and instance selection10.1016/j.ins.2018.10.029 ↗