Ensemble of Deep Convolutional Neural Networks for Automatic Pavement Crack Detection and Measurement
Source abstract
Abstract
Automated pavement crack detection and measurement are important road issues. Agencies have to guarantee the improvement of road safety. Conventional crack detection and measurement algorithms can be extremely time-consuming and low efficiency. Therefore, recently, innovative algorithms have received increased attention from researchers. In this paper, we propose an ensemble of convolutional neural networks (without a pooling layer) based on probability fusion for automated pavement crack detection and measurement. Specifically, an ensemble of convolutional neural networks was employed to identify the structure of small cracks with raw images. Secondly, outputs of the individual convolutional neural network model for the ensemble were averaged to produce the final crack probability value of each pixel, which can obtain a predicted probability map. Finally, the predicted morphological features of the cracks were measured by using the skeleton extraction algorithm. To validate the proposed method, some experiments were performed on two public crack databases (CFD and AigleRN) and the results of the different state-of-the-art methods were compared. To evaluate the efficiency of crack detection methods, three parameters were considered: precision (Pr), recall (Re) and F1 score (F1). For the two public databases of pavement images, the proposed method obtained the highest values of the three evaluation parameters: for the CFD database, Pr = 0.9552, Re = 0.9521 and F1 = 0.9533 (which reach values up to 0.5175 higher than the values obtained on the same database with the other methods), for the AigleRN database, Pr = 0.9302, Re = 0.9166 and F1 = 0.9238 (which reach values up to 0.7313 higher than the values obtained on the same database with the other methods). The experimental results show that the proposed method outperforms the other methods. For crack measurement, the crack length and width can be measure based on different crack types (complex, common, thin, and intersecting cracks.). The results show that the proposed algorithm can be effectively applied for crack measurement.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Infrastructure Maintenance and Monitoring · Asphalt Pavement Performance Evaluation · Concrete Corrosion and Durability
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Key Lab of Digital Signal and Image Processing of Guangdong Province, Shan’tou 515063, China
College of Engineering, Shantou University, Shan’tou 515063, China
Department of Civil, Construction and Environmental Engineering, Sapienza University of Rome, 00184 Rome, Italy
Department of Industrial Engineering, Pusan National University, Busan 609735, Korea
Funding
Science and Technology Planning Project of Guangdong Province of China · Project of Educational Commission of Guangdong Province of China · State Key Lab of Digital Manufacturing Equipment and Technology · Project of Robot Automatic Design Platform combining Multi-Objective Evolutionary Computation and Deep Neural Network
References (47)
Show source-provided references
1. Bazzaz, M., Darabi, M.K., Little, D.N., and Garg, N. (2020). Effect of Evotherm-M1 on Properties of Asphaltic Materials Used at NAPMRC Testing Facility. J. Test. Eval., in press.10.1520/JTE20190446 ↗
2. DeepCrack: Learning hierarchical convolutional features for crack detection10.1109/TIP.2018.2878966 ↗
3. Pantuso, A., Loprencipe, G., Bonin, G., and Teltayev, B.B. (2019). Analysis of pavement condition survey data for effective implementation of a network level pavement management program for Kazakhstan. Sustainability, 11.10.3390/su11030901 ↗
4. Di Mascio, P., Loprencipe, G., and Moretti, L. (2019). Technical and economic criteria to select pavement surfaces of port handling plants. Coatings, 9.10.3390/coatings9020126 ↗
5. Loprencipe, G., and Zoccali, P. (2017). Ride quality due to road surface irregularities: Comparison of different methods applied on a set of real road profiles. Coatings, 7.10.3390/coatings7050059 ↗
6. Bonin, G., Polizzotti, S., Loprencipe, G., Folino, N., Oliviero Rossi, C., and Teltayev, B.B. (2017, January 10–12). Development of a road asset management system in Kazakhstan. Proceedings of the AIIT International Congress on Transport Infrastructure and Systems, TIS 2017, Rome, Italy.10.1201/9781315281896-70 ↗
7. Image Based Techniques for Crack Detection, Classification and Quantification in Asphalt Pavement: A Review10.1007/s11831-016-9194-z ↗
8. Loprencipe, G., and Pantuso, A. (2017). A specified procedure for distress identification and assessment for urban road surfaces based on PCI. Coatings, 7.10.3390/coatings7050065 ↗
9. Evaluation methods for improving surface geometry of concrete floors: A case study10.1016/j.csse.2015.06.002 ↗
10. Visual retrieval of concrete crack properties for automated post-earthquake structural safety evaluation10.1016/j.autcon.2011.03.004 ↗
11. Bridge inspection robot system with machine vision10.1016/j.autcon.2009.04.003 ↗
12. Oliveira, H., and Correia, P.L. (2009, January 24–28). Automatic road crack segmentation using entropy and image dynamic thresholding. Proceedings of the 17th European Signal Processing Conference, Glasgow, Scotland, UK.
13. Zhao, H., Qin, G., and Wang, X. (2010, January 16–18). Improvement of canny algorithm based on pavement edge detection. Proceedings of the 2010 3rd International Congress on Image and Signal Processing, CISP 2010, Yantai, China.10.1109/CISP.2010.5646923 ↗
14. Evaluating pavement cracks with bidimensional empirical mode decomposition
15. Tanaka, N., and Uematsu, K. (1998, January 17–19). A Crack Detection Method in Road Surface Images Using Morphology. Proceedings of the Workshop on Machine Vision Applications, Chiba, Japan.
16. Tang, J., and Gu, Y. (2013, January 13–16). Automatic crack detection and segmetnation using a hybrid algorithm for road distress analysis. Proceedings of the 2013 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2013, Manchester, UK.10.1109/SMC.2013.516 ↗
17. Subirats, P., Dumoulin, J., Legeay, V., and Barba, D. (2006, January 8–11). Automation of pavement surface crack detection using the continuous wavelet transform. Proceedings of the International Conference on Image Processing, ICIP 2006, Atlanta, GA, USA.10.1109/ICIP.2006.313007 ↗
18. Detecting curves with unknown endpoints and arbitrary topology using minimal paths10.1109/TPAMI.2011.267 ↗
19. Nguyen, T.S., Begot, S., Duculty, F., and Avila, M. (2011, January 11–14). Free-form anisotropy: A new method for crack detection on pavement surface images. Proceedings of the International Conference on Image Processing, ICIP 2011, Brussels, Belgium.10.1109/ICIP.2011.6115610 ↗
20. Automatic Crack Detection on Two-Dimensional Pavement Images: An Algorithm Based on Minimal Path Selection10.1109/TITS.2015.2477675 ↗
21. The Aircraft Skin Crack Inspection Based on Different-Source Sensors and Support Vector Machines10.1007/s10921-016-0359-3 ↗
22. Automatic road crack detection and characterization10.1109/TITS.2012.2208630 ↗
23. Oliveira, H., and Correia, P.L. (2014, January 27–30). CrackIT—An image processing toolbox for crack detection and characterization. Proceedings of the 2014 IEEE International Conference on Image Processing, ICIP 2014, Paris, France.10.1109/ICIP.2014.7025160 ↗
24. Automatic road crack detection using random structured forests10.1109/TITS.2016.2552248 ↗
25. Deep Learning-Based Crack Damage Detection Using Convolutional Neural Networks10.1111/mice.12263 ↗
26. Autonomous Structural Visual Inspection Using Region-Based Deep Learning for Detecting Multiple Damage Types10.1111/mice.12334 ↗
27. Automated Pixel-Level Pavement Crack Detection on 3D Asphalt Surfaces Using a Deep-Learning Network10.1111/mice.12297 ↗
28. Automated Pixel-Level Pavement Crack Detection on 3D Asphalt Surfaces with a Recurrent Neural Network10.1111/mice.12409 ↗
29. Fan, Z., Wu, Y., Lu, J., and Li, W. (2018). Automatic Pavement Crack Detection Based on Structured Prediction with the Convolutional Neural Network. arXiv.
30. Automatic Pixel-Level Crack Detection and Measurement Using Fully Convolutional Network10.1111/mice.12412 ↗
31. An Ensemble of Fine-Tuned Convolutional Neural Networks for Medical Image Classification10.1109/JBHI.2016.2635663 ↗
32. Ensemble of Deep Neural Networks with Probability-Based Fusion for Facial Expression Recognition10.1007/s12559-017-9472-6 ↗
33. Maji, D., Santara, A., Mitra, P., and Sheet, D. (2016). Ensemble of Deep Convolutional Neural Networks for Learning to Detect Retinal Vessels in Fundus Images. arXiv.
34. Building Skeleton Models via 3-D Medial Surface Axis Thinning Algorithms10.1006/cgip.1994.1042 ↗
35. Comparison of thinning algorithms on a parallel processor10.1016/0262-8856(83)90063-X ↗
36. Improvement of Crack-Detection Accuracy Using a Novel Crack Defragmentation Technique in Image-Based Road Assessment10.1061/(ASCE)CP.1943-5487.0000451 ↗
37. Goodfellow, I., Bengio, Y., and Courville, A. (2016). Deep Learning, MIT Press.
38. Nair, V., and Hinton, G.E. (2010, January 21–24). Rectified linear units improve Restricted Boltzmann machines. Proceedings of the ICML 2010 Proceeding 27th International Conference on Machine Learning, Haifa, Israel.
39. Large-scale multi-label text classification—Revisiting neural networks10.1007/978-3-662-44851-9_28 ↗
40. Simonyan, K., and Zisserman, A. (2015, January 7–9). Very deep convolutional networks for large-scale image recognition. Proceedings of the 3rd International Conference on Learning Representations, ICLR 2015—Conference Track Proceedings, San Diego, CA, USA.
41. Springenberg, J.T., Dosovitskiy, A., Brox, T., and Riedmiller, M. (2015, January 7–9). Striving for simplicity: The all convolutional net. Proceedings of the 3rd International Conference on Learning Representations, ICLR 2015—Workshop Track Proceedings, San Diego, CA, USA.
42. Dropout: A simple way to prevent neural networks from overfitting
43. Ensemble methods in machine learning
44. Chambon, S., and Moliard, J.M. (2011). Automatic road pavement assessment with image processing: Review and comparison. Int. J. Geophys., 2011.10.1155/2011/989354 ↗
45. Automatic Pixel-Level Pavement Crack Detection Using Information of Multi-Scale Neighborhoods10.1109/ACCESS.2018.2829347 ↗
46. Konig, J., David Jenkins, M., Barrie, P., Mannion, M., and Morison, G. (2019, January 22–29). A Convolutional Neural Network for Pavement Surface Crack Segmentation Using Residual Connections and Attention Gating. Proceedings of the IEEE International Conference on Image Processing (ICIP), Taipei, Taiwan.10.1109/ICIP.2019.8803060 ↗
47. U-net: Convolutional networks for biomedical image segmentation