journal article·2023-08-13Open access

The Combination of Transformer and You Only Look Once for Automatic Concrete Pavement Crack Detection

Xin Zheng, Songrong Qian, Shaodong Wei, Shiyun Zhou, Yi Hou

Applied Sciences· Volume 13· Issue 16

Abstract

The real-time detection of cracks is an important part of road maintenance and an important initiative to reduce traffic accidents caused by road cracks. In response to the lack of efficiency of current research results for the real-time detection of road cracks and the low storage and computational capacity of edge devices, a new automatic crack detection algorithm is proposed: BT–YOLO. We combined Bottleneck Transformer with You Only Look Once (YOLO), which is more conducive to extracting the features of small cracks than YOLOv5s. The introduction of DWConv to the feature extraction network reduced the number of parameters and improved the inference speed of the network. We embedded the SimAM (Simple, Parameter-Free Attention Module) non-parametric attention mechanism to make the crack features more prominent. The experimental results showed that the accuracy of BT–YOLO in crack detection was increased by 4.5%, the mapped value was increased by 8%, and the parameter amount was decreased by 24.9%. Eventually, we deployed edge devices for testing. The frame rate reached 89, which satisfied the requirements of real-time crack detection.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: Infrastructure Maintenance and Monitoring · Asphalt Pavement Performance Evaluation · Concrete Corrosion and Durability

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

School of Mechanical Engineering, Guizhou University, Guiyang 550025, China

State Key Laboratory of Public Big Data, Guizhou University, Guiyang 550025, China

Funding

Guizhou optoelectronic information and intelligent application International Joint Research Center

References (43)

Show source-provided references
1. Crack detection using image processing: A critical review and analysis10.1016/j.aej.2017.01.020 ↗
2. Zaloshnja, E., and Miller, T.R. (2009, January 5–7). Cost of crashes related to road conditions, United States, 2006. Proceedings of the Annals of Advances in Automotive Medicine/Annual Scientific Conference, Baltimore, Maryland.
3. Visual retrieval of concrete crack properties for automated post-earthquake structural safety evaluation10.1016/j.autcon.2011.03.004 ↗
4. Gkantou, M., Muradov, M., Kamaris, G.S., Hashim, K., Atherton, W., and Kot, P. (2019). Novel Electromagnetic Sensors Embedded in Reinforced Concrete Beams for Crack Detection. Sensors, 19.10.3390/s19235175 ↗
5. Koshti, A.M. (2018). Health Monitoring of Structural and Biological Systems XII, SPIE.
6. Hosseini, Z., Momayez, M., Hassani, F., and Lévesque, D. (2008). AIP Conference Proceedings, American Institute of Physics.
7. Cooling analysis of welded materials for crack detection using infrared thermography10.1016/j.infrared.2014.09.025 ↗
8. Pavement distress analysis using image processing techniques10.1111/j.1467-8667.1991.tb00393.x ↗
9. Oliveira, H., and Correia, P.L. (2009, January 24–28). Automatic road crack segmentation using entropy and image dynamic thresholding. Proceedings of the 17th European Signal Processing Conference, Scotland, UK.
10. Automatic inspection of pavement cracking distress10.1117/1.2177650 ↗
11. A Threshold Selection Method from Gray-Level Histograms10.1109/TSMC.1979.4310076 ↗
12. A fast two-dimensional median filtering algorithm10.1109/TASSP.1979.1163188 ↗
13. Adaptive histogram equalization and its variations10.1016/S0734-189X(87)80186-X ↗
14. Road crack detection using visual features extracted by Gabor filters10.1111/mice.12042 ↗
15. The Aircraft Skin Crack Inspection Based on Different-Source Sensors and Support Vector Machines10.1007/s10921-016-0359-3 ↗
16. Automatic Road Crack Detection Using Random Structured Forests10.1109/TITS.2016.2552248 ↗
17. Pavement crack detection and classification based on fusion feature of LBP and PCA with SVM10.1080/10298436.2021.1888092 ↗
18. Chen, C., Seo, H.S., Zhao, Y., Chen, B., Kim, J.W., Choi, Y., and Bang, M. (2019, January 8–10). Automatic pavement crack detection based on image recognition. Proceedings of the International Conference on Smart Infrastructure and Construction 2019 (ICSIC) Driving Data-Informed Decision Making, Cambridge, UK.10.1680/icsic.64669.361 ↗
19. Automatic pixel-level crack detection and measurement using the fully convolutional network10.1111/mice.12412 ↗
20. Automatic pavement crack detection and classification using multiscale feature attention network10.1109/ACCESS.2019.2956191 ↗
21. Automated Crack Detection on Concrete Bridges10.1109/TASE.2014.2354314 ↗
22. Fujita, Y., Mitani, Y., and Hamamoto, Y. (2006, January 20–24). A method for crack detection on a concrete structure. Proceedings of the 18th International Conference on Pattern Recognition (ICPR’06), Hong Kong, China.10.1109/ICPR.2006.98 ↗
23. A multi-hop task offloading decision model in MEC-enabled internet of vehicles10.1109/JIOT.2022.3143529 ↗
24. Fan, Z., Wu, Y., Lu, J., and Li, W. (2018). Automatic Pavement Crack Detection Based on Structured Prediction with the Convolutional Neural Network. arXiv.
25. Autonomous concrete crack detection using deep fully convolutional neural network10.1016/j.autcon.2018.11.028 ↗
26. Computer vision-based concrete crack detection using U-net fully convolutional networks10.1016/j.autcon.2019.04.005 ↗
27. Automated pavement crack detection and segmentation based on two-step convolutional neural network10.1111/mice.12622 ↗
28. Multi-scale classification network for road crack detection10.1049/iet-its.2018.5280 ↗
29. Concrete road crack detection using deep learning-based faster R-CNN method10.1007/s40996-021-00671-2 ↗
30. Improved MobileNetV2-SSDLite for automatic fabric defect detection system based on cloud-edge computing10.1016/j.measurement.2022.111665 ↗
31. Concrete crack segmentation based on UAV-enabled edge computing10.1016/j.neucom.2021.03.139 ↗
32. Deep learning-based linear defects detection system for large-scale photovoltaic plants based on an edge-cloud computing infrastructure10.1016/j.solener.2021.11.016 ↗
33. Wu, Y., Guo, H., Chakraborty, C., Khosravi, M., Berretti, S., and Wan, S. (2022). Edge Computing Driven Low-Light Image Dynamic Enhancement for Object Detection. IEEE Trans. Netw. Sci. Eng.10.1109/TNSE.2022.3151502 ↗
34. IoMT-Enabled Real-time Blood Glucose Prediction with Deep Learning and Edge Computing10.1109/JIOT.2022.3143375 ↗
35. Automated bridge surface crack detection and segmentation using computer vision-based deep learning model10.1016/j.engappai.2022.105225 ↗
36. Tzutalin, D. (2023, July 04). LabelImg. GitHub Repository. Available online: https://pypi.org/project/labelImg/.
37. Trinh, H.-C., Le, D.-H., and Kwon, Y.-K. (2014). PANET: A GPU-based tool for fast parallel analysis of robustness dynamics and feed-forward/feedback loop structures in large-scale biological networks. PLoS ONE, 9.10.1371/journal.pone.0103010 ↗
38. Heterogeneous Systolic Array Architecture for Compact CNNs Hardware Accelerators
39. Srinivas, A., Lin, T.Y., Parmar, N., Shlens, J., Abbeel, P., and Vaswani, A. (2021, January 20–25). Bottleneck transformers for visual recognition. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA.10.1109/CVPR46437.2021.01625 ↗
40. Yang, L., Zhang, R.Y., Li, L., and Xie, X. (2021, January 18–24). Simam: A simple, parameter-free attention module for convolutional neural networks. Proceedings of the International Conference on Machine Learning, Virtual Event.
41. Gevorgyan, Z. (2022). SIoU loss: More powerful learning for bounding box regression. arXiv.
42. Model compression and hardware acceleration for neural networks: A comprehensive survey10.1109/JPROC.2020.2976475 ↗
43. Bottou, L. (2012). Neural Networks: Tricks of the Trade, Springer. [2nd ed.].