journal article·2021-01-16Open access

Deep Learning-Based Applications for Safety Management in the AEC Industry: A Review

Lei Hou, Haosen Chen, Guomin (Kevin) Zhang, Xiangyu Wang

Applied Sciences· Volume 11· Issue 2

Abstract

Safety is an essential topic to the architecture, engineering and construction (AEC) industry. However, traditional methods for structural health monitoring (SHM) and jobsite safety management (JSM) are not only inefficient, but also costly. In the past decade, scholars have developed a wide range of deep learning (DL) applications to address automated structure inspection and on-site safety monitoring, such as the identification of structural defects, deterioration patterns, unsafe workforce behaviors and latent risk factors. Although numerous studies have examined the effectiveness of the DL methodology, there has not been one comprehensive, systematic, evidence-based review of all individual articles that investigate the effectiveness of using DL in the SHM and JSM industry to date, nor has there been an examination of this body of evidence in regard to these methodological problems. Therefore, the objective of this paper is to disclose the state of the art of current research progress and determine the relevant gaps, challenges and future work. Methodically, CiteSpace was employed to summarize the research trends, advancements and frontiers of DL applications from 2010 to 2020. Next, an application-focused literature review was conducted, which led to a summary of research gaps, recommendations and future research directions. Overall, this review gains insight into SHM and JSM and aims to help researchers formulate more types of effective DL applications which have not been addressed sufficiently for the time being.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: Infrastructure Maintenance and Monitoring · Occupational Health and Safety Research · Concrete Corrosion and Durability

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

School of Engineering, RMIT University, Melbourne, VIC 3000, Australia

School of Civil Engineering and Architecture, East China Jiaotong University, Nanchang 330013, China

Australasian Joint Research Centre for Building Information Modelling, Curtin University, Perth, WA 6102, Australia

References (100+)

Show source-provided references
1. A review on computer vision based defect detection and condition assessment of concrete and asphalt civil infrastructure10.1016/j.aei.2015.01.008 ↗
2. Nonlinear numerical analysis and progressive damage assessment of a cable-stayed bridge pier subjected to ship collision10.1016/j.marstruc.2019.102662 ↗
3. Advances in computer vision-based civil infrastructure inspection and monitoring10.1016/j.eng.2018.11.030 ↗
4. Effect of Axial Compression Ratio on Seismic Behavior of GFRP Reinforced Concrete Columns10.1142/S0219455420400040 ↗
5. Large deflection behavior effect in reinforced concrete columns exposed to extreme dynamic loads10.1007/s11709-020-0604-9 ↗
6. Flah, M., Nunez, I., Chaabene, W.B., and Nehdi, M.L. (2020). Machine learning algorithms in civil structural health monitoring: A systematic review. Arch. Comput. Methods Eng., 1–23.10.1007/s11831-020-09471-9 ↗
7. Sun, L., Li, C., Zhang, C., Liang, T., and Zhao, Z. (2019). The strain transfer mechanism of fiber bragg grating sensor for extra large strain monitoring. Sensors, 19.10.3390/s19081851 ↗
8. Kordestani, H., and Zhang, C. (2020). Direct use of the savitzky–golay filter to develop an output-only trend line-based damage detection method. Sensors, 20.10.3390/s20071983 ↗
9. Computer vision applications in construction safety assurance10.1016/j.autcon.2019.103013 ↗
10. Prasanna, P., Dana, K., Gucunski, N., and Basily, B. (2012, January 12–15). Computer-vision based crack detection and analysis. Proceedings of the Sensors and Smart Structures Technologies for Civil, Mechanical, and Aerospace Systems, San Diego, CA, USA.10.1117/12.915384 ↗
11. Risk-based management of occupational safety and health in the construction industry–Part 1: Background knowledge10.1016/j.ssci.2014.02.008 ↗
12. Capturing and Understanding Workers’ Activities in Far-Field Surveillance Videos with Deep Action Recognition and Bayesian Nonparametric Learning10.1111/mice.12419 ↗
13. Wang, M., Wong, P., Luo, H., Kumar, S., Delhi, V., and Cheng, J. (2019, January 21–24). Predicting Safety Hazards Among Construction Workers and Equipment Using Computer Vision and Deep Learning Techniques. Proceedings of the International Symposium on Automation and Robotics in Construction (ISARC), Banff Alberta, AB, Canada.10.22260/ISARC2019/0054 ↗
14. A novel methodology for modal parameters identification of large smart structures using MUSIC, empirical wavelet transform, and Hilbert transform10.1016/j.engstruct.2017.05.054 ↗
15. Automatic Pavement Crack Detection by Multi-Scale Image Fusion10.1109/TITS.2018.2856928 ↗
16. Event-Triggered Synchronization for Neutral-Type Semi-Markovian Neural Networks With Partial Mode-Dependent Time-Varying Delays10.1109/TNNLS.2019.2955287 ↗
17. Automatic detection of feather defects using Lie group and fuzzy Fisher criterion for shuttlecock production10.1016/j.ymssp.2020.106690 ↗
18. Oliveira, H., and Correia, P.L. (2008, January 25–29). Supervised strategies for cracks detection in images of road pavement flexible surfaces. Proceedings of the IEEE 2008 16th European Signal Processing Conference, Lausanne, Switzerland.
19. A deep hybrid learning model to detect unsafe behavior: Integrating convolution neural networks and long short-term memory10.1016/j.autcon.2017.11.002 ↗
20. A deep learning CNN architecture applied in smart near-infrared analysis of water pollution for agricultural irrigation resources10.1016/j.agwat.2020.106303 ↗
21. Imagenet classification with deep convolutional neural networks10.1145/3065386 ↗
22. Deep-learning-enabled geometric constraints and phase unwrapping for single-shot absolute 3D shape measurement10.1063/5.0003217 ↗
23. Deep Learning: A Rapid and Efficient Route to Automatic Metasurface Design10.1002/advs.201900128 ↗
24. Segnet: A deep convolutional encoder-decoder architecture for image segmentation10.1109/TPAMI.2016.2644615 ↗
25. Image recognition with deep neural networks in presence of noise–dealing with and taking advantage of distortions10.3233/ICA-170551 ↗
26. Vehicle type detection by ensembles of convolutional neural networks operating on super resolved images10.3233/ICA-180577 ↗
27. Regional parallel structure based CNN for thermal infrared face identification10.3233/ICA-180560 ↗
28. A framework for learning from incidents in the workplace10.1016/j.ssci.2011.12.032 ↗
29. Incident reporting or storytelling? Competing schemes in a safety-critical and hazardous work setting10.1016/j.ssci.2007.06.024 ↗
30. Chen, C., and Song, M. (2019). Visualizing a field of research: A methodology of systematic scientometric reviews. PLoS ONE, 14.10.1371/journal.pone.0223994 ↗
31. CiteSpace II: Detecting and visualizing emerging trends and transient patterns in scientific literature10.1002/asi.20317 ↗
32. The citespace manual
33. Rose, P., Aaron, B., Tamir, D.E., Lu, L., Hu, J., and Shi, H. (2014, January 12–16). Supervised computer-vision-based sensing of concrete bridges for crack-detection and assessment. Proceedings of the Transportation Research Board 93rd Annual Meeting, Washington, DC, USA.
34. Foreground–background separation technique for crack detection10.1111/mice.12428 ↗
35. Defining a conceptual framework for the integration of modelling and advanced imaging for improving the reliability and efficiency of bridge assessments10.1007/s13349-016-0191-6 ↗
36. Unified No-Reference Quality Assessment of Singly and Multiply Distorted Stereoscopic Images10.1109/TIP.2018.2881828 ↗
37. Synchrosqueezed wavelet transform-fractality model for locating, detecting, and quantifying damage in smart highrise building structures10.1088/0964-1726/24/6/065034 ↗
38. Defect detection in reinforced concrete using random neural architectures10.1111/mice.12039 ↗
39. Pseudospectra, MUSIC, and dynamic wavelet neural network for damage detection of highrise buildings10.1002/nme.1964 ↗
40. Experimental validation of cost-effective vision-based structural health monitoring10.1016/j.ymssp.2016.11.021 ↗
41. Image-based automated 3D crack detection for post-disaster building assessment10.1061/(ASCE)CP.1943-5487.0000334 ↗
42. Fast crack detection method for large-size concrete surface images using percolation-based image processing10.1007/s00138-009-0189-8 ↗
43. Vision-based automated crack detection for bridge inspection10.1111/mice.12141 ↗
44. Target-free approach for vision-based structural system identification using consumer-grade cameras10.1002/stc.1850 ↗
45. Texture analysis based damage detection of ageing infrastructural elements10.1111/j.1467-8667.2012.00790.x ↗
46. Concrete crack detection by multiple sequential image filtering10.1111/j.1467-8667.2011.00716.x ↗
47. Road crack detection using visual features extracted by Gabor filters10.1111/mice.12042 ↗
48. Achievements and challenges in machine vision-based inspection of large concrete structures10.1260/1369-4332.17.3.303 ↗
49. Support-vector-machine-based method for automated steel bridge rust assessment10.1016/j.autcon.2011.12.001 ↗
50. Machine vision-based model for spalling detection and quantification in subway networks10.1016/j.autcon.2017.06.008 ↗
51. Rapid entropy-based detection and properties measurement of concrete spalling with machine vision for post-earthquake safety assessments10.1016/j.aei.2012.06.005 ↗
52. Vision-based detection of loosened bolts using the Hough transform and support vector machines10.1016/j.autcon.2016.06.008 ↗
53. Girshick, R., Donahue, J., Darrell, T., and Malik, J. (2014, January 23–28). Rich feature hierarchies for accurate object detection and semantic segmentation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA.10.1109/CVPR.2014.81 ↗
54. Selective search for object recognition10.1007/s11263-013-0620-5 ↗
55. Recent progress and future trends on damage identification methods for bridge structures10.1002/stc.2416 ↗
56. Autonomous structural visual inspection using region-based deep learning for detecting multiple damage types10.1111/mice.12334 ↗
57. Girshick, R. (2015, January 7–13). Fast r-cnn. Proceedings of the IEEE International Conference on Computer Vision, Santiago, Chile.10.1109/ICCV.2015.169 ↗
58. Faster r-cnn: Towards real-time object detection with region proposal networks10.1109/TPAMI.2016.2577031 ↗
59. Unified vision-based methodology for simultaneous concrete defect detection and geolocalization10.1111/mice.12351 ↗
60. Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C.-Y., and Berg, A.C. (2016, January 11–14). Ssd: Single shot multibox detector. Proceedings of the European Conference on Computer Vision, Amsterdam, The Netherlands.10.1007/978-3-319-46448-0_2 ↗
61. Redmon, J., Divvala, S., Girshick, R., and Farhadi, A. (2016, January 27–30). You only look once: Unified, real-time object detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.10.1109/CVPR.2016.91 ↗
62. Automatic pixel-level crack detection and measurement using fully convolutional network10.1111/mice.12412 ↗
63. Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs10.1109/TPAMI.2017.2699184 ↗
64. Cui, Z., Ke, R., Pu, Z., and Wang, Y. (2018). Deep bidirectional and unidirectional LSTM recurrent neural network for network-wide traffic speed prediction. arXiv.
65. A fast detection method via region-based fully convolutional neural networks for shield tunnel lining defects10.1111/mice.12367 ↗
66. Jin, Y., Zhang, J., Li, M., Tian, Y., Zhu, H., and Fang, Z. (2017). Towards the automatic anime characters creation with generative adversarial networks. arXiv.
67. Encoder–decoder network for pixel-level road crack detection in black-box images10.1111/mice.12440 ↗
68. Pavement defect detection with fully convolutional network and an uncertainty framework10.1111/mice.12533 ↗
69. Nondestructive test methods for concrete bridges: A review10.1016/j.conbuildmat.2015.12.011 ↗
70. Influence of the Position of Artificial Boundary on Computation Accuracy of Conjugated Infinite Element for a Finite Length Cylindrical Shell10.1007/s40857-020-00175-5 ↗
71. Adeli, H., and Jiang, X. (2008). Intelligent Infrastructure: Neural Networks, Wavelets, and Chaos Theory for Intelligent Transportation Systems and Smart Structures, CRC Press.
72. Ofsthun, S.C., and Wilmering, T.J. (2004, January 6–13). Model-driven development of integrated health management architectures. Proceedings of the 2004 IEEE Aerospace Conference Proceedings (IEEE Cat. No. 04TH8720), Big Sky, MT, USA.10.1109/AERO.2004.1368185 ↗
73. A method of data recovery based on compressive sensing in wireless structural health monitoring10.1155/2014/546478 ↗
74. Wu, C., Wu, P., Wang, J., Jiang, R., Chen, M., and Wang, X. (2020). Critical review of data-driven decision-making in bridge operation and maintenance. Struct. Infrastruct. Eng., 1–24.10.1080/15732479.2020.1833946 ↗
75. Mousavi, A.A., Zhang, C., Masri, S.F., and Gholipour, G. (2020). Structural Damage Localization and Quantification Based on a CEEMDAN Hilbert Transform Neural Network Approach: A Model Steel Truss Bridge Case Study. Sensors, 20.10.3390/s20051271 ↗
76. Genetic algorithm-support vector regression for high reliability SHM system based on FBG sensor network10.1016/j.optlaseng.2011.09.015 ↗
77. One-year dynamic monitoring of a historic tower: Damage detection under changing environment10.1007/s11012-016-0482-3 ↗
78. Temperature effects on static and dynamic behavior of Consoli Palace in Gubbio, Italy10.1016/j.ymssp.2018.10.021 ↗
79. Zonno, G., Aguilar, R., Boroschek, R., and Lourenço, P.B. (2019). Environmental and Ambient Vibration Monitoring of Historical Adobe Buildings: Applications in Emblematic Andean Churches. Int. J. Archit. Herit., 1–17.10.1080/15583058.2019.1653402 ↗
80. Environmental effects on the identified natural frequencies of the Dowling Hall Footbridge10.1016/j.ymssp.2011.03.005 ↗
81. One-year monitoring of the Z24-Bridge: Environmental effects versus damage events10.1002/1096-9845(200102)30:2<149::AID-EQE1>3.0.CO;2-Z ↗
82. Analysis of the long and short-term effects of temperature and humidity on the structural properties of adobe buildings using continuous monitoring10.1016/j.engstruct.2019.109299 ↗
83. Deep learning for data anomaly detection and data compression of a long-span suspension bridge10.1111/mice.12528 ↗
84. Wireless and real-time structural damage detection: A novel decentralized method for wireless sensor networks10.1016/j.jsv.2018.03.008 ↗
85. Structural health monitoring using extremely compressed data through deep learning10.1111/mice.12517 ↗
86. Structural damage detection with automatic feature-extraction through deep learning10.1111/mice.12313 ↗
87. Gulgec, N.S., Takáč, M., and Pakzad, S.N. (2020). Structural sensing with deep learning: Strain estimation from acceleration data for fatigue assessment. Comput. Aided Civ. Infrastruct. Eng.10.1111/mice.12565 ↗
88. Structural health monitoring by using a sparse coding-based deep learning algorithm with wireless sensor networks10.1007/s00779-014-0800-5 ↗
89. Automated object identification using optical video cameras on construction sites10.1111/j.1467-8667.2010.00690.x ↗
90. Automated 2D detection of construction equipment and workers from site video streams using histograms of oriented gradients and colors10.1016/j.autcon.2012.12.002 ↗
91. Data-driven scene parsing method for recognizing construction site objects in the whole image10.1016/j.autcon.2016.08.018 ↗
92. Vision-based framework for intelligent monitoring of hardhat wearing on construction sites10.1061/(ASCE)CP.1943-5487.0000813 ↗
93. Xie, Z., Liu, H., Li, Z., and He, Y. (2018, January 14–16). A convolutional neural network based approach towards real-time hard hat detection. Proceedings of the 2018 IEEE International Conference on Progress in Informatics and Computing (PIC), Suzhou, China.10.1109/PIC.2018.8706269 ↗
94. Detecting non-hardhat-use by a deep learning method from far-field surveillance videos10.1016/j.autcon.2017.09.018 ↗
95. Automatic detection of hardhats worn by construction personnel: A deep learning approach and benchmark dataset10.1016/j.autcon.2019.102894 ↗
96. Falls from heights: A computer vision-based approach for safety harness detection10.1016/j.autcon.2018.02.018 ↗
97. Transfer learning and deep convolutional neural networks for safety guardrail detection in 2D images10.1016/j.autcon.2018.01.003 ↗
98. Unsupervised feature learning for objects of interest detection in cluttered construction roof site images10.1016/j.proeng.2016.04.010 ↗
99. Long short-term memory10.1162/neco.1997.9.8.1735 ↗
100. Image-based construction hazard avoidance system using augmented reality in wearable device10.1016/j.autcon.2017.06.014 ↗