A Scalogram-Based CNN Approach for Audio Classification in Construction Sites
Source abstract
Abstract
The automatic monitoring of activities in construction sites through the proper use of acoustic signals is a recent field of research that is currently in continuous evolution. In particular, the use of techniques based on Convolutional Neural Networks (CNNs) working on the spectrogram of the signal or its mel-scale variants was demonstrated to be quite successful. Nevertheless, the spectrogram has some limitations, which are due to the intrinsic trade-off between temporal and spectral resolutions. In order to overcome these limitations, in this paper, we propose employing the scalogramas a proper time–frequency representation of the audio signal. The scalogram is defined as the square modulus of the Continuous Wavelet Transform (CWT) and is known as a powerful tool for analyzing real-world signals. Experimental results, obtained on real-world sounds recorded in construction sites, have demonstrated the effectiveness of the proposed approach, which is able to clearly outperform most state-of-the-art solutions.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Infrastructure Maintenance and Monitoring · Music and Audio Processing · Structural Health Monitoring Techniques
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Department of Information Engineering, Electronics and Telecommunications (DIET), Sapienza University of Rome, via Eudossiana 18, 00184 Rome, Italy
Department of Construction Management, Louisiana State University, Baton Rouge, LA 70803, USA
Funding
Sapienza University of Rome
References (54)
Show source-provided references
1. Environmental Sound Classification: A descriptive review of the literature
2. A Survey of Audio Classification Using Deep Learning10.1109/ACCESS.2023.3318015 ↗
3. A New Deep CNN Model for Environmental Sound Classification10.1109/ACCESS.2020.2984903 ↗
4. Piczak, K.J. (2015, January 17–20). Environmental sound classification with convolutional neural networks. Proceedings of the 2015 IEEE 25th International Workshop on Machine Learning for Signal Processing (MLSP 2015), Boston, MA, USA.10.1109/MLSP.2015.7324337 ↗
5. Advanced Sound Classifiers and Performance Analyses for Accurate Audio-Based Construction Project Monitoring10.1061/(ASCE)CP.1943-5487.0000911 ↗
6. Xiong, W., Xu, X., Chen, L., and Yang, J. (2022). Sound-Based Construction Activity Monitoring with Deep Learning. Buildings, 12.10.3390/buildings12111947 ↗
7. Kim, I.C., Kim, Y.J., and Chin, S.Y. (2022). Sound Localization Framework for Construction Site Monitoring. Appl. Sci., 12.10.3390/app122110783 ↗
8. Activity classification using accelerometers and machine learning for complex construction worker activities10.1016/j.jobe.2020.102001 ↗
9. Jungmann, M., Ungureanu, L., Hartmann, T., Posada, H., and Chacon, R. (2022, January 11–14). Real-Time Activity Duration Extraction of Crane Works for Data-Driven Discrete Event Simulation. Proceedings of the 2022 Winter Simulation Conference (WSC 2022), Singapore.10.1109/WSC57314.2022.10015250 ↗
10. Automated Methods for Activity Recognition of Construction Workers and Equipment: State-of-the-Art Review10.1061/(ASCE)CO.1943-7862.0001843 ↗
11. Real-time monitoring of construction sites: Sensors, methods, and applications10.1016/j.autcon.2021.104099 ↗
12. Construction safety management in the data-rich era: A hybrid review based upon three perspectives of nature of dataset, machine learning approach, and research topic10.1016/j.aei.2023.102144 ↗
13. Assessing research issues in Automated Project Performance Control (APPC)10.1016/j.autcon.2006.08.001 ↗
14. A learning model for automated construction site monitoring using ambient sounds10.1016/j.autcon.2021.104094 ↗
15. Meng, Q., Peng, Q., Li, Z., and Hu, X. (2022). Big Data Technology in Construction Safety Management: Application Status, Trend and Challenge. Buildings, 12.10.3390/buildings12050533 ↗
16. Activity identification in modular construction using audio signals and machine learning10.1016/j.autcon.2020.103361 ↗
17. Deep Learning in Construction: Review of Applications and Potential Avenues10.1061/(ASCE)CP.1943-5487.0001010 ↗
18. Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., and Rabinovich, A. (2015, January 18–22). Going deeper with convolutions. Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Los Alamitos, CA, USA.10.1109/CVPR.2015.7298594 ↗
19. Wyse, L. (2017, January 17–18). Audio Spectrogram Representations for Processing with Convolutional Neural Networks. Proceedings of the First International Workshop on Deep Learning and Music joint with IJCNN, Anchorage, AK, USA.
20. Dörfler, M., Bammer, R., and Grill, T. (2017, January 8–12). Inside the spectrogram: Convolutional Neural Networks in audio processing. Proceedings of the 2017 International Conference on Sampling Theory and Applications (SampTA), Bordeaux, France.10.1109/SAMPTA.2017.8024472 ↗
21. Analytical expressions for the tonotopic sensory scale10.1121/1.399849 ↗
22. Time-Frequency Reassignment and Synchrosqueezing: An Overview10.1109/MSP.2013.2265316 ↗
23. Mallat, S. (2009). A Wavelet Tour of Signal Processing: The Sparse Way, Academic Press. [3rd ed.].
24. Construction with digital twin information systems10.1017/dce.2020.16 ↗
25. Digital Intelligent Management Platform for High-Rise Building Construction Based on BIM Technology
26. Scientometric analysis and critical review on the application of deep learning in the construction industry10.1139/cjce-2022-0379 ↗
27. Machine learning techniques applied to construction: A hybrid bibliometric analysis of advances and future directions10.1016/j.autcon.2022.104532 ↗
28. Excavation Equipment Recognition Based on Novel Acoustic Statistical Features10.1109/TCYB.2016.2609999 ↗
29. Jeong, G., Ahn, C.R., and Park, M. (2022, January 11–14). Constructing an Audio Dataset of Construction Equipment from Online Sources for Audio-Based Recognition. Proceedings of the 2022 Winter Simulation Conference (WSC), Singapore.10.1109/WSC57314.2022.10015388 ↗
30. Automated activity recognition of construction workers using single in-pocket smartphone and machine learning methods
31. A CNN Approach for Audio Classification in Construction Sites
32. Sound-based multiple-equipment activity recognition using convolutional neural networks10.1016/j.autcon.2021.104104 ↗
33. Deep Belief Network based audio classification for construction sites monitoring10.1016/j.eswa.2021.114839 ↗
34. Esposito, A., Faudez-Zanuy, M., Morabito, F.C., and Pasero, E. (2023). Applications of Artificial Intelligence and Neural Systems to Data Science, Springer.10.1007/978-981-99-3592-5 ↗
35. Kim, I., Kim, Y., and Chin, S. (2023). Deep-Learning-Based Sound Classification Model for Concrete Pouring Work Monitoring at a Construction Site. Appl. Sci., 13.10.3390/app13084789 ↗
36. Development of acoustic denoising learning network for communication enhancement in construction sites10.1016/j.aei.2023.101981 ↗
37. DesPatNet25: Data encryption standard cipher model for accurate automated construction site monitoring with sound signals10.1016/j.eswa.2021.116447 ↗
38. Sherafat, B., Rashidi, A., Lee, Y.C., and Ahn, C.R. (2019). A Hybrid Kinematic-Acoustic System for Automated Activity Detection of Construction Equipment. Sensors, 19.10.3390/s19194286 ↗
39. Action recognition of earthmoving excavators based on sequential pattern analysis of visual features and operation cycles10.1016/j.autcon.2019.03.025 ↗
40. Framework for Location Data Fusion and Pose Estimation of Excavators Using Stereo Vision10.1061/(ASCE)CP.1943-5487.0000783 ↗
41. Visual–auditory learning network for construction equipment action detection10.1111/mice.12983 ↗
42. Elelu, K., Le, T., and Le, C. (2023). Collision Hazard Detection for Construction Worker Safety Using Audio Surveillance. J. Constr. Eng. Manag., 149.10.1061/JCEMD4.COENG-12561 ↗
43. Dang, K., Elelu, K., Le, T., and Le, C. (2022). Augmented Hearing of Auditory Safety Cues for Construction Workers: A Systematic Literature Review. Sensors, 22.10.3390/s22239135 ↗
44. Chen, H., Zhang, P., Bai, H., Yuan, Q., Bao, X., and Yan, Y. (2018, January 2–6). Deep Convolutional Neural Network with Scalogram for Audio Scene Modeling. Proceedings of the Interspeech, Hyderabad, India.10.21437/Interspeech.2018-1524 ↗
45. Faysal, A., Ngui, W.K., Lim, M.H., and Leong, M.S. (2021). Noise Eliminated Ensemble Empirical Mode Decomposition Scalogram Analysis for Rotating Machinery Fault Diagnosis. Sensors, 21.10.3390/s21238114 ↗
46. Copiaco, A., Ritz, C., Fasciani, S., and Abdulaziz, N. (2019, January 10–12). Scalogram Neural Network Activations with Machine Learning for Domestic Multi-channel Audio Classification. Proceedings of the 2019 IEEE International Symposium on Signal Processing and Information Technology (ISSPIT), Ajman, United Arab Emirates.10.1109/ISSPIT47144.2019.9001814 ↗
47. Deep Scalogram Representations for Acoustic Scene Classification10.1109/JAS.2018.7511066 ↗
48. Flanagan, J.L. (1972). Speech Analysis, Synthesis and Perception, Springer. [2nd ed.].10.1007/978-3-662-01562-9 ↗
49. Gupta, P., Chodingala, P.K., and Patil, H.A. (September, January 29). Morlet Wavelet-Based Voice Liveness Detection using Convolutional Neural Network. Proceedings of the 2022 30th European Signal Processing Conference (EUSIPCO), Belgrade, Serbia.10.23919/EUSIPCO55093.2022.9909835 ↗
50. Feature extraction of machine sound using wavelet and its application in fault diagnosis10.1016/S0963-8695(00)00025-6 ↗
51. Krizhevsky, A., Sutskever, I., and Hinton, G.E. (2012, January 3–8). ImageNet Classification with Deep Convolutional Neural Networks. Proceedings of the 25th International Conference on Neural Information Processing Systems, Stateline, NV, USA.
52. Kingma, D.P., and Ba, J. (2015, January 7–9). Adam: A Method for Stochastic Optimization. Proceedings of the 3rd International Conference for Learning Representations (ICLR 2015), San Diego, CA, USA.
53. Evaluation: From Precision, Recall and F-Measure to ROC, Informedness, Markedness and Correlation
54. Scarpiniti, M., Comminiello, D., Uncini, A., and Lee, Y.C. (2020, January 24–28). Deep recurrent neural networks for audio classification in construction sites. Proceedings of the 28th European Signal Processing Conference (EUSIPCO 2020), Amsterdam, The Netherlands.10.23919/Eusipco47968.2020.9287802 ↗