A Deep Attention Model for Environmental Sound Classification from Multi-Feature Data
Source abstract
Abstract
Automated environmental sound recognition has clear engineering benefits; it allows audio to be sorted, curated, and searched. Unlike music and language, environmental sound is loaded with noise and lacks the rhythm and melody of music or the semantic sequence of language, making it difficult to find common features representative enough of various environmental sound signals. To improve the accuracy of environmental sound recognition, this paper proposes a recognition method based on multi-feature parameters and time–frequency attention module. It begins with a pretreatment that relies on multi-feature parameters to extract the sound, which supplements the phase information lost by the Log-Mel spectrogram in the current mainstream methods, and enhances the expressive ability of input features. A time–frequency attention module with multiple convolutions is designed to extract the attention weight of the input feature spectrogram and reduce the interference coming from the background noise and irrelevant frequency bands in the audio. Comparative experiments were conducted on three general datasets: environmental sound classification datasets (ESC-10, ESC-50) and an UrbanSound8K dataset. Experiments demonstrated that the proposed method performs better.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Music and Audio Processing · Speech and Audio Processing · Animal Vocal Communication and Behavior
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
National Key Laboratory of Electronic Testing Technology, North University of China, Taiyuan 030051, China
Hunan Vanguard Group Co., Ltd., Changsha 410100, China
Funding
National Science Foundation of China
References (27)
Show source-provided references
1. Baum, E., Harper, M., Alicea, R., and Ordonez, C. (February, January 31). Sound identification for fire-fighting mobile robots. Proceedings of the IEEE International Conference on Robotic Computing, Laguna Hills, CA, USA.
2. Robust environmental sound recognition for home automation10.1109/TASE.2007.911680 ↗
3. Radhakrishnan, R., Divakaran, A., and Smaragdis, A. (2005, January 16–19). Audio analysis for surveillance applications. Proceedings of the IEEE Workshop on Applications of Signal Processing to Audio and Acoustics, New Paltz, NY, USA.
4. Cotton, C.V., and Ellis, D. (2011, January 16–19). Spectral vs. spectro-temporal features for acoustic event detection. Proceedings of the 2011 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA), New Paltz, NY, USA.10.1109/ASPAA.2011.6082331 ↗
5. A Lung Sound Category Recognition Method Based on Wavelet Decomposition and BP Neural Network10.7150/ijbs.29863 ↗
6. Geiger, J.T., and Helwani, K. (September, January 31). Improving event detection for audio surveillance using Gabor filterbank features. Proceedings of the 2015 23rd European Signal Processing Conference (EUSIPCO), Nice, France.10.1109/EUSIPCO.2015.7362476 ↗
7. Wang, J.C., Wang, J.F., He, K.W., and Hsu, C.S. (2006, January 16–21). Environmental sound classification using hybrid SVM/KNN classifier and MPEG-7 audio low-level descriptor. Proceedings of the International Joint Conference on Neural Networks, Vancouver, BC, Canada.
8. Urban sound event classification based on local and global features aggregation10.1016/j.apacoust.2016.08.002 ↗
9. Feature learning with matrix factorization applied to acoustic scene classification. IEEE/ACM Trans
10. Multi-kernel extreme learning machine for EEG classification in brain-computer interfaces
11. Li, C., Li, S., Gao, Y., Zhang, X., and Li, W. (2021). A Two-stream Neural Network for Pose-based Hand Gesture Recognition. IEEE Trans. Cogn. Dev. Syst., 1–10.
12. Tokozume, Y., and Harada, T. (2017, January 5–9). Learning environmental sounds with end-to-end convolutional neural network. Proceedings of the 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), New Orleans, LA, USA.10.1109/ICASSP.2017.7952651 ↗
13. Dai, W., Dai, C., Qu, S., Li, J., and Das, S. (2017, January 5–9). Very deep convolutional neural networks for raw waveforms. Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), New Orleans, LA, USA.10.1109/ICASSP.2017.7952190 ↗
14. Zhu, B., Xu, K., Wang, D., Zhang, L., Li, B., and Peng, Y. (2018). Environmental sound classification based on multi-temporal resolution convolutional neural network combining with multi-level features. Advances in Multimedia Information Processing—PCM 2018, Proceedings of the Pacific Rim Conference on Multimedia, Hefei, China, 21–22 September 2018, Springer.10.1007/978-3-030-00767-6_49 ↗
15. End-to-end environmental sound classification using a 1D convolutional neural network10.1016/j.eswa.2019.06.040 ↗
16. Environmental Sound Recognition with Time–Frequency Audio Features10.1109/TASL.2009.2017438 ↗
17. Piczak, K.J. (2015, January 17–20). Environmental sound classification with convolutional neural networks. Proceedings of the 25th International Workshop on Machine Learning for Signal Processing, Boston, MA, USA.10.1109/MLSP.2015.7324337 ↗
18. Deep convolutional neural networks and data augmentation for environmental sound classification10.1109/LSP.2017.2657381 ↗
19. Zhang, Z., Xu, S., Cao, S., and Zhang, S. (2018). Deep convolutional neural network with mixup for environmental sound classification. Pattern Recognition and Computer Vision. PRCV 2018, Proceedings of the Chinese Conference PRCV, Guangzhou, China, 23–26 November 2018, Springer.10.1007/978-3-030-03335-4_31 ↗
20. Environment sound event classification with a two-stream convolutional neural network10.1109/ACCESS.2020.3007906 ↗
21. Piczak, K.J. (2015, January 26–30). ESC: Dataset for environmental sound classification. Proceedings of the 23rd ACM International Conference on Multimedia, Brisbane, Australia.10.1145/2733373.2806390 ↗
22. Salamon, J., Jacoby, C., and Bello, J.P. (2014, January 3–7). A dataset and taxonomy for urban sound research. Proceedings of the 22nd ACM International Conference on Multimedia, Orlando, FL, USA.10.1145/2647868.2655045 ↗
23. Guzhov, A., Raue, F., Hees, J., and Dengel, A. (2020). ESResNet: Environmental Sound Classification Based on Visual Domain Models. arXiv.
24. Loshchilov, I., and Hutter, F. (2016). SGDR: Stochastic Gradient Descent with Restarts. arXiv.
25. Zhang, H., Cisse, M., Dauphin, Y.N., and Lopez-Paz, D. (May, January 30). Mixup: Beyond empirical risk minimization. Proceedings of the International Conference on Learning Representations, Vancouver, BC, Canada.
26. Classifying environmental sounds using image recognition networks10.1016/j.procs.2017.08.250 ↗
27. Wang, H., Zou, Y., Chong, D., and Wang, W. (2019). Learning discriminative and robust time-frequency representations for environmental sound classification. arXiv.