An Ensemble Stacked Convolutional Neural Network Model for Environmental Event Sound Recognition
Source abstract
Abstract
Convolutional neural networks (CNNs) with log-mel audio representation and CNN-based end-to-end learning have both been used for environmental event sound recognition (ESC). However, log-mel features can be complemented by features learned from the raw audio waveform with an effective fusion method. In this paper, we first propose a novel stacked CNN model with multiple convolutional layers of decreasing filter sizes to improve the performance of CNN models with either log-mel feature input or raw waveform input. These two models are then combined using the Dempster–Shafer (DS) evidence theory to build the ensemble DS-CNN model for ESC. Our experiments over three public datasets showed that our method could achieve much higher performance in environmental sound recognition than other CNN models with the same types of input features. This is achieved by exploiting the complementarity of the model based on log-mel feature input and the model based on learning features directly from raw waveforms.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Music and Audio Processing · Speech and Audio Processing · Music Technology and Sound Studies
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
School of Mechanical Engineering, Guizhou University, Guiyang 550025, China
Guizhou Provincial Key Laboratory of Public Big Data Guizhou University, Guiyang 550025, China
Key Laboratory of Advanced Manufacturing Technology, Ministry of Education, Guizhou University, Guiyang 550025, China
Department of Computer Science and Engineering, University of South Carolina, Columbia, SC 29208, USA
Funding
National Natural Science Foundation of China · Guizhou Science and Technology Department
References (39)
Show source-provided references
1. Piczak, K.J. (2015, January 26–30). ESC: Dataset for Environmental Sound Classification. Proceedings of the ACM International Conference on Multimedia, Brisbane, Australia.10.1145/2733373.2806390 ↗
2. Łopatka, K., Zwan, P., and Czyżewski, A. (2010). Dangerous Sound Event Recognition Using Support Vector Machine Classifiers, Springer.10.1007/978-3-642-14989-4_5 ↗
3. The implementation of low-cost urban acoustic monitoring devices10.1016/j.apacoust.2016.06.010 ↗
4. Piczak, K.J. (2015, January 17–20). Environmental sound classification with convolutional neural networks. Proceedings of the IEEE International Workshop on Machine Learning for Signal Processing, Boston, MA, USA.10.1109/MLSP.2015.7324337 ↗
5. Tokozume, Y., and Harada, T. (2017, January 5–9). Learning environmental sounds with end-to-end convolutional neural network. Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing, New Orleans, LA, USA.10.1109/ICASSP.2017.7952651 ↗
6. Han, Y., and Lee, K. (arXiv, 2016). Acoustic scene classification using convolutional neural network and multiple-width frequency-delta data augmentation, arXiv.
7. Cotton, C.V., and Ellis, D.P.W. (2011, January 16–19). Spectral vs. spectro-temporal features for acoustic event detection. Proceedings of the Applications of Signal Processing to Audio and Acoustics, New Paltz, NY, USA.10.1109/ASPAA.2011.6082331 ↗
8. Geiger, J.T., Schuller, B., and Rigoll, G. (2013, January 20–23). Large-scale audio feature extraction and SVM for acoustic scene classification. Proceedings of the Applications of Signal Processing to Audio and Acoustics, New Paltz, NY, USA.10.1109/WASPAA.2013.6701857 ↗
9. Ntalampiras, S., Potamitis, I., and Fakotakis, N. (2008, January 9–10). Automatic recognition of urban environmental sounds events. Proceedings of the IAPR Workshop on Cognitive Information Processing Cip, Santorini, Greece.
10. Acoustic Scene Classification: Classifying environments from the sounds they produce10.1109/MSP.2014.2326181 ↗
11. Chachada, S., and Kuo, C.C.J. (2012, January 3–6). Environmental sound recognition: A survey. Proceedings of the Signal and Information Processing Association Summit and Conference, Hollywood, CA, USA.10.1109/APSIPA.2013.6694338 ↗
12. Theodorou, T., Mporas, I., and Fakotakis, N. (2015). Automatic Sound Recognition of Urban Environment Events, Springer International Publishing.10.1007/978-3-319-23132-7_16 ↗
13. Su, F., Yang, L., Lu, T., and Wang, G. (December, January 28). Environmental sound classification for scene recognition using local discriminant bases and HMM. Proceedings of the International Conference on Multimedea, Scottsdale, AZ, USA.
14. Gencoglu, O., Virtanen, T., and Huttunen, H. (2014, January 1–5). Recognition of acoustic events using deep neural networks. Proceedings of the Signal Processing Conference, Lisbon, Portugal.
15. Robust Sound Event Classification Using Deep Neural Networks10.1109/TASLP.2015.2389618 ↗
16. Kons, Z., and Toledo-Ronen, O. (2013, January 25–29). Audio event classification using deep neural networks. Proceedings of the INTERSPEECH 2013, Lyon, France.10.21437/Interspeech.2013-384 ↗
17. A robust environmental sound event recognition using spectral features
18. Huzaifah, M. (arXiv, 2017). Comparison of Time-Frequency Representations for Environmental Sound Classification using Convolutional Neural Networks, arXiv.
19. Deep Convolutional Neural Networks and Data Augmentation for Environmental Sound Classification10.1109/LSP.2017.2657381 ↗
20. Hoshen, Y., Weiss, R.J., and Wilson, K.W. (2015, January 19–24). Speech acoustic modeling from raw multichannel waveforms. Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing, Brisbane, Australia.10.1109/ICASSP.2015.7178847 ↗
21. Zazo, R., Sainath, T.N., Simko, G., and Parada, C. (2016, January 8–12). Feature Learning with Raw-Waveform CLDNNs for Voice Activity Detection. Proceedings of the INTERSPEECH 2016, San Francisco, CA, USA.10.21437/Interspeech.2016-268 ↗
22. Dai, W., Dai, C., Qu, S., Li, J., and Das, S. (2017, January 5–9). Very deep convolutional neural networks for raw waveforms. Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing, New Orleans, LA, USA.10.1109/ICASSP.2017.7952190 ↗
23. Aytar, Y., Vondrick, C., and Torralba, A. (arXiv, 2016). SoundNet: Learning Sound Representations from Unlabeled Video, arXiv.10.1109/CVPR.2016.18 ↗
24. Application of Dempster-Shafer evidence theory to unsupervised classification in multisource remote sensing10.1109/36.602544 ↗
25. Li, Y.-B., Wang, N., and Zhou, C. (2010, January 22–24). Based on D-S evidence theory of information fusion improved method. Proceedings of the International Conference on Computer Application and System Modeling, Taiyuan, China.10.1109/ICCASM.2010.5620424 ↗
26. Improving Rolling Bearing Fault Diagnosis by DS Evidence Theory Based Fusion Model10.1155/2017/6737295 ↗
27. Application of D-S Evidence Fusion Method in the Fault Detection of Temperature Sensor10.1155/2014/395057 ↗
28. Dempster-Shafer evidence theory for multi-bearing faults diagnosis10.1016/j.engappai.2016.10.017 ↗
29. Salamon, J., Jacoby, C., and Bello, J.P. (2014, January 3–7). A Dataset and Taxonomy for Urban Sound Research. Proceedings of the ACM International Conference on Multimedia, Orlando, FL, USA.10.1145/2647868.2655045 ↗
30. Comparison of Parametric Representations for Monosyllabic Word Recognition in Continuously Spoken Sentences
31. Convolutional Neural Networks for Speech Recognition10.1109/TASLP.2014.2339736 ↗
32. Jaitly, N., and Hinton, G. (2011, January 22–27). Learning a better representation of speech soundwaves using restricted boltzmann machines. Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing, Prague, Czech Republic.10.1109/ICASSP.2011.5947700 ↗
33. Palaz, D., Collobert, R., and Doss, M.M. (arXiv, 2013). Estimating Phoneme Class Conditional Probabilities from Raw Speech Signal using Convolutional Neural Networks, arXiv.10.21437/Interspeech.2013-438 ↗
34. Lee, J., Park, J., Kim, K.L., and Nam, J. (arXiv, 2017). Sample-level Deep Convolutional Neural Networks for Music Auto-tagging Using Raw Waveforms, arXiv.
35. Dinkel, H., Chen, N., Qian, Y., and Yu, K. (2017, January 5–9). End-to-end spoofing detection with raw waveform CLDNNS. Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing, New Orleans, LA, USA.10.1109/ICASSP.2017.7953080 ↗
36. Muckenhirn, H., Magimai-Doss, M., and Marcel, S. (2017, January 1–4). End-to-End convolutional neural network-based voice presentation attack detection. Proceedings of the IEEE International Joint Conference on Biometrics, Denver, CO, USA.10.1109/BTAS.2017.8272715 ↗
37. Upper and Lower Probabilities Induced by a Multivalued Mapping10.1214/aoms/1177698950 ↗
38. A mathematical theory of evidence
39. Simonyan, K., and Zisserman, A. (arXiv, 2014). Very Deep Convolutional Networks for Large-Scale Image Recognition, arXiv.