Sound Event Detection Using Derivative Features in Deep Neural Networks
Source abstract
Abstract
We propose using derivative features for sound event detection based on deep neural networks. As input to the networks, we used log-mel-filterbank and its first and second derivative features for each frame of the audio signal. Two deep neural networks were used to evaluate the effectiveness of these derivative features. Specifically, a convolutional recurrent neural network (CRNN) was constructed by combining a convolutional neural network and a recurrent neural networks (RNN) followed by a feed-forward neural network (FNN) acting as a classification layer. In addition, a mean-teacher model based on an attention CRNN was used. Both models had an average pooling layer at the output so that weakly labeled and unlabeled audio data may be used during model training. Under the various training conditions, depending on the neural network architecture and training set, the use of derivative features resulted in a consistent performance improvement by using the derivative features. Experiments on audio data from the Detection and Classification of Acoustic Scenes and Events 2018 and 2019 challenges indicated that a maximum relative improvement of 16.9% was obtained in terms of the F-score.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Music and Audio Processing · Speech and Audio Processing · Speech Recognition and Synthesis
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Department of Electronics, Keimyung University, Daegu 42601, Korea
Funding
Ministry of Education, Science and Technology
References (24)
Show source-provided references
1. Turpault, N., Serizel, R., Shah, A., and Salamon, J. (2019, January 25–26). Sound event detection in domestic environments with weakly labeled data and soundscape synthesis. Proceedings of the Workshop on Detection and Classification of Acoustic Scenes and Events, New York, NY, USA. hal-02160855v2.10.33682/006b-jx26 ↗
2. Nandwana, M.K., Ziaei, A., and Hansen, J. (2015, January 19–24). Robust unsupervised detection of human screams in noisy acoustic environments. Proceedings of the International Conference on Acoustics, Speech and Signal Processing, Brisbane, Australia.10.1109/ICASSP.2015.7177952 ↗
3. Audio surveillance: A systematic review10.1145/2871183 ↗
4. Salamon, J., and Bello, J.P. (September, January 31). Feature learning with deep scattering for urban sound analysis. Proceedings of the 23rd European Signal Processing Conference (EUSIPCO), Nice, France.
5. Ntalampiras, S., Potamitis, I., and Fakotakise, N. (2009, January 19–24). On acoustic surveillance of hazardous situations. Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Taipei, Taiwan.10.1109/ICASSP.2009.4959546 ↗
6. Wang, Y., Neves, L., and Metze, F. (2016, January 20–25). Audio-based multimedia event detection using deep recurrent neural networks. Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Shanghai, China.10.1109/ICASSP.2016.7472176 ↗
7. Dekkers, G., Vuegen, L., Waterschoot, T., Vanrumste, B., and Karsmakers, P. (2018). DCASE 2018 challenge—Task 5: Monitoring of domestic activities based on multi-channel acoustics. arXiv.
8. Krizhevsky, A., Sutskever, I., and Hinton, G. (2012, January 3–8). Imagenet classification with deep convolutional neural networks. Proceedings of the Advances in Neural Information Processing Systems, Stateline, NV, USA.
9. Graves, A., Mohamed, A., and Hinton, G. (2013, January 26–31). Speech recognition with deep recurrent neural Networks. Proceedings of the IEEE International Conference on Acoustics Speech and Signal Processing (ICASSP), Vancouver, BC, Canada.10.1109/ICASSP.2013.6638947 ↗
10. Cho, K., Merrienboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., and Bengio, Y. (2014, January 25–29). Learning phrase representations using RNN encoder-decoder for statistical machine translation. Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), Doha, Qatar.10.3115/v1/D14-1179 ↗
11. Convolutional recurrent neural networks for polyphonic sound event detection10.1109/TASLP.2017.2690575 ↗
12. Cakir, E., Heittola, T., Huttunen, H., and Virtanen, T. (2015, January 12–16). Polyphonic sound event detection using multilabel deep neural networks. Proceedings of the International Joint Conference on Neural Networks (IJCNN), Killarney, Ireland.10.1109/IJCNN.2015.7280624 ↗
13. Robust sound event classification using deep neural networks10.1109/TASLP.2015.2389618 ↗
14. Bahdanau, D., Chorowski, J., Serdyuk, D., Brakel, P., and Bengio, Y. (2016, January 20–25). End-to-end attention-based large vocabulary speech recognition. Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Shaghai, China.10.1109/ICASSP.2016.7472618 ↗
15. Xu, Y., Kong, Q., Huang, Q., Wang, W., and Plumbley, M.D. (2017, January 20–24). Attention and localization based on a deep convolutional recurrent model for weakly supervised audio tagging. Proceedings of the International Conference on Spoken Language Processing (Interspeech), Stockholm, Sweden.10.21437/Interspeech.2017-486 ↗
16. Chorowski, K., Bahdanau, D., Serdyuk, D., Cho, K., and Bengio, Y. (2015, January 7–12). Attention-based models for speech recognition. Proceedings of the Advances in Neural Information Processing Systems, Montreal, QC, Canada.
17. Mnih, V., Heess, N., Graves, A., and Kavukcuoglu, K. (2014, January 8–13). Recurrent models of visual attention. Proceedings of the Advances in Neural Information Processing Systems, Montreal, QC, Canada.
18. Bahdanau, D., Cho, K., and Bengio, Y. (2014). Neural machine translation by jointly learning to align and translate. arXiv.
19. Harb, R., and Pernkopf, F. (2018). Sound event detection using weakly labeled semi-supervised data with GCRNNs, VAT and self-adaptive label refinement. arXiv.
20. JiaKai, L. (2018, January 19–20). Mean teacher convolution system for DCASE 2018 task 4. Proceedings of the Workshop on Detection and Classification of Acoustic Scenes and Events, Surrey, UK.
21. Tarvainen, A., and Valpola, H. (2017, January 4–9). Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results. Proceedings of the Advances in Neural Information Processing Systems, Long Beach, CA, USA.
22. Convolutional neural networks for speech recognition10.1109/TASLP.2014.2339736 ↗
23. Mesaros, A., Heittola, T., and Virtanen, T. (2016). Metrics for polyphonic sound event detection. Appl. Sci., 6.10.3390/app6060162 ↗
24. Serizel, R., Turpault, N., Eghbal-Zadeh, H., and Shah, A.P. (2018, January 19–20). Large-Scale Weakly Labeled Semi-Supervised Sound Event Detection in Domestic Environments. Proceedings of the Workshop on Detection and Classification of Acoustic Scenes and Events, Surrey, UK.