Deep Learning for Audio Event Detection and Tagging on Low-Resource Datasets
Source abstract
Abstract
In training a deep learning system to perform audio transcription, two practical problems may arise. Firstly, most datasets are weakly labelled, having only a list of events present in each recording without any temporal information for training. Secondly, deep neural networks need a very large amount of labelled training data to achieve good quality performance, yet in practice it is difficult to collect enough samples for most classes of interest. In this paper, we propose factorising the final task of audio transcription into multiple intermediate tasks in order to improve the training performance when dealing with this kind of low-resource datasets. We evaluate three data-efficient approaches of training a stacked convolutional and recurrent neural network for the intermediate tasks. Our results show that different methods of training have different advantages and disadvantages.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Music and Audio Processing · Speech Recognition and Synthesis · Speech and Audio Processing
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Machine Listening Lab, Centre for Digital Music (C4DM), Queen Mary University of London, London E1 4NS, UK
Funding
Engineering and Physical Sciences Research Council
References (39)
Show source-provided references
1. Deep Learning10.1038/nature14539 ↗
2. Choi, K., Fazekas, G., and Sandler, M.B. (2016, January 7–11). Automatic Tagging Using Deep Convolutional Neural Networks. Proceedings of the 17th International Society for Music Information Retrieval Conference (ISMIR), New York, NY, USA.
3. Unsupervised Feature Learning Based on Deep Models for Environmental Audio Tagging10.1109/TASLP.2017.2690563 ↗
4. Xu, Y., Kong, Q., Huang, Q., Wang, W., and Plumbley, M.D. (2017, January 14–19). Convolutional gated recurrent neural network incorporating spatial features for audio tagging. Proceedings of the 2017 International Joint Conference on Neural Networks (IJCNN), Anchorage, AK, USA.10.1109/IJCNN.2017.7966291 ↗
5. Adavanne, S., Drossos, K., Çakir, E., and Virtanen, T. (September, January 28). Stacked convolutional and recurrent neural networks for bird audio detection. Proceedings of the 2017 25th European Signal Processing Conference (EUSIPCO), Kos, Greece.10.23919/EUSIPCO.2017.8081505 ↗
6. Pons, J., Nieto, O., Prockup, M., Schmidt, E.M., Ehmann, A.F., and Serra, X. (2017, January 4–9). End-to-end learning for music audio tagging at scale. Presented at the Workshop Machine Learning for Audio Signal Processing at NIPS (ML4Audio), Long Beach, CA, USA.
7. Cakir, E., Heittola, T., Huttunen, H., and Virtanen, T. (2015, January 12–17). Polyphonic sound event detection using multi label deep neural networks. Proceedings of the 2015 International Joint Conference on Neural Networks (IJCNN), Killarney, Ireland.10.1109/IJCNN.2015.7280624 ↗
8. Parascandolo, G., Huttunen, H., and Virtanen, T. (2016, January 20–25). Recurrent neural networks for polyphonic sound event detection in real life recordings. Proceedings of the 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Shanghai, China.10.1109/ICASSP.2016.7472917 ↗
9. Lee, D., Lee, S., Han, Y., and Lee, K. (2017, January 16–17). Ensemble of Convolutional Neural Networks for Weakly-supervised Sound Event Detection Using Multiple Scale Input. Proceedings of the Detection and Classification of Acoustic Scenes and Events 2017 Workshop (DCASE 2017), Munich, Germany.
10. Adavanne, S., and Virtanen, T. (2017, January 16–17). Sound Event Detection Using Weakly Labeled Dataset with Stacked Convolutional and Recurrent Neural Network. Proceedings of the Detection and Classification of Acoustic Scenes and Events 2017 Workshop (DCASE 2017), Munich, Germany.
11. Acoustic classification of multiple simultaneous bird species: A multi-instance multi-label approach10.1121/1.4707424 ↗
12. Ruiz-Muñoz, J.F., Orozco-Alzate, M., and Castellanos-Dominguez, G. (2015, January 25–31). Multiple Instance Learning-based Birdsong Classification Using Unsupervised Recording Segmentation. Proceedings of the 24th International Conference on Artificial Intelligence (IJCAI’15), Buenos Aires, Argentina.
13. Fanioudakis, L., and Potamitis, I. (arXiv, 2017). Deep Networks tag the location of bird vocalisations on audio spectrograms, arXiv.
14. Schlüter, J. (2016, January 7–11). Learning to Pinpoint Singing Voice from Weakly Labeled Examples. Proceedings of the 17th International Society for Music Information Retrieval Conference (ISMIR 2016), New York, NY, USA.
15. Kong, Q., Xu, Y., Wang, W., and Plumbley, M.D. (2017, January 5–9). A joint detection-classification model for audio tagging of weakly labelled data. Proceedings of the 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), New Orleans, LA, USA.10.1109/ICASSP.2017.7952234 ↗
16. Hershey, S., Chaudhuri, S., Ellis, D.P.W., Gemmeke, J.F., Jansen, A., Moore, R.C., Plakal, M., Platt, D., Saurous, R.A., and Seybold, B. (2017, January 5–9). CNN architectures for large-scale audio classification. Proceedings of the 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), New Orleans, LA, USA.10.1109/ICASSP.2017.7952132 ↗
17. Kumar, A., and Raj, B. (2016, January 15–19). Audio Event Detection Using Weakly Labeled Data. Proceedings of the 2016 ACM on Multimedia Conference (MM’16), Amsterdam, The Netherlands.10.1145/2964284.2964310 ↗
18. Kumar, A., and Raj, B. (arXiv, 2017). Deep CNN Framework for Audio Event Recognition using Weakly Labeled Web Data, arXiv.10.1145/2964284.2964310 ↗
19. Yu, D., and Deng, L. (2016). Automatic Speech Recognition, Springer.10.1007/978-1-4471-5779-3 ↗
20. Speaker Diarization: A Review of Recent Research10.1109/TASL.2011.2125954 ↗
21. Garcia-Romero, D., Snyder, D., Sell, G., Povey, D., and McCree, A. (2017, January 5–9). Speaker diarization using deep neural network embeddings. Proceedings of the 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), New Orleans, LA, USA.10.1109/ICASSP.2017.7953094 ↗
22. Tirumala, S.S., and Shahamiri, S.R. (2016, January 21–24). A Review on Deep Learning Approaches in Speaker Identification. Proceedings of the 8th International Conference on Signal Processing Systems (ICSPS 2016), Auckland, New Zealand.10.1145/3015166.3015210 ↗
23. Solving the multiple instance problem with axis-parallel rectangles10.1016/S0004-3702(96)00034-3 ↗
24. Zhou, Z.H., and Zhang, M.L. (2002). Neural Networks for Multi-Instance Learning, Computer Science and Technology Department, Nanjing University. Technical Report, AI Lab.
25. Amar, R., Dooly, D.R., Goldman, S.A., and Zhang, Q. (July, January 28). Multiple-Instance Learning of Real-Valued Data. Proceedings of the Eighteenth International Conference on Machine Learning (ICML ’01), Williamstown, MA, USA.
26. Liu, D., Zhou, Y., Sun, X., Zha, Z., and Zeng, W. (2017, January 22–29). Adaptive Pooling in Multi-instance Learning for Web Video Annotation. Proceedings of the 2017 IEEE International Conference on Computer Vision Workshops (ICCVW), Venice, Italy.10.1109/ICCVW.2017.46 ↗
27. Wang, Y., Li, J., and Metze, F. (arXiv, 2018). Comparing the Max and Noisy-Or Pooling Functions in Multiple Instance Learning for Weakly Supervised Sequence Learning Tasks, arXiv.10.21437/Interspeech.2018-990 ↗
28. Multitask Learning10.1023/A:1007379606734 ↗
29. A Survey on Transfer Learning10.1109/TKDE.2009.191 ↗
30. A Review on Multi-Label Learning Algorithms10.1109/TKDE.2013.39 ↗
31. An overview of multi-task learning10.1093/nsr/nwx105 ↗
32. (2018, August 15). Neural Information Processing Scaled for Bioacoustics (NIPS4B) 2013 Bird Song Competition. Available online: http://sabiod.univ-tln.fr/nips4b/challenge1.html.
33. (2018, August 15). Transcriptions for the NIPS4B 2013 Bird Song Competition Training Set. Available online: https://figshare.com/articles/Transcriptions_of_NIPS4B_2013_Bird_Challenge_Training_Dataset/6798548.
34. Kingma, D.P., and Ba, J. (2015, January 7–9). Adam: A Method for Stochastic Optimization. Proceedings of the 3rd International Conference for Learning Representations, San Diego, CA, USA.
35. Lasseck, M. (2013, January 10). Bird song classification in field recordings: Winning solution for NIPS4B 2013 competition. Proceedings of the International Symposium Neural Information Scaled for Bioacoustics, Lake Tahoe, NV, USA.
36. Duong, L., Cohn, T., Bird, S., and Cook, P. (2015, January 27–29). Low Resource Dependency Parsing: Cross-lingual Parameter Sharing in a Neural Network Parser. Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing, Beijing, China.10.3115/v1/P15-2139 ↗
37. Misra, I., Shrivastava, A., Gupta, A., and Hebert, M. (2016, January 27–30). Cross-Stitch Networks for Multi-task Learning. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Vegas, NV, USA.10.1109/CVPR.2016.433 ↗
38. Yang, Y., and Hospedales, T. (2017, January 24–26). Trace Norm Regularised Deep Multi-Task Learning. Proceedings of the 5th International Conference on Learning Representations Workshop, Toulon, France.
39. Ruder, S., Bingel, J., Augenstein, I., and Søgaard, A. (arXiv, 2017). Sluice networks: Learning what to share between loosely related tasks, arXiv.