Data-Dependent Feature Extraction Method Based on Non-Negative Matrix Factorization for Weakly Supervised Domestic Sound Event Detection
Source abstract
Abstract
In this paper, feature extraction methods are developed based on the non-negative matrix factorization (NMF) algorithm to be applied in weakly supervised sound event detection. Recently, the development of various features and systems have been attempted to tackle the problems of acoustic scene classification and sound event detection. However, most of these systems use data-independent spectral features, e.g., Mel-spectrogram, log-Mel-spectrum, and gammatone filterbank. Some data-dependent feature extraction methods, including the NMF-based methods, recently demonstrated the potential to tackle the problems mentioned above for long-term acoustic signals. In this paper, we further develop the recently proposed NMF-based feature extraction method to enable its application in weakly supervised sound event detection. To achieve this goal, we develop a strategy for training the frequency basis matrix using a heterogeneous database consisting of strongly- and weakly-labeled data. Moreover, we develop a non-iterative version of the NMF-based feature extraction method so that the proposed feature extraction method can be applied as a part of the model structure similar to the modern “on-the-fly” transform method for the Mel-spectrogram. To detect the sound events, the temporal basis is calculated using the NMF method and then used as a feature for the mean-teacher-model-based classifier. The results are improved for the event-wise post-processing method. To evaluate the proposed system, simulations of the weakly supervised sound event detection were conducted using the Detection and Classification of Acoustic Scenes and Events 2020 Task 4 database. The results reveal that the proposed system has F1-score performance comparable with the Mel-spectrogram and gammatonegram and exhibits 3–5% better performance than the log-Mel-spectrum and constant-Q transform.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Music and Audio Processing · Speech and Audio Processing · Music Technology and Sound Studies
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
School of Electronic and Electrical Engineering, Kyungpook National University, Daegu 41566, Korea
School of Electronics Engineering, Kyungpook National University, Daegu 41566, Korea
Media Research Division, Electronics and Telecommunications Research Institute, Daejeon 34129, Korea
Funding
Institue for Information & Communications Technology Promotion (IITP)
References (46)
Show source-provided references
1. Chu, S., Narayanan, S., Kuo, C.C.J., and Mataric, M.J. (2006, January 9–12). Where am I? Scene recognition for mobile robots using audio features. Proceedings of the 2006 IEEE International Conference on Multimedia and Expo, Toronto, Canada.10.1109/ICME.2006.262661 ↗
2. Ellis, D.P., and Lee, K. (2004, January 15). Minimal-impact audio-based personal archives. Proceedings of the the 1st ACM Workshop on Continuous Archival and Retrieval of Personal Experiences, New York, NY, USA.10.1145/1026653.1026659 ↗
3. Acoustic scene classification: Classifying environments from the sounds they produce10.1109/MSP.2014.2326181 ↗
4. Mesaros, A., Heittola, T., and Virtanen, T. (2018, January 19–20). A multi-device dataset for urban acoustic scene classification. Proceedings of the Detection and Classification of Acoustic Scenes and Events 2018 Workshop (DCASE2018), Surrey, UK.
5. Feature learning with matrix factorization applied to acoustic scene classification10.1109/TASLP.2017.2690570 ↗
6. Convolutional recurrent neural networks for polyphonic sound event detection10.1109/TASLP.2017.2690575 ↗
7. Huang, Z., and Jiang, D. (2019). Acoustic Scene Classification Based on Deep Convolutional Neuralnetwork with Spatial-Temporal Attention Pooling, DCASE Community. Technical Report; DCASE2019 Challenge.
8. Liu, H., Wang, F., Liu, X., and Guo, D. (2018). An Ensemble System for Domestic Activity Recognition, DCASE Community. Technical Report; DCASE2018 Challenge.
9. Chen, H., Liu, Z., Liu, Z., Zhang, P., and Yan, Y. (2019). Integrating the Data Augmentation Scheme with Various Classifiers for Acoustic Scene Modeling, DCASE Community. Technical Report; DCASE2019 Challenge.
10. Inoue, T., Vinayavekhin, P., Wang, S., Wood, D., Greco, N., and Tachibana, R. (2018). Domestic Activities Classification Based on CNN Using Shuffling and Mixing Data Augmentation, DCASE Community. Technical Report; DCASE2018 Challenge.
11. Valenti, M., Squartini, S., Diment, A., Parascandolo, G., and Virtanen, T. (2017, January 14–19). A convolutional neural network approach for acoustic scene classification. Proceedings of the 2017 International Joint Conference on Neural Networks (IJCNN), Anchorage, AK, USA.10.1109/IJCNN.2017.7966035 ↗
12. Moore, B.C. (2012). An Introduction to the Psychology of Hearing, Brill Academy Press.
13. Bisot, V., Essid, S., and Richard, G. (September, January 31). HOG and subband power distribution image features for acoustic scene classification. Proceedings of the 2015 23rd European Signal Processing Conference (EUSIPCO), Nice, France.10.1109/EUSIPCO.2015.7362477 ↗
14. Roma, G., Nogueira, W., Herrera, P., and De Boronat, R. (2013). Recurrence quantification analysis features for auditory scene classification. IEEE Aasp Chall. Detect. Classif. Acoust. Scenes Events.10.1109/WASPAA.2013.6701890 ↗
15. Gowdy, J.N., and Tufekci, Z. (2000, January 5–9). Mel-scaled discrete wavelet coefficients for speech recognition. Proceedings of the 2000 IEEE International Conference on Acoustics, Speech, and Signal Processing, (Cat. No. 00CH37100), Istanbul, Turkey.
16. Waldekar, S., and Saha, G. (2018, January 2–6). Wavelet Transform Based Mel-scaled Features for Acoustic Scene Classification. Proceedings of the Interspeech 2018, Hyderabad, India.10.21437/Interspeech.2018-2083 ↗
17. Waldekar, S., Kumar A, K., and Saha, G. (2020). Mel-Scaled Wavelet-Based Features for Sub-Task A and Texture Features for Sub-Task B of DCASE 2020 Task 1, DCASE Community. Technical Report; DCASE2020 Challenge.
18. Phan, H., Koch, P., Katzberg, F., Maass, M., Mazur, R., and Mertins, A. (2017). Audio scene classification with deep recurrent neural networks. arXiv.10.21437/Interspeech.2017-101 ↗
19. Gammatone cepstral coefficients: Biologically inspired features for non-speech audio classification10.1109/TMM.2012.2199972 ↗
20. Principal component analysis10.1002/wics.101 ↗
21. Learning the parts of objects by non-negative matrix factorization10.1038/44565 ↗
22. Smaragdis, P., and Brown, J.C. (2003, January 19–22). Non-negative matrix factorization for polyphonic music transcription. Proceedings of the 2003 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics, New Paltz, NY, USA.
23. Enforcing harmonicity and smoothness in Bayesian non-negative matrix factorization applied to polyphonic music transcription10.1109/TASL.2010.2041381 ↗
24. Automatic music transcription: Challenges and future directions10.1007/s10844-013-0258-3 ↗
25. Wilson, K.W., Raj, B., Smaragdis, P., and Divakaran, A. (April, January 30). Speech denoising using nonnegative matrix factorization with priors. Proceedings of the 2008 IEEE International Conference on Acoustics, Speech and Signal Processing, Las Vegas, NV, USA.10.1109/ICASSP.2008.4518538 ↗
26. NMF-based speech enhancement using bases update10.1109/LSP.2014.2362556 ↗
27. Fan, H.T., Hung, J.w., Lu, X., Wang, S.S., and Tsao, Y. (2014, January 4–9). Speech enhancement using segmental nonnegative matrix factorization. Proceedings of the 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Florence, Italy.10.1109/ICASSP.2014.6854450 ↗
28. Cauchi, B. (2011). Non-Negative Matrix Factorisation Applied to Auditory Scenes Classification. [Master’s Thesis, Master ATIAM, Université Pierre et Marie Curie].
29. Bisot, V., Serizel, R., Essid, S., and Richard, G. (2016, January 20–25). Acoustic scene classification with matrix factorization for unsupervised feature learning. Proceedings of the 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Shanghai, China.10.1109/ICASSP.2016.7472918 ↗
30. On the limited memory BFGS method for large scale optimization10.1007/BF01589116 ↗
31. Feature Extraction Based on the Non-Negative Matrix Factorization of Convolutional Neural Networks for Monitoring Domestic Activity With Acoustic Signals10.1109/ACCESS.2020.3007199 ↗
32. Choi, K., Joo, D., and Kim, J. (2017). Kapre: On-gpu audio preprocessing layers for a quick implementation of deep neural network models with keras. arXiv.
33. nnAudio: An on-the-Fly GPU Audio to Spectrogram Conversion Toolbox Using 1D Convolutional Neural Networks10.1109/ACCESS.2020.3019084 ↗
34. Turpault, N., Serizel, R., Parag Shah, A., and Salamon, J. (2019, January 25–26). Sound event detection in domestic environments with weakly labeled data and soundscape synthesis. Proceedings of the Detection and Classification of Acoustic Scenes and Events 2019 Workshop, New York, NY, USA.10.33682/006b-jx26 ↗
35. Lee, D.D., and Seung, H.S. (2001). Algorithms for non-negative matrix factorization. Advances in Neural Information Processing Systems, MIT Press.
36. Cichocki, A., Zdunek, R., Phan, A.H., and Amari, S.I. (2009). Nonnegative Matrix and Tensor Factorizations: Applications to Exploratory Multi-Way Data Analysis and Blind Source Separation, John Wiley & Sons.10.1002/9780470747278 ↗
37. Reverberation suppression using non-negative matrix factorization to detect low-Doppler target with continuous wave active sonar10.1186/s13634-019-0608-6 ↗
38. Nonnegative matrix factorization with the Itakura-Saito divergence: With application to music analysis10.1162/neco.2008.04-08-771 ↗
39. Ben-Israel, A., and Greville, T.N. (2003). Generalized Inverses: Theory and Applications, Springer Science & Business Media.
40. Delphin-Poulat, L., and Plapous, C. (2019). Mean Teacher with Data Augmentation for Dcase 2019 Task 4, Orange Labs. Technical Report.
41. Jiakai, L. (2018). Mean Teacher Convolution System for Dcase 2018 Task 4, DCASE Community. Technical Report; DCASE2018 Challenge.
42. Tarvainen, A., and Valpola, H. (2017). Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results. arXiv.
43. Gemmeke, J.F., Ellis, D.P.W., Freedman, D., Jansen, A., Lawrence, W., Moore, R.C., Plakal, M., and Ritter, M. (2017). Audio Set: An ontology and human-labeled dataset for audio events. Proc. IEEE ICASSP.10.1109/ICASSP.2017.7952261 ↗
44. Fonseca, E., Pons, J., Favory, X., Font, F., Bogdanov, D., Ferraro, A., Oramas, S., Porter, A., and Serra, X. (2017, January 23–27). Freesound Datasets: A platform for the creation of open audio datasets. Proceedings of the 18th International Society for Music Information Retrieval Conference (ISMIR 2017), Suzhou, China.
45. Dekkers, G., Lauwereins, S., Thoen, B., Adhana, M.W., Brouckxon, H., Van Waterschoot, T., Vanrumste, B., Verhelst, M., and Karsmakers, P. (2017, January 16–17). The SINS Database for Detection of Daily Activities in a Home Environment Using an Acoustic Sensor Network. Proceedings of the Detection and Classification of Acoustic Scenes and Events 2017 Workshop (DCASE2017), Munich, Germany.
46. Cornell, S.C., Pepe, G., Principi, E., Pariente, M., Olvera, M., Gabrielli, L., and Squartini, S. (2020). The UNIVPM-INRIA Systems for The DCASE 2020 Task 4, DCASE Community. Technical Report; DCASE2020 Challenge.