Robust Detection of Background Acoustic Scene in the Presence of Foreground Speech
Source abstract
Abstract
The characterising sound required for the Acoustic Scene Classification (ASC) system is contained in the ambient signal. However, in practice, this is often distorted by e.g., foreground speech of the speakers in the surroundings. Previously, based on the iVector framework, we proposed different strategies to improve the classification accuracy when foreground speech is present. In this paper, we extend these methods to deep-learning (DL)-based ASC systems, for improving foreground speech robustness. ResNet models are proposed as the baseline, in combination with multi-condition training at different signal-to-background ratios (SBRs). For further robustness, we first investigate the noise-floor-based Mel-FilterBank Energies (NF-MFBE) as the input feature of the ResNet model. Next, speech presence information is incorporated within the ASC framework obtained from a speech enhancement (SE) system. As the speech presence information is time-frequency specific, it allows the network to learn to distinguish better between background signal regions and foreground speech. While the proposed modifications improve the performance of ASC systems when foreground speech is dominant, in scenarios with low-level or absent foreground speech, performance is slightly worse. Therefore, as a last consideration, ensemble methods are introduced, to integrate classification scores from different models in a weighted manner. The experimental study systematically validates the contribution of each proposed modification and, for the final system, it is shown that with the proposed input features and meta-learner, the classification accuracy is improved in all tested SBRs. Especially for SBRs of 20 dB, absolute improvements of up to 9% can be obtained.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Speech and Audio Processing · Music and Audio Processing · Speech Recognition and Synthesis
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
IDLab, Department of Electronics and Information Systems, Ghent University—imec, 9000 Ghent, Belgium
References (30)
Show source-provided references
1. Acoustic Scene Classification: Classifying environments from the sounds they produce10.1109/MSP.2014.2326181 ↗
2. Eronen, A., Tuomi, J., Klapuri, A., Fagerlund, S., Sorsa, T., Lorho, G., and Huopaniemi, J. (2003, January 6–10). Audio-based context awareness-acoustic modeling and perceptual evaluation. Proceedings of the 2003 IEEE International Conference on Acoustics, Speech, and Signal Processing, 2003. Proceedings. (ICASSP ’03), Hong Kong, China.10.1109/ASPAA.2003.1285814 ↗
3. Audio-based context recognition10.1109/TSA.2005.854103 ↗
4. Elizalde, B., Lei, H., Friedland, G., and Peters, N. (April, January 31). An i-vector based approach for audio scene detection. Proceedings of the IEEE AASP Challenge on Detection and Classification of Acoustic Scenes and Events, Online.
5. Front-End Factor Analysis for Speaker Verification10.1109/TASL.2010.2064307 ↗
6. Mesaros, A., Heittola, T., and Virtanen, T. (September, January 29). TUT database for acoustic scene classification and sound event detection. Proceedings of the 2016 24th European Signal Processing Conference (EUSIPCO), Budapest, Hungary.10.1109/EUSIPCO.2016.7760424 ↗
7. Mesaros, A., Heittola, T., and Virtanen, T. (2018, January 19–20). A multi-device dataset for urban acoustic scene classification. Proceedings of the Detection and Classification of Acoustic Scenes and Events 2018 Workshop (DCASE 2018), Surrey, UK.
8. Abeßer, J. (2020). A Review of Deep Learning Based Methods for Acoustic Scene Classification. Appl. Sci., 10.10.3390/app10062020 ↗
9. Valenti, M., Squartini, S., Diment, A., Parascandolo, G., and Virtanen, T. (2017, January 14–19). A convolutional neural network approach for acoustic scene classification. Proceedings of the 2017 International Joint Conference on Neural Networks (IJCNN), Anchorage, AK, USA.10.1109/IJCNN.2017.7966035 ↗
10. Seo, H., Park, J., and Park, Y. (2019, January 25–26). Acoustic scene classification using various pre-processed features and convolutional neural networks. Proceedings of the Detection and Classification of Acoustic Scenes and Events Workshop (DCASE 2019), New York, NY, USA.
11. Jallet, H., Cakir, E., and Virtanen, T. (2017, January 16). Acoustic Scene Classification Using CRNN. Proceedings of the DCASE2017 Challenge, Munich, Germany.
12. Mulimani, M., and Koolagudi, S.G. (2021, January 2–4). Acoustic scene classification using deep learning architectures. Proceedings of the 2021 6th International Conference for Convergence in Technology (I2CT), Maharashtra, India.
13. Bisot, V., Serizel, R., Essid, S., and Richard, G. (2017, January 16). Nonnegative Feature Learning Methods for Acoustic Scene Classification. Proceedings of the DCASE2017 Challenge, Munich, Germany.
14. Takahashi, G., Yamada, T., Ono, N., and Makino, S. (2017, January 12–15). Performance evaluation of acoustic scene classification using DNN-GMM and frame-concatenated acoustic features. Proceedings of the 2017 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC), Kuala Lumpur, Malaysia.10.1109/APSIPA.2017.8282314 ↗
15. Song, S., Desplanques, B., De Moor, C., Demuynck, K., and Madhu, N. (2021, January 23–27). Robust Acoustic Scene Classification in the Presence of Active Foreground Speech. Proceedings of the 2021 29th European Signal Processing Conference (EUSIPCO), Dublin, Ireland.10.23919/EUSIPCO54536.2021.9615984 ↗
16. Song, S., Desplanques, B., Demuynck, K., and Madhu, N. (September, January 29). SoftVAD in iVector-Based Acoustic Scene Classification for Robustness to Foreground Speech. Proceedings of the 2022 30th European Signal Processing Conference (EUSIPCO), Belgrade, Serbia.10.23919/EUSIPCO55093.2022.9909938 ↗
17. DNN-Based Cepstral Excitation Manipulation for Speech Enhancement10.1109/TASLP.2019.2933698 ↗
18. Liu, S., Triantafyllopoulos, A., Ren, Z., and Schuller, B.W. (2020, January 25–29). Towards Speech Robustness for Acoustic Scene Classification. Proceedings of the Interspeech 2020, Shanghai, China.10.21437/Interspeech.2020-2365 ↗
19. Noise power spectral density estimation based on optimal smoothing and minimum statistics10.1109/89.928915 ↗
20. A noise-estimation algorithm for highly non-stationary environments10.1016/j.specom.2005.08.005 ↗
21. Unbiased MMSE-Based Noise Power Estimation With Low Complexity and Low Tracking Delay10.1109/TASL.2011.2180896 ↗
22. Byttebier, L., Desplanques, B., Thienpondt, J., Song, S., Demuynck, K., and Madhu, N. (2021, January 15–19). Small-footprint acoustic scene classification through 8-bit quantization-aware training and pruning of ResNet models. Proceedings of the DCASE2021 Challenge, Online.
23. Braun, S., Gamper, H., Reddy, C.K., and Tashev, I. (2021, January 6–11). Towards Efficient Models for Real-Time Deep Noise Suppression. Proceedings of the ICASSP 2021—2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Toronto, ON, Canada.10.1109/ICASSP39728.2021.9413580 ↗
24. Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods
25. Guo, C., Pleiss, G., Sun, Y., and Weinberger, K.Q. (2017, January 6–11). On Calibration of Modern Neural Networks. Proceedings of the 34th International Conference on Machine Learning, Sydney, Australia.
26. Reddy, C.K.A., Dubey, H., Gopal, V., Cutler, R., Braun, S., Gamper, H., Aichner, R., and Srinivasan, S. (2021, January 6–11). ICASSP 2021 Deep Noise Suppression Challenge. Proceedings of the ICASSP 2021—2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Toronto, ON, Canada.10.1109/ICASSP39728.2021.9415105 ↗
27. Huang, J., Lu, H., Lopez Meyer, P., Cordourier, H., and Del Hoyo Ontiveros, J. (2019, January 25–26). Acoustic Scene Classification Using Deep Learning-based Ensemble Averaging. Proceedings of the Workshop on Detection and Classification of Acoustic Scenes and Events 2019 (DCASE 2019), New York, NY, USA.10.33682/8rd2-g787 ↗
28. A novel acoustic scene classification model using the late fusion of convolutional neural networks and different ensemble classifiers10.1016/j.apacoust.2020.107829 ↗
29. Panayotov, V., Chen, G., Povey, D., and Khudanpur, S. (2015, January 19–24). Librispeech: An ASR corpus based on public domain audio books. Proceedings of the 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), South Brisbane, QLD, Australia.10.1109/ICASSP.2015.7178964 ↗
30. Smith, L.N. (2017, January 24–31). Cyclical Learning Rates for Training Neural Networks. Proceedings of the 2017 IEEE Winter Conference on Applications of Computer Vision (WACV), Santa Rosa, CA, USA.10.1109/WACV.2017.58 ↗