A Multi-Source Separation Approach Based on DOA Cue and DNN
Source abstract
Abstract
Multiple sound source separation in a reverberant environment has become popular in recent years. To improve the quality of the separated signal in a reverberant environment, a separation method based on a DOA cue and a deep neural network (DNN) is proposed in this paper. Firstly, a pre-processing model based on non-negative matrix factorization (NMF) is utilized for recorded signal dereverberation, which makes source separation more efficient. Then, we propose a multi-source separation algorithm combining sparse and non-sparse component points recovery to obtain each sound source signal from the dereverberated signal. For sparse component points, the dominant sound source for each sparse component point is determined by a DOA cue. For non-sparse component points, a DNN is used to recover each sound source signal. Finally, the signals separated from the sparse and non-sparse component points are well matched by temporal correlation to obtain each sound source signal. Both objective and subjective evaluation results indicate that compared with the existing method, the proposed separation approach shows a better performance in the case of a high-reverberation environment.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Speech and Audio Processing · Blind Source Separation Techniques · Advanced Adaptive Filtering Techniques
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Faculty of Information Technology, Beijing University of Technology, Beijing 100124, China
National Space Science Center, The Chinese Academy of Sciences, Beijing 100190, China
Department of Computer Science and Information Engineering, National Taipei University of Technology, Taipei 106344, Taiwan
Funding
Cooperative Research Project of BJUT-NTUT
References (36)
Show source-provided references
1. Bahdanau, D., Chorowski, J., Serdyuk, D., Brakel, P., and Bengio, Y. (2016, January 20–25). End-to-end attention-based large vocabulary speech recognition. Proceedings of the 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Shanghai, China.10.1109/ICASSP.2016.7472618 ↗
2. Petridis, S., and Pantic, M. (2016, January 20–25). Deep complementary bottleneck features for visual speech recognition. Proceedings of the 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Shanghai, China.10.1109/ICASSP.2016.7472088 ↗
3. Binary Sparse Coding of Convolutive Mixtures for Sound Localization and Separation via Spatialization10.1109/TSP.2015.2488598 ↗
4. Mysore, G.J., Smaragdis, P., and Raj, B. (2010, January 27–30). Non-negative hidden Markov modeling of audio with application to source separation. Proceedings of the 9th International Conference on Latent Variable Analysis and Signal Separation (LVA/ICA’10), St. Malo, France.10.1007/978-3-642-15995-4_18 ↗
5. Rennie, S.J., Hershey, J.R., and Olsen, P.A. (2009, January 19–24). Single channel speech separation and recognition using loopy belief propagation. Proceedings of the 2009 IEEE International Conference on Acoustics, Speech and Signal Processing, Taipei, Taiwan.10.1109/ICASSP.2009.4960466 ↗
6. Computational Models of Auditory Scene Analysis: A Review
7. Informed Audio Source Separation Using Linearly Constrained Spatial Filters10.1109/TASL.2012.2208629 ↗
8. Multi-speaker LCMV Beamformer and Postfilter for Source Separation and Noise Reduction10.1109/TASLP.2017.2655258 ↗
9. Liu, J., Yang, Q., Jia, M., and Zhang, X. (November, January 30). Multiple Sound Source Separation by Jointing Single Source Zone Detection and Linearly Constrained Minimum Variance. Proceedings of the 2020 9th International Conference on Computing and Pattern Recgnition (ICCPR), Xiamen, China.10.1145/3436369.3437435 ↗
10. Comon, P., and Jutten, C. (2010). Handbook of Blind Source Separation: Independent Component Analysis and Applications, Academic Press.
11. Blind separation of speech mixtures via time-frequency masking10.1109/TSP.2004.828896 ↗
12. Loesch, B., and Yang, B. (2008, January 14–17). Source number estimation and clustering for underdetermined blind source separation. Proceedings of the International Workshop on Acoustic Echo and Noise Control (IWAENC), Seattle, WA, USA.
13. Bosi, M., and Goldberg, R.E. (2003). Introduction to Digital Audio Coding and Standards, Kluwer Academic Publishers.10.1007/978-1-4615-0327-9 ↗
14. Separation of Multiple Speech Sources by Recovering Sparse and Non-Sparse Components from B-Format Microphone Recordings10.1016/j.specom.2017.12.010 ↗
15. On training targets for supervised speech separation10.1109/TASLP.2014.2352935 ↗
16. Huang, P.-S., Kim, M., Hasegawa-Johnson, M., and Smaragdis, P. (2014, January 4–9). Deep learning for monaural speech separation. Proceedings of the 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Florence, Italy.10.1109/ICASSP.2014.6853860 ↗
17. Supervised Speech Separation Based on Deep Learning: An Overview10.1109/TASLP.2018.2842159 ↗
18. Multichannel Audio Source Separation with Deep Neural Networks10.1109/TASLP.2016.2580946 ↗
19. Two-Stage Monaural Source Separation in Reverberant Room Environments Using Deep Neural Networks10.1109/TASLP.2018.2874708 ↗
20. Deep Learning for Talker-Dependent Reverberant Speaker Separation: An Empirical Study10.1109/TASLP.2019.2934319 ↗
21. Switching Divergences for Spectral Learning in Blind Speech Dereverberation10.1109/TASLP.2019.2901643 ↗
22. Jia, X., Jia, M., Li, L., and Jia, Y. (2020, January 18–20). Multiple sound source localization by using diffuseness estimation. Proceedings of the 2020 7th International Conference on Information Science and Control Engineering (ICISCE), Changsha, China.10.1109/ICISCE50968.2020.00181 ↗
23. Real-time Multiple Sound Source Localization and Counting Using A Soundfield Microphone10.1007/s12652-016-0388-x ↗
24. Jia, X., Jia, M., Gao, S., and Zhang, Y. (2021, January 8–10). Multi-source separation based on non-negative matrix factorization and source distribution. Proceedings of the 2021 Immersive and 3D Audio: From Architecture to Automotive (I3DA), Bologna, Italy.10.1109/I3DA48870.2021.9610860 ↗
25. A generalized divergence measure for nonnegative matrix factorization10.1162/neco.2007.19.3.780 ↗
26. A Matlab Simulation of “shoebox” Room Acoustics for Use in Research and Teaching
27. A bayesian approach to convolutive nonnegative matrix factorization for blind speech dereverberation10.1016/j.sigpro.2018.04.024 ↗
28. Algorithms for nonnegative matrix factorization with the beta-divergence
29. Shujau, M., Ritz, C.H., and Burnett, I.S. (2011, January 17–19). Separation of speech sources using an Acoustic Vector Sensor. Proceedings of the 2011 IEEE 13th International Workshop on Multimedia Signal Processing (MMSP), Hangzhou, China.10.1109/MMSP.2011.6093797 ↗
30. Jia, M., and Sun, J. (2018). Multiple Speech Source Separation Using Inter-Channel Correlation and Relaxed Sparsity. Appl. Sci.-Basel, 8.10.3390/app8010123 ↗
31. Acoustic Source Separation of Convolutive Mixtures Based on Intensity Vector Statistics10.1109/TASL.2008.918967 ↗
32. Spatial Sound Reproduction with Directional Audio Coding
33. (2005). Wideband Extension to Recommendation P.862 for the Assessment of Wideband Telephone Networks and Speech Codecs, International Telecommunication Union. ITU-T Rec P.862.2.
34. Taal, C.H., Hendriks, R.C., Heusdens, R., and Jensen, J. (2010, January 14–19). A short-time objective intelligibility measure for time-frequency weighted noisy speech. Proceedings of the 2010 IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), Dallas, TX, USA.10.1109/ICASSP.2010.5495701 ↗
35. (1997). Method for the Subjective Assessment of Intermediate Quality Levels of Coding Systems. Standard No. BS.1534.
36. Reverberant speech separation with probabilistic time-frequency masking for B-format recordings10.1016/j.specom.2015.01.002 ↗