Enhancing Target Speech Based on Nonlinear Soft Masking Using a Single Acoustic Vector Sensor
Source abstract
Abstract
Enhancing speech captured by distant microphones is a challenging task. In this study, we investigate the multichannel signal properties of the single acoustic vector sensor (AVS) to obtain the inter-sensor data ratio (ISDR) model in the time-frequency (TF) domain. Then, the monotone functions describing the relationship between the ISDRs and the direction of arrival (DOA) of the target speaker are derived. For the target speech enhancement (SE) task, the DOA of the target speaker is given, and the ISDRs are calculated. Hence, the TF components dominated by the target speech are extracted with high probability using the established monotone functions, and then, a nonlinear soft mask of the target speech is generated. As a result, a masking-based speech enhancement method is developed, which is termed the AVS-SMASK method. Extensive experiments with simulated data and recorded data have been carried out to validate the effectiveness of our proposed AVS-SMASK method in terms of suppressing spatial speech interferences and reducing the adverse impact of the additive background noise while maintaining less speech distortion. Moreover, our AVS-SMASK method is computationally inexpensive, and the AVS is of a small physical size. These merits are favorable to many applications, such as robot auditory systems.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Speech and Audio Processing · Indoor and Outdoor Localization Technologies · Advanced Adaptive Filtering Techniques
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
ADSPLAB, School of Electronic Computer Engineering, Peking University Shenzhen Graduate School, Shenzhen 518055, China
School of Electrical, Computer, and Telecommunications Engineering, University of Wollongong, Wollongong, NSW 2500, Australia
References (27)
Show source-provided references
1. Yang, Y., Song, H., and Liu, J. (2010, January 23–25). Service robot speech enhancement method using acoustic micro-sensor array. Proceedings of the International Conference on Advanced Intelligence and Awareness Internet (IET), Beijing, China.
2. Gomez, R., Ivanchuk, L., Nakamura, K., Mizumoto, T., and Nakadai, K. (October, January 28). Utilizing visual cues in robot audition for sound source discrimination in speech-based human-robot communication. Proceedings of the IEEE International Conference on Intelligent Robots and Systems, Hamburg, Germany.
3. Development and validation of a robust speech interface for improved human-robot interaction10.1007/s12369-009-0032-4 ↗
4. Chen, M., Wang, L., Xu, C., and Li, R. (2015, January 10–12). A novel approach of system design for dialect speech interaction with NAO robot. Proceedings of the International Conference on Advanced Robotics (ICAR), Hong Kong, China.
5. Philipos, C., and Loizou, P.C. (2017). Speech Enhancement: Theory and Practice, CRC Press. [2nd ed.].
6. New insights into the noise reduction Wiener filter10.1109/TSA.2005.860851 ↗
7. Soft mask methods for single-channel speaker separation10.1109/TASL.2007.901310 ↗
8. Lightburn, L., De Sena, E., Moore, A., Naylor, P.A., and Brookes, M. (2017, January 5–9). Improving the perceptual quality of ideal binary masked speech. Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), New Orleans, LA, USA.10.1109/ICASSP.2017.7952238 ↗
9. Wang, Z., and Wang, D. (2018, January 15–20). Mask Weighted Stft Ratios for Relative Transfer Function Estimation and Its Application to Robust ASR. Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Calgary, AB, Canada.10.1109/ICASSP.2018.8461663 ↗
10. Xiao, X., Zhao, S., Jones, D.L., Chng, E.S., and Li, H. (2017, January 5–9). On time-frequency mask estimation for MVDR beamforming with application in robust speech recognition. Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), New Orleans, LA, USA.10.1109/ICASSP.2017.7952756 ↗
11. Heymann, J., Drude, L., and Haeb-Umbach, R. (2016, January 20–25). Neural network based spectral mask estimation for acoustic beamforming. Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Shanghai, China.10.1109/ICASSP.2016.7471664 ↗
12. Li, B., and Zou, Y.X. (2012, January 25–30). Improved DOA Estimation with Acoustic Vector Sensor Arrays Using Spatial Sparsity and Subarray Manifold. Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Kyoto, Japan.10.1109/ICASSP.2012.6288438 ↗
13. Zou, Y.X., Shi, W., Li, B., Ritz, C.H., Shujau, M., and Xi, J. (2013, January 26–31). Multisource DOA Estimation Based On Time-Frequency Sparsity and Joint Inter-Sensor Data Ratio with Single Acoustic Vector Sensor. Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Vancouver, BC, Canada.10.1109/ICASSP.2013.6638412 ↗
14. Zou, Y.X., Guo, Y., Zheng, W., Ritz, C.H., and Xi, J. (2014, January 9–13). An effective DOA estimation by exploring the spatial sparse representation of the inter-sensor data ratio model. Proceedings of the IEEE China Summit & International Conference on Signal and Information Processing (ChinaSIP), Xi’an, China.10.1109/ChinaSIP.2014.6889198 ↗
15. Zou, Y.X., Guo, Y., and Wang, Y.Q. (2014, January 13–16). A robust high-resolution speaker DOA estimation under reverberant environment. Proceedings of the International Symposium on Chinese Spoken Language Processing (ISCSLP), Singapore.
16. Zou, Y., Gu, R., Wang, D., Jiang, A., and Ritz, C.H. (2017, January 12–15). Learning a Robust DOA Estimation Model with Acoustic Vector Sensor Cues, In Proceedings the of Asia-Pacific Signal and Information Processing Association (APSIPA). Kuala Lumpur, Malaysia.10.1109/APSIPA.2017.8282304 ↗
17. Performance of Time- and Frequency-domain Binaural Beamformers Based on Recorded Signals from Real Rooms10.1121/1.1624064 ↗
18. Beamformer Performance with Acoustic Vector Sensors in Air10.1121/1.2139073 ↗
19. Shujau, M., Ritz, C.H., and Burnett, I.S. (2010, January 14–19). Speech Enhancement via Separation of Sources from Co-located Microphone Recordings. Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSAP), Dallas, TX, USA.10.1109/ICASSP.2010.5496124 ↗
20. Wu, P.K.T., Jin, C., and Kan, A. (September, January 30). A Multi-Microphone SPE Algorithm Tested Using Acoustic Vector Sensors. Proceedings of the International Workshop on Acoustic Echo and Noise Control, Tel-Aviv, Israel.
21. Zou, Y.X., Wang, P., Wang, Y.Q., Ritz, C.H., and Xi, J. (2014). Speech enhancement with an acoustic vector sensor: An effective adaptive beamforming and post-filtering approach. EURASIP J. Audio Speech Music Process, 17.10.1186/1687-4722-2014-17 ↗
22. Zou, Y.X., Wang, P., Wang, Y.Q., Ritz, C.H., and Xi, J. (2014, January 20–23). An effective target speech enhancement with single acoustic vector sensor based on the speech time-frequency sparsity. Proceedings of the 19th International Conference on Digital Signal Processing (DSP), Hong Kong, China.10.1109/ICDSP.2014.6900725 ↗
23. Distortion measures for speech processing10.1109/TASSP.1980.1163421 ↗
24. ITU-T (2001). 862-Perceptual Evaluation of Speech Quality (PESQ): An Objective Method for End-to-End Speech Quality Assessment of Narrow-Band Telephone Networks and Speech Codecs, International Telecommunication Union-Telecommunication Standardization Sector (ITU-T).
25. Hirsch, H.G., and Pearce, D. (2000, January 18–20). The AURORA experimental framework for the performance evaluations of speech recognition systems under noisy conditions. Proceedings of the Automatic Speech Recognition: Challenges for the Next Millennium, Paris, France.
26. Shujau, M., Ritz, C.H., and Burnett, I.S. (2011, January 19–22). Separation of speech sources using an Acoustic Vector Sensor. Proceedings of the IEEE International Workshop on Multimedia Signal Processing, Hangzhou, China.10.1109/MMSP.2011.6093797 ↗
27. IEEE Recommended Practice for Speech Quality Measurements10.1109/TAU.1969.1162058 ↗