Hybrid Dilated and Recursive Recurrent Convolution Network for Time-Domain Speech Enhancement
Source abstract
Abstract
In this paper, we propose a fully convolutional neural network based on recursive recurrent convolution for monaural speech enhancement in the time domain. The proposed network is an encoder-decoder structure using a series of hybrid dilated modules (HDM). The encoder creates low-dimensional features of a noisy input frame. In the HDM, the dilated convolution is used to expand the receptive field of the network model. In contrast, the standard convolution is used to make up for the under-utilized local information of the dilated convolution. The decoder is used to reconstruct enhanced frames. The recursive recurrent convolutional network uses GRU to solve the problem of multiple training parameters and complex structures. State-of-the-art results are achieved on two commonly used speech datasets.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Speech and Audio Processing · Speech Recognition and Synthesis · Music and Audio Processing
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
School of Electronics and Information, Northwestern Polytechnical University, Xi’an 710129, China
College of Computer and Cyber Security, Hebei Normal University, Shijiazhuang 050024, China
Funding
National Natural Science Foundation of China · Key Research and Development Program of Shaanxi · International Science and Technology Cooperation Research Project of Shenzhen
References (48)
Show source-provided references
1. Suppression of acoustic noise in speech using spectral subtraction10.1109/TASSP.1979.1163209 ↗
2. Speech enhancement using a minimum-mean square error short-time spectral amplitude estimator10.1109/TASSP.1984.1164453 ↗
3. A signal subspace approach for speech enhancement10.1109/89.397090 ↗
4. Wang, D. (2005). On ideal binary mask as the computational goal of auditory scene analysis. Speech Separation by Humans and Machines, Springer.10.1007/0-387-22794-6_12 ↗
5. On training targets for supervised speech separation10.1109/TASLP.2014.2352935 ↗
6. A regression approach to speech enhancement based on deep neural networks10.1109/TASLP.2014.2364452 ↗
7. Tan, K., and Wang, D. (2018, January 2–6). A Convolutional Recurrent Neural Network for Real-Time Speech Enhancement. Proceedings of the Interspeech, Hyderabad, India.10.21437/Interspeech.2018-1405 ↗
8. Zhao, H., Zarar, S., Tashev, I., and Lee, C.H. (2018, January 15–20). Convolutional-recurrent neural networks for speech enhancement. Proceedings of the 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Calgary, AB, Canada.10.1109/ICASSP.2018.8462155 ↗
9. The importance of phase in speech enhancement10.1016/j.specom.2010.12.003 ↗
10. Pascual, S., Bonafonte, A., and Serra, J. (2017). SEGAN: Speech enhancement generative adversarial network. arXiv.10.21437/Interspeech.2017-1428 ↗
11. On loss functions for supervised monaural time-domain speech enhancement10.1109/TASLP.2020.2968738 ↗
12. Abdulbaqi, J., Gu, Y., Chen, S., and Marsic, I. (2020, January 4–8). Residual Recurrent Neural Network for Speech Enhancement. Proceedings of the ICASSP 2020—2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Barcelona, Spain.10.1109/ICASSP40776.2020.9053544 ↗
13. Fu, S.W., Tsao, Y., Lu, X., and Kawai, H. (2017, January 12–15). Raw waveform-based speech enhancement by fully convolutional networks. Proceedings of the 2017 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC), Kuala Lumpur, Malaysia.10.1109/APSIPA.2017.8281993 ↗
14. Oord, A.V.D., Dieleman, S., Zen, H., Simonyan, K., Vinyals, O., Graves, A., Kalchbrenner, N., Senior, A., and Kavukcuoglu, K. (2016). Wavenet: A generative model for raw audio. arXiv.
15. Rethage, D., Pons, J., and Serra, X. (2018, January 15–20). A wavenet for speech denoising. Proceedings of the 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Calgary, AB, Canada.10.1109/ICASSP.2018.8462417 ↗
16. Pandey, A., and Wang, D. (2019, January 12–17). TCNN: Temporal convolutional neural network for real-time speech enhancement in the time domain. Proceedings of the ICASSP 2019—2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Brighton, UK.10.1109/ICASSP.2019.8683634 ↗
17. Ren, D., Zuo, W., Hu, Q., Zhu, P., and Meng, D. (2019, January 15–20). Progressive Image Deraining Networks: A Better and Simpler Baseline. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA.10.1109/CVPR.2019.00406 ↗
18. Tai, Y., Yang, J., and Liu, X. (2017, January 21–26). Image Super-Resolution via Deep Recursive Residual Network. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.10.1109/CVPR.2017.298 ↗
19. Berouti, M., Schwartz, R., and Makhoul, J. (1979, January 2–4). Enhancement of speech corrupted by acoustic noise. Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP’79), Washington, DC, USA.10.1109/ICASSP.1979.1170788 ↗
20. A parametric formulation of the generalized spectral subtraction method10.1109/89.701361 ↗
21. Kamath, S.D., and Loizou, P.C. (2002, January 13–17). A multi-band spectral subtraction method for enhancing speech corrupted by colored noise. Proceedings of the ICASSP international Conference on Acoustics Speech and Signal Processing, Orlando, FL, USA.10.1109/ICASSP.2002.5745591 ↗
22. Sovka, P., Pollak, P., and Kybic, J. (1996, January 10–13). Extended Spectral Subtraction. Proceedings of the 8th European Signal Processing Conference, Trieste, Italy.
23. Relaxed statistical model for speech enhancement and a priori SNR estimation10.1109/TSA.2005.851940 ↗
24. Speech enhancement for non-stationary noise environments10.1016/S0165-1684(01)00128-1 ↗
25. A modified a priori SNR for speech enhancement using spectral subtraction rules10.1109/LSP.2004.824017 ↗
26. Elimination of the musical noise phenomenon with the Ephraim and Malah noise suppressor10.1109/89.279283 ↗
27. Li, A., Zheng, C., Cheng, L., Peng, R., and Li, X. (2020, January 7–10). A Time-domain Monaural Speech Enhancement with Feedback Learning. Proceedings of the 2020 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC), Auckland, New Zealand.
28. Speech Enhancement Using Deep Learning Methods: A Review10.14203/jet.v21.19-26 ↗
29. Yan, Z., Xu, B., Giri, R., and Tao, Z. (2018, January 15–20). Perceptually Guided Speech Enhancement Using Deep Neural Networks. Proceedings of the ICASSP 2018—2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Calgary, AB, Canada.
30. Karjol, P., Ajay Kumar, M., and Ghosh, P.K. (2018, January 15–20). Speech Enhancement Using Multiple Deep Neural Networks. Proceedings of the 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Calgary, AB, Canada.10.1109/ICASSP.2018.8462649 ↗
31. Xu, Y., Du, J., Huang, Z., Dai, L.R., and Lee, C.H. (2017). Multi-objective learning and mask-based post-processing for deep neural network based speech enhancement. arXiv.
32. Zaremba, W., Sutskever, I., and Vinyals, O. (2015). Recurrent Neural Network Regularization. arXiv.
33. Long Short-Term Memory10.1162/neco.1997.9.8.1735 ↗
34. Cho, K., Merrienboer, B.V., Gulcehre, C., BaHdanau, D., Bougares, F., Schwenk, H., and Bengio, Y. (2014). Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation. arXiv.10.3115/v1/D14-1179 ↗
35. Gao, T., Du, J., Dai, L.R., and Lee, C.H. (2018, January 15–20). Densely Connected Progressive Learning for LSTM-Based Speech Enhancement. Proceedings of the 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Calgary, AB, Canada.10.1109/ICASSP.2018.8461861 ↗
36. Yu, F., and Koltun, V. (2015). Multi-scale context aggregation by dilated convolutions. arXiv.
37. Ye, S., Hu, X., and Xu, X. (2020). Tdcgan: Temporal Dilated Convolutional Generative Adversarial Network for End-to-end Speech Enhancement. arXiv.
38. He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27–30). Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Vegas, NV, USA.10.1109/CVPR.2016.90 ↗
39. He, K., Zhang, X., Ren, S., and Sun, J. (2015, January 7–13). Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. Proceedings of the IEEE International Conference on Computer Vision, Santiago, Chile.10.1109/ICCV.2015.123 ↗
40. Noh, H., Hong, S., and Han, B. (2015, January 7–13). Learning deconvolution network for semantic Segmentation. Proceedings of the IEEE International Conference on Computer Vision, Santiago, Chile.10.1109/ICCV.2015.178 ↗
41. Garofolo, J.S., Lamel, L.F., Fisher, W.M., Fiscus, J.G., and Pallett, D.S. (2022, March 08). DARPA TIMIT Acoustic-Phonetic Continous Speech Corpus CD-ROM. NIST Speech Disc 1-1.1. NASA STI/Recon Technical Report n. 1993. Volume 93, p. 27403, Available online: https://nvlpubs.nist.gov/nistpubs/Legacy/IR/nistir4930.pdf.
42. Assessment for automatic speech recognition: II. NOISEX-92: A database and an experiment to study the effect of additive noise on speech recognition systems10.1016/0167-6393(93)90095-3 ↗
43. Kingma, D.P., and Ba, J. (2014). Adam: A method for stochastic optimization. arXiv.
44. Speech enhancement using a minimum mean-square error log-spectral amplitude estimator10.1109/TASSP.1985.1164550 ↗
45. A New Framework for CNN-Based Speech Enhancement in the Time Domain10.1109/TASLP.2019.2913512 ↗
46. Rix, A.W., Beerends, J.G., Hollier, M.P., and Hekstra, A.P. (2001, January 7–11). Perceptual Evaluation of Speech Quality (PESQ): A New Method for Speech Quality Assessment of Telephone Networks and Codecs. Proceedings of the 2001 IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP’01), Salt Lake City, UT, USA.
47. An Algorithm for Intelligibility Prediction of Time–Frequency Weighted Noisy Speech10.1109/TASL.2011.2114881 ↗
48. Statistical Comparisons of Classifiers over Multiple Data Sets