Design of a Dual-Path Speech Enhancement Model
Source abstract
Abstract
Although both noise suppression and speech restoration are fundamental to speech enhancement, many Deep neural network (DNN)-based approaches tend to focus disproportionately on one, often overlooking the importance of their joint handling. In this study, we propose a dual-path architecture designed to balance noise suppression and speech restoration. The main path consists of an encoder and two specialized decoders: one dedicated to estimating the clean speech spectrum and the other to predicting a noise suppression mask. To reinforce the joint modeling of noise suppression and speech restoration, we introduce an auxiliary refinement path. This path consists of a separate encoder–decoder structure and is designed to further refine the enhanced speech by incorporating complementary information, learned independently from the main path. By using this dual-path architecture, the model better preserves fine speech details while reducing residual noise. Experimental results on the VoiceBank + DEMAND dataset show that our model surpasses conventional methods across multiple evaluation metrics in the causal setup. Specifically, it achieves a PESQ score of 3.33, reflecting improved speech quality, and a CSIG score of 4.48, indicating enhanced intelligibility. Furthermore, it demonstrates superior noise suppression, achieving an SNRseg of 10.44 and a CBAK score of 3.75.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Speech and Audio Processing · Speech Recognition and Synthesis · Infant Health and Development
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Intelligent Signal Processing Lab, Yonsei University, Wonju 26493, Republic of Korea
Department of Electric and Semiconductor Engineering, Gangneung-Wonju National University, Gangneung 25457, Republic of Korea
References (37)
Show source-provided references
1. Ochiai, T., Watanabe, S., Hori, T., and Hershey, J. (2017, January 6–11). Multichannel end-to-end speech recognition. Proceedings of the 34th International Conference on Machine Learning, Sydney, Australia.
2. Two-stage binaural speech enhancement with Wiener filter for high-quality speech communication10.1016/j.specom.2010.04.009 ↗
3. Improving Privacy and Security of User Data in Location Based Services10.4018/IJACI.2018010102 ↗
4. Medical Cyber-Physical Systems: A Survey10.1007/s10916-018-0921-x ↗
5. Fundamentals, Present and Future Perspectives of Speech Enhancement10.1007/s10772-020-09674-2 ↗
6. Speech enhancement using a minimum-mean square error short-time spectral amplitude estimator10.1109/TASSP.1984.1164453 ↗
7. Yousheng, X., and Jianwen, H. (2014, January 7–9). Speech enhancement based on combination of wiener filter and subspace filter. Proceedings of the 2014 International Conference on Audio, Language and Image Processing, Shanghai, China.10.1109/ICALIP.2014.7009836 ↗
8. A regression approach to speech enhancement based on deep neural networks10.1109/TASLP.2014.2364452 ↗
9. Weninger, F., Erdogan, H., Watanabe, S., Vincent, E., Le Roux, J., Hershey, J.R., and Schuller, B. (2015, January 25–28). Speech enhancement with LSTM recurrent neural networks and its application to noise-robust ASR. Proceedings of the Latent Variable Analysis and Signal Separation: 12th International Conference, LVA/ICA 2015, Liberec, Czech Republic. Proceedings 12.10.1007/978-3-319-22482-4_11 ↗
10. Supervised Speech Separation Based on Deep Learning: An Overview10.1109/TASLP.2018.2842159 ↗
11. Nuthakki, R., Masanta, P., and Yukta, T.N. (2021, January 9–10). A Literature Survey on Speech Enhancement Based on Deep Neural Network Technique. Proceedings of the 4th International Conference on Communications and Cyber Physical Engineering (ICCCE 2021), Hyderabad, India.
12. Hu, Y., Liu, Y., Lv, S., Xing, M., Zhang, S., Fu, Y., Wu, J., Zhang, B., and Xie, L. (2020). DCCRN: Deep complex convolution recurrent network for phase-aware speech enhancement. arXiv.10.21437/Interspeech.2020-2537 ↗
13. Cao, R., Abdulatif, S., and Yang, B. (2022, January 18–22). CMGAN: Conformer-based Metric GAN for Speech Enhancement. Proceedings of the Interspeech 2022, Incheon, Republic of Korea.10.21437/Interspeech.2022-517 ↗
14. Two-Stage Refinement of Magnitude and Complex Spectra for Real-Time Speech Enhancement10.1109/LSP.2022.3215100 ↗
15. Ju, Y., Rao, W., Yan, X., Fu, Y., Lv, S., Cheng, L., Wang, Y., Xie, L., and Shang, S. (2022, January 22–27). TEA-PSE: Tencent-Ethereal-Audio-Lab Personalized Speech Enhancement System for ICASSP 2022 DNS Challenge. Proceedings of the ICASSP 2022—2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Singapore.10.1109/ICASSP43922.2022.9747765 ↗
16. Zhang, Z., Zhang, L., Zhuang, X., Qian, Y., Li, H., and Wang, M. (2022, January 22–27). FB-MSTCN: A Full-Band Single-Channel Speech Enhancement Method Based on Multi-Scale Temporal Convolutional Network. Proceedings of the ICASSP 2022—2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Singapore.10.1109/ICASSP43922.2022.9746387 ↗
17. Glance and gaze: A collaborative learning framework for single-channel speech enhancement10.1016/j.apacoust.2021.108499 ↗
18. Fu, Y., Liu, Y., Li, J., Luo, D., Lv, S., Jv, Y., and Xie, L. (2022, January 22–27). Uformer: A Unet Based Dilated Complex & Real Dual-Path Conformer Network for Simultaneous Speech Enhancement and Dereverberation. Proceedings of the ICASSP 2022—2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Singapore.10.1109/ICASSP43922.2022.9746020 ↗
19. Hwang, S., Park, S.W., and Park, Y. (2025, January 6–11). Causal Speech Enhancement Based on a Two-Branch Nested U-Net Architecture Using Self-Supervised Speech Embeddings. Proceedings of the ICASSP 2025—2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Hyderabad, India.10.1109/ICASSP49660.2025.10888248 ↗
20. ICASSP 2023 Speech Signal Improvement Challenge10.1109/OJSP.2024.3376293 ↗
21. U2-Net: Going deeper with nested U-structure for salient object detection10.1016/j.patcog.2020.107404 ↗
22. Hwang, S., Park, S.W., and Park, Y. (2022, January 18–22). Monoaural Speech Enhancement Using a Nested U-Net with Two-Level Skip Connections. Proceedings of the Interspeech 2022, Incheon, Republic of Korea.10.21437/Interspeech.2022-10025 ↗
23. A Nested U-Net With Self-Attention and Dense Connectivity for Monaural Speech Enhancement10.1109/LSP.2021.3128374 ↗
24. A Time-Frequency Attention Module for Neural Speech Enhancement10.1109/TASLP.2022.3225649 ↗
25. On the Importance of Power Compression and Phase Estimation in Monaural Speech Dereverberation10.1121/10.0003321 ↗
26. Valentini-Botinhao, C., Wang, X., Takaki, S., and Yamagishi, J. (2016, January 13–15). Investigating RNN-based speech enhancement methods for noise-robust Text-to-Speech. Proceedings of the 9th ISCA Speech Synthesis Workshop, Sunnyvale, CA, USA.10.21437/SSW.2016-24 ↗
27. Rix, A., Beerends, J., Hollier, M., and Hekstra, A. (2001, January 7–11). Perceptual evaluation of speech quality (PESQ)—A new method for speech quality assessment of telephone networks and codecs. Proceedings of the 2001 IEEE International Conference on Acoustics, Speech, and Signal Processing, Proceedings (Cat. No.01CH37221), Salt Lake City, UT, USA,.
28. Hu, Y., and Loizou, P.C. (2006, January 17–21). Evaluation of Objective Measures for Speech Enhancement. Proceedings of the Interspeech 2006, Pittsburgh, PA, USA.10.21437/Interspeech.2006-84 ↗
29. A deep learning loss function based on the perceptual evaluation of the speech quality10.1109/LSP.2018.2871419 ↗
30. Robust Speech Recognition via Large-Scale Weak Supervision
31. Loizou, P.C. (2007). Speech Enhancement: Theory and Practice, CRC Press.10.1201/9781420015836 ↗
32. Defossez, A., Synnaeve, G., and Adi, Y. (2020). Real time speech enhancement in the waveform domain. arXiv.10.21437/Interspeech.2020-2409 ↗
33. Two Heads are Better Than One: A Two-Stage Complex Spectral Mapping Approach for Monaural Speech Enhancement10.1109/TASLP.2021.3079813 ↗
34. Chen, J., Wang, Z., Tuo, D., Wu, Z., Kang, S., and Meng, H. (2022, January 22–27). FullSubNet+: Channel Attention FullSubNet with Complex Spectrograms for Speech Enhancement. Proceedings of the ICASSP 2022—2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Singapore.10.1109/ICASSP43922.2022.9747888 ↗
35. Zhao, S., Ma, B., Watcharasupat, K., and Gan, W. (2022, January 22–27). FRCRN: Boosting feature representation using frequency recurrence for monaural speech enhancement. Proceedings of the ICASSP 2022—2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Singapore.10.1109/ICASSP43922.2022.9747578 ↗
36. CompNet: Complementary network for single-channel speech enhancement10.1016/j.neunet.2023.09.041 ↗
37. Zhang, Y., Zou, H., and Zhu, J. (2024, January 14–19). A Two-Stage Framework in Cross-Spectrum Domain for Real-Time Speech Enhancement. Proceedings of the ICASSP 2024—2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Seoul, Republic of Korea.10.1109/ICASSP48485.2024.10447096 ↗