Efficient Speech Enhancement via Flow Matching with Gated Bidirectional Mamba2
Source abstract
Abstract
Speech enhancement (SE) aims to recover clean speech from noisy speech while preserving intelligibility, speaker identity, and runtime efficiency. Existing language-model (LM)-based methods may lose fine acoustic details due to discretization, whereas diffusion models often require many iterative denoising steps. This study proposes an efficient speech enhancement framework based on flow matching and a gated bidirectional Mamba2 backbone. The model predicts a continuous velocity field in the Mel-spectrogram domain and introduces a DiMamba block that captures past and future context through shared-weight bidirectional state-space modeling with adaptive gating. Experiments on the DNS Challenge test set and additional VoiceBank test data show that the proposed method achieves strong perceptual quality and speaker preservation while substantially reducing inference cost. The model reaches a real-time factor of 0.31, more than five times faster than diffusion baselines, and achieves a word error rate of 4.7% and a quality mean opinion score of 3.58. These results indicate that flow matching combined with gated bidirectional Mamba2 provides an effective quality–efficiency trade-off for offline speech enhancement.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Speech and Audio Processing · Speech Recognition and Synthesis · Face recognition and analysis
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
School of Intelligent Science and Technology, Beijing University of Civil Engineering and Architecture, Beijing 102616, China
State Key Laboratory of Opto-Electronic Information Acquisition and Protection Technology, Anhui University, Hefei 230093, China
Funding
State Key Laboratory of Opto-Electronic Information Acquisition and Protection Technology
References (40)
Show source-provided references
1. Westhausen, N.L., and Meyer, B.T. (2020, January 25–29). Dual-Signal Transformation LSTM Network for Real-Time Noise Suppression. Proceedings of the Interspeech 2020, Shanghai, China.10.21437/Interspeech.2020-2631 ↗
2. Le, X., Chen, H., Chen, K., and Lu, J. (September, January 30). DPCRN: Dual-Path Convolution Recurrent Network for Single Channel Speech Enhancement. Proceedings of the Interspeech 2021, Brno, Czechia.10.21437/Interspeech.2021-296 ↗
3. Lu, Y.X., Ai, Y., and Ling, Z.H. (2023, January 20–24). MP-SENet: A Speech Enhancement Model with Parallel Denoising of Magnitude and Phase Spectra. Proceedings of the Interspeech 2023, Dublin, Ireland.10.21437/Interspeech.2023-1441 ↗
4. Li, X., Wang, Q., and Liu, X. (2024, January 1–5). MaskSR: Masked Language Model for Full-band Speech Restoration. Proceedings of the Interspeech 2024, Kos, Greece.10.21437/Interspeech.2024-1584 ↗
5. Wang, Z., Zhu, X., Zhang, Z., Lv, Y., Jiang, N., Zhao, G., and Xie, L. (2024). Selm: Speech enhancement using discrete tokens and language models. Proceedings of the ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Seoul, Republic of Korea, 14–19 April 2024, IEEE.10.1109/ICASSP48485.2024.10447464 ↗
6. Yang, H., Su, J., Kim, M., and Jin, Z. (2024, January 1–5). Genhancer: High-fidelity speech enhancement via generative modeling on discrete codec tokens. Proceedings of the Interspeech, Kos, Greece.10.21437/Interspeech.2024-590 ↗
7. Anyenhance: A unified generative model with prompt-guidance and self-critic for voice enhancement10.1109/TASLPRO.2025.3587393 ↗
8. Kang, B., Zhu, X., Zhang, Z., Ye, Z., Liu, M., Wang, Z., Zhu, Y., Ma, G., Chen, J., and Xiao, L. (2025). LLaSE-G1: Incentivizing generalization capability for llama-based speech enhancement. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Association for Computational Linguistics.10.18653/v1/2025.acl-long.651 ↗
9. Lu, Y.J., Wang, Z.Q., Watanabe, S., Richard, A., Yu, C., and Tsao, Y. (2022). Conditional diffusion probabilistic model for speech enhancement. Proceedings of the ICASSP 2022—2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Singapore, 23–27 May 2022, IEEE.10.1109/ICASSP43922.2022.9746901 ↗
10. Storm: A diffusion-based stochastic regeneration model for speech enhancement and dereverberation10.1109/TASLP.2023.3294692 ↗
11. Welker, S., Richter, J., and Gerkmann, T. (2022, January 18–22). Speech Enhancement with Score-Based Generative Models in the Complex STFT Domain. Proceedings of the Interspeech 2022, Incheon, Republic of Korea.10.21437/Interspeech.2022-10653 ↗
12. Lipman, Y., Chen, R.T., Ben-Hamu, H., Nickel, M., and Le, M. (2022, January 25–29). Flow Matching for Generative Modeling. Proceedings of the Eleventh International Conference on Learning Representations, Virtual.
13. Wang, Z., Liu, Z., Zhu, X., Zhu, Y., Liu, M., Chen, J., Xiao, L., Weng, C., and Xie, L. (2025). FlowSE: Efficient and high-quality speech enhancement via flow matching. arXiv.10.21437/Interspeech.2025-1745 ↗
14. Lee, S., Cheong, S., Han, S., and Shin, J.W. (2025). Flowse: Flow matching-based speech enhancement. Proceedings of the ICASSP 2025—2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Hyderabad, India, 6–11 April 2025, IEEE.10.1109/ICASSP49660.2025.10888274 ↗
15. Rong, X., Gao, J., Wang, Z., Yesilbursa, M., Wojcicki, K., and Lu, J. (2026). StuPASE: Towards Low-Hallucination Studio-Quality Generative Speech Enhancement. arXiv.
16. Gu, A., and Dao, T. (2024, January 7–9). Mamba: Linear-time sequence modeling with selective state spaces. Proceedings of the First Conference on Language Modeling, Philadelphia, PA, USA.
17. BSDB-net: Band-split dual-branch network with selective state spaces mechanism for monaural speech enhancement
18. Dao, T., and Gu, A. (2024). Transformers are SSMs: Generalized models and efficient algorithms through structured state space duality. Proceedings of the International Conference on Machine Learning, Vienna, Austria, 21–27 July 2024, PMLR.
19. Zhu, L., Liao, B., Zhang, Q., Wang, X., Liu, W., and Wang, X. (2024). Vision mamba: Efficient visual representation learning with bidirectional state space model. arXiv.
20. Wang, J., Lin, Z., Wang, T., Ge, M., Wang, L., and Dang, J. (2025). Mamba-SEUNet: Mamba UNet for monaural speech enhancement. Proceedings of the ICASSP 2025—2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Hyderabad, India, 6–11 April 2025, IEEE.10.1109/ICASSP49660.2025.10889525 ↗
21. Mamba in speech: Towards an alternative to self-attention10.1109/TASLPRO.2025.3566210 ↗
22. Lan, Z., Chen, M., Goodman, S., Gimpel, K., Sharma, P., and Soricut, R. (2020, January 26–30). ALBERT: A Lite BERT for Self-supervised Learning of Language Representations. Proceedings of the International Conference on Learning Representations, Addis Ababa, Ethiopia.
23. Dehghani, M., Gouws, S., Vinyals, O., Uszkoreit, J., and Kaiser, Ł. (2018). Universal transformers. arXiv.
24. Dauphin, Y.N., Fan, A., Auli, M., and Grangier, D. (2017). Language modeling with gated convolutional networks. Proceedings of the International Conference on Machine Learning, Sydney, Australia, 6–11 August 2017, PMLR.
25. Shazeer, N. (2020). Glu variants improve transformer. arXiv.
26. Peebles, W., and Xie, S. (2023). Scalable diffusion models with transformers. Proceedings of the IEEE/CVF International Conference on Computer Vision, Paris, France, 1–6 October 2023, IEEE.10.1109/ICCV51070.2023.00387 ↗
27. Ho, J., and Salimans, T. (2022). Classifier-free diffusion guidance. arXiv.
28. Siuzdak, H. (2023). Vocos: Closing the gap between time-domain and fourier-based neural vocoders for high-quality audio synthesis. arXiv.
29. Panayotov, V., Chen, G., Povey, D., and Khudanpur, S. (2015). Librispeech: An asr corpus based on public domain audio books. Proceedings of the 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), South Brisbane, QLD, Australia, 19–24 April 2015, IEEE.10.1109/ICASSP.2015.7178964 ↗
30. Veaux, C., Yamagishi, J., and King, S. (2013, January 25–27). The voice bank corpus: Design, collection and data analysis of a large regional accent speech database. Proceedings of the 2013 International Conference Oriental COCOSDA Held Jointly with 2013 Conference on Asian Spoken Language Research and Evaluation (O-COCOSDA/CASLRE), Gurgaon, India.10.1109/ICSDA.2013.6709856 ↗
31. Chen, G., Chai, S., Wang, G., Du, J., Zhang, W., Weng, C., Su, D., Povey, D., Trmal, J., and Zhang, J. (September, January 30). GigaSpeech: An Evolving, Multi-domain ASR Corpus with 10,000 Hours of Transcribed Audio. Proceedings of the Interspeech 2021, Brno, Czechia.10.21437/Interspeech.2021-1965 ↗
32. Thiemann, J., Ito, N., and Vincent, E. (2013, January 2–6). The diverse environments multi-channel acoustic noise database (demand): A database of multichannel environmental noise recordings. Proceedings of the Meetings on Acoustics, Acoustical Society of America, San Francisco, CA, USA.10.1121/1.4799597 ↗
33. Wichern, G., Antognini, J., Flynn, M., Zhu, L.R., McQuinn, E., Crow, D., Manilow, E., and Le Roux, J. (2019, January 15–19). WHAM!: Extending Speech Separation to Noisy Environments. Proceedings of the Interspeech 2019, Graz, Austria.10.21437/Interspeech.2019-2821 ↗
34. Ko, T., Peddinti, V., Povey, D., Seltzer, M.L., and Khudanpur, S. (2017). A study on data augmentation of reverberant speech for robust speech recognition. Proceedings of the 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), New Orleans, LA, USA, 5–9 March 2017, IEEE.10.1109/ICASSP.2017.7953152 ↗
35. Reddy, C.K., Dubey, H., Koishida, K., Nair, A., Gopal, V., Cutler, R., Braun, S., Gamper, H., Aichner, R., and Srinivasan, S. (September, January 30). INTERSPEECH 2021 Deep Noise Suppression Challenge. Proceedings of the Interspeech 2021, Brno, Czechia.10.21437/Interspeech.2021-1609 ↗
36. Reddy, C.K., Gopal, V., and Cutler, R. (2021). DNSMOS: A non-intrusive perceptual objective speech quality metric to evaluate noise suppressors. Proceedings of the ICASSP 2021—2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Toronto, ON, Canada, 6–11 June 2021, IEEE.10.1109/ICASSP39728.2021.9414878 ↗
37. Wang, H., Liang, C., Wang, S., Chen, Z., Zhang, B., Xiang, X., Deng, Y., and Qian, Y. (2023). Wespeaker: A research and production oriented speaker embedding learning toolkit. Proceedings of the ICASSP 2023—2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Rhodes Island, Greece, 4–10 June 2023, IEEE.10.1109/ICASSP49357.2023.10096626 ↗
38. Radford, A., Kim, J.W., Xu, T., Brockman, G., McLeavey, C., and Sutskever, I. (2023). Robust speech recognition via large-scale weak supervision. Proceedings of the International Conference on Machine Learning, Honolulu, HI, USA, 23–29 July 2023, PMLR.
39. Conv-tasnet: Surpassing ideal time–frequency magnitude masking for speech separation10.1109/TASLP.2019.2915167 ↗
40. Defossez, A., Synnaeve, G., and Adi, Y. (2020, January 25–29). Real Time Speech Enhancement in the Waveform Domain. Proceedings of the Interspeech 2020, Shanghai, China.10.21437/Interspeech.2020-2409 ↗