SuperFormer: Enhanced Multi-Speaker Speech Separation Network Combining Channel and Spatial Adaptability
Source abstract
Abstract
Speech separation is a hot topic in multi-speaker speech recognition. The long-term autocorrelation of speech signal sequences is an essential task for speech separation. The keys are effective intra-autocorrelation learning for the speaker’s speech, modelling the local (intra-blocks) and global (intra- and inter- blocks) dependence features of the speech sequence, with the real-time separation of as few parameters as possible. In this paper, the local and global dependence features of speech sequence information are extracted by utilizing different transformer structures. A forward adaptive module of channel and space autocorrelation is proposed to give the separated model good channel adaptability (channel adaptive modeling) and space adaptability (space adaptive modeling). In addition, at the back end of the separation model, a speaker enhancement module is considered to further enhance or suppress the speech of different speakers by taking advantage of the mutual suppression characteristics of each source signal. Experiments show that the scale-invariant signal-to-noise ratio improvement (SI-SNRi) of the proposed separation network on the public corpus WSJ0-2mix achieves better separation performance compared with the baseline models. The proposed method can provide a solution for speech separation and speech recognition in multi-speaker scenarios.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Speech and Audio Processing · Music and Audio Processing · Speech Recognition and Synthesis
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
School of Software, Liaoning Technical University, Huludao 125105, China
Suzhou Automotive Research Institute, Tsinghua University, Suzhou 215100, China
Department of Civil and Environmental Engineering, The Hong Kong Polytechnic University, Hung Hom, Kowloon, Hong Kong, China
Funding
National Natural Science Foundation of China · Jiangsu Science and Technology Project · Research Project of Hubei Provincial Department of Education · Hubei Science and Technology Project · Hong Kong Scholars Program · Research Project of Liaoning Provincial Department of Education · Supported by China Postdoctoral Foundation
References (19)
Show source-provided references
1. Luo, Y., and Mesgarani, N. (2018, January 15–20). TaSNet: Time-Domain Audio Separation Network for Real-Time, Single-Channel Speech Separation. Proceedings of the 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Calgary, AB, Canada.10.1109/ICASSP.2018.8462116 ↗
2. Conv-TasNet: Surpassing Ideal Time–Frequency Magnitude Masking for Speech Separation10.1109/TASLP.2019.2915167 ↗
3. Luo, Y., Chen, Z., and Yoshioka, T. (2020, January 4–8). Dual-path RNN: Efficient long sequence modeling for time-domain single-channel speech separation. Proceedings of the ICASSP 2020—2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Virtual.10.1109/ICASSP40776.2020.9054266 ↗
4. Subakan, C., Ravanelli, M., Cornell, S., Bronzi, M., and Zhong, J. (2021, January 6–11). Attention is All You Need in Speech Separation. Proceedings of the 2021 IEEE International Conference on Acoustics, Speech and Signal Processing, Toronto, ON, Canada.10.1109/ICASSP39728.2021.9413901 ↗
5. Lea, C., Vidal, R., Reiter, A., and Hager, G.D. (October, January 8–10,). Temporal Convolutional Networks: A Unified Approach to Action Segmentation. Proceedings of the Computer Vision–ECCV 2016 Workshops, Amsterdam, The Netherlands.10.1007/978-3-319-49409-8_7 ↗
6. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, L., and Polosukhin, I. (2017). Attention Is All You Need. arXiv.
7. Liu, Y., Shao, Z., and Hoffmann, N. (2021). Global Attention Mechanism: Retain Information to Enhance Channel-Spatial Interactions. arXiv.
8. Liu, Z., Mao, H., Wu, C.Y., Feichtenhofer, C., Darrell, T., and Xie, S. (2022). A ConvNet for the 2020s. arXiv.10.1109/CVPR52688.2022.01167 ↗
9. Garofolo, J.S., Graff, D., Paul, D., and Pallett, D. (1993). CSR-I (WSJ0) Complete LDC93S6A. Web Download, Linguistic Data Consortium.
10. Hershey, J.R., Chen, Z., Le Roux, J., and Watanabe, S.J. (2016, January 20–25). Deep clustering: Discriminative embeddings for segmentation and separation. Proceedings of the 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Shanghai, China.10.1109/ICASSP.2016.7471631 ↗
11. Multi-talker speech separation with utterance-level permutation invariant training of deep recurrent neural networks10.1109/TASLP.2017.2726762 ↗
12. Isik, Y., Roux, J.L., Chen, Z., Watanabe, S., and Hershey, J.R. (2016, January 8–12). Single-channel multi-speaker separation using deep clustering. Proceedings of the Interspeech 2016, San Francisco, CA, USA.10.21437/Interspeech.2016-1176 ↗
13. Chen, Z., Luo, Y., and Mesgarani, N. (2017, January 5–9). Deep attractor network for single microphone speaker separation. Proceedings of the 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), New Orleans, LA, USA.10.1109/ICASSP.2017.7952155 ↗
14. Speaker-independent Speech Separation with Deep Attractor Network10.1109/TASLP.2018.2795749 ↗
15. Xu, C.L., Rao, W., Xiao, X., Chng, E.S., and Li, H.Z. (2018, January 15–20). Single Channel Speech Separation with Constrained Utterance Level Permutation Invariant Training Using Grid LSTM. Proceedings of the 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Calgary, AB, Canada.10.1109/ICASSP.2018.8462471 ↗
16. Li, C.X., Zhu, L., Xu, S., Gao, P., and Xu, B. (2018, January 15–20). CBLDNN-Based Speaker-Independent Speech Separation Via Generative Adversarial Training. Proceedings of the 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Calgary, AB, Canada.10.1109/ICASSP.2018.8462505 ↗
17. Wang, Z.Q., Roux, J.L., and Hershey, J.R. (2018, January 15–20). Alternative objective functions for deep clustering. Proceedings of the 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Calgary, AB, Canada.10.1109/ICASSP.2018.8462507 ↗
18. Wang, Z.Q., Le Roux, J., Wang, D.L., and Hershey, J. (2018, January 2–6). End-to-End Speech Separation with Unfolded Iterative Phase Reconstruction. Proceedings of the Interspeech, Hyderabad, India.10.21437/Interspeech.2018-1629 ↗
19. Luo, Y., and Mesgarani, N. (2018, January 2–6). Real-time Single-channel Dereverberation and Separation with Time-domain Audio Separation Network. Proceedings of the Interspeech, Hyderabad, India.10.21437/Interspeech.2018-2290 ↗