A Light-Weight Autoregressive CNN-Based Frame Level Transducer Decoder for End-to-End ASR
Source abstract
Abstract
A convolutional neural network (CNN) transducer decoder was proposed to reduce the decoding time of an end-to-end automatic speech recognition (ASR) system while maintaining accuracy. The CNN of 177 k parameters and a kernel size of 6 generates the probabilities of the current token at the token level, at the token transition of the output token sequence. Two probabilities of the current token, one from the encoder and the other from the CNN are added to the frame level to reduce the decoding step to the number of input frames. An encoder composed of an 18-layer conformer was combined with the proposed decoder for training with the Librispeech data set. The forward-backward algorithm was used for training. The space and re-appearance tokens are added to the 300-word piece tokens to represent the token string. A space token appears at a frame between two words. A comparison with the autoregressive decoders such as transformer and RNN-T decoders demonstrates that this work provides comparable WERs with much less decoding time. A comparison with non-autoregressive decoders such as CTC indicates that this work enhanced WERs.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Speech Recognition and Synthesis · Speech and Audio Processing · Music and Audio Processing
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Department of Electronic and Electrical Engineering, Pohang University of Science and Technology (POSTECH), Pohang 37673, Republic of Korea
Funding
National Research Foundation of Korea (NRF)
References (30)
Show source-provided references
1. Wang, D., Wang, X., and Lv, S. (2019). An overview of end-to-end automatic speech recognition. Symmetry, 11.10.3390/sym11081018 ↗
2. Graves, A., and Jaitly, N. (2014, January 21–26). Towards end-to-end speech recognition with recurrent neural networks. Proceedings of the International Conference on Machine Learning, PMLR, Beijing, China.
3. Chan, W., Jaitly, N., Le, Q., and Vinyals, O. (2016, January 20–25). Listen, attend and spell: A neural network for large vocabulary conversational speech recognition. Proceedings of the 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Shanghai, China.10.1109/ICASSP.2016.7472621 ↗
4. Convolutional neural networks for speech recognition10.1109/TASLP.2014.2339736 ↗
5. Hannun, A., Lee, A., Xu, Q., and Collobert, R. (2019). Sequence-to-sequence speech recognition with time-depth separable convolutions. arXiv.10.21437/Interspeech.2019-2460 ↗
6. Kriman, S., Beliaev, S., Ginsburg, B., Huang, J., Kuchaiev, O., Lavrukhin, V., Leary, R., Li, J., and Zhang, Y. (2020, January 4–8). Quartznet: Deep automatic speech recognition with 1d time-channel separable convolutions. Proceedings of the ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Barcelona, Spain.10.1109/ICASSP40776.2020.9053889 ↗
7. Graves, A., Jaitly, N., and Mohamed, A.R. (2013, January 8–12). Hybrid speech recognition with deep bidirectional LSTM. Proceedings of the 2013 IEEE Workshop on Automatic Speech Recognition and Understanding, Olomouc, Czech Republic.10.1109/ASRU.2013.6707742 ↗
8. Performance evaluation of deep neural networks applied to speech recognition: RNN, LSTM and GRU10.2478/jaiscr-2019-0006 ↗
9. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., and Polosukhin, I. (2017, January 4–9). Attention is all you need. Proceedings of the 31st Conference on Neural Information Processing Systems (NIPS 2017), Long Beach, CA, USA.
10. Zhang, Q., Lu, H., Sak, H., Tripathi, A., McDermott, E., Koo, S., and Kumar, S. (2020, January 4–8). Transformer transducer: A streamable speech recognition model with transformer encoders and rnn-t loss. Proceedings of the ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Barcelona, Spain.10.1109/ICASSP40776.2020.9053896 ↗
11. Wang, Y., Mohamed, A., Le, D., Liu, C., Xiao, A., Mahadeokar, J., Huang, H., Tjandra, A., Zhang, X., and Zhang, F. (2020, January 4–8). Transformer-based acoustic modeling for hybrid speech recognition. Proceedings of the ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Barcelona, Spain.10.1109/ICASSP40776.2020.9054345 ↗
12. Graves, A., Fernández, S., Gomez, F., and Schmidhuber, J. (2006, January 25–29). Connectionist temporal classification: Labelling unsegmented sequence data with recurrent neural networks. Proceedings of the 23rd International Conference on Machine Learning, Pittsburgh, PA, USA.10.1145/1143844.1143891 ↗
13. Chan, W., Saharia, C., Hinton, G., Norouzi, M., and Jaitly, N. (2020, January 13–18). Imputer: Sequence modelling via imputation and dynamic programming. Proceedings of the International Conference on Machine Learning, PMLR, Virtual.
14. Higuchi, Y., Watanabe, S., Chen, N., Ogawa, T., and Kobayashi, T. (2020). Mask CTC: Non-autoregressive end-to-end ASR with CTC and mask predict. arXiv.10.21437/Interspeech.2020-2404 ↗
15. Fujita, Y., Watanabe, S., Omachi, M., and Chan, X. (2020). Insertion-based modeling for end-to-end automatic speech recognition. arXiv.10.21437/Interspeech.2020-1619 ↗
16. Lee, J., and Watanabe, S. (2021, January 6–11). Intermediate loss regularization for ctc-based speech recognition. Proceedings of the ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Toronto, ON, Canada.10.1109/ICASSP39728.2021.9414594 ↗
17. Nozaki, J., and Komatsu, T. (2021). Relaxing the conditional independence assumption of CTC-based ASR by conditioning on intermediate predictions. arXiv.10.21437/Interspeech.2021-911 ↗
18. Graves, A. (2012). Sequence transduction with recurrent neural networks. arXiv.10.1007/978-3-642-24797-2 ↗
19. Ghahremani, P., BabaAli, B., Povey, D., Riedhammer, K., Trmal, J., and Khudanpur, S. (2014, January 4–9). A pitch extraction algorithm tuned for automatic speech recognition. Proceedings of the 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Florence, Italy.10.1109/ICASSP.2014.6854049 ↗
20. Kudo, T. (2018). Subword regularization: Improving neural network translation models with multiple subword candidates. arXiv.10.18653/v1/P18-1007 ↗
21. Higuchi, Y., Chen, N., Fujita, Y., Inaguma, H., Komatsu, T., Lee, J., Nozaki, J., Wang, T., and Watanabe, S. (2021, January 13–17). A comparative study on non-autoregressive modelings for speech-to-text generation. Proceedings of the 2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU), Cartagena, Colombia.10.1109/ASRU51503.2021.9688157 ↗
22. Collobert, R., Puhrsch, C., and Synnaeve, G. (2016). Wav2letter: An end-to-end convnet-based speech recognition system. arXiv.
23. Gulati, A., Qin, J., Chiu, C.C., Parmar, N., Zhang, Y., Yu, J., Han, W., Wang, S., Zhang, Z., and Wu, Y. (2020). Conformer: Convolution-augmented transformer for speech recognition. arXiv.10.21437/Interspeech.2020-3015 ↗
24. Panayotov, V., Chen, G., Povey, D., and Khudanpur, S. (2015, January 19–24). Librispeech: An asr corpus based on public domain audio books. Proceedings of the 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), South Brisbane, QLD, Australia.10.1109/ICASSP.2015.7178964 ↗
25. Siivola, V., and Pellom, B.L. (2005, January 4–8). Growing an n-gram language model. Proceedings of the Ninth European Conference on Speech Communication and Technology, Lisbon, Portugal.10.21437/Interspeech.2005-24 ↗
26. Ko, T., Peddinti, V., Povey, D., and Khudanpur, S. (2015, January 6–10). Audio augmentation for speech recognition. Proceedings of the Sixteenth Annual Conference of the International Speech Communication Association, Dresden, Germany.10.21437/Interspeech.2015-711 ↗
27. Park, D.S., Chan, W., Zhang, Y., Chiu, C.C., Zoph, B., Cubuk, E.D., and Le, Q.V. (2019). Specaugment: A simple data augmentation method for automatic speech recognition. arXiv.10.21437/Interspeech.2019-2680 ↗
28. Kim, S., Hori, T., and Watanabe, S. (2017, January 5–9). Joint CTC-attention based end-to-end speech recognition using multi-task learning. Proceedings of the 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), New Orleans, LA, USA.10.1109/ICASSP.2017.7953075 ↗
29. Hybrid CTC/attention architecture for end-to-end speech recognition10.1109/JSTSP.2017.2763455 ↗
30. Watanabe, S., Hori, T., Karita, S., Hayashi, T., Nishitoba, J., Unno, Y., Soplin, N.E.Y., Heymann, J., Wiesner, M., and Chen, N. (2018). Espnet: End-to-end speech processing toolkit. arXiv.10.21437/Interspeech.2018-1456 ↗