Entropy-Based Dynamic Rescoring with Language Model in E2E ASR Systems
Source abstract
Abstract
Language models (LM) have played crucial roles in automatic speech recognition (ASR), whether as an essential part of a conventional ASR system composed of an acoustic model and LM, or as an integrated model to enhance the performance of novel end-to-end ASR systems. With the development of machine learning and deep learning, language modeling has made great progress in natural language processing applications. In recent years, efforts have been made to leverage the advantages of novel LM to ASR. The most common way to apply an integration is still shallow fusion because it can be easily implemented by zero-overhead while obtaining significant improvement. Our method can further enhance the applicability of shallow fusion without hyperparameter tuning while maintaining similar performance.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Speech Recognition and Synthesis · Natural Language Processing Techniques · Music and Audio Processing
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Department of Electrical Engineering and Information Systems, Graduate School of Engineering, The University of Tokyo, Tokyo 113-8656, Japan
References (25)
Show source-provided references
1. Juang, B.H., and Rabiner, L.R. (2005). Automatic Speech Recognition—A Brief History of the Technology Development, Atlanta Rutgers University and the University of California. Georgia Institute of Technology.10.1016/B0-08-044854-2/00906-8 ↗
2. Attention is all you need
3. Watanabe, S., Hori, T., Karita, S., Hayashi, T., Nishitoba, J., Unno, Y., Soplin, N.E.Y., Heymann, J., Wiesner, M., and Chen, N. (2018). ESPnet: End-to-end speech processing toolkit. arXiv.10.21437/Interspeech.2018-1456 ↗
4. Chan, W., Jaitly, N., Le, Q., and Vinyals, O. (2016, January 20–25). Listen, attend and spell: A neural network for large vocabulary conversational speech recognition. Proceedings of the 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Shanghai, China.10.1109/ICASSP.2016.7472621 ↗
5. Bahdanau, D., Cho, K., and Bengio, Y. (2014). Neural machine translation by jointly learning to align and translate. arXiv.
6. Graves, A., Fernández, S., Gomez, F., and Schmidhuber, J. (2006, January 25–29). Connectionist temporal classification: Labelling unsegmented sequence data with recurrent neural networks. Proceedings of the 23rd International Conference on Machine Learning, Pittsburgh, PA, USA.10.1145/1143844.1143891 ↗
7. Kim, S., Hori, T., and Watanabe, S. (2017, January 5–9). Joint CTC-attention based end-to-end speech recognition using multi-task learning. Proceedings of the 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), New Orleans, LA, USA.10.1109/ICASSP.2017.7953075 ↗
8. Drexler, J., and Glass, J. (2019, January 12–17). Subword regularization and beam search decoding for end-to-end automatic speech recognition. Proceedings of the ICASSP 2019—2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Brighton, UK.10.1109/ICASSP.2019.8683531 ↗
9. Zeineldeen, M., Glushko, A., Michel, W., Zeyer, A., Schlüter, R., and Ney, H. (2021). Investigating methods to improve language model integration for attention-based encoder-decoder asr models. arXiv.10.21437/Interspeech.2021-1255 ↗
10. Narisetty, C.P., Hayashi, T., Ishizaki, R., Watanabe, S., and Takeda, K. (2021, January 15–19). Leveraging State-of-the-art ASR Techniques to Audio Captioning. Proceedings of the Detection and Classification of Acoustic Scenes and Events 2021, Online.
11. McDermott, E., Sak, H., and Variani, E. (2017, January 5–9). A Density Ratio Approach to Language Model Fusion in End-to-End Automatic Speech Recognition. Proceedings of the 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), New Orleans, LA, USA.
12. Andrés-Ferrer, J., Albesano, D., Zhan, P., and Vozila, P. (2022). Contextual density ratio for language model biasing of sequence to sequence ASR systems. arXiv.10.21437/Interspeech.2021-443 ↗
13. Toshniwal, S., Kannan, A., Chiu, C.-C., Wu, Y., Sainath, T.N., and Livescu, K. (2018, January 18–21). A comparison of techniques for language model integration in encoder-decoder speech recognition. Proceedings of the 2018 IEEE Spoken Language Technology Workshop (SLT), Athens, Greece.10.1109/SLT.2018.8639038 ↗
14. Zhao, D., Sainath, T.N., Rybach, D., Rondon, P., Bhatia, D., Li, B., and Pang, R. (2019, January 15–19). Shallow-Fusion End-to-End Contextual Biasing. Proceedings of the Interspeech, Graz, Austria.10.21437/Interspeech.2019-1209 ↗
15. Shan, C., Weng, C., Wang, G., Su, D., Luo, M., Yu, D., and Xie, L. (2019, January 12–17). Component fusion: Learning replaceable language model component for end-to-end speech recognition system. Proceedings of the ICASSP 2019—2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Brighton, UK.10.1109/ICASSP.2019.8682490 ↗
16. Sriram, A., Jun, H., Satheesh, S., and Coates, A. (2017). Cold fusion: Training seq2seq models together with language models. arXiv.10.21437/Interspeech.2018-1392 ↗
17. Battenberg, E., Chen, J., Child, R., Coates, A., Li, Y.G.Y., Liu, H., Satheesh, S., Sriram, A., and Zhu, Z. (2017, January 16–20). Exploring neural transducers for end-to-end speech recognition. Proceedings of the 2017 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU), Okinawa, Japan.10.1109/ASRU.2017.8268937 ↗
18. Wang, D., and Zheng, T.F. (2015, January 16–19). Transfer learning for speech and language processing. Proceedings of the 2015 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA), Hong Kong, China.10.1109/APSIPA.2015.7415532 ↗
19. Zhao, W.X., Jiang, J., Weng, J., He, J., Lim, E.-P., Yan, H., and Li, X. (2011). Comparing twitter and traditional media using topic models. European Conference on Information Retrieval, Springer.10.1007/978-3-642-20161-5_34 ↗
20. Xing, C., Wu, W., Wu, Y., Liu, J., Huang, Y., Zhou, M., and Ma, W.Y. (2017, January 4–9). Topic aware neural response generation. Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence, San Francisco, CA, USA.10.1609/aaai.v31i1.10981 ↗
21. A cache-based natural language model for speech recognition10.1109/34.56193 ↗
22. Sennrich, R., Haddow, B., and Birch, A. (2016). Improving Neural Machine Translation Models with Monolingual Data. arXiv.10.18653/v1/P16-1009 ↗
23. Renduchintala, A., Ding, S., Wiesner, M., and Watanabe, S. (2018). Multi-Modal Data Augmentation for End-to-End ASR. arXiv.10.21437/Interspeech.2018-2456 ↗
24. Panayotov, V., Chen, G., Povey, D., and Khudanpur, S. (2015, January 19–24). Librispeech: An asr corpus based on public domain audio books. Proceedings of the 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), South Brisbane, QLD, Australia.10.1109/ICASSP.2015.7178964 ↗
25. Rousseau, A., Deléglise, P., and Esteve, Y. (2012, January 21–27). TED-LIUM: An Automatic Speech Recognition dedicated corpus. Proceedings of the Eighth International Conference on Language Resources and Evaluation (LREC’12), Istanbul, Turkey.