Evaluating Novel Speech Transcription Architectures on the Spanish RTVE2020 Database
Source abstract
Abstract
This work presents three novel speech recognition architectures evaluated on the Spanish RTVE2020 dataset, employed as the main evaluation set in the Albayzín S2T Transcription Challenge 2020. The main objective was to improve the performance of the systems previously submitted by the authors to the challenge, in which the primary system scored the second position. The novel systems are based on both DNN-HMM and E2E acoustic models, for which fully- and self-supervised learning methods were included. As a result, the new speech recognition engines clearly outperformed the performance of the initial systems from the previous best WER of 19.27 to the new best of 17.60 achieved by the DNN-HMM based system. This work therefore describes an interesting benchmark of the latest acoustic models over a highly challenging dataset, and identifies the most optimal ones depending on the expected quality, the available resources and the required latency.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Speech Recognition and Synthesis · Music and Audio Processing · Speech and Audio Processing
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Vicomtech Foundation, Basque Research and Technology Alliance (BRTA), Mikeletegi 57, 20009 Donostia-San Sebastián, Spain
Department of Electronics Engineering and Communications, University of Zaragoza, Pedro Cerbuna 12, 50009 Zaragoza, Spain
References (41)
Show source-provided references
1. Performance vs. hardware requirements in state-of-the-art automatic speech recognition10.1186/s13636-021-00217-4 ↗
2. Baevski, A., Zhou, H., Mohamed, A., and Auli, M. (2020). Wav2vec 2.0: A framework for self-supervised learning of speech representations. arXiv.
3. Hsu, W.N., Tsai, Y.H.H., Bolte, B., Salakhutdinov, R., and Mohamed, A. (2021, January 6–12). HuBERT: How much can a bad teacher benefit ASR pre-training?. Proceedings of the 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Toronto, ON, Canada.10.1109/ICASSP39728.2021.9414460 ↗
4. Lleida, E., Ortega, A., Miguel, A., Bazán-Gil, V., Pérez, C., Gómez, M., and de Prada, A. (2021, December 28). Rtve2020 Database Description. Available online: http://catedrartve.unizar.es/reto2020/RTVE2020DB.pdf.
5. Povey, D., Ghoshal, A., Boulianne, G., Burget, L., Glembek, O., Goel, N., Hannemann, M., Motlicek, P., Qian, Y., and Schwarz, P. (2011, January 11–15). The Kaldi speech recognition toolkit. Proceedings of the 2011 IEEE Workshop on Automatic Speech Recognition and Understanding, Waikoloa, HI, USA.
6. Kriman, S., Beliaev, S., Ginsburg, B., Huang, J., Kuchaiev, O., Lavrukhin, V., Leary, R., Li, J., and Zhang, Y. (2020, January 4–8). Quartznet: Deep automatic speech recognition with 1d time-channel separable convolutions. Proceedings of the 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Barcelona, Spain.10.1109/ICASSP40776.2020.9053889 ↗
7. Alvarez, A., Arzelus, H., Torre, I.G., and González-Docasal, A. (2021, January 24–25). The Vicomtech Speech Transcription Systems for the Albayzın-RTVE 2020 Speech to Text Transcription Challenge. Proceedings of the IberSPEECH 2021, Valladolid, Spain.10.21437/IberSPEECH.2021-22 ↗
8. Deep neural networks for acoustic modeling in speech recognition: The shared views of four research groups10.1109/MSP.2012.2205597 ↗
9. Amodei, D., Ananthanarayanan, S., Anubhai, R., Bai, J., Battenberg, E., Case, C., Casper, J., Catanzaro, B., Cheng, Q., and Chen, G. (2016, January 20–22). Deep speech 2: End-to-end speech recognition in english and mandarin. Proceedings of the International Conference on Machine Learning, New York, NY, USA.
10. Graves, A., and Jaitly, N. (2014, January 21–26). Towards End-to-end Speech Recognition with Recurrent Neural Networks. Proceedings of the 31st International Conference on International Conference on Machine Learning, Beijing, China.
11. Chan, W., Jaitly, N., Le, Q., and Vinyals, O. (2016, January 20–25). Listen, attend and spell: A neural network for large vocabulary conversational speech recognition. Proceedings of the 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Shanghai, China.10.1109/ICASSP.2016.7472621 ↗
12. Attention-based models for speech recognition
13. Lu, L., Zhang, X., and Renals, S. (2016, January 20–25). On training the recurrent neural network encoder-decoder for large vocabulary end-to-end speech recognition. Proceedings of the 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Shanghai, China.10.1109/ICASSP.2016.7472641 ↗
14. Yao, Z., Wu, D., Wang, X., Zhang, B., Yu, F., Yang, C., Peng, Z., Chen, X., Xie, L., and Lei, X. (2021). Wenet: Production oriented streaming and non-streaming end-to-end speech recognition toolkit. arXiv.10.21437/Interspeech.2021-1983 ↗
15. Chang, X., Maekaku, T., Guo, P., Shi, J., Lu, Y.J., Subramanian, A.S., Wang, T., Yang, S.w., Tsao, Y., and Lee, H.y. (2021). An exploration of self-supervised pretrained representations for end-to-end speech recognition. arXiv.10.1109/ASRU51503.2021.9688137 ↗
16. Panayotov, V., Chen, G., Povey, D., and Khudanpur, S. (2015, January 19–24). Librispeech: An ASR corpus based on public domain audio books. Proceedings of the 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), South Brisbane, QLD, Australia.10.1109/ICASSP.2015.7178964 ↗
17. Han, K.J., Pan, J., Tadala, V.K.N., Ma, T., and Povey, D. (2021, January 6–12). Multistream CNN for robust acoustic modeling. Proceedings of the 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Toronto, ON, Canada.10.1109/ICASSP39728.2021.9414639 ↗
18. Han, K.J., Prieto, R., and Ma, T. (2019, January 14–18). State-of-the-art speech recognition using multi-stream self-attention with dilated 1d convolutions. Proceedings of the 2019 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU), Sentosa, Singapore.10.1109/ASRU46091.2019.9003730 ↗
19. Pan, J., Shapiro, J., Wohlwend, J., Han, K.J., Lei, T., and Ma, T. (2020). ASAPP-ASR: Multistream CNN and self-attentive SRU for SOTA speech recognition. arXiv.10.21437/Interspeech.2020-2947 ↗
20. Li, J., Lavrukhin, V., Ginsburg, B., Leary, R., Kuchaiev, O., Cohen, J.M., Nguyen, H., and Gadde, R.T. (2019). Jasper: An end-to-end convolutional neural acoustic model. arXiv.10.21437/Interspeech.2019-1819 ↗
21. Hsu, W.N., Bolte, B., Tsai, Y.H.H., Lakhotia, K., Salakhutdinov, R., and Mohamed, A. (2021). HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units. arXiv.10.1109/TASLP.2021.3122291 ↗
22. Wang, Y., Mohamed, A., Le, D., Liu, C., Xiao, A., Mahadeokar, J., Huang, H., Tjandra, A., Zhang, X., and Zhang, F. (2020, January 4–8). Transformer-based acoustic modeling for hybrid speech recognition. Proceedings of the 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Barcelona, Spain.10.1109/ICASSP40776.2020.9054345 ↗
23. Lleida, E., Ortega, A., Miguel, A., Bazán-Gil, V., Pérez, C., Gómez, M., and de Prada, A. (2019). Albayzin 2018 evaluation: The iberspeech-RTVE challenge on speech technologies for spanish broadcast media. Appl. Sci., 9.10.3390/app9245412 ↗
24. del Pozo, A., Aliprandi, C., Álvarez, A., Mendes, C., Neto, J.P., Paulo, S., Piccinini, N., and Raffaelli, M. (2021, December 21). SAVAS: Collecting, Annotating and Sharing Audiovisual Language Resources for Automatic Subtitling. Available online: https://d1wqtxts1xzle7.cloudfront.net/47591506/SAVAS_Collecting_Annotating_and_Sharing_20160728-14677-ftvov1.pdf?1469710441=&response-content-disposition=inline%3B+filename%3DSAVAS_Collecting_Annotating_and_Sharing.pdf&Expires=1644576491&Signature=U30UJaJhWnxGGyeWM1nrbsH8X6OmBzgCFnHR-6~yPtA1zWbi~QwWg9nfIXFGc7-iRF5q4FvKblsf0o5-O665DKoLwhtMpAVwuEX71lTmY9qjRRaSMAa3AfEFKyrCNwuKgEWWRmlkaftiUVOTZRyt8T2S3z9Y0TmDTFtJ7Nyvd4~~096vT3y7sdjgd5j~R54br24q13pXlHh64yJozRV41xGvb76yjYF1~yS3oivFYiP0GjZ1jckXgSRB0WzwVkDkve6JThKuxKwO58VP3~WfRIIb2DUrtKYPO-C8EUkPY7e2ZrpLNTjCuTO0JqfIaXmvdKk4XMn7T0KqZA-~fbHABQ__&Key-Pair-Id=APKAJLOHF5GGSLRBV4ZA.
25. Ardila, R., Branson, M., Davis, K., Henretty, M., Kohler, M., Meyer, J., Morais, R., Saunders, L., Tyers, F.M., and Weber, G. (2019). Common voice: A massively-multilingual speech corpus. arXiv.
26. Casacuberta, F., Garcia, R., Llisterri, J., Nadeu, C., Pardo, J., and Rubio, A. (1991, January 26). Development of Spanish corpora for speech research (Albayzin). Proceedings of the Workshop on International Cooperation and Standarization of Speech Databases and Speech I/O Assesment Methods, Chiavari, Italy.
27. Campione, E., and Véronis, J. (December, January 30). A multilingual prosodic database. Proceedings of the Fifth International Conference on Spoken Language Processing, Sydney, Australia.
28. Park, D.S., Chan, W., Zhang, Y., Chiu, C.C., Zoph, B., Cubuk, E.D., and Le, Q.V. (2019). Specaugment: A simple data augmentation method for automatic speech recognition. arXiv.10.21437/Interspeech.2019-2680 ↗
29. Ko, T., Peddinti, V., Povey, D., and Khudanpur, S. (2015, January 6–10). Audio augmentation for speech recognition. Proceedings of the Sixteenth Annual Conference of the International Speech Communication Association, Dresden, Germany.10.21437/Interspeech.2015-711 ↗
30. Peddinti, V., Povey, D., and Khudanpur, S. (2015, January 6–10). A time delay neural network architecture for efficient modeling of long temporal contexts. Proceedings of the Sixteenth Annual Conference of the International Speech Communication Association, Dresden, Germany.10.21437/Interspeech.2015-647 ↗
31. Xu, H., Chen, T., Gao, D., Wang, Y., Li, K., Goel, N., Carmiel, Y., Povey, D., and Khudanpur, S. (2018, January 15–20). A pruned rnnlm lattice-rescoring algorithm for automatic speech recognition. Proceedings of the 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Calgary, AB, Canada.10.1109/ICASSP.2018.8461974 ↗
32. Smith, L.N. (2017, January 24–31). Cyclical learning rates for training neural networks. Proceedings of the 2017 IEEE Winter conference on Applications of Computer Vision (WACV), Santa Rosa, CA, USA.10.1109/WACV.2017.58 ↗
33. Kucsko, G. (2021, December 28). Pyctcdecode. Available online: https://github.com/kensho-technologies/pyctcdecode.
34. Conneau, A., Baevski, A., Collobert, R., Mohamed, A., and Auli, M. (2020). Unsupervised cross-lingual representation learning for speech recognition. arXiv.10.21437/Interspeech.2021-329 ↗
35. Wang, C., Rivière, M., Lee, A., Wu, A., Talnikar, C., Haziza, D., Williamson, M., Pino, J., and Dupoux, E. (2021). Voxpopuli: A large-scale multilingual speech corpus for representation learning, semi-supervised learning and interpretation. arXiv.10.18653/v1/2021.acl-long.80 ↗
36. Pratap, V., Xu, Q., Sriram, A., Synnaeve, G., and Collobert, R. (2020). Mls: A large-scale multilingual dataset for speech research. arXiv.10.21437/Interspeech.2020-2826 ↗
37. Valk, J., and Alumäe, T. (2021, January 19–22). Voxlingua107: A dataset for spoken language recognition. Proceedings of the 2021 IEEE Spoken Language Technology Workshop (SLT), Shenzhen, China.10.1109/SLT48900.2021.9383459 ↗
38. Babu, A., Wang, C., Tjandra, A., Lakhotia, K., Xu, Q., Goyal, N., Singh, K., von Platen, P., Saraf, Y., and Pino, J. (2021). XLS-R: Self-supervised Cross-lingual Speech Representation Learning at Scale. arXiv.10.21437/Interspeech.2022-143 ↗
39. Povey, D., Cheng, G., Wang, Y., Li, K., Xu, H., Yarmohammadi, M., and Khudanpur, S. (2018, January 2–6). Semi-Orthogonal Low-Rank Matrix Factorization for Deep Neural Networks. Proceedings of the Interspeech 2018, Hyderabad, India.10.21437/Interspeech.2018-1417 ↗
40. Lleida, E., Ortega, A., Miguel, A., Bazán, V., Pérez, C., Zotano, M., and de Prada, A. (2018). RTVE2018 Database Description, Vivolab and Corporación Radiotelevisión Espanola.
41. Yang, S.W., Chi, P.H., Chuang, Y.S., Lai, C.I.J., Lakhotia, K., Lin, Y.Y., Liu, A.T., Shi, J., Chang, X., and Lin, G.T. (2021). SUPERB: Speech processing Universal PERformance Benchmark. arXiv.10.21437/Interspeech.2021-1775 ↗