Semisupervised Speech Data Extraction from Basque Parliament Sessions and Validation on Fully Bilingual Basque–Spanish ASR
Source abstract
Abstract
In this paper, a semisupervised speech data extraction method is presented and applied to create a new dataset designed for the development of fully bilingual Automatic Speech Recognition (ASR) systems for Basque and Spanish. The dataset is drawn from an extensive collection of Basque Parliament plenary sessions containing frequent code switchings. Since session minutes are not exact, only the most reliable speech segments are kept for training. To that end, we use phonetic similarity scores between nominal and recognized phone sequences. The process starts with baseline acoustic models trained on generic out-of-domain data, then iteratively updates the models with the extracted data and applies the updated models to refine the training dataset until the observed improvement between two iterations becomes small enough. A development dataset, involving five plenary sessions not used for training, has been manually audited for tuning and evaluation purposes. Cross-validation experiments (with 20 random partitions) have been carried out on the development dataset, using the baseline and the iteratively updated models. On average, Word Error Rate (WER) reduces from 16.57% (baseline) to 4.41% (first iteration) and further to 4.02% (second iteration), which corresponds to relative WER reductions of 73.4% and 8.8%, respectively. When considering only Basque segments, WER reduces on average from 16.57% (baseline) to 5.51% (first iteration) and further to 5.13% (second iteration), which corresponds to relative WER reductions of 66.7% and 6.9%, respectively. As a result of this work, a new bilingual Basque–Spanish resource has been produced based on Basque Parliament sessions, including 998 h of training data (audio segments + transcriptions), a development set (17 h long) designed for tuning and evaluation under a cross-validation scheme and a fully bilingual trigram language model.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Speech Recognition and Synthesis · Phonetics and Phonology Research · Natural Language Processing Techniques
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Department of Electricity and Electronics, Faculty of Science and Technology, University of the Basque Country (UPV/EHU), Barrio Sarriena, 48940 Leioa, Spain
Funding
Spanish Ministry of Science and Innovation · Basque Government
References (38)
Show source-provided references
1. Gardner-Chloros, P. (2009). Code-Switching, Cambridge University Press.10.1017/CBO9780511609787 ↗
2. Semi-supervised acoustic model training for speech with code-switching10.1016/j.specom.2018.10.006 ↗
3. Dalmia, S., Liu, Y., Ronanki, S., and Kirchhoff, K. (2021, January 6–11). Transformer-Transducers for Code-Switched Speech Recognition. Proceedings of the ICASSP 2021—2021 IEEE International Conference on Acoustics, Speech and Signal Processing, Toronto, ON, Canada.10.1109/ICASSP39728.2021.9413562 ↗
4. Code-switched automatic speech recognition in five South African languages10.1016/j.csl.2021.101262 ↗
5. Alvarez, A., Arzelus, H., Prieto, S., and del Pozo, A. (2016, January 23–25). Rich Transcription and Automatic Subtitling for Basque and Spanish. Proceedings of the Iberspeech 2016, Lisbon, Portugal.
6. Yılmaz, E., van den Heuvel, H., and van Leeuwen, D. (2016, January 13–16). Code-switching detection using multilingual DNNs. Proceedings of the 2016 IEEE Spoken Language Technology Workshop (SLT), San Diego, CA, USA.10.1109/SLT.2016.7846326 ↗
7. Yilmaz, E., McLaren, M., van den Heuvel, H., and van Leeuwen, D.A. (2017, January 16–20). Language diarization for semi-supervised bilingual acoustic model training. Proceedings of the 2017 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU), Okinawa, Japan.10.1109/ASRU.2017.8268921 ↗
8. Seki, H., Watanabe, S., Hori, T., Roux, J.L., and Hershey, J.R. (2018, January 15–20). An End-to-End Language-Tracking Speech Recognizer for Mixed-Language Speech. Proceedings of the 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Calgary, AB, Canada.10.1109/ICASSP.2018.8462180 ↗
9. Zeng, Z., Khassanov, Y., Pham, V.T., Xu, H., Chng, E.S., and Li, H. (2019, January 15–19). On the End-to-End Solution to Mandarin-English Code-Switching Speech Recognition. Proceedings of the Interspeech 2019, Graz, Austria.10.21437/Interspeech.2019-1429 ↗
10. Qiu, Z., Li, Y., Li, X., Metze, F., and Campbell, W.M. (2020, January 25–29). Towards Context-Aware End-to-End Code-Switching Speech Recognition. Proceedings of the Interspeech 2020, Shanghai, China.10.21437/Interspeech.2020-1980 ↗
11. Shi, X., Feng, Q., and Xie, L. (2020). The ASRU 2019 Mandarin-English Code-Switching Speech Recognition Challenge: Open Datasets, Tracks, Methods and Results. arXiv.
12. Li, C., Deng, S., Wang, Y., Wang, G., Gong, Y., Chen, C., and Bai, J. (2022). TALCS: An Open-Source Mandarin-English Code-Switching Corpus anda Speech Recognition Baseline. arXiv.10.21437/Interspeech.2022-877 ↗
13. King, A. (2012). The Basque Language: A Practical Introduction, University of Nevada Press.
14. Grenoble, L., Lane, P., and Røyneland, U. (2019). Linguistic Minorities in Europe Online, De Gruyter Mouton.
15. Grenoble, L., Lane, P., and Røyneland, U. (2019). Linguistic Minorities in Europe Online, De Gruyter Mouton.
16. Lightly supervised and unsupervised acoustic model training10.1006/csla.2001.0186 ↗
17. Unsupervised Training of Acoustic Models for Large Vocabulary Continuous Speech Recognition10.1109/TSA.2004.838537 ↗
18. Active learning and semi-supervised learning for speech recognition: A unified framework using the global entropy reduction maximization criterion10.1016/j.csl.2009.03.004 ↗
19. Liao, H., McDermott, E., and Senior, A. (2013, January 8–12). Large scale deep neural network acoustic modeling with semi-supervised training data for YouTube video transcription. Proceedings of the ASRU, Olomouc, Czech Republic.10.1109/ASRU.2013.6707758 ↗
20. Veselý, K., Hannemann, M., and Burget, L. (2013, January 8–12). Semi-supervised training of Deep Neural Networks. Proceedings of the 2013 IEEE Workshop on Automatic Speech Recognition and Understanding, Olomouc, Czech Republic.10.1109/ASRU.2013.6707741 ↗
21. Manohar, V., Hadian, H., Povey, D., and Khudanpur, S. (2018, January 15–20). Semi-Supervised Training of Acoustic Models Using Lattice-Free MMI. Proceedings of the 2018 IEEE International Conference on Acoustics, Speech and Signal Processing, (ICASSP 2018), Calgary, AB, Canada.10.1109/ICASSP.2018.8462331 ↗
22. Large-Scale Semi-Supervised Training in Deep Learning Acoustic Model for ASR10.1109/ACCESS.2019.2940961 ↗
23. Wotherspoon, S., Hartmann, W., Snover, M., and Kimball, O. (2021, January 6–11). Improved Data Selection for Domain Adaptation in ASR. Proceedings of the ICASSP 2021—2021 IEEE International Conference on Acoustics, Speech and Signal Processing, Toronto, ON, Canada.10.1109/ICASSP39728.2021.9413869 ↗
24. Etchegoyhen, T., Arzelus, H., Gete Ugarte, H., Alvarez, A., González-Docasal, A., and Benites Fernandez, E. (2021, January 24–25). Mintzai-ST: Corpus and Baselines for Basque-Spanish Speech Translation. Proceedings of the IberSPEECH 2021, Valladolid, Spain.10.21437/IberSPEECH.2021-41 ↗
25. Penagarikano, M., Varona, A., Bordel, G., and Rodriguez-Fuentes, L.J. (2022, January 14–16). Semisupervised training of a fully bilingual ASR system for Basque and Spanish. Proceedings of the IberSPEECH 2022, Granada, Spain.10.21437/IberSPEECH.2022-8 ↗
26. Lopez de Ipina, K., Torres, I., and Onederra, L. (1995, January 18–21). Design of a phonetic corpus for a speech database in basque language. Proceedings of the 4th European Conference on Speech Communication and Technology (Eurospeech 1995), Madrid, Spain.10.21437/Eurospeech.1995-194 ↗
27. Bordel, G., Nieto, S., Penagarikano, M., Rodriguez-Fuentes, L.J., and Varona, A. (2012, January 9–13). A simple and efficient method to align very long speech signals to acoustically imperfect transcriptions. Proceedings of the Interspeech 2012, Portland, OR, USA.10.21437/Interspeech.2012-402 ↗
28. Bordel, G., Penagarikano, M., Rodriguez-Fuentes, L.J., and Varona, A. (2012, January 21–23). Aligning very long speech signals to bilingual transcriptions of parliamentary sessions. Proceedings of the Iberspeech 2012, Madrid, Spain.10.1007/978-3-642-35292-8_8 ↗
29. Ardila, R., Branson, M., Davis, K., Henretty, M., Kohler, M., Meyer, J., Morais, R., Saunders, L., Tyers, F.M., and Weber, G. (2019). Common Voice: A Massively-Multilingual Speech Corpus. arXiv.
30. Odriozola, I., Hernaez, I., Torres, M., Rodriguez-Fuentes, L.J., Penagarikano, M., and Navas, E. (2014, January 26–31). Basque Speecon-like and Basque SpeechDat MDB-600: Speech Databases for the Development of ASR Technology for Basque. Proceedings of the 9th International Conference on Language Resources and Evaluation (LREC 2014), Reykjavik, Iceland.
31. Moreno, A., Poch, D., Bonafonte, A., Lleida, E., Llisterri, J., Marino, J.B., and Nadeu, C. (1993, January 22–25). Albayzin speech database: Design of the phonetic corpus. Proceedings of the 3rd European Conference on Speech Communication and Technology (Eurospeech 1993), Berlin, Germany.10.21437/Eurospeech.1993-66 ↗
32. Quilis Morales, A. (2019). Tratado de Fonología y Fonética Españolas, Gredos.
33. Hualde, J. (2004). Basque Phonology, Taylor & Francis.10.4324/9780203168004 ↗
34. Bordel, G., Nieto, S., Penagarikano, M., Rodriguez-Fuentes, L.J., and Varona, A. (2011, January 28–31). Automatic Subtitling of the Basque Parliament Plenary Sessions Videos. Proceedings of the Interspeech 2011, Florence, Italy.10.21437/Interspeech.2011-483 ↗
35. Probabilistic Kernels for Improved Text-to-Speech Alignment in Long Audio Tracks10.1109/LSP.2015.2505140 ↗
36. Kjartansson, O., Gutkin, A., Butryna, A., Demirsahin, I., and Rivera, C. (2020, January 11–12). Open-Source High Quality Speech Datasets for Basque, Catalan and Galician. Proceedings of the 1st Joint Workshop on SLTU and CCURL, Marseille, France.
37. Collobert, R., Puhrsch, C., and Synnaeve, G. (2016). Wav2Letter: An End-to-End ConvNet-based Speech Recognition System. arXiv.
38. Heafield, K. (2011, January 30–31). KenLM: Faster and Smaller Language Model Queries. Proceedings of the Sixth Workshop on Statistical Machine Translation, Edinburgh, UK.