The Development and Experimental Evaluation of a Multilingual Speech Corpus for Low-Resource Turkic Languages
Source abstract
Abstract
The development of parallel audio corpora for Turkic languages, such as Kazakh, Uzbek, and Tatar, remains a significant challenge in the development of multilingual speech synthesis, recognition systems, and machine translation. These languages are low-resource in speech technologies, lacking sufficiently large audio datasets with aligned transcriptions that are crucial for modern recognition, synthesis, and understanding systems. This article presents the development and experimental evaluation of a speech corpus focused on Turkic languages, intended for use in speech synthesis and automatic translation tasks. The primary objective is to create parallel audio corpora using a cascade generation method, which combines artificial intelligence and text-to-speech (TTS) technologies to generate both audio and text, and to evaluate the quality and suitability of the generated data. To evaluate the quality of synthesized speech, metrics measuring naturalness, intonation, expressiveness, and linguistic adequacy were applied. As a result, a multimodal (Kazakh–Turkish, Kazakh–Tatar, Kazakh–Uzbek) corpus was created, combining high-quality natural Kazakh audio with transcription and translation, along with synthetic audio in Turkish, Tatar, and Uzbek. These corpora offer a unique resource for speech and text processing research, enabling the integration of ASR, MT, TTS, and speech-to-speech translation (STS).
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Speech Recognition and Synthesis · Face recognition and analysis · Advanced Computational Techniques in Science and Engineering
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Faculty of Information Technology and Artificial Intelligence, Farabi University, Almaty 050040, Kazakhstan
Funding
Ministry of Science and Higher Education of the Republic of Kazakhstan
References (88)
Show source-provided references
1. Mussakhojayeva, S., Dauletbek, K., Yeshpanov, R., and Varol, H.A. (2023). Multilingual Speech Recognition for Turkic Languages. Information, 14.10.3390/info14020074 ↗
2. Bekarystankyzy, A., Mamyrbayev, O., Mendes, M., Fazylzhanova, A., and Assam, M. (2024). Multilingual end-to-end ASR for low-resource Turkic languages with common alphabets. Sci. Rep., 14.10.1038/s41598-024-64848-1 ↗
3. (2025, July 02). Whisper. Available online: https://github.com/openai/whisper.
4. (2025, July 15). MMS (Massively Multilingual Speech). Available online: https://github.com/facebookresearch/fairseq/tree/main/examples/mms.
5. (2025, July 20). Soyle. Available online: https://github.com/IS2AI/Soyle.
6. Duquenne, P.-A., Gong, H., Dong, N., Du, J., Lee, A., Goswami, V., Wang, C., Pino, J., Sagot, B., and Schwenk, H. (2023, January 9–14). SpeechMatrix: A Large-Scale Mined Corpus of Multilingual Speech-to-Speech Translations. Proceedings of the 61st Annual Meeting of the As-sociation for Computational Linguistics (Volume 1: Long Papers), Toronto, ON, Canada.10.18653/v1/2023.acl-long.899 ↗
7. Salesky, E., Wiesner, M., Bremerman, J., Cattoni, R., Negri, M., Turchi, M., Oard, D.W., and Post, M. (September, January 30). The Multilingual TEDx Corpus for Speech Recognition and Translation. Proceedings of the 22nd Annual Conference of the International Speech Communication Association (Interspeech 2021), Brno, Czechia.10.21437/Interspeech.2021-11 ↗
8. Wang, C., Riviere, M., Lee, A., Wu, A., Talnikar, C., Haziza, D., Williamson, M., Pino, J., and Dupoux, E. (2021, January 1–6). VoxPopuli: A Large-Scale Multilingual Speech Corpus for Representation Learning, Semi-Supervised Learning and Interpretation. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), Online.10.18653/v1/2021.acl-long.80 ↗
9. Dogan-Schönberger, P., Mäder, J., and Hofmann, T. (2021). SwissDial: Parallel Multidialectal Corpus of Swiss German Speech and Text. arXiv.
10. Gong, H., Dong, N., Popuri, S., Goswami, V., Lee, A., and Pino, J. (2023). Multilingual Speech-to-Speech Translation into Multiple Target Languages. arxiv.
11. Ardila, R., Branson, M., Davis, K., Kohler, M., Meyer, J., Henretty, M., Morais, R., Saunders, L., Tyers, F.M., and Weber, G. (2020, January 11–16). Common Voice: A Massively-Multilingual Speech Corpus. Proceedings of the 12th Conference on Language Resources and Evaluation (LREC 2020), Marseille, France.
12. Jia, Y., Ramanovich, M.T., Wang, Q., and Zen, H. (2022, January 20–25). CVSS Corpus and Massively Multilingual Speech-to-Speech Translation. Proceedings of the 13th Conference on Language Resources and Evaluation (LREC 2022), Marseille, France.
13. Baali, M., El-Hajj, W., and Ali, A. (2022). Creating Speech-to-Speech Corpus from Dubbed Series. arXiv.
14. Di Gangi, M.A., Cattoni, R., Bentivogli, L., Negri, M., and Turchi, M. (2019, January 2–7). MuST-C: A Multilingual Speech Translation Corpus. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Minneapolis, MN, USA.
15. Singhal, B., Naini, A.R., and Ghosh, P.K. (2021, January 18–20). wSPIRE: A Parallel Multi-Device Corpus in Neutral and Whispered Speech. Proceedings of the 24th Conference of the Oriental COCOSDA International Committee for the Co-Ordination and Standardisation of Speech Databases and Assessment Techniques (O-COCOSDA), Singapore.10.1109/O-COCOSDA202152914.2021.9660449 ↗
16. Mussakhojayeva, S., Khassanov, Y., and Varol, H.A. (2022, January 20–25). KazakhTTS2: Extending the Open-Source Kazakh TTS Corpus with More Data, Speakers, and Topics. Proceedings of the 13th International Conference on Language Resources and Evaluation (LREC 2022), Marseille, France.
17. Kozhirbayev, Z., and Islamgozhayev, T. (2023). Cascade Speech Translation for the Kazakh Language. Appl. Sci., 13.10.3390/app13158900 ↗
18. National Corpus of the Tatar Language “Tugan Tel”: Grammatical Annotation and Implementation10.1016/j.sbspro.2013.10.623 ↗
19. Grammatical Disambiguation in the Tatar National Corpus
20. Ibragimov, T.I., and Saikhunov, M.R. (2014). Written Corpus of the Tatar Language: Structural and Functional Characteristics. Current Issues of Dialectology of the Languages of the Peoples of Russia, Proceedings of the 14th All-Russian Scientific Conference, Pereslavl-Zalessky, Russia, 20–22 November 2014, Kazan Federal University. (In Russian).
21. Saikhunov, M.R., Khusainov, R.R., and Ibragimov, T.I. (2017). System of Complex Morphological Search in the Written Corpus of the Tatar Language. Traditional Culture of Turkic Peoples in a Changing World, Proceedings of the 1st International Scientific Conference, Kazan, Russia, 12–15 April 2017, Ministry of Culture of the Republic of Tatarstan, the Republican Center for Development of Traditional Culture, and Kazan State Conservatoire named after N.G. Zhiganov. (In Russian).
22. Saikhunov, M.R., Khusainov, R.R., and Ibragimov, T.I. (2019). Challenges in Creating a Text Corpus Exceeding 400 Million Tokens. The Finno-Ugric World in the Multiethnic Space of Russia: Cultural Heritage and New Challenges, Proceedings of the 6th All-Russian Scientific Conference of Finno-Ugric Studies, Izhevsk, Russia, 4–7 June 2019, Udmurt State University. (In Russian).
23. Mindubaev, A., and Gatiatullin, A. (2024, January 15–17). Problems of Semantic Relation Extraction from Tatar Text Corpus “Tugan Tel”. Proceedings of the IEEE 3rd International Conference on Problems of Informatics, Electronics and Radio Engineering (PIERE), Novosibirsk, Russia.10.1109/PIERE62470.2024.10804976 ↗
24. Incorporation of Iterative Self-supervised Pre-training in the Creation of the ASR System for the Tatar Language
25. Orel, D., Kuzdeuov, A., Gilmullin, R., Khakimov, B., and Varol, H.A. (2024, January 19–22). TatarTTS: An Open-Source Text-to-Speech Synthesis Dataset for the Tatar Language. Proceedings of the 6th International Conference on Artificial Intelligence in Information and Communication (ICAIIC), Osaka, Japan.10.36227/techrxiv.170723255.52161895/v1 ↗
26. Mussakhojayeva, S., Gilmullin, R., Khakimov, B., Galimov, M., Orel, D., Abilbekov, A., and Varol, H.A. (2024, January 19–22). Noise-Robust Multilingual Speech Recognition and the Tatar Speech Corpus. Proceedings of the 6th International Conference on Artificial Intelligence in Information and Communication (ICAIIC), Osaka, Japan.10.22541/au.170750106.64166861/v1 ↗
27. Musaev, M., Khujayorov, I., and Ochilov, M. (2021). Automatic recognition of Uzbek speech based on integrated neural networks. 11th World Conference “Intelligent System for Industrial Automation” (WCIS-2020), Springer.10.1007/978-3-030-68004-6_28 ↗
28. Musaev, M., Khujayorov, I., and Ochilov, M. (2020, January 7–9). Development of integral model of speech recognition system for Uzbek language. Proceedings of the IEEE International Conference on Application of Information and Communication Technologies (AICT), Tashkent, Uzbekistan.10.1109/AICT50176.2020.9368719 ↗
29. Musaev, M., Mussakhojayeva, S., Khujayorov, I., Khassanov, Y., Ochilov, M., and Varol, H.A. (2021). USC: An Open-Source Uzbek Speech Corpus and Initial Speech Recognition Experiments. arXiv.10.1007/978-3-030-87802-3_40 ↗
30. Povey, A., and Povey, K. (2024, January 19–20). FERUZASPEECH: A 60 Hour Uzbek Read Speech Corpus with Punctuation, Casing, and Context. Proceedings of the 7th International Conference on Natural Language and Speech Processing (ICNLSP 2024), Trento, Italy.
31. A detailed survey of Turkish automatic speech recognition
32. Zen, H., and Sak, H. (2015, January 19–24). Unidirectional Long Short-Term Memory Recurrent Neural Network with Recurrent Output Layer for Low-Latency Speech Synthesis. Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), South Brisbane, Australia.10.1109/ICASSP.2015.7178816 ↗
33. Oyucu, S. (2023). A Novel End-to-End Turkish Text-to-Speech (TTS) System via Deep. Electronics, 12.10.3390/electronics12081900 ↗
34. Mercan, O.B., Cepni, S., Tasar, D.E., and Ozan, Ş. (2023). Performance Comparison of Pre-Trained Models for Speech-to-Text in Turkish: Whisper Small and Wav2Vec2 XLS-R 300M. arXiv.
35. Polat, H., Turan, A.K., Koçak, C., and Ulaş, H.B. (2024). Implementation of a Whisper Architecture-Based Turkish Automatic Speech Recognition (ASR) System and Evaluation of the Effect of Fine-Tuning with a Low-Rank Adaptation (LoRA) Adapter on Its Performance. Electronics, 13.10.3390/electronics13214227 ↗
36. Transformer-Based Turkish Automatic Speech Recognition
37. Salor, Ö., Pellom, B.L., Çılgın, T., Hacıoğlu, K., and Demirekler, M. (2002, January 16–20). On Developing New Text and Audio Corpora and Speech Recognition Tools for the Turkish Language. Proceedings of the 7th International Conference on Spoken Language Processing (ICSLP 2002), Denver, CO, USA.10.21437/ICSLP.2002-152 ↗
38. Turkish Speech Corpora and Recognition Tools Developed by Porting SONIC: Towards Multilingual Speech Recognition10.1016/j.csl.2007.01.001 ↗
39. A Corpus-Based Study on Turkish Spoken Productions of Bilingual Adults10.13189/ujer.2016.040913 ↗
40. Polat, H., and Oyucu, S. (2020). Building a Speech and Text Corpus of Turkish: Large Corpus Collection with Initial Speech Recognition Results. Symmetry, 12.10.3390/sym12020290 ↗
41. (2025, August 01). Institute of Smart Systems and Artificial Intelligence. Available online: https://issai.nu.edu.kz/.
42. (2025, August 01). Kazakh Language Text-to-Speech—2. Available online: https://issai.nu.edu.kz/tts2-eng/.
43. (2025, August 01). Kazakh Speech Corpus 2. Available online: https://github.com/IS2AI/Kazakh_ASR.
44. (2025, August 01). KazTTS. Available online: https://github.com/IS2AI/Kazakh_TTS.
45. National Corpus of Kazakh Language (2025, August 01). General Information. Available online: https://qazcorpus.kz/header/jalpyMalimeten.php.
46. (2025, August 01). National Corpus of Kazakh Language. Available online: https://qazcorpus.kz/indexen.php.
47. (2025, August 01). TatSC_ASR (Tatar Speech Corpus). Available online: https://huggingface.co/datasets/yasalma/TatSC_ASR.
48. (2025, August 01). Hackathon Tatar ASR. Available online: https://huggingface.co/datasets/yasalma/tat_hackathon_asr.
49. (2025, August 01). TatarTTS. Available online: https://github.com/IS2AI/TatarTTS.
50. (2025, August 01). Tugan Tel. Available online: https://tugantel.tatar/.
51. (2025, August 01). Corpus of Written Tatar. Available online: https://www.corpus.tatar/.
52. (2025, August 01). Tatar–English Corpus. Available online: https://huggingface.co/datasets/yasalma/tt-en-language-corpus.
53. (2025, August 01). Uzbek Speech Corpus (USC). Available online: https://huggingface.co/datasets/issai/Uzbek_Speech_Corpus.
54. (2025, August 01). FERUZASPEECH. Available online: https://huggingface.co/datasets/k2speech/FeruzaSpeech.
55. (2025, August 01). Common Voice. Available online: https://commonvoice.mozilla.org/en/datasets.
56. (2025, August 01). Uzbek Speech Recognition Corpus (Mobile). Available online: https://dataoceanai.com/datasets/asr/uzbek-speech-recognition-corpus-mobile-3/?utm_source=chatgpt.com.
57. (2025, August 01). Uzbek Text Corpora. Available online: https://www.sketchengine.eu/corpora-and-languages/uzbek-text-corpora/.
58. (2025, August 01). Uzbek National Corpus. Available online: https://uzbekcorpus.uz/enNashrlar.
59. (2025, August 01). TIL Project: Uzbek-English Parallel Corpus. Available online: http://uzbekcorpora.uz/.
60. (2025, August 01). Leipzig Corpora Collection: Uzbek (uzb_community_2017). Available online: https://corpora.uni-leipzig.de/en?corpusId=uzb_community_2017.
61. (2025, August 01). Uzbek Corpus Sample. Available online: https://github.com/elmurod1202/Uzbek-Corpus-Sample/tree/main.
62. Text Classification Dataset and Analysis for Uzbek Language (2025, August 01). Uzbek Text Classification Dataset. Available online: https://github.com/elmurod1202/textclassification?tab=readme-ov-file.
63. (2025, August 01). Open Parallel Corpora (OPUS). Available online: https://opus.nlpl.eu/.
64. (2006, January 17–19). METU Turkish Microphone Speech Corpus. Proceedings of the 2006 IEEE 14th Signal Processing and Communications Applications, Antalya, Turkey.
65. (2025, August 01). METU Turkish Microphone Speech v1.0—LDC Catalog. Available online: https://catalog.ldc.upenn.edu/LDC2006S33.
66. (2025, August 01). ASR-BigTurkCSC: Turkish Conversational Speech Corpus. Available online: https://magichub.com/datasets/turkish-conversational-speech-corpus/.
67. (2025, August 01). PhonBank Turkish Istanbul Corpus. Available online: https://talkbank.org/phon/access/Other/Turkish/Istanbul.html.
68. (2025, August 01). ITU Turkish Natural Language Processing Pipeline Prepared by ITU NLP Group. Available online: http://tools.nlp.itu.edu.tr/.
69. Say, B., Zeyrek, D., Oflazer, K., and Özge, U. (2002, January 7–9). Development of a Corpus and a Treebank for Present-Day Written Turkish. Proceedings of the Eleventh International Conference of Turkish Linguistics, Eastern Mediterranean University, Famagusta, Cyprus.
70. European Language Grid (ELG) (2025, August 01). METU Turkish Corpus (MTC)—Resource Catalogue. Available online: https://live.european-language-grid.eu/catalogue/corpus/15024.
71. (2025, August 01). TS Corpus: A Large-Scale Turkish Corpus with Morphological Annotation. Available online: https://tscorpus.com.
72. Resources for Turkish natural language processing: A critical survey10.1007/s10579-022-09605-4 ↗
73. Kilgarriff, A., Baisa, V., Bušta, J., Jakubíček, M., Kovář, V., Michelfeit, J., Rychlý, P., and Suchomel, V. (2014). The Sketch Engine: Ten Years on. Lexicography, Springer.10.1007/s40607-014-0009-9 ↗
74. Aksan, Y., Aksan, M., Kolcu, E., and Yildiz, I. (2012, January 23–25). Construction of the Turkish National Corpus (TNC). Proceedings of the Eighth International Conference on Language Resources and Evaluation (LREC’12), Istanbul, Turkey.
75. Turkish National Corpus (TNC) Official Website (2025, December 02). Türkçe Ulusal Derlemİ. Available online: https://live.european-language-grid.eu/catalogue/corpus/12920.
76. The Linguist List (2025, August 01). Demo Access Announcement for Turkish National Corpus (TNC). Available online: https://linguistlist.org/issues/24/1024/.
77. ITU NLP Group (2025, August 01). trWaC: Turkish Web as Corpus (Free CLARIN Version). Available online: https://nlp.itu.edu.tr/en/toolsandresources.
78. Sulubacak, U., Gökırmak, M., Tyers, F.M., Çöltekin, Ç., Nivre, J., and Eryiğit, G. (2016, January 11–16). Universal Dependencies for Turkish. Proceedings of the 26th International Conference on Computational Linguistics: Technical Papers (COLING 2016), Osaka, Japan.
79. Resources for Turkish dependency parsing: Introducing the BOUN Treebank and the BoAT annotation tool10.1007/s10579-021-09558-0 ↗
80. The ParlaMint corpora of parliamentary proceedings10.1007/s10579-021-09574-0 ↗
81. (2025, June 02). MMS: Multilingual Text-to-Speech (TTS). Available online: https://huggingface.co/facebook/mms-tts.
82. (2025, July 10). TurkicTTS. Available online: https://github.com/IS2AI/TurkicTTS.
83. (2025, July 20). ElevenLabs TTS. Available online: https://elevenlabs.io/docs/capabilities/text-to-speech.
84. (2025, July 15). CoquiTTS. Available online: https://github.com/coqui-ai/TTS.
85. Lo, C.-C., Fu, S.-W., Huang, W.-C., Wang, X., Yamagishi, J., Tsao, Y., and Wang, H.-M. (2019, January 15–19). MOSNet: Deep Learning-Based Objective Assessment for Voice Conversion. Proceedings of the 20th Annual Conference of the International Speech Communication Association (Interspeech 2019), Graz, Austria.10.21437/Interspeech.2019-2003 ↗
86. Reddy, C.K., Gopal, V., and Cutler, R. (2021, January 6–11). Dnsmos: A Non-Intrusive Perceptual Objective Speech Quality Metric to Evaluate Noise Suppressors. Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Toronto, ON, Canada.10.1109/ICASSP39728.2021.9414878 ↗
87. Manocha, P., and Kumar, A. (2022, January 18–22). Speech Quality Assessment through MOS using Non-Matching References. Proceedings of the 23rd Annual Conference of the International Speech Communication Association (Interspeech 2022), Incheon, Republic of Korea.10.21437/Interspeech.2022-407 ↗
88. Yang, Y., Wang, H., Han, B., Liu, S., Li, J., Qin, Y., and Chen, X. (2025). Towards Responsible Evaluation for Text-to-Speech. arXiv.