Comparative Study for Multi-Speaker Mongolian TTS with a New Corpus
Source abstract
Abstract
Low-resource text-to-speech synthesis is a very promising research direction. Mongolian is the official language of the Inner Mongolia Autonomous Region and is spoken by more than 10 million people worldwide. Mongolian, as a representative low-resource language, has a relative lack of open-source datasets for its TTS. Therefore, we make public an open-source multi-speaker Mongolian TTS dataset, named MnTTS2, for related researchers. In this work, we invited three Mongolian announcers to record topic-rich speeches. Each announcer recorded 10 h of Mongolian speech, and the whole dataset was 30 h in total. In addition, we built two baseline systems based on state-of-the-art neural architectures, including a multi-speaker Fastspeech 2 model with HiFi-GAN vocoder and a full end-to-end VITS model for multi-speakers. On the system of FastSpeech2+HiFi-GAN, the three speakers scored 4.0 or higher on both naturalness evaluation and speaker similarity. In addition, the three speakers achieved scores of 4.5 or higher on the VITS model for naturalness evaluation and speaker similarity scores. The experimental results show that the published MnTTS2 dataset can be used to build robust Mongolian multi-speaker TTS models.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Natural Language Processing Techniques · Speech Recognition and Synthesis · Topic Modeling
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
College of Computer Science, Inner Mongolia University, Hohhot 010031, China
Funding
High-level Talents Introduction Project of Inner Mongolia University · Young Scientists Fund of the National Natural Science Foundation of China
References (34)
Show source-provided references
1. Shen, J., Pang, R., Weiss, R.J., Schuster, M., Jaitly, N., Yang, Z., Chen, Z., Zhang, Y., Wang, Y., and Skerrv-Ryan, R. (2018, January 15–20). Natural tts synthesis by conditioning wavenet on mel spectrogram predictions. Proceedings of the 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Calgary, AB, Canada.10.1109/ICASSP.2018.8461368 ↗
2. Charpentier, F., and Stella, M. (1986, January 7–11). Diphone synthesis using an overlap-add technique for speech waveforms concatenation. Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP ’86), Tokyo, Japan.10.1109/ICASSP.1986.1168657 ↗
3. A tutorial on hidden Markov models and selected applications in speech recognition10.1109/5.18626 ↗
4. Cho, K., Van Merriënboer, B., Bahdanau, D., and Bengio, Y. (2014). On the properties of neural machine translation: Encoder-decoder approaches. arXiv.10.3115/v1/W14-4012 ↗
5. Wang, Y., Skerry-Ryan, R., Stanton, D., Wu, Y., Weiss, R.J., Jaitly, N., Yang, Z., Xiao, Y., Chen, Z., and Bengio, S. (2017, January 20–24). Tacotron: Towards End-to-End Speech Synthesis. Proceedings of the Interspeech 2017, Stockholm, Sweden.10.21437/Interspeech.2017-1452 ↗
6. Li, N., Liu, S., Liu, Y., Zhao, S., and Liu, M. (2019, January 29–31). Neural speech synthesis with transformer network. Proceedings of the AAAI Conference on Artificial Intelligence, Honolulu, HI, USA.
7. Arık, S.Ö., Chrzanowski, M., Coates, A., Diamos, G., Gibiansky, A., Kang, Y., Li, X., Miller, J., Ng, A., and Raiman, J. (2017, January 6–11). Deep voice: Real-time neural text-to-speech. Proceedings of the International Conference on Machine Learning, Sydney, Australia.
8. Gu, J., Bradbury, J., Xiong, C., Li, V.O., and Socher, R. (2017, January 24–26). Non-Autoregressive Neural Machine Translation. Proceedings of the International Conference on Learning Representations, Toulon, France.
9. Ren, Y., Ruan, Y., Tan, X., Qin, T., Zhao, S., Zhao, Z., and Liu, T.Y. (2019). Fastspeech: Fast, robust and controllable text to speech. Adv. Neural Inf. Process. Syst., 32.
10. Ren, Y., Hu, C., Tan, X., Qin, T., Zhao, S., Zhao, Z., and Liu, T.Y. (2020, January 26–30). FastSpeech 2: Fast and High-Quality End-to-End Text to Speech. Proceedings of the International Conference on Learning Representations, Addis Ababa, Ethiopia.
11. van den Oord, A., Dieleman, S., Zen, H., Simonyan, K., Vinyals, O., Graves, A., Kalchbrenner, N., Senior, A., and Kavukcuoglu, K. (2016, January 13–15). WaveNet: A Generative Model for Raw Audio. Proceedings of the 9th ISCA Speech Synthesis Workshop, Sunnyvale, CA, USA.
12. Kalchbrenner, N., Elsen, E., Simonyan, K., Noury, S., Casagrande, N., Lockhart, E., Stimberg, F., Oord, A., Dieleman, S., and Kavukcuoglu, K. (2018, January 25–31). Efficient neural audio synthesis. Proceedings of the International Conference on Machine Learning, Vienna, Austria.
13. Kumar, K., Kumar, R., de Boissiere, T., Gestin, L., Teoh, W.Z., Sotelo, J., de Brébisson, A., Bengio, Y., and Courville, A.C. (2019). Melgan: Generative adversarial networks for conditional waveform synthesis. Adv. Neural Inf. Process. Syst., 32.
14. Hifi-gan: Generative adversarial networks for efficient and high fidelity speech synthesis
15. Mongolian ethnicity and linguistic anxiety in China10.1525/aa.2003.105.4.753 ↗
16. Hu, Y., Yin, P., Liu, R., Bao, F., and Gao, G. (2022, January 27–28). MnTTS: An Open-Source Mongolian Text-to-Speech Synthesis Dataset and Accompanied Baseline. Proceedings of the 2022 International Conference on Asian Language Processing (IALP), Singapore.10.1109/IALP57159.2022.9961271 ↗
17. Kim, J., Kong, J., and Son, J. (2021, January 18–24). Conditional variational autoencoder with adversarial learning for end-to-end text-to-speech. Proceedings of the International Conference on Machine Learning, Virtual.
18. Liang, K., Liu, B., Hu, Y., Liu, R., Bao, F., and Gao, G. (2022). MnTTS2: An Open-Source Multi-Speaker Mongolian Text-to-Speech Synthesis Dataset. arXiv.
19. Ito, K., and Johnson, L. (2023, March 01). The LJ Speech Dataset. Available online: https://keithito.com/LJ-Speech-Dataset.
20. Zen, H., Dang, V., Clark, R., Zhang, Y., Weiss, R.J., Jia, Y., Chen, Z., and Wu, Y. (2019). LibriTTS: A corpus derived from LibriSpeech for text-to-speech. arXiv.10.21437/Interspeech.2019-2441 ↗
21. Veaux, C., Yamagishi, J., and MacDonald, K. (2017). CSTR VCTK Corpus: English Multi-Speaker Corpus for CSTR Voice Cloning Toolkit, The Centre for Speech Technology Research (CSTR), University of Edinburgh.
22. Shi, Y., Bu, H., Xu, X., Zhang, S., and Li, M. (2020). AISHELL-3: A Multi-speaker Mandarin TTS Corpus and the Baselines. arXiv.10.21437/Interspeech.2021-755 ↗
23. Park, D.S., Chan, W., Zhang, Y., Chiu, C.C., Zoph, B., Cubuk, E.D., and Le, Q.V. (2019). SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition. arXiv.10.21437/Interspeech.2019-2680 ↗
24. Sheng, P., Yang, Z., Hu, H., Tan, T., and Qian, Y. (2018, January 26–29). Data augmentation using conditional generative adversarial networks for robust speech recognition. Proceedings of the 2018 11th International Symposium on Chinese Spoken Language Processing (ISCSLP), Taipei City, Taiwan.10.1109/ISCSLP.2018.8706651 ↗
25. Zhou, X., Tian, X., Lee, G., Das, R.K., and Li, H. (2020, January 4–8). End-to-end code-switching tts with cross-lingual language model. Proceedings of the ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Barcelona, Spain.10.1109/ICASSP40776.2020.9054722 ↗
26. Unsupervised learning10.1162/neco.1989.1.3.295 ↗
27. Zhu, X.J. (2005). Semi-Supervised Learning Literature Survey, Department of Computer Sciences, University of Wisconsin-Madison.
28. A survey of transfer learning10.1186/s40537-016-0043-6 ↗
29. Huang, A., Bao, F., Gao, G., Shan, Y., and Liu, R. (2021, January 23–25). Mongolian emotional speech synthesis based on transfer learning and emotional embedding. Proceedings of the 2021 International Conference on Asian Language Processing (IALP), Yantai, China.10.1109/IALP54817.2021.9675192 ↗
30. Liu, R., Bao, F., Gao, G., and Wang, W. (2016, January 21–23). Mongolian prosodic phrase prediction using suffix segmentation. Proceedings of the 2016 International Conference on Asian Language Processing (IALP), Tainan, Taiwan.10.1109/IALP.2016.7875979 ↗
31. Liu, R., Bao, F., Gao, G., and Wang, Y. (2017, January 11–13). Mongolian text-to-speech system based on deep neural network. Proceedings of the National Conference on Man-Machine Speech Communication, Lianyungang, China.10.1007/978-981-10-8111-8_10 ↗
32. Liu, R., Bao, F., Gao, G., Zhang, H., and Wang, Y. (2018, January 2–6). Improving Mongolian Phrase Break Prediction by Using Syllable and Morphological Embeddings with BiLSTM Model. Proceedings of the Interspeech 2018, Hyderabad, India.10.21437/Interspeech.2018-1706 ↗
33. Zhi, T., Shi, Y., Du, W., Li, G., and Wang, D. (2021, January 18–20). M2ASR-MONGO: A Free Mongolian Speech Database and Accompanied Baselines. Proceedings of the 24th Conference of the Oriental COCOSDA International Committee for the Co-ordination and Standardisation of Speech Databases and Assessment Techniques (O-COCOSDA 2021), Singapore.10.1109/O-COCOSDA202152914.2021.9660401 ↗
34. Mean opinion score (MOS) revisited: Methods and applications, limitations and alternatives10.1007/s00530-014-0446-1 ↗