Evaluation of Tacotron Based Synthesizers for Spanish and Basque
Source abstract
Abstract
In this paper, we describe the implementation and evaluation of Text to Speech synthesizers based on neural networks for Spanish and Basque. Several voices were built, all of them using a limited number of data. The system applies Tacotron 2 to compute mel-spectrograms from the input sequence, followed by WaveGlow as neural vocoder to obtain the audio signals from the spectrograms. The limited number of data used for training the models leads to synthesis errors in some sentences. To automatically detect those errors, we developed a new method that is able to find the sentences that have lost the alignment during the inference process. To mitigate the problem, we implemented a guided attention providing the system with the explicit duration of the phonemes. The resulting system was evaluated to assess its robustness, quality and naturalness both with objective and subjective measures. The results reveal the capacity of the system to produce good quality and natural audios.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Speech Recognition and Synthesis · Speech and Audio Processing · Music and Audio Processing
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
HiTZ Basque Center for Language Technologies—Aholab, University of the Basque Country UPV/EHU, 48013 Bilbao, Spain
Funding
Basque Government · MCIN/AEI/10.13039/501100011033
References (34)
Show source-provided references
1. Black, A.W., and Taylor, P. (1997, January 22–25). Automatically Clustering Similar Units for Unit Selection Speech Synthesis. Proceedings of the EUROSPEECH 1997, Rhodes, Greece.10.21437/Eurospeech.1997-219 ↗
2. Campbell, N., and Black, A.W. (1997). Prosody and the Selection of Source Units for Concatenative Synthesis. Progress in Speech Synthesis, Springer.10.1007/978-1-4612-1894-4_22 ↗
3. Wu, Y.J., and Wang, R.H. (2006, January 14–19). Minimum generation error training for HMM-based speech synthesis. Proceedings of the IEEE International Conference on Acoustics Speech and Signal Processing Proceedings (ICASSP), Toulouse, France.
4. Statistical parametric speech synthesis10.1016/j.specom.2009.04.004 ↗
5. Ze, H., Senior, A., and Schuster, M. (2013, January 26–31). Statistical parametric speech synthesis using deep neural networks. Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Vancouver, BC, Canada.
6. Fan, Y., Qian, Y., Xie, F.L., and Soong, F.K. (2014, January 14–18). TTS synthesis with bidirectional LSTM based recurrent neural networks. Proceedings of the INTERSPEECH 2014, Singapore.10.21437/Interspeech.2014-443 ↗
7. Qian, Y., Fan, Y., Hu, W., and Soong, F.K. (2014, January 4–9). On the training aspects of deep neural network (DNN) for parametric TTS synthesis. Proceedings of the 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Florence, Italy.10.1109/ICASSP.2014.6854318 ↗
8. Oord, A.v.d., Dieleman, S., Zen, H., Simonyan, K., Vinyals, O., Graves, A., Kalchbrenner, N., Senior, A., and Kavukcuoglu, K. (2016). Wavenet: A generative model for raw audio. arXiv.
9. Arık, S.Ö., Chrzanowski, M., Coates, A., Diamos, G., Gibiansky, A., Kang, Y., Li, X., Miller, J., Ng, A., and Raiman, J. (2017, January 6–11). Deep voice: Real-time neural text-to-speech. Proceedings of the International Conference on Machine Learning, Sydney, Australia.
10. Gibiansky, A., Arik, S.Ö., Diamos, G.F., Miller, J., Peng, K., Ping, W., Raiman, J., and Zhou, Y. (2017, January 4–9). Deep Voice 2: Multi-Speaker Neural Text-to-Speech. Proceedings of the Neural Information Processing Systems 2017, Long Beach, CA, USA.
11. Ping, W., Peng, K., Gibiansky, A., Arik, S.O., Kannan, A., Narang, S., Raiman, J., and Miller, J. (May, January 30). Deep voice 3: 2000-speaker neural text-to-speech. Proceedings of the ICLR, Vancouver, BC, Canada.
12. Sotelo, J., Mehri, S., Kumar, K., Santos, J.F., Kastner, K., Courville, A., and Bengio, Y. (2017, January 24–26). Char2wav: End-to-end speech synthesis. Proceedings of the 5th International Conference on Learning Representations, Toulon, France.
13. Shen, J., Pang, R., Weiss, R.J., Schuster, M., Jaitly, N., Yang, Z., Chen, Z., Zhang, Y., Wang, Y., and Skerrv-Ryan, R. (2018, January 15–20). Natural TTS synthesis by conditioning wavenet on mel spectrogram predictions. Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Calgary, AB, Canada.10.1109/ICASSP.2018.8461368 ↗
14. Sequence to sequence learning with neural networks
15. Prenger, R., Valle, R., and Catanzaro, B. (2019, January 12–17). Waveglow: A flow-based generative network for speech synthesis. Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Brighton, UK.10.1109/ICASSP.2019.8683143 ↗
16. Glow: Generative flow with invertible 1 × 1 convolutions
17. Chung, Y.A., Wang, Y., Hsu, W.N., Zhang, Y., and Skerry-Ryan, R.J. (2019, January 12–17). Semi-supervised Training for Improving Data Efficiency in End-to-end Speech Synthesis. Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Brighton, UK.10.1109/ICASSP.2019.8683862 ↗
18. Pre-alignment guided attention for improving training efficiency and model stability in end-to-end speech synthesis10.1109/ACCESS.2019.2914149 ↗
19. Mittag, G., and Möller, S. (2020, January 25–29). Deep Learning Based Assessment of Synthetic Speech Naturalness. Proceedings of the INTERSPEECH 2020, Shanghai, China.10.21437/Interspeech.2020-2382 ↗
20. Garcia, V., Hernaez, I., and Navas, E. (2021, January 24–25). Implementation of neural network based synthesizers for Spanish and Basque. Proceedings of the IberSPEECH 2021, Valladolid, Spain.10.21437/IberSPEECH.2021-48 ↗
21. Erro, D., Sainz, I., Luengo, I., Odriozola, I., Sánchez, J., Saratxaga, I., Navas, E., and Hernáez, I. (2010, January 10–12). HMM-based speech synthesis in Basque language using HTS. Proceedings of the FALA. RTTH, Vigo, Spain.
22. Wells, J., Barry, W., Grice, M., Fourcin, A., and Gibbon, D. (2020, December 20). Standard Computer-Compatible Transcription. Esprit Project 2589 (SAM), Doc. no. SAM-UCL-037, 1992; Volume 37. Available online: https://www.phon.ucl.ac.uk/home/sampa/.
23. MINTZAI: Sistemas de Aprendizaje Profundo E2E para Traducción Automática del Habla MINTZAI: End-to-end Deep Learning for Speech Translation
24. Ren, Y., Qin, T., Ruan, Y., Zhao, S., Liu, T.Y., Tan, X., and Zhao, Z. (2019). FastSpeech: Fast, robust and controllable text to speech. arXiv.
25. Chorowski, J., Bahdanau, D., Serdyuk, D., Cho, K., and Bengio, Y. (2014, January 8–13). Attention-Based Models for Speech Recognition. Proceedings of the 28th International Conference on Neural Information Processing Systems—Volume 1, Montreal, QC, Canada.
26. Normalizing flows: An introduction and review of current methods10.1109/TPAMI.2020.2992934 ↗
27. Ito, K., and Johnson, L. (2020, December 20). The LJ Speech Dataset, v1.1. Available online: https://keithito.com/LJ-Speech-Dataset/.
28. He, M., Deng, Y., and He, L. (2019, January 15–19). Robust Sequence-to-Sequence Acoustic Modeling with Stepwise Monotonic Attention for Neural TTS. Proceedings of the INTERSPEECH 2019, ISCA, Graz, Austria.10.21437/Interspeech.2019-1972 ↗
29. Battenberg, E., Skerry-Ryan, R.J., Mariooryad, S., Stanton, D., Kao, D., Shannon, M., and Bagby, T. (2020, January 4–8). Location-Relative Attention Mechanisms for Robust Long-Form Speech Synthesis. Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Barcelona, Spain.10.1109/ICASSP40776.2020.9054106 ↗
30. Liu, R., Sisman, B., Li, J., Bao, F., Gao, G., and Li, H. (2020, January 4–8). Teacher-student training for robust tacotron-based tts. Proceedings of the ICASSP 2020–2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Barcelona, Spain.10.1109/ICASSP40776.2020.9054681 ↗
31. McAuliffe, M., Socolof, M., Mihuc, S., Wagner, M., and Sonderegger, M. (2017, January 20–24). Montreal forced aligner: Trainable text-speech alignment using kaldi. Proceedings of the INTERSPEECH 2017, ISCA, Stockholm, Sweden.10.21437/Interspeech.2017-1386 ↗
32. (2020, March 10). NVIDIA. Available online: https://ngc.nvidia.com/catalog/models/nvidia:waveglow_ljs_256channels.
33. (2021, January 12). NVIDIA. Available online: https://drive.google.com/file/d/1rpK8CzAAirq9sWZhe9nlfvxMF1dRgFbF/view.
34. Ling, Z.H., Zhou, X., and King, S. (2021, January 23). The Blizzard Challenge 2021. Proceedings of the Blizzard Challenge Workshop 2021, Online.10.21437/Blizzard.2021-1 ↗