Automatic Classification of Synthetic Voices for Voice Banking Using Objective Measures
Source abstract
Abstract
Speech is the most common way of communication among humans. People who cannot communicate through speech due to partial of total loss of the voice can benefit from Alternative and Augmentative Communication devices and Text to Speech technology. One problem of using these technologies is that the included synthetic voices might be impersonal and badly adapted to the user in terms of age, accent or even gender. In this context, the use of synthetic voices from voice banking systems is an attractive alternative. New voices can be obtained applying adaptation techniques using recordings from people with healthy voice (donors) or from the user himself/herself before losing his/her own voice. In this way, the goal is to offer a wide voice catalog to potential users. However, as there is no control over the recording or the adaptation processes, some method to control the final quality of the voice is needed. We present the work developed to automatically select the best synthetic voices using a set of objective measures and a subjective Mean Opinion Score evaluation. A prediction algorithm of the MOS has been build which correlates similarly to the most correlated individual measure.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Speech Recognition and Synthesis · Speech and Audio Processing · Music and Audio Processing
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
HiTZ Basque Center for Language Technologies—Aholab, University of the Basque Country UPV/EHU, 48013 Bilbao, Spain
Funding
Basque Government · MCIN AEI
References (58)
Show source-provided references
1. How does familiarity with a voice affect trait judgements?10.1111/bjop.12454 ↗
2. Influence of speaker familiarity on blind and visually impaired children’s and young adults’ perception of synthetic voices10.1016/j.csl.2017.05.010 ↗
3. Hunt, A.J., and Black, A.W. (1996, January 9). Unit selection in a concatenative speech synthesis system using a large speech database. Proceedings of the 1996 IEEE International Conference on Acoustics, Speech, and Signal Processing Conference Proceedings, Atlanta, GA, USA.
4. Statistical parametric speech synthesis10.1016/j.specom.2009.04.004 ↗
5. Pollet, V., and Breen, A. (2008, January 22–26). Synthesis by generation and concatenation of multiform segments. Proceedings of the Ninth Annual Conference of the International Speech Communication Association, Brisbane, Australia.10.21437/Interspeech.2008-175 ↗
6. Remaking speech10.1121/1.1916020 ↗
7. Restructuring speech representations using a pitch-adaptive time–frequency smoothing and an instantaneous-frequency-based F0 extraction: Possible role of a repetitive structure in sounds10.1016/S0167-6393(98)00085-5 ↗
8. WORLD: A vocoder-based high-quality speech synthesis system for real-time applications10.1587/transinf.2015EDP7457 ↗
9. Harmonics plus Noise Model based Vocoder for Statistical Parametric Speech Synthesis10.1109/JSTSP.2013.2283471 ↗
10. Thousands of voices for HMM-based speech synthesis—Analysis and application of TTS systems built on various ASR corpora10.1109/TASL.2010.2045237 ↗
11. Suni, A., Raitio, T., Vainio, M., and Alku, P. (2010). The GlottHMM speech synthesis entry for Blizzard Challenge 2010. The Blizzard Challenge 2010 Workshop, Language Technologies Institute LTI.10.21437/Blizzard.2010-11 ↗
12. Ze, H., Senior, A., and Schuster, M. (2013, January 26–31). Statistical parametric speech synthesis using deep neural networks. Proceedings of the 2013 IEEE International Conference on Acoustics, Speech and Signal Processing, Vancouver, BC, Canada.
13. Deep learning for acoustic modeling in parametric speech generation: A systematic review of existing techniques and future trends10.1109/MSP.2014.2359987 ↗
14. Arık, S.Ö., Chrzanowski, M., Coates, A., Diamos, G., Gibiansky, A., Kang, Y., Li, X., Miller, J., Ng, A., and Raiman, J. (2017, January 6–11). Deep voice: Real-time neural text-to-speech. Proceedings of the International Conference on Machine Learning (PMLR), Sydney, NSW, Australia.
15. Arik, S., Diamos, G., Gibiansky, A., Miller, J., Peng, K., Ping, W., Raiman, J., and Zhou, Y. (2017). Deep voice 2: Multi-speaker neural text-to-speech. arXiv.
16. Ping, W., Peng, K., Gibiansky, A., Arik, S.Ö., Kannan, A., Narang, S., Raiman, J., and Miller, J. (2017). Deep Voice 3: 2000-Speaker Neural Text-to-Speech. 2017. arXiv.
17. Sotelo, J., Mehri, S., Kumar, K., Santos, J.F., Kastner, K., Courville, A., and Bengio, Y. (2022, February 10). Char2wav: End-to-End Speech Synthesis. Available online: https://mila.quebec/wp-content/uploads/2017/02/end-end-speech.pdf.
18. Wang, Y., Skerry-Ryan, R., Stanton, D., Wu, Y., Weiss, R.J., Jaitly, N., Yang, Z., Xiao, Y., Chen, Z., and Bengio, S. (2017). Tacotron: Towards end-to-end speech synthesis. arXiv.10.21437/Interspeech.2017-1452 ↗
19. Shen, J., Pang, R., Weiss, R.J., Schuster, M., Jaitly, N., Yang, Z., Chen, Z., Zhang, Y., Wang, Y., and Skerrv-Ryan, R. (2018, January 15–20). Natural tts synthesis by conditioning wavenet on mel spectrogram predictions. Proceedings of the 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Calgary, AB, Canada.10.1109/ICASSP.2018.8461368 ↗
20. Ping, W., Peng, K., and Chen, J. (2018). Clarinet: Parallel wave generation in end-to-end text-to-speech. arXiv.
21. van den Oord, A., Dieleman, S., Zen, H., Simonyan, K., Vinyals, O., Graves, A., Kalchbrenner, N., Senior, A., and Kavukcuoglu, K. (2016). Wavenet: A generative model for raw audio. arXiv.
22. Signal estimation from modified short-time Fourier transform10.1109/TASSP.1984.1164317 ↗
23. Toman, M., Meltzner, G.S., and Patel, R. (2018, January 2–6). Data Requirements, Selection and Augmentation for DNN-based Speech Synthesis from Crowdsourced Data. Proceedings of the INTERSPEECH, Hyderabad, India.10.21437/Interspeech.2018-1316 ↗
24. Taigman, Y., Wolf, L., Polyak, A., and Nachmani, E. (2017). Voiceloop: Voice fitting and synthesis via a phonological loop. arXiv.
25. Jia, Y., Zhang, Y., Weiss, R.J., Wang, Q., Shen, J., Ren, F., Chen, Z., Nguyen, P., Pang, R., and Moreno, I.L. (2018). Transfer learning from speaker verification to multispeaker text-to-speech synthesis. arXiv.
26. Cooper, E., Lai, C.I., Yasuda, Y., Fang, F., Wang, X., Chen, N., and Yamagishi, J. (2020, January 4–8). Zero-shot multi-speaker text-to-speech with state-of-the-art neural speaker embeddings. Proceedings of the ICASSP 2020—2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Barcelona, Spain.10.1109/ICASSP40776.2020.9054535 ↗
27. Erro, D., Hernáez, I., Navas, E., Alonso, A., Arzelus, H., Jauk, I., Hy, N.Q., Magariños, C., Pérez-Ramón, R., and Sulír, M. (July, January 9). ZureTTS: Online Platform for Obtaining Personalized Synthetic Voices. Proceedings of the eNTERFACE’14, Bilbao, Spain.
28. Laboratory, A.S.P. (2022, February 10). ZureTTS. Available online: https://aholab.ehu.eus/ahomytts/.
29. Alonso, A., García, V., Hernaez, I., Navas, E., and Sanchez, J. (2021, January 24–26). Automatic Speaker Adaptation Assessment Based on Objective Measures for Voice Banking Donors. Proceedings of the IBERSPEECH 2020, Valladolid, Spain.10.21437/IberSPEECH.2021-45 ↗
30. (2022, February 10). Model Talker. Available online: https://www.modeltalker.org/.
31. (2022, February 10). Speak Unique. Available online: https://www.speakunique.co.uk/.
32. (2022, February 10). VocalID. Available online: https://vocalid.ai/.
33. (2022, February 10). The Voice Keeper. Available online: https://thevoicekeeper.com/.
34. (2022, February 10). Acapela Group. Available online: http://www.acapela-group.com/solutions/my-own-voice/.
35. (2022, February 10). CereVoice. Available online: https://www.cereproc.com/en/products/cerevoiceme.
36. Speech synthesis technologies for individuals with vocal disabilities: Voice banking and reconstruction10.1250/ast.33.1 ↗
37. Building personalised synthetic voices for individuals with severe speech impairment10.1016/j.csl.2012.10.001 ↗
38. Pierard, A., Erro, D., Hernaez, I., Navas, E., and Dutoit, T. (2016, January 23–25). Surgery of speech synthesis models to overcome the scarcity of training data. Proceedings of the International Conference on Advances in Speech and Language Technologies for Iberian Languages, Lisbon, Portugal.10.1007/978-3-319-49169-1_8 ↗
39. Robust Speaker-Adaptive HMM-Based Text-to-Speech Synthesis10.1109/TASL.2009.2016394 ↗
40. Moreno Bilbao, M.A., Poig, D., Bonafonte Cávez, A., Lleida, E., Llisterri, J., Mariño Acebal, J.B., and Nadeu Camprubí, C. (1993, January 22–25). Albayzin speech database: Design of the phonetic corpus. Proceedings of the EUROSPEECH 1993: 3rd European Conference on Speech Communication and Technology, Berlin, Germany.10.21437/Eurospeech.1993-66 ↗
41. Sainz, I., Erro, D., Navas, E., Hernáez, I., Sanchez, J., Saratxaga, I., and Odriozola, I. (2012, January 23–25). Versatile Speech Databases for High Quality Synthesis for Basque. Proceedings of the Eighth International Conference on Language Resources and Evaluation (LREC’12), Istanbul, Turkey.
42. Taal, C.H., Hendriks, R.C., Heusdens, R., and Jensen, J. (2010, January 14–19). A short-time objective intelligibility measure for time-frequency weighted noisy speech. Proceedings of the 2010 IEEE International Conference on Acoustics, Speech and Signal Processing, Dallas, TX, USA.10.1109/ICASSP.2010.5495701 ↗
43. An algorithm for predicting the intelligibility of speech masked by modulated noise maskers10.1109/TASLP.2016.2585878 ↗
44. An instrumental intelligibility metric based on information theory
45. Mittag, G., and Möller, S. (2020, January 25–29). Deep Learning Based Assessment of Synthetic Speech Naturalness. Proceedings of the Interspeech 2020, Shanghai, China.10.21437/Interspeech.2020-2382 ↗
46. Speech intelligibility evaluation for mobile phones10.3813/AAA.918896 ↗
47. An algorithm for intelligibility prediction of time—Frequency weighted noisy speech10.1109/TASL.2011.2114881 ↗
48. Objective quality and intelligibility prediction for users of assistive listening devices: Advantages and limitations of existing tools10.1109/MSP.2014.2358871 ↗
49. Xia, R., Li, J., Akagi, M., and Yan, Y. (2012, January 25–30). Evaluation of objective intelligibility prediction measures for noise-reduced signals in mandarin. Proceedings of the 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Kyoto, Japan.10.1109/ICASSP.2012.6288911 ↗
50. Speech intelligibility prediction based on mutual information10.1109/TASLP.2013.2295914 ↗
51. Schlittenlacher, J., and Baer, T. (2020). Text-to-speech for the hearing impaired. arXiv.
52. Ayllón, D., Sánchez-Hevia, H.A., Figueroa, C., and Lanchantin, P. (2019, January 15–19). Investigating the Effects of Noisy and Reverberant Speech in Text-to-Speech Systems. Proceedings of the INTERSPEECH 2019, Graz, Austria.10.21437/Interspeech.2019-3104 ↗
53. Janbakhshi, P., Kodrasi, I., and Bourlard, H. (2019, January 12–17). Pathological Speech Intelligibility Assessment Based on the Short-time Objective Intelligibility Measure. Proceedings of the ICASSP 2019—2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Brighton, UK.10.1109/ICASSP.2019.8683741 ↗
54. Mittag, G., and Möller, S. (2019, January 12–17). Non-intrusive speech quality assessment for super-wideband speech communication networks. Proceedings of the ICASSP 2019—2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Brighton, UK.10.1109/ICASSP.2019.8683770 ↗
55. (2022, February 10). NISQA. Available online: https://github.com/gabrielmittag/NISQA.
56. McAuliffe, M., Socolof, M., Mihuc, S., Wagner, M., and Sonderegger, M. (2017, January 20–24). Montreal Forced Aligner: Trainable text-speech alignment using Kaldi. Proceedings of the INTERSPEECH 2017, Stockholm, Sweden.10.21437/Interspeech.2017-1386 ↗
57. Tokuda, K., Zen, H., Yamagishi, J., Black, A., Masuko, T., Sako, S., Oura, K., Hasimoto, K., Sawada, K., and Yoshimura, T. (2022, February 10). The HMM-based speech synthesis system (HTS). Available online: http://hts.sp.nitech.ac.jp.
58. Erro, D., Hernaez, I., Alonso, A., García-Lorenzo, D., Navas, E., Ye, J., Arzelus, H., Jauk, I., Hy, N.Q., and Magariños, C. (2015, January 6–10). Personalized synthetic voices for speaking impaired: Website and app. Proceedings of the Sixteenth Annual Conference of the International Speech Communication Association, Dresden, Germany.10.21437/Interspeech.2015-314 ↗