A Survey of Grapheme-to-Phoneme Conversion Methods
Source abstract
Abstract
Grapheme-to-phoneme conversion (G2P) is the task of converting letters (grapheme sequences) into their pronunciations (phoneme sequences). It plays a crucial role in natural language processing, text-to-speech synthesis, and automatic speech recognition systems. This paper provides a systematical overview of the G2P conversion from different perspectives. The conversion methods are first presented in the paper; detailed discussions are conducted on methods based on deep learning technology. For each method, the key ideas, advantages, disadvantages, and representative models are summarized. This paper then mentioned the learning strategies and multilingual G2P conversions. Finally, this paper summarized the commonly used monolingual and multilingual datasets, including Mandarin, Japanese, Arabic, etc. Two tables illustrated the performance of various methods with relative datasets. After making a general overall of G2P conversion, this paper concluded with the current issues and the future directions of deep learning-based G2P conversion.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Natural Language Processing Techniques · Topic Modeling · Speech Recognition and Synthesis
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
School of Envronment and Spatial Informatics, China University of Mining and Technology, No.1 Daxue Road, Xuzhou 221000, China
School of Computer Science and Technology, China University of Mining and Technology, No.1 Daxue Road, Xuzhou 221000, China
Department of Electrical and Computer Engineering, University of British Columbia, 2332 Main Mall, Vancouver, BC V6T 1Z4, Canada
Funding
Fundamental Research Funds for the Central Universities
References (68)
Show source-provided references
1. To-sound rules for automatic translation of English text to phonetics10.1109/TASSP.1976.1162873 ↗
2. Regular models of phonological rule systems
3. Black, A.W., Lenzo, K., and Pagel, V. (1998, January 26–29). Issues in building general letter to sound rules. Proceedings of the The Third ESCA/COCOSDA Workshop (ETRW) on Speech Synthesis, Blue Mountains, Australia.
4. Kominek, J., and Black, A.W. (2006, January 4–9). Learning pronunciation dictionaries: Language complexity and word selection strategies. Proceedings of the Human Language Technology Conference of the NAACL, Main Conference, New York, NY, USA.10.3115/1220835.1220865 ↗
5. Galescu, L., and Allen, J.F. (September, January 29). Bi-directional conversion between graphemes and phonemes using a joint n-gram model. Proceedings of the 4th ISCA Tutorial and Research Workshop (ITRW) on Speech Synthesis, Perthshire, Scotland.
6. Hahn, S., Vozila, P., and Bisani, M. (2012, January 9–13). Comparison of grapheme-to-phoneme methods on large pronunciation dictionaries and LVCSR tasks. Proceedings of the Thirteenth Annual Conference of the International Speech Communication Association, Portland, OR, USA.10.21437/Interspeech.2012-657 ↗
7. Kubo, Yotaro, T.H., and Nakamura, A. (2013, January 25–29). A method for structure estimation of weighted finite-state transducers and its application to grapheme-to-phoneme conversion. Proceedings of the INTERSPEECH, Lyon, France.10.21437/Interspeech.2013-186 ↗
8. Joint-sequence models for grapheme-to-phoneme conversion10.1016/j.specom.2008.01.002 ↗
9. Novak, J.R., Minematsu, N., and Hirose, K. (2012, January 23–25). WFST-based grapheme-to-phoneme conversion: Open source tools for alignment, model-building and decoding. Proceedings of the 10th International Workshop on Finite State Methods and Natural Language Processing, Donostia–San Sebastián, Spain.
10. Novak, J.R., Minematsu, N., Hirose, K., Hori, C., Kashioka, H., and Dixon, P.R. (2012, January 9–13). Improving WFST-based G2P Conversion with Alignment Constraints and RNNLM N-best Rescoring. Proceedings of the Interspeech, Portland, OR, USA.10.21437/Interspeech.2012-654 ↗
11. Novak, J.R., Minematsu, N., and Hirose, K. (2013, January 25–29). Failure transitions for joint n-gram models and G2p conversion. Proceedings of the Interspeech, Lyon, France.10.21437/Interspeech.2013-449 ↗
12. Taylor, P. (2005, January 4–8). Hidden Markov models for grapheme to phoneme conversion. Proceedings of the Ninth European Conference on Speech Communication and Technology, Lisboa, Portugal.10.21437/Interspeech.2005-615 ↗
13. Ogbureke, K.U., Cahill, P., and Carson-Berndsen, J. (2010, January 26–30). Hidden markov models with context-sensitive observations for grapheme-to-phoneme conversion. Proceedings of the Eleventh Annual Conference of the International Speech Communication Association, Makuhari, Chiba, Japan.10.21437/Interspeech.2010-352 ↗
14. Jiampojamarn, S., Kondrak, G., and Sherif, T. (2007, January 22–27). Applying many-to-many alignments and hidden markov models to letter-to-phoneme conversion. Proceedings of the Human Language Technologies 2007: The Conference of the North American Chapter of the Association for Computational Linguistics; Proceedings of the Main Conference, Rochester, NY, USA.
15. Lehnen, P., Hahn, S., Guta, A., and Ney, H. (2011, January 22–27). Incorporating alignments into Conditional Random Fields for grapheme to phoneme conversion. Proceedings of the 2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Prague, Czech Republic.10.1109/ICASSP.2011.5947458 ↗
16. Lehnen, P., Hahn, S., Guta, V.A., and Ney, H. (2012, January 9–13). Hidden conditional random fields with m-to-n alignments for grapheme-to-phoneme conversion. Proceedings of the Thirteenth Annual Conference of the International Speech Communication Association, Portland, OR, USA.10.21437/Interspeech.2012-661 ↗
17. Lehnen, P., Allauzen, A., Lavergne, T., Yvon, F., Hahn, S., and Ney, H. (2013, January 25–29). Structure learning in hidden conditional random fields for grapheme-to-phoneme conversion. Proceedings of the Annual Conference of the International Speech Communication Association, Lyon, France.10.21437/Interspeech.2013-544 ↗
18. Wu, K., Allauzen, C., Hall, K., Riley, M., and Roark, B. (2014, January 14–18). Encoding linear models as weighted finite-state transducers. Proceedings of the INTERSPEECH, Singapore.10.21437/Interspeech.2014-315 ↗
19. Sutskever, I., Vinyals, O., and Le, Q.V. (2014, January 8–13). Sequence to sequence learning with neural networks. Proceedings of the 27th International Conference on Neural Information Processing Systems, Cambridge, MA, USA.
20. Rao, K., Peng, F., Sak, H., and Beaufays, F. (2015, January 19–24). Grapheme-to-phoneme conversion using long short-term memory recurrent neural networks. Proceedings of the 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Brisbane, Australia.10.1109/ICASSP.2015.7178767 ↗
21. Yao, K., and Zweig, G. (2015). Sequence-to-sequence neural net models for grapheme-to-phoneme conversion. arXiv.10.21437/Interspeech.2015-134 ↗
22. Mousa, A.E.D., and Schuller, B. (2016, January 8–12). Deep bidirectional long short-term memory recurrent neural networks for grapheme-to-phoneme conversion utilizing complex many-to-many alignments. Proceedings of the Interspeech 2016, San Francisco, CA, USA.10.21437/Interspeech.2016-1229 ↗
23. Arık, S.Ö., Chrzanowski, M., Coates, A., Diamos, G., Gibiansky, A., Kang, Y., Li, X., Miller, J., Ng, A., and Raiman, J. (2017, January 7–10). Deep voice: Real-time neural text-to-speech. Proceedings of the International Conference on Machine Learning, Amsterdam, The Netherlands.
24. Juzová, M., Tihelka, D., and Vít, J. (2019, January 15–19). Unified Language-Independent DNN-Based G2P Converter. Proceedings of the INTERSPEECH, Graz, Austria.10.21437/Interspeech.2019-2335 ↗
25. Wang, C., Huang, P., Zou, Y., Zhang, H., Liu, S., Yin, X., and Ma, Z. (2023, January 4–10). LiteG2P: A fast, light and high accuracy model for grapheme-to-phoneme conversion. Proceedings of the ICASSP 2023–2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Rhodes Island, Greece.10.1109/ICASSP49357.2023.10096225 ↗
26. Chae, M.J., Park, K., Bang, J., Suh, S., Park, J., Kim, N., and Park, L. (2018, January 15–20). Convolutional sequence to sequence model with non-sequential greedy decoding for grapheme to phoneme conversion. Proceedings of the 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Calgary, AB, Canada.10.1109/ICASSP.2018.8462678 ↗
27. Yolchuyeva, S., Németh, G., and Gyires-Tóth, B. (2019). Grapheme-to-phoneme conversion with convolutional neural networks. Appl. Sci., 9.10.3390/app9061143 ↗
28. Toshniwal, S., and Livescu, K. (2016, January 13–16). Jointly learning to align and convert graphemes to phonemes with neural attention models. Proceedings of the 2016 IEEE Spoken Language Technology Workshop (SLT), San Diego, CA, USA.10.1109/SLT.2016.7846248 ↗
29. Luong, T., Pham, H., and Manning, C.D. (2015). Effective Approaches to Attention-based Neural Machine Translation. arXiv.10.18653/v1/D15-1166 ↗
30. Sun, H., Tan, X., Gan, J.W., Liu, H., Zhao, S., Qin, T., and Liu, T.Y. (2019). Token-level ensemble distillation for grapheme-to-phoneme conversion. arXiv.10.21437/Interspeech.2019-1208 ↗
31. Yolchuyeva, S., Németh, G., and Gyires-Tóth, B. (2020). Transformer based grapheme-to-phoneme conversion. arXiv.10.21437/Interspeech.2019-1954 ↗
32. Kim, H.Y., Kim, J.H., and Kim, J.M. (2021, January 6–11). Nn-kog2p: A novel grapheme-to-phoneme model for korean language. Proceedings of the ICASSP 2021–2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Toronto, ON, Canada.10.1109/ICASSP39728.2021.9414653 ↗
33. Řezáčková, M., Švec, J., and Tihelka, D. (September, January 30). T5g2p: Using text-to-text transfer transformer for grapheme-to-phoneme conversion. Proceedings of the INTERSPEECH, Brno, Czechia.
34. Řezáčková, M., Frémund, A., Švec, J., and Matoušek, J. (2023, January 5–8). T5G2P: Multilingual Grapheme-to-Phoneme Conversion with Text-to-Text Transfer Transformer. Proceedings of the Asian Conference on Pattern Recognition, Springer, Kitakyushu, Japan.10.1007/978-3-031-47665-5_27 ↗
35. T5g2p: Text-to-text transfer transformer based grapheme-to-phoneme conversion10.1109/TASLP.2024.3426332 ↗
36. Dong, L., Guo, Z.Q., Tan, C.H., Hu, Y.J., Jiang, Y., and Ling, Z.H. (2022, January 22–27). Neural grapheme-to-phoneme conversion with pre-trained grapheme models. Proceedings of the ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Singapore.10.1109/ICASSP43922.2022.9746447 ↗
37. Tsvetkov, Y., Sitaram, S., Faruqui, M., Lample, G., Littell, P., Mortensen, D., Black, A.W., Levin, L., and Dyer, C. (2016). Polyglot neural language models: A case study in cross-lingual phonetic representation learning. arXiv.10.18653/v1/N16-1161 ↗
38. Milde, B., Schmidt, C., and Köhler, J. (2017, January 20–24). Multitask Sequence-to-Sequence Models for Grapheme-to-Phoneme Conversion. Proceedings of the INTERSPEECH, Stockholm, Sweden.10.21437/Interspeech.2017-1436 ↗
39. Pritzen, J., Gref, M., Zühlke, D., and Schmidt, C. (2021). Multitask learning for grapheme-to-phoneme conversion of anglicisms in German speech recognition. arXiv.
40. Route, J., Hillis, S., Etinger, I.C., Zhang, H., and Black, A.W. (2019, January 3). Multimodal, multilingual grapheme-to-phoneme conversion for low-resource languages. Proceedings of the 2nd Workshop on Deep Learning Approaches for Low-Resource NLP (DeepLo 2019), Hong Kong, China.10.18653/v1/D19-6121 ↗
41. Vesik, K., Abdul-Mageed, M., and Silfverberg, M. (2020). One model to pronounce them all: Multilingual grapheme-to-phoneme conversion with a transformer ensemble. arXiv.10.18653/v1/2020.sigmorphon-1.16 ↗
42. Sokolov, A., Rohlin, T., and Rastrow, A. (2020). Neural machine translation for multilingual grapheme-to-phoneme conversion. arXiv.10.21437/Interspeech.2019-3176 ↗
43. Engelhart, E., Elyasi, M., and Bharaj, G. (2021). Grapheme-to-phoneme transformer model for transfer learning dialects. arXiv.
44. Ni, J., Shiga, Y., and Kawai, H. (2018, January 2–6). Multilingual Grapheme-to-Phoneme Conversion with Global Character Vectors. Proceedings of the Interspeech, Hyderabad, India.10.21437/Interspeech.2018-1626 ↗
45. Deri, A., and Knight, K. (2016, January 7–12). Grapheme-to-phoneme models for (almost) any language. Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, Berlin, Germany.10.18653/v1/P16-1038 ↗
46. Peters, B., Dehdari, J., and van Genabith, J. (2017). Massively multilingual neural grapheme-to-phoneme conversion. arXiv.10.18653/v1/W17-5403 ↗
47. Lee, J.L., Ashby, L.F., Garza, M.E., Lee-Sikka, Y., Miller, S., Wong, A., McCarthy, A.D., and Gorman, K. (2020, January 11–16). Massively multilingual pronunciation modeling with WikiPron. Proceedings of the Twelfth Language Resources and Evaluation Conference, Marseille, France.
48. Gorman, K., Ashby, L.F., Goyzueta, A., McCarthy, A.D., Wu, S., and You, D. (2020, January 10). The SIGMORPHON 2020 shared task on multilingual grapheme-to-phoneme conversion. Proceedings of the 17th SIGMORPHON Workshop on Computational Research in Phonetics, Phonology, and Morphology, Online.10.18653/v1/2020.sigmorphon-1.2 ↗
49. Mortensen, D.R., Dalmia, S., and Littell, P. (2018, January 7–12). Epitran: Precision G2P for many languages. Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018), Miyazaki, Japan.
50. Yu, M., Nguyen, H.D., Sokolov, A., Lepird, J., Sathyendra, K.M., Choudhary, S., Mouchtaris, A., and Kunzmann, S. (2020, January 4–8). Multilingual grapheme-to-phoneme conversion with byte representation. Proceedings of the ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Online.10.1109/ICASSP40776.2020.9054696 ↗
51. Kim, H.Y., Kim, J.H., and Kim, J.M. (2022, January 10–15). Fast Bilingual Grapheme-To-Phoneme Conversion. Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies: Industry Track, Seattle, DC, USA.10.18653/v1/2022.naacl-industry.32 ↗
52. Zhu, J., Zhang, C., and Jurgens, D. (2022). ByT5 model for massively multilingual grapheme-to-phoneme conversion. arXiv.10.21437/Interspeech.2022-538 ↗
53. Glocker, K., Herygers, A., and Georges, M. (2023). Allophant: Cross-lingual Phoneme Recognition with Articulatory Attributes. arXiv.10.21437/Interspeech.2023-772 ↗
54. Sejnowski, T., and Rosenberg, C. (1988). Connectionist Bench (Nettalk Corpus). Uci Mach. Learn. Repos.
55. Park, K., and Lee, S. (2020). g2pm: A neural grapheme-to-phoneme conversion package for mandarin chinese based on a new open benchmark dataset. arXiv.10.21437/Interspeech.2020-1094 ↗
56. Toward Kurdish language processing: Experiments in collecting and processing the AsoSoft text corpus
57. Rahmati, E., and Sameti, H. (2024, January 12–16). GE2PE: Persian End-to-End Grapheme-to-Phoneme Conversion. Proceedings of the Findings of the Association for Computational Linguistics: EMNLP 2024, Miami, Florida.10.18653/v1/2024.findings-emnlp.196 ↗
58. Alghmadi, M. (2003, January 3–9). KACST arabic phonetic database. Proceedings of the the Fifteenth International Congress of Phonetics Science, Barcelona, Spain.
59. Halpern, J. (2024, November 09). THE ROLE OF PHONETICS AND PHONETIC DATABASES IN JAPANESE SPEECH TECHNOLOGY. Available online: https://api.semanticscholar.org/CorpusID:43967749.
60. Kurian, C. (2016, January 26–28). Speech database and text corpora for Malayalam language automatic speech recognition technology. Proceedings of the IEEE 2016 Conference of The Oriental Chapter of International Committee for Coordination and Standardization of Speech Databases and Assessment Techniques (O-COCOSDA), Bali, Indonesia.10.1109/ICSDA.2016.7918975 ↗
61. Ipa-Dict (2016, September 20). Monolingual Wordlists with Pronunciation Information in IPA. Available online: https://github.com/open-dict-data/ipa-dict.
62. Chen, S.F. (2003, January 1–4). Conditional and joint models for grapheme-to-phoneme conversion. Proceedings of the INTERSPEECH, Geneva, Switzerland.
63. Galescu, L., and Allen, J.F. (2002, January 16–20). Pronunciation of proper names with a joint n-gram model for bi-directional grapheme-to-phoneme conversion. Proceedings of the Seventh International Conference on Spoken Language Processing, Denver, CO, USA.10.21437/ICSLP.2002-79 ↗
64. Zhao, C., Wang, J., Qu, X., Wang, H., and Xiao, J. (2022, January 23–27). r-g2p: Evaluating and enhancing robustness of grapheme to phoneme conversion by controlled noise introducing and contextual information incorporation. Proceedings of the 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Singapore.10.1109/ICASSP43922.2022.9746051 ↗
65. Wang, Y., Bao, F., Zhang, H., and Gao, G. (2021, January 6–11). Joint alignment learning-attention based model for grapheme-to-phoneme conversion. Proceedings of the ICASSP 2021–2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Toronto, ON, Canada.10.1109/ICASSP39728.2021.9413679 ↗
66. Cao, D., Zhao, Y., and Wu, L. (2023). Near-Optimal Active Learning for Multilingual Grapheme-to-Phoneme Conversion. Appl. Sci., 13.10.3390/app13169408 ↗
67. Qharabagh, M.F., Dehghanian, Z., and Rabiee, H.R. (2024). LLM-Powered Grapheme-to-Phoneme Conversion: Benchmark and Case Study. arXiv.
68. Han, D., Cui, M., Kang, J., Wu, X., Liu, X., and Meng, H. (2024). Improving Grapheme-to-Phoneme Conversion through In-Context Knowledge Retrieval with Large Language Models. arXiv.