Acoustic Data-Driven Subword Units Obtained through Segment Embedding and Clustering for Spontaneous Speech Recognition
Source abstract
Abstract
We propose a method to extend a phoneme set by using a large amount of broadcast data to improve the performance of Korean spontaneous speech recognition. In the proposed method, we first extract variable-length phoneme-level segments from broadcast data and then convert them into fixed-length embedding vectors based on a long short-term memory architecture. We use decision tree-based clustering to find acoustically similar embedding vectors and then build new acoustic subword units by gathering the clustered vectors. To update the lexicon of a speech recognizer, we build a lookup table between the tri-phone units and the units derived from the decision tree. Finally, the proposed lexicon is obtained by updating the original phoneme-based lexicon by referencing the lookup table. To verify the performance of the proposed unit, we compare the proposed unit with the previous units obtained by using the segment-based k-means clustering method or the frame-based decision-tree clustering method. As a result, the proposed unit is shown to produce better performance than the previous units in both spontaneous, and read Korean speech recognition tasks.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Speech Recognition and Synthesis · Speech and Audio Processing · Music and Audio Processing
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
School of Electronics Engineering, Chungbuk National University, 1 Chungdae-ro, Seowon-gu, Cheongju, Chungbuk 28644, Korea
Artificial Intelligence Research Laboratory, Electronics and Telecommunications Research Institute (ETRI), 218 Gajeong-ro, Yuseong-gu, Daejeon 34129, Korea
References (36)
Show source-provided references
1. Subword modeling for automatic speech recognition: Past, present, and emerging approaches10.1109/MSP.2012.2210952 ↗
2. Chomsky, N., and Halle, M. (1968). The Sound Pattern of English, Harper and Row.
3. Sainath, T.N., Prabhavalkar, R., Kumar, S., Lee, S., Kannan, A., Rybach, D., Schoglo, V., Nguyen, P., Li, B., and Wu, Y. (2018, January 15–20). No need for a lexicon? Evaluating the value of the pronunciation lexica in end-to-end models. Proceedings of the 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Calgary, AB, Canada.10.1109/ICASSP.2018.8462380 ↗
4. Young, S.J., Odell, J.J., and Woodland, P.C. (1994, January 8–11). Tree-based state tying for high accuracy acoustic modelling. Proceedings of the Workshop on Human Language Technology, Plainsboro, NJ, USA.10.3115/1075812.1075885 ↗
5. Implicit modelling of pronunciation variation in automatic speech recognition10.1016/j.specom.2005.03.008 ↗
6. Lee, K.-N., and Chung, M. (2003, January 1–4). Modeling cross-morpheme pronunciation variations for Korean large vocabulary continuous speech recognition. Proceedings of the 8th European Conference on Speech Communication and Technology (EUROSPEECH), Geneva, Switzerland.10.21437/Eurospeech.2003-114 ↗
7. Differences between acoustic characteristics of spontaneous and read speech and their effects on speech recognition performance10.1016/j.csl.2007.07.003 ↗
8. Nakamura, M., Furui, S., and Iwano, K. (2006, January 20). Acoustic and linguistic characterization of spontaneous speech. Proceedings of the Workshop on Speech Recognition and Intrinsic Variation (SRIV), Toulouse, France.
9. Bang, J.-U., Choi, M.-Y., Kim, S.-H., and Kwon, O.-W. (2019, January 15–19). Extending an acoustic data-driven phone set for spontaneous speech recognition. Proceedings of the Annual Conference of the International Speech Communication Association (INTERSPEECH), Graz, Austria.10.21437/Interspeech.2019-1979 ↗
10. High accuracy phone recognition using context clustering and quasi-triphonic models10.1006/csla.1994.1006 ↗
11. Towards weakly supervised acoustic subword unit discovery and lexicon development using hidden Markov models10.1016/j.specom.2017.11.011 ↗
12. MacQueen, J. (1967, January 18–21). Some methods for classification and analysis of multivariate observations. Proceedings of the 5th Berkeley Symposium on Mathematical Statistics and Probability, Berkeley, CA, USA.
13. Lee, C.H., Soong, F.K., and Juang, B.H. (1988, January 11–14). A segment model based approach to speech recognition. Proceedings of the 1988 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), New York, NY, USA.
14. Svendsen, T., Paliwal, K.K., Harborg, E., and Husoy, P.O. (1989, January 23–26). An improved sub-word based speech recognizer. Proceedings of the 1989 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Glasgow, Scotland.
15. Paliwal, K.K. (1990, January 3–6). Lexicon-building methods for an acoustic sub-word based speech recognizer. Proceedings of the 1990 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Albuquerque, NM, USA.
16. Bacchiani, M., and Ostendorf, M. (December, January 30). Using automatically-derived acoustic sub-word units in large vocabulary speech recognition. Proceedings of the 5th International Conference on Spoken Language Processing (ICSLP), Sydney, Australia.
17. Holter, T., and Svendsen, T. (1997, January 14–17). Combined optimisation of baseforms and model parameters in speech recognition based on acoustic subword units. Proceedings of the 1997 IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU), Santa Barbara, CA, USA.
18. Kanthak, S., and Ney, H. (2002, January 13–17). Context-dependent acoustic modeling using graphemes for large vocabulary speech recognition. Proceedings of the 2002 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Orlando, FL, USA.10.1109/ICASSP.2002.1005872 ↗
19. Killer, M., Stuker, S., and Schultz, T. (2003, January 1–4). Grapheme based speech recognition. Proceedings of the 8th European Conference on Speech Communication and Technology (EUROSPEECH), Geneva, Switzerland.10.21437/Eurospeech.2003-785 ↗
20. Dines, J., and Doss, M.M. (2007, January 28–30). A study of phoneme and grapheme based context-dependent ASR systems. Proceedings of the International Workshop on Machine Learning for Multimodal Interaction (MLMI), Brno, Czech Republic.
21. Magimai-Doss, M., Rasipuram, R., Aradilla, G., and Bourlard, H. (2011, January 27–31). Grapheme-based automatic speech recognition using KL-HMM. Proceedings of the 12th Annual Conference of the International Speech Communication Association (INTERSPEECH), Florence, Italy.10.21437/Interspeech.2011-171 ↗
22. Acoustic and lexical resource constrained ASR using language-independent acoustic model and language-dependent probabilistic lexical model10.1016/j.specom.2014.12.006 ↗
23. Young, S., Kershaw, D., Odell, J., Ollason, D., Valtchev, V., and Woodland, P. (2000). The HTK Book Version 3.0, Cambridge University Press.
24. Ljolje, A., and Riley, M. (1991, January 19–22). Automatic segmentation and labeling of speech. Proceedings of the 1991 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Washington, DC, USA.10.1109/ICASSP.1991.150379 ↗
25. Bang, J.-U., Choi, M.-Y., Kim, S.-H., and Kwon, O.-W. (2017, January 20–24). Improving speech recognizers by refining broadcast data with inaccurate subtitle timestamps. Proceedings of the Annual Conference of the International Speech Communication Association (INTERSPEECH), Stockholm, Sweden.10.21437/Interspeech.2017-650 ↗
26. Chung, Y.A., Wu, C.C., Shen, C.H., Lee, H.Y., and Lee, L.S. (2016, January 8–12). Audio Word2Vec: Unsupervised Learning of audio segment representations using sequence-to-sequence autoencoder. Proceedings of the 17th Annual Conference of the International Speech Communication Association (INTERSPEECH), San Francisco, CA, USA.10.21437/Interspeech.2016-82 ↗
27. Levin, K., Henry, K., Jansen, A., and Livescu, K. (2013, January 8–13). Fixed-dimensional acoustic embeddings of variable-length segments in low-resource settings. Proceedings of the 2013 IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU), Olomouc, Czech Republic.10.1109/ASRU.2013.6707765 ↗
28. Mitra, V., Vergyri, D., and Franco, H. (2016, January 8–12). Unsupervised Learning of acoustic units using autoencoders and Kohonen nets. Proceedings of the Annual Conference of the International Speech Communication Association (INTERSPEECH), San Francisco, CA, USA.10.21437/Interspeech.2016-1374 ↗
29. Povey, D., Ghoshal, A., Boulianne, G., Burget, L., Glembek, O., Goel, N., Hannemann, M., Motlicek, P., Qian, Y., and Schwarz, P. (2011, January 11–15). The Kaldi speech recognition toolkit. Proceedings of the 2011 IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU), Waikoloa, HI, USA.
30. Stolcke, A. (2002, January 13–14). SRILM—An extensible language modeling toolkit. Proceedings of the 7th International Conference on Spoken Language Processing (ICSLP), Warsaw, Poland.10.21437/ICSLP.2002-303 ↗
31. Automatic construction of a large-scale speech recognition database using multi-genre broadcast data with inaccurate subtitle timestamps10.1587/transinf.2019EDP7234 ↗
32. Lightly supervised and unsupervised acoustic model training10.1006/csla.2001.0186 ↗
33. Korean large vocabulary continuous speech recognition with morpheme-based recognition units10.1016/S0167-6393(02)00031-6 ↗
34. Computer codes for Korean sounds: K-SAMPA
35. A cluster separation measure10.1109/TPAMI.1979.4766909 ↗
36. Gillick, L., and Cox, S.J. (1989, January 23–26). Some statistical issues in the comparison of speech recognition algorithms. Proceedings of the 1989 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Glasgow, UK.