Voice Activation for Low-Resource Languages
Source abstract
Abstract
Voice activation systems are used to find a pre-defined word or phrase in the audio stream. Industry solutions, such as “OK, Google” for Android devices, are trained with millions of samples. In this work, we propose and investigate several ways to train a voice activation system when the in-domain data set is small. We compare self-training exemplar pre-training, fine-tuning a model pre-trained on another domain, joint training on both an out-of-domain high-resource and a target low-resource data set, and unsupervised pre-training. In our experiments, the unsupervised pre-training and the joint-training with a high-resource data set from another domain significantly outperform a strong baseline of fine-tuning a model trained on another data set. We obtain 7–25% relative improvement depending on the model architecture. Additionally, we improve the best test accuracy on the Lithuanian data set from 90.77% to 93.85%.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Speech Recognition and Synthesis · Topic Modeling · Natural Language Processing Techniques
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Department of Information Technologies, Vilnius Gediminas Technical University, Saulėtekio al. 11, LT-10223 Vilnius, Lithuania
References (48)
Show source-provided references
1. Voice Activation Systems for Embedded Devices: Systematic Literature Review10.15388/20-INFOR398 ↗
2. Alon, U., Pundak, G., and Sainath, T.N. (2019, January 12–17). Contextual Speech Recognition with Difficult Negative Training Examples. Proceedings of the ICASSP 2019—2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Brighton, UK.10.1109/ICASSP.2019.8682738 ↗
3. Meng, H., Xu, B., and Zheng, T.F. (2020, January 25–29). Streaming Keyword Spotting on Mobile Devices. Proceedings of the Interspeech 2020, 21st Annual Conference of the International SpeechCommunication Association, Virtual Event, Shanghai, China.
4. Kavya, H.P., and Karjigi, V. (2014, January 10–12). Sensitive keyword spotting for crime analysis. Proceedings of the 2014 IEEE National Conference on Communication, Signal Processing and Networking (NCCSN), Palakkad, India.10.1109/NCCSN.2014.7001149 ↗
5. A voice command detection system for aerospace applications10.1007/s10772-017-9467-4 ↗
6. Gruenstein, A., Alvarez, R., Thornton, C., and Ghodrat, M. (2017). A Cascade Architecture for Keyword Spotting on Mobile Devices. arXiv.
7. Meng, H., Xu, B., and Zheng, T.F. (2020, January 25–29). Metadata-Aware End-to-End Keyword Spotting. Proceedings of the Interspeech 2020, 21st Annual Conference of the International Speech Communication Association, Virtual Event, Shanghai, China.
8. Meng, H., Xu, B., and Zheng, T.F. (2020, January 25–29). Domain Aware Training for Far-Field Small-Footprint Keyword Spotting. Proceedings of the Interspeech 2020, 21st Annual Conference of the International Speech Communication Association, Virtual Event, Shanghai, China.
9. Meng, H., Xu, B., and Zheng, T.F. (2020, January 25–29). Re-Weighted Interval Loss for Handling Data Imbalance Problem of End-to-End Keyword Spotting. Proceedings of the Interspeech 2020, 21st Annual Conference of the International Speech Communication Association, Virtual Event, Shanghai, China.
10. Meng, H., Xu, B., and Zheng, T.F. (2020, January 25–29). Deep Template Matching for Small-Footprint and Configurable Keyword Spotting. Proceedings of the Interspeech 2020, 21st Annual Conference of the International Speech Communication Association, Virtual Event, Shanghai, China.
11. Meng, H., Xu, B., and Zheng, T.F. (2020, January 25–29). State Sequence Pooling Training of Acoustic Models for Keyword Spotting. Proceedings of the Interspeech 2020, 21st Annual Conference of the International Speech Communication Association, Virtual Event, Shanghai, China.
12. Kumatani, K., Panchapagesan, S., Wu, M., Kim, M., Strom, N., Tiwari, G., and Mandal, A. (2017, January 16–20). Direct modeling of raw audio with DNNS for wake word detection. Proceedings of the 2017 IEEE Automatic Speech Recognition and Understanding Workshop, ASRU 2017, Okinawa, Japan.10.1109/ASRU.2017.8268943 ↗
13. Yegnanarayana, B. (2018, January 2–6). Efficient Keyword Spotting Using Time Delay Neural Networks. Proceedings of the Interspeech 2018, 19th Annual Conference of the International Speech Communication Association, Hyderabad, India.
14. Manor, E., and Greenberg, S. (2017, January 14–17). Voice trigger system using fuzzy logic. Proceedings of the 2017 International Conference on Circuits, System and Simulation (ICCSS), London, UK.10.1109/CIRSYSSIM.2017.8023193 ↗
15. Tang, R., and Lin, J. (2018, January 15–20). Deep Residual Learning for Small-Footprint Keyword Spotting. Proceedings of the 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Calgary, AB, Canada.10.1109/ICASSP.2018.8462688 ↗
16. Chen, G., Parada, C., and Heigold, G. (2014, January 4–9). Small-footprint keyword spotting using deep neural networks. Proceedings of the 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Florence, Italy.10.1109/ICASSP.2014.6854370 ↗
17. Rohlicek, J.R., Russell, W., Roukos, S., and Gish, H. (1989, January 23–26). Continuous hidden Markov modeling for speaker-independent word spotting. Proceedings of the International Conference on Acoustics, Speech, and Signal Processing, Glasgow, UK.
18. Zeppenfeld, T., and Waibel, A.H. (1992, January 23–26). A hybrid neural network, dynamic programming word spotter. Proceedings of the ICASSP-92: 1992 IEEE International Conference on Acoustics, Speech, and Signal Processing, San Francisco, CA, USA.10.1109/ICASSP.1992.226116 ↗
19. Morgan, D.P., Scofield, C.L., Lorenzo, T.M., Real, E.C., and Loconto, D.P. (1990, January 3–6). A keyword spotter which incorporates neural networks for secondary processing. Proceedings of the International Conference on Acoustics, Speech, and Signal Processing, Albuquerque, NM, USA.
20. Morgan, D.P., Scofield, C.L., and Adcock, J.E. (1991, January 14–17). Multiple neural network topologies applied to keyword spotting. Proceedings of the ICASSP 91: 1991 International Conference on Acoustics, Speech, and Signal Processing, Toronto, ON, Canada.10.1109/ICASSP.1991.150339 ↗
21. Naylor, J.A., Huang, W.Y., Nguyen, M., and Li, K.P. (1992, January 26–28). The application of neural networks to wordspotting. Proceedings of the 1992 Conference Record of the Twenty-Sixth Asilomar Conference on Signals, Systems Computers, Pacific Grove, CA, USA.
22. Wu, M., Panchapagesan, S., Sun, M., Gu, J., Thomas, R., Vitaladevuni, S.N.P., Hoffmeister, B., and Mandal, A. (2018, January 15–20). Monophone-Based Background Modeling for Two-Stage On-Device Wake Word Detection. Proceedings of the 2018 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2018, Calgary, AB, Canada.10.1109/ICASSP.2018.8462227 ↗
23. Meng, H., Xu, B., and Zheng, T.F. (2020, January 25–29). Accurate Detection of Wake Word Start and End Using a CNN. Proceedings of the Interspeech 2020, 21st Annual Conference of the International Speech Communication Association, Virtual Event, Shanghai, China.
24. Kolesau, A., and Šešok, D. (2021, May 04). Lithuanian Speech Commands Dataset. Available online: https://github.com/kolesov93/lt_speech_commands.
25. Menon, R., Kamper, H., van der Westhuizen, E., Quinn, J., and Niesler, T. (2019). Feature Exploration for Almost Zero-Resource ASR-Free Keyword Spotting Using a Multilingual Bottleneck Extractor and Correspondence Autoencoders. arXiv.10.21437/Interspeech.2019-1665 ↗
26. Knill, K., Gales, M., Ragni, A., and Rath, S. (2014, January 14–18). Language independent and unsupervised acoustic models for speech recognition and keyword spotting. Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH, Singapore.10.21437/Interspeech.2014-4 ↗
27. Wang, H., Ragni, A., Gales, M.J.F., Knill, K.M., Woodland, P.C., and Zhang, C. (2015, January 6–10). Joint decoding of tandem and hybrid systems for improved keyword spotting on low resource languages. Proceedings of the INTERSPEECH 2015, 16th Annual Conference of the International Speech Communication Association, Dresden, Germany.10.21437/Interspeech.2015-726 ↗
28. Cross-language phoneme mapping for phonetic search keyword spotting in continuous speech of under-resourced languages10.5430/air.v4n2p72 ↗
29. Lin, J., Kilgour, K., Roblek, D., and Sharifi, M. (2020, January 4–8). Training Keyword Spotters with Limited and Synthesized Speech Data. Proceedings of the ICASSP 2020—2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Barcelona, Spain.10.1109/ICASSP40776.2020.9053193 ↗
30. wav2vec: Unsupervised Pre-Training for Speech Recognition10.21437/Interspeech.2019-1873 ↗
31. Kolesau, A., and Šešok, D. (2020). Unsupervised Pre-Training for Voice Activation. Appl. Sci., 10.10.3390/app10238643 ↗
32. Wav2KWS: Transfer Learning From Speech Representations for Keyword Spotting10.1109/ACCESS.2021.3078715 ↗
33. Why Does Unsupervised Pre-training Help Deep Learning?
34. Deep Neural Networks for Acoustic Modeling in Speech Recognition10.1109/MSP.2012.2205597 ↗
35. Dosovitskiy, A., Springenberg, J., Riedmiller, M., and Brox, T. (2014). Discriminative Unsupervised Feature Learning with Exemplar Convolutional Neural Networks. IEEE Trans. Pattern Anal. Mach. Intell., 1.
36. Triguero, I., García, S., and Herrera, F. (2015). Self-labeled techniques for semi-supervised learning: Taxonomy, software and empirical study. Knowl. Inf. Syst., 42.10.1007/s10115-013-0706-y ↗
37. Kahn, J., Lee, A., and Hannun, A. (2020, January 4–8). Self-Training for End-to-End Speech Recognition. Proceedings of the ICASSP 2020—2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Barcelona, Spain.10.1109/ICASSP40776.2020.9054295 ↗
38. Warden, P. (2018). Speech Commands: A Dataset for Limited-Vocabulary Speech Recognition. arXiv.
39. Li, H., Chaudhari, P., Yang, H., Lam, M., Ravichandran, A., Bhotika, R., and Soatto, S. (2020, January 26–30). Rethinking the Hyperparameters for Fine-tuning. Proceedings of the International Conference on Learning Representations, Addis Ababa, Ethiopia.
40. Ge, W., and Yu, Y. (2017, January 21–26). Borrowing Treasures From the Wealthy: Deep Transfer Learning Through Selective Joint Fine-Tuning. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.10.1109/CVPR.2017.9 ↗
41. DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs10.1109/TPAMI.2017.2699184 ↗
42. Cortes, C., Lawrence, N., Lee, D., Sugiyama, M., and Garnett, R. (2015). Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks. Advances in Neural Information Processing Systems, Curran Associates, Inc.
43. Bansal, S., Kamper, H., Livescu, K., Lopez, A., and Goldwater, S. (2021, January 6–11). Pre-training on high-resource speech recognition improves low-resource speech-to-text translation. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), Minneapolis, MN, USA.10.18653/v1/N19-1006 ↗
44. SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition10.21437/Interspeech.2019-2680 ↗
45. Meng, H., Xu, B., and Zheng, T.F. (2020, January 25–29). Iterative Pseudo-Labeling for Speech Recognition. Proceedings of the Interspeech 2020, 21st Annual Conference of the International Speech Communication Association, Virtual Event, Shanghai, China.
46. Meng, H., Xu, B., and Zheng, T.F. (2020, January 25–29). Small-Footprint Keyword Spotting with Multi-Scale Temporal Convolution. Proceedings of the Interspeech 2020, 21st Annual Conference of the International Speech Communication Association, Virtual Event, Shanghai, China.
47. Meng, H., Xu, B., and Zheng, T.F. (2020, January 25–29). Neural Architecture Search for Keyword Spotting. Proceedings of the Interspeech 2020, 21st Annual Conference of the International Speech Communication Association, Virtual Event, Shanghai, China.
48. Scikit-learn: Machine Learning in Python