Spoken Language Identification System Using Convolutional Recurrent Neural Network
Source abstract
Abstract
Following recent advancements in deep learning and artificial intelligence, spoken language identification applications are playing an increasingly significant role in our day-to-day lives, especially in the domain of multi-lingual speech recognition. In this article, we propose a spoken language identification system that depends on the sequence of feature vectors. The proposed system uses a hybrid Convolutional Recurrent Neural Network (CRNN), which combines a Convolutional Neural Network (CNN) with a Recurrent Neural Network (RNN) network, for spoken language identification on seven languages, including Arabic, chosen from subsets of the Mozilla Common Voice (MCV) corpus. The proposed system exploits the advantages of both CNN and RNN architectures to construct the CRNN architecture. At the feature extraction stage, it compares the Gammatone Cepstral Coefficient (GTCC) feature and Mel Frequency Cepstral Coefficient (MFCC) feature, as well as a combination of both. Finally, the speech signals were represented as frames and used as the input for the CRNN architecture. After conducting experiments, the results of the proposed system indicate higher performance with combined GTCC and MFCC features compared to GTCC or MFCC features used individually. The average accuracy of the proposed system was 92.81% in the best experiment for spoken language identification. Furthermore, the system can learn language-specific patterns in various filter size representations of speech files.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Speech Recognition and Synthesis · Music and Audio Processing · Handwritten Text Recognition Techniques
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Computer Engineering Department, College of Computer and Information Sciences, King Saud University, Riyadh 11543, Saudi Arabia
Funding
Researchers Supporting Project
References (54)
Show source-provided references
1. Lounnas, K., Satori, H., Hamidi, M., Teffahi, H., Abbas, M., and Lichouri, M. (2020, January 16–19). CLIASR: A Combined Automatic Speech Recognition and Language Identification System. Proceedings of the 2020 1st International Conference on Innovative Research in Applied Science, Engineering and Technology (IRASET), Meknes, Morocco.10.1109/IRASET48871.2020.9092020 ↗
2. Bartz, C., Herold, T., Yang, H., and Meinel, C. (2017). Language Identification Using Deep Convolutional Recurrent Neural Networks. arXiv.10.1007/978-3-319-70136-3_93 ↗
3. Fromkin, V., Rodman, R., and Hyams, N.M. (2014). An Introduction to Language, Wadsworth/Cengage Learning. [10th ed.].
4. (2008). The World’s Major Languages, Routledge Handbooks Online.
5. Crystal, D. (2010). The Cambridge Encyclopedia of Language, Cambridge University Press. [3rd ed.].
6. Shaalan, K., Siddiqui, S., Alkhatib, M., and Monem, A.A. (2018). Challenges in Arabic Natural Language Processing. Systems Computational Linguistics, Speech and Image Processing for Arabic Language, World Scientific.10.1142/9789813229396_0003 ↗
7. Study on pharyngeal and uvular consonants in foreign accented Arabic for ASR10.1016/j.csl.2009.04.005 ↗
8. (2013). Spoken Language Recognition: From Fundamentals to Practice. IEEE J. Mag. IEEE Xplore, 101, 1136–1159. Available online: https://ieeexplore.ieee.org/document/6451097.10.1109/JPROC.2012.2237151 ↗
9. Multilinguality in speech and spoken language systems10.1109/5.880085 ↗
10. Language-independent and language-adaptive acoustic modeling for speech recognition10.1016/S0167-6393(00)00094-7 ↗
11. Kim, H., and Park, J.-S. (2020). Automatic Language Identification Using Speech Rhythm Features for Multi-Lingual Speech Recognition. Appl. Sci., 10.10.3390/app10072225 ↗
12. A unified system for multilingual speech recognition and language identification10.1016/j.specom.2020.12.008 ↗
13. Retrieval and browsing of spoken content10.1109/MSP.2008.917992 ↗
14. Walker, K., and Strassel, S. (2012). The RATS radio traffic collection system. Odyssey Speaker and Language Recognition Workshop, ISCA.
15. Knowledge Distillation-Based Representation Learning for Short-Utterance Spoken Language Identification10.1109/TASLP.2020.3023627 ↗
16. Recognition of Spoken Languages from Acoustic Speech Signals Using Fourier Parameters10.1007/s00034-019-01100-6 ↗
17. Multi-Level Cross-Lingual Transfer Learning With Language Shared and Specific Knowledge for Spoken Language Understanding10.1109/ACCESS.2020.2972925 ↗
18. Towards Relevance and Sequence Modeling in Language Recognition10.1109/TASLP.2020.2983580 ↗
19. Nofal, M., Abdel-Reheem, E., and El Henawy, H. (1999, January 22–24). Arabic/English automatic spoken language identification. Proceedings of the 1999 IEEE Pacific Rim Conference on Communications, Computers and Signal Processing (PACRIM 1999). Conference Proceedings (Cat. No.99CH36368), Victoria, BC, Canada.
20. Zazo, R., Lozano-Diez, A., Gonzalez-Dominguez, J., Toledano, D.T., and Gonzalez-Rodriguez, J. (2016). Language Identification in Short Utterances Using Long Short-Term Memory (LSTM) Recurrent Neural Networks. PLoS ONE, 11.10.1371/journal.pone.0146917 ↗
21. Draghici, A., Abeßer, J., and Lukashevich, H. A study on spoken language identification using deep neural networks. Proceedings of the 15th International Conference on Audio Mostly, New York, NY, USA, 15–17 September 2020, Available online: https://doi.org/10.1145/3411109.3411123.10.1145/3411109.3411123 ↗
22. Hybrid Feature Selection Method Based on Harmony Search and Naked Mole-Rat Algorithms for Spoken Language Identification From Audio Signals10.1109/ACCESS.2020.3028121 ↗
23. Performance of a language identification system using hybrid features and ANN learning algorithms10.1016/j.apacoust.2020.107815 ↗
24. FuzzyGCP: A deep learning architecture for automatic spoken language identification from speech signals10.1016/j.eswa.2020.114416 ↗
25. Shen, P., Lu, X., and Kawai, H. (2022). Transducer-based language embedding for spoken language identification. arXiv.10.21437/Interspeech.2022-11281 ↗
26. A Hybrid Meta-Heuristic Feature Selection Method for Identification of Indian Spoken Languages From Audio Signals10.1109/ACCESS.2020.3028241 ↗
27. Ma, Z., and Yu, H. (2020). Language Identification with Deep Bottleneck Features. arXiv, Available online: http://arxiv.org/abs/1809.08909.
28. Alshutayri, A., and Albarhamtoshy, H. (2011, January 12–14). Arabic Spoken Language Identification System (ASLIS): A Proposed System to Identifying Modern Standard Arabic (MSA) and Egyptian Dialect. Proceedings of the Informatics Engineering and Information Science Conference, Kuala Lumpur, Malaysia.10.1007/978-3-642-25453-6_33 ↗
29. LPC and MFCC Performance Evaluation with Artificial Neural Network for Spoken Language Identification
30. Pimentel, I. (2022, August 21). The Top 10 Languages in Higher Demand for Business. Available online: https://blog.acolad.com/the-top-10-languages-in-higher-demand-for-business.
31. (2022, August 21). “10 Foreign Languages in Demand across the Globe”. Education World, 19 November 2018. Available online: https://www.educationworld.in/foreign-languages-in-demand-across-the-globe/.
32. Sisodia, D.S., Nikhil, S., Kiran, G.S., and Sathvik, P. (2020, January 28–29). Ensemble Learners for Identification of Spoken Languages using Mel Frequency Cepstral Coefficients. Proceedings of the 2nd International Conference on Data, Engineering and Applications (IDEA), Bhopal, India.10.1109/IDEA49133.2020.9170720 ↗
33. Singh, G., Sharma, S., Kumar, V., Kaur, M., Baz, M., and Masud, M. (2021). Spoken Language Identification Using Deep Learning. Comput. Intell. Neurosci.10.1155/2021/5123671 ↗
34. Alashban, A.A., and Alotaibi, Y.A. (2021, January 26–28). Speaker Gender Classification in Mono-Language and Cross-Language Using BLSTM Network. Proceedings of the 2021 44th International Conference on Telecommunications and Signal Processing (TSP), Brno, Czech Republic.10.1109/TSP52935.2021.9522623 ↗
35. (2022, February 27). Mozilla Common Voice. Available online: https://commonvoice.mozilla.org/.
36. Ardila, R., Branson, M., Davis, K., Henretty, M., Kohler, M., Meyer, J., Morais, R., Saunders, L., Tyers, F.M., and Weber, G. (2020). Common Voice: A Massively-Multilingual Speech Corpus. arXiv, Available online: http://arxiv.org/abs/1912.06670.
37. (2022, March 30). Automatic Speech Recognition: A Deep Learning Approach—PDF Drive. Available online: http://www.pdfdrive.com/automatic-speech-recognition-a-deep-learning-approach-e177783075.html.
38. Alashban, A.A., and Alotaibi, Y.A. (2022, January 12–14). Language Effect on Speaker Gender Classification Using Deep Learning. Proceedings of the 2022 2nd International Conference on Artificial Intelligence and Signal Processing (AISP), Vijayawada, India.10.1109/AISP53593.2022.9760599 ↗
39. (2022, August 17). Detect Boundaries of Speech in Audio Signal—MATLAB detectSpeech—MathWorks Switzerland. Available online: https://ch.mathworks.com/help/audio/ref/detectspeech.html.
40. Journal, I. (2022, March 31). Extracting Mfcc and Gtcc Features for Emotion Recognition from Audio Speech Signals. Available online: https://www.academia.edu/8088548/EXTRACTING_MFCC_AND_GTCC_FEATURES_FOR_EMOTION_RECOGNITION_FROM_AUDIO_SPEECH_SIGNALS.
41. Kotsakis, R., Matsiola, M., Kalliris, G., and Dimoulas, C. (2020). Investigation of Spoken-Language Detection and Classification in Broadcasted Audio Content. Information, 11.10.3390/info11040211 ↗
42. Dua, S., Kumar, S.S., Albagory, Y., Ramalingam, R., Dumka, A., Singh, R., Rashid, M., Gehlot, A., Alshamrani, S.S., and AlGhamdi, A.S. (2022). Developing a Speech Recognition System for Recognizing Tonal Speech Signals Using a Convolutional Neural Network. Appl. Sci., 12.10.3390/app12126223 ↗
43. Nisar, S., Shahzad, I., Khan, M.A., and Tariq, M. (2017, January 4–6). Pashto spoken digits recognition using spectral and prosodic based feature extraction. Proceedings of the 2017 Ninth International Conference on Advanced Computational Intelligence (ICACI), Doha, Qatar.10.1109/ICACI.2017.7974488 ↗
44. Liu, G.K. (2018). Evaluating Gammatone Frequency Cepstral Coefficients with Neural Networks for Emotion Recognition from Speech. arXiv.
45. Liu, J.-M., You, M., Li, G.-Z., Wang, Z., Xu, X., Qiu, Z., Xie, W., An, C., and Chen, S. (2013, January 6–10). Cough signal recognition with Gammatone Cepstral Coefficients. Proceedings of the 2013 IEEE China Summit and International Conference on Signal and Information Processing, Beijing, China.10.1109/ChinaSIP.2013.6625319 ↗
46. Efficiency of deep neural networks for joint angle modeling in digital gait assessment10.1186/s13634-020-00715-1 ↗
47. (2022, August 18). Sequence Folding Layer—MATLAB—MathWorks Switzerland. Available online: https://ch.mathworks.com/help/deeplearning/ref/nnet.cnn.layer.sequencefoldinglayer.html#mw_e600a552-2ab0-48a8-b1d9-ae672b821805.
48. (2022, August 18). Sequence Unfolding Layer—MATLAB—MathWorks Switzerland. Available online: https://ch.mathworks.com/help/deeplearning/ref/nnet.cnn.layer.sequenceunfoldinglayer.html?searchHighlight=unfolding%20layer&s_tid=srchtitle_unfolding%20layer_1.
49. (2022, August 18). Flatten Layer—MATLAB—MathWorks Switzerland. Available online: https://ch.mathworks.com/help/deeplearning/ref/nnet.cnn.layer.flattenlayer.html?searchHighlight=flatten%20layer&s_tid=srchtitle_flatten%20layer_1.
50. (2022, March 30). Time Series Forecasting Using Hybrid CNN—RNN. Available online: https://ch.mathworks.com/matlabcentral/fileexchange/91360-time-series-forecasting-using-hybrid-cnn-rnn.
51. Digital Audio Forensics: Microphone and Environment Classification Using Deep Learning10.1109/ACCESS.2021.3073786 ↗
52. Explainable Artificial Intelligence (XAI): Concepts, taxonomies, opportunities and challenges toward responsible AI10.1016/j.inffus.2019.12.012 ↗
53. Saeed, W., and Omlin, C. (2021). Explainable AI (XAI): A Systematic Meta-Survey of Current Challenges and Future Opportunities. arXiv.
54. The role of typology and formal similarity in third language acquisition (German and Spanish)10.22210/strjez/50-1/1 ↗