Creating a Parallel Corpus for the Kazakh Sign Language and Learning
Source abstract
Abstract
Kazakh Sign Language (KSL) is a crucial communication tool for individuals with hearing and speech impairments. Deep learning, particularly Transformer models, offers a promising approach to improving accessibility in education and communication. This study analyzes the syntactic structure of KSL, identifying its unique grammatical features and deviations from spoken Kazakh. A custom parser was developed to convert Kazakh text into KSL glosses, enabling the creation of a large-scale parallel corpus. Using this resource, a Transformer-based machine translation model was trained, achieving high translation accuracy and demonstrating the feasibility of this approach for enhancing communication accessibility. The research highlights key challenges in sign language processing, such as the limited availability of annotated data. Future work directions include the integration of video data and the adoption of more comprehensive evaluation metrics. This paper presents a methodology for constructing a parallel corpus through gloss annotations, contributing to advancements in sign language translation technology.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Hand Gesture Recognition Systems · Hearing Impairment and Communication · Interpreting and Communication in Healthcare
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Laboratory of Computer Engineering of Intelligent Systems, Institute of Information and Computational Technologies, Almaty 050010, Kazakhstan
Global Education and Training, University of Illinois Urbana-Champaign, Champaign, IL 61820, USA
Department of Information Systems, L.N. Gumilyov Eurasian National University, Astana 010008, Kazakhstan
Department of Software Engineering, Satbayev University, Almaty 050010, Kazakhstan
Faculty of Information Technology, Al-Farabi Kazakh National University, Almaty 050038, Kazakhstan
Department of Communication Technology and Network, Faculty of Computer Science and Information Technology, Universiti Putra Malaysia, Serdang 43400, Malaysia
Funding
Committee of Science of the Ministry of Science and Higher Education of the Republic of Kazakhstan
References (29)
Show source-provided references
1. SignFi: Sign Language Recognition Using WiFi10.1145/3191755 ↗
2. Bragg, D. (2019, January 28–30). Sign Language Recognition, Generation, and Translation: An Interdisciplinary Perspective. Proceedings of the ACM SIGACCESS Conference on Computers and Accessibility, Pittsburgh, PA, USA.10.1145/3308561.3353774 ↗
3. Sign language recognition: A deep survey10.1016/j.eswa.2020.113794 ↗
4. Sowmya. Speech to Sign Conversion Using Natural Language Processing
5. Sparsha, U., Priyanka, M., Mukthashree, S., and Kiran, K.N. (2024, January 26–27). System for Sign Language to Speech Conversion. Proceedings of the 2024 International Conference on Intelligent and Innovative Technologies in Computing, Electrical and Electronics (IITCEE), Bengaluru, India.
6. Machine Translation from Signed to Spoken Languages: State of the Art and Challenges10.1007/s10209-023-00992-1 ↗
7. Gesture Recognition of the Kazakh Alphabet Based on Machine and Deep Learning Models10.1016/j.procs.2024.08.064 ↗
8. Application of Neural Networks Ensemble Method for the Kazakh Sign Language Recognition10.11591/eei.v13i5.7803 ↗
9. Yerimbetova, A., Sakenov, B., Berzhanova, U., Mukazhanov, N., Daiyrbayeva, E., and Othman, M. (2024, January 4–6). Development of a Model of Kazakh Sign Language Recognition Based on Deep Learning Method. Proceedings of the 2024 9th International Conference on Computer Science and Engineering (UBMK), Antalya, Turkey.10.1109/UBMK63289.2024.10773578 ↗
10. Yerimbetova, A., Kaidina, D., Sakenov, B., Daiyrbayeva, E., Turdalyuly, M., and Berzhanova, U. (2024, January 4–6). Recognising Kazakh Sign Language with Mediapipe. Proceedings of the 2024 9th International Conference on Computer Science and Engineering (UBMK), Antalya, Turkey.10.1109/UBMK63289.2024.10773406 ↗
11. Othman, A., and Jemni, M. (2012, January 21–22). English-ASL Gloss Parallel Corpus 2012: ASLG-PC12. Proceedings of the sign-lang@LREC 2012 Workshop, Istanbul, Turkey.
12. Forster, J. (2020). Automatic Sign Language Recognition: From Video Corpora to Gloss Sentences. [Master’s Thesis, RWTH Aachen University].
13. Hanke, T., Schulder, M., Konrad, R., and Jahn, E. (2020, January 11–16). Extending the Public DGS Corpus in Size and Depth. Proceedings of the Sign-lang@LREC 2020 Workshop, Marseille, France.
14. Kim, Y., Kwak, M., Lee, D., Kim, Y., and Baek, H. (2022). Keypoint-based sign language translation without glosses. arXiv.10.3390/s23063231 ↗
15. Mahmoud, H.E., Elattar, M.A., and Darweesh, M.S. (2023, January 18–20). Joint Sign Language Recognition and Translation Using Keypoint Estimation. Proceedings of the 2023 11th International Japan-Africa Conference on Electronics, Communications, and Computations (JAC-ECC), Alexandria, Egypt.10.1109/JAC-ECC61002.2023.10479621 ↗
16. Kim, Y., and Baek, H. (2023). Preprocessing for keypoint-based sign language translation without glosses. Sensors, 23.10.3390/s23063231 ↗
17. English–Georgian parallel corpus and its application in Georgian lexicography10.5788/32-2-1701 ↗
18. Jiang, Z., Göhring, A., Moryossef, A., Sennrich, R., and Ebling, S. (2024, January 20–25). SwissSLi: The Multi-parallel Sign Language Corpus for Switzerland. Proceedings of the 2024 Joint International Conference on Computational Linguistics and Language Resources and Evaluation (LREC-COLING 2024), Torino, Italy.
19. Sign Language Structure10.1146/annurev.an.09.100180.002053 ↗
20. Prillwitz, S., and Vollhaber, T. (1990). Hamburg Notation System for Sign Language Research: Development of a Sign Writing with Computer Application. Current Trends in European Sign Language Research, Signum.
21. Sutton, V., and Gleaves, R. (1995). SignWriter—The World’s First Sign Language Processor, Deaf Action Committee for SignWriting.
22. Word order in Russian sign language10.1353/sls.2012.0001 ↗
23. Zaitseva, G.L. (2006). Gesture and Word: Scientific and Methodological Articles, Moscow Press.
24. Qi, P., Zhang, Y., Zhang, Y., Bolton, J., and Manning, C.D. (2020). Stanza: A Python natural language processing toolkit for many human languages. arXiv.10.18653/v1/2020.acl-demos.14 ↗
25. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., and Polosukhin, I. (2017, January 4–9). Attention Is All You Need. Proceedings of the 31st Conference on Neural Information Processing Systems (NeurIPS 2017), Long Beach, CA, USA.
26. Mars, M. (2022). From word embeddings to pre-trained language models: A state-of-the-art walkthrough. Appl. Sci., 12.10.3390/app12178805 ↗
27. Velay, M., and Daniel, F. (2018). Seq2seq and multi-task learning for joint intent and content extraction for domain-specific interpreters. arXiv.
28. Gómez, S.E., Fabra, S.U.P., McGill, E., and Saggion, H. (2021, January 18). Syntax-Aware Transformers for Neural Machine Translation: The Case of Text to Sign Gloss Translation. Proceedings of the 14th Workshop on Building and Using Comparable Corpora (BUCC 2021), Online.10.26615/978-954-452-076-2_004 ↗
29. Saunders, B., Camgoz, N.C., and Bowden, R. (2020). Progressive transformers for end-to-end sign language production. Computer Vision–ECCV 2020 Proceedings of the 16th European Conference, Glasgow, UK, 23–28 August 2020, Springer International Publishing.10.1007/978-3-030-58621-8_40 ↗