journal article·2025Open access

VTAN: A Novel Video Transformer Attention-Based Network for Dynamic Sign Language Recognition

Ziyang Deng, Mengxue Liu, Longfei Li, Qing Han, Weidong Min

Computers, Materials & Continua· Volume 82· Issue 2

Abstract

Dynamic sign language recognition holds significant importance, particularly with the application of deep learning to address its complexity. However, existing methods face several challenges. Firstly, recognizing dynamic sig... | Find, read and cite all the research you need on Tech Science Press
Source: openalex_inverted_index · Quality: baseline

Research context

OpenAlex topics: Hand Gesture Recognition Systems · Hearing Impairment and Communication · Gait Recognition and Analysis

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

References (41)

Show source-provided references
1. Global-local enhancement network for NMF-aware sign language recognition
2. A review of hand gesture and sign language recognition techniques10.1007/s13042-017-0705-5 ↗
3. Sign language recognition using artificial intelligence10.1007/s10639-022-11391-z ↗
4. Conditional sentence generation and cross-modal reranking for sign language translation10.1109/TMM.2021.3087006 ↗
5. HAKA: HierArchical Knowledge Acquisition in a sign language tutor10.1016/j.eswa.2022.119365 ↗
6. MEN: Mutual enhancement networks for sign language recognition and education
7. W. -G. Liou, C. -Y. Hsieh, and W. -Y. Lin, “Trajectory-based sign language recognition using discriminant analysis in higher-dimensional feature space,” presented at the 2011 IEEE Int. Conf. Multimedia Expo, Barcelona, Spain, Jul. 11–15, 2011, pp. 1–4. doi: 10.1109/ICME.2011.6012048.10.1109/ICME.2011.6012048 ↗
8. Human skeleton feature optimizer and adaptive structure enhancement graph convolution network for action recognition10.1109/TCSVT.2022.3201186 ↗
9. Structure-aware multi-view image inpainting using dual consistency attention10.1016/j.inffus.2023.102174 ↗
10. Deep learning in neural networks: An overview10.1016/j.neunet.2014.09.003 ↗
11. Transformers in computational visual media: A survey10.1007/s41095-021-0247-3 ↗
12. American sign language recognition and training method with recurrent neural network10.1016/j.eswa.2020.114403 ↗
13. A. Kozlov, V. Andronov, and Y. Gritsenko, “Lightweight network architecture for real-time action recognition,” presented at the 35th Annu. ACM Symp. Appl. Comput., Brno, Czech Republic, Mar. 30–Apr. 3, 2020, pp. 2074–2080. doi: 10.1145/3341105.3373906.10.1145/3341105.3373906 ↗
14. K. Y. Lynn and F. Wong, “Recognition of dynamic hand gesture using hidden markov model,” presented at the 2022 Int. Conf. Green Energy, Comput. Sustain. Technol. (GECOST), Jakarta, Indonesia, Jul. 28–30, 2022, pp. 28–30. doi: 10.1109/GECOST55694.2022.10010517.10.1109/GECOST55694.2022.10010517 ↗
15. Trajectory-based recognition of dynamic Persian sign language using hidden Markov model10.1016/j.csl.2019.101053 ↗
16. Continuous sign language recognition using level building based on fast hidden Markov model10.1016/j.patrec.2016.03.030 ↗
17. A. Hamed, N. A. Belal, and K. M. Mahar, “Arabic sign language alphabet recognition based on HOG-PCA using microsoft kinect in complex backgrounds,” presented at the 2016 IEEE 6th Int. Conf. Adv. Comput. (IACC), Bhimtal, India, Feb. 27–28, 2016, pp. 27–28. doi: 10.1109/IACC.2016.90.10.1109/IACC.2016.90 ↗
18. Latent support vector machine modeling for sign language recognition with Kinect10.1145/2629481 ↗
19. K. O’shea and R. Nash, “An introduction to convolutional neural networks,” Nov. 2015, arXiv:1511.08458.
20. R. M. Schmidt, “Recurrent neural networks (RNNs): A gentle introduction and overview,” Dec. 2019, arXiv:1912.05911.
21. Deep learning-based sign language recognition system for static signs10.1007/s00521-019-04691-y ↗
22. Sign language recognition based on R(2+1)D with spatial-temporal-channel attention10.1109/THMS.2022.3144000 ↗
23. ASL-3DCNN: American sign language recognition technique using 3-D convolutional neural networks10.1007/s11042-021-10768-5 ↗
24. A wearable system for sign language recognition enabled by a convolutional neural network
25. Improving continuous sign language recognition with consistency constraints and signer removal
26. Intelligent real-time Arabic sign language classification using attention-based inception and BiLSTM
27. Weakly supervised learning with multi-stream CNN-LSTM-HMMs to discover sequential parallelism in sign language videos10.1109/TPAMI.2019.2911077 ↗
28. Spatial-temporal feature-based end-to-end Fourier network for 3D sign language recognition10.1016/j.eswa.2024.123258 ↗
29. Sign language recognition using modified deep learning network and hybrid optimization: A hybrid optimizer (HO) based optimized CNNSa-LSTM approach10.1038/s41598-024-76174-7 ↗
30. SIGNFORMER: Deepvision transformer for sign language recognition10.1109/ACCESS.2022.3231130 ↗
31. Arabic sign language letters recognition using Vision Transformer
32. AUTSL: A large scale multi-modal turkish sign language dataset and baseline methods10.1109/ACCESS.2020.3028072 ↗
33. Attention-based 3D-CNNs for large-vocabulary sign language recognition10.1109/TCSVT.2018.2870740 ↗
34. Dynamic sign language recognition based on video sequence with BLSTM-3D residual networks10.1109/ACCESS.2019.2904749 ↗
35. Lexicon-free fingerspelling recognition from video: Data, models, and signer adaptation
36. P. Kumar, H. Gauba, P. P. Roy, and D. P. Dogra, “A multimodal framework for sensor-based sign language recognition,” presented at the Neurocomput. Conf., Boston, MA, USA, Jun. 7–10, 2017, vol. 259, no. 11, pp. 7–10. doi: 10.1016/j.neucom.2016.08.132.10.1016/j.neucom.2016.08.132 ↗
37. H. Wang, X. Chai, and X. Chen, “Sparse observation (SO) alignment for sign language recognition,” presented at the Neurocomput. Conf., New York, NY, USA, Jan. 15–18, 2016, vol. 175, no. 1, pp. 674–685. doi: 10.1016/j.neucom.2015.10.112.10.1016/j.neucom.2015.10.112 ↗
38. M. De Coster, M. Van Herreweghe, and J. Dambre, “Isolated sign recognition from rgb video using pose flow and self-attention,” presented at the IEEE/CVF Conf. Comput. Vis. Pattern Recognit., Nashville, TN, USA, Jun. 19–25, 2021, pp. 3441–3450. doi: 10.1109/CVPRW53098.2021.00383.10.1109/CVPRW53098.2021.00383 ↗
39. D. Ryumin, D. Ivanko, and A. Axyonov, “Cross-language transfer learning using visual information for automatic sign gesture recognition,” presented at the Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci., Nice, France, May 22–28, 2023, vol. 48, pp. 209–216. doi: 10.5194/isprs-archives-XLVIII-1-2023-209-2023.10.5194/isprs-archives-XLVIII-2-W3-2023-209-2023 ↗
40. Using motion history images with 3D convolutional networks in isolated sign language recognition10.1109/ACCESS.2022.3151362 ↗
41. Audio-visual speech and gesture recognition by sensors of mobile devices10.3390/s23042284 ↗