Intelligent Sign Language Recognition System for E-Learning Context
Source abstract
Abstract
In this research work, an efficient sign language recognition tool for e-learning has been proposed with a new type of feature set based on angle and lines. This feature set has the ability to increase the overall performance of machine learning algorithms in an efficient way. The hand gesture recognition based on these features has been implemented for usage in real-time. The feature set used hand landmarks, which were generated using media-pipe (MediaPipe) and open computer vision (openCV) on each frame of the incoming video. The overall algorithm has been tested on two well-known ASL-alphabet (American Sign Language) and ISL-HS (Irish Sign Language) sign language datasets. Different machine learning classifiers including random forest, decision tree, and naïve Bayesian have been used to classify hand gestures using this unique feature set and their respective results have been compared. Since the random forest classifier performed better, it has been selected as the base classifier for the proposed system. It showed 96.7% accuracy with ISL-HS and 93.7% accuracy with ASL-alphabet dataset using the extracted features.
Source: openalex_inverted_index · Quality: baseline
Research context
OpenAlex topics: Hand Gesture Recognition Systems · Hearing Impairment and Communication · Gait Recognition and Analysis
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
References (40)
Show source-provided references
1. On the design of and interaction with conversational agents: An organizing and assessing review of human-computer interaction research10.17705/1jais.00724 ↗
2. Deep learning-driven distributed communication systems for cluster online educational platform considering human–computer interaction10.1002/dac.5009 ↗
3. Design & implementation of real time autonomous car by using image processing & IoT
4. Crowd anomaly detection in public surveillance via spatio-temporal descriptors and zero-shot classifier
5. Semantic recognition of human-object interactions via gaussian-based elliptical modeling and pixel-level labeling10.1109/ACCESS.2021.3101716 ↗
6. Facial expression recognition based on deep learning10.1016/j.cmpb.2022.106621 ↗
7. Human gesture recognition using kinect camera
8. Hand gesture recognition using convolutional neural network
9. Gesture recognition using kinect in a virtual classroom environment
10. Hand gesture recognition for post-stroke rehabilitation using leap motion
11. M. J. Hussain, “Video for real time hand gesture recognition,” [Online]. Available: https://www.youtube.com/watch?v=xMXxg_eudvg. 2022.
12. Dynamic hand gesture recognition of sign language using geometric features learning10.1504/IJCVR.2022.119239 ↗
13. Real-time surface EMG pattern recognition for hand gestures based on an artificial neural network10.3390/s19143170 ↗
14. Hand gestures for elderly care using a microsoft kinect10.5101/nbe.v12i3.p197-204 ↗
15. Recognition and classification of dynamic hand gestures by a wearable data-glove10.1007/s42979-020-00396-5 ↗
16. Performance evaluation of convolutional neural network for hand gesture recognition using EMG10.3390/s20061642 ↗
17. Real-time hand gesture recognition based on electromyographic signals and artificial neural networks
18. Hand gesture recognition using machine learning and the Myo armband
19. Surface EMG hand gesture recognition system based on PCA and GRNN10.1007/s00521-019-04142-8 ↗
20. Hand gesture recognition based on active ultrasonic sensing of smartphone: A survey10.1109/ACCESS.2019.2933987 ↗
21. Wisture: Touch-less hand gesture classification in unmodified smartphones using Wi-Fi signals10.1109/JSEN.2018.2876448 ↗
22. A Smartphone-based application using machine learning for gesture recognition: Using feature extraction and template matching via Hu image moments to recognize gestures10.1109/MCE.2018.2868109 ↗
23. Visual pattern recognition by moment invariants10.1109/TIT.1962.1057692 ↗
24. Automatic static hand gesture recognition using ToF cameras
25. A hand gesture recognition method based on multi-feature fusion and template matching
26. Vision based hand gesture recognition using dynamic time warping for Indian sign language
27. Vision-based approach for American sign language recognition using edge orientation histogram
28. Hand gesture recognition based on auto-landmark localization and reweighted genetic algorithm for healthcare muscle activities10.3390/su13052961 ↗
29. American sign language alphabet recognition by extracting feature from hand pose estimation10.3390/s21175856 ↗
30. A. Costa and U. Edu, “ASLScribe: Real-time American sign language alphabet image classification using mediapipe hands and artificial neural networks,” final project, University of Georgia, 2019.
31. “OpenCV,” [Online]. Available: https://opencv.org/. 2021.
32. “Mediapipe,” [Online]. Available: https://google.github.io/mediapipe/solutions/hands. 2020.
33. Wearable sensors based exertion recognition using statistical features and random forest for physical healthcare monitoring
34. Akash, “ASL alphabet,” [Online]. Available: https://www.kaggle.com/grassknoted/asl-alphabet. 2018.
35. A dataset for Irish sign language recognition
36. Compact spatial pyramid pooling deep convolutional neural network based hand gestures decoder10.3390/app10217898 ↗
37. Squeezecapsnet–transfer learning-based ASL interpretation using squeezeNet with multi-lane capsules
38. Hand orientation redundancy filter applied to hand-shapes dataset
39. Sign language fingerspelling recognition using synthetic data
40. Irish sign language recognition using principal component analysis and convolutional neural networks