A Four-Stage Mahalanobis-Distance-Based Method for Hand Posture Recognition
Source abstract
Abstract
Automatic recognition of hand postures is an important research topic with many applications, e.g., communication support for deaf people. In this paper, we present a novel four-stage, Mahalanobis-distance-based method for hand posture recognition using skeletal data. The proposed method is based on a two-stage classification algorithm with two additional stages related to joint preprocessing (normalization) and a rule-based system, specific to hand shapes that the algorithm is meant to classify. The method achieves superior effectiveness on two benchmark datasets, the first of which was created by us for the purpose of this work, while the second is a well-known and publicly available dataset. The method’s recognition rate measured by leave-one-subject-out cross-validation tests is 94.69% on the first dataset and 97.44% on the second. Experiments, including comparison with other state-of-the-art methods and ablation studies related to classification accuracy and time, confirm the effectiveness of our approach.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Hand Gesture Recognition Systems · Hearing Impairment and Communication · Human Pose and Action Recognition
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Department of Computer and Control Engineering, Faculty of Electrical and Computer Engineering, Rzeszów University of Technology, W. Pola 2, 35-959 Rzeszów, Poland
Funding
Minister of Education and Science of the Republic of Poland
References (45)
Show source-provided references
1. A review of hand gesture and sign language recognition techniques10.1007/s13042-017-0705-5 ↗
2. Wang, Y., Wang, D., Fu, Y., Yao, D., Xie, L., and Zhou, M. (2022). Multi-Hand Gesture Recognition Using Automotive FMCW Radar Sensor. Remote Sens., 14.10.3390/rs14102374 ↗
3. Khalaf, A.S., Alharthi, S.A., Dolgov, I., and Toups, Z.O. (2019, January 10–13). A Comparative Study of Hand Gesture Recognition Devices in the Context of Game Design. Proceedings of the 2019 ACM International Conference on Interactive Surfaces and Spaces, New York, NY, USA.10.1145/3343055.3360758 ↗
4. Hand Gestures Identification for Fine-Grained Human Activity Recognition in Smart Homes10.1016/j.procs.2022.03.007 ↗
5. OpenPose: Realtime Multi-Person 2D Pose Estimation using10.1109/TPAMI.2019.2929257 ↗
6. Lugaresi, C., Tang, J., Nash, H., McClanahan, C., Uboweja, E., Hays, M., Zhang, F., Chang, C.L., Yong, M., and Lee, J. (2019, January 17). Mediapipe: A framework for perceiving and processing reality. Proceedings of the Third Workshop on Computer Vision for AR/VR at IEEE Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA.
7. Enhanced computer vision with microsoft kinect sensor: A review10.1109/TCYB.2013.2265378 ↗
8. Guzsvinecz, T., Szucs, V., and Sik-Lanyi, C. (2019). Suitability of the Kinect sensor and Leap Motion controller—A literature review. Sensors, 19.10.3390/s19051072 ↗
9. Hand Gesture Recognition for Characters Understanding Using Convex Hull Landmarks and Geometric Features10.1109/ACCESS.2023.3300712 ↗
10. Japanese fingerspelling identification by using MediaPipe10.1587/nolta.13.288 ↗
11. Shin, J., Matsuoka, A., Hasan, M.A.M., and Srizon, A.Y. (2021). American sign language alphabet recognition by extracting feature from hand pose estimation. Sensors, 21.10.3390/s21175856 ↗
12. Sign language dactyl recognition based on machine learning algorithms
13. Hagimoto, S., Nitta, T., Okada, R., and Nakanishi, T. (2022, January 12–14). A Dynamic Finger Character Recognition Method Using Landmark Behavior Rule Base. Proceedings of the 2022 13th International Congress on Advanced Applied Informatics Winter (IIAI-AAI-Winter), Phuket, Thailand.10.1109/IIAI-AAI-Winter58034.2022.00045 ↗
14. Sanalohit, J., and Katanyukul, T. (2022). Thai Finger Spelling Recognition: Investigating MediaPipe Hands Potentials. arXiv.
15. John, J., and Sherif, B.V. (2022). Machine Learning and Autonomous Systems: Proceedings of ICMLAS 2021, Springer.
16. Sign language recognition using deep learning
17. Rodríguez-Moreno, I., Martínez-Otzeta, J.M., Goienetxea, I., and Sierra, B. (2022). Sign language recognition by means of common spatial patterns: An analysis. PLoS ONE, 17.10.1371/journal.pone.0276941 ↗
18. Hisham, E., and Saleh, S.N. (2022, January 27–29). ESMAANI: A Static and Dynamic Arabic Sign Language Recognition System Based on Machine and Deep Learning Models. Proceedings of the 2022 5th International Conference on Communications, Signal Processing, and their Applications (ICCSPA), Cairo, Egypt.10.1109/ICCSPA55860.2022.10019112 ↗
19. Fowley, F., and Ventresque, A. (2021, January 29–31). Sign Language Fingerspelling Recognition using Synthetic Data. Proceedings of the AICS, Beijing, China.
20. Papadimitriou, K., Potamianos, G., Sapountzaki, G., Goulas, T., Efthimiou, E., Fotinea, S.E., and Maragos, P. (2023). Greek sign language recognition for an education platform. Univers. Access Inf. Soc., 1–18.10.1007/s10209-023-01017-7 ↗
21. Papadimitriou, K., and Potamianos, G. (2023, January 4–10). Sign Language Recognition via Deformable 3D Convolutions and Modulated Graph Convolutional Networks. Proceedings of the ICASSP 2023—2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Rhodes Island, Greece.10.1109/ICASSP49357.2023.10096714 ↗
22. Kushwaha, R., Kaur, G., and Kumar, M. (2023, January 26–28). Hand Gesture Based Sign Language Recognition Using Deep Learning. Proceedings of the 2023 Third International Conference on Secure Cyber Computing and Communication (ICSCCC), Jalandhar, India.10.1109/ICSCCC58608.2023.10176912 ↗
23. Al-Hammadi, M., Bencherif, M.A., Alsulaiman, M., Muhammad, G., Mekhtiche, M.A., Abdul, W., Alohali, Y.A., Alrayes, T.S., Mathkour, H., and Faisal, M. (2022). Spatial attention-based 3d graph convolutional neural network for sign language recognition. Sensors, 22.10.3390/s22124558 ↗
24. Alyami, S., Luqman, H., and Hammoudeh, M. (2023). Isolated Arabic Sign Language Recognition Using A Transformer-based Model and Landmark Keypoints. ACM Trans. Asian Low-Resour. Lang. Inf. Process.10.1145/3584984 ↗
25. Addressing Resource Scarcity across Sign Languages with Multilingual Pretraining and Unified-Vocabulary Datasets
26. Real-time recognition of American sign language using long-short term memory neural network and hand detection
27. Sheela, K.A., Kumar, C.A., Sandhya, J., and Ravindra, G. (2022, January 18–22). Indian Sign Language Translator. Proceedings of the 2022 IEEE International Symposium on Smart Electronic Systems (iSES), Warangal, India.
28. Kashyap, N. (2022). Perspectives in Communication, Embedded-systems and Signal-processing-PiCES, WorldServe Online.
29. Xia, K., Lu, W., Fan, H., and Zhao, Q. (2022). A Sign Language Recognition System Applied to Deaf-Mute Medical Consultation. Sensors, 22.10.3390/s22239107 ↗
30. Marsico, M.D., di Baja, G.S., and Fred, A.L.N. (2023, January 22–24). Video-Based Sign Language Digit Recognition for the Thai Language: A New Dataset and Method Comparisons. Proceedings of the Proceedings of the 12th International Conference on Pattern Recognition Applications and Methods, ICPRAM 2023, Lisbon, Portugal.
31. Guo, Y., Zhao, J., Ding, B., Tan, C., Ling, W., Tan, Z., Miyaki, J., Du, H., and Lu, S. (2023, January 2–6). Sign-to-911: Emergency Call Service for Sign Language Users with Assistive AR Glasses. Proceedings of the 29th Annual International Conference on Mobile Computing and Networking, New York, NY, USA.10.1145/3570361.3613260 ↗
32. Tobias, J.L., and Di Mitri, D. (2023, January 4–8). Using Accessible Motion Capture in Educational Games for Sign language Learning. Proceedings of the European Conference on Technology Enhanced Learning, Aveiro, Portugal.10.1007/978-3-031-42682-7_74 ↗
33. Ivanska, L., and Korotyeyeva, T. (2022, January 10–12). Mobile real-time gesture detection application for sign language learning. Proceedings of the 2022 IEEE 17th International Conference on Computer Sciences and Information Technologies (CSIT), Lviv, Ukraine.10.1109/CSIT56902.2022.10000440 ↗
34. Kapuscinski, T. (July, January 29). Handshape Recognition in an Educational Game for Finger Alphabet Practicing. Proceedings of the International Conference on Intelligent Tutoring Systems, Bucharest, Romania.
35. Chandwani, L., Khilari, J., Gurjar, K., Maragale, P., Sonare, A., Kakade, S., Bhatt, A., and Kulkarni, R. (2023). Gesture based Sign Language Recognition system using Mediapipe. Res. Sq.10.21203/rs.3.rs-3106646/v1 ↗
36. Cucurull, X., and Garrell, A. (2023). Continual Learning of Hand Gestures for Human-Robot Interaction. arXiv.
37. A review on computational methods based automated sign language recognition system for hearing and speech impaired community10.1002/cpe.7653 ↗
38. On the generalized distance in statistics
39. The clinical usefulness of the fingers-to-palm ratio in different hand microcirculatory abnormalities10.1097/00006231-200007000-00010 ↗
40. (2023, July 12). SPAS Dataset and Matlab scripts for the Recognition Method Presented in This Paper. Available online: http://vision.kia.prz.edu.pl.
41. A New 2D Static Hand Gesture Colour Image Dataset for ASL Gestures
42. A method for static hand gesture recognition based on non-negative matrix factorization and compressive sensing
43. Sahoo, J.P., Ari, S., and Patra, S.K. (2019, January 16–18). Hand Gesture Recognition Using PCA Based Deep CNN Reduced Features and SVM Classifier. Proceedings of the 2019 IEEE International Symposium on Smart Electronic Systems (iSES), Rourkela, India.10.1109/iSES47678.2019.00056 ↗
44. Real-time hand posture recognition using hand geometric features and Fisher Vector10.1016/j.image.2019.115729 ↗
45. DeReFNet: Dual-stream Dense Residual Fusion Network for static hand gesture recognition10.1016/j.displa.2023.102388 ↗