Gesture Classification Using a Smartwatch: Focusing on Unseen Non-Target Gestures
Source abstract
Abstract
Hand gestures serve as a fundamental means of communication, and extensive research has been conducted to develop automated recognition systems. These systems are expected to improve human/computer interaction, particularly in environments where verbal communication is limited. A key challenge in these systems is the classification of non-target actions, as everyday movements are often not included in the training set, but resembling target gestures can lead to misclassification. Unlike previous studies that primarily focused on target action recognition, this study explicitly addresses the unseen non-target classification problem through an experiment to distinguish target and non-target activities based on movement characteristics. This study examines the ability of deep learning models to generalize classification criteria beyond predefined training sets. The proposed method was validated with arm movement data from 20 target group participants and 11 non-target group participants, achieving an average F1-score of 84.23%, with a non-target classification score of 73.23%. Furthermore, we confirmed that data augmentation and incorporating a loss factor significantly improved the recognition of unseen non-target gestures. The results suggest that improving classification performance on untrained, non-target movements will enhance the applicability of gesture recognition systems in real-world environments. This is particularly relevant for wearable devices, assistive technologies, and human/computer interaction systems.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Hand Gesture Recognition Systems · Robotics and Automated Systems · Hearing Impairment and Communication
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Department of Computer Engineering, Pukyong National University, Busan 48513, Republic of Korea
Funding
National Research Foundation of Korea (NRF)
References (45)
Show source-provided references
1. Air-Writing Recognition Based on Deep Convolutional Neural Networks10.1109/ACCESS.2021.3121093 ↗
2. Metrics and Models for Handwritten Character Recognition10.1214/ss/1028905973 ↗
3. Singh, D., Khan, M.A., Bansal, A., and Bansal, N. (2015, January 7–8). An Application of SVM in Character Recognition with Chain Code. Proceedings of the International Conference Communication, Control and Intelligent Systems, CCIS 2015, Mathura, India.10.1109/CCIntelS.2015.7437901 ↗
4. Alqudah, A., Alqudah, A.M., Alquran, H., Al-zoubi, H.R., Al-qodah, M., and Al-khassaweneh, M.A. (2021). Recognition of Handwritten Arabic and Hindi Numerals Using Convolutional Neural Networks. Appl. Sci., 11.10.3390/app11041573 ↗
5. Handwritten Character Recognition from Images Using CNN-ECOC10.1016/j.procs.2020.03.293 ↗
6. Roy, P., Ghosh, S., and Pal, U. (2018, January 5–8). A CNN Based Framework for Unistroke Numeral Recognition in Air-Writing. Proceedings of the International Conference on Frontiers in Handwriting Recognition (ICFHR), Niagara Falls, NY, USA.10.1109/ICFHR-2018.2018.00077 ↗
7. A Letter Input System Based on Handwriting Gestures. Electron. Commun
8. Hand Gesture and Character Recognition Based on Kinect Sensor
9. Vision-Based Mid-Air Unistroke Character Input Using Polar Signatures10.1109/THMS.2017.2706695 ↗
10. Non-Touch Character Input System Based on Hand Tapping Gestures Using Kinect Sensor10.1109/ACCESS.2017.2703783 ↗
11. Saez-Mingorance, B., Mendez-Gomez, J., Mauro, G., Castillo-Morales, E., Pegalajar-Cuellar, M., and Morales-Santos, D.P. (2021). Air-Writing Character Recognition with Ultrasonic Transceivers. Sensors, 21.10.3390/s21206700 ↗
12. Otsubo, Y., Matsuki, K., and Nakai, M. (2013, January 4–6). A Study on Restoration of Acceleration Feature Vectors for Aerial Handwritten Character Recognition. Proceedings of the Forum on Information Technology, Tottori, Japan.
13. Ultra-Range Gesture Recognition Using a Web-Camera in Human–Robot Interaction10.1016/j.engappai.2024.108443 ↗
14. Chen, X., Su, L., Zhao, J., Qiu, K., Jiang, N., and Zhai, G. (2023). Sign Language Gesture Recognition and Classification Based on Event Camera with Spiking Neural Networks. Electronics, 12.10.3390/electronics12040786 ↗
15. Kuramochi, K., Tsukamoto, K., and Yanai, H.F. (2017, January 19–22). Accuracy Improvement of Aerial Handwritten Katakana Character Recognition. Proceedings of the 2017 56th Annual Conference of the Society of Instrument and Control Engineers of Japan (SICE), Kanazawa, Japan.10.23919/SICE.2017.8105492 ↗
16. Matsuki, K., and Nakai, M. (2011, January 7–9). Aerial Handwritten Character Recognition Using an Acceleration Sensor and Gyro Sensor. Proceedings of the Forum on Information Technology 2011, Hakodate, Japan.
17. Air-Writing Recognition Using Smart-Bands10.1016/j.pmcj.2020.101183 ↗
18. Novel Wearable HD-EMG Sensor with Shift-Robust Gesture Recognition Using Deep Learning10.1109/TBCAS.2023.3314053 ↗
19. A Methodological and Structural Review of Hand Gesture Recognition Across Diverse Data Modalities10.1109/ACCESS.2024.3456436 ↗
20. Deep Learning for Anomaly Detection: A Survey
21. Hsu, Y., Shen, Y., Jin, H., and Karaletsos, T. (2020). Generalized ODIN: Detecting Out-of-Distribution Image without Learning from Out-of-Distribution Data. arXiv.10.1109/CVPR42600.2020.01096 ↗
22. Exploring the Limits of Out-of-Distribution Detection
23. Deep Learning for Time Series Anomaly Detection: A Survey
24. Deep Learning for Anomaly Detection in Multivariate Time Series: Approaches, Applications, and Challenges
25. Deep Learning for Time Series Anomaly Detection: A Survey
26. Hong, Z., Yue, Y., Chen, Y., Cong, L., Lin, H., Luo, Y., and Xie, S. (2024). Out-of-Distribution Detection in Medical Image Analysis: A Survey. arXiv.
27. Reliable Deep Learning in Anomalous Diffusion Against Out-of-Distribution Dynamics10.1038/s43588-024-00703-7 ↗
28. Li, D., Song, K., Wang, J., and Zhang, Y. (2024). Time-Series Forecasting for OOD Generalization Using Invariant Learning. arXiv.
29. Zhang, K., Li, Q., Chen, Y., Liu, S., Li, Z., Liu, X., and Li, D. (2024, January 7–11). Diversify: A General Framework for Time Series OOD Detection and Generalization. Proceedings of the International Conference on Learning Representations (ICLR), Vienna, Austria.
30. Li, Y., Liu, J., Chen, Z., Liu, X., Wu, H., and Wang, L. (2024). Out-of-Distribution Detection via Deep Multi-Comprehension Ensemble. arXiv.
31. Wang, Z., Wang, S., Xu, H., Qiu, J., Liu, C., and Zhou, Z. (2024). DACR: Distribution-Augmented Contrastive Reconstruction for Time-Series Anomaly Detection. arXiv.10.1109/ICASSP48485.2024.10447891 ↗
32. Xu, S., Guo, M., Zhao, X., Liu, Y., Hu, J., and Huang, Y. (2025). TS-OOD: Evaluating Time-Series Out-of-Distribution Detection and Prospective Directions for Progress. arXiv.
33. Wen, Q., Sun, L., Yang, F., Song, X., Gao, J., Wang, X., and Xu, H. (2020). Time Series Data Augmentation for Deep Learning: A Survey. arXiv.10.24963/ijcai.2021/631 ↗
34. Yang, H., and Desell, T. (2022). Robust Augmentation for Multivariate Time Series Classification. arXiv.
35. Iwana, B.K., and Uchida, S. (2021). An Empirical Survey of Data Augmentation for Time Series Classification with Neural Networks. PLoS ONE, 16.10.1371/journal.pone.0254841 ↗
36. Le Guennec, A., Malinowski, S., and Tavenard, R. (2016, January 19–23). Data Augmentation for Time Series Classification Using Convolutional Neural Networks. Proceedings of the ECML/PKDD Workshop on Advanced Analytics and Learning on Temporal Data, Riva Del Garda, Italy.
37. Simonyan, K., and Zisserman, A. (2015, January 7–9). Very Deep Convolutional Networks for Large-Scale Image Recognition. Proceedings of the International Conference on Learning Representations (ICLR), San Diego, CA, USA.
38. Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., and Wojna, Z. (2016, January 27–30). Rethinking the Inception Architecture for Computer Vision. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.10.1109/CVPR.2016.308 ↗
39. Chollet, F. (2017, January 21–26). Xception: Deep Learning with Depthwise Separable Convolutions. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.10.1109/CVPR.2017.195 ↗
40. Zoph, B., Vasudevan, V., Shlens, J., and Le, Q.V. (2018, January 18–22). Learning Transferable Architectures for Scalable Image Recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Salt Lake City, UT, USA.10.1109/CVPR.2018.00907 ↗
41. Tan, M., and Le, Q.V. (2019, January 9–15). EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks. Proceedings of the International Conference on Machine Learning (ICML), Long Beach, CA, USA.
42. Liu, Z., Mao, H., Wu, C.Y., Feichtenhofer, C., Darrell, T., and Xie, S. (2022, January 19–24). A ConvNet for the 2020s. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA.10.1109/CVPR52688.2022.01167 ↗
43. Wang, S., Wu, H., Shi, X., Hu, T., Luo, H., Ma, L., Zhang, J.Y., and Zhou, J. (2024, January 1–5). TimeMixer: Decomposable Multiscale Mixing for Time Series Forecasting. Proceedings of the International Conference on Learning Representations (ICLR), Virtual Event.
44. Luo, D., and Wang, X. (2024, January 1–5). ModernTCN: A Modern Pure Convolution Structure for General Time Series Analysis. Proceedings of the International Conference on Learning Representations (ICLR), Virtual Event.
45. Cheng, H., Wen, Q., Liu, Y., and Sun, L. (2024, January 1–5). RobustTSF: Towards Theory and Design of Robust Time Series Forecasting with Anomalies. Proceedings of the International Conference on Learning Representations (ICLR), Virtual Event.