A User-Specific Hand Gesture Recognition Model Based on Feed-Forward Neural Networks, EMGs, and Correction of Sensor Orientation
Source abstract
Abstract
Hand gesture recognition systems have several applications including medicine and engineering. A gesture recognition system should identify the class, time, and duration of a gesture executed by a user. Gesture recognition systems based on electromyographies (EMGs) produce good results when the EMG sensor is placed on the same orientation for training and testing. However, when the orientation of the sensor changes between training and testing, which is very common in practice, the classification and recognition accuracies degrade significantly. In this work, we propose a system for recognizing, in real time, five gestures of the right hand. These gestures are the same ones recognized by the proprietary system of the Myo armband. The proposed system is based on the use of a shallow artificial feed-forward neural network. This network takes as input the covariances between the channels of an EMG and the result of a bag of five functions applied to each channel of an EMG. To correct the rotation of the EMG sensor, we also present an algorithm based on finding the channel of maximum energy given a set of synchronization EMGs, which for this work correspond to the gesture waveout. The classification and recognition accuracies obtained here show that the recognition system, together with the algorithm for correcting the orientation, allows a user to wear the EMG sensor in different orientations for training and testing, without a significant performance reduction. Finally, to reproduce the results obtained in this paper, we have made the code and the dataset used here publicly available.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Muscle activation and electromyography studies · Hand Gesture Recognition Systems · EEG and Brain-Computer Interfaces
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Artificial Intelligence and Computer Vision Research Lab, Departamento de Informática y Ciencias de la Computación (DICC), Escuela Politécnica Nacional, Quito 170517, Ecuador
Funding
CEDIA
References (46)
Show source-provided references
1. Young, S., Stephens-Fripp, B., Gillett, A., Zhou, H., and Alici, G. (2019, January 8–12). Pattern Recognition for Prosthetic Hand User’s Intentions using EMG Data and Machine Learning Techniques. Proceedings of the 2019 IEEE/ASME International Conference on Advanced Intelligent Mechatronics (AIM), Hong Kong, China.10.1109/AIM.2019.8868766 ↗
2. Jiménez, L.A.E., Benalcázar, M.E., and Sotomayor, N. (2016, January 26–28). Gesture recognition and machine learning applied to sign language translation. Proceedings of the VII Latin American Congress on Biomedical Engineering CLAIB 2016, Bucaramanga, Santander, Colombia.
3. Saggio, G., and Sbernini, L. (2011, January 30–31). New scenarios in human trunk posture measurements for clinical applications. Proceedings of the 2011 IEEE International Symposium on Medical Measurements and Applications, Bari, Italy.10.1109/MeMeA.2011.5966651 ↗
4. A fuzzy integral ensemble method in visual P300 brain-computer interface10.1155/2016/9845980 ↗
5. León, M.A., Romero, P.A., Quevedo, W.X., Arteaga, O.B., Terán, C., Benalcázar, M.E., and Andaluz, V.H. (2018, January 24–27). Virtual Rehabilitation System for Fine Motor Skills Using a Functional Hand Orthosis. Proceedings of the International Conference on Augmented Reality, Virtual Reality and Computer Graphics, Otranto, Italy.10.1007/978-3-319-95282-6_6 ↗
6. Saggio, G., Bisegna, P., Latessa, G., and Bocchetti, S. (2009, January 15–19). Mechanical modeling of bend sensors exploited to measure human joint movements. Proceedings of the 2009 IEEE International Symposium on a World of Wireless, Mobile and Multimedia Networks & Workshops, Kos, Greece.10.1109/WOWMOM.2009.5282407 ↗
7. Jaramillo-Yánez, A., Benalcázar, M.E., and Mena-Maldonado, E. (2020). Real-Time Hand Gesture Recognition Using Surface Electromyography and Machine Learning: A Systematic Literature Review. Sensors, 20.10.3390/s20092467 ↗
8. Chung, E.A., and Benalcázar, M.E. (2019, January 2–6). Real-Time Hand Gesture Recognition Model Using Deep Learning Techniques and EMG Signals. Proceedings of the 2019 27th European Signal Processing Conference (EUSIPCO), A Coruna, Spain.10.23919/EUSIPCO.2019.8903136 ↗
9. Qi, J., Jiang, G., Li, G., Sun, Y., and Tao, B. (2019). Surface 539 EMG hand gesture recognition system based on PCA and 540 GRNN. Neural Comput. Appl.10.1007/s00521-019-04142-8 ↗
10. Robust hand gesture recognition with a double channel surface EMG wearable armband and SVM classifier10.1016/j.bspc.2018.07.010 ↗
11. Zhang, Z., Yang, K., Qian, J., and Zhang, L. (2019). Real-Time Surface EMG Pattern Recognition for Hand Gestures Based on an Artificial Neural Network. Sensors, 19.10.3390/s19143170 ↗
12. Abhishek, K.S., Qubeley, L.C.K., and Ho, D. (2016, January 3–5). Glove-Based Hand Gesture Recognition Sign Language Translator Using Capacitive Touch Sensor. Proceedings of the 2016 IEEE International Conference on Electron Devices and Solid-State Circuits (EDSSC), Hong Kong, China.10.1109/EDSSC.2016.7785276 ↗
13. Design and Myoelectric Control of an Anthropomorphic Prosthetic Hand10.1016/S1672-6529(16)60377-3 ↗
14. Sign Languages Recognition Based on Neural Network Architecture10.1007/978-3-319-59480-4_12 ↗
15. Benatti, S., Rovere, G., Bosser, J., Montagna, F., Farella, E., Glaser, H., Schonle, P., Burger, T., Fateh, S., and Huang, Q. (2017, January 15–16). A Sub-10mW Real-Time Implementation for EMG Hand Gesture Recognition Based on a Multi-Core Biomedical SoC. Proceedings of the 2017 7th International Workshop on Advances in Sensors and Interfaces, IWASI 2017, Vieste, Italy.10.1109/IWASI.2017.7974234 ↗
16. A hand gesture recognition sensor using reflected impulses10.1109/JSEN.2017.2679220 ↗
17. A framework for hand gesture recognition based on accelerometer and EMG sensors10.1109/TSMCA.2011.2116004 ↗
18. Generalized Hand Gesture Recognition for Wearable Devices in IoT: Application and Implementation Challenges
19. Moschetti, A., Fiorini, L., Esposito, D., Dario, P., and Cavallo, F. (2016). Recognition of Daily Gestures with Wearable Inertial Rings and Bracelets. Sensors, 16.10.3390/s16081341 ↗
20. Dynamic hand gesture recognition using vision-based approach for human–computer interaction10.1007/s00521-016-2525-z ↗
21. Luh, G.C., Lin, H.A., Ma, Y.H., and Yen, C.J. (2015, January 12–15). Intuitive muscle-gesture based robot navigation control using wearable gesture armband. Proceedings of the 2015 International Conference on Machine Learning and Cybernetics (ICMLC), Guangzhou, China.10.1109/ICMLC.2015.7340953 ↗
22. Benalcazar, M.E., Motoche, C., Zea, J.A., Jaramillo, A.G., Anchundia, C.E., Zambrano, P., Segura, M., Benalcazar Palacios, F., and Perez, M. (2017, January 16–20). Real-Time Hand Gesture Recognition Using the Myo Armband and Muscle Activity Detection. Proceedings of the 2017 IEEE 2nd Ecuador Technical Chapters Meeting (ETCM 2017), Salinas, Ecuador.10.1109/ETCM.2017.8247458 ↗
23. Fortunato, A., King, J., and Bonneau, P.J. (2020, November 26). Data Display, Acquisition and Feedback System for Biomedical Experiments. Available online: https://digitalcommons.wpi.edu/mqp-all/340.
24. Weiss, L.D., Weiss, J.M., and Silver, J.K. (2015). Easy EMG: A Guide to Performing Nerve Conduction Studies and Electromyography, Elsevier.10.1016/B978-0-323-28664-0.00004-7 ↗
25. Vimos, V.H., Benalcázar, M., Oña, A.F., and Cruz, P.J. (2019, January 28–31). A Novel Technique for Improving the Robustness to Sensor Rotation in Hand Gesture Recognition Using sEMG. Proceedings of the International Conference on Computer Science, Electronics and Industrial Engineering (CSEI), Ambato, Ecuador.10.1007/978-3-030-33614-1_16 ↗
26. Boschmann, A., and Platzner, M. (September, January 28). Reducing classification accuracy degradation of pattern recognition based myoelectric control caused by electrode shift using a high density electrode array. Proceedings of the 2012 Annual International Conference of the IEEE Engineering in Medicine and Biology Society, San Diego, CA, USA.10.1109/EMBC.2012.6346923 ↗
27. Wearing-independent Hand Gesture Recognition Method Based on EMG Armband10.1007/s00779-018-1152-3 ↗
28. Xu, Z., Shen, L., Qian, J., and Zhang, Z. (2020). Advanced Hand Gesture Prediction Robust to Electrode Shift with an Arbitrary Angle. Sensors, 20.10.3390/s20041113 ↗
29. A Multi-Window Majority Voting Strategy to Improve Hand Gesture Recognition Accuracies Using Electromyography Signal10.1109/TNSRE.2019.2961706 ↗
30. Single Channel Surface EMG Control of Advanced Prosthetic Hands: A Simple, Low Cost and Efficient Approach10.1016/j.eswa.2017.03.012 ↗
31. Identification of Gesture Based on Combination of Raw sEMG and sEMG Envelope Using Supervised Learning and Univariate Feature Selection10.1007/s42235-019-0052-1 ↗
32. A Dynamic Gesture Recognition and Prediction System Using the Convexity Approach10.1016/j.cviu.2016.10.006 ↗
33. Duda, R.O., Hart, P.E., and Stork, D.G. (2012). Pattern Classification, John Wiley & Sons.
34. Arai, K., Kapoor, S., and Bhatia, R. (2020). Effect of Analysis Window and Feature Selection on Classification of Hand Movements Using EMG Signal. Intelligent Systems and Applications, Springer International Publishing.
35. Cortes Tobar, D.F., Hoang Duy, V., and Trong Dao, T. (2019). Movement Control System for a Transradial Prosthesis Using Myoelectric Signals. AETA 2019—Recent Advances in Electrical Engineering and Related Sciences: Theory and Application, Springer International Publishing.
36. Gesture Recognition for Human-Robot Collaboration: A Review10.1016/j.ergon.2017.02.004 ↗
37. A Study of Personal Recognition Method Based on EMG Signal10.1109/TBCAS.2020.3005148 ↗
38. Jabbari, M., Khushaba, R.N., and Nazarpour, K. (2020, January 20–24). EMG-Based Hand Gesture Classification with Long Short-Term Memory Deep Recurrent Neural Networks. Proceedings of the 2020 42nd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC), Montreal, QC, Canada.10.1109/EMBC44109.2020.9175279 ↗
39. Techniques of EMG signal analysis: Detection, processing, classification and applications10.1251/bpo115 ↗
40. Too, J., Abdullah, A.R., and Saad, N.M. (2019). Classification of Hand Movements based on Discrete Wavelet Transform and Enhanced Feature Extraction. Int. J. Adv. Comput. Sci. Appl., 10.10.14569/IJACSA.2019.0100612 ↗
41. Classification of EMG signal based on time domain and frequency domain features
42. Neural networks with a continuous squashing function in the output are universal approximators10.1016/S0893-6080(00)00031-9 ↗
43. Conjugate Gradient Back-propagation with Modified Polack -Rebier updates for training feed forward neural network
44. Chen, L., Fu, J., Wu, Y., Li, H., and Zheng, B. (2020). Hand Gesture Recognition Using Compact CNN via Surface Electromyography Signals. Sensors, 20.10.3390/s20030672 ↗
45. (2020, November 26). Dataset EMG-EPN-120. Available online: https://laboratorio-ia.epn.edu.ec/en/resources/dataset/2020_emg_dataset_120.
46. (2020, November 26). laboratorioAI/2020_ROT_SPECIFIC_ANN_EMG_120: Code Example for the Paper “A User-Specific Hand Gesture Recognition Model Based on Feed-Forward Neural Networks, EMGs and Correction of Sensor Orientation”. Available online: https://github.com/laboratorioAI/2020_ROT_SPECIFIC_ANN_EMG_120.