Stretchable array electromyography sensor with graph neural network for static and dynamic gestures recognition system
Source abstract
Abstract
Abstract With advances in artificial intelligence (AI)-based algorithms, gesture recognition accuracy from sEMG signals has continued to increase. Spatiotemporal multichannel-sEMG signals substantially increase the quantity and reliability of the data for any type of study. Here, we report an array of bipolar stretchable sEMG electrodes with a self-attention-based graph neural network to recognize gestures with high accuracy. The array is designed to spatially cover the skeletal muscles to acquire the regional sampling data of EMG activity from 18 different gestures. The system can differentiate individual static and dynamic gestures with ~97% accuracy when training a single trial per gesture. Moreover, a sticky patchwork of holes adhered to an array sensor enables skin-like attributes such as stretchability and water vapor permeability and aids in delivering stable EMG signals. In addition, the recognition accuracy (~95%) remained unchanged even after long-term testing for over 72 h and being reused more than 10 times.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Muscle activation and electromyography studies · Advanced Sensor and Energy Harvesting Materials · EEG and Brain-Computer Interfaces
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
References (48)
Show source-provided references
1. Hayashi, H. & Tsuji, T. Human–machine interfaces based on bioelectric signals: a narrative review with a novel system proposal. IEEJ Trans. Electr. Electron. Eng. 17, 1536–1544 (2022).10.1002/tee.23646 ↗
2. Zheng, M., Crouch, M. S. & Eggleston, M. S. Surface electromyography as a natural human–machine interface: a review. IEEE Sens. J. 22, 9198–9214 (2022).10.1109/JSEN.2022.3165988 ↗
3. Tian, L. et al. Large-area MRI-compatible epidermal electronic interfaces for prosthetic control and cognitive monitoring. Nat. Biomed. Eng. 3, 194–205 (2019).10.1038/s41551-019-0347-x ↗
4. Kapur, A., Kapur, S. & Maes, P. AlterEgo. In 23rd Int. Conf. Intell. User Interfaces 43–53 (ACM, 2018).10.1145/3172944.3172977 ↗
5. Torres-Castillo, J. R., López-López, C. O. & Padilla-Castañeda, M. A. Neuromuscular disorders detection through time-frequency analysis and classification of multi-muscular EMG signals using Hilbert-Huang transform. Biomed. Signal Process. Control 71, 103037 (2022).10.1016/j.bspc.2021.103037 ↗
6. Goen, A. Classification of EMG signals for assessment of neuromuscular disorders. Int. J. Electron. Electr. Eng. 2, 242–248 (2014).10.12720/ijeee.2.3.242-248 ↗
7. Verma, A. R. & Gupta, B. Detecting neuromuscular disorders using EMG signals based on TQWT features. Augment. Hum. Res. 5, 8 (2020).10.1007/s41133-019-0020-7 ↗
8. Singh, A., Dutta, M. K. & Travieso, C. M. Analysis of EMG signals for automated diagnosis of myopathy. In 2017 4th IEEE Uttar Pradesh Sect. Int. Conf. Electr. Comput. Electron. 628–631 (IEEE, 2017).10.1109/UPCON.2017.8251122 ↗
9. Kim, H., Lee, J. & Kim, J. Electromyography-signal-based muscle fatigue assessment for knee rehabilitation monitoring systems. Biomed. Eng. Lett. 8, 345–353 (2018).10.1007/s13534-018-0078-z ↗
10. EdwardJero, S., Divya Bharathi, K., Karthick, P. A., Ramakrishnan, S. Musclefatigue analysis in isometric contractions using geometric features of surfaceelectromyography signals. Biomed. Signal Process. Control 68, 102603 (2021).10.1016/j.bspc.2021.102603 ↗
11. Kakoty, N. M., Gohain, L., Saikia, J. B., Kalita, A. J. & Borah, S. Real-time EMG based prosthetic hand controller realizing neuromuscular constraint. Int. J. Intell. Robot. Appl. 6, 530–542 (2022).10.1007/s41315-021-00221-z ↗
12. Farnsworth, B. D., Triolo, R. J. & Young, D. J. Wireless implantable EMG sensing microsystem. In 2008 IEEE Sensors 1245–1248 (IEEE, 2008).10.1109/ICSENS.2008.4716669 ↗
13. Kocejko, T., Ruminski, J., Przystup, P., Polinski, A. & Wtorek, J. The role of EMG module in hybrid interface of prosthetic arm. In 2017 10th Int. Conf. Hum. Syst. Interact. 36–40 (IEEE, 2017).10.1109/HSI.2017.8004992 ↗
14. Holobar, A., Minetto, M. A., Botter, A., Negro, F. & Farina, D. Experimental analysis of accuracy in the identification of motor unit spike trains from high-density surface EMG. IEEE Trans. Neural Syst. Rehabil. Eng. 18, 221–229 (2010).10.1109/TNSRE.2010.2041593 ↗
15. Zhu, K., Guo, W., Yang, G., Li, Z. & Wu, H. High-fidelity recording of EMG signals by multichannel on-skin electrode arrays from target muscles for effective human–machine interfaces. ACS Appl. Electron. Mater. 3, 1350–1358 (2021).10.1021/acsaelm.0c01129 ↗
16. Savur, C. & Sahin, F. American Sign Language Recognition system by using surface EMG signal. In 2016 IEEE Int. Conf. Syst. Man, Cybern. 002872–002877 (IEEE, 2016).10.1109/SMC.2016.7844675 ↗
17. Rossi, M., Benatti, S., Farella, E. & Benini, L. Hybrid EMG classifier based on HMM and SVM for hand gesture recognition in prosthetics. In 2015 IEEE Int. Conf. Ind. Technol. 1700–1705 (IEEE, 2015).10.1109/ICIT.2015.7125342 ↗
18. Benatti, S. et al. A versatile embedded platform for EMG acquisition and gesture recognition. IEEE Trans. Biomed. Circuits Syst. 9, 620–630 (2015).10.1109/TBCAS.2015.2476555 ↗
19. Chen, X. & Wang, Z. J. Pattern recognition of number gestures based on a wireless surface EMG system. Biomed. Signal Process. Control 8, 184–192 (2013).10.1016/j.bspc.2012.08.005 ↗
20. Huang, Y., Englehart, K. B., Hudgins, B. & Chan, A. D. C. A Gaussian mixture model based classification scheme for myoelectric control of powered upper limb prostheses. IEEE Trans. Biomed. Eng. 52, 1801–1811 (2005).10.1109/TBME.2005.856295 ↗
21. Ajiboye, A. B. & Weir, R. F. A heuristic fuzzy logic approach to EMG pattern recognition for multifunctional prosthesis control. IEEE Trans. Neural Syst. Rehabil. Eng. 13, 280–291 (2005).10.1109/TNSRE.2005.847357 ↗
22. Hasan, M. M., Rahaman, A., Faisal Shuvo, M., Abu Saleh Ovi, M., & Rahman, M. M. Human hand gesture detection based on EMG signal using ANN. In 2014 Int. Conf. Informatics, Electron. Vis. 1–5 (IEEE, 2014).10.1109/ICIEV.2014.6850687 ↗
23. Kondo, G., Kato, R., Yokoi, H. & Arai, T. Classification of individual finger motions hybridizing electromyogram in transient and converged states. In 2010 IEEE Int. Conf. Robot. Autom. 2909–2915 (IEEE, 2010).10.1109/ROBOT.2010.5509493 ↗
24. Zhang, H. et al. An adaptation strategy of using LDA classifier for EMG pattern recognition. In 2013 35th Annu. Int. Conf. IEEE Eng. Med. Biol. Soc. 4267–4270 (IEEE, 2013).
25. Xu, K., Weihua, H., Jure, L. & Stefanie, J. How powerful are graph neural networks? International Conference on Learning Representations (ICLR), (2019).
26. Lai, Z. et al. In Lecture Notes in Computer Science. Vol. 13110. 27–39 (Springer, 2021).
27. Gandla, S. et al. Ultrafast prototyping of large-area stretchable electronic systems by laser ablation technique for controllable robotic arm operations. IEEE Trans. Ind. Electron. 69, 4245–4253 (2022).10.1109/TIE.2021.3073355 ↗
28. Chae, H. et al. Laser-processed nature-inspired deformable structures for breathable and reusable electrophysiological sensors toward controllable home electronic appliances and psychophysiological stress monitoring. ACS Appl. Mater. Interfaces 11, 28387–28396 (2019).10.1021/acsami.9b06363 ↗
29. Daubechies, I. The wavelet transform, time-frequency localization and signal analysis. IEEE Trans. Inf. Theory 36, 961–1005 (1990).10.1109/18.57199 ↗
30. Reaz, M. B. I., Hussain, M. S. & Mohd-Yasin, F. Techniques of EMG signal analysis: detection, processing, classification and applications. Biol. Proced. Online 8, 11–35 (2006).10.1251/bpo115 ↗
31. Cote-Allard, U. et al. Deep learning for electromyographic hand gesture signal classification using transfer learning. IEEE Trans. Neural Syst. Rehabil. Eng. 27, 760–771 (2019).10.1109/TNSRE.2019.2896269 ↗
32. Jarrett, K., Kavukcuoglu, K., Ranzato, M. A. & LeCun, Y. What is the best multi-stage architecture for object recognition? In 2009 IEEE 12th Int. Conf. Comput. Vis. 2146–2153 (IEEE, 2009).10.1109/ICCV.2009.5459469 ↗
33. Dauphin, Y. N., Fan, A., Auli, M. & Grangier, D. Language modeling with gated convolutional networks. Int. Conf. Mach. Learn. PMLR 70, 933–941 (2017).
34. Kipf, T. N. & Welling, M. Semi-supervised classification with graph convolutional networks. International Conference on Learning Representations (ICLR), (2017).
35. Bai, D. et al. Multi-channel sEMG signal gesture recognition based on improved CNN-LSTM hybrid models. In 2021 IEEE Int. Conf. Intell. Saf. Robot. 111–116 (IEEE, 2021).10.1109/ISR50024.2021.9419532 ↗
36. Yang, Z. et al. Dynamic gesture recognition using surface emg signals based on multi-stream residual network. Front. Bioeng. Biotechnol. 9, 779353 (2021).10.3389/fbioe.2021.779353 ↗
37. Joodaki, H. & Panzer, M. B. Skin mechanical properties and modeling: a review. Proc. Inst. Mech. Eng. Part H. J. Eng. Med. 232, 323–343 (2018).10.1177/0954411918759801 ↗
38. Park, G. B., Courtney, J. M., McNair, A. & Gaylor, J. D. S. The design and evaluation of a burn wound covering. Eng. Med. 7, 11–15 (1978).10.1243/EMED_JOUR_1978_007_006_02 ↗
39. Wang, L., Khor, E., Wee, A. & Lim, L. Y. Chitosan-alginate PEC membrane as a wound dressing: assessment of incisional wound healing. J. Biomed. Mater. Res. 63, 610–618 (2002).10.1002/jbm.10382 ↗
40. Schwope, A. D., Wise, D. L., Sell, K. W., Dressler, D. P. & Skornick, W. A. Evaluation of wound-covering materials. J. Biomed. Mater. Res. 11, 489–502 (1977).10.1002/jbm.820110405 ↗
41. Amma, C., Krings, T., Böer, J. & Schultz, T. Advancing Muscle-Computer Interfaces with High-Density Electromyography. In Proc. 33rd Annu. ACM Conf. Hum. Factors Comput. Syst. 929–938 (ACM, 2015).10.1145/2702123.2702501 ↗
42. Liu, X. et al. The virtual trackpad: an electromyography-based, wireless, real-time, low-power, embedded hand-gesture-recognition system using an event-driven artificial neural network. IEEE Trans. Circuits Syst. II Express Briefs 64, 1257–1261 (2017).
43. Moin, A. et al. An EMG gesture recognition system with flexible high-density sensors and brain-inspired high-dimensional classifier. In 2018 IEEE Int. Symp. Circuits Syst. 1–5 (IEEE, 2018).10.1109/ISCAS.2018.8351613 ↗
44. Pancholi, S. & Joshi, A. M. Electromyography-based hand gesture recognition system for upper limb amputees. IEEE Sens. Lett. 3, 1–4 (2019).
45. Benatti, S. et al. Online learning and classification of EMG-based gestures on a parallel ultra-low power platform using hyperdimensional computing. IEEE Trans. Biomed. Circuits Syst. 13, 516–528 (2019).10.1109/TBCAS.2019.2914476 ↗
46. Cerone, G. L., Botter, A. & Gazzoni, M. A modular, smart, and wearable system for high density sEMG detection. IEEE Trans. Biomed. Eng. 66, 3371–3380 (2019).10.1109/TBME.2019.2904398 ↗
47. Moin, A. et al. A wearable biosensing system with in-sensor adaptive machine learning for hand gesture recognition. Nat. Electron. 4, 54–63 (2021).10.1038/s41928-020-00510-8 ↗
48. Vaswani, A. et al. Attention is all you need. Advances in Neural Information Processing Systems (NIPS), (2017).