journal article·2025-05-09Open access

Analysis of a Novel Training Game with Eye Tracking and Electromyography for Autonomous Wheelchair Control

Peter Smith, Matt Dombrowski, Viviana Rivera, Maanya Pradeep, Delaney Gunnell, John Sparkman, Albert Manero

Applied Sciences· Volume 15· Issue 10

Abstract

A novel electromyography (EMG)-based wheelchair interface was developed that uses contractions from the temporalis muscle to control a wheelchair. To aid in the training process for users of this interface, a serious training game, Limbitless Journey, was developed to support patients. Amyotrophic Lateral Sclerosis (ALS) is a condition that causes progressive motor function loss, and while many people with ALS use wheelchairs as mobility devices, a traditional joystick-based wheelchair interface may become inaccessible as the condition progresses. Limbitless Journey simulates the wheelchair interface by utilizing the same temporalis muscle contractions for control of in-game movements, but in a low-stress learning environment. A usability study was conducted to evaluate the serious-game-based training platform. A major outcome of this study was qualitative data gathered through a concurrent think-aloud methodology. Three cohorts of five participants participated in the study. Audio recordings of participants using Limbitless Journey were transcribed, and a sentiment analysis was performed to evaluate user perspectives. The goal of the study was twofold: first, to perform a think-aloud usability study on the game; second, to determine whether accessible controls could be as effective as manual controls. The user comments were coded into the following categories: game environment, user interface interactions, and controller usability. The game environment category had the most positive comments, while the most negative comments were primarily related to usability challenges with the flexion-based controller. Interactions with the user interface were the main topic of feedback for improvement in future game versions. This game will be utilized in subsequent trials conducted at the facility to test its efficacy as a novel training system for the ALS population. The feedback collected will be implemented in future versions of the game to improve the training process.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: Gaze Tracking and Assistive Technology · Stroke Rehabilitation and Recovery · Ergonomics and Musculoskeletal Disorders

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

Nicholson School of Communication and Media, University of Central Florida, 12405 Aquarius Agora Dr., Orlando, FL 32816, USA

School of Visual Arts and Design, University of Central Florida, 12400 Aquarius Agora Dr., Orlando, FL 32816, USA

Limbitless Solutions, University of Central Florida, 12703 Research Parkway, Suite 100, Orlando, FL 32826, USA

References (57)

Show source-provided references
1. Limchesing, T.J.C., Vidad, A.J.P., Munsayac, F.E.T., Bugtai, N.T., and Baldovino, R.G. (2021, January 28–30). Application of EEG Device and BCI for the Brain-controlled Automated Wheelchair. Proceedings of the 2021 IEEE 13th International Conference on Humanoid, Nanotechnology, Information Technology, Communication and Control, Environment, and Management (HNICEM), Manila, Philippines.10.1109/HNICEM54116.2021.9731931 ↗
2. Felzer, T. (2025, March 05). On the Possibility of Developing a Brain-Computer Interface (BCI). Available online: https://api.semanticscholar.org/CorpusID:18261749.
3. Solea, R., Margarit, A., Cernega, D., and Serbencu, A. (2019, January 9–11). Head movement control of powered wheelchair. Proceedings of the 2019 23rd International Conference on System Theory, Control and Computing (ICSTCC), Sinaia, Romania.10.1109/ICSTCC.2019.8885844 ↗
4. Palumbo, A., Ielpo, N., Calabrese, B., Garropoli, R., Gramigna, V., Ammendolia, A., and Marotta, N. (2024). An Innovative Device Based on Human-Machine Interface (HMI) for Powered Wheelchair Control for Neurodegenerative Disease: A Proof-of-Concept. Sensors, 24.10.3390/s24154774 ↗
5. Rash, G.S., and Quesada, P. (2024, November 13). Electromyography Fundamentals. Available online: https://people.stfx.ca/smackenz/courses/HK474/Labs/EMG%20Lab/EMGfundamentals.pdf.
6. Evaluating surface EMG control of motorized wheelchairs for amyotrophic lateral sclerosis patients10.1186/s12984-022-01066-8 ↗
7. Soderberg, G.L. (1992). Selected Topics in Surface Electromyography for Use in the Occupational Setting: Expert Perspectives, DHHS (NIOSH) Publication Number 91-100.
8. Surface electromyography as a natural human–machine interface: A review10.1109/JSEN.2022.3165988 ↗
9. Zhang, J., Gao, S., Zhou, K., Cheng, Y., and Mao, S. (2023). An online hybrid BCI combining SSVEP and EOG-based eye movements. Front. Hum. Neurosci., 17.10.3389/fnhum.2023.1103935 ↗
10. An autonomous hybrid brain-computer interface system combined with eye-tracking in virtual environment10.1016/j.jneumeth.2021.109442 ↗
11. Dillen, A., Omidi, M., Díaz, M.A., Ghaffari, F., Roelands, B., Vanderborght, B., Romain, O., and De Pauw, K. (2024). Evaluating the real-world usability of BCI control systems with augmented reality: A user study protocol. Front. Hum. Neurosci., 18.10.3389/fnhum.2024.1448584 ↗
12. Hardiman, O., Doherty, C.P., Elamin, M., and Bede, P. (2016). Amyotrophic Lateral Sclerosis. Neurodegenerative Disorders: A Clinical Guide, Springer International Publishing.10.1007/978-3-319-23309-3 ↗
13. Clinical Spectrum of Amyotrophic Lateral Sclerosis (ALS)10.1101/cshperspect.a024117 ↗
14. Wheelchair use by patients with amyotrophic lateral sclerosis: A survey of user characteristics and selection preferences10.1053/apmr.2001.18062 ↗
15. Moore, K.L., and Dalley, A.F. (2018). Clinically Oriented Anatomy, Wolters Kluwer India Pvt Ltd.
16. Alterations in the masticatory system in patients with amyotrophic lateral sclerosis10.11607/ofph.1882 ↗
17. Smith, P., Dombrowski, M., MacDonald, C., Williams, C., Pradeep, M., Barnum, E., Rivera, V.P., Sparkman, J., and Manero, A. (2024, January 4–7). Initial Evaluation of a Hybrid eye tracking and Electromyography Training Game for Hands-Free Wheelchair Use. Proceedings of the 2024 Symposium on Eye Tracking Research and Applications, Glasgow, UK.10.1145/3649902.3653006 ↗
18. Susi, T., Johannesson, M., and Backlund, P. (2025, March 05). Serious Games: An Overview. Available online: https://www.diva-portal.org/smash/get/diva2:2416/fulltext01.pdf.
19. PhantomAR: Gamified mixed reality system for alleviating phantom limb pain in upper limb amputees—Design, implementation, and clinical usability evaluation10.1186/s12984-025-01554-7 ↗
20. Sánchez-Gil, J.J., Sáez-Manzano, A., López-Luque, R., Ochoa-Sepúlveda, J.J., and Cañete-Carmona, E. (2024). Gamified devices for stroke rehabilitation: A systematic review. Comput. Methods Programs Biomed., 258.10.1016/j.cmpb.2024.108476 ↗
21. PlayBionic: Game-Based Interventions to Encourage Patient Engagement and Performance in Prosthetic Motor Rehabilitation10.1016/j.pmrj.2018.09.027 ↗
22. Mechanism of gamification: Role of flow in the behavioral and emotional pathways of engagement in management education
23. Smith, P.A., and Dombrowski, M. (2017, January 2–4). Designing games to help train children to use Prosthetic Arms. Proceedings of the 2017 IEEE 5th International Conference on Serious Games and Applications for Health (SeGAH), Perth, Australia.10.1109/SeGAH.2017.7939265 ↗
24. Utilizing Electromyographic Video Games Controllers to Improve Outcomes for Prosthesis Users10.1007/s10484-023-09598-y ↗
25. Wäppling, A., Walchshofer, L., and Lewin, R. (2022). What Makes a Cozy Game? A Study of Three Games Considered Cozy. [Bachelor’s Thesis, Uppsala University].
26. Csikszentmihalyi, M., and Csikzentmihaly, M. (1990). Flow: The Psychology of Optimal Experience, Harper & Row.
27. Nielsen, J., and Landauer, T.K. (1993, January 24–29). A mathematical model of the finding of usability problems. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems—CHI’93, Amsterdam, The Netherlands.10.1145/169059.169166 ↗
28. An Empirical Evaluation of the System Usability Scale10.1080/10447310802205776 ↗
29. The development and validation of the game user experience satisfaction scale (GUESS)10.1177/0018720816669646 ↗
30. Prokop, M., Pilař, L., and Tichá, I. (2020). Impact of Think-Aloud on Eye-Tracking: A Comparison of Concurrent and Retrospective Think-Aloud for Research on Decision-Making in the Game Environment. Sensors, 20.10.3390/s20102750 ↗
31. Using the think aloud protocol in health professions education: An interview method for exploring thought processes: AMEE Guide No. 15110.1080/0142159X.2022.2155123 ↗
32. Do Procedures for Verbal Reporting of Thinking Have to Be Reactive? A Meta-Analysis and Recommendations for Best Reporting Methods10.1037/a0021663 ↗
33. Bellini-Leite, S.C. (2022). Dual process theory: Embodied and predictive; symbolic and classical. Front. Psychol., 13.10.3389/fpsyg.2022.805386 ↗
34. Does thinking-aloud affect learning, visual information processing and cognitive load when learning with seductive details as expected from self-regulation perspective?10.1016/j.chb.2020.106411 ↗
35. (2025, February 12). Zoom. Available online: https://www.zoom.com/.
36. (2025, February 12). Temi. Available online: https://www.temi.com/.
37. Muhr, T. (ATLAS. ti 6.0, 2004). ATLAS. ti 6.0, version 6.
38. Assessing concurrent think-aloud protocol as a usability test method: A technical communication approach10.1109/TPC.2010.2052859 ↗
39. Analysis of academic research data with the use of ATLAS. ti. Experiences of use in the area of Tourism and Hospitality Administration10.56294/dm2024306 ↗
40. The Validity of Sentiment Analysis: Comparing Manual Annotation, Crowd-Coding, Dictionary Approaches, and Machine Learning Algorithms10.1080/19312458.2020.1869198 ↗
41. Becker, I., Parkin, S., and Sasse, M.A. (November, January 30). Combining qualitative coding and sentiment analysis: Deconstructing perceptions of usable security in organisations. Proceedings of the the LASER Workshop: Learning from Authoritative Security Experiment Results (LASER 2016), Boston, MA, USA.
42. A team-based approach to open coding: Considerations for creating intercoder consensus10.1177/1525822X19838237 ↗
43. Dry electrodes for electrocardiography10.1088/0967-3334/34/9/R47 ↗
44. Yoganathan, K., Dharmadasa, T., Northall, A., Talbot, K., Thompson, A.G., and Turner, M.R. (2025). Asymmetry in amyotrophic lateral sclerosis: Clinical, neuroimaging and histological observations. Brain, awaf121.10.1093/brain/awaf121 ↗
45. Ericsson, K.A., and Simon, H.A. (1993). Protocol Analysis: Verbal Reports as Data, EUA Massachusetts Institute of Technology.10.7551/mitpress/5657.001.0001 ↗
46. Retrospective vs. concurrent think-aloud protocols: Testing the usability of an online library catalogue10.1080/0044929031000 ↗
47. Evaluation of different modalities of verbalization in a sorting task10.1016/S0020-7373(83)80011-X ↗
48. A protocol for identifying and sampling from proxy populations10.1111/ssqu.12519 ↗
49. Lindgaard, G., and Chattratichart, J. (May, January 28). Usability testing: What have we overlooked?. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, San Jose, CA, USA.
50. Lazar, J., Feng, J.H., and Hochheiser, H. (2017). Research Methods in Human-Computer Interaction, Morgan Kaufmann.
51. Cognitive impairment in amyotrophic lateral sclerosis10.1016/S1474-4422(07)70265-X ↗
52. Picksche Krankheit und Amyotrophische Lateralsklerose
53. Technology Usability for People Living with Dementia: Concept Analysis10.2196/51987 ↗
54. Usability evaluation of an electric-powered wheelchair driven by eye tracking10.1007/s10209-021-00809-z ↗
55. Usability studies of an egocentric vision-based robotic wheelchair
56. Use of virtual technology as an intervention for wheelchair skills training: A systematic review10.1016/j.apmr.2018.02.007 ↗
57. Cooper, N., Millela, F., Cant, I., White, M.D., and Meyer, G. (2021). Transfer of training—Virtual reality training with augmented multisensory cues improves user experience during training and task performance in the real world. PLoS ONE, 16.10.1371/journal.pone.0248225 ↗