Interacting with Smart Virtual Assistants for Individuals with Dysarthria: A Comparative Study on Usability and User Preferences
Source abstract
Abstract
This study explores the effectiveness and user experience of different interaction methods used by individuals with dysarthria when engaging with Smart Virtual Assistants (SVAs). It focuses on three primary modalities: direct speech commands through Alexa, non-verbal voice cues via the Daria system, and eye gaze control. The objective is to assess the usability, workload, and user preferences associated with each method, catering to the varying communication capabilities of individuals with dysarthria. While Alexa and Daria facilitate voice-based interactions, eye gaze control offers an alternative for those unable to use voice commands, including users with severe dysarthria. This comparative approach aims to determine how the usability of each interaction method varies, conducted with eight participants with dysarthria. The results indicated that non-verbal voice interactions, particularly with the Daria system, were favored because of their lower workload and ease of use. The eye gaze technology, while viable, presented challenges in terms of the higher workload and usability. These findings highlight the necessity of diversifying interaction methods with SVAs to accommodate the unique needs of individuals with dysarthria.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: AI in Service Interactions · Assistive Technology in Communication and Mobility · Social Robot Interaction and HRI
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
School of Computer Science and Informatics, Cardiff University, Cardiff CF24 4AG, UK
Computer Science and Engineering Department, Yanbu Industrial College, Yanbu 46411, Saudi Arabia
References (63)
Show source-provided references
1. Kepuska, V., and Bohouta, G. (2018, January 8–10). Next-generation of virtual personal assistants (microsoft cortana, apple siri, amazon alexa and google home). Proceedings of the 2018 IEEE 8th Annual Computing and Communication Workshop and Conference (CCWC), Las Vegas, NV, USA.10.1109/CCWC.2018.8301638 ↗
2. Alexa, Siri, Cortana, and more: An introduction to voice assistants10.1080/02763869.2018.1404391 ↗
3. Understanding the long-term use of smart speaker assistants10.1145/3264901 ↗
4. Moore, C. (2023, December 15). OK, Google: What Can Home Do? The Speaker’s Most Useful Skills. Available online: https://www.digitaltrends.com/home/google-home-most-useful-skills/.
5. Music, search, and IoT: How people (really) use voice assistants10.1145/3311956 ↗
6. Sciarretta, E., and Alimenti, L. (2021). Human-Computer Interaction. Theory, Methods and Tools: Thematic Area, HCI 2021, Held as Part of the 23rd HCI International Conference, HCII 2021, Virtual Event, 24–29 July 2021, Proceedings, Part I 23, Springer.
7. Investigating the accessibility of voice assistants with impaired users: Mixed methods study10.2196/18431 ↗
8. Corbett, E., and Weber, A. (2016, January 6–9). What can I say? addressing user experience challenges of a mobile voice user interface for accessibility. Proceedings of the 18th International Conference on Human-Computer Interaction with Mobile Devices and Services, Florence, Italy.10.1145/2935334.2935386 ↗
9. User personas: Smart speakers, home automation and people with disabilities
10. Takashima, Y., Takiguchi, T., and Ariki, Y. (2019, January 12–17). End-to-end dysarthric speech recognition using multiple databases. Proceedings of the ICASSP 2019–2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Brighton, UK.10.1109/ICASSP.2019.8683803 ↗
11. Pradhan, A., Mehta, K., and Findlater, L. (2018, January 21–26). “Accessibility Came by Accident” Use of Voice-Controlled Intelligent Personal Assistants by People with Disabilities. Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, Montreal, QC, Canada.10.1145/3173574.3174033 ↗
12. Masina, F., Pluchino, P., Orso, V., Ruggiero, R., Dainese, G., Mameli, I., Volpato, S., Mapelli, D., and Gamberini, L. (2021). Proceedings of the Ambient Assisted Living: Italian Forum 2019, Springer.
13. On the impact of dysarthric speech on contemporary ASR cloud platforms10.1007/s40860-019-00085-y ↗
14. Teixeira, A., Braga, D., Coelho, L., Fonseca, J., Alvarelhão, J., Martín, I., Queirós, A., Rocha, N., Calado, A., and Dias, M. (2009, January 3–5). Speech as the basic interface for assistive technology. Proceedings of the DSAI 2009, 2th International Conference on Software Development for Enhancing Accessibility and Fighting Info-Exclusion, Lisboa, Portugal.
15. Jaddoh, A., Loizides, F., Lee, J., and Rana, O. (2023). An interaction framework for designing systems for virtual home assistants and people with dysarthria. Univers. Access Inf. Soc., 1–13.10.1007/s10209-023-01037-3 ↗
16. Purposes of AAC device use for persons with ALS as reported by caregivers10.1080/07434610600650276 ↗
17. Beukelman, D.R., and Mirenda, P. (1998). Augmentative and Alternative Communication, Paul H. Brookes.
18. What adults who use AAC say about their use of mainstream mobile technologies
19. Ballati, F., Corno, F., and De Russis, L. (2018, January 22–24). Assessing virtual assistant capabilities with italian dysarthric speech. Proceedings of the 20th International ACM SIGACCESS Conference on Computers and Accessibility, Galway, Ireland.10.1145/3234695.3236354 ↗
20. Curtis, H., Neate, T., and Vazquez Gonzalez, C. (2022, January 23–26). State of the Art in AAC: A Systematic Review and Taxonomy. Proceedings of the 24th International ACM SIGACCESS Conference on Computers and Accessibility, Athens, Greece.10.1145/3517428.3544810 ↗
21. Matters, C. (2023, December 11). TYPES OF AAC. Available online: https://www.communicationmatters.org.uk/what-is-aac/types-of-aac/.
22. Bissoli, A., Lavino-Junior, D., Sime, M., Encarnação, L., and Bastos-Filho, T. (2019). A human–machine interface based on eye tracking for controlling and monitoring a smart home using the internet of things. Sensors, 19.10.3390/s19040859 ↗
23. Usability and workload of access technology for people with severe motor impairment: A comparison of brain-computer interfacing and eye tracking10.1177/1545968315575611 ↗
24. Acoustic-phonetic contrasts and intelligibility in the dysarthria associated with mixed cerebral palsy10.1044/jshr.3502.296 ↗
25. (2023, December 15). Alexa. Available online: https://www.amazon.com/b?node=21576558011.
26. (2023, December 12). Statista.com. Number of Households with Smart Home Products and Services in Use Worldwide from 2017 to 2025. Available online: https://www.statista.com/statistics/1252975/smart-home-households-worldwide/.
27. Non-verbal interaction with virtual home assistants for people with dysarthria
28. (2018). Ergonomic Requirements for Office Work with Visual Display Terminals (VDT)s-Part 11 Guidance on Usability (Standard No. ISO 9241-11:2018).
29. Ballati, F., Corno, F., and De Russis, L. (2018, January 25–28). “Hey Siri, Do You Understand Me?”: Virtual Assistants and Dysarthria. Proceedings of the International Workshop on the Reliability of Intelligent Environments (Workshops), Rome, Italy.
30. Moore, M., Venkateswara, H., and Panchanathan, S. (2018, January 2–6). Whistle-blowing asrs: Evaluating the need for more inclusive automatic speech recognition systems. Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH, Hyderabad, India.10.21437/Interspeech.2018-2391 ↗
31. Derboven, J., Huyghe, J., and De Grooff, D. (2014, January 26–30). Designing voice interaction for people with physical and speech impairments. Proceedings of the 8th Nordic Conference on Human-Computer Interaction: Fun, Fast, Foundational, Helsinki, Finland.10.1145/2639189.2639252 ↗
32. Moore, M. (2021). Multimedia for Accessible Human Computer Interfaces, Springer.
33. Corno, F., Farinetti, L., and Signorile, I. (2002, January 26–29). A cost-effective solution for eye-gaze assistive technology. Proceedings of the IEEE International Conference on Multimedia and Expo, Lausanne, Switzerland.
34. Hemmingsson, H., and Borgestig, M. (2020). Usability of eye-gaze controlled computers in Sweden: A total population survey. Int. J. Environ. Res. Public Health, 17.10.3390/ijerph17051639 ↗
35. Eye tracking communication devices in amyotrophic lateral sclerosis: Impact on disability and quality of life10.3109/21678421.2013.803576 ↗
36. Eye-gaze control technology for children, adolescents and adults with cerebral palsy with significant physical disability: Findings from a systematic review10.1080/17518423.2017.1362057 ↗
37. Understanding users and their needs10.1007/s10209-009-0148-1 ↗
38. Two case studies describing assessment and provision of eye gaze technology for people with severe physical disabilities10.1108/17549450200800013 ↗
39. Gil-Gómez, J.A., Manzano-Hernández, P., Albiol-Pérez, S., Aula-Valero, C., Gil-Gómez, H., and Lozano-Quilis, J.A. (2017). USEQ: A short questionnaire for satisfaction evaluation of virtual rehabilitation systems. Sensors, 17.10.3390/s17071589 ↗
40. Kocabalil, A.B., Laranjo, L., and Coiera, E. (2018, January 4–6). Measuring user experience in conversational interfaces: A comparison of six questionnaires. Proceedings of the 32nd International BCS Human Computer Interaction Conference, Belfast, UK.10.14236/ewic/HCI2018.21 ↗
41. Vtyurina, A., and Fourney, A. (2018, January 21–26). Exploring the role of conversational cues in guided task support with virtual assistants. Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, Montreal, QC, Canada.10.1145/3173574.3173782 ↗
42. Pyae, A., and Joelsson, T.N. (2018, January 3–6). Investigating the usability and user experiences of voice user interface: A case of Google home smart speaker. Proceedings of the 20th International Conference on Human-Computer Interaction with Mobile Devices and Services Adjunct, Barcelona, Spain.10.1145/3236112.3236130 ↗
43. Bogers, T., Al-Basri, A.A.A., Ostermann Rytlig, C., Bak Møller, M.E., Juhl Rasmussen, M., Bates Michelsen, N.K., and Gerling Jørgensen, S. (2019). Information in Contemporary Society, Proceedings of the 14th International Conference, iConference 2019, Washington, DC, USA, 31 March–3 April 2019, Springer. Proceedings 14.
44. and Lee, J.S. (2018, January 22). Speech and gestures for smart-home control and interaction for older adults. Proceedings of the 3rd International Workshop on Multimedia for Personal Health and Health Care, Seoul, Republic of Korea.
45. The relationship between system effectiveness and subjective usability scores using the System Usability Scale10.1080/10447318.2014.904177 ↗
46. Hey Google, Help Doing My Homework: Surveying Voice Interactive Systems
47. Iannizzotto, G., Bello, L.L., Nucita, A., and Grasso, G.M. (2018, January 4–6). A vision and speech enabled, customizable, virtual assistant for smart environments. Proceedings of the 2018 11th International Conference on Human System Interaction (HSI), Gdansk, Poland.10.1109/HSI.2018.8431232 ↗
48. Barricelli, B.R., Fogli, D., Iemmolo, L., and Locoro, A. (2022, January 6–10). A multi-modal approach to creating routines for smart speakers. Proceedings of the 2022 International Conference on Advanced Visual Interfaces, Frascati, Italy.10.1145/3531073.3531168 ↗
49. Frøkjær, E., Hertzum, M., and Hornbæk, K. (2000, January 1–6). Measuring usability: Are effectiveness, efficiency, and satisfaction really correlated?. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, The Hague, The Netherlands.10.1145/332040.332455 ↗
50. Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research10.1016/S0166-4115(08)62386-9 ↗
51. Kim, S., and Ko, I.Y. (May, January 29). A Conversational Approach for Modifying Service Mashups in IoT Environments. Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems, New Orleans, LA, USA.10.1145/3491102.3517655 ↗
52. Tobii (2023, December 11). Tobii Eye Tracker. Available online: https://www.tobii.com/.
53. Feit, A.M., Williams, S., Toledo, A., Paradiso, A., Kulkarni, H., Kane, S., and Morris, M.R. (2017, January 6–11). Toward everyday gaze input: Accuracy and precision of eye tracking and implications for design. Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, Denver, CO, USA.10.1145/3025453.3025599 ↗
54. Ability-based design: Concept, principles and examples10.1145/1952383.1952384 ↗
55. Patel, R., Dromey, C., and Kunov, H. (1998). Control of Prosodic Parameters by an Individual with Severe Dysarthria, University of Toronto. Technical Report.
56. Dysarthric speakers’ intelligibility and speech characteristics in relation to computer speech recognition10.1080/07434619512331277289 ↗
57. Perceived usefulness, perceived ease of use, and user acceptance of information technology10.2307/249008 ↗
58. Unified theory of acceptance and use of technology: A synthesis and the road ahead
59. Munteanu, C., Jones, M., Oviatt, S., Brewster, S., Penn, G., Whittaker, S., Rajput, N., and Nanavati, A. (2013). CHI’13 Extended Abstracts on Human Factors in Computing Systems, Association for Computing Machinery.
60. Gaps between continuous measurement methods: A longitudinal study of perceived usability10.1093/iwc/iwab023 ↗
61. “Functioning better is doing better”: Older adults’ priorities for the evaluation of assistive technology10.1080/10400435.2022.2113180 ↗
62. Wang, K., Wang, S., and Ji, Q. (2016, January 14–17). Deep eye fixation map learning for calibration-free eye gaze tracking. Proceedings of the Ninth Biennial ACM Symposium on Eye Tracking Research & Applications, Charleston, SC, USA.10.1145/2857491.2857515 ↗
63. Conceptualization and measurement of assistive technology usability10.1080/17483100701343665 ↗