Emotion Recognition in Usability Testing: A Framework for Improving Web Application UI Design
Source abstract
Abstract
Traditional usability testing methods often lack the ability to fully capture different aspects of the user experience (UX). The focus of this research work is to propose a framework and develop its comprehensive prototype to improve usability testing and UX analysis by integrating session recording, interaction logging, and emotion recognition methods. A trained emotion recognition model based on MobileNetV2 architecture in conjunction with Hotjar and Google Analytics is proposed to add more context to the user experience during usability testing. The results obtained during the testing of a developed framework prototype showed that UI testing based on UX principles and integration of emotion recognition can be a powerful tool for improving the UI of web applications. It is recommended to improve UI testing processes by incorporating these aspects and data analysis methods. This would provide a more complete and more objective picture of the usability of the interface.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Emotion and Mood Recognition · Color perception and design · Gaze Tracking and Assistive Technology
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Marine Research Institute, Klaipeda University, H. Manto Str. 84, LT-92294 Klaipeda, Lithuania
Funding
Lithuanian Research Council and the Ministry of Education, Science, and Sports of the Republic of Lithuania
References (30)
Show source-provided references
1. (2019). Ergonomics of Human-System Interaction, Part 210: Human-Centred Design for Interactive Systems (Standard No. ISO 9241-210).
2. Attitudes towards User Experience (UX) Measurement10.1016/j.ijhcs.2013.09.006 ↗
3. Lessons Learned to Improve the UX Practices in Agile Projects Involving Data Science and Process Automation10.1016/j.infsof.2022.107106 ↗
4. A User Interface (UI) and User eXperience (UX) Evaluation Framework for Cyberlearning Environments in Computer Science and Software Engineering Education10.1016/j.heliyon.2020.e03917 ↗
5. Evaluating on User Experience and User Interface (UX/UI) of EnerTrApp a Mobile Web Energy Monitoring System10.1016/j.procs.2019.11.236 ↗
6. Deep Learning for Emotion Driven User Experiences10.1016/j.patrec.2021.09.004 ↗
7. Are Three Methods Better than One? A Comparative Assessment of Usability Evaluation Methods in an EHR10.1016/j.ijmedinf.2014.01.010 ↗
8. Marques, L., Matsubara, P.G., Nakamura, W.T., Ferreira, B.M., Wiese, I.S., Gadelha, B.F., Zaina, L.M., Redmiles, D., and Conte, T.U. (2021). Understanding UX Better: A New Technique to Go beyond Emotion Assessment. Sensors, 21.10.3390/s21217183 ↗
9. Enhancing Player Experience in Game With Affective Computing10.1016/j.procs.2021.01.066 ↗
10. Liu, J., Tong, J., Han, J., Yang, F., and Chen, S. (2013, January 23–25). Affective Computing Applications in Distance Education. Proceedings of the 2013 the International Conference on Education Technology and Information System (ICETIS 2013), San Francisco, CA, USA.10.2991/icetis-13.2013.212 ↗
11. Yao, K., Huang, W.-T., Chen, T.-Y., Wu, C.-C., and Ho, W.-S. (2022). Establishing an Intelligent Emotion Analysis System for Long-Term Care Application Based on LabVIEW. Sustainability, 14.10.3390/su14148932 ↗
12. Design and Analysis of an Automatic UI Adaptation Framework from Multimodal Emotion Recognition Using an RGB-D Sensor10.1016/j.procs.2020.03.011 ↗
13. A Model of the Perception of Facial Expressions of Emotion by Humans: Research Overview and Perspectives
14. The Role of Emotional Valence for the Processing of Facial and Verbal Stimuli-Positivity or Negativity Bias?10.3389/fpsyg.2019.01654 ↗
15. Read My Face: Automatic Facial Coding Versus Psychophysiological Indicators of Emotional Valence and Arousal10.3389/fpsyg.2020.01388 ↗
16. The Nature of Emotions: Human Emotions Have Deep Evolutionary Roots, a Fact That May Explain Their Complexity and Provide Tools for Clinical Practice10.1511/2001.28.344 ↗
17. An Argument for Basic Emotions10.1080/02699939208411068 ↗
18. Tao, J., and Tan, T. (2005, January 22). Affective Computing: A Review. Proceedings of the International Conference on Affective Computing and Intelligent Interaction, Beijing, China.10.1007/11573548 ↗
19. Yao, J., Xiao, Y., You, P., and Sun, G. (2022). Proceedings of the International Conference on Image, Vision and Intelligent Systems (ICIVIS 2021), Springer Nature.10.1007/978-981-16-6963-7 ↗
20. Mood Perception Model for Social Robot Based on Facial and Bodily Expression Using a Hidden Markov Model10.20965/jrm.2019.p0629 ↗
21. ML-DCNNet: Multi-Level Deep Convolutional Neural Network for Facial Expression Recognition and Intensity Estimation10.1007/s13369-020-04811-0 ↗
22. Natural Facial Expression Recognition Using Differential-AAM and Manifold Learning10.1016/j.patcog.2008.10.010 ↗
23. Kriging Predictor for Facial Emotion Recognition Using Numerical Proximities of Human Emotions
24. Ullah, M., Li, X., Hassan, M.A., Ullah, F., Muhammad, Y., Granelli, F., Vilcekova, L., and Sadad, T. (2023). An Intelligent Multi-Floor Navigational System Based on Speech, Facial Recognition and Voice Broadcasting Using Internet of Things. Sensors, 23.10.3390/s23010275 ↗
25. Affective State Recognition from Hand Gestures and Facial Expressions Using Grassmann Manifolds10.1007/s11042-020-10341-6 ↗
26. Emotion Detection from Text and Speech: A Survey10.1007/s13278-018-0505-2 ↗
27. Deep Convolutional Neural Networks: Structure, Feature Extraction and Training
28. Convolutional Neural Networks: An Overview and Application in Radiology10.1007/s13244-018-0639-9 ↗
29. Dong, K., Zhou, C., Yihan, R., and Li, Y. (2020, January 1). MobileNetV2 Model for Image Classification. Proceedings of the 2020 2nd International Conference on Information Technology and Computer Application (ITCA), Guangzhou, China.10.1109/ITCA52113.2020.00106 ↗
30. Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., and Chen, L.-C. (2018, January 18–23). MobileNetV2: Inverted Residuals and Linear Bottlenecks. Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00474 ↗