journal article·2024-07-27Open access

Testing the Quality of the Mobile Application Interface Using Various Methods—A Case Study of the T1DCoach Application

Marek Milosz, Małgorzata Plechawska-Wójcik, Mariusz Dzieńkowski

Applied Sciences· Volume 14· Issue 15

Abstract

The T1DCoach mobile application is designed to educate patients—children with type 1 diabetes (T1D) and their caregivers and diabetes educators. The idea behind the mobile application is that its users perform actions that the patient needs to perform in real life. These include measuring blood glucose levels, operating the insulin pump, meal calculation, bolus administration, etc. These in-application activities are performed on the patient’s digital twin. To increase user engagement, gamification elements have been implemented in the application. An important element of the T1DCoach mobile application is its interface, which should be adapted to very different groups of users: children, their caregivers and educators. In addition to presenting the T1DCoach application, the paper presents the stage examining the quality of the interface using three research groups: children, their caregivers and educators. The research was conducted using the scenario method, using eye-tracking, recording activities and thinking aloud. After the application testing sessions, surveys were carried out using the System Usability Scale method and focus group interviews were conducted. The research results are presented in the article along with the most important recommendations for improving the application interface.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: Mobile Health and mHealth Applications · E-Learning and Knowledge Management · Technology Use by Older Adults

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

Department of Computer Science, Lublin University of Technology, 36B Nadbystrzycka Str., 20-618 Lublin, Poland

Funding

Polish National Center for Research and Development

References (50)

Show source-provided references
1. Building patient relationships: A smartphone application supporting communication between teenagers with asthma and the RN care coordinator. CIN: Computers, Informatics
2. Supporting patient-clinician interaction in chronic HIV care: Design and development of a patient-reported outcomes software application10.2196/27861 ↗
3. Lambrecht, A., Vuillerme, N., Raab, C., Simon, D., Messner, E.M., Hagen, M., Bayat, S., Kleyer, A., Aubourg, T., and Schett, G. (2021). Quality of a supporting mobile app for rheumatic patients: Patient-based assessment using the user version of the mobile application scale (uMARS). Front. Med., 8.10.3389/fmed.2021.715345 ↗
4. Advancement Level of Mobile Applications Intended for Type 1 Diabetes Therapy Supporting10.12775/JEHS.2020.10.09.085 ↗
5. A robot goes to rehab: A novel gamified system for long-term stroke rehabilitation using a socially assistive robot—Methodology and usability testing10.1186/s12984-021-00915-2 ↗
6. Facilitating and automating usability testing of educational technologies10.1002/cae.22725 ↗
7. Als, B.S., Jensen, J.J., and Skov, M.B. (2005, January 8–10). Comparison of think-aloud and constructive interaction in usability testing with children. Proceedings of the 2005 Conference on Interaction Design and Children, Boulder, CO, USA.10.1145/1109540.1109542 ↗
8. Usability guidelines for usable user interface10.15623/ijret.2014.0309011 ↗
9. A structured methodology for comparative evaluation of user interface designs using usability criteria and measures10.1016/S0169-8141(97)00059-0 ↗
10. Usability testing
11. How do consumers search for and appraise health information on the world wide web? Qualitative study using focus groups, usability tests, and in-depth interviews10.1136/bmj.324.7337.573 ↗
12. Usability inspection: Novice crowd inspectors versus expert10.1016/j.jss.2021.111122 ↗
13. Measuring subjectively experienced time in usability and user experience testing scenarios10.1016/j.ijhcs.2020.102399 ↗
14. Usability of mobile applications: A systematic literature study10.1109/ACCESS.2020.2981892 ↗
15. QUEST e-learning portal: Applying heuristic evaluation, usability testing and eye tracking10.1007/s10209-020-00774-z ↗
16. Exploring Two Methods of Usability Testing: System Usability Scale And Retrospective Think-Aloud
17. Kamińska, D., Zwoliński, G., and Laska-Leśniewicz, A. (2022). Usability testing of virtual reality applications—The pilot study. Sensors, 22.10.3390/s22041342 ↗
18. Usability testing of mobile learning applications: A systematic mapping study10.1108/IJILT-03-2023-0029 ↗
19. Perceived usability evaluation of educational technology using the System Usability Scale (SUS): A systematic review10.1080/15391523.2020.1867938 ↗
20. Systematic review of applied usability metrics within usability evaluation methods for hospital electronic healthcare record systems: Metrics and Evaluation Methods for eHealth Systems10.1111/jep.13582 ↗
21. Borys, M., and Milosz, M. (2018, January 4–6). Mobile application usability testing in quasi-real conditions—The synergy of using different methods. 11th International Conference on Human System Interaction, HSI2018, Gdansk, Poland.10.1109/HSI.2018.8431342 ↗
22. Akmal Muhamat, N., Hasan, R., Saddki, N., Mohd Arshad, M.R., and Ahmad, M. (2021). Development and usability testing of mobile application on diet and oral health. PLoS ONE, 16.10.1371/journal.pone.0257035 ↗
23. User Experience Design on Visualization of Mobile-Based Land Monitoring System Using a User-Centered Design Approach10.3991/ijim.v16i03.28499 ↗
24. Nugroho, A., Santosa, P.I., and Hartanto, R. (2022, January 27–28). Usability evaluation methods of mobile applications: A systematic literature review. Proceedings of the 2022 International Symposium on Information Technology and Digital Innovation (ISITDI), Padang, Indonesia.10.1109/ISITDI55734.2022.9944401 ↗
25. Comparison of heuristic and cognitive walkthrough usability evaluation methods for evaluating health information systems10.1093/jamia/ocw100 ↗
26. Usability evaluation of a nursing information system by applying cognitive walkthrough method10.1016/j.ijmedinf.2021.104459 ↗
27. Farzandipour, M., Nabovati, E., and Sadeqi Jabali, M. (2022). Comparison of usability evaluation methods for a health information system: Heuristic evaluation versus cognitive walkthrough method. BMC Med. Inform. Decis. Mak., 22.10.1186/s12911-022-01905-7 ↗
28. Milosz, M., Plechawska-Wójcik, M., Borys, M., and Laskowski, M. (2013, January 6–8). Quality improvement of ERP system GUI using expert method: A case study. Proceedings of the 6th International Conference on Human System Interaction, HSI2013, Sopot, Poland.10.1109/HSI.2013.6577815 ↗
29. System usability scale benchmarking for digital health apps: Meta-analysis10.2196/37290 ↗
30. Novák, J.Š., Masner, J., Benda, P., Šimek, P., and Merunka, V. (2023). Eye tracking, usability, and user experience: A systematic review. Int. J. Hum. Comput. Interact., 1–17.10.1080/10447318.2023.2221600 ↗
31. Julian, I., Murad, D.F., and Riva’i, R.Y. (2021, January 25–26). Combining UEQ and Eye-Tracking Method as Usability Evaluation for Mobile Apps. Proceedings of the 2021 3rd International Conference on Cybernetics and Intelligent System (ICORIS), Makassar, Indonesia.
32. Țichindelean, M., Țichindelean, M.T., Cetină, I., and Orzan, G. (2021). A comparative eye tracking study of usability—Towards sustainable web design. Sustainability, 13.10.3390/su131810415 ↗
33. Usability evaluation using eye-tracking on E-commerce and education domains10.48185/jitc.v1i1.43 ↗
34. Key methodological considerations for usability testing of electronic patient-reported outcome (ePRO) systems10.1007/s11136-019-02329-z ↗
35. Milosz, M., and Chmielewska, M. (2020, January 6–8). Usability Testing of e-Government Online Services Using Different Methods—A Case Study. Proceedings of the 13th International Conference on Human System Interaction, HSI2020, Tokyo, Japan.10.1109/HSI49210.2020.9142628 ↗
36. Virtual therapy using Type 1 Diabetes Direct Simulator10.1088/1742-6596/1736/1/012031 ↗
37. Ten factors to consider when developing usability scenarios and tasks for health information technology10.1016/j.jbi.2018.01.001 ↗
38. Lowry, S.Z., Quinn, M.T., Ramaiah, M., Schumacher, R.M., Patterson, E.S., North, R., Zhang, J., Gibbons, M.C., and Abbott, P. (2012). (NISTIR 7804) Technical Evaluation National Institute of Standards and Technology, Testing and Validation of the Usability of Electronic Health Records. NIST Interagency/Internal Report (NISTIR) 7804, National Institute of Standards and Technology.10.6028/NIST.IR.7804 ↗
39. Usability standards meet scenario-based design: Challenges and opportunities10.1016/j.jbi.2014.11.008 ↗
40. Patterns in Eyetracking Scanpaths and the Affecting Factors
41. Knowledge-Based Patterns of Remembering: Eye Movement Scanpaths Reflect Domain Experience10.1007/978-3-540-89350-9_10 ↗
42. Ehmke, C., and Wilson, S. (2007, January 3–7). Identifying Web Usability Problems from Eye-Tracking Data. Proceedings of the 21st British HCI Group Annual Conference on People and Computers: HCI...But Not As We Know It, Swinton, UK.10.14236/ewic/HCI2007.12 ↗
43. Salvendy, G. (2006). Usability Testing. Handbook of Human Factors and Ergonomics, John Wiley & Sons, Inc.. [3rd ed.].10.1002/0470048204 ↗
44. Tullis, T.S., and Stetson, J.N. (2004, January 7–11). A comparison of questionnaires for assessing website usability. Proceedings of the UPA: Usability Professionals’ Association Conference, Minneapolis, MN, USA.
45. SUS: A Retrospective
46. Jordan, P.W., Thomas, B., Weerdmeester, B.A., and McClelland, A.L. (1996). SUS: A “quick and dirty” usability scale. Usability Evaluation in Industry, Taylor and Francis.10.1201/9781498710411 ↗
47. Morgan, D.L. (1997). Focus Groups as Qualitative Research, Sage. [2nd ed.].10.4135/9781412984287 ↗
48. Langford, J., and McDonaugh, D. (2003). Focus Groups. Supporting Effective Product Development, Taylor and Francis.10.1201/9780203302743 ↗
49. How to specify the participant group size for usability studies: A practitioner’s guide
50. Adjustment of Insulin Pump Settings in Type 1 Diabetes Management: Advisor Pro Device Compared to Physicians’ Recommendations10.1177/1932296820965561 ↗