journal article·2025-08-27Open access

Current Status and Challenges in Usability Evaluation Research on Human–Machine Interaction in Intelligent Vehicles: A Systematic Review

Datao Zhou, Xiaofang Yuan, Yidi Sun, Yu Wu

Applied Sciences· Volume 15· Issue 17

Abstract

Usability evaluation is a crucial stage in optimizing human–machine interaction design for intelligent vehicles. The academic community has accumulated substantial research findings on this topic. However, the global landscape, core focus areas, and potential challenges of these findings remain unclear and require systematic review and analysis. This is essential for understanding progress in the field and clarifying future research and technological development directions. To address this research need, we followed the PRISMA guidelines and conducted a systematic review of 158 applied research cases in the field of usability evaluation for intelligent vehicle human–machine interaction. We focused on analyzing the automation levels, task scenarios, and interaction modalities examined in these cases. Based on this analysis, we examined the current development status and potential challenges and identified future development recommendations for this field. Our research reveals that current usability evaluation studies of intelligent vehicle human–machine interaction exhibit a contradictory pattern of “coexisting development and limitations.” On the one hand, the research scope and depth continue to expand with technological advances. On the other hand, related research topics show obvious lag, with particularly insufficient research on non-driving tasks and multimodal interaction topics. The potential challenges arising from this research status seriously constrain industry development and require practitioners to actively address them. In this regard, our research findings provide a valuable reference for practitioners in understanding the current research progress and planning future development directions.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: Human-Automation Interaction and Safety · Safety Warnings and Signage · Ergonomics and Human Factors

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

School of Art and Design, Wuhan University of Technology, Wuhan 430070, China

College of Design and Innovation, Tongji University, Shanghai 200092, China

Hubei Provincial Engineering Research Centre for Intelligent Industrial Design of Advanced Equipment, Wuhan 430070, China

Funding

Key Research and Development Program of Hubei Province · Hubei Province Foreign Talent and Intelligence Introduction Project

References (100+)

Show source-provided references
1. Intelligent Cockpit for Intelligent Connected Vehicles: Definition, Taxonomy, Technology and Evaluation10.1109/TIV.2023.3339798 ↗
2. Milestones in Autonomous Driving and Intelligent Vehicles—Part I: Control, Computing System Design, Communication, HD Map, Testing, and Human Behaviors10.1109/TSMC.2023.3276218 ↗
3. The development and validation of the perceived safety of intelligent connected vehicles scale10.1016/j.aap.2021.106092 ↗
4. Tufano, F., Bahadure, S.W., Tufo, M., Novella, L., Fiengo, G., and Santini, S. (2023). An Optimization Framework for Information Management in Adaptive Automotive Human–Machine Interfaces. Appl. Sci., 13.10.3390/app131910687 ↗
5. Li, J., Liu, J., Wang, X., and Liu, L. (2024). The Impact of Transparency on Driver Trust and Reliance in Highly Automated Driving: Presenting Appropriate Transparency in Automotive HMI. Appl. Sci., 14.10.3390/app14083203 ↗
6. Bellani, P., Picardi, A., Caruso, F., Gaetani, F., Brevi, F., Arquilla, V., and Caruso, G. (2023). Enhancing User Engagement in Shared Autonomous Vehicles: An Innovative Gesture-Based Windshield Interaction System. Appl. Sci., 13.10.3390/app13179901 ↗
7. Kim, S., Oh, J., Seong, M., Jeon, E., Moon, Y.-K., and Kim, S. (2023). Assessing the Impact of AR HUDs and Risk Level on User Experience in Self-Driving Cars: Results from a Realistic Driving Simulation. Appl. Sci., 13.10.3390/app13084952 ↗
8. Human–machine interaction in intelligent and connected vehicles: A review of status quo, issues, and opportunities10.1109/TITS.2021.3127217 ↗
9. Automated driving: Safety blind spots10.1016/j.ssci.2017.07.018 ↗
10. Challenges to Human Drivers in Increasingly Automated Vehicles10.1177/0018720819900402 ↗
11. Yan, M., Rampino, L., and Caruso, G. (2025). Comparing User Acceptance in Human–Machine Interfaces Assessments of Shared Autonomous Vehicles: A Standardized Test Procedure. Appl. Sci., 15.10.3390/app15010045 ↗
12. Human–Vehicle Cooperation in Automated Driving: A Multidisciplinary Review and Appraisal10.1080/10447318.2018.1561792 ↗
13. Effects of Gesture-Based Interaction on Driving Behavior: A Driving Simulator Study Using the Projection-Based Vehicle-in-the-Loop10.1177/0018720820943284 ↗
14. Behavioral Changes to Repeated Takeovers in Highly Automated Driving: Effects of the Takeover-Request Design and the Nondriving-Related Task Modality10.1177/0018720818814963 ↗
15. Agent Transparency, Situation Awareness, Mental Workload, and Operator Performance: A Systematic Literature Review10.1177/00187208221077804 ↗
16. Automotive HMI design and participatory user involvement: Review and perspectives10.1080/00140139.2016.1188218 ↗
17. Harms, I.M., Auerbach, D.A.M., Papadimitriou, E., and Hagenzieker, M.P. (2025). Frequently Used Vehicle Controls While Driving: A Real-World Driving Study Assessing Internal Human–Machine Interface Task Frequencies and Influencing Factors. Appl. Sci., 15.10.3390/app15105230 ↗
18. Intelligence testing for autonomous vehicles: A new approach10.1109/TIV.2016.2608003 ↗
19. Use Cases for Assessing, Testing, and Validating the Human Machine Interface of Automated Driving Systems10.1177/1541931218621426 ↗
20. Specification-based autonomous driving system testing
21. Learning to use automation: Behavioral changes in interaction with automated driving systems10.1016/j.trf.2019.02.013 ↗
22. Forster, Y., Hergeth, S., Naujoks, F., Krems, J., and Keinath, A. (2019). User Education in Automated Driving: Owner’s Manual and Interactive Tutorial Support Mental Model Formation and Human-Automation Interaction. Information, 10.10.3390/info10040143 ↗
23. A Survey on Automated Driving System Testing: Landscapes and Trends10.1145/3579642 ↗
24. Forster, Y., Frison, A.-K., Wintersberger, P., Geisel, V., Hergeth, S., and Riener, A. (2019, January 21–25). Where we come from and where we are going: A review of automated driving studies. Proceedings of the AutomotiveUI ‘19: Proceedings of the 11th International Conference on Automotive User Interfaces and Interactive Vehicular Applications: Adjunct Proceedings, Utrecht, The Netherlands.10.1145/3349263.3351341 ↗
25. Su, Y., Tan, Z., and Dai, N. (2021, January 25–29). Changes in Usability Evaluation of Human-Machine Interfaces from the Perspective of Automated Vehicles. Proceedings of the AHFE 2021 Virtual Conferences on Usability and User Experience, Human Factors and Wearable Technologies, Human Factors in Virtual Environments and Game Design, and Human Factors and Assistive Technology, Virtual.10.1007/978-3-030-80091-8_105 ↗
26. Albers, D., Radlmayr, J., Loew, A., Hergeth, S., Naujoks, F., Keinath, A., and Bengler, K. (2020). Usability Evaluation—Advances in Experimental Design in the Context of Automated Driving Human–Machine Interfaces. Information, 11.10.3390/info11050240 ↗
27. Scientific procedures and rationales for systematic literature reviews (SPAR-4-SLR)10.1111/ijcs.12695 ↗
28. The PRISMA 2020 statement: An updated guideline for reporting systematic reviews10.1136/bmj.n71 ↗
29. PRISMA 2020 explanation and elaboration: Updated guidance and exemplars for reporting systematic reviews10.1136/bmj.n160 ↗
30. Boboc, R.G., Gîrbacia, F., and Butilă, E.V. (2020). The Application of Augmented Reality in the Automotive Industry: A Systematic Literature Review. Appl. Sci., 10.10.3390/app10124259 ↗
31. Alanazi, F. (2023). A Systematic Literature Review of Autonomous and Connected Vehicles in Traffic Management. Appl. Sci., 13.10.3390/app13031789 ↗
32. (2021). Taxonomy and Definitions for Terms Related to Driving Automation Systems for On-Road Motor Vehicles (Standard No. ISO/SAE PAS 22736:2021).
33. (2021). Taxonomy and Definitions for Terms Related to Driving Automation Systems for On-Road Motor Vehicles (Standard No. J3016_202104).
34. (2021). Taxonomy of Driving Automation for Vehicles (Standard No. GB/T 40429—2021). (In Chinese).
35. Llaneras, R., Glaser, Y., Green, C., August, M., and Landry, S. (2025). Operational Evaluation of a Pre-Production L2 System HMI, SAE International.10.4271/2025-01-8050 ↗
36. Assessing drivers’ response during automated driver support system failures with non-driving tasks10.1016/j.jsr.2017.02.009 ↗
37. User-centred design evaluation of symbols for adaptive cruise control (ACC) and lane-keeping assistance (LKA)10.1007/s10111-021-00673-0 ↗
38. Roberts, S., Ebadi, Y., Talreja, N., Michael A Knodler, J., and Fisher, D.L. (2022, January 17–20). Designing and Evaluating an Informative Interface for Transfer of Control in a Level 2 Automated Driving System. Proceedings of the 14th International Conference on Automotive User Interfaces and Interactive Vehicular Applications, Seoul, Republic of Korea.10.1145/3543174.3546839 ↗
39. Sánchez–Mateo, S., Pérez–Moreno, E., and Jiménez, F. (2020). Driver Monitoring for a Driver-Centered Design and Assessment of a Merging Assistance System Based on V2V Communications. Sensors, 20.10.3390/s20195582 ↗
40. Earcons to reduce mode confusions in partially automated vehicles: Development and application of an evaluation method10.1016/j.ijhcs.2023.103044 ↗
41. Awakening the Disengaged: Can Driving-Related Prompts Engage Drivers in Partial Automation?10.1177/00187208251314248 ↗
42. Marcano, M., Tango, F., Sarabia, J., Chiesa, S., Pérez, J., and Díaz, S. (2022). Can Shared Control Improve Overtaking Performance? Combining Human and Automation Strengths for a Safer Maneuver. Sensors, 22.10.3390/s22239093 ↗
43. Driver assist system for human–machine interaction10.1007/s10111-017-0439-x ↗
44. Liu, Y., Zhang, J., Li, Y., Hansen, P., and Wang, J. (2021, January 24–29). Human-Computer Collaborative Interaction Design of Intelligent Vehicle—A Case Study of HMI of Adaptive Cruise Control. Proceedings of the Third International Conference, MobiTAS 2021, Held as Part of the 23rd HCI International Conference, HCII 2021, Virtual Event.10.1007/978-3-030-78358-7_21 ↗
45. Classification of Human-Vehicle Interaction: User Perspectives on Design10.2224/sbp.6242 ↗
46. (2003). Road Vehicles—Ergonomic Aspects of Transport Information and Control Systems—Procedure for Assessing Suitability for Use While Driving (Standard No. ISO 17287: 2003).
47. Pfleging, B., and Schmidt, A. (2015, January 18–23). (Non-) Driving-Related Activities in the Car: Defining Driver Activities for Manual and Automated Driving. Proceedings of the Workshop on Experiencing Autonomous Vehicles: Crossing the Boundaries between a Drive and a Ride at CHI’15, Seoul, Republic of Korea.
48. Intelligent in-vehicle interaction technologies10.1002/aisy.202100122 ↗
49. (2016). Road Vehicles—Transport Information and Control Systems—Detection-Response Task (DRT) for Assessing Attentional Effects of Cognitive Load in Driving (Standard No. ISO 17488: 2016).
50. (2020). Road Vehicles—Human Performance and State in the Context of Automated Driving—Part 1: Common Underlying Concepts (Standard No. ISO/TR 21959-1: 2020).
51. Safety evaluation and prediction of takeover performance in automated driving considering drivers’ cognitive load: A driving simulator study10.1016/j.trf.2024.03.021 ↗
52. Scharfe, M.S.L., Zeeb, K., and Russwinkel, N. (2020). The Impact of Situational Complexity and Familiarity on Takeover Quality in Uncritical Highly Automated Driving Scenarios. Information, 11.10.3390/info11020115 ↗
53. Johansson, M., Mullaart Söderholm, M., Novakazi, F., and Rydström, A. (2021). The Decline of User Experience in Transition from Automated Driving to Manual Driving. Information, 12.10.3390/info12030126 ↗
54. Du, H., Tao, S., Feng, X., Ma, J., and Wang, H. (2024, January 15). From Passive to Active: Towards Conversational In-Vehicle Navigation Through Large Language Models. Proceedings of the Design, User Experience, and Usability, HCII 2024, Cham, Switzerland.10.1007/978-3-031-61353-1_11 ↗
55. Is control necessary for drivers? Exploring the influence of human–machine collaboration modes on driving behavior and subjective perception under different hazard visibility scenarios10.1016/j.aap.2025.108067 ↗
56. Comparative safety analysis of take-over control mechanisms of conditionally automated vehicles10.1016/j.aap.2025.108068 ↗
57. Takeover Request Design in Automated Driving: A Systematic Review10.1177/1071181321651296 ↗
58. Human machine interface design for continuous support of mode awareness during automated driving: An online simulation10.1016/j.trf.2022.03.020 ↗
59. The influence of system transparency on trust: Evaluating interfaces in a highly automated vehicle10.1016/j.trf.2020.06.001 ↗
60. Towards Implicit Interaction in Highly Automated Vehicles-A Systematic Literature Review10.1145/3546726 ↗
61. A Refined Non-Driving Activity Classification Using a Two-Stream Convolutional Neural Network10.1109/JSEN.2020.3005810 ↗
62. Multitasking While Driving: How Drivers Self-Regulate Their Interaction with In-Vehicle Touchscreens in Automated Driving10.1080/10447318.2023.2215634 ↗
63. A comfort evaluation method based on an intelligent car cockpit10.1002/hfm.20973 ↗
64. Effects of Quantity and Size of Buttons of In-Vehicle Touch Screen on Drivers’ Eye Glance Behavior10.1080/10447318.2017.1415688 ↗
65. Effect of Touch Button Interface on In-Vehicle Information Systems Usability10.1080/10447318.2021.1886484 ↗
66. The Effect of Touchscreen Layout Division on the Usability and Driving Safety of In-Vehicle Information Systems
67. Chinese handwriting while driving: Effects of handwritten box size on in-vehicle information systems usability and driver distraction10.1080/15389588.2022.2126936 ↗
68. Exploring relationships between design features and system usability of intelligent car human–machine interface10.1016/j.robot.2021.103829 ↗
69. Cao, J., Hsuan-Lin, C., Yan-Lin, C., Chih-Hsing, C., Ying-Yin, H., and Lee, Y.-J. (2025). Enhancing in-car interface efficiency: The influence of menu configuration on cognitive load and visuospatial memory. Ergonomics, 1–17.10.1080/00140139.2025.2470951 ↗
70. Shao, J., Yang, Z., Li, Y., Li, X., Huang, Y., and Deng, J. (2023, January 23–28). Research on Chinese Font Size of Automobile Central Control Interface Driving. Proceedings of the HCI in Mobility, Transport, and Automotive Systems: 5th International Conference, MobiTAS 2023, Held as Part of the 25th HCI International Conference, HCII 2023, Copenhagen, Denmark.
71. Graefe, J., Rittger, L., Carollo, G., Engelhardt, D., and Bengler, K. (2023, January 23–28). Evaluating the Potential of Interactivity in Explanations for User-Adaptive In-Vehicle Systems—Insights from a Real-World Driving Study. Proceedings of the HCI International 2023—Late Breaking Papers: 25th International Conference on Human-Computer Interaction, HCII 2023, Copenhagen, Denmark.10.1007/978-3-031-48047-8_19 ↗
72. Lin, M.-C., Lin, Y.-H., Lin, C.-C., and Lin, J.-Y. (2014, January 22–27). A Study on the Interface Design of a Functional Menu and Icons for In-Vehicle Navigation Systems. Proceedings of the 16th International Conference, HCI International 2014, Heraklion, Crete, Greece.10.1007/978-3-319-07863-2_26 ↗
73. How do drivers interact with navigation systems in real life conditions?: Results of a field-operational-test on navigation systems10.1016/j.trf.2014.04.011 ↗
74. The use of navigation systems in naturalistic driving10.1080/15389588.2015.1077384 ↗
75. The impact of navigation system display size and environmental illumination on young driver mental workload10.1016/j.trf.2020.08.027 ↗
76. A usability evaluation on the XVC framework for in-vehicle user interfaces10.1587/transinf.E93.D.3321 ↗
77. Driver–Vehicle–Environment monitoring for on-board driver support systems: Lessons learned from design and implementation10.1016/j.apergo.2009.03.002 ↗
78. Influence of prompt timing and messages of an audio navigation system on driver behavior on an urban expressway with five exits10.1016/j.aap.2021.106155 ↗
79. Kim, J.C., Laine, T.H., and Åhlund, C. (2021). Multimodal Interaction Systems Based on Internet of Things and Augmented Reality: A Systematic Literature Review. Appl. Sci., 11.10.3390/app11041738 ↗
80. Multimodal interaction: A review10.1016/j.patrec.2013.07.003 ↗
81. (2017). Ergonomics of Human-System Interaction—Part 112: Principles for the Presentation of Information (Standard No. IISO 9241-112:2017).
82. Qualitative Research of the Multimodal In-Vehicle Interaction Systems Latency Perception10.1145/3701204 ↗
83. The Application Value of Human-Vehicle Interaction Theory in Intelligent Cockpit Design10.54097/0edtqn79 ↗
84. Multimodal interaction and IoT applications10.1007/s11042-022-14278-w ↗
85. Manawadu, U.E., Kamezaki, M., Ishikawa, M., Kawano, T., and Sugano, S. (2017, January 11–14). A multimodal human-machine interface enabling situation-adaptive control inputs for highly automated vehicles. Proceedings of the 2017 IEEE Intelligent Vehicles Symposium (IV), Los Angeles, CA, USA.10.1109/IVS.2017.7995875 ↗
86. Impact of non-driving related task types, request modalities, and automation on driver takeover: A meta-analysis10.1016/j.ssci.2024.106704 ↗
87. Hong, J., Kim, S., Jeon, G., Boo, S., Jo, J., Park, J., Jung, S., Park, J., and Lee, S. (2022, January 17–20). A Preliminary Study of Multimodal Feedback Interfaces in Takeover Transition of Semi-Autonomous Vehicles. Proceedings of the Adjunct Proceedings of the 14th International Conference on Automotive User Interfaces and Interactive Vehicular Applications, Seoul, Republic of Korea.10.1145/3544999.3554783 ↗
88. A user study of directional tactile and auditory user interfaces for take-over requests in conditionally automated vehicles10.1016/j.aap.2022.106766 ↗
89. Effect of multimodal takeover request issued through A-pillar LED light, earcon, speech message, and haptic seat in conditionally automated driving10.1016/j.trf.2022.07.012 ↗
90. Take-over requests in highly automated driving: A crowdsourcing survey on auditory, vibrotactile, and visual displays10.1016/j.trf.2018.04.001 ↗
91. Effects of Head-Up Display Information Layout Design on Driver Performance: Driving Simulator Studies10.1080/10447318.2024.2414487 ↗
92. Speech Auditory Alerts Promote Memory for Alerted Events in a Video-Simulated Self-Driving Car Ride10.1177/0018720816629279 ↗
93. Haptic Seat Interfaces for Driver Information and Warning Systems10.1080/10447318.2011.555321 ↗
94. The Effects of Vibration Patterns of Take-Over Request and Non-Driving Tasks on Taking-Over Control of Automated Vehicles10.1080/10447318.2017.1404778 ↗
95. García-Díaz, J.M., García-Ruiz, M.A., Aquino-Santos, R., and Edwards-Block, A. (2013, January 2–6). Evaluation of a Driving Simulator with a Visual and Auditory Interface. Proceedings of the 6th Latin American Conference, CLIHC 2013, Carrillo, Costa Rica.
96. Löcken, A., Frison, A.-K., Fahn, V., Kreppold, D., Götz, M., and Riener, A. (2020, January 5–8). Increasing User Experience and Trust in Automated Vehicles via an Ambient Light Display. Proceedings of the 22nd International Conference on Human-Computer Interaction with Mobile Devices and Services, Oldenburg, Germany.10.1145/3379503.3403567 ↗
97. Peintner, J., Manger, C., and Riener, A. (2023, January 18–22). Communication of Uncertainty Information in Cooperative, Automated Driving: A Comparative Study of Different Modalities. Proceedings of the 15th International Conference on Automotive User Interfaces and Interactive Vehicular Applications, Ingolstadt, Germany.10.1145/3580585.3607154 ↗
98. Kuhlmann, K., Borowy, M., and Vollrath, M. (2024, January 22–25). Illuminating Safety: Can Light Cues Enhance Situation Awareness During Take-Overs in Conditional Automated Driving?. Proceedings of the Adjunct Proceedings of the 16th International Conference on Automotive User Interfaces and Interactive Vehicular Applications, Stanford, CA, USA.10.1145/3641308.3685035 ↗
99. REViVe: Development of a reactive environmental vigilance in-vehicle system to mitigate drowsiness-induced inattention during automated driving10.1016/j.aap.2025.108045 ↗
100. Effectiveness of replacement of automotive side mirrors by in-vehicle LCD–Effect of location and size of LCD on safety and efficiency–10.1016/j.ergon.2018.03.010 ↗