Employing Emerging Technologies to Develop and Evaluate In-Vehicle Intelligent Systems for Driver Support: Infotainment AR HUD Case Study
Source abstract
Abstract
The plurality of current infotainment devices within the in-vehicle space produces an unprecedented volume of incoming data that overwhelm the typical driver, leading to higher collision probability. This work presents an investigation to an alternative option which aims to manage the incoming information while offering an uncluttered and timely manner of presenting and interacting with the incoming data safely. The latter is achieved through the use of an augmented reality (AR) head-up display (HUD) system, which projects the information within the driver’s field of view. An uncluttered gesture recognition interface provides the interaction with the AR visuals. For the assessment of the system’s effectiveness, we developed a full-scale virtual reality driving simulator which immerses the drivers in challenging, collision-prone, scenarios. The scenarios unfold within a digital twin model of the surrounding motorways of the city of Glasgow. The proposed system was evaluated in contrast to a typical head-down display (HDD) interface system by 30 users, showing promising results that are discussed in detail.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Human-Automation Interaction and Safety · Traffic and Road Safety · Virtual Reality Applications and Impacts
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
School of Computing, Engineering and Built Environment, Glasgow Caledonian University, Glasgow G4-0BA, UK
Department of Autonomous Systems, Al-Balqa Applied University, Al Salt 19117, Jordan
King Abdullah II School of Information Technology, The University of Jordan, Amman 11942, Jordan
Volkswagen, Research and Development, Beijing 100068, China
School of Design and Architecture, Dar Al-Hekma University, Jeddah 22246, Saudi Arabia
Defence and Security Research Institute, The University of Nicosia, Nicosia CY-2417, Cyprus
References (68)
Show source-provided references
1. Augmented Reality Interface Design Approaches for Goal-directed and Stimulus-driven Driving Tasks10.1109/TVCG.2018.2868531 ↗
2. Human-machine collaboration through vehicle head-up display interface10.1007/s10111-008-0117-0 ↗
3. Charissis, V., Papanastasiou, S., Chan, W., and Peytchev, E. (2013, January 6–9). Evolution of a full-windshield HUD designed for current VANET communication standards. Proceedings of the 16th International IEEE Conference on Intelligent Transportation Systems (ITSC), The Hague, The Netherlands.10.1109/ITSC.2013.6728464 ↗
4. Understanding Driver Response Patterns to Mental Workload Increase in Typical Driving Scenarios10.1109/ACCESS.2018.2851309 ↗
5. Lagoo, R., Charissis, V., Chan, W., Khan, S., and Harrison, D. (2018, January 12–15). Prototype gesture recognition interface for a vehicular head-up display system. Proceedings of the 36th IEEE International Conference on Consumer Electronics (ICCE), Las Vegas, NV, USA.10.1109/ICCE.2018.8326146 ↗
6. Coley, G., Wesley, A., Reed, N., and Parry, I. (2008). Driver Reaction Times to Familiar but Unexpected Events, HIS. Published Project Report (PPR 313); Transport Research Laboratory (TRL).
7. Basacik, D., Reed, N., and Robbins, R. (2012). Smartphone Use while Driving: A Simulator Study, HIS. Published Project Report (PPR 592); Transport Research Laboratory (TRL).
8. Reed, N., and Robbins, R. (2008). The Effects of Text Messaging on Driver Behaviour: A Simulator Study, HIS. Published Project Report (PPR 367); Transport Research Laboratory (TRL).
9. Evaluating the effectiveness of a front windshield sticker reminder in reducing texting while driving in young adults
10. Regan, M.A., Lee, J.D., and Young, K.L. (2009). Cellular phones and driver distraction. Driver Distraction Theory, Effects and Mitigation, CRC Press.10.1201/9781420007497 ↗
11. Safety-critical event risk associated with cell phone tasks as measured in naturalistic driving studies: A systematic review and meta-analysis10.1016/j.aap.2015.11.015 ↗
12. Morris, A., Reed, S., Welsh, R., Brown, L., and Birrell, S. (2015). Distraction effects of navigation and green-driving systems—Results from field operational tests (FOTs) in the UK. Eur. Transp. Res. Rev., 7.10.1007/s12544-015-0175-3 ↗
13. The use of navigation systems in naturalistic driving10.1080/15389588.2015.1077384 ↗
14. Selection of Measurement Method for Detection of Driver Visual Cognitive Distraction: A Review10.1109/ACCESS.2017.2750743 ↗
15. Mitigating Driver’s Distraction with the use of Augmented Reality Head-Up Display and Gesture Recognition system10.1109/MCE.2019.2923896 ↗
16. Driver distraction and driver inattention: Definition, relationship and taxonomy10.1016/j.aap.2011.04.008 ↗
17. A comparison of two contemporary types of in-car multifunctional interfaces10.1016/j.apergo.2011.08.004 ↗
18. Toward Social Internet of Vehicles: Concept, Architecture, and Applications10.1109/ACCESS.2015.2416657 ↗
19. Privacy Management in Social Internet of Vehicles: Review, Challenges and Blockchain-Based Solutions10.1109/ACCESS.2019.2922236 ↗
20. Pickering, C.A., Burnham, K.J., and Richardson, M.J. (2007, January 22–24). A review of automotive human-machine interface technologies and techniques to reduce driver distraction. Proceedings of the 2nd IET International Conference on System Safety, London, UK.10.1049/cp:20070468 ↗
21. Mihai, D., Dumitru, A., Postelnicu, C., and Mogan, G. (2015, January 6–8). Video-based evaluation of driver’s visual attention using smartphones. Proceedings of the 2015 6th International Conference on Information, Intelligence, Systems and Applications (IISA), Corfu, Greece.10.1109/IISA.2015.7387983 ↗
22. Automotive Antenna Roof for Cooperative Connected Driving10.1109/ACCESS.2019.2897219 ↗
23. Charissis, V., Papanastasiou, S., and Vlachos, G. (2008, January 14–17). Comparative Study of Prototype Automotive HUD vs. HDD: Collision Avoidance Simulation and Results. Proceedings of the Society of Automotive Engineers (SAE) World Congress and Exhibition, Detroit, MI, USA.10.4271/2008-01-0203 ↗
24. Charissis, V., and Naef, M. (2007, January 13–15). Evaluation of Prototype Automotive Head-Up Display Interface: Testing Driver’s Focusing Ability through a VR Simulation. Proceedings of the IEEE Intelligent Vehicle Symposium, (IV ‘07), Istanbul, Turkey.10.1109/IVS.2007.4290174 ↗
25. Perceived Importance of Automotive HUD Information Items: A Study With Experienced HUD Users10.1109/ACCESS.2018.2828615 ↗
26. Behind the glass: Driver challenges and opportunities for AR automotive applications10.1109/JPROC.2013.2294642 ↗
27. Hyungil, K., Xuefang, W., Gabbard, J.L., and Polys, N.F. (2013, January 27–30). Exploring head-up augmented reality interfaces for crash warning systems. Proceedings of the 5th International Conference on Automotive User Interfaces and Interactive Vehicular Applications, ACM Automotive UI 2013, Eindhoven, The Netherlands.
28. Weinberg, G., Harsham, B., and Medenica, Z. (December, January 30). Evaluating the usability of a head-up display for selection from choice lists in cars. Proceedings of the 3rd ACM International Conference on Automotive User Interfaces and Interactive Vehicular Applications, Automotive UI 2011, Salzburg, Austria.10.1145/2381416.2381423 ↗
29. Brand, D., Büchele, K., and Meschtscherjakov, A. (2016, January 24–26). Pointing at the HUD: Gesture Interaction Using a Leap Motion. Proceedings of the 8th ACM International Conference on Automotive User Interfaces and Interactive Vehicular Applications, ACM Automotive UI 2016, Ann Arbor, MI, USA.10.1145/3004323.3004343 ↗
30. Beyond 5G: THz-based medium access protocol for mobile heterogeneous networks10.1109/MCOM.2018.1700924 ↗
31. Twidle, K. (2007). The Anatomy and Physiology of the Used Car Business: A Global Analysis with Particular Reference to Four Key Markets: U.S., Canada, France, Germany, Car Internet Research Program II and Capgemini.
32. User-defined gesture interaction for in-vehicle information systems10.1007/s11042-019-08075-1 ↗
33. Pickering, C.A., Burnham, K.J., and Richardson, M.J. (2007, January 28–29). A Research Study of Hand Gesture Recognition Technologies and Applications for Human Vehicle Interaction. Proceedings of the 2007 3rd Institution of Engineering and Technology Conference on Automotive Electronics, Warwick, UK.
34. Zobl, M., Geiger, M., Bengler, K., and Lang, M. (2001, January 5–10). A usability study on hand-gesture controlled operation of in-car devices. Proceedings of the Human-Computer Interaction International, New Orleans, LA, USA.
35. Alpern, M., and Minardo, K. (2003, January 5–10). Developing a car gesture interface for use as a secondary task. Proceedings of the Extended abstracts of the 2003 Conference on Human Factors in Computing Systems, CHI 2003, Ft. Lauderdale, FL, USA.10.1145/765891.766078 ↗
36. Reimer, B., Mehler, B., Dobres, J., and Coughlin, J.F. (2013). The Effects of a Production Level “Voice-Command” Interface on Driver Behaviour: Reported Workload, Physiology, Visual Attention, and Driving Performance, MIT. MIT Age-Lab Technical Report (No. 2013-18A); MIT Agelab White Paper.
37. SAE International (2020). Development of Design and Engineering Recommendations for in-Vehicle Alphanumeric Messages, SAE International. Document SAE J2831_202007.
38. Inzuka, Y., Osumi, Y., and Shinkai, H. (1999, January 2–6). Visibility of Head-Up Display for Automobiles. Proceedings of the 35th Annual Meeting of the Human Factors Society, San Francisco, CA, USA.
39. Yoo, H., Tsimhoni, O., Watanabe, H., Green, P., and Shah, R. (1999). Display of HUD Warnings to Drivers: Determining on Optimal Location, University of Michigan, Transportation Research Institute. Technical Report UMTRI-99-9.
40. Tsimhoni, O., Green, P., and Watanabe, H. (, January May). Detecting and Reading Text on HUDs: Effects of Driving Workload and Message Location. Proceedings of the 11th Annual Intelligent Transportation Systems America (ITSA) Meeting, Miami, FL, USA.
41. Designing a direct manipulation HUD interface for in-vehicle infotainment
42. Kortum, P. (2008). HCI Beyond the GUI, Design for Haptic, Speech, Olfactory, and Other Non-Traditional Interfaces, Elsevier.
43. Zhang, Y., Owechko, Y., and Zhang, J. (2004, January 3–6). Driver cognitive workload estimation: A data-driven perspective. Proceedings of the 7th International IEEE Conference on Intelligent Transportation Systems, Washington, DC, USA.
44. Examining the relationship between driver distraction and driving errors: A discussion of theory, studies and methods10.1016/j.ssci.2011.07.008 ↗
45. Strayer, D.L., Cooper, J.M., Goethe, R.M., McCarty, M.M., Getty, D., and Biondi, F. (2017). Visual and Cognitive Demands of Using In-Vehicle Infotainment Systems, AAA Foundation for Traffic Safety.
46. Effects of display position of a visual in-vehicle task on simulated driving10.1016/j.apergo.2005.06.002 ↗
47. Driver performance while text messaging using handheld and in-vehicle systems10.1016/j.aap.2010.11.019 ↗
48. Assessing the driving distraction effect of vehicle HMI displays using data mining techniques10.1016/j.trf.2020.01.016 ↗
49. Frank, M., Drikakis, D., and Charissis, V. (2020). Machine-Learning Methods for Computational Science and Engineering. Computation, 8.10.3390/computation8010015 ↗
50. Charissis, V., and Papanastasiou, S. (2008). Artificial Intelligence Rationale for Autonomous Vehicle Agents Behaviour in Driving Simulation Environment. Advances in Robotics, Automation and Control, InTechOpen.10.5772/5820 ↗
51. Stephanidis, C., Duffy, V.G., Streitz, N., Konomi, S., and Krömker, H. (2020, January 19–24). Improving Emergency Vehicles’ Response Times with the Use of Augmented Reality and Artificial Intelligence. Proceedings of the HCI International 2020—Late Breaking Papers: Digital Human Modeling and Ergonomics, Mobility and Intelligent Environments, Copenhagen, Denmark.10.1007/978-3-030-59987-4 ↗
52. Scottish Government (2009). Go Safe on Scotland’s Roads It’s Everyone’s Responsibility, Safety Framework to 2020, Report.
53. Jacko, J.A. (2011). Evaluation of Collision Avoidance Prototype Head-Up Display Interface for Older Drivers. Human-Computer Interaction. Towards Mobile and Intelligent Interaction Environments. Lecture Notes in Computer Science, Springer.
54. A time-series analysis of motorway collisions in England considering road infrastructure, socio-demographics, traffic and weather characteristics10.1016/j.jth.2015.10.005 ↗
55. Park, S., Lee, S., Park, S., and Park, S. (2019). AI-Based Physical and Virtual Platform with 5-Layered Architecture for Sustainable Smart Energy City Development. Sustainability, 11.10.3390/su11164479 ↗
56. Al-Ali, A.R., Gupta, R., Zaman Batool, T., Landolsi, T., Aloul, F., and Al Nabulsi, A. (2020). Digital Twin Conceptual Model within the Context of Internet of Things. Future Internet, 12.10.3390/fi12100163 ↗
57. Park, S., Park, S.H., Park, L.W., Park, S., Lee, S., Lee, T., Lee, S.H., Jang, H., Kim, S.M., and Chang, H. (2018). Design and Implementation of a Smart IoT Based Building and Town Disaster Management System in Smart City Infrastructure. Appl. Sci., 8.10.3390/app8112239 ↗
58. Wang, S., Charissis, V., and Harrison, D. (2017). Augmented Reality Prototype HUD for Passenger Infotainment in a Vehicular Environment. Adv. Sci. Technol. Eng. Syst. J., 634–641.10.25046/aj020381 ↗
59. Take-Over Performance and Safety Analysis Under Different Scenarios and Secondary Tasks in Conditionally Automated Driving10.1109/ACCESS.2019.2914864 ↗
60. Jose, R., Lee, G.A., and Billinghurst, M.A. (December, January 29). Comparative study of simulated augmented reality displays for vehicle navigation. Proceedings of the 28th Australian Conference on Computer-Human Interaction, OzCHI 2016, Launceston, Australia.10.1145/3010915.3010918 ↗
61. Parada-Loira, F., González-Agulla, E., and Alba-Castro, J.L. (2014, January 8–11). Hand gestures to control infotainment equipment in cars. Proceedings of the 2014 IEEE Intelligent Vehicles Symposium Proceedings, Dearborn, MI, USA.10.1109/IVS.2014.6856614 ↗
62. Keep your eyes on the road: Young driver crash risk increases according to duration of distraction10.1016/j.jadohealth.2013.11.021 ↗
63. Cognitive distraction while multitasking in the automobile10.1016/B978-0-12-385527-5.00002-4 ↗
64. Reading, typing, and driving: How interactions with in-vehicle systems degrade driving performance10.1016/j.trf.2014.06.001 ↗
65. Villalobos-Zúñiga, G., Kujala, T., and Oulasvirta, A. (2016, January 24–26). T9 + HUD: Physical keypad and HUD can improve driving performance while typing and driving. Proceedings of the 8th International Conference on Automotive User Interfaces and Interactive Vehicular Applications, Ann Arbor, MI, USA.
66. Sensitivity of eye-movement measures to in-vehicle task difficulty10.1016/j.trf.2005.04.014 ↗
67. Ranney, T.A., Baldwin, G.H.S., Smith, L.A., Mazzae, E.N., and Pierce, R.S. (2014). Detection Response Task Evaluation for Driver Distraction Measurement Application, Report No. DOT HS 812 077.
68. Stojmenova, K., and Sodnik, J. (2018). Detection-Response Task—Uses and Limitations. Sensors, 18.10.3390/s18020594 ↗