A Comprehensive Review: Multisensory and Cross-Cultural Approaches to Driver Emotion Modulation in Vehicle Systems
Source abstract
Abstract
Road accidents are caused by multiple factors. Aggressive driving and traffic violations account for 74% of road traffic accidents. In total, 92% of fatalities occur in low- and middle-income countries. Drivers’ emotions significantly influence driving performance, making emotional modulation critical during vehicle interaction. With the rise of smart vehicles, in-vehicle affective computing and human-centered design have gained importance. This review analyzes 802 studies related to driver emotional regulation, focusing on 74 studies regarding sensory stimuli and cultural contexts. The results show that single-sensory methods dominate, yet multisensory approaches using auditory and visual elements are more effective. Most studies overlook cultural factors, particularly the differences in East–West cultural values, indicating a need to tailor modulation methods based on cultural preferences. Designs must emphasize adaptability and cultural consistency. This review aims to analyze driver emotional modulation thoroughly, providing key insights for developing vehicle systems that meet the diverse emotional and cultural needs of global drivers. Future research should focus on creating multisensory emotional modulation systems that offer positive reinforcement without causing excessive relaxation or aggression, accommodating subtle cultural and individual differences, thus enhancing the safety of autonomous driving.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Color perception and design · Multisensory perception and integration · Olfactory and Sensory Function Studies
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Department of Mechanical Engineering, Faculty of Engineering, Centre for Sustainable and Smart Manufacturing (CSSM), Universiti Malaya, Kuala Lumpur 50603, Malaysia
References (100+)
Show source-provided references
1. (2024, July 28). Global Status Report on Road Safety 2023. Available online: https://www.who.int/teams/social-determinants-of-health/safety-and-mobility/global-status-report-on-road-safety-2023.
2. Measuring the perception of aggression in driving behavior10.1016/j.aap.2020.105709 ↗
3. Analyzing driver behavior under naturalistic driving conditions: A review10.1016/j.aap.2020.105908 ↗
4. Braun, M., Chadowitz, R., and Alt, F. (2019). User Experience of Driver State Visualizations: A Look at Demographics and Personalities. Human-Computer Interaction—INTERACT 2019, Springer.10.1007/978-3-030-29390-1_9 ↗
5. Intelligent In-Vehicle Interaction Technologies10.1002/aisy.202100122 ↗
6. Emotion Recognition from Multiple Modalities: Fundamentals and methodologies10.1109/MSP.2021.3106895 ↗
7. The Empathetic Car10.1145/3478078 ↗
8. Intelligent Cockpit for Intelligent Connected Vehicles: Definition, Taxonomy, Technology and Evaluation10.1109/TIV.2023.3339798 ↗
9. Nareshkumar, R., Suseela, G., Nimala, K., and Niranjana, G. (2023). Feasibility and Necessity of Affective Computing in Emotion Sensing of Drivers for Improved Road Safety. Principles and Applications of Socio-Cognitive and Affective Computing, IGI Global.10.4018/978-1-6684-3843-5.ch007 ↗
10. From Intelligent Vehicles to Smart Societies: A Parallel Driving Approach10.1109/TCSS.2018.2862058 ↗
11. Affective Automotive User Interfaces–Reviewing the State of Driver Affect Research and Emotion Regulation in the Car
12. Driver Emotion Recognition for Intelligent Vehicles
13. Overview of intelligent vehicle core technology and development10.1088/1755-1315/769/4/042054 ↗
14. Magana, V.C., Scherz, W.D., Seepold, R., Madrid, N.M., Paneda, X.G., and Garcia, R. (2020). The Effects of the Driver’s Mental State and Passenger Compartment Conditions on Driving Performance and Driving Stress. Sensors, 20.10.3390/s20185274 ↗
15. Braun, M., Schubert, J., Pfleging, B., and Alt, F. (2019). Improving Driver Emotions with Affective Strategies. Multimodal Technol. Interact., 3.10.3390/mti3010021 ↗
16. Susindar, S., and Ferris, T. (2023, November 01). Evaluating Emotion Regulation Techniques for Supporting Driving Safety and Performance, Available online: https://rosap.ntl.bts.gov/view/dot/73295.
17. Zepf, S., El Haouij, N., Lee, J., Ghandeharioun, A., Hernandez, J., and Picard, R.W. (2020, January 12–18). Studying Personalized Just-in-time Auditory Breathing Guides and Potential Safety Implications during Simulated Driving. Proceedings of the 28th ACM Conference on User Modeling, Adaptation and Personalization, Genoa, Italy.10.1145/3340631.3394854 ↗
18. Effects of emotions on driving behavior10.1016/j.trf.2018.08.012 ↗
19. The emotional side of cognitive distraction: Implications for road safety10.1016/j.aap.2012.04.004 ↗
20. The cold driver: Cold stress while driving results in dangerous behavior10.1016/j.biopsycho.2016.09.011 ↗
21. Hiroike, S.Y. (2009, January 9–11). Study of olfactory effect on individual driver under driving. Proceedings of the 2009 ICME International Conference on Complex Medical Engineering, Tempe, Arizona.10.1109/ICCME.2009.4906634 ↗
22. Review and Perspectives on Human Emotion for Connected Automated Vehicles10.1007/s42154-023-00270-z ↗
23. Gu, H., Liang, B., and Cao, H. (2024). User-centered framework for assessing the performance of smart car cockpits. Int. J. Adv. Manuf. Technol., 1–14.10.1007/s00170-024-12994-1 ↗
24. Al-Hussein, W.A., Kiah, M.L.M., Por, L.Y., and Zaidan, B.B. (2021). Investigating the Effect of Social and Cultural Factors on Drivers in Malaysia: A Naturalistic Driving Study. Int. J. Environ. Res. Public Health, 18.10.3390/ijerph182211740 ↗
25. Shinar, D. (2017). Distraction and Inattention. Traffic Safety and Human Behavior, Emerald Publishing Limited.10.1108/9781786352217 ↗
26. Review of Cultures and Organizations: Software of the Mind., by G. Hofstede10.2307/2393257 ↗
27. A Review of Research on Driving Styles and Road Safety10.1177/0018720815591313 ↗
28. In-Vehicle Sensing for Smart Cars10.1109/OJVT.2022.3174546 ↗
29. An EEG Data Processing Approach for Emotion Recognition10.1109/JSEN.2022.3168572 ↗
30. A multi-modal driver emotion dataset and study: Including facial expressions and synchronized physiological signals10.1016/j.engappai.2023.107772 ↗
31. On-road driver facial expression emotion recognition with parallel multi-verse optimizer (PMVO) and optical flow reconstruction for partial occlusion in internet of things (IoT)10.1016/j.measen.2023.100711 ↗
32. Cai, Y., Li, X., and Li, J. (2023). Emotion Recognition Using Different Sensors, Emotion Models, Methods and Datasets: A Comprehensive Review. Sensors, 23.10.3390/s23052455 ↗
33. Oh, G., Ryu, J., Jeong, E., Yang, J.H., Hwang, S., Lee, S., and Lim, S. (2021). DRER: Deep Learning–Based Driver’s Real Emotion Recognizer. Sensors, 21.10.3390/s21062166 ↗
34. Driver Emotion Recognition with a Hybrid Attentional Multimodal Fusion Framework10.1109/TAFFC.2023.3250460 ↗
35. Lisetti, C.L., and Nasoz, F. (2005, January 22–27). Affective Intelligent Car Interfaces with Emotion Recognition. Proceedings of the 11th International Conference on Human Computer Interaction, Las Vegas, NV, USA.
36. Multimodal System to Detect Driver Fatigue Using EEG, Gyroscope, and Image Processing10.1109/ACCESS.2020.3009226 ↗
37. European Commission (2024, July 28). Research and Artificial Intelligence—Digital Strategy. Available online: https://rea.ec.europa.eu/research-and-artificial-intelligence_en.
38. Measuring the impact of accounting journals using Google Scholar and the g-index10.1016/j.bar.2009.10.002 ↗
39. The Moderating Effects of Emotions on the Relationship Between Self-Reported Individual Traits and Actual Risky Driving Behaviors10.2147/PRBM.S301156 ↗
40. Emotions, behaviour, and the adolescent driver: A literature review10.1016/j.trf.2017.06.019 ↗
41. How appraisals shape driver emotions: A study from discrete and dimensional emotion perspectives10.1016/j.trf.2014.09.012 ↗
42. Road rage and aggressive driving behaviors: The role of state-trait anxiety and coping strategies
43. The Effects of Personality and Attitude on Risky Driving Behavior Among Public van Drivers: Hierarchical Modeling10.1016/j.shaw.2022.03.005 ↗
44. Negative or positive? The effect of emotion and mood on risky driving10.1016/j.trf.2012.08.009 ↗
45. Zhang, X., Chang, R., Sui, X., and Li, Y. (2022). Influences of Emotion on Driving Decisions at Different Risk Levels: An Eye Movement Study. Front. Psychol., 13.10.3389/fpsyg.2022.788712 ↗
46. Effects of driving anger on driver behavior—Results from naturalistic driving data10.1016/j.trf.2016.10.019 ↗
47. Development and validation of the Drivers’ Emotional Intelligence Inventory (EMOVIAL)10.1016/j.trf.2020.05.007 ↗
48. Detecting aggressive driving patterns in drivers using vehicle sensor data
49. HARMONY: A Human-Centered Multimodal Driving Study in the Wild10.1109/ACCESS.2021.3056007 ↗
50. Factors underpinning unsafe driving: A systematic literature review of car drivers10.1016/j.trf.2020.04.008 ↗
51. A review on key challenges in intelligent vehicles: Safety and driver-oriented features10.1049/itr2.12088 ↗
52. Just Breathe: In-Car Interventions for Guided Slow Breathing10.1145/3191760 ↗
53. Balters, S., Murnane, E.L., Landay, J.A., and Paredes, P.E. (2018, January 21). Breath Booster! Exploring In-Car, Fast-Paced Breathing Interventions to Enhance Driver Arousal State. Proceedings of the 12th Eai International Conference on Pervasive Computing Technologies for Healthcar, New York, NY, USA.10.1145/3240925.3240939 ↗
54. Evaluation of driver stress intervention with guided breathing and positive comments10.1016/j.apergo.2023.104144 ↗
55. The impact of emotions and predominant emotion regulation technique on driving performance10.3233/WOR-2012-0666-3608 ↗
56. The Effect of Vehicle Automation Styles on Drivers’ Emotional State10.1109/TITS.2023.3239880 ↗
57. Hassib, M., Braun, M., Pfleging, B., and Alt, F. (2019). Detecting and Influencing Driver Emotions Using Psycho-Physiological Sensors and Ambient Light. Human-Computer Interaction—INTERACT 2019, Springer.10.1007/978-3-030-29381-9_43 ↗
58. Visual-Attribute-Based Emotion Regulation of Angry Driving Behaviors10.1109/MITS.2021.3050890 ↗
59. Wang, C., Terken, J., Hu, J., and Rauterberg, M. (2016, January 24). “Likes” and “Dislikes” on the Road: A Social Feedback System for Improving Driving Behavior. Proceedings of the AutomotiveUI’16: 8th International Conference on Automotive User Interfaces and Interactive Vehicular Applications, Ann Arbor, MI, USA.10.1145/3003715.3005403 ↗
60. Harris, H., and Nass, C. (2011, January 7). Emotion regulation for frustrating driving contexts. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, Vancouver, BC, Canada.10.1145/1978942.1979050 ↗
61. Oehl, M., Ihme, K., Drewitz, U., Pape, A.-A., Cornelsen, S., and Schramm, M. (2019, January 21–25). Towards a frustration-aware assistant for increased in-vehicle UX. Proceedings of the 11th International Conference on Automotive User Interfaces and Interactive Vehicular Applications: Adjunct Proceedings, Utrecht, The Netherlands.10.1145/3349263.3351518 ↗
62. FakhrHosseini, M., and Jeon, M. (2016, January 24–26). The Effects of Various Music on Angry Drivers’ Subjective, Behavioral, and Physiological States. Proceedings of the Adjunct 8th International Conference on Automotive User Interfaces and Interactive Vehicular Applications, Ann Arbor, MI, USA.10.1145/3004323.3004346 ↗
63. Zhu, Y., Wang, Y., Li, G., and Guo, X. (2016, January 24). Recognizing and Releasing Drivers’ Negative Emotions by Using Music: Evidence from Driver Anger. Proceedings of the AutomotiveUI’16: 8th International Conference on Automotive User Interfaces and Interactive Vehicular Applications, New York, NY, USA.10.1145/3004323.3004344 ↗
64. How do angry drivers respond to emotional music? A comprehensive perspective on assessing emotion10.1007/s12193-019-00300-3 ↗
65. Exploring an alternative in-car music background designed for driver safety10.1016/j.trf.2011.12.001 ↗
66. Effects of mood induction via music on cardiovascular measures of negative emotion during simulated driving10.1016/j.physbeh.2014.02.049 ↗
67. The effect of the dominance of an in-vehicle agent’s voice on driver situation awareness, emotion regulation, and trust: A simulated lab study of manual and automated driving10.1016/j.trf.2022.01.009 ↗
68. Bankar, C., Bhide, A., Kulkarni, A., Ghube, C., and Bedekar, M. (December, January 30). Driving Control Using Emotion Analysis Via EEG. Proceedings of the 2018 IEEE Punecon, Pune, India.10.1109/PUNECON.2018.8745412 ↗
69. Effects of speech-based intervention with positive comments on reduction of driver’s anger state and perceived workload, and improvement of driving performance10.1016/j.apergo.2020.103098 ↗
70. Hsieh, L., Seaman, S., and Young, R. (2010, January 11–12). Effect of emotional speech tone on driving from lab to road: fMRI and ERP studies. Proceedings of the 2nd International Conference on Automotive User Interfaces and Interactive Vehicular Applications, Pittsburgh, PE, USA.10.1145/1969773.1969778 ↗
71. Jonsson, I.-M., Nass, C., Endo, J., Reaves, B., Harris, H., Ta, J.L., Chan, N., and Knapp, S. (2004, January 24). Don’t blame me I am only the driver: Impact of blame attribution on attitudes and attention to driving task. Proceedings of the CHI04: CHI 2004 Conference on Human Factors in Computing Systems, New York, NY, USA.10.1145/985921.986028 ↗
72. Toward Decreasing the Driving Risk: Speech-Based Driver’s Anger Regulation in Smart Cockpit10.1109/JRFID.2022.3208199 ↗
73. SAfeDJ: A Crowd-Cloud Codesign Approach to Situation-Aware Music Delivery for Drivers10.1145/2808201 ↗
74. The impact of environmental noise on drivers’ cognitive abilities: A case study on in-vehicle voice interaction interfaces10.1016/j.apergo.2024.104247 ↗
75. Mitigating passive fatigue during monotonous drives with thermal stimuli: Insights into the effect of different stimulation durations10.1016/j.aap.2017.12.005 ↗
76. Dmitrenko, D., Maggioni, E., Brianza, G., Holthausen, B.E., Walker, B.N., and Obrist, M. (2020, January 21). CARoma Therapy: Pleasant Scents Promote Safer Driving, Better Mood, and Improved Well-Being in Angry Drivers. Proceedings of the 2020 Chi Conference on Human Factors in Computing Systems, New York, NY, USA.10.1145/3313831.3376176 ↗
77. Yoshida, M., Kato, C., Kakamu, Y., Kawasumi, M., Yamasaki, H., Yamamoto, S., Nakano, T., and Yamada, M. (2011, January 13–15). Study on Stimulation Effects for Driver Based on Fragrance Presentation. Proceedings of the MVA2011 IAPR Conference on Machine Vision Applications, Nara, Japan. Available online: https://www.mva-org.jp/Proceedings/2011CD/papers/09-26.pdf.
78. Dmitrenko, D., Maggioni, E., and Obrist, M. (2019, January 2). Towards a Framework for Validating the Matching Between Notifications and Scents in Olfactory In-Car Interaction. Proceedings of the CHI ’19: CHI Conference on Human Factors in Computing Systems, New York, NY, USA.10.1145/3290607.3313001 ↗
79. Dmitrenko, D., Maggioni, E., and Obrist, M. (2018, January 2). I Smell Trouble: Using Multiple Scents to Convey Driving-Relevant Information. Proceedings of the ICMI ‘18: International Conference on Multimodal Interaction, New York, NY, USA.10.1145/3242969.3243015 ↗
80. Research on the olfactory stimulation-based method for positively regulating drivers’ negative emotions
81. Holman, A.C., and Popușoi, S.A. (2020). How You Deal with Your Emotions Is How You Drive. Emotion Regulation Strategies, Traffic Offenses, and the Mediating Role of Driving Styles. Sustainability, 12.10.3390/su12124929 ↗
82. A multimodal psychological, physiological and behavioural dataset for human emotions in driving tasks10.1038/s41597-022-01557-2 ↗
83. Taourarti, I., Ramaswamy, A., Ibanez-Guzman, J., Monsuez, B., and Tapus, A. (2023, January 13–17). Towards Cross-Cultural Intelligent Vehicles: A Review. Proceedings of the Twelfth International Conference on Advances in Vehicular Systems, Technologies and Applications, Barcelona, Spain.
84. Baltruschat, S., Mas-Cuesta, L., Cándido, A., Maldonado, A., Verdejo-Lucas, C., Catena-Verdejo, E., and Catena, A. (2021). Repeat Traffic Offenders Improve Their Performance in Risky Driving Situations and Have Fewer Accidents Following a Mindfulness-Based Intervention. Front. Psychol., 11.10.3389/fpsyg.2020.567278 ↗
85. Revisiting cultural differences in emotion perception between easterners and westerners: Chinese perceivers are accurate, but see additional non-intended emotions in negative facial expressions10.1016/j.jesp.2019.02.003 ↗
86. Emotions Across Cultures and Methods
87. Emotion regulation of social exclusion: A cross-cultural study10.1057/s41599-021-00857-z ↗
88. Pauw, L.S., Vu, T.-V., Sun, R., Vuillier, L., Milek, A., and Sauter, D. (2020). Emotion Regulation and Wellbeing: A Cross-Cultural Study during the COVID-19 Outbreak. PsyArXiv.10.31234/osf.io/9qrw5 ↗
89. Emotion regulation among Chinese and German children and adolescents: A binational comparative study10.1007/s12144-022-03578-x ↗
90. Effects of Cultural Diversity and Emotional Regulation Strategies on Team Member Performance10.1017/orp.2017.4 ↗
91. Cultural Differences in Perceptual Strategies Underlying Emotion Regulation10.1177/0022022119876102 ↗
92. Liddell, B.J., and Williams, E.N. (2019). Cultural Differences in Interpersonal Emotion Regulation. Front. Psychol., 10.10.3389/fpsyg.2019.00999 ↗
93. Huggins, C.F., Williams, J.H.G., and Sato, W. (2023). Cross-cultural differences in self-reported and behavioural emotional self-awareness between Japan and the UK. BMC Res. Notes, 16.10.1186/s13104-023-06660-0 ↗
94. Chen, P., Chung-Fat-Yim, A., and Marian, V. (2022). Cultural Experience Influences Multisensory Emotion Perception in Bilinguals. Languages, 7.10.3390/languages7010012 ↗
95. I Feel Your Voice: Cultural Differences in the Multisensory Perception of Emotion10.1177/0956797610380698 ↗
96. Differences in Emotional Labor Across Cultures: A Comparison of Chinese and U.S. Service Workers10.1007/s10869-013-9288-7 ↗
97. Emotion experience and regulation in China and the United States: How do culture and gender shape emotion responding?10.1080/00207594.2011.626043 ↗
98. Cultural conditioning: Understanding interpersonal accommodation in India and the United States in terms of the modal characteristics of interpersonal influence situations10.1037/a0021083 ↗
99. Chen, P. (2019). The Influence of Language and Culture on the Multisensory Perception of Emotion. [Ph.D. Thesis, Northwestern University].
100. Emotion Regulation Choice: Differences in U.S. and Indian Populations