journal article·2024-11-15Open access

Emotion Recognition in a Closed-Cabin Environment: An Exploratory Study Using Millimeter-Wave Radar and Respiration Signals

Hanyu Wang, Dengkai Chen, Sen Gu, Yao Zhou, Jianghao Xiao, Yiwei Sun, Jianhua Sun, Yuexin Huang, Xian Zhang, Hao Fan

Applied Sciences· Volume 14· Issue 22

Abstract

In the field of psychology and cognition within closed cabins, noncontact vital sign detection holds significant potential as it can enhance the user’s experience by utilizing objective measurements to assess emotions, making the process more sustainable and easier to deploy. To evaluate the capability of noncontact methods for emotion recognition in closed spaces, such as submarines, this study proposes an emotion recognition method that employs a millimeter-wave radar to capture respiration signals and uses a machine-learning framework for emotion classification. Respiration signals were collected while the participants watched videos designed to elicit different emotions. An automatic sparse encoder was used to extract features from respiration signals, and two support vector machines were employed for emotion classification. The proposed method was experimentally validated using the FaceReader software, which is based on audiovisual signals, and achieved an emotion classification accuracy of 68.21%, indicating the feasibility and effectiveness of using respiration signals to recognize and assess the emotional states of individuals in closed cabins.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: Optical and Acousto-Optic Technologies

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

Key Laboratory of Industrial Design and Ergonomics, Ministry of Industry and Information Technology, Northwestern Polytechnical University, Xi’an 710072, China

Industrial Design and Ergonomics Innovation Center, Ningbo Institute of Northwestern Polytechnical University, Ningbo 315103, China

School of Mechanical and Electrical Engineering, Henan University of Technology, Zhengzhou 450001, China

Department of Design, Southeast University, Nanjing 211189, China

Funding

the Fundamental Research Funds for the Central Universities (CN) · the Special Support Program for High-Level Talents of Shaanxi Province (CN)

References (53)

Show source-provided references
1. Are Emotions Natural Kinds?10.1111/j.1745-6916.2006.00003.x ↗
2. Emotion Experience and Its Varieties10.1177/1754073909103595 ↗
3. What Are Emotions? And How Can They Be Measured?10.1177/0539018405058216 ↗
4. Cowie, R., and de Rosis, F. (2007). Real-time recognition of the affective user state with physiological signals. Affective Computing and Intelligent Interaction, Proceedings of the Doctoral Consortium, OPUS Augsburg.
5. Jones, C.M., and Troen, T. (2007, January 28). Biometric Valence and Arousal Recognition. Proceedings of the 19th Australasian Conference on Computer-Human Interaction: Entertaining User Interfaces; Association for Computing Machinery, New York, NY, USA.10.1145/1324892.1324929 ↗
6. DEAP: A Database for Emotion Analysis; Using Physiological Signals10.1109/T-AFFC.2011.15 ↗
7. Oceanaut’s Personal Acoustic Comfort Prediction Model and Sound Environment Improvement Method in the Cabin of a Deep-Sea Manned Submersible10.1080/00140139.2022.2152110 ↗
8. Psychophysiology and Adaptive Automation10.1016/0301-0511(95)05161-9 ↗
9. Thermal Comfort Analysis Based on PMV/PPD in Cabins of Manned Submersibles10.1016/j.buildenv.2018.10.033 ↗
10. Analysis of Oceanaut Operating Performance Using an Integrated Bayesian Network Aided by the Fuzzy Logic Theory10.1016/j.ergon.2021.103129 ↗
11. Positive Emotions at Work10.1146/annurev-orgpsych-012119-044908 ↗
12. Fisher, C.D., and Noble, C.S. (2004). A Within-Person Examination of Correlates of Performance and Emotions While Working. Emotion and Performance, CRC Press.10.1207/s15327043hup1702_2 ↗
13. The Impact of a View from a Window on Thermal Comfort, Emotion, and Cognitive Performance10.1016/j.buildenv.2020.106779 ↗
14. Paakkanen, M.A., Martela, F., and Pessi, A.B. (2021). Responding to Positive Emotions at Work–The Four Steps and Potential Benefits of a Validating Response to Coworkers’ Positive Experiences. Front. Psychol., 12.10.3389/fpsyg.2021.668160 ↗
15. Automatic Emotion Recognition Based on Body Movement Analysis: A Survey10.1109/MCG.2014.106 ↗
16. Emotion Recognition From Body Movement10.1109/ACCESS.2019.2963113 ↗
17. Virtual Reality Facial Emotion Recognition in Social Environments: An Eye-Tracking Study10.1016/j.invent.2021.100432 ↗
18. A Novel Facial Emotion Recognition Method for Stress Inference of Facial Nerve Paralysis Patients10.1016/j.eswa.2022.116705 ↗
19. A Bimodal Approach for Speech Emotion Recognition Using Audio and Text
20. Lighting Environmental Assessment in Enclosed Spaces Based on Emotional Model10.1016/j.scitotenv.2023.161933 ↗
21. DREAMER: A Database for Emotion Recognition Through EEG and ECG Signals From Wireless Low-Cost Off-the-Shelf Devices10.1109/JBHI.2017.2688239 ↗
22. Electroencephalogram Emotion Recognition Based on Empirical Mode Decomposition and Optimal Feature Selection10.1109/TCDS.2018.2868121 ↗
23. Raja, M., and Sigg, S. (2016, January 14–18). Applicability of RF-Based Methods for Emotion Recognition: A Survey. Proceedings of the 2016 IEEE International Conference on Pervasive Computing and Communication Workshops (PerCom Workshops), Sydney, Australia.10.1109/PERCOMW.2016.7457119 ↗
24. Electrocardiogram-based Emotion Recognition System Using Empirical Mode Decomposition and Discrete Fourier Transform10.1111/exsy.12014 ↗
25. Shu, L., Xie, J., Yang, M., Li, Z., Li, Z., Liao, D., Xu, X., and Yang, X. (2018). A Review of Emotion Recognition Using Physiological Signals. Sensors, 18.10.3390/s18072074 ↗
26. Healey, J., and Picard, R. (1998, January 12–15). Digital Processing of Affective Signals. Proceedings of the 1998 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP ’98 (Cat. No.98CH36181), Seattle, WA, USA.
27. Zhao, M., Adib, F., and Katabi, D. (2016, January 3). Emotion Recognition Using Wireless Signals. Proceedings of the 22nd Annual International Conference on Mobile Computing and Networking, New York, NY, USA.10.1145/2973750.2973762 ↗
28. Padilla-Buritica, J.I., Martinez-Vargas, J.D., and Castellanos-Dominguez, G. (2016). Emotion Discrimination Using Spatially Compact Regions of Interest Extracted from Imaging EEG Activity. Front. Comput. Neurosci., 10.10.3389/fncom.2016.00055 ↗
29. Brooker, G. (November, January 30). Understanding Millimetre Wave FMCW Radars. Proceedings of the 1st International Conference Sensors Technology, Wuhan, China.
30. A Circumplex Model of Affect10.1037/h0077714 ↗
31. Convenience Sampling10.1136/bmj.f6304 ↗
32. A Sniff of Happiness10.1177/0956797614566318 ↗
33. Emotional Body Odors as Context: Effects on Cardiac and Subjective Responses10.1093/chemse/bjy021 ↗
34. (2024, May 22). Facial Expression Recognition Software|FaceReader. Available online: https://www.noldus.com/facereader.
35. Xia, S., Yang, J., Cai, W., Zhang, C., Hua, L., and Zhou, Z. (2021). Adaptive Complex Variational Mode Decomposition for Micro-Motion Signal Processing Applications. Sensors, 21.10.3390/s21051637 ↗
36. Ensemble Empirical Mode Decomposition: A Noise-Assisted Data Analysis Method10.1142/S1793536909000047 ↗
37. Improved Complete Ensemble EMD: A Suitable Tool for Biomedical Signal Processing10.1016/j.bspc.2014.06.009 ↗
38. Variational Mode Extraction: A New Efficient Method to Derive Respiratory Signals from ECG10.1109/JBHI.2017.2734074 ↗
39. Ju, Y., Guo, J., and Liu, S. (2015, January 17–19). A Deep Learning Method Combined Sparse Autoencoder with SVM. Proceedings of the 2015 International Conference on Cyber-Enabled Distributed Computing and Knowledge Discovery, Xi’an, China.10.1109/CyberC.2015.39 ↗
40. Respiration-Based Emotion Recognition with Deep Learning10.1016/j.compind.2017.04.005 ↗
41. On the Limited Memory BFGS Method for Large Scale Optimization10.1007/BF01589116 ↗
42. Modular Neural-SVM Scheme for Speech Emotion Recognition Using ANOVA Feature Selection Method10.1007/s00521-012-0814-8 ↗
43. Study of Human Comfort in Autonomous Vehicles Using Wearable Sensors10.1109/TITS.2021.3104827 ↗
44. Gouveia, C., Tomé, A., Barros, F., Soares, S.C., Vieira, J., and Pinho, P. (2020). Study on the Usage Feasibility of Continuous-Wave Radar for Emotion Recognition. Biomed. Signal Process. Control, 58.10.1016/j.bspc.2019.101835 ↗
45. Emotional States of Drivers and the Impact on Speed, Acceleration and Traffic Violations—A Simulator Study10.1016/j.aap.2014.04.010 ↗
46. Castiblanco Jimenez, I.A., Olivetti, E.C., Vezzetti, E., Moos, S., Celeghin, A., and Marcolin, F. (2024). Effective Affective EEG-Based Indicators in Emotion-Evoking VR Environments: An Evidence from Machine Learning. Neural Comput. Appl.10.1007/s00521-024-10240-z ↗
47. Jerath, R., and Beveridge, C. (2020). Respiratory Rhythm, Autonomic Modulation, and the Spectrum of Emotions: The Future of Emotion Recognition and Modulation. Front. Psychol., 11.10.3389/fpsyg.2020.01980 ↗
48. Mutual Information between Heart Rate Variability and Respiration for Emotion Characterization10.1088/1361-6579/ab310a ↗
49. Automated Facial Coding: Validation of Basic Emotions and FACS AUs in FaceReader10.1037/npe0000028 ↗
50. Validity Study on Facereader’s Images Recognition from Chinese Facial Expression Database
51. Placing the Face in Context: Cultural Differences in the Perception of Facial Emotion10.1037/0022-3514.94.3.365 ↗
52. Li, Y.-T., Yeh, S.-L., and Huang, T.-R. (2023). The Cross-Race Effect in Automatic Facial Expression Recognition Violates Measurement Invariance. Front. Psychol., 14.10.3389/fpsyg.2023.1201145 ↗
53. An Accurate Emotion Recognition System Using ECG and GSR Signals and Matching Pursuit Method10.1016/j.bj.2017.11.001 ↗