Fast Emotion Recognition Based on Single Pulse PPG Signal with Convolutional Neural Network
Source abstract
Abstract
Physiological signals contain considerable information regarding emotions. This paper investigated the ability of photoplethysmogram (PPG) signals to recognize emotion, adopting a two-dimensional emotion model based on valence and arousal to represent human feelings. The main purpose was to recognize short term emotion using a single PPG signal pulse. We used a one-dimensional convolutional neural network (1D CNN) to extract PPG signal features to classify the valence and arousal. We split the PPG signal into a single 1.1 s pulse and normalized it for input to the neural network based on the personal maximum and minimum values. We chose the dataset for emotion analysis using physiological (DEAP) signals for the experiment and tested the 1D CNN as a binary classification (high or low valence and arousal), achieving the short-term emotion recognition of 1.1 s with 75.3% and 76.2% valence and arousal accuracies, respectively, on the DEAP data.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: EEG and Brain-Computer Interfaces · Non-Invasive Vital Sign Monitoring · Emotion and Mood Recognition
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
School of Electrical Engineering, Korea University, Seoul 02841, Korea
Department of Digital Electronics, Inha Technical College, Incheon 22212, Korea
Department of Human Intelligence and Robot Engineering, Sangmyung University, Cheonan 31066, Korea
Funding
National Research Foundation of Korea (NRF) funded by the Ministry of Education
References (39)
Show source-provided references
1. Hönig, F., Batliner, A., and Nöth, E. (2007, January 12–14). Real-time recognition of the affective user state with physiological signals. Proceedings of the Doctoral Consortium, Affective Computing and Intelligent Interaction, Lisbon, Portugal.
2. Emotion recognition in human-computer interaction10.1109/79.911197 ↗
3. Progressive Filtering Approach for Early Human Action Recognition10.1007/s12555-017-0532-1 ↗
4. Soft robot review10.1007/s12555-016-0462-3 ↗
5. Busso, C., Deng, Z., Yildirim, S., Bulut, M., Lee, C.M., Kazemzadeh, A., Lee, S., Neumann, U., and Narayanan, S. (2004, January 13–15). Analysis of emotion recognition using facial expressions, speech and multimodal information. Proceedings of the 6th International Conference on Multimodal Interfaces, State College, PA, USA.10.1145/1027933.1027968 ↗
6. A Sociable Human-robot Interaction Scheme Based on Body Emotion Analysis10.1007/s12555-017-0423-5 ↗
7. Emotion recognition: The role of facial movement and the relative importance of upper and lower areas of the face10.1037/0022-3514.37.11.2049 ↗
8. Survey on speech emotion recognition: Features, classification schemes, and databases10.1016/j.patcog.2010.09.020 ↗
9. EEG-based emotion recognition in music listening10.1109/TBME.2010.2048568 ↗
10. Wagner, J., Kim, J., and André, E. (2005, January 6). From physiological signals to emotions: Implementing and comparing selected methods for feature extraction and classification. Proceedings of the 2005 IEEE International Conference on Multimedia and Expo, Amsterdam, The Netherlands.
11. Jerritta, S., Murugappan, M., Nagarajan, R., and Wan, K. (2011, January 4–6). Physiological signals based human emotion recognition: A review. Proceedings of the 2011 IEEE 7th International Colloquium on Signal Processing and Its Applications, Penang, Malaysia.10.1109/CSPA.2011.5759912 ↗
12. Liu, Y., Sourina, O., and Nguyen, M.K. (2011). Real-time EEG-based emotion recognition and its applications. Transactions on Computational Science XII, Springer.10.1007/978-3-642-22336-5_13 ↗
13. Nakasone, A., Prendinger, H., and Ishizuka, M. (2005, January 6–8). Emotion recognition from electromyography and skin conductance. Proceedings of the 5th International Workshop on Biosignal Interpretation, Tokyo, Japan.
14. Analysis of physiological for emotion recognition with the IRS model10.1016/j.neucom.2015.07.112 ↗
15. Emotion recognition based on physiological changes in music listening10.1109/TPAMI.2008.26 ↗
16. Automatic recognition of boredom in video games using novel biosignal moment-based features10.1109/T-AFFC.2011.4 ↗
17. Emotional state classification from EEG data using machine learning approach10.1016/j.neucom.2013.06.046 ↗
18. Feature extraction and selection for emotion recognition from EEG10.1109/TAFFC.2014.2339834 ↗
19. Emotion recognition from EEG signals by using multivariate empirical mode decomposition10.1007/s10044-016-0567-6 ↗
20. EEG-based emotion recognition using deep learning network with principal component based covariate shift adaptation10.1155/2014/627892 ↗
21. Learning deep physiological models of affect10.1109/MCI.2013.2247823 ↗
22. Emotion Recognition based on EEG using LSTM Recurrent Neural Network
23. Yang, Y., Wu, Q., Qiu, M., Wang, Y., and Chen, X. (2018, January 8–13). Emotion Recognition from Multi-Channel EEG through Parallel Convolutional Recurrent Neural Network. Proceedings of the 2018 International Joint Conference on Neural Networks (IJCNN), Rio de Janeiro, Brazil.10.1109/IJCNN.2018.8489331 ↗
24. Rakshit, R., Reddy, V.R., and Deshpande, P. (2016, January 16). Emotion detection and recognition using HRV features derived from photoplethysmogram signals. Proceedings of the 2nd Workshop on Emotion Representations and Modelling for Companion Systems, Tokyo, Japan.10.1145/3009960.3009962 ↗
25. The relationship between autonomic indicators of psychological stress: Heart rate and skin conductance10.1097/00006842-196301000-00004 ↗
26. Emotion extraction based on multi bio-signal using back-propagation neural network10.1007/s11042-016-4213-5 ↗
27. Guo, H.W., Huang, Y.S., Lin, C.H., Chien, J.C., Haraikawa, K., and Shieh, J.S. (November, January 31). Heart rate variability signal features for emotion recognition by using principal component analysis and support vectors machine. Proceedings of the 2016 IEEE 16th International Conference on Bioinformatics and Bioengineering (BIBE), Taichung, Taiwan.10.1109/BIBE.2016.40 ↗
28. Poincaré’s section analysis for PPG-based automatic emotion recognition10.1016/j.chaos.2018.07.035 ↗
29. A circumplex model of affect10.1037/h0077714 ↗
30. Human emotion recognition using deep belief network architecture10.1016/j.inffus.2018.10.009 ↗
31. Zheng, W.L., Zhu, J.Y., Peng, Y., and Lu, B.L. (2014, January 14–18). EEG-based emotion classification using deep belief networks. Proceedings of the 2014 IEEE International Conference on Multimedia and Expo (ICME), Chengdu, China.10.1109/ICME.2014.6890166 ↗
32. Respiration-based emotion recognition with deep learning10.1016/j.compind.2017.04.005 ↗
33. Deap: A database for emotion analysis; using physiological signals10.1109/T-AFFC.2011.15 ↗
34. Martínez, H.P. (2013). Advancing Affect Modeling via Preference Learning and Unsupervised Feature Extraction, Center for Computer Cames Research, IT University of Copenhagen.
35. Zhuang, X., Rozgic, V., and Crystal, M. (2014, January 1–4). Compact unsupervised eeg response representation for emotion recognition. Proceedings of the IEEE-EMBS International Conference on Biomedical and Health Informatics (BHI), Valencia, Spain.10.1109/BHI.2014.6864469 ↗
36. Torres-Valencia, C.A., Daza-Santacoloma, G., and Orozco-Gutiérrez, A.A. (2014, January 17–19). Electric propagation modeling of Deep Brain Stimulation (DBS) using the finite element method (FEM). Proceedings of the 2014 XIX Symposium on Image, Signal Processing and Artificial Vision, Armenia, Colombia.10.1109/STSIVA.2014.7010180 ↗
37. Xu, H., and Plataniotis, K.N. (2016, January 4–6). EEG-based affect states classification using deep belief networks. Proceedings of the 2016 Digital Media Industry & Academic Forum (DMIAF), Santorini, Greece.10.1109/DMIAF.2016.7574921 ↗
38. Liu, W., Zheng, W.L., and Lu, B.L. (2016). Emotion recognition using multimodal deep learning. International Conference on Neural Information Processing, Springer.10.1007/978-3-319-46672-9_58 ↗
39. Chao, H., Dong, L., Liu, Y., and Lu, B. (2019). Emotion Recognition from Multiband EEG Signals Using CapsNet. Sensors, 19.10.3390/s19092212 ↗