journal article·2022-10-06Open access

A Novel Approach for Emotion Recognition Based on EEG Signal Using Deep Learning

Awf Abdulrahman, Muhammet Baykara, Talha Burak Alakus

Applied Sciences· Volume 12· Issue 19

Abstract

Emotion can be defined as a voluntary or involuntary reaction to external factors. People express their emotions through actions, such as words, sounds, facial expressions, and body language. However, emotions expressed in such actions are sometimes manipulated by people and real feelings cannot be conveyed clearly. Therefore, understanding and analyzing emotions is essential. Recently, emotion analysis studies based on EEG signals appear to be in the foreground, due to the more reliable data collected. In this study, emotion analysis based on EEG signals was performed and a deep learning model was proposed. The study consists of four stages. In the first stage, EEG data were obtained from the GAMEEMO dataset. In the second stage, EEG signals were transformed with both VMD (variation mode decomposition) and EMD (empirical mode decomposition), and a total of 14 (nine from EMD, five from VMD) IMFs were obtained from each signal. In the third stage, statistical features were obtained from IMFs and maximum value, minimum value, and average values were used for this. In the last stage, both binary-class and multi-class classifications were made. The proposed deep learning model is compared with kNN (k nearest neighbor), SVM (support vector machines), and RF (random forest). At the end of the study, an accuracy of 70.89% in binary-class classification and 90.33% in multi-class classification was obtained with the proposed deep learning method.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: EEG and Brain-Computer Interfaces · Emotion and Mood Recognition · ECG Monitoring and Analysis

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

Department of Public Health, Duhok Polytechnic University, Duhok 42001, Iraq

Department of Software Engineering, Fırat University, Elazığ 23119, Turkey

Department of Software Engineering, Kırklareli University, Kırklareli 39000, Turkey

References (65)

Show source-provided references
1. Emotion classification during music listening from forehead biosignals10.1007/s11760-013-0591-6 ↗
2. Happy people live longer: Subjective well-being contributes to health and longevity10.1111/j.1758-0854.2010.01045.x ↗
3. Determination of effective EEG channels for discrimination of positive and negative emotions with wavelet decomposition and support vector machines
4. Backpropagation neural networks training for EEG-SSVEP classification of emotion recognition
5. Audio-textual emotion recognition based on improved neural networks
6. Lasri, I., Solh, A.R., and Belkacemi, M.E. (2019, January 29–30). Facial emotion recognition of students using convolutional neural network. Proceedings of the 3rd International Conference on Intelligent Computing in Data Sciences, Marrakech, Morocco.10.1109/ICDS47004.2019.8942386 ↗
7. Garber-Barron, M., and Si, M. (2012, January 10–15). Using body movement and posture for emotion detection in non-acted scenarios. Proceedings of the IEEE International Conference on Fuzzy Systems, Brisbane, QLD, Australia.10.1109/FUZZ-IEEE.2012.6250780 ↗
8. Sassenrath, C., Sassenberg, K., Ray, D.G., Scheiter, K., and Jarodzka, H. (2014). A motivational determinant of facial emotion recognition: Regulatory focus affects recognition of emotions in faces. PLoS ONE, 9.10.1371/journal.pone.0112383 ↗
9. Wioleta, S. (2013, January 6–8). Using physiological signals for emotion recognition. Proceedings of the 6th Conference on Human System Interactions, Sopot, Poland.10.1109/HSI.2013.6577880 ↗
10. A EEG-based emotion recognition model with rhythm and time characteristics10.1186/s40708-019-0100-y ↗
11. Wearable electroencephalography10.1109/MEMB.2010.936545 ↗
12. Bashivan, P., Rish, I., and Heisig, S. (2016). Mental state recognition via wearable EEG. arXiv.
13. Horlings, R., Datcu, D., and Rothkrantz, L.J.M. (2008, January 12–13). Emotion recognition using brain activity. Proceedings of the 9th International Conference on Computer Systems and Technologies and Workshop for PhD Students in Computing, Gabrovo, Bulgaria.10.1145/1500879.1500888 ↗
14. A model for basic emotions using observations of behavior in drosophila10.3389/fpsyg.2019.00781 ↗
15. Hania, W.M.B., and Lachiri, Z. (2017, January 8–10). Emotion classification in arousal-valence dimension using discrete affective keywords tagging. Proceedings of the International Conference on Engineering & MIS, Monastir, Tunisia.
16. Stickel, C., Ebner, M., Steinbach-Nordmann, S., Searle, G., and Holzinger, A. (2009, January 19–24). Emotion detection: Application of the valence arousal space for rapid biological usability testing to enhance universal access. Proceedings of the International Conference on Universal Access in Human-Computer Interaction, San Diego, CA, USA.10.1007/978-3-642-02707-9_70 ↗
17. Bradley, M.M., and Lang, P.J. (1999). International affective digitized sounds (IADS): Stimuli, instruction manual and affective ratings. Technical Report, No: B-2, The Center for Research in Psychophysiology, University of Florida.
18. A literature review on emotion recognition system using various facial expression
19. Database for an emotion recognition system based on EEG signals and various computer games—GAMEEMO10.1016/j.bspc.2020.101951 ↗
20. Emotion recognition with deep learning using GAMEEMO data set10.1049/el.2020.2460 ↗
21. Discrimination of genuine and acted emotional expressions using eeg signal and machine learning10.1109/ACCESS.2020.3032380 ↗
22. Subject independent emotion recognition from EEG using VMD and deep learning
23. Sharan, R.V., Berkovsky, S., Taib, R., Koprinska, I., and Li, J. (2020, January 20–24). Detecting personality traits using inter-hemispheric asynchrony of the brainwaves. Proceedings of the 42nd Annual International Conference of the IEEE Engineering in Medicine & Biology Society, Montreal, QC, Canada.10.1109/EMBC44109.2020.9176108 ↗
24. Priya, T.H., Mahalakshmi, P., Naidu, V.P.S., and Srinivas, M. (2020, January 24–25). Stress detection from EEG using power ratio. Proceedings of the International Conference on Emerging Trends in Information Technology and Engineering, Vellore, India.10.1109/ic-ETITE47903.2020.401 ↗
25. Matlovic, T., Gaspar, P., Moro, R., Simko, J., and Bielikova, M. (2016, January 20–21). Emotions detection using facial expressions recognition and EEG. Proceedings of the 11th International Workshop on Semantic and Social Media Adaptation and Personalization, Thessaloniki, Greece.10.1109/SMAP.2016.7753378 ↗
26. Single-trial EEG emotion recognition using granger causality/transfer entropy analysis10.1016/j.jneumeth.2020.108904 ↗
27. A 3D-convolutional neural network framework with ensemble learning techniques for multi-modal emotion recognition
28. Negative news recognition during social media news consumption using EEG10.1109/ACCESS.2019.2941251 ↗
29. emotion recognition based on double tree complex wavelet transform and machine learning in internet of things10.1109/ACCESS.2019.2948884 ↗
30. Improved complete ensemble EMD: A suitable tool for biomedical signal processing10.1016/j.bspc.2014.06.009 ↗
31. Li, X., Dong, L., Li, B., Lei, Y., and Xu, N. (2020). Microseismic signal denoising via empirical mode decomposition, compressed sensing, and soft-thresholding. Appl. Sci., 10.10.3390/app10062191 ↗
32. Boudraa, A.O., Cexus, J.C., Benramdane, S., and Beghdadi, A. (2007, January 12–15). Noise filtering using empirical mode decomposition. Proceedings of the 9th International Symposium on Signal Processing and Its Applications, Sharjah, UAE.10.1109/ISSPA.2007.4555624 ↗
33. Empirical mode decomposition analysis of climate changes with special reference to rainfall data10.1155/DDNS/2006/45348 ↗
34. Epileptic seizure classifications using empirical mode decomposition and its derivative10.1186/s12938-020-0754-y ↗
35. Empirical mode decomposition: Applications on signal and image processing10.1142/S1793536909000059 ↗
36. Lahmiri, S., and Boukadoum, M. (2014, January 22–24). Biomedical image denoising using variational mode decomposition. Proceedings of the IEEE Biomedical Circuits and Systems Conference, Lausanne, Switzerland.10.1109/BioCAS.2014.6981732 ↗
37. Variational mode decomposition10.1109/TSP.2013.2288675 ↗
38. Jiang, L., Zhou, X., Che, L., Rong, S., and Wen, H. (2019). Feature extraction and reconstruction by using 2D-VMD based on carrier-free UWB radar application in human motion recognition. Sensors, 19.10.3390/s19091962 ↗
39. Islam, M., Ahmed, T., Mostafa, S.S., Yusuf, S.U., and Ahmad, M. (2013, January 17–18). Human emotion recognition using frequency & statistical measures of EEG signal. Proceedings of the International Conference on Informatics, Electronics and Vision, Dhaka, Bangladesh.10.1109/ICIEV.2013.6572658 ↗
40. Li, C., Zhang, Y., and Ren, X. (2020). Modeling hourly soil temperature using deep BiLSTM neural network. Algorithms, 13.10.3390/a13070173 ↗
41. Siami-Namini, S., Tavakoli, N., and Namin, A.S. (2019, January 9–12). The performance of LSTM and BiLSTM in forecasting time series. Proceedings of the IEEE International Conference on Big Data, Los Angeles, CA, USA.10.1109/BigData47090.2019.9005997 ↗
42. Glorot, X., and Bengio, Y. (2010, January 13–15). Understanding the difficulty of training deep feedforward neural networks. Proceedings of the 13th International Conference on Artificial Intelligence and Statistics, Sardinia, Italy.
43. He, K., Zhang, X., Ren, S., and Sun, J. (2015, January 7–13). Delving deep into rectifiers: Surpassing human-level performance on ImageNet classification. Proceedings of the IEEE International Conference on Computer Vision, Santiago, Chile.10.1109/ICCV.2015.123 ↗
44. Graves, A., Fernandez, S., and Schmidhuber, J. (2005, January 11–15). Bidirectional LSTM networks for improved phoneme classification and recognition. Proceedings of the 15th International Conference on Artificial Neural Networks, Warsaw, Poland.10.1007/11550907_126 ↗
45. Sensitivity analysis of k-fold cross validation in prediction error estimation10.1109/TPAMI.2009.187 ↗
46. Introduction to machine learning: K-nearest neighbors10.21037/atm.2016.03.37 ↗
47. Application of k-nearest neighbor (KNN) approach for predicting economic events: Theoretical background
48. Vapnik, V.N. (2000). The Nature of Statistical Learning Theory, Springer. [2nd ed.].10.1007/978-1-4757-3264-1 ↗
49. Lihong, Z., Ying, S., Yushi, Z., Cheng, Z., and Yi, Z. (2009, January 17–19). Face recognition based on multi-class SVM. Proceedings of the Chinese Control and Decision Conference, Guilin, China.
50. Business analytics using random forest trees for credit risk prediction: A comparison study10.14257/ijast.2014.72.02 ↗
51. Clustering-based speech emotion recognition by incorporating learned features and deep BiLSTM10.1109/ACCESS.2020.2990405 ↗
52. Bidirectional recurrent neural networks10.1109/78.650093 ↗
53. Pasupa, K., and Sunhem, W. (2016, January 5–6). A comparison between shallow and deep architecture classifiers on small dataset. Proceedings of the 8th International Conference on Information Technology and Electrical Engineering, Yogyakarta, Indonesia.10.1109/ICITEED.2016.7863293 ↗
54. Deng, L., and Yu, D. (2014). Deep Learning: Methods and Applications, Now Foundations and Trends. [1st ed.].
55. Classification of COVID-19 in chest X-ray images using DeTraC deep convolutional neural network10.1007/s10489-020-01829-7 ↗
56. Understanding receiver operating characteristic (ROC) curves10.1017/S1481803500013336 ↗
57. Flach, P.A., and Wu, S. (August, January 30). Repairing concavities in ROC curves. Proceedings of the 19th International Joint Conference on Artificial Intelligence, Edinburgh, Scotland.
58. Methods of EEG signal features extraction using linear analysis in frequency and time-frequency domains
59. EEG signals classification based on time frequency analysis10.1142/S0218126617501985 ↗
60. CNN based efficient approach for emotion recognition
61. DEEPHER: Human emotion recognition using an eeg-based deep learning network model
62. Huang, G., and Song, Z. (2021, January 15–17). Analysis of bimodal emotion recognition method based on EEG signals. Proceedings of the 2nd International Seminar on Artificial Intelligence, Networking and Information Technology, Shanghai, China.10.1109/AINIT54228.2021.00051 ↗
63. GameEmo-CapsNet: Emotion recognition from single-channel EEG signals using the 1D capsule networks10.18280/ts.380612 ↗
64. Abdulrahman, A., and Baykara, M. (2021, January 25–27). Feature extraction approach based on statistical methods and wavelet packet decomposition for emotion recognition using EEG signals. Proceedings of the International Conference on Innovations in Intelligent SysTems and Applications, Kocaeli, Turkey.10.1109/INISTA52262.2021.9548406 ↗
65. LEDPatNet19: Automated emotion recognition model based on nonlinear LED pattern feature extraction function using EEG signals10.1007/s11571-021-09748-0 ↗