journal article·2024-01-15Open access

EEG Emotion Classification Based on Graph Convolutional Network

Zhiqiang Fan, Fangyue Chen, Xiaokai Xia, Yu Liu

Applied Sciences· Volume 14· Issue 2

Abstract

EEG-based emotion recognition is a task that uses scalp-EEG data to classify the emotion states of humans. The study of EEG-based emotion recognition can contribute to a large spectrum of application fields including healthcare and human–computer interaction. Recent studies in neuroscience reveal that the brain regions and their interactions play an essential role in the processing of different stimuli and the generation of corresponding emotional states. Nevertheless, such regional interactions, which have been proven to be critical in recognizing emotions in neuroscience, are largely overlooked in existing machine learning or deep learning models, which focus on individual channels in brain signals. Motivated by this, in this paper, we present RGNet, a model that is designed to learn the regional level representation of EEG signal for accurate emotion recognition. Specifically, after applying preprocessing and feature extraction techniques on raw signals, RGNet adopts a novel region-wise encoder to extract the features of channels located within each region as input to compute the regional level features, enabling the model to effectively explore the regional functionality. A graph is then constructed by considering each region as a node and connections between regions as edges, upon which a graph convolutional network is designed with spectral filtering and learned adjacency matrix. Instead of focusing on only the spatial proximity, it allows the model to capture more complex functional relationships. We conducted experiments from the perspective of region division strategies, region encoders and input feature types. Our model has achieved 98.64% and 99.33% for Deap and Dreamer datasets, respectively. The comparison studies show that RGNet outperforms the majority of the existing models for emotion recognition from EEG signals.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: EEG and Brain-Computer Interfaces · Emotion and Mood Recognition · Functional Brain Connectivity Studies

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

Artificial Intelligence Institute of China Electronics Technology Group Corporation, Beijing 100041, China

Beijing Institute of System Engineering, Beijing 100101, China

State Key Laboratory of Software Development Environment, Beihang University, Beijing 100191, China

References (53)

Show source-provided references
1. Improving multimodal fusion with Main Modal Transformer for emotion recognition in conversation10.1016/j.knosys.2022.109978 ↗
2. Self-labeling with feature transfer for speech emotion recognition10.1016/j.knosys.2022.109589 ↗
3. Deep learning based multimodal emotion recognition using model-level fusion of audio–visual modalities10.1016/j.knosys.2022.108580 ↗
4. Intelligent facial emotion recognition using moth-firefly optimization10.1016/j.knosys.2016.08.018 ↗
5. Klonowski, W. (2009). Everything you wanted to ask about EEG but were afraid to get the right answer. Nonlinear Biomed. Phys., 3.10.1186/1753-4631-3-2 ↗
6. Schmorrow, D.D., and Fidopiastis, C.M. (2013). Proceedings of the Foundations of Augmented Cognition, Springer. Lecture Notes in Computer Science.10.1007/978-3-642-39454-6 ↗
7. Garg, A., Kapoor, A., Bedi, A.K., and Sunkaria, R.K. (2019, January 26–28). Merged LSTM Model for emotion classification using EEG signals. Proceedings of the 2019 International Conference on Data Science and Engineering (ICDSE), Patna, India.10.1109/ICDSE47409.2019.8971484 ↗
8. Chen, S., and Jin, Q. (2015, January 26). Multi-modal Dimensional Emotion Recognition using Recurrent Neural Networks. Proceedings of the 5th International Workshop on Audio/Visual Emotion Challenge, Brisbane, Australia.10.1145/2808196.2811638 ↗
9. Regional brain function, emotion and disorders of emotion10.1016/S0959-4388(99)80032-4 ↗
10. Beyond brain regions: Network perspective of cognition-emotion interactions10.1017/S0140525X11001567 ↗
11. Functional grouping and cortical–subcortical interactions in emotion: A meta-analysis of neuroimaging studies10.1016/j.neuroimage.2008.03.059 ↗
12. The brain basis of emotion: A meta-analytic review10.1017/S0140525X11000446 ↗
13. From Regional to Global Brain: A Novel Hierarchical Spatial-Temporal Neural Network Model for EEG Emotion Recognition10.1109/TAFFC.2019.2922912 ↗
14. Ding, Y., Robinson, N., Zeng, Q., Chen, D., Phyo Wai, A.A., Lee, T.S., and Guan, C. (2020, January 19–24). TSception:A Deep Learning Framework for Emotion Detection Using EEG. Proceedings of the 2020 International Joint Conference on Neural Networks (IJCNN), Glasgow, UK.10.1109/IJCNN48605.2020.9206750 ↗
15. A Bi-Hemisphere Domain Adversarial Neural Network Model for EEG Emotion Recognition10.1109/TAFFC.2018.2885474 ↗
16. Ding, Y., Robinson, N., Zeng, Q., and Guan, C. (2022). LGGNet: Learning from Local-Global-Graph Representations for Brain-Computer Interface. arXiv.10.1109/TNNLS.2023.3236635 ↗
17. EEG-Based Emotion Recognition in Music Listening10.1109/TBME.2010.2048568 ↗
18. Emotional state classification from EEG data using machine learning approach10.1016/j.neucom.2013.06.046 ↗
19. Bazgir, O., Mohammadi, Z., and Habibi, S.A.H. (2018, January 29–30). Emotion Recognition with Machine Learning Using EEG Signals. Proceedings of the 2018 25th National and 3rd International Iranian Conference on Biomedical Engineering (ICBME), Qom, Iran.10.1109/ICBME.2018.8703559 ↗
20. Alhalaseh, R., and Alasasfeh, S. (2020). Machine-Learning-Based Emotion Recognition System Using EEG Signals. Computers, 9.10.3390/computers9040095 ↗
21. Emotion Recognition based on EEG using LSTM Recurrent Neural Network
22. Deep learning with convolutional neural networks for EEG decoding and visualization: Convolutional Neural Networks in EEG Analysis10.1002/hbm.23730 ↗
23. EEG-Based Emotion Recognition using 3D Convolutional Neural Networks
24. EEGNet: A compact convolutional neural network for EEG-based brain–computer interfaces10.1088/1741-2552/aace8c ↗
25. Affect recognition from scalp-EEG using channel-wise encoder networks coupled with geometric deep learning and multi-channel feature fusion10.1016/j.knosys.2022.109038 ↗
26. EEG-Based Emotion Recognition Using Regularized Graph Neural Networks10.1109/TAFFC.2020.2994159 ↗
27. EEG Emotion Recognition Using Dynamical Graph Convolutional Neural Networks10.1109/TAFFC.2018.2817622 ↗
28. SAE+LSTM: A New Framework for Emotion Recognition From Multi-Channel EEG10.3389/fnbot.2019.00037 ↗
29. EEG-based Emotion Recognition via Channel-wise Attention and Self Attention10.1109/TAFFC.2020.3025777 ↗
30. EEG emotion recognition using fusion model of graph convolutional neural networks and LSTM10.1016/j.asoc.2020.106954 ↗
31. EEG-based emotion recognition using spatial-temporal graph convolutional LSTM with attention mechanism10.1109/JBHI.2022.3198688 ↗
32. Moontaha, S., Schumann, F.E.F., and Arnrich, B. (2023). Online learning for wearable eeg-based emotion classification. Sensors, 23.10.20944/preprints202301.0156.v1 ↗
33. Li, C., Wang, B., Zhang, S., Liu, Y., Song, R., Cheng, J., and Chen, X. (2022). Emotion recognition from EEG based on multi-task learning with capsule network and attention mechanism. Comput. Biol. Med., 143.10.1016/j.compbiomed.2022.105303 ↗
34. Li, D., Xie, L., Wang, Z., and Yang, H. (2023). Brain emotion perception inspired eeg emotion recognition with deep reinforcement learning. IEEE Trans. Neural Netw. Learn. Syst., 1–14.10.1109/TNNLS.2023.3265730 ↗
35. Yang, Y., Wu, Q., Qiu, M., Wang, Y., and Chen, X. (2018, January 8–13). Emotion Recognition from Multi-Channel EEG through Parallel Convolutional Recurrent Neural Network. Proceedings of the 2018 International Joint Conference on Neural Networks (IJCNN), Rio de Janeiro, Brazil.10.1109/IJCNN.2018.8489331 ↗
36. Mapping Mental Function to Brain Structure: How Can Cognitive Neuroimaging Succeed?10.1177/1745691610388777 ↗
37. On the relationship between emotion and cognition10.1038/nrn2317 ↗
38. Functional specialization does not require a one-to-one mapping between brain regions and emotions10.1017/S0140525X11001749 ↗
39. Neuroimaging Support for Discrete Neural Correlates of Basic Emotions: A Voxel-based Meta-analysis10.1162/jocn.2009.21366 ↗
40. Graph analysis of the human connectome: Promise, progress, and pitfalls10.1016/j.neuroimage.2013.04.087 ↗
41. Complex brain networks: Graph theoretical analysis of structural and functional systems10.1038/nrn2575 ↗
42. Fair, D.A., Cohen, A.L., Power, J.D., Dosenbach, N.U.F., Church, J.A., Miezin, F.M., Schlaggar, B.L., and Petersen, S.E. (2009). Functional Brain Networks Develop from a “Local to Distributed” Organization. PLoS Comput. Biol., 5.10.1371/journal.pcbi.1000381 ↗
43. The emerging field of signal processing on graphs: Extending high-dimensional data analysis to networks and other irregular domains10.1109/MSP.2012.2235192 ↗
44. Defferrard, M., Bresson, X., and Vandergheynst, P. (2016, January 5–10). Convolutional neural networks on graphs with fast localized spectral filtering. Proceedings of the NIPS’16: Proceedings of the 30th International Conference on Neural Information Processing Systems, Barcelona, Spain.
45. DEAP: A Database for Emotion Analysis; Using Physiological Signals10.1109/T-AFFC.2011.15 ↗
46. DREAMER: A Database for Emotion Recognition Through EEG and ECG Signals From Wireless Low-cost Off-the-Shelf Devices10.1109/JBHI.2017.2688239 ↗
47. EEG-based emotion recognition using an end-to-end regional-asymmetric convolutional neural network10.1016/j.knosys.2020.106243 ↗
48. Duan, R.N., Zhu, J.Y., and Lu, B.L. (2013, January 6–8). Differential entropy feature for EEG-based emotion classification. Proceedings of the 2013 6th International IEEE/EMBS Conference on Neural Engineering (NER), San Diego, CA, USA.10.1109/NER.2013.6695876 ↗
49. Investigating critical frequency bands and channels for EEG-based emotion recognition with deep neural networks10.1109/TAMD.2015.2431497 ↗
50. The cerebral sulci and gyri10.3171/2009.11.FOCUS09245 ↗
51. The ten-twenty electrode system of the International Federation. The International Federation of Clinical Neurophysiology
52. Differing emotional response from right and left hemispheres10.1038/261690a0 ↗
53. Approach-withdrawal and cerebral asymmetry: Emotional expression and brain physiology: I10.1037/0022-3514.58.2.330 ↗