Affective EEG Decoding Generalizes Across Colormap and Exposure Time
Source abstract
Abstract
Viewing emotional pictures modulates electrocortical activity during the first second, with functional properties that reflect the type of processing that is being carried out. Recently, the investigation of electrocortical activity has been aided by machine learning techniques, such as multivariate pattern analysis (MVPA). Building on previous studies that used MVPA to classify between emotional and neutral stimuli, here we investigate electroencephalographic (EEG) changes while a sample of n = 15 participants viewed emotional and neutral scenes that could be presented in color or in grayscale, and for either a short (24 ms) or a long (6 s) exposure time. A linear classifier was used to classify EEG patterns as consequential to the viewing of emotional (pleasant, unpleasant) vs. neutral scenes, and to assess the extent to which scalp activation patterns are specific to the perceptual conditions under which a scene is viewed (i.e., color or greyscale, short vs. long exposure time) or generalize across viewing conditions. We observed that emotional content could be significantly decoded through MVPA, with earlier classification onset for pleasant-neutral vs. unpleasant-neutral classification. Moreover, this classification generalized across perceptual conditions, indicating that the symbolic meaning of natural scenes drives the emotional modulation of scalp activity. These results further indicate that, within the first second after the onset of natural scenes, emotional states can be decoded from the EEG signal, and that such learning can be applied to flexibly classify emotional states under perceptually different conditions.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Multisensory perception and integration · Emotion and Mood Recognition · Face Recognition and Perception
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Department of Psychology, University of Bologna, Viale Berti Pichat 5, 40127 Bologna, Italy
Center for Mind/Brain Sciences (CIMeC), University of Trento, Corso Bettini 31, 38068 Rovereto, Italy
Department of Medicine and Surgery, University of Parma, Via Volturno 39, 43125 Parma, Italy
References (80)
Show source-provided references
1. Decoding Neural Representations of Affective Scenes in Retinotopic Visual Cortex10.1093/cercor/bhaa411 ↗
2. Decoding the Temporal Dynamics of Affective Scene Processing10.1016/j.neuroimage.2022.119532 ↗
3. Cacioppo, J.T., Tassinary, L.G., and Berntson, G. (2007). Emotion and Motivation. Handbook of Psychophysiology, Cambridge University Press.
4. Arnold, M.B. (1970). Feelings and Emotions, Academia Press.
5. Image as Action: A Reply to Watts and Blackstock10.1080/02699938708408060 ↗
6. Frijda, N.H. (1986). The Emotions, Cambridge University Press. Studies in emotion and social interaction.
7. Advances in EEG-Based Emotion Recognition: Challenges, Methodologies, and Future Directions10.1016/j.asoc.2025.113478 ↗
8. Investigating Critical Frequency Bands and Channels for EEG-Based Emotion Recognition with Deep Neural Networks10.1109/TAMD.2015.2431497 ↗
9. DEAP: A Database for Emotion Analysis; Using Physiological Signals10.1109/T-AFFC.2011.15 ↗
10. DREAMER: A Database for Emotion Recognition Through EEG and ECG Signals From Wireless Low-Cost Off-the-Shelf Devices10.1109/JBHI.2017.2688239 ↗
11. Emotion Recognition in EEG Signals Using Deep Learning Methods: A Review10.1016/j.compbiomed.2023.107450 ↗
12. EEG-Based Emotion Recognition Using 4D Convolutional Recurrent Neural Network10.1007/s11571-020-09634-1 ↗
13. 4D Attention-Based Neural Network for EEG Emotion Recognition10.1007/s11571-021-09751-5 ↗
14. EEG-Based Emotion Recognition Using a Temporal-Difference Minimizing Neural Network10.1007/s11571-023-10004-w ↗
15. Domain Adversarial Learning with Multiple Adversarial Tasks for EEG Emotion Recognition10.1016/j.eswa.2024.126028 ↗
16. Contrastive Learning of EEG Representation of Brain Area for Emotion Recognition10.1109/TIM.2025.3533618 ↗
17. Decoding Driving States Based on Normalized Mutual Information Features and Hyperparameter Self-Optimized Gaussian Kernel-Based Radial Basis Function Extreme Learning Machine10.1016/j.chaos.2025.116751 ↗
18. Driver Fatigue Detection Using EEG-Based Graph Attention Convolutional Neural Networks: An End-to-End Learning Approach with Mutual Information-Driven Connectivity10.1016/j.asoc.2025.114097 ↗
19. Handy, T.C. (2005). EEG Oscillations and Wavelet Analysis. Event-Related Potentials: A Methods Handbook, MIT Press. A Bradford book.
20. Fleeting Images: A New Look at Early Emotion Discrimination10.1111/1469-8986.3820175 ↗
21. Attention and Emotion: An ERP Analysis of Facilitated Emotional Stimulus Processing10.1097/00001756-200306110-00002 ↗
22. Emotional Facilitation of Sensory Processing in the Visual Cortex10.1111/1467-9280.01411 ↗
23. Implicit and Explicit Categorization of Natural Scenes10.1016/S0079-6123(06)56003-0 ↗
24. Brain Potentials in Affective Picture Processing: Covariation with Autonomic Arousal and Affective Report10.1016/S0301-0511(99)00044-7 ↗
25. Multiple P3s to Emotional Stimuli and Their Theoretical Significance10.1111/j.1469-8986.1986.tb00694.x ↗
26. Large-scale Neural Correlates of Affective Picture Processing10.1111/1469-8986.3950641 ↗
27. The Late Positive Component of Visual Evoked Response Sensitive to Emotional Factors
28. Brain Potentials in Perception: Picture Complexity and Emotional Arousal10.1111/j.1469-8986.2007.00520.x ↗
29. When Does Size Not Matter? Effects of Stimulus Size on Affective Modulation10.1111/j.1469-8986.2006.00392.x ↗
30. Repetition and Event-Related Potentials: Distinguishing Early and Late Processes in Affective Picture Perception10.1162/jocn.2007.19.4.577 ↗
31. Repetitive Exposure: Brain and Reflex Measures of Emotion and Attention10.1111/j.1469-8986.2010.01083.x ↗
32. Repetition and ERPs during Emotional Scene Processing: A Selective Review10.1016/j.ijpsycho.2016.07.496 ↗
33. Dissociable Decoding of Spatial Attention and Working Memory from EEG Oscillations and Sustained Potentials10.1523/JNEUROSCI.2860-17.2017 ↗
34. Decoding Motion Direction Using the Topography of Sustained ERPs and Alpha Oscillations10.1016/j.neuroimage.2018.09.029 ↗
35. Decoding Dynamic Brain Patterns from Evoked Responses: A Tutorial on Multivariate Pattern Analysis Applied to Time Series Neuroimaging Data10.1162/jocn_a_01068 ↗
36. Choosing Prediction Over Explanation in Psychology: Lessons From Machine Learning10.1177/1745691617693393 ↗
37. Fahrenfort, J.J., Van Driel, J., Van Gaal, S., and Olivers, C.N.L. (2018). From ERPs to MVPA Using the Amsterdam Decoding and Modeling Toolbox (ADAM). Front. Neurosci., 12.10.3389/fnins.2018.00368 ↗
38. Beyond Mind-Reading: Multi-Voxel Pattern Analysis of fMRI Data10.1016/j.tics.2006.07.005 ↗
39. The Time Course of Cross-Modal Representations of Conceptual Categories10.1016/j.neuroimage.2023.120254 ↗
40. Decoding Neural Patterns for the Processing of Fearful Faces under Different Visual Awareness Conditions: A Multivariate Pattern Analysis10.1111/psyp.14368 ↗
41. Decoding Emotion from High-Frequency Steady State Visual Evoked Potential (SSVEP)10.1016/j.jneumeth.2023.109919 ↗
42. The Influence of Color on Emotional Perception of Natural Scenes10.1111/j.1469-8986.2011.01284.x ↗
43. G*Power 3: A Flexible Statistical Power Analysis Program for the Social, Behavioral, and Biomedical Sciences10.3758/BF03193146 ↗
44. Cohen, J. (1988). Statistical Power Analysis for the Behavioral Sciences, Lawrence Erlbaum Associates. [2nd ed.].
45. Zeigler-Hill, V., and Shackelford, T.K. (2017). International Affective Picture System. Encyclopedia of Personality and Individual Differences, Springer International Publishing.10.1007/978-3-319-28099-8 ↗
46. Schneider, W., Eschman, A., and Zuccolotto, A. (2012). E-Prime Reference Guide, Psychology Software Tools, Inc.
47. On the Time Course of Perceptual Information That Results from a Brief Visual Presentation10.1037/0096-1523.18.2.530 ↗
48. Sidowski, J.B., Johnson, J.H., and Williams, T.A. (1980). Behavioral Treatment and Bio-Behavioral Assessment: Computer Applications. Technology in Mental Health Care Delivery Systems, Ablex Pub. Corp.
49. Carlson, T.A., Grootswagers, T., and Robinson, A.K. (2019). An Introduction to Time-Resolved Decoding Analysis for M/EEG. arXiv.10.7551/mitpress/11442.003.0075 ↗
50. Attenuated Conflict Self-Referential Information Facilitating Conflict Resolution10.1038/s41539-024-00256-4 ↗
51. Internal Noise in Contrast Discrimination Propagates Forwards from Early Visual Cortex10.1016/j.neuroimage.2019.02.049 ↗
52. High-Pass Filtering Artifacts in Multivariate Classification of Neural Time Series Data10.1016/j.jneumeth.2021.109080 ↗
53. VanRullen, R., Busch, N.A., Drewes, J., and Dubois, J. (2011). Ongoing EEG Phase as a Trial-by-Trial Predictor of Perceptual and Attentional Variability. Front. Psychol., 2.10.3389/fpsyg.2011.00060 ↗
54. On the Interpretation of Weight Vectors of Linear Models in Multivariate Neuroimaging10.1016/j.neuroimage.2013.10.067 ↗
55. The Use of the Area under the ROC Curve in the Evaluation of Machine Learning Algorithms10.1016/S0031-3203(96)00142-2 ↗
56. Nonparametric Statistical Testing of EEG- and MEG-Data10.1016/j.jneumeth.2007.03.024 ↗
57. Brain Processes in Emotional Perception: Motivated Attention10.1080/02699930341000239 ↗
58. Neural Response to Emotional Pictures Is Unaffected by Concurrent Task Difficulty: An Event-Related Potential Study10.1037/0735-7044.121.6.1156 ↗
59. Aversive Picture Processing: Effects of a Concurrent Task on Sustained Defensive System Engagement10.1111/j.1469-8986.2010.01041.x ↗
60. Alpha-band Oscillations and Emotion: A Review of Studies on Picture Perception10.1111/psyp.14438 ↗
61. Open Science Collaboration (2015). Estimating the Reproducibility of Psychological Science. Science, 349, aac4716.10.1126/science.aac4716 ↗
62. A Case-by-Case Analysis of EPN and LPP Components within a “One-Picture-per-Emotion-Category” Protocol10.1111/psyp.14718 ↗
63. Case-by-case: Emotional Stimulus Significance and the Modulation of the EPN and LPP10.1111/psyp.13766 ↗
64. Loftus, G.R. (2010). Null Hypothesis. Encyclopedia of Research Design, SAGE Publications, Inc.
65. Looking at Pictures: Affective, Facial, Visceral, and Behavioral Reactions10.1111/j.1469-8986.1993.tb03352.x ↗
66. Time Will Tell: Object Categorization and Emotional Engagement during Processing of Degraded Natural Scenes10.1111/psyp.13704 ↗
67. Scene Identification and Emotional Response: Which Spatial Frequencies Are Critical?10.1523/JNEUROSCI.3745-11.2011 ↗
68. Affective Valence, Stimulus Attributes, and P300: Color vs. Black/White and Normal vs. Scrambled Images10.1016/j.ijpsycho.2008.07.016 ↗
69. Bringing Color to Emotion: The Influence of Color on Attentional Bias to Briefly Presented Emotional Images10.3758/s13415-017-0530-z ↗
70. Kuniecki, M., Pilarczyk, J., and Wichary, S. (2015). The Color Red Attracts Attention in an Emotional Context. An ERP Study. Front. Hum. Neurosci., 9.10.3389/fnhum.2015.00212 ↗
71. Understanding Natural Scenes: Contributions of Image Statistics10.1016/j.neubiorev.2017.01.012 ↗
72. Spatial Frequencies and Emotional Perception
73. Mastria, S., Codispoti, M., Tronelli, V., and De Cesarei, A. (2024). Subjective Affective Responses to Natural Scenes Require Understanding, Not Spatial Frequency Bands. Vision, 8.10.3390/vision8020036 ↗
74. On the Latency of Object Recognition and Affect: Evidence from Temporal Order and Simultaneity Judgments10.1037/xge0001244 ↗
75. Semantic Processing Precedes Affect Retrieval: The Neurological Case for Cognitive Primacy in Visual Processing10.1037/1089-2680.10.1.41 ↗
76. Bad Is Stronger than Good10.1037/1089-2680.5.4.323 ↗
77. Spatio-Temporal Attention toward Emotional Scenes across Adulthood10.1037/emo0000891 ↗
78. Decoding the Brain: Neural Representation and the Limits of Multivariate Pattern Analysis in Cognitive Neuroscience10.1093/bjps/axx023 ↗
79. Garg, M., and Prasad, R.S. (2024). The Science of Emotion: Decoding and Analysis of Human Emotional Landscape. Affective Computing for Social Good, Springer Nature. The Springer Series in Applied Machine Learning.10.1007/978-3-031-63821-3 ↗
80. Characterizing the Dynamics of Mental Representations: The Temporal Generalization Method10.1016/j.tics.2014.01.002 ↗