GCE: An Audio-Visual Dataset for Group Cohesion and Emotion Analysis
Source abstract
Abstract
We present the Group Cohesion and Emotion (GCE) dataset, which comprises 1029 segmented films sourced from YouTube. These videos encompass a range of interactions, including interviews, meetings, informal discussions, and other similar contexts. In the annotation process, graduate psychology students were tasked with assigning coherence levels, ranging from 1 to 7, and affective states as negative, neutral, or positive for each 30 s film. We introduce a foundational model that utilizes advanced visual and audio embedding techniques to investigate the concepts of group cohesion and group emotion prediction. The application of Multi-Head Attention (MHA) fusion is utilized to enhance the process of cross-representation learning. The scope of our investigation includes both unimodal and multimodal techniques, which provide insights into the prediction of group cohesion and the detection of group emotion. The results emphasize the effectiveness of the GCE dataset in examining the level of group unity and emotional conditions.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Mental Health Research Topics · Media Influence and Health · Action Observation and Synchronization
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Department of AI Convergence, Chonnam National University, Gwangju 61186, Republic of Korea
Department of Psychology, Chonnam National University, Gwangju 61186, Republic of Korea
Funding
National Research Foundation of Korea (NRF) · Institute of Information and Communications Technology Planning and Evaluation (IITP) under the Artificial Intelligence Convergence Innovation Human Resources Development · Korean government (MSIT), and in part by the MSIT (Ministry of Science and ICT), Korea, under the Innovative Human Resource Development for Local Intellectualization support program
References (33)
Show source-provided references
1. The development of an instrument to assess cohesion in sport teams: The Group Environment Questionnaire
2. Gruenfeld, D.H. (1998). Group Emotion: A View from Top and Bottom, Elsevier Science.
3. Constants across cultures in the face and emotion10.1037/h0030377 ↗
4. The circumplex model of affect: An integrative approach to affective neuroscience, cognitive development, and psychopathology10.1017/S0954579405050340 ↗
5. Guo, D., Wang, K., Yang, J., Zhang, K., Peng, X., and Qiao, Y. (2019, January 14–18). Exploring regularizations with face, body and image cues for group cohesion prediction. Proceedings of the 2019 International Conference on Multimodal Interaction, Suzhou, China.10.1145/3340555.3355712 ↗
6. D2C-Based Hybrid Network for Predicting Group Cohesion Scores10.1109/ACCESS.2021.3088340 ↗
7. Zhu, B., Guo, X., Barner, K., and Boncelet, C. (2019, January 14–18). Automatic group cohesiveness detection with multi-modal features. Proceedings of the 2019 International Conference on Multimodal Interaction, Suzhou, China.10.1145/3340555.3355716 ↗
8. Maman, L. (2020, January 25–29). Multimodal Groups’ Analysis for Automated Cohesion Estimation. Proceedings of the 2020 International Conference on Multimodal Interaction, Utrecht, The Netherlands.10.1145/3382507.3421153 ↗
9. Corbellini, N., Ceccaldi, E., Varni, G., and Volpe, G. (2022, January 21–25). An exploratory study on group potency classification from non-verbal social behaviours. Proceedings of the International Conference on Pattern Recognition, Montréal, QC, Canada.10.1007/978-3-031-37660-3_17 ↗
10. Ghosh, S., Dhall, A., and Sebe, N. (2018, January 7–10). Automatic group affect analysis in images via visual attribute and feature networks. Proceedings of the 2018 25th IEEE International Conference on Image Processing (ICIP), Athens, Greece.10.1109/ICIP.2018.8451242 ↗
11. Mou, W., Celiktutan, O., and Gunes, H. (2015, January 4–8). Group-level arousal and valence recognition in static images: Face, body and context. Proceedings of the 2015 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG), Ljubljana, Slovenia.
12. Sharma, G., Ghosh, S., and Dhall, A. (2019, January 3–6). Automatic group level affect and cohesion prediction in videos. Proceedings of the 2019 8th International Conference on Affective Computing and Intelligent Interaction Workshops and Demos (ACIIW), Cambridge, MA, USA.10.1109/ACIIW.2019.8925231 ↗
13. Guo, X., Polania, L., Zhu, B., Boncelet, C., and Barner, K. (2020, January 1–5). Graph neural networks for image understanding based on multiple cues: Group emotion recognition and event recognition as use cases. Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, Snowmass Village, CO, USA.10.1109/WACV45572.2020.9093547 ↗
14. Dhall, A., Joshi, J., Sikka, K., Goecke, R., and Sebe, N. (2015, January 4–8). The more the merrier Analysing the affect of a group of people in images. Proceedings of the 2015 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG), Ljubljana, Slovenia.10.1109/FG.2015.7163151 ↗
15. Ghosh, S., Dhall, A., Sebe, N., and Gedeon, T. (2019, January 14–19). Predicting group cohesiveness in images. Proceedings of the 2019 International Joint Conference on Neural Networks (IJCNN), Budapest, Hungary.10.1109/IJCNN.2019.8852184 ↗
16. Rabiee, H., Haddadnia, J., Mousavi, H., Kalantarzadeh, M., Nabi, M., and Murino, V. (2016, January 23–26). Novel dataset for fine-grained abnormal behavior understanding in crowd. Proceedings of the 2016 13th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS), Colorado Springs, CO, USA.10.1109/AVSS.2016.7738074 ↗
17. Alameda-Pineda, X., Subramanian, R., Ricci, E., Lanz, O., and Sebe, N. (2017). SALSA: A multimodal dataset for the automated analysis of free-standing social interactions. Group and Crowd Behavior for Computer Vision, Academic Press.10.1016/B978-0-12-809276-7.00017-5 ↗
18. The MatchNMingle dataset: A novel multi-sensor resource for the analysis of social interactions and group dynamics in-the-wild during free-standing conversations and speed dates
19. Koutsombogera, M., and Vogel, C. (2018, January 7–12). Modeling collaborative multimodal behavior in group dialogues: The MULTISIMO corpus. Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018), Miyazaki, Japan.
20. Game-on: A multimodal dataset for cohesion and group analysis10.1109/ACCESS.2020.3005719 ↗
21. Zheng, W.-L., Dong, B.-N., and Lu, B.-L. (2014, January 26–30). Multimodal emotion recognition using EEG and eye tracking data. Proceedings of the 2014 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, Chicago, IL, USA.
22. Emotional Evaluation about IAPS in Korean University Students10.19066/cogsci.2009.20.2.004 ↗
23. The structure of group cohesion10.1177/0146167295216003 ↗
24. Likert scale: Explored and explained10.9734/BJAST/2015/14975 ↗
25. Arasenti-tweet: A corpus for arabic sentiment analysis of saudi tweets10.1016/j.procs.2017.10.094 ↗
26. Computing inter-rater reliability for observational data: An overview and tutorial10.20982/tqmp.08.1.p023 ↗
27. Measuring nominal scale agreement among many raters10.1037/h0031619 ↗
28. Florian, S., Dmitry, K., and James, P. (2015, January 7–12). FaceNet: A Unified Embedding for Face Recognition and Clustering. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2015, Boston, MA, USA.
29. Mingxing, T., and Quoc, L. (2019, January 9–15). EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks. Proceedings of the 36th International Conference on Machine Learning 2019, PMLR 97, Long Beach, CA, USA.
30. Christoph, F., Haoqi, F., Jitendra, M., and Kaiming, H. (November, January 27). SlowFast Networks for Video Recognition. Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) 2019, Seoul, Republic of Korea.
31. Florian, E., Martin, W., and Björn, S. (2010, January 25–29). Opensmile: The munich versatile and fast open-source audio feature extractor. Proceedings of the 18th ACM international conference on Multimedia (MM ‘10), Firenze, Italy.
32. wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations
33. Attention is all you need