DevEmo—Software Developers’ Facial Expression Dataset
Source abstract
Abstract
The COVID-19 pandemic has increased the relevance of remote activities and digital tools for education, work, and other aspects of daily life. This reality has highlighted the need for emotion recognition technology to better understand the emotions of computer users and provide support in remote environments. Emotion recognition can play a critical role in improving the remote experience and ensuring that individuals are able to effectively engage in computer-based tasks remotely. This paper presents a new dataset, DevEmo, that can be used to train deep learning models for the purpose of emotion recognition of computer users. The dataset consists of 217 video clips of 33 students solving programming tasks. The recordings were collected in the participants’ actual work environment, capturing the students’ facial expressions as they engaged in programming tasks. The DevEmo dataset is labeled to indicate the presence of the four emotions (anger, confusion, happiness, and surprise) and a neutral state. The dataset provides a unique opportunity to explore the relationship between emotions and computer-related activities, and has the potential to support the development of more personalized and effective tools for computer-based learning environments.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Emotion and Mood Recognition · EEG and Brain-Computer Interfaces · Gaze Tracking and Assistive Technology
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Faculty of Electronics, Telecommunications and Informatics, Gdansk University of Technology, 80-233 Gdańsk, Poland
References (29)
Show source-provided references
1. Methodology of affective intervention design for intelligent systems10.1093/iwc/iwv047 ↗
2. A survey on data collection for machine learning: A big data-ai integration perspective10.1109/TKDE.2019.2946162 ↗
3. Chollet, F. (2017, January 21–26). Xception: Deep learning with depthwise separable convolutions. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.10.1109/CVPR.2017.195 ↗
4. Transfer learning using vgg-16 with deep convolutional neural network for classifying images
5. Detecting affect states using VGG16, ResNet50 and SE-ResNet50 networks10.1007/s42979-020-0114-9 ↗
6. Lyons, M.J., Akamatsu, S., Kamachi, M., Gyoba, J., and Budynek, J. (1998, January 14–16). The Japanese female facial expression (JAFFE) database. Proceedings of the third International Conference on Automatic Face and Gesture Recognition, Nara, Japan.
7. The Karolinska directed emotional faces: A validation study10.1080/02699930701626582 ↗
8. Multi-pie10.1016/j.imavis.2009.08.002 ↗
9. Presentation and validation of the Radboud Faces Database10.1080/02699930903485076 ↗
10. Compound facial expressions of emotion10.1073/pnas.1322355111 ↗
11. Barsoum, E., Zhang, C., Ferrer, C.C., and Zhang, Z. (2016, January 12–16). Training deep networks for facial expression recognition with crowd-sourced label distribution. Proceedings of the 18th ACM International Conference on Multimodal Interaction, Tokyo, Japan.10.1145/2993148.2993165 ↗
12. Fabian Benitez-Quiroz, C., Srinivasan, R., and Martinez, A.M. (2016, January 27–30). Emotionet: An accurate, real-time algorithm for the automatic annotation of a million facial expressions in the wild. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.10.1109/CVPR.2016.600 ↗
13. Affectnet: A database for facial expression, valence, and arousal computing in the wild10.1109/TAFFC.2017.2740923 ↗
14. Dhall, A., Goecke, R., Lucey, S., and Gedeon, T. (2011, January 6–13). Static facial expression analysis in tough conditions: Data, evaluation protocol and benchmark. Proceedings of the 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops), Barcelona, Spain.10.1109/ICCVW.2011.6130508 ↗
15. Pantic, M., Valstar, M., Rademaker, R., and Maat, L. (2005, January 6). Web-based database for facial expression analysis. Proceedings of the 2005 IEEE international conference on multimedia and Expo, Amsterdam, The Netherlands.
16. Lucey, P., Cohn, J.F., Kanade, T., Saragih, J., Ambadar, Z., and Matthews, I. (2010, January 13–18). The extended cohn-kanade dataset (ck+): A complete dataset for action unit and emotion-specified expression. Proceedings of the 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition-Workshops, San Francisco, CA, USA.10.1109/CVPRW.2010.5543262 ↗
17. Deap: A database for emotion analysis; using physiological signals10.1109/T-AFFC.2011.15 ↗
18. The belfast induced natural emotion database10.1109/T-AFFC.2011.26 ↗
19. Ringeval, F., Sonderegger, A., Sauer, J., and Lalanne, D. (2013, January 22–26). Introducing the RECOLA multimodal corpus of remote collaborative and affective interactions. Proceedings of the 2013 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG), Shanghai, China.10.1109/FG.2013.6553805 ↗
20. The Indian spontaneous expression database for emotion recognition10.1109/TAFFC.2015.2498174 ↗
21. Zafeiriou, S., Kollias, D., Nicolaou, M.A., Papaioannou, A., Zhao, G., and Kotsia, I. (2017, January 21–26). Aff-wild: Valence and arousal’In-the-Wild’challenge. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, Honolulu, HI, USA.10.1109/CVPRW.2017.248 ↗
22. Livingstone, S.R., and Russo, F.A. (2018). The Ryerson Audio-Visual Database of Emotional Speech and Song (RAVDESS): A dynamic, multimodal set of facial and vocal expressions in North American English. PLoS ONE, 13.10.1371/journal.pone.0196391 ↗
23. A spontaneous driver emotion facial expression (defe) dataset for intelligent vehicles: Emotions triggered by video-audio clips in driving scenarios10.1109/TAFFC.2021.3063387 ↗
24. Wrobel, M.R. (2018). Applicability of emotion recognition and induction methods to study the behavior of programmers. Appl. Sci., 8.10.3390/app8030323 ↗
25. BORIS: A free, versatile open-source event-logging software for video/audio coding and live observations10.1111/2041-210X.12584 ↗
26. Brodny, G., Kołakowska, A., Landowska, A., Szwoch, M., Szwoch, W., and Wróbel, M.R. (2016, January 6–8). Comparison of selected off-the-shelf solutions for emotion recognition based on facial expressions. Proceedings of the 2016 9th International Conference on Human System Interactions (HSI), Portsmouth, UK.10.1109/HSI.2016.7529664 ↗
27. Borsos, Z., Jakab, Z., Stefanik, K., Bogdán, B., and Gyori, M. (2022). Test–Retest Reliability in Automated Emotional Facial Expression Analysis: Exploring FaceReader 8.0 on Data from Typically Developing Children and Children with Autism. Appl. Sci., 12.10.3390/app12157759 ↗
28. Wohlin, C., Runeson, P., Höst, M., Ohlsson, M.C., Regnell, B., and Wesslén, A. (2012). Experimentation in Software Engineering, Springer Science & Business Media.10.1007/978-3-642-29044-2 ↗
29. Understanding the Hawthorne effect10.1136/bmj.h4672 ↗