A Method for Automatic Emotion Detection Through Machine Learning
Source abstract
Abstract
Facial expression recognition (FER) is a fundamental component of Affective Computing and is gaining increasing relevance in mental health applications. This study presents an approach for facial expression recognition using feature extraction and machine learning techniques. Starting from a publicly available dataset, a manual cleaning and relabeling process led to the creation of a refined dataset of 35,625 facial images grouped into four emotional macroclasses. Features were extracted using the SqueezeNet and Inception v3 embedders and classified using various algorithms. The experimental results show that Inception v3 consistently outperforms SqueezeNet and that feature normalization improves classification stability and robustness. The results highlight the importance of data quality and preprocessing in applied FER systems.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Emotion and Mood Recognition · Mental Health via Writing · Sentiment Analysis and Opinion Mining
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Department of Medicine and Health Sciences “Vincenzo Tiberio”, University of Molise, 86100 Campobasso, Italy
Department of Engineering, University of Sannio, 82100 Benevento, Italy
ICAR-CNR, 87036 Rende, Italy
Funding
EU DUCA · Health Operational Plan · PRIN-MUR-Ministry of Health, Progetto MolisCTe, Ministero delle Imprese e del Made in Italy · FORESEEN: FORmal mEthodS for attack dEtEction in autonomous driviNg systems · ALOHA: a framework for monitoring the physical and psychological health status of the Worker through Object detection and federated machine learning, Call for Collaborative Research BRiC -2024, INAIL
References (60)
Show source-provided references
1. Campeau, S., Falls, W.A., Cullinan, W., and Picard, R.W. (1997). Informatica Affettiva, Cambridge University Press.
2. Affective Computing: Recent advances, challenges, and future trends10.34133/icomputing.0076 ↗
3. Cesarelli, M., Martinelli, F., Mercaldo, F., and Santone, A. (2022). Emotion recognition from facial expression using explainable deep learning. 2022 IEEE International Conference on Dependable, Autonomic and Secure Computing, International Conference on Pervasive Intelligence and Computing, International Conference on Cloud and Big Data Computing, International Conference on Cyber Science and Technology Congress (DASC/PiCom/CBDCom/CyberSciTech), Calabria, Italy, 12–15 September 2022, IEEE.10.1109/DASC/PiCom/CBDCom/Cy55231.2022.9927951 ↗
4. MMATrans: Muscle movement aware representation learning for facial expression recognition via transformers10.1109/TII.2024.3431640 ↗
5. NGDNet: Nonuniform Gaussian-label distribution learning for infrared head pose estimation and on-task behavior understanding in the classroom10.1016/j.neucom.2020.12.090 ↗
6. Liu, H., Deng, L., Liu, T., Meng, R., Zhang, Z., and Li, Y.F. (2024). ACAForms: Learning Adaptive Context-Aware Feature for Facial Expression Recognition in Human-Robot Interaction. Proceedings of the 2024 9th International Conference on Robotics and Automation Engineering (ICRAE), Singapore, 15–17 November 2024, IEEE.10.1109/ICRAE64368.2024.10851638 ↗
7. Not in my face: Challenges and ethical considerations in automatic face emotion recognition technology10.3390/make6040109 ↗
8. A survey on facial emotion recognition techniques: A state-of-the-art literature review10.1016/j.ins.2021.10.005 ↗
9. Pan-cultural elements in facial displays of emotion10.1126/science.164.3875.86 ↗
10. Facial recognition technology in schools: Critical questions and concerns10.1080/17439884.2020.1686014 ↗
11. Eigenfaces for recognition10.1162/jocn.1991.3.1.71 ↗
12. Dimino, G. (2025, December 18). Introduction to Modern Artificial Intelligence Techniques. Elettronica e Telecomunicazioni, 2020. Available online: http://www.crit.rai.it/eletel/2020-1/201-2.pdf.
13. Facial emotion recognition methods, datasets and technologies: A literature survey10.1016/j.matpr.2021.07.046 ↗
14. Trends and challenges of image analysis in facial emotion recognition: A review10.1007/s13721-022-00376-0 ↗
15. Jones, M., and Viola, P. (2003). Fast Multi-View Face Detection, Mitsubishi Electric Research Laboratories, Inc.. Mitsubishi Electric Research Lab TR-20003-96.
16. Soo, S. (2014). Object Detection Using Haar-Cascade Classifier, Institute of Computer Science, University of Tartu.
17. Real time face recognition using AdaBoost improved fast PCA algorithm
18. Rajesh, K., and Naveenkumar, M. (2016, January 9–10). A robust method for face recognition and face emotion detection system using support vector machines. Proceedings of the 2016 International Conference on Electrical, Electronics, Communication, Computer and Optimization Techniques (ICEECCOT), Mysuru, India.10.1109/ICEECCOT.2016.7955175 ↗
19. Wang, Y., Li, Y., Song, Y., and Rong, X. (2019). Facial expression recognition based on random forest and convolutional Neural Network. Information, 10.10.3390/info10120375 ↗
20. Active affective state detection and user assistance with dynamic Bayesian Networks10.1109/TSMCA.2004.838454 ↗
21. A comprehensive review of facial expression recognition techniques10.1007/s00530-022-00984-w ↗
22. Facial expression recognition based on local region specifc features and support vector machines10.1007/s11042-016-3418-y ↗
23. Face description with local binary patterns: Application to face recognition10.1109/TPAMI.2006.244 ↗
24. Learning multiscale active facial patches for expression analysis10.1109/TCYB.2014.2354351 ↗
25. Less is more: Micro-expression recognition from video using apex frame
26. Extended local binary patterns for efficient and robust spontaneous facial micro-expression recognition10.1109/ACCESS.2019.2942358 ↗
27. Micro expression recognition using delaunay triangulation and voronoi tessellation10.1080/03772063.2022.2068680 ↗
28. Multi-objective based spatio-temporal feature representation learning robust to expression intensity variations for facial expression recognition10.1109/TAFFC.2017.2695999 ↗
29. Facial expression recognition based on deep evolutional spatial-temporal networks10.1109/TIP.2017.2689999 ↗
30. Deep spatial-temporal feature fusion for facial expression recognition in static images10.1016/j.patrec.2017.10.022 ↗
31. Local learning with deep and handcrafted features for facial expression recognition10.1109/ACCESS.2019.2917266 ↗
32. Ruan, D., Yan, Y., Lai, S., Chai, Z., Shen, C., and Wang, H. (2021, January 20–25). Feature decomposition and reconstruction learning for effective facial expression recognition. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA.10.1109/CVPR46437.2021.00757 ↗
33. Cai, J., Meng, Z., Khan, A.S., O’Reilly, J., Li, Z., Han, S., and Tong, Y. (2021). Identity-free facial expression recognition using conditional generative adversarial network. Proceedings of the 2021 IEEE International Conference on Image Processing (ICIP), Anchorage, AK, USA, 19–22 September 2021, IEEE.10.1109/ICIP42928.2021.9506593 ↗
34. Clip-aware expressive feature learning for video-based facial expression recognition10.1016/j.ins.2022.03.062 ↗
35. Deep multi-path convolutional neural network joint with salient region attention for facial expression recognition10.1016/j.patcog.2019.03.019 ↗
36. Reliable crowdsourcing and deep locality-preserving learning for unconstrained facial expression recognition10.1109/TIP.2018.2868382 ↗
37. A roi-guided deep architecture for robust facial expressions recognition10.1016/j.ins.2020.02.047 ↗
38. Krizhevsky, A., Sutskever, I., and Hinton, G.E. (2012). Imagenet classification with deep convolutional neural networks. Adv. Neural Inf. Process. Syst., 25.
39. Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., and Rabinovich, A. (2015, January 7–12). Going deeper with convolutions. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA.10.1109/CVPR.2015.7298594 ↗
40. Three convolutional neural network models for facial expression recognition in the wild10.1016/j.neucom.2019.05.005 ↗
41. Safarov, F., Kutlimuratov, A., Khojamuratova, U., Abdusalomov, A., and Cho, Y.I. (2025). Enhanced AlexNet with Gabor and Local Binary Pattern Features for Improved Facial Emotion Recognition. Sensors, 25.10.3390/s25123832 ↗
42. A fine-grained human facial key feature extraction and fusion method for emotion recognition10.1038/s41598-025-90440-2 ↗
43. So, J., and Han, Y. (2025). Facial Landmark-Driven Keypoint Feature Extraction for Robust Facial Expression Recognition. Sensors, 25.10.3390/s25123762 ↗
44. Navigating label ambiguity for facial expression recognition in the wild
45. Head pose estimation: An extensive survey on recent techniques and applications10.1016/j.patcog.2022.108591 ↗
46. A novel eye center localization method for head poses with large rotations10.1109/TIP.2020.3044209 ↗
47. Kazemi, V., and Sullivan, J. (2014). One millisecond face alignment with an ensemble of regression trees. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA, 23–28 June 2014, IEEE.10.1109/CVPR.2014.241 ↗
48. Hempel, T., Abdelrahman, A.A., and Al-Hamadi, A. (2022). 6d rotation representation for unconstrained head pose estimation. Proceedings of the 2022 IEEE International Conference on Image Processing (ICIP), Bordeaux, France, 16–19 October 2022, IEEE.10.1109/ICIP46576.2022.9897219 ↗
49. Orientation cues-aware facial relationship representation for head pose estimation via transformer10.1109/TIP.2023.3331309 ↗
50. Narayan, K., VS, V., Chellappa, R., and Patel, V.M. (2025, January 19–23). Facexformer: A unified transformer for facial analysis. Proceedings of the IEEE/CVF International Conference on Computer Vision, Honolulu, HI, USA.
51. Transformer-based weakly supervised 3D human pose estimation10.1016/j.jvcir.2025.104432 ↗
52. Wang, A., Wang, M., Jiang, K., Cao, M., and Iwahori, Y. (2019). A dual neural architecture combined SqueezeNet with OctConv for LiDAR data classification. Sensors, 19.10.3390/s19224927 ↗
53. Genetic hyperparameter optimization with Modified Scalable-Neighbourhood Component Analysis for breast cancer prognostication10.1016/j.neunet.2023.02.035 ↗
54. (2025, September 01). Available online: https://orangedatamining.com/widget-catalog/model/logisticregression/.
55. (2025, September 01). Available online: https://orangedatamining.com/widget-catalog/model/neuralnetwork/.
56. (2025, September 01). Available online: https://xgboost.readthedocs.io/en/latest/index.html.
57. (2025, September 01). Available online: https://orangedatamining.com/widget-catalog/model/randomforest/.
58. (2025, September 01). Available online: https://www.kaggle.com/datasets/ananthu017/emotion-detection-fer/data.
59. (2025, September 01). Available online: https://orangedatamining.com/download/.
60. Cai, Y., Li, X., and Li, J. (2023). Emotion recognition using different sensors, emotion models, methods and datasets: A comprehensive review. Sensors, 23.10.3390/s23052455 ↗