Comparative Analysis of AI-Based Facial Identification and Expression Recognition Using Upper and Lower Facial Regions
Source abstract
Abstract
The COVID-19 pandemic has significantly impacted society, having led to a lack of social skills in children who became used to interacting with others while wearing masks. To analyze this issue, we investigated the effects of masks on face identification and facial expression recognition, using deep learning models for these operations. The results showed that when using the upper or lower facial regions for face identification, the upper facial region allowed for an accuracy of 81.36%, and the lower facial region allowed for an accuracy of 55.52%. Regarding facial expression recognition, the upper facial region allowed for an accuracy of 39% compared to 49% for the lower facial region. Furthermore, our analysis was conducted for a number of facial expressions, and specific emotions such as happiness and contempt were difficult to distinguish using only the upper facial region. Because this study used a model trained on data generated from human labeling, it is assumed that the effects on humans would be similar. Therefore, this study is significant because it provides engineering evidence of a decline in facial expression recognition; however, wearing masks does not cause difficulties in identification.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Face recognition and analysis · Emotion and Mood Recognition · Face and Expression Recognition
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Department of AI & Informatics, Graduate School, Sangmyung University, Hongjimun 2-gil 20, Jongno-gu, Seoul 03016, Republic of Korea
Department of Human-Centered Artificial Intelligence, Graduate School, Sangmyung University, Hongjimun 2-gil 20, Jongno-gu, Seoul 03016, Republic of Korea
Funding
Sangmyung University
References (31)
Show source-provided references
1. The Role of Face Masks in the Recognition of Emotions by Preschool Children10.1001/jamapediatrics.2021.4556 ↗
2. Age and social competence in preschoolers’ decoding of facial expression10.1111/j.2044-8309.1990.tb00885.x ↗
3. Ejaz, S., Islam, R., and Sarker, A. (2019, January 3–5). Implementation of Principal Component Analysis on Masked and Non-masked Face Recognition. Proceedings of the 2019 1st International Conference on Advances in Science, Engineering and Robotics Technology, Dhaka, Bangladesh.10.1109/ICASERT.2019.8934543 ↗
4. Recognizing action units for facial expression analysis10.1109/34.908962 ↗
5. Park, S., and Wallraven, C. (2021, January 9–12). Comparing Facial Expression Recognition in Humans and Machines: Using CAM, GradCAM, and Extremal Perturbation. Proceedings of the Pattern Recognition: 6th Asian Conference, ACPR 2021, Jeju Island, Republic of Korea.10.1007/978-3-031-02375-0_30 ↗
6. Comparison of Computer and Human Face Recognition According to Facial Components10.9717/kmms.2012.15.1.040 ↗
7. (2023, April 21). Comparison of Human and Computer Performance across Face Recognition Experiments—ScienceDirect. Available online: https://www.sciencedirect.com/science/article/pii/S0262885613001741.
8. The contribution of the upper and lower face in happy and sad facial expression classification10.1016/j.visres.2010.06.002 ↗
9. (2023, April 21). Abrosoft FantaMorph—Photo Morphing Software for Creating Morphing Photos and Animations. Available online: https://www.fantamorph.com/.
10. Relative importance of upper and lower parts of the face in recognizing facial expressions of emotion10.4189/shes.9.89 ↗
11. Facial Expressions Recognition in a Single Static as well as Dynamic Facial Images Using Tracking and Probabilistic Neural Networks10.1007/11949534_29 ↗
12. Khoeun, R., Chophuk, P., and Chinnasarn, K. (2022). Emotion Recognition for Partial Faces Using a Feature Vector Technique. Sensors, 22.10.3390/s22124633 ↗
13. Deng, H., Feng, Z., Qian, G., Lv, X., Li, H., and Li, G. (2021). MFCosface: A Masked-Face Recognition Algorithm Based on Large Margin Cosine Loss. Appl. Sci., 11.10.3390/app11167310 ↗
14. Mukhiddinov, M., Djuraev, O., Akhmedov, F., Mukhamadiyev, A., and Cho, J. (2023). Masked Face Emotion Recognition Based on Facial Landmarks and Deep Learning Approaches for Visually Impaired People. Sensors, 23.10.3390/s23031080 ↗
15. Pann, V., and Lee, H.J. (2022). Effective Attention-Based Mechanism for Masked Face Recognition. Appl. Sci., 12.10.3390/app12115590 ↗
16. Face masks disrupt holistic processing and face perception in school-age children10.1186/s41235-022-00360-2 ↗
17. Deep face recognition: A survey10.1016/j.neucom.2020.10.081 ↗
18. Schroff, F., Kalenichenko, D., and Philbin, J. (2015, January 7–12). FaceNet: A Unified Embedding for Face Recognition and Clustering. Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA.10.1109/CVPR.2015.7298682 ↗
19. Parkhi, O.M., Vedaldi, A., and Zisserman, A. (2015, January 7–10). Deep Face Recognition. Proceedings of the British Machine Vision Conference 2015, Swansea, UK.10.5244/C.29.41 ↗
20. ArcFace: Additive Angular Margin Loss for Deep Face Recognition10.1109/TPAMI.2021.3087709 ↗
21. He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27–30). Deep residual learning for image recognition. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.10.1109/CVPR.2016.90 ↗
22. Leibe, B., Matas, J., and Sebe, N. (2016, January 11–14). MS-Celeb-1M: A Dataset and Benchmark for Large-Scale Face Recognition. Proceedings of the European Conference on Computer Vision, Amsterdam, The Netherlands.
23. AffectNet: A Database for Facial Expression, Valence, and Arousal Computing in the Wild10.1109/TAFFC.2017.2740923 ↗
24. Estimation of Continuous Valence and Arousal Levels from Faces in Naturalistic Conditions10.1038/s42256-020-00280-0 ↗
25. Classifying Emotions and Engagement in Online Learning Based on a Single Facial Expression Recognition Neural Network10.1109/TAFFC.2022.3188390 ↗
26. (2023, April 11). The Latest in Machine Learning|Papers with Code. Available online: https://paperswithcode.com/.
27. Joint Face Detection and Alignment Using Multitask Cascaded Convolutional Networks10.1109/LSP.2016.2603342 ↗
28. Huang, G.B., Ramesh, M., Berg, T., and Learned-Miller, E. (2007). Labeled Faces in the Wild: A Database for Studying Face Recognition in Unconstrained Environments, University of Massachusetts.
29. InfoGAN: Interpretable Representation Learning by In-formation Maximizing Generative Adversarial Nets
30. Moschoglou, S., Papaioannou, A., Sagonas, C., Deng, J., Kotsia, I., and Zafeiriou, S. (2017, January 21–26). AgeDB: The First Manually Collected, In-the-Wild Age Database. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, Honolulu, HI, USA.10.1109/CVPRW.2017.250 ↗
31. Ekman, P., and Friesen, W. (2023, April 23). Facial Action Coding System: A Technique for the Measurement of Facial Movement. Available online: https://www.paulekman.com/facial-action-coding-system/.