A Lightweight Teaching Assessment Framework Using Facial Expression Recognition for Online Courses
Source abstract
Abstract
To ensure the effectiveness of online teaching, educators must understand students’ learning progress. This study proposes LWKD-ViT, a framework designed to accurately capture students’ emotions during online courses. The framework is built on a lightweight facial expression recognition (FER) model with modifications to the fusion block. In addition, knowledge distillation (KD) is integrated into the online course platform to enhance performance. The framework follows a defined process involving face detection, tracking, and clustering to extract facial sequences for each student. An improved model, MobileViT-Local, developed by the authors, extracts emotion features from individual frames of students’ facial video streams for classification and prediction. Students’ facial images are captured through their device cameras and analyzed in real time on their devices, eliminating the need to transmit videos to the teacher’s computer or a remote server. To evaluate the performance of MobileViT-Local, comprehensive tests were conducted on benchmark datasets, including RAFD, RAF-DB, and FER2013, as well as a self-built dataset, SCAUOL. Experimental results demonstrate the model’s competitive performance and superior efficiency. Due to the use of knowledge distillation, the proposed model achieves a prediction accuracy of 94.96%, surpassing other mainstream models. It also exhibits excellent performance, with optimal FLOPs of 0.265 G and a compact size of 4.96 M, while maintaining acceptable accuracy.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Emotion and Mood Recognition · Face recognition and analysis · Advanced Technologies in Various Fields
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
College of Mathematics and Informatics, South China Agricultural University, Guangzhou 510642, China
Funding
Scientific Research Platforms and Projects of Guangdong Provincial Education Department of China
References (46)
Show source-provided references
1. Understanding learner continuance intention: A comparison of live video learning, pre-recorded video learning and hybrid video learning in COVID-19 pandemic10.1080/10447318.2021.1938389 ↗
2. Assessing learning engagement based on facial expression recognition in MOOC’s scenario10.1007/s00530-021-00854-x ↗
3. Darwin, C., and Prodger, P. (1998). The Expression of the Emotions in Man and Animals, Oxford University Press.10.1093/oso/9780195112719.002.0002 ↗
4. Recognizing action units for facial expression analysis10.1109/34.908962 ↗
5. Emotional intelligence and attachment in adulthood: A meta-analysis10.1016/j.paid.2021.111174 ↗
6. The faces of engagement: Automatic recognition of student engagementfrom facial expressions10.1109/TAFFC.2014.2316163 ↗
7. Survey on facial expressions recognition: Databases, features and classification schemes10.1007/s11042-023-15139-w ↗
8. A comprehensive survey on deep facial expression recognition: Challenges, applications, and future guidelines10.1016/j.aej.2023.01.017 ↗
9. Goodfellow, I., Bengio, Y., and Courville, A. (2016). Deep Learning, MIT Press.
10. NGDNet: Nonuniform Gaussian-label distribution learning for infrared head pose estimation and on-task behavior understanding in the classroom10.1016/j.neucom.2020.12.090 ↗
11. Zhu, B., Lan, X., Guo, X., Barner, K.E., and Boncelet, C. (2020, January 25–29). Multi-rate attention based gru model for engagement prediction. Proceedings of the 2020 International Conference on Multimodal Interaction, Utrecht, The Netherlands.10.1145/3382507.3417965 ↗
12. Liu, C., Jiang, W., Wang, M., and Tang, T. (2020, January 25–29). Group level audio-video emotion recognition using hybrid networks. Proceedings of the 2020 International Conference on Multimodal Interaction, Online.10.1145/3382507.3417968 ↗
13. Facial expression recognition method with multi-label distribution learning for non-verbal behavior understanding in the classroom10.1016/j.infrared.2020.103594 ↗
14. Preference prediction based on a photo gallery analysis with scene recognition and object detection10.1016/j.patcog.2021.108248 ↗
15. Constants across cultures in the face and emotion10.1037/h0030377 ↗
16. Strong evidence for universals in facial expressions: A reply to Russell’s mistaken critique10.1037/0033-2909.115.2.268 ↗
17. A new pan-cultural facial expression of emotion10.1007/BF00992253 ↗
18. Who knows what about contempt: A reply to Izard and Haynes10.1007/BF00992470 ↗
19. The universality of a contempt expression: A replication10.1007/BF00993116 ↗
20. More evidence for the universality of a contempt expression10.1007/BF00992972 ↗
21. Mercado-Diaz, L.R., Veeranki, Y.R., Marmolejo-Ramos, F., and Posada-Quintero, H.F. (2024). EDA-Graph: Graph Signal Processing of Electrodermal Activity for Emotional States Detection. J. Biomed. Health Inform. J-BHI, 28.10.36227/techrxiv.24311716 ↗
22. Veeranki, Y.R., Posada-Quintero, H.F., and Swaminathan, R. (2024). Transition Network-Based Analysis of Electrodermal Activity Signals for Emotion Recognition. Innov. Res. Biomed. Eng. IRBM, 45.10.1016/j.irbm.2024.100849 ↗
23. Veeranki, Y.R., Mercado-Diaz, L.R., and Posada-Quintero, H.F. (2024, January 26–28). Autoencoder Based Nonlinear Feature Extraction from EDA Signals for Emotion Recognition. Proceedings of the 2024 IEEE International Symposium on Medical Measurements and Applications (MeMeA), Eindhoven, The Netherlands.10.1109/MeMeA60663.2024.10596800 ↗
24. Allen, I.E., and Seaman, J. (2017). Digital Compass Learning: Distance Education Enrollment Report 2017, Babson Survey Research Group.
25. Shea, P., Bidjerano, T., and Vickers, J. (2016). Faculty Attitudes Toward Online Learning: Failures and Successes, SUNY Research Network.
26. Lederman, D. (2018). Conflicted Views of Technology: A Survey of Faculty Attitudes, Inside Higher Ed.
27. Performance outcomes of an online first aid and CPR course for laypersons10.1177/0017896910379696 ↗
28. Investigating the efficacy of practical skill teaching: A pilot-study comparing three educational methods10.1007/s10459-012-9355-2 ↗
29. An evaluation of online learning to teach practical competencies in undergraduate health science students10.1016/j.iheduc.2014.09.003 ↗
30. Goodfellow, I.J., Erhan, D., Carrier, P.L., Courville, A., Mirza, M., Hamner, B., Cukierski, W., Tang, Y., Thaler, D., and Lee, D.H. (2013, January 3–7). Challenges in representation learning: A report on three machine learning contests. Proceedings of the Neural Information Processing: 20th International Conference, ICONIP 2013, Daegu, Republic of Korea. Proceedings, Part III 20.
31. Li, S., Deng, W., and Du, J. (2017, January 21–26). Reliable Crowdsourcing and Deep Locality-Preserving Learning for Expression Recognition in the Wild. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.10.1109/CVPR.2017.277 ↗
32. Reliable Crowdsourcing and Deep Locality-Preserving Learning for Unconstrained Facial Expression Recognition10.1109/TIP.2018.2868382 ↗
33. Presentation and validation of the Radboud Faces Database10.1080/02699930903485076 ↗
34. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, L., and Polosukhin, I. (2017). Attention is all you need. Adv. Neural Inf. Process. Syst., 30.
35. Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., and Gelly, S. (2020). An image is worth 16x16 words: Transformers for image recognition at scale. arXiv.
36. Touvron, H., Cord, M., Douze, M., Massa, F., Sablayrolles, A., and Jégou, H. (2021, January 18–24). Training data-efficient image transformers & distillation through attention. Proceedings of the International Conference on Machine Learning, PMLR, Online.
37. Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., and Guo, B. (2021, January 10–17). Swin transformer: Hierarchical vision transformer using shifted windows. Proceedings of the IEEE/CVF International Conference on Computer Vision, Online.10.1109/ICCV48922.2021.00986 ↗
38. Volo: Vision outlooker for visual recognition
39. Mehta, S., and Rastegari, M. (2021). Mobilevit: Light-weight, general-purpose, and mobile-friendly vision transformer. arXiv.
40. Howard, A.G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., Andreetto, M., and Adam, H. (2017). Mobilenets: Efficient convolutional neural networks for mobile vision applications. arXiv.
41. Zhang, X., Zhou, X., Lin, M., and Sun, J. (2018, January 18–23). Shufflenet: An extremely efficient convolutional neural network for mobile devices. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00716 ↗
42. Tan, M., and Le, Q. (2019, January 9–15). Efficientnet: Rethinking model scaling for convolutional neural networks. Proceedings of the International Conference on Machine Learning, PMLR, Long Beach, CA, USA.
43. Yu, J., Markov, K., and Matsui, T. (2016, January 13–16). Articulatory and spectrum features integration using generalized distillation framework. Proceedings of the 2016 IEEE 26th International Workshop on Machine Learning for Signal Processing (MLSP), Salerno, Italy.10.1109/MLSP.2016.7738813 ↗
44. Wang, C.Y., Bochkovskiy, A., and Liao, H.Y.M. (2023, January 18–22). YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, BC, Canada.10.1109/CVPR52729.2023.00721 ↗
45. A Style-Based Generator Architecture for Generative Adversarial Networks10.1109/TPAMI.2020.2970919 ↗
46. Touvron, H., Cord, M., Douze, M., Massa, F., and Hervé, J. (2020). Training data-efficient image transformers & distillation through attention. arXiv.