Student Behavior Recognition in Classroom Based on Deep Learning
Source abstract
Abstract
With the widespread application of information technology in education, the real-time detection of student behavior in the classroom has become a key issue in improving teaching quality. This paper proposes a Student Behavior Detection (SBD) model that combines YOLOv5, the Contextual Attention (CA) mechanism and OpenPose, aiming to achieve efficient and accurate behavior recognition in complex classroom environments. By integrating YOLOv5 with the CA attention mechanism to enhance feature extraction capabilities, the model’s recognition performance in complex backgrounds, such as those with occlusion, is significantly improved. In addition, the feature map generated by the improved YOLOv5 is used to replace VGG-19 in OpenPose, which effectively improves the accuracy of student posture recognition. The experimental results demonstrate that the proposed model achieves a maximum mAP of 82.1% in complex classroom environments, surpassing Faster R-CNN by 5.2 percentage points and YOLOv5 by 4.6 percentage points. Additionally, the F1 score and R value of this model exhibit clear advantages over the other two traditional methods. This model offers an effective solution for intelligent classroom behavior analysis and the optimization of educational management.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Hand Gesture Recognition Systems · Human Pose and Action Recognition
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Key Laboratory of Data Science and Intelligence Education, Hainan Normal University, Ministry of Education, Haikou 571158, China
College of Information and Communication Engineering, Dalian Nationalities University, Dalian 116000, China
Funding
Key Laboratory of Data Science and Intelligence Education (Hainan Normal University), the Ministry of Education, China
References (23)
Show source-provided references
1. A review on artificial intelligence in education10.36941/ajis-2021-0077 ↗
2. Automatic detection of mind wandering from video in the lab and in the classroom10.1109/TAFFC.2019.2908837 ↗
3. A Review of Yolo algorithm developments10.1016/j.procs.2022.01.135 ↗
4. Chen, H., Zhou, G., and Jiang, H. (2023). Student behavior detection in the classroom based on improved YOLOv8. Sensors, 23.10.3390/s23208385 ↗
5. BiTNet: A lightweight object detection network for real-time classroom behavior recognition with transformer and bi-directional pyramid network
6. Liu, C., Tao, Y., Liang, J., Li, K., and Chen, Y. (2018, January 14–16). Object detection based on YOLO network. Proceedings of the 2018 IEEE 4th Information Technology and Mechatronics Engineering Conference (ITOEC), Chongqing, China.10.1109/ITOEC.2018.8740604 ↗
7. Human pose estimation using deep learning: A systematic literature review10.3390/make5040081 ↗
8. Data augmentation method for improving the accuracy of human pose estimation with cropped images10.1016/j.patrec.2020.06.015 ↗
9. Human action recognition from various data modalities: A review
10. Automatic visual detection of human behavior: A review from 2000 to 201410.1016/j.eswa.2015.05.023 ↗
11. Batool, M., Jalal, A., and Kim, K. (2019, January 27–29). Sensors technologies for human activity analysis based on SVM optimized by PSO algorithm. Proceedings of the 2019 International Conference on Applied and Engineering Mathematics (ICAEM), Taxila, Pakistan.10.1109/ICAEM.2019.8853770 ↗
12. Real-time and accurate abnormal behavior detection in videos10.1007/s00138-020-01111-3 ↗
13. Zhu, Y., Lan, Z., Newsam, S., and Hauptmann, A. (2018, January 2–6). Hidden two-stream convolutional networks for action recognition. Proceedings of the Computer Vision–ACCV 2018, 14th Asian Conference on Computer Vision, Perth, Australia. Revised Selected Papers, Part III 14.
14. Asymmetric 3d convolutional neural networks for action recognition10.1016/j.patcog.2018.07.028 ↗
15. Zhang, P., Lan, C., Zeng, W., Xing, J., Xue, J., and Zheng, N. (2020, January 14–19). Semantics-guided neural networks for efficient skeleton-based human action recognition. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.10.1109/CVPR42600.2020.00119 ↗
16. Ji, H., Zeng, X., Li, H., Ding, W., Nie, X., Zhang, Y., and Xiao, Z. (2020, January 10–12). Human abnormal behavior detection method based on T-TINY-YOLO. Proceedings of the 5th International Conference on Multimedia and Image Processing, Nanjing, China.10.1145/3381271.3381273 ↗
17. Deep convolutional neural networks for human action recognition using depth maps and postures10.1109/TSMC.2018.2850149 ↗
18. An intelligent system for monitoring students’ engagement in large classroom teaching through facial expression recognition10.1111/exsy.12839 ↗
19. OpenPose: Realtime multi-person 2D pose estimation using part affinity fields10.1109/TPAMI.2019.2929257 ↗
20. Duan, H., Zhao, Y., Chen, K., Lin, D., and Dai, B. (2022, January 18–24). Revisiting skeleton-based action recognition. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.10.1109/CVPR52688.2022.00298 ↗
21. Jocher, G., Stoken, A., Borovec, J., Changyu, L., Hogan, A., Diaconu, L., Ingham, F., Poznanski, J., Fang, J., and Yu, L. (2020). Ultralytics/ YOLOv5, V3.1-bug fixes and preformance improvements. Zenodo.
22. A robust and lightweight deep attention multiple instance learning algorithm for predicting genetic alterations10.1016/j.compmedimag.2023.102189 ↗
23. Viswakumar, A., Rajagopalan, V., Ray, T., and Parimi, C. (2019, January 15–17). Human gait analysis using OpenPose. Proceedings of the 2019 Fifth International Conference on Image Information Processing (ICIIP), Shimla, India.10.1109/ICIIP47207.2019.8985781 ↗