Real-Time Analysis and Intervention of Classroom Behavior Using Multi-Modal Fusion and Spatiotemporal Context
Source abstract
Abstract
Analyzing classroom engagement is essential for developing effective smart learning environments. Conventional methods often face challenges in achieving reliable identification of individual students, accurately recognizing their behavioral states, and providing timely support. This paper presents a multimodal sensing and supportive feedback system built upon an end–edge–cloud collaborative architecture. By integrating RFID-based seat association, fingerprint verification, and computer vision-based activity analysis, the system establishes a reliable link between student identity and observed activities. Key computational tasks, including activity recognition, spatiotemporal context matching, and rule-based assessment, are executed locally on edge nodes. This enables low-latency, privacy-conscious feedback delivered via Bluetooth, effectively avoiding delays associated with cloud processing. Experimental results indicate that the proposed system significantly enhances both activity recognition accuracy and identity–behavior association reliability in typical classroom scenarios while substantially reducing the average feedback latency compared to traditional approaches.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Context-Aware Activity Recognition Systems · Human Pose and Action Recognition · Emotion and Mood Recognition
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
College of Electronic Information and Optical Engineering, Nankai University, Tianjin 300350, China
Funding
National College Students’ Innovation and Entrepreneurship Training Program · Tianjin Key Laboratory of Optical-electronic Sensor and Sensor Network Technology
References (24)
Show source-provided references
1. Valdes-Ramirez, D., Sobrino Caamal, V., Huertas Cardozo, J.I., Corchado, J.M., and Zavala, G. (2026). Edge computing model for real-time classroom behavior analysis in face-to-face lessons. Distributed Computing and Artificial Intelligence, Special Sessions II, 22nd International Conference, Springer.10.1007/978-3-032-05748-8_42 ↗
2. Multi-Modal Student Engagement Detection Using Eye-Gaze and Posture Analysis under Obstructed Conditions10.22161/ijtle.4.6.3 ↗
3. YOLOv8-Based Student Behavior Detection in the Classroom: Introducing BAR Attention and Shape IoU Improvements to Enhance Detection Capabilities
4. A WAD-YOLOv8-based method for classroom student behavior detection10.1038/s41598-025-87661-w ↗
5. Liu, Q., Jiang, X., and Jiang, R. (2025). Classroom Behavior Recognition Using Computer Vision: A Systematic Review. Sensors, 25.10.3390/s25020373 ↗
6. Enhancing Person Identification for Smart Cities: Fusion of Video Surveillance and Wearable Device Data Based on Machine Learning10.1109/JSEN.2024.3422844 ↗
7. Edge intelligence: The confluence of edge computing and artificial intelligence10.1109/JIOT.2020.2984887 ↗
8. Hossen, M.A., and Abas, P.E. (2025). Machine Learning for Human Activity Recognition: State-of-the-Art Techniques and Emerging Trends. J. Imaging, 11.10.3390/jimaging11030091 ↗
9. GaitFi: Robust Device-Free Human Identification via WiFi and Vision Multimodal Learning10.1109/JIOT.2022.3203559 ↗
10. Maltoni, D., Maio, D., Jain, A.K., and Prabhakar, S. (2022). Handbook of Fingerprint Recognition, Springer. [3rd ed.].10.1007/978-3-030-83624-5 ↗
11. Pereira, J.S. (2025). Long-Range RFID Indoor Positioning System for an Autonomous Wheelchair. Sensors, 25.10.3390/s25082542 ↗
12. Zhao, M., Xu, K., Liu, J., Zhang, J., Mi, T., Shi, X., and Qiu, R.C. (2025). RISAR: RIS-Assisted Human Activity Recognition with Commercial WiFi Devices. IEEE Trans. Wirel. Commun.
13. CMD-HAR: Cross-Modal Disentanglement for Wearable Human Activity Recognition
14. MaskFi: Unsupervised Learning of WiFi and Vision Representations for Multimodal Human Activity Recognition
15. Mao, C., Tan, C., Liu, H., Hu, J., and Zheng, M. (2024). ViFi-ReID: A Two-Stream Vision-WiFi Multimodal Approach for Person Re-Identification. CoRR, abs/2410.09875.
16. Padilla, R., Passos, W.L., Dias, T.L.B., Netto, S.L., and da Silva, E.A.B. (2021). A comparative analysis of object detection metrics with a companion open-source toolkit. Electronics, 10.10.3390/electronics10030279 ↗
17. Personalized feedback in digital learning environments: Classification framework and literature review10.1016/j.caeai.2022.100080 ↗
18. Tian, H., Su, X., and Hou, Y. (2024). Feedback stabilization of probabilistic finite state machines based on deep Q-network. Front. Comput. Neurosci., 18.10.3389/fncom.2024.1385047 ↗
19. Golipoor, S., and Sigg, S. (2025). RF-Behavior: A Multimodal Radio-Frequency Dataset for Human Behavior and Emotion Analysis. arXiv.
20. Zhao, J., Guo, G., and Liu, B. (2025). An Improved YOLOv10-Based Algorithm for Student Behavior Detection in Classroom Environments. Proceedings of the 2025 International Conference on Trustworthy Big Data and Artificial Intelligence (ICTBAI), IEEE.10.1109/ICTBAI68361.2025.00014 ↗
21. Towards Student Actions in Classroom Scenes: New Dataset and Baseline10.1109/TMM.2025.3590899 ↗
22. Ye, X., Sakurai, K., Nair, N.K.C., and Wang, K.I.K. (2024). Machine Learning Techniques for Sensor-Based Human Activity Recognition with Data Heterogeneity—A Review. Sensors, 24.10.3390/s24247975 ↗
23. Quy, T.D., Lin, C.Y., and Shih, T.K. (2025). Enhanced Human Activity Recognition Using Wi-Fi Sensing: Leveraging Phase and Amplitude with Attention Mechanisms. Sensors, 25.10.3390/s25041038 ↗
24. Bandyopadhyay, A., Basu, R., Sen, T., and Das, S. (2025). MHARFedLLM: Multimodal Human Activity Recognition Using Federated Large Language Model. arXiv.