A Computer-Vision Based Application for Student Behavior Monitoring in Classroom
Source abstract
Abstract
Automated learning analytics is becoming an essential topic in the educational area, which needs effective systems to monitor the learning process and provides feedback to the teacher. Recent advances in visual sensors and computer vision methods enable automated monitoring of behavior and affective states of learners at different levels, from university to pre-school. The objective of this research was to build an automatic system that allowed the faculties to capture and make a summary of student behaviors in the classroom as a part of data acquisition for the decision making process. The system records the entire session and identifies when the students pay attention in the classroom, and then reports to the facilities. Our design and experiments show that our system is more flexible and more accurate than previously published work.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: IoT-based Smart Home Systems · Anomaly Detection Techniques and Applications
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
ICT Department, FPT University, Hanoi 10000, Vietnam
School of Information and Software Engineering, University of Electronic Science and Technology of China, Chengdu 610054, China
Department of Computer Science, University of Freiburg, 79098 Freiburg, Germany
References (54)
Show source-provided references
1. Grasha, A.F. (2002). Teaching with Style: A Practical Guide to Enhancing Learning by Understanding Teaching and Learning Styles, Alliance Publishers.
2. Modeling antecedents of university students’ study behavior and grade point average10.1111/j.1559-1816.2013.01043.x ↗
3. The interrelationship between student learning experience and study behaviour10.1080/07294360.2010.539598 ↗
4. Dividing attention in the classroom reduces exam performance10.1080/01443410.2018.1489046 ↗
5. American Psychological Association (2019, June 08). Multitasking: Switching Costs. Available online: https://www.apa.org/research/action/multitask.
6. Hale, G.A., and Lewis, M. (1979). Attention in the Classroom. Attention and Cognitive Development, Springer.10.1007/978-1-4613-2985-5 ↗
7. McCoy, B. (2013). Digital Distractions in the Classroom: Student Classroom Use of Digital Devices for NonClass Related Purposes, Journalism and Mass Communications.
8. Rethinking college students’ self-regulation and sustained attention: Does text messaging during class influence cognitive learning?10.1080/03634523.2012.672755 ↗
9. Students’ silent messages: Can teacher verbal and nonverbal immediacy moderate student use of text messaging in class?10.1080/03634523.2010.496092 ↗
10. Yu, T., Jo, I., Lee, H., and Kim, Y. (2014). Relations between Student Online Learning Behavior and Academic Achievement in Higher Education: A Learning Analytics Approach, Springer.
11. Academic analytics and data mining in higher education
12. Raca, M., and Dillenbourg, P. (2013, January 8–12). System for assessing classroom attention. Proceedings of the Third International Conference on Learning Analytics and Knowledge—LAK 2013, Leuven, Belgium.10.1145/2460296.2460351 ↗
13. Raca, M. (2019, June 15). Camera-Based Estimation of Student’s Attention in Class. Available online: https://infoscience.epfl.ch/record/212929.
14. Lim, J.H., Teh, E.Y., Geh, M.H., and Lim, C.H. (2017, January 12–15). Automated classroom monitoring with connected visioning system. Proceedings of the 2017 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC 2017), Kuala Lumpur, Malaysia.10.1109/APSIPA.2017.8282063 ↗
15. Predicting students’ attention in the classroom from Kinect facial and body features10.1186/s13640-017-0228-8 ↗
16. Dar, P. (2019, June 15). A Chinese School Is Using Facial Recognition to Analyze Students’ Behavior. Available online: https://www.analyticsvidhya.com/blog/2018/06/china-school-facial-recognition-analyse-students/.
17. Son, N.T., Chi, l., Lam, P.T., and van Dinh, T. (2019, January 19–21). Combination of facial recognition and interaction with academic portal in automatic attendance system. Proceedings of the 2019 8th ACM International Conference on Software and Computer Applications (ICSCA 2019), New York, NY, USA.10.1145/3316615.3316638 ↗
18. Srivastava, A., Mane, S., Shah, A., Shrivastava, N., and Thakare, B. (2017, January 19–20). A survey of face detection algorithms. Proceedings of the 2017 International Conference on Inventive Systems and Control (ICISC 2017), Coimbatore, India.10.1109/ICISC.2017.8068607 ↗
19. Zhao, Z., Zheng, P., Xu, S., and Wu, X. (2019). Object detection with deep learning: A review. IEEE Trans. Neural Netw. Learn. Syst., 1–21.10.1109/TNNLS.2019.2933467 ↗
20. Girshick, R., Donahue, J., Darrell, T., and Malik, J. (2014, January 23–28). Rich feature hierarchies for accurate object detection and semantic segmentation. Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA.10.1109/CVPR.2014.81 ↗
21. Girshick, R. (17–18, January 11–12). Fast R-CNN. Proceedings of the 2015 IEEE International Conference on Computer Vision (ICCV 2015), Santiago, Chile.10.1109/ICCV.2015.169 ↗
22. Faster r-cnn: Towards real-time object detection with region proposal networks10.1109/TPAMI.2016.2577031 ↗
23. Dai, J., Li, Y., He, K., and Sun, J. (2016, January 5–10). R-FCN: Object detection via region-based fully convolutional networks. Proceedings of the 30th International Conference on Neural Information Processing Systems (NIPS 2016), Barcelona, Spain.
24. He, K., Gkioxari, G., Dollár, P., and Girshick, R.B. (2017, January 22–29). Mask R-CNN. Proceedings of the 2017 IEEE International Conference on Computer Vision (ICCV), Venice, Italy.10.1109/ICCV.2017.322 ↗
25. Joint face detection and alignment using multitask cascaded convolutional networks10.1109/LSP.2016.2603342 ↗
26. Redmon, J., Divvala, S., Girshick, R., and Farhadi, A. (2016, January 27–30). You only look once: Unified, real-time object detection. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2016), Los Alamitos, CA, USA.10.1109/CVPR.2016.91 ↗
27. Redmon, J., and Farhadi, A. (2017, January 21–26). Yolo9000: Better, faster, stronger. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2017), Honolulu, HI, USA.10.1109/CVPR.2017.690 ↗
28. Redmon, J., and Farhadi, A. (2019, August 10). Yolov3: An Incremental Improvement. Available online: http://arxiv.org/abs/1804.02767.
29. Leibe, B., Matas, J., Sebe, N., and Welling, M. (2016). Ssd: Single shot multibox detector. Computer Vision-ECCV 2016, Springer International Publishing.10.1007/978-3-319-46454-1 ↗
30. Lin, T., Goyal, P., Girshick, R., He, K., and Dollar, P. (2018). Focal loss for dense object detection. IEEE Trans. Pattern Anal. Mach. Intell.10.1109/ICCV.2017.324 ↗
31. Najibi, M., Samangouei, P., Chellappa, R., and Davis, L.S. (2017, January 22–29). SSH: Single stage headless face detector. Proceedings of the 2017 IEEE International Conference on Computer Vision (ICCV 2017), Venice, Italy.10.1109/ICCV.2017.522 ↗
32. Deng, J., Guo, J., Zhou, Y., Yu, J., Kotsia, I., and Zafeiriou, S. (2019). Retinaface: Single-stage dense face localisation in the wild. arXiv.10.1109/CVPR42600.2020.00525 ↗
33. (2019, August 10). Face Landmark Detection in an Image. Available online: https://docs.opencv.org/3.4.2/d2/d42/tutorial_face_landmark_detection_in_an_image.html.
34. (2019, August 10). Face Landmark Detection. Available online: http://dlib.net/face_landmark_detection.py.html.
35. Masi, I., Wu, Y., Hassner, T., and Natarajan, P. (November, January 29). Deep face recognition: A survey. Proceedings of the 2018 31st SIBGRAPI Conference on Graphics, Patterns and Images (SIBGRAPI 2018), Parana, Brazil.10.1109/SIBGRAPI.2018.00067 ↗
36. Taigman, Y., Yang, M., Ranzato, M., and Wolf, L. (2014, January 23–28). Deepface: Closing the gap to human-level performance in face verification. Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA.10.1109/CVPR.2014.220 ↗
37. Schroff, F., Kalenichenko, D., and Philbin, J. (2015, January 11–12). FaceNet: A unified embedding for face recognition and clustering. Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA.10.1109/CVPR.2015.7298682 ↗
38. Gary, K.L., Xie, T.X., and Jones, M.W. (2015). Deep face recognition. Proceedings of the British Machine Vision Conference (BMVC), BMVA Press.
39. Liu, W., Wen, Y., Yu, Z., Li, M., Raj, B., and Song, L. (2017, January 21–26). SphereFace: Deep Hypersphere Embedding for Face Recognition. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.10.1109/CVPR.2017.713 ↗
40. Deng, J., Guo, J., Xue, N., and Zafeiriou, S. (2019, January 16–20). Arcface: Additive angular margin loss for deep face recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA.10.1109/CVPR.2019.00482 ↗
41. K-nearest neighbor classification approach for face and fingerprint at feature level fusion
42. Tabletgaze: Dataset and analysis for unconstrained appearance-based gaze estimation in mobile tablets10.1007/s00138-017-0852-4 ↗
43. Krafka, K., Khosla, A., Kellnhofer, P., Kannan, H., Bhandarkar, S., Matusik, W., and Torralba, A. (July, January 26). Eye Tracking for Everyone. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.10.1109/CVPR.2016.239 ↗
44. Zhu, X., Lei, Z., Liu, X., Shi, H., and Li, S.Z. (July, January 26). Face Alignment Across Large Poses: A 3D Solution. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.10.1109/CVPR.2016.23 ↗
45. Kumar, A., Alavi, A., and Chellappa, R. (June, January 30). Kepler: Keypoint and pose estimation of unconstrained faces by learning efficient h-cnn regressors. Proceedings of the 2017 12th IEEE International Conference on Automatic Face Gesture Recognition (FG 2017), Washington, DC, USA.10.1109/FG.2017.149 ↗
46. Ruiz, N., Chong, E., and Rehg, J. (2018, January 18–22). Fine-grained head pose estimation without keypoints. Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW 2018), Salt Lake City, UT, USA.10.1109/CVPRW.2018.00281 ↗
47. Yang, T.-Y., Chen, Y.-T., Lin, Y.-Y., and Chuang, Y.-Y. (2019, January 16–20). FSA-Net: Learning fine-grained structure aggregation for head pose estimation from a single image. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA.10.1109/CVPR.2019.00118 ↗
48. (2019, October 15). Camera Calibration and 3D Reconstruction. Available online: https://docs.opencv.org/2.4/modules/calib3d/doc/camera_calibration_and_3d_reconstruction.html.
49. Yang, S., Luo, P., Loy, C.C., and Tang, X. (July, January 26). WIDER FACE: A Face Detection Benchmark. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.10.1109/CVPR.2016.596 ↗
50. Support-vector networks10.1007/BF00994018 ↗
51. Induction of decision trees10.1007/BF00116251 ↗
52. Greedy Function Approximation: A Gradient Boosting Machine10.1214/aos/1013203451 ↗
53. Random Forest10.1023/A:1010933404324 ↗
54. SMOTE: Synthetic minority over-sampling technique10.1613/jair.953 ↗