High-Quality Classroom Dialogue Automatic Analysis System
Source abstract
Abstract
Classroom dialogue analysis is crucial as it significantly impacts both knowledge transmission and teacher–student interactions. Since the inception of classroom analysis research, traditional methods such as manual transcription and coding have served as foundational tools for understanding these interactions. While precise and insightful, these methods are inherently time-consuming, labor-intensive, and susceptible to human bias. Moreover, they struggle to handle the scale and complexity of modern classroom data effectively. In contrast, many contemporary deep learning approaches focus primarily on dialogue classification, but often lack the capability to provide deeper interpretative insights. To address these challenges, this study introduces an automated dialogue analysis system that combines scalability, efficiency, and objectivity in evaluating teaching quality. We first collected a large dataset of classroom recordings from primary and secondary schools in China and manually annotated the dialogues using multiple encoding frameworks. Based on these data, we developed an automated analysis system featuring a novel dialogue classification algorithm that incorporates speaker role information for more accurate insights. Additionally, we implemented innovative visualization techniques to automatically generate comprehensive classroom analysis reports, effectively bridging the gap between traditional manual methods and modern automated approaches. Experimental results demonstrated the system’s high accuracy in distinguishing various types of classroom dialogue. Large-scale analysis revealed key patterns in classroom dynamics, showcasing the strong potential of our system to enhance teaching evaluation and provide valuable insights for improving education practices.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Robotics and Automated Systems · Speech and dialogue systems
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Lab of Artificial Intelligence for Education, East China Normal University, Shanghai 200062, China
Shanghai Institute of Artificial Intelligence for Education, East China Normal University, Shanghai 200062, China
School of Computer Science and Technology, East China Normal University, Shanghai 200062, China
College of Computer and Cyber Security, Fujian Normal University, Fuzhou 350007, China
International Classroom Analysis Lab, Institute of Curriculum and Instruction, East China Normal University, Shanghai 200062, China
References (47)
Show source-provided references
1. Classroom dialogue: A systematic review across four decades of research10.1080/0305764X.2013.786024 ↗
2. Asterhan, C., Clarke, S., and Resnick, L. (2015). Socializing Intelligence Through Academic Talk and Dialogue, American Educational Research Association.
3. Clarke, S.N., Resnick, L.B., and Rosé, C.P. (2015). Dialogic instruction: A new frontier. Handbook of Educational Psychology, Routledge.
4. How (well structured) talk builds the mind
5. Jensen, E., Pugh, S.L., and D’Mello, S.K. (2021, January 12–16). A deep transfer learning approach to modeling teacher discourse in the classroom. Proceedings of the LAK21: 11th International Learning Analytics and Knowledge Conference, Irvine, CA, USA.10.1145/3448139.3448168 ↗
6. Formative assessment and self-regulated learning: A model and seven principles of good feedback practice10.1080/03075070600572090 ↗
7. Providing performance feedback to teachers: A review10.1177/088840640402700407 ↗
8. Focus on formative feedback10.3102/0034654307313795 ↗
9. Archer, J., Cantrell, S., Holtzman, S.L., Joe, J.N., Tocci, C.M., and Wood, J. (2016). Better Feedback for Better Teaching: A Practical Guide to Improving Classroom Observations, John Wiley & Sons.
10. Flanders, N.A. (1965). Teacher Influence, Pupil Attitudes, and Achievement: Ned A. Flanders, Number 12.
11. Volodina, E., Grigonytė, G., Pilán, I., Björkenstam, K.N., and Borin, L. (2016). The Language ENvironment Analysis (LENA) system: A literature review. Proceedings of the Joint Workshop on NLP for Computer Assisted Language Learning and NLP for Language Acquisition, Umeå, Sweden, 16 November 2016, LiU Electronic Press.
12. Using the LENA in teacher training: Promoting student involvement through automated feedback
13. D’Mello, S.K., Olney, A.M., Blanchard, N., Samei, B., Sun, X., Ward, B., and Kelly, S. (2015, January 9–13). Multimodal capture of teacher-student interactions for automated dialogic analysis in live classrooms. Proceedings of the 2015 ACM on International Conference on Multimodal Interaction, Seattle, WA, USA.10.1145/2818346.2830602 ↗
14. Xu, L., He, X., Zhang, J., and Li, Y. (2019, January 28–31). Automatic classification of discourse in Chinese classroom based on multi-feature fusion. Proceedings of the 2019 International Conference on Computer, Information and Telecommunication Systems (CITS), Beijing, China.10.1109/CITS.2019.8862101 ↗
15. Automatic classification of activities in classroom discourse10.1016/j.compedu.2014.05.010 ↗
16. Blanchard, N., D’Mello, S., Olney, A.M., and Nystrand, M. (2015, January 26–29). Automatic Classification of Question & Answer Discourse Segments from Teacher’s Speech in Classrooms. Proceedings of the 8th International Conference on Educational Data Mining, Madrid, Spain.
17. Lei, S., Li, J., Song, Y., Shen, Y., Lee, L.K., and Hao, T. (2020, January 19–22). Leveraging Neural Network-Based Model for Context Classification of Classroom Dialogue Text. Proceedings of the Technology in Education, Innovations for Online Teaching and Learning: 5th International Conference, ICTE 2020, Macau, China. Revised Selected Papers 5.
18. Sinclair, J.M., and Coulthard, M. (1975). Towards an Analysis of Discourse: The English Used by Teachers and Pupils, Oxford University Press.
19. Developing a coding scheme for analysing classroom dialogue across educational contexts10.1016/j.lcsi.2015.12.001 ↗
20. Song, Y., Hao, T., Liu, Z., and Lan, Z. (2019, January 23–25). A systematic review of frameworks for coding towards classroom dialogue. Proceedings of the Emerging Technologies for Education: 4th International Symposium, SETE 2019, Held in Conjunction with ICWL 2019, Magdeburg, Germany. Revised Selected Papers 4.
21. The teacher scheme for educational dialogue analysis (T-SEDA): Developing a research-based observation tool for supporting teacher inquiry into pupils’ participation in classroom dialogue10.1080/1743727X.2018.1467890 ↗
22. Deliberative discourse idealized and realized: Accountable talk in the classroom and in civic life10.1007/s11217-007-9071-1 ↗
23. Coding classroom dialogue: Methodological considerations for researchers10.1016/j.lcsi.2020.100404 ↗
24. Suresh, A., Jacobs, J., Lai, V., Tan, C., Ward, W., Martin, J.H., and Sumner, T. (2021). Using transformers to provide teachers with personalized feedback on their classroom discourse: The TalkMoves application. arXiv.
25. Long, Y., and Zhang, Y. (2024). Enhanced Classroom Dialogue Sequences Analysis with a Hybrid AI Agent: Merging Expert Rule-Base with Large Language Models. arXiv.
26. Rafique, M.A., Khaskheli, F., Hassan, M.T., Naseer, S., and Jeon, M. (2022). Employing automatic content recognition for teaching methodology analysis in classroom videos. PLoS ONE, 17.10.1371/journal.pone.0263448 ↗
27. Zhao, G., Yi, J., Chu, J., Zhang, Y., and Yin, J. (2022, January 28–30). Design and Implementation of the Teacher-student Dialogue Automatic Speech Recognition Tool Based on Classroom Verbal Characteristics. Proceedings of the 14th International Conference on Education Technology and Computers, Barcelona, Spain.10.1145/3572549.3572629 ↗
28. Su, Z., and Zhou, Q. (2022, January 12–17). Speaker clustering in textual dialogue with pairwise utterance relation and cross-corpus dialogue act supervision. Proceedings of the 29th International Conference on Computational Linguistics, Gyeongju, Republic of Korea.
29. He, Z., Tavabi, L., Lerman, K., and Soleymani, M. (2021). Speaker turn modeling for dialogue act classification. arXiv.10.18653/v1/2021.findings-emnlp.185 ↗
30. Mezza, S., Wobcke, W., and Blair, A. (2022, January 12–17). A Multi-Dimensional, Cross-Domain and Hierarchy-Aware Neural Architecture for ISO-Standard Dialogue Act Tagging. Proceedings of the 29th International Conference on Computational Linguistics, Gyeongju, Republic of Korea.
31. Zheng, R., Jiang, F., and Shen, R. (2021, January 7–15). GestureDet: Real-time student gesture analysis with multi-dimensional attention-based detector. Proceedings of the Twenty-Ninth International Conference on International Joint Conferences on Artificial Intelligence, Yokohama, Japan.10.24963/ijcai.2020/95 ↗
32. Wang, Z., Wang, M., Zeng, C., and Li, L. (2024). SBD-Net: Incorporating Multi-Level Features for an Efficient Detection Network of Student Behavior in Smart Classrooms. Appl. Sci., 14.10.3390/app14188357 ↗
33. Zhang, Z., Zhang-Li, D., Yu, J., Gong, L., Zhou, J., Liu, Z., Hou, L., and Li, J. (2024). Simulating classroom education with llm-empowered agents. arXiv.
34. Evaluating large language models in analysing classroom dialogue10.1038/s41539-024-00273-3 ↗
35. Jensen, E., Dale, M., Donnelly, P.J., Stone, C., Kelly, S., Godley, A., and D’Mello, S.K. (2020, January 25–30). Toward automated feedback on teacher discourse to enhance teacher learning. Proceedings of the 2020 Chi Conference on Human Factors in Computing Systems, Honolulu, HI, USA.10.1145/3313831.3376418 ↗
36. Lin, F.C., Ngo, H.H., Dow, C.R., Lam, K.H., and Le, H.L. (2021). Student behavior recognition system for the classroom environment based on skeleton pose estimation and person detection. Sensors, 21.10.3390/s21165314 ↗
37. Lenart, C., Ahadian, P., Yang, Y., Suo, S., Corsello, A., Kosko, K., and Guan, Q. (November, January 29). Gaze Analysis System for Immersive 360 Video for Preservice Teacher Education. Proceedings of the 31st ACM International Conference on Multimedia, Ottawa, ON, Canada.
38. A multimodal facial cues based engagement detection system in e-learning context using deep learning approach10.1007/s11042-023-14392-3 ↗
39. Uchiyama, S., Umemura, K., and Morita, Y. (2023). Large Language Model-based System to Provide Immediate Feedback to Students in Flipped Classroom Preparation Learning. arXiv.
40. Ngoon, T.J., Sushil, S., Stewart, A.E., Lee, U.S., Venkatraman, S., Thawani, N., Mitra, P., Clarke, S., Zimmerman, J., and Ogan, A. (2024, January 11–16). ClassInSight: Designing Conversation Support Tools to Visualize Classroom Discussion for Personalized Teacher Professional Development. Proceedings of the CHI Conference on Human Factors in Computing Systems, Honolulu, HI, USA.10.1145/3613904.3642487 ↗
41. Liu, Z., Zhu, Z., Zhu, L., Jiang, E., Hu, X., Peppler, K.A., and Ramani, K. (2024, January 11–16). ClassMeta: Designing Interactive Virtual Classmate to Promote VR Classroom Participation. Proceedings of the CHI Conference on Human Factors in Computing Systems, Honolulu, HI, USA.10.1145/3613904.3642947 ↗
42. Bain, M., Huh, J., Han, T., and Zisserman, A. (2023). WhisperX: Time-Accurate Speech Transcription of Long-Form Audio. arXiv.10.21437/Interspeech.2023-78 ↗
43. Kruse, K. (2024, September 01). Gagne’s Nine Events of Instruction: An Introduction. Available online: https://psu.pb.unizin.org/app/uploads/sites/88/2018/06/Gagne-Nine-Events-of-Instructoin.pdf.
44. Mehan, H. (1979). Learning Lessons: Social Organization in the Classroom, Harvard University Press.10.4159/harvard.9780674420106 ↗
45. Interrater reliability: The kappa statistic
46. Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., and Stoyanov, V. (2019). Roberta: A robustly optimized bert pretraining approach. arXiv.
47. Jun, Y., and Huanghai, O. (2011, January 3–5). ST analysis of classroom teaching. Proceedings of the 2011 6th International Conference on Computer Science & Education (ICCSE), Singapore.10.1109/ICCSE.2011.6028602 ↗