Enhancing Online Learning Through Multi-Agent Debates for CS University Students

As recent advancements in large language models enhance reasoning across various domains, educators are increasingly exploring their use in conversation-based tutoring systems. However, since LLMs are black-box models to users and lack human-like problem-solving strategies, users are hardly convinced by the answers provided by LLMs. This lack of trust can potentially undermine the effectiveness of learning in educational scenarios. To address these issues, we introduce a novel approach that integrates multi-agent debates into a lecture video Q&A system, aiming to assist computer science (CS) university students in self-learning by using LLMs to simulate debates between affirmative and negative debaters and a judge to reach a final answer and presenting the entire process to users for review. This approach is expected to lead to better learning outcomes and the improvement of students’ critical thinking. To validate the effectiveness of this approach, we carried out a user study through a prototype system and conducted preliminary experiments based on video lecture learning involving 90 CS students from three universities. The study compared different conditions and demonstrated that students who had access to a combination of video-based Q&A and multi-agent debates performed significantly better on quizzes compared to those who only had access to the video or video-based Q&A. These findings indicate that integrating multi-agent debates with lecture videos can substantially enhance the learning experience, which is also beneficial for the development of students’ high-order thinking abilities in the future.