An Enhanced Deep Knowledge Tracing Model via Multiband Attention and Quantized Question Embedding
Source abstract
Abstract
Knowledge tracing plays a crucial role in effectively representing learners’ understanding and predicting their future learning progress. However, existing deep knowledge tracing methods, reliant on the forgetting model and Rasch model, often fail to account for the varying rates at which learners forget different knowledge concepts and the variations in question embedding covering the same concept. To address these limitations, this paper introduces an enhanced deep knowledge tracing model that combines the transformer network model with two innovative components. The first component is a multiband attention mechanism, which comprehensively summarizes a learner’s past response history across various temporal scales. By computing attention weights using different decay rates, this mechanism adaptively captures both long-term and short-term interactions for different knowledge concepts. The second component utilizes a quantized question embedding module to effectively capture variations among questions addressing the same knowledge concept. This module represents these differences in a rich embedding space, avoiding overparameterization or overfitting issues. The proposed model is evaluated on popular benchmark datasets, demonstrating its superiority over existing knowledge tracing methods in accuracy. This enhancement holds potential for improving personalized learning systems by providing more precise insights into learners’ progress.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Intelligent Tutoring Systems and Adaptive Learning · Topic Modeling · Robotics and Automated Systems
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Faculty of Artificial Intelligence in Education, Central China Normal University, Wuhan 430079, China
Funding
MOE Project of Humanities and Social Sciences · National Natural Science Foundation of China
References (43)
Show source-provided references
1. Knowledge tracing: Modeling the acquisition of procedural knowledge10.1007/BF01099821 ↗
2. Liu, Q., Shen, S., Huang, Z., and Chen, E. (2021). A Survey of Knowledge Tracing. arXiv.
3. Student modeling approaches: A literature review for the last decade10.1016/j.eswa.2013.02.007 ↗
4. The efectiveness of online learning: Beyond no signifcant diference and future horizons
5. Investigating users’ perspectives on e-learning: An integration of TAM and IS success model10.1016/j.chb.2014.07.044 ↗
6. Wang, F., Liu, Q., Chen, E., Huang, Z., Chen, Y., Yin, Y., Huang, Z., and Wang, S. (2020, January 7–12). Neural cognitive diagnosis for intelligent education systems. Proceedings of the AAAI Conference on Artifcial Intelligence, New York, NY, USA.
7. Deep knowledge tracing
8. A learning algorithm for continually running fully recurrent neural networks10.1162/neco.1989.1.2.270 ↗
9. Long short-term memory10.1162/neco.1997.9.8.1735 ↗
10. Yeung, C., and Yeung, D. (2018, January 26–28). Addressing two problems in deep knowledge tracing via prediction-consistent regularization. Proceedings of the Fifth Annual ACM Conference on Learning at Scale, London, UK.10.1145/3231644.3231647 ↗
11. Zhang, J., Shi, X., King, I., and Yeung, D. (2017, January 3–7). Dynamic key-value memory networks for knowledge tracing. Proceedings of the 26th International Conference on World Wide Web, Perth, Australia.10.1145/3038912.3052580 ↗
12. Pandey, S., and Karypis, G. (2019, January 2–5). A self-attentive model for knowledge tracing. Proceedings of the 12th International Conference on Educational Data Mining, Montreal, QC, Canada.
13. Ghosh, A., Hefernan, N., and Lan, A. (2020, January 23–27). Context-aware attentive knowledge tracing. Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Virtual, CA, USA.10.1145/3394486.3403282 ↗
14. Lee, W., Chun, J., Lee, Y., Park, K., and Park, S. (2022, January 21–24). Contrastive Learning for Knowledge Tracing. Proceedings of the ACM Web Conference, Lisbon, Portugal.10.1145/3485447.3512105 ↗
15. Pashler, H., Cepeda, N., Lindsey, R., Vul, E., and Mozer, M. (2009, January 7–10). Predicting the optimal spacing of study: A multiscale context model of memory. Proceedings of the Conference on Advances in Neural Information Processing Systems, Vancouver, BC, Canada.
16. Nagatani, K., Zhang, Q., Sato, M., Chen, Y.Y., Chen, F., and Ohkuma, T. (2019, January 13–17). Augmenting knowledge tracing by considering forgetting behavior. Proceedings of the World Wide Web Conference, San Francisco, CA, USA.10.1145/3308558.3313565 ↗
17. Lord, F. (1980). Applications of Item Response Theory to Practical Testing Problems, Erlbaum Associates.
18. Rasch, G. (1993). Probabilistic Models for Some Intelligence and Attainment Tests, MESA Press.
19. Adapting bayesian knowledge tracing to a massive open online course in edx
20. Cen, H., Koedinger, K., and Junker, B. (2006, January 26–30). Learning factors analysis–a general method for cognitive model evaluation and improvement. Proceedings of the International Conference on Intelligent Tutoring Systems, Jhongli, Taiwan.10.1007/11774303_17 ↗
21. Pavlik, P., Cen, H., and Koedinger, K. (2009, January 6–10). Performance factors analysis—A new alternative to knowledge tracing. Proceedings of the 14th International Conference on Artificial Intelligence in Education, Brighton, UK.
22. A review of recent advances in learner and skill modeling in intelligent learning environments10.1007/s11257-011-9106-8 ↗
23. Huang, Z., Liu, Q., Zhai, C., Yin, Y., and Hu, G. (2019, January 3–7). Exploring multi-objective exercise recommendations in online education systems. Proceedings of the 28th ACM International Conference on Information and Knowledge Management, Beijing, China.10.1145/3357384.3357995 ↗
24. Liu, Q., Tong, S., Liu, C., Zhao, H., Chen, E., Ma, H., and Wang, S. (2019, January 4–8). Exploiting cognitive structure for adaptive learning. Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Anchorage, AK, USA.10.1145/3292500.3330922 ↗
25. Attention is all you need
26. Khajah, M., Lindsey, R., and Mozer, M. (2016). How deep is knowledge tracing?. arXiv.
27. Yeung, C., and Yeung, D. (2018). Addressing two problems in deep knowledge tracing via prediction-consistent regularization. arXiv.10.1145/3231644.3231647 ↗
28. Abdelrahman, G., and Wang, Q. (2019, January 21–25). Knowledge tracing with sequential key-value memory networks. Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval, Paris, France.10.1145/3331184.3331195 ↗
29. Devlin, J., Chang, M., Lee, K., and Toutanova, K. (2019, January 3–5). BERT: Pre-training of deep bidirectional transformers for language understanding. Proceedings of the 2019 Conference of the North. Association for Computational Linguistics, Minneapolis, MN, USA.
30. Language models are few-shot learners
31. Fine-grained learning performance prediction via adaptive sparse self-attention networks10.1016/j.ins.2020.08.017 ↗
32. Zhu, J., Yu, W., Zheng, Z., Huang, C., and Fung, G. (2020, January 9–11). Learning from Interpretable Analysis: Attention Based Knowledge Tracing. Proceedings of the International Conference on Artificial Intelligence in Education, Ifrane, Morocco.10.1007/978-3-030-52240-7_66 ↗
33. Choi, Y., Lee, Y., Cho, J., Baek, J., Kim, B., Cha, Y., Shin, D., Bae, C., and Heo, J. (2020, January 12–14). Towards an appropriate query, key, and value computation for knowledge tracing. Proceedings of the Seventh ACM Conference on Learning, Online.10.1145/3386527.3405945 ↗
34. Shin, D., Shim, Y., Yu, H., Lee, S., Kim, B., and Choi, Y. (2021, January 11). Saint+: Integrating temporal features for ednet correctness prediction. Proceedings of the 11th International Learning Analytics and Knowledge Conference, Irvine, CA, USA.10.1145/3448139.3448188 ↗
35. Mikolov, T., Sutskever, I., Chen, K., Corrado, G., and Dean, J. (2013). Distributed representations of words and phrases and their compositionality. Adv. Neural Inf. Process. Syst., 3111–3119.
36. Oord, V., Vinyals, A., and Kavukcuoglu, K. (2017, January 4–9). Neural Discrete Representation Learning. Proceedings of the International Conference on Neural Information Processing Systems, Long Beach, CA, USA.
37. Stamper, J., Niculescu-Mizil, A., Ritter, S., Gordon, G., and Koedinger, K. (2010, January 25–28). Algebra 2005–2006 and Bridge to Algebra 2006–2007. Proceedings of the 16th ACM SIGKDD Conference on Knowledge Discovery and Data Mining Workshop, Washington, DC, USA.
38. Addressing the assessment challenge with an online system that tutors as it assesses10.1007/s11257-009-9063-7 ↗
39. Patikorn, T., Hefernan, N., and Baker, R. (2018, January 12–15). Assistments longitudinal data mining competition 2017: A preface. Proceedings of the International Conference on Educational Data Mining, Buffalo, NY, USA.
40. Accelerating t-SNE using tree-based algorithms
41. When is deep learning the best approach to knowledge tracing?
42. Paszke, A., Gross, S., Chintala, S., Chanan, G., Yang, E., DeVito, Z., Lin, Z., Desmaison, A., Antiga, L., and Lerer, A. (2017, January 4–9). Automatic diferentiation in PyTorch. Proceedings of the International Conference on Neural Information Processing Systems Workshop, Long Beach, CA, USA.
43. Kingma, D., and Ba, J. (2015, January 7–9). Adam: A Method for Stochastic Optimization. Proceedings of the International Conference on Learning Representations, San Diego, CA, USA.