journal article·2024-09-06Open access

Knowledge Graph and Personalized Answer Sequences for Programming Knowledge Tracing

Jianguo Pan, Zhengyang Dong, Lijun Yan, Xia Cai

Applied Sciences· Volume 14· Issue 17

Abstract

Knowledge tracing is a significant research area in educational data mining, aiming to predict future performance based on students’ historical learning data. In the field of programming, several challenges are faced in knowledge tracing, including inaccurate exercise representation and limited student information. These issues can lead to biased models and inaccurate predictions of students’ knowledge states. To effectively address these issues, we propose a novel programming knowledge tracing model named GPPKT (Knowledge Graph and Personalized Answer Sequences for Programming Knowledge Tracing), which enhances performance by using knowledge graphs and personalized answer sequences. Specifically, we establish the associations between well-defined knowledge concepts and exercises, incorporating student learning abilities and latent representations generated from personalized answer sequences using Variational Autoencoders (VAE) in the model. This deep knowledge tracing model employs Long Short-Term Memory (LSTM) networks and attention mechanisms to integrate the embedding vectors, such as exercises and student information. Extensive experiments are conducted on two real-world programming datasets. The results indicate that GPPKT outperforms state-of-the-art methods, achieving an AUC of 0.8840 and an accuracy of 0.8472 on the Luogu dataset, and an AUC of 0.7770 and an accuracy of 0.8799 on the Codeforces dataset. This demonstrates the superiority of the proposed model, with an average improvement of 9.03% in AUC and 2.02% in accuracy across both datasets.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: Intelligent Tutoring Systems and Adaptive Learning · AI-based Problem Solving and Planning · Topic Modeling

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

The College of Information, Mechanical and Electrical Engineering, Shanghai Normal University, Shanghai 200234, China

Funding

National Natural Science Foundation of China · Natural Science Foundation of Shanghai · National Key Research and Development Program of China

References (36)

Show source-provided references
1. Piech, C., Bassen, J., Huang, J., Ganguli, S., Sahami, M., Guibas, L.J., and Sohl-Dickstein, J. (2015, January 7–12). Deep knowledge tracing. Proceedings of the Advances in Neural Information Processing Systems, Montreal, QC, Canada.
2. Yeung, C.K., and Yeung, D.Y. (2018, January 26–28). Addressing two problems in deep knowledge tracing via prediction-consistent regularization. Proceedings of the Fifth Annual ACM Conference on Learning at Scale, London, UK.10.1145/3231644.3231647 ↗
3. Zhang, J., Shi, X., King, I., and Yeung, D.Y. (2017, January 3–7). Dynamic key-value memory networks for knowledge tracing. Proceedings of the 26th International Conference on World Wide Web, Perth, Australia.10.1145/3038912.3052580 ↗
4. Pandey, S., and Karypis, G. (2019). A self-attentive model for knowledge tracing. arXiv.
5. Long, T., Liu, Y., Shen, J., Zhang, W., and Yu, Y. (2021, January 11–15). Tracing knowledge state with individual cognition and acquisition estimation. Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, Virtual.10.1145/3404835.3462827 ↗
6. Ghosh, A., Heffernan, N., and Lan, A.S. (2020, January 23–27). Context-aware attentive knowledge tracing. Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, Virtual.10.1145/3394486.3403282 ↗
7. Nakagawa, H., Iwasawa, Y., and Matsuo, Y. (2019, January 14–17). Graph-based knowledge tracing: Modeling student proficiency using graph neural network. Proceedings of the IEEE/WIC/ACM International Conference on Web Intelligence, Thessaloniki, Greece.10.1145/3350546.3352513 ↗
8. Kingma, D.P., and Welling, M. (2013). Auto-encoding variational bayes. arXiv.
9. Long short-term memory10.1162/neco.1997.9.8.1735 ↗
10. Su, Y., Liu, Q., Liu, Q., Huang, Z., Yin, Y., Chen, E., Ding, C., Wei, S., and Hu, G. (2018, January 2–7). Exercise-enhanced sequential modeling for student performance prediction. Proceedings of the AAAI Conference on Artificial Intelligence, New Orleans, LA, USA.10.1609/aaai.v32i1.11864 ↗
11. Minn, S., Yu, Y., Desmarais, M.C., Zhu, F., and Vie, J.J. (2018, January 17–20). Deep knowledge tracing and dynamic student classification for knowledge tracing. Proceedings of the 2018 IEEE International conference on data mining (ICDM), Singapore.10.1109/ICDM.2018.00156 ↗
12. Lee, J., and Yeung, D.Y. (2019, January 4–8). Knowledge query network for knowledge tracing: How knowledge interacts with skills. Proceedings of the 9th International Conference on Learning Analytics & Knowledge, Tempe, AZ, USA.10.1145/3303772.3303786 ↗
13. Yeung, C.K. (2019). Deep-IRT: Make deep learning based knowledge tracing explainable using item response theory. arXiv.
14. Abdelrahman, G., and Wang, Q. (2019, January 21–25). Knowledge tracing with sequential key-value memory networks. Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval, Paris, France.10.1145/3331184.3331195 ↗
15. Guo, X., Huang, Z., Gao, J., Shang, M., Shu, M., and Sun, J. (2021, January 20–24). Enhancing knowledge tracing via adversarial training. Proceedings of the 29th ACM International Conference on Multimedia, Virtual.10.1145/3474085.3475554 ↗
16. Pandey, S., and Srivastava, J. (2020, January 19–23). RKT: Relation-aware self-attention for knowledge tracing. Proceedings of the 29th ACM International Conference on Information & Knowledge Management, Virtual.10.1145/3340531.3411994 ↗
17. Shen, S., Liu, Q., Chen, E., Wu, H., Huang, Z., Zhao, W., Su, Y., Ma, H., and Wang, S. (2020, January 25–30). Convolutional knowledge tracing: Modeling individualization in student learning process. Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval, Virtual.10.1145/3397271.3401288 ↗
18. Wang, W., Liu, T., Chang, L., Gu, T., and Zhao, X. (2020, January 29–30). Convolutional recurrent neural networks for knowledge tracing. Proceedings of the 2020 International Conference on Cyber-Enabled Distributed Computing and Knowledge Discovery (CyberC), Chongqing, China.10.1109/CyberC49757.2020.00054 ↗
19. Ekt: Exercise-aware knowledge tracing for student performance prediction10.1109/TKDE.2019.2924374 ↗
20. Memory: A contribution to experimental psychology10.5214/ans.0972.7531.200408 ↗
21. Nagatani, K., Zhang, Q., Sato, M., Chen, Y.Y., Chen, F., and Ohkuma, T. (2019, January 13–17). Augmenting knowledge tracing by considering forgetting behavior. Proceedings of the World Wide Web Conference, San Francisco, CA, USA.10.1145/3308558.3313565 ↗
22. Yang, Y., Shen, J., Qu, Y., Liu, Y., Wang, K., Zhu, Y., Zhang, W., and Yu, Y. (2020, January 14–18). GIKT: A graph-based interaction model for knowledge tracing. Proceedings of the Machine Learning and Knowledge Discovery in Databases: European Conference, ECML PKDD 2020, Ghent, Belgium. Proceedings, Part I.10.1007/978-3-030-67658-2_18 ↗
23. Song, Q., and Luo, W. (2023). SFBKT: A Synthetically Forgetting Behavior Method for Knowledge Tracing. Appl. Sci., 13.10.3390/app13137704 ↗
24. Estimating programming knowledge with Bayesian knowledge tracing10.1145/1595496.1562972 ↗
25. Wang, L., Sy, A., Liu, L., and Piech, C. (2017, January 25–28). Learning to Represent Student Knowledge on Programming Exercises Using Deep Learning. Proceedings of the International Educational Data Mining Society, Paper Presented at the International Conference on Educational Data Mining (EDM), Wuhan, China.10.1145/3051457.3053985 ↗
26. Piech, C., Huang, J., Nguyen, A., Phulsuksombati, M., Sahami, M., and Guibas, L. (2015, January 6–11). Learning program embeddings to propagate feedback on student code. Proceedings of the International Conference on Machine Learning, PMLR, Lille, France.
27. Swamy, V., Guo, A., Lau, S., Wu, W., Wu, M., Pardos, Z., and Culler, D. (2018, January 27–30). Deep knowledge tracing for free-form student code progression. Proceedings of the Artificial Intelligence in Education: 19th International Conference, AIED 2018, London, UK. Proceedings, Part II 19.10.1007/978-3-319-93846-2_65 ↗
28. Shi, Y., Chi, M., Barnes, T., and Price, T. (2022). Code-dkt: A code-based knowledge tracing model for programming tasks. arXiv.
29. Knowledge tracing within single programming practice using problem-solving process data10.1109/TLT.2020.3032980 ↗
30. Liang, Y., Peng, T., Pu, Y., and Wu, W. (2022). HELP-DKT: An interpretable cognitive model of how students learn programming based on deep knowledge tracing. Sci. Rep., 12.10.1038/s41598-022-07956-0 ↗
31. Sun, Z., Deng, Z.H., Nie, J.Y., and Tang, J. (2019). Rotate: Knowledge graph embedding by relational rotation in complex space. arXiv.
32. Modèles connexionnistes de l’apprentissage
33. A data repository for the EDM community: The PSLC DataShop
34. Addressing the assessment challenge with an online system that tutors as it assesses10.1007/s11257-009-9063-7 ↗
35. Choi, Y., Lee, Y., Cho, J., Baek, J., Kim, B., Cha, Y., Shin, D., Bae, C., and Heo, J. (2020, January 12–14). Towards an appropriate query, key, and value computation for knowledge tracing. Proceedings of the Seventh ACM Conference on Learning@ Scale, Virtual.10.1145/3386527.3405945 ↗
36. pyKT: A python library to benchmark deep learning based knowledge tracing models