journal article·2025-04-30Open access

AI-Powered System to Facilitate Personalized Adaptive Learning in Digital Transformation

Yao Yao, Horacio González-Vélez

Applied Sciences· Volume 15· Issue 9

Abstract

As Large Language Models (LLMs) incorporate generative Artificial Intelligence (AI) and complex machine learning algorithms, they have proven to be highly effective in assisting human users with complex professional tasks through natural language interaction. However, in addition to their current capabilities, LLMs occasionally generate responses that contain factual inaccuracies, stemming from their dependence on the parametric knowledge they encapsulate. To avoid such inaccuracies, also known as hallucinations, people use domain-specific knowledge (expertise) to support LLMs in the corresponding task, but the necessary knowledge engineering process usually requires considerable manual effort from experts. In this paper, we developed an approach to leverage the collective strengths of multiple agents to automatically facilitate the knowledge engineering process and then use the learned knowledge and Retrieval Augmented Generation (RAG) pipelines to optimize the performance of LLMs in domain-specific tasks. Through this approach, we effectively build AI assistants based on particular customized knowledge to help students better carry out personalized adaptive learning in digital transformation. Our initial tests demonstrated that integrating a Knowledge Graph (KG) within a RAG framework significantly improved the quality of domain-specific outputs generated by the LLMs. The results also revealed performance fluctuations for LLMs across varying contexts, underscoring the critical need for domain-specific knowledge support to enhance AI-driven adaptive learning systems.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: Engineering Education and Technology

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

National College of Ireland, D01 N6P6 Dublin, Ireland

Funding

European Commission DIGITAL

References (68)

Show source-provided references
1. Xing, M., Zhang, R., Xue, H., Chen, Q., Yang, F., and Xiao, Z. (2024, January 25–29). Understanding the weakness of large language model agents within a complex android environment. Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, Barcelona, Spain.10.1145/3637528.3671650 ↗
2. Using early LLMs for corpus linguistics: Examining ChatGPT’s potential and limitations10.1016/j.acorp.2024.100089 ↗
3. Fan, W., Ding, Y., Ning, L., Wang, S., Li, H., Yin, D., Chua, T.S., and Li, Q. (2024, January 25–29). A survey on rag meeting llms: Towards retrieval-augmented large language models. Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, Barcelona, Spain.10.1145/3637528.3671470 ↗
4. Wang, H., and Li, Y.F. (2023, January 20–23). Large Language Model Empowered by Domain-Specific Knowledge Base for Industrial Equipment Operation and Maintenance. Proceedings of the 2023 5th International Conference on System Reliability and Safety Engineering (SRSE), Beijing, China.10.1109/SRSE59585.2023.10336112 ↗
5. A survey on LLM-generated text detection: Necessity, methods, and future directions10.1162/coli_a_00549 ↗
6. Hofer, M., Obraczka, D., Saeedi, A., Köpcke, H., and Rahm, E. (2024). Construction of Knowledge Graphs: Current State and Challenges. Information, 15.10.3390/info15080509 ↗
7. From task structures to world models: What do LLMs know?10.1016/j.tics.2024.02.008 ↗
8. Harnessing the power of llms in practice: A survey on chatgpt and beyond
9. Limitations of large language models in medical applications10.1093/postmj/qgad069 ↗
10. Asher, N., Bhar, S., Chaturvedi, A., Hunter, J., and Paul, S. (2023, January 13–14). Limits for Learning with Language Models. Proceedings of the 12th Joint Conference on Lexical and Computational Semantics (*SEM 2023), Toronto, ON, Canada.10.18653/v1/2023.starsem-1.22 ↗
11. A domain-specific next-generation large language model (LLM) or ChatGPT is required for biomedical engineering and research10.1007/s10439-023-03306-x ↗
12. PreparedLLM: Effective pre-pretraining framework for domain-specific large language models10.1080/20964471.2024.2396159 ↗
13. Li, Y., Ma, S., Wang, X., Huang, S., Jiang, C., Zheng, H.T., Xie, P., Huang, F., and Jiang, Y. (2023). EcomGPT: Instruction-tuning Large Language Model with Chain-of-Task Tasks for E-commerce. arXiv.10.1609/aaai.v38i17.29820 ↗
14. Wu, S., Irsoy, O., Lu, S., Dabravolski, V., Dredze, M., Gehrmann, S., Kambadur, P., Rosenberg, D., and Mann, G. (2023). Bloomberggpt: A large language model for finance. arXiv.
15. Nguyen, H.T. (2023). A Brief Report on LawGPT 1.0: A Virtual Legal Assistant Based on GPT-3. arXiv.
16. BioGPT: Generative pre-trained transformer for biomedical text generation and mining10.1093/bib/bbac409 ↗
17. Biomedlm: A domain-specific large language model for biomedical text
18. Deng, C., Zhang, T., He, Z., Chen, Q., Shi, Y., Zhou, L., Fu, L., Zhang, W., Wang, X., and Zhou, C. (2023). Learning A Foundation Language Model for Geoscience Knowledge Understanding and Utilization. arXiv.
19. Bi, Z., Zhang, N., Xue, Y., Ou, Y., Ji, D., Zheng, G., and Chen, H. (2023). Oceangpt: A large language model for ocean science tasks. arXiv.10.18653/v1/2024.acl-long.184 ↗
20. Soman, S., and HG, R. (2023, January 25–28). Observations on LLMs for telecom domain: Capabilities and limitations. Proceedings of the Third International Conference on AI-ML Systems, Bangalore, India.10.1145/3639856.3639892 ↗
21. Unifying large language models and knowledge graphs: A roadmap10.1109/TKDE.2024.3352100 ↗
22. Improving the domain adaptation of retrieval augmented generation (RAG) models for open domain question answering10.1162/tacl_a_00530 ↗
23. Amarnath, N.S., and Nagarajan, R. (2024, January 15–17). An Intelligent Retrieval Augmented Generation Chatbot for Contextually-Aware Conversations to Guide High School Students. Proceedings of the 2024 4th International Conference on Sustainable Expert Systems (ICSES), Kaski, Nepal.10.1109/ICSES63445.2024.10762977 ↗
24. Ko, H.T., Liu, Y.K., Tsai, Y.C., and Suen, S. (2024, January 14–16). Enhancing Python Learning Through Retrieval-Augmented Generation: A Theoretical and Applied Innovation in Generative AI Education. Proceedings of the International Conference on Innovative Technologies and Learning, Tartu, Estonia.10.1007/978-3-031-65884-6_17 ↗
25. Long, C., Liu, Y., Ouyang, C., and Yu, Y. (2024). Bailicai: A Domain-Optimized Retrieval-Augmented Generation Framework for Medical Applications. arXiv.
26. RefAI: A GPT-powered retrieval-augmented generative tool for biomedical literature recommendation and summarization10.1093/jamia/ocae129 ↗
27. Xu, Z., Cruz, M.J., Guevara, M., Wang, T., Deshpande, M., Wang, X., and Li, Z. (2024, January 14–18). Retrieval-augmented generation with knowledge graphs for customer service question answering. Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval, Washington, DC, USA.10.1145/3626772.3661370 ↗
28. Halgin, F., and Mohammedand, A.T. (2024, January 11–13). Intelligent patent processing: Leveraging retrieval-augmented generation for enhanced consultant services. Proceedings of the International Symposium on Digital Transformation, Växjö, Sweden.
29. Retrieval-augmented generation for knowledge-intensive nlp tasks
30. Perković, G., Drobnjak, A., and Botički, I. (2024, January 20–24). Hallucinations in LLMs: Understanding and Addressing Challenges. Proceedings of the 2024 47th MIPRO ICT and Electronics Convention (MIPRO), Opatija, Croatia.10.1109/MIPRO60963.2024.10569238 ↗
31. Ayala, O., and Bechard, P. (2024, January 16–21). Reducing hallucination in structured outputs via Retrieval-Augmented Generation. Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 6: Industry Track), Mexico City, Mexico.10.18653/v1/2024.naacl-industry.19 ↗
32. Knowledge graphs: Opportunities and challenges10.1007/s10462-023-10465-9 ↗
33. Song, Y., Li, W., Dai, G., and Shang, X. (2023). Advancements in Complex Knowledge Graph Question Answering: A Survey. Electronics, 12.10.3390/electronics12214395 ↗
34. Edge, D., Trinh, H., Cheng, N., Bradley, J., Chao, A., Mody, A., Truitt, S., and Larson, J. (2024). From local to global: A graph rag approach to query-focused summarization. arXiv.
35. Meyer, L.P., Stadler, C., Frey, J., Radtke, N., Junghanns, K., Meissner, R., Dziwis, G., Bulert, K., and Martin, M. (2023, January 20–29). Llm-assisted knowledge graph engineering: Experiments with chatgpt. Proceedings of the Working Conference on Artificial Intelligence Development for a Resilient and Sustainable Tomorrow, Leipzig, Germany.10.1007/978-3-658-43705-3_8 ↗
36. Llms for knowledge graph construction and reasoning: Recent capabilities and future opportunities10.1007/s11280-024-01297-w ↗
37. Personalized adaptive learning: An emerging pedagogical approach enabled by a smart learning environment10.1186/s40561-019-0089-y ↗
38. A systematic literature review on personalised learning in the higher education context10.1007/s10758-022-09628-4 ↗
39. Chen, D.L., Aaltonen, K., Lampela, H., and Kujala, J. (2024). The Design and Implementation of an Educational Chatbot with Personalized Adaptive Learning Features for Project Management Training. Technol. Knowl. Learn., 1–26.10.1007/s10758-024-09807-5 ↗
40. Shin, D., Shim, Y., Yu, H., Lee, S., Kim, B., and Choi, Y. (2021, January 12–16). Saint+: Integrating temporal features for ednet correctness prediction. Proceedings of the LAK21: 11th International Learning Analytics and Knowledge Conference, Irvine, CA, USA.10.1145/3448139.3448188 ↗
41. Mejeh, M., and Sarbach, L. (2024). Co-design: From Understanding to Prototyping an Adaptive Learning Technology to Enhance Self-regulated Learning. Technol. Knowl. Learn., 1–26.10.1007/s10758-024-09788-5 ↗
42. Lozano, E.A., Sánchez-Torres, C.E., López-Nava, I.H., and Favela, J. (2023, January 28–29). An open framework for nonverbal communication in human-robot interaction. Proceedings of the International Conference on Ubiquitous Computing and Ambient Intelligence, Riviera Maya, Mexico.10.1007/978-3-031-48642-5_3 ↗
43. Social Robots for Healthcare and Education in Latin America10.1145/3653320 ↗
44. Reinforcement learning tutor better supported lower performers in a math task10.1007/s10994-023-06423-9 ↗
45. Kulshreshtha, D., Shayan, M., Belfer, R., Reddy, S., Serban, I.V., and Kochmar, E. (2022). Few-shot question generation for personalized feedback in intelligent tutoring systems. PAIS 2022, IOS Press.10.3233/FAIA220062 ↗
46. Design of an adaptive e-learning system based on multi-agent approach and reinforcement learning10.48084/etasr.3905 ↗
47. Yekollu, R.K., Bhimraj Ghuge, T., Sunil Biradar, S., Haldikar, S.V., and Farook Mohideen Abdul Kader, O. (2024, January 2–3). AI-driven personalized learning paths: Enhancing education through adaptive systems. Proceedings of the International Conference on Smart data intelligence, Trichy, India.10.1007/978-981-97-3191-6_38 ↗
48. An adaptable and personalized framework for top-N course recommendations in online learning10.1038/s41598-024-56497-1 ↗
49. Bilyalova, A., Salimova, D., and Zelenina, T. (2019, January 15–18). Digital transformation in education. Proceedings of the Integrated Science in Digital Age: ICIS 2019, Munich, Germany.10.1007/978-3-030-22493-6_24 ↗
50. Digital transformation in education: A systematic review of education 4.010.1016/j.techfore.2023.122664 ↗
51. Extrinsic and Intrinsic Personalization in the Digital Transformation of Education
52. An artificial intelligence educational strategy for the digital transformation10.1007/s12008-020-00702-8 ↗
53. LearnSmart, adaptive teaching, and student learning effectiveness: An empirical investigation10.1080/08832323.2016.1274711 ↗
54. Systematic review of adaptive learning research designs, context, strategies, and technologies from 2009 to 201810.1007/s11423-020-09793-2 ↗
55. Exploring the efficacy of adaptive learning technologies in online education: A longitudinal analysis of student engagement and performance
56. Deception abilities emerged in large language models10.1073/pnas.2317967121 ↗
57. Oruganti, S., Nirenburg, S., English, J., and McShane, M. (2023, January 25–27). Automating Knowledge Acquisition for Content-Centric Cognitive Agents Using LLMs. Proceedings of the AAAI Symposium Series, Arlington, VA, USA.
58. Yao, Y., González-Vélez, H., and Croitoru, M. (2024, January 16–18). Explanatory Dialogues with Active Learning for Rule-based Expertise. Proceedings of the RuleML+ RR (Companion), Bucharest, Romania.
59. Shekhar, S., Favier, A., Alami, R., and Croitoru, M. (2023, January 12–14). A Knowledge Rich Task Planning Framework for Human-Robot Collaboration. Proceedings of the International Conference on Innovative Techniques and Applications of Artificial Intelligence, Cambridge, UK.10.1007/978-3-031-47994-6_25 ↗
60. Dong, Q., Li, L., Dai, D., Zheng, C., Ma, J., Li, R., Xia, H., Xu, J., Wu, Z., and Chang, B. (2024, January 12–16). A survey on in-context learning. Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, Miami, FL, USA.10.18653/v1/2024.emnlp-main.64 ↗
61. Team, G., Mesnard, T., Hardin, C., Dadashi, R., Bhupatiraju, S., Pathak, S., Sifre, L., Rivière, M., Kale, M.S., and Love, J. (2024). Gemma: Open models based on gemini research and technology. arXiv.
62. Meta, A. (2024, November 01). Llama 3. Available online: https://llama.meta.com/llama3/.
63. Jiang, A.Q., Sablayrolles, A., Mensch, A., Bamford, C., Chaplot, D.S., Casas, D.d.l., Bressand, F., Lengyel, G., Lample, G., and Saulnier, L. (2023). Mistral 7B. arXiv.
64. Hendrycks, D., Burns, C., Basart, S., Zou, A., Mazeika, M., Song, D., and Steinhardt, J. (2020). Measuring massive multitask language understanding. arXiv.
65. Goutte, C., and Gaussier, E. (2005, January 21–23). A probabilistic interpretation of precision, recall and F-score, with implication for evaluation. Proceedings of the European Conference on Information Retrieval, Santiago de Compostela, Spain.10.1007/978-3-540-31865-1_25 ↗
66. Reimers, N., and Gurevych, I. (2019, January 3–7). Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing, Hong Kong, China.10.18653/v1/D19-1410 ↗
67. Bhawal, S. (2025, April 27). Resume Dataset. Data Set. Available online: https://www.kaggle.com/datasets/snehaanbhawal/resume-dataset.
68. Grattafiori, A., Dubey, A., Jauhri, A., Pandey, A., Kadian, A., Al-Dahle, A., Letman, A., Mathur, A., Schelten, A., and Vaughan, A. (2024). The llama 3 herd of models. arXiv.