Can Generative Artificial Intelligence Outperform Self-Instructional Learning in Computer Programming?: Impact on Motivation and Knowledge Acquisition
Source abstract
Abstract
Generative artificial intelligence tools, such as Microsoft Copilot, are transforming the teaching of programming by providing real-time feedback and personalized assistance; however, their impact on learning, motivation, and cognitive absorption remains underexplored, particularly in university settings. This study evaluates the effectiveness of Microsoft Copilot compared to instructional videos in teaching web programming in PHP, implementing a quasi-experimental design with 71 industrial engineering students in Chile, divided into two groups: one using Microsoft Copilot and the other following instructional videos, with pre- and post-tests applied to measure knowledge acquisition while surveys based on the Hedonic-Motivation System Adoption Model (HMSAM) assessed cognitive absorption (enjoyment, control, immersion, curiosity) and technology acceptance (perceived usefulness, ease of use, and intention to adopt). The results show that, while both methods improved learning, students who used instructional videos achieved greater knowledge gains, higher levels of curiosity, and a stronger intention to continue using the technique, suggesting that instructional videos, by providing structured explanations and reducing cognitive load, may be more effective in the early stages of programming learning. In contrast, AI tools could be more beneficial in advanced stages where students require adaptive feedback, providing empirical evidence on the comparative effectiveness of AI-based and video-based instruction in teaching programming and highlighting the importance of balancing structured learning with AI-driven interactivity, with the recommendation that educators integrate both approaches to optimize the learning experience, using videos for initial instruction and AI tools for personalized support.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Online Learning and Analytics · Intelligent Tutoring Systems and Adaptive Learning · Educational Games and Gamification
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Escuela de Comercio, Pontificia Universidad Católica de Valparaíso, Valparaíso 2340025, Chile
Escuela de Ingeniería Informática, Pontificia Universidad Católica de Valparaíso, Valparaíso 2340025, Chile
Funding
Regular Fondecyt Project of the National Research · Development Agency of the State of Chile
References (97)
Show source-provided references
1. Bahroun, Z., Anane, C., Ahmed, V., and Zacca, A. (2023). Transforming Education: A Comprehensive Review of Generative Artificial Intelligence in Educational Settings through Bibliometric and Content Analysis. Sustainability, 15.10.3390/su151712983 ↗
2. Luckin, R., and Holmes, W. (2016). Intelligence Unleashed: An Argument for AI in Education, UCL Knowledge Lab.
3. Jackson, V., Vasilescu, B., Russo, D., Ralph, P., Izadi, M., Prikladnicki, R., D’angelo, S., Inman, S., Andrade, A., and van der Hoek, A. (ACM Trans. Softw. Eng. Methodol., 2024). The Impact of Generative AI on Creativity in Software Development: A Research Agenda, ACM Trans. Softw. Eng. Methodol., just accepted.10.1145/3708523 ↗
4. Time Flies When You’re Having Fun: Cognitive Absorption and Beliefs about Information Technology Usage10.2307/3250951 ↗
5. The Impact of Cognitive Absorption on Perceived Usefulness and Perceived Ease of Use in On-Line Learning: An Extension of the Technology Acceptance Model10.1016/j.im.2003.12.013 ↗
6. Park, T.H., and Wiedenbeck, S. (2011, January 8–9). Learning Web Development: Challenges at an Earlier Stage of Computing Education. Proceedings of the ICER’11: ACM SIGCSE 2011 International Computing Education Research Workshop, Providence, RI, USA.10.1145/2016911.2016937 ↗
7. Ling, H.C., and Chiang, H. (2022). Sen Learning Performance in Adaptive Learning Systems: A Case Study of Web Programming Learning Recommendations. Front. Psychol., 13.10.3389/fpsyg.2022.770637 ↗
8. A Study of Programming Learning Perceptions and Effectiveness under a Blended Learning Model with Live Streaming: Comparisons between Full-Time and Working Students10.1080/10494820.2023.2198586 ↗
9. Learning and Teaching Programming: A Review and Discussion10.1076/csed.13.2.137.14200 ↗
10. Guo, P.J., Kim, J., and Rubin, R. (2014, January 4–5). How Video Production Affects Student Engagement: An Empirical Study of MOOC Videos. Proceedings of the L@S 2014—1st ACM Conference on Learning at Scale, Atlanta, GA, USA.10.1145/2556325.2566239 ↗
11. Artificial Intelligence and Its Scope in Latin American Higher Education10.56183/iberoeds.v3i1.627 ↗
12. De La Torre, A., and Baldeon-Calisto, M. (2024, January 29–30). Generative Artificial Intelligence in Latin American Higher Education: A Systematic Literature Review. Proceedings of the 12th International Symposium on Digital Forensics and Security, ISDFS, San Antonio, TX, USA.10.1109/ISDFS60797.2024.10527283 ↗
13. Hinostroza, J.E., Isaacs, S., and Bougroum, M. (2014). Information and Communications Technologies for Improving Learning Opportunities and Outcomes in Developing Countries. Learning and Education in Developing Countries, Palgrave Pivot.10.1057/9781137455970_3 ↗
14. A Study of the Difficulties of Novice Programmers10.1145/1151954.1067453 ↗
15. Active Learning Strategies for Engaging Students in Higher Education
16. Taking “Fun and Games” Seriously: Proposing the Hedonic-Motivation System Adoption Model (HMSAM)
17. Extrinsic and Intrinsic Motivation to Use Computers in the Workplace110.1111/j.1559-1816.1992.tb00945.x ↗
18. Consumer Acceptance and Use of Information Technology: Extending the Unified Theory of Acceptance and Use of Technology10.2307/41410412 ↗
19. Measuring and Defining the Experience of Immersion in Games10.1016/j.ijhcs.2008.04.004 ↗
20. Exploring the Structural Relationships of Microinteractions in Perception and Behavior by the Hedonic Motivation System Adoption Model10.1080/10447318.2024.2303552 ↗
21. Qomarul Huda, M., Aeni Hidayah, N., Nur Hafizah Hersyaf, T., and Sujoko, I. (2020, January 23–24). Asmawi Analysis of Continuance Use of Video on Demand Applications by Using the Hedonic Motivation System Adoption Model. Proceedings of the 2020 8th International Conference on Cyber and IT Service Management, CITSM, Pangkal, Indonesia.10.1109/CITSM50537.2020.9268910 ↗
22. Chakraborty, S. (2024). Generative AI in Modern Education Society. arXiv.
23. Why Do People Use Gamification Services?
24. Understanding Gender Differences in M-Health Adoption: A Modified Theory of Reasoned Action Model10.1089/tmj.2013.0092 ↗
25. Qadhi, S., and Qadhi, S. (2023). Knowledge Dynamics: Educational Pathways from Theories to Tangible Outcomes. From Theory of Knowledge Management to Practice, IntechOpen.10.5772/intechopen.1002979 ↗
26. Bratianu, C., and Garcia-Perez, A. (2023, January 7–8). Knowledge Dynamics and Expert Knowledge Translation: A Case Study. Proceedings of the 24th European Conference on Knowledge Management 2023, Lisbon, Portugal.10.34190/eckm.24.1.1382 ↗
27. Knowledge Dynamics: A Thermodynamics Approach10.1108/K-02-2019-0122 ↗
28. Bishop, J.L., and Verleger, M.A. (2013, January 23–26). The Flipped Classroom: A Survey of the Research. Proceedings of the ASEE Annual Conference and Exposition, Atlanta, GA, USA.
29. The Effects of Integrating Mobile Devices with Teaching and Learning on Students’ Learning Performance: A Meta-Analysis and Research Synthesis10.1016/j.compedu.2015.11.008 ↗
30. A Meta-Analysis of Three Types of Interaction Treatments in Distance Education10.3102/0034654309333844 ↗
31. Comparing the Effectiveness of Video-Based Learning and Game-Based Learning Using Teacher-Authored Video Games for Online Software Engineering Education10.1109/TE.2022.3142688 ↗
32. Deci, E.L., and Ryan, R.M. (1985). Intrinsic Motivation and Self-Determination in Human Behavior. Intrinsic Motivation and Self-Determination in Human Behavior, Springer.10.1007/978-1-4899-2271-7 ↗
33. Exploring the Effects of Web 2.0 Technology on Individual and Collaborative Learning Performance in Relation to Self-Regulation of Learners
34. Examining the Effect of Generative AI on Students’ Motivation and Writing Self-Efficacy10.29140/dal.v1.102324 ↗
35. Curiosity and the Pleasures of Learning: Wanting and Liking New Information10.1080/02699930541000101 ↗
36. Dousay, T.A. (2014). Multimedia Design and Situational Interest: A Look at Juxtaposition and Measurement. Educational Media and Technology Yearbook, Springer.10.1007/978-3-319-06314-0_5 ↗
37. Krouska, A., Mylonas, P., Kabassi, K., Caro, J., Sgouropoulou, C., Hmoud, M., Swaity, H., Hamad, N., Karram, O., and Daher, W. (2024). Higher Education Students’ Task Motivation in the Generative Artificial Intelligence Context: The Case of ChatGPT. Information, 15.10.3390/info15010033 ↗
38. Abdelshiheed, M., Maniktala, M., Barnes, T., and Chi, M. (2023). Assessing Competency Using Metacognition and Motivation: The Role of Time-Awareness in Preparation for Future Learning. arXiv.
39. Perceived Usefulness, Perceived Ease of Use, and User Acceptance of Information Technology10.2307/249008 ↗
40. Ghimire, A., and Edwards, J. (2024). Generative AI Adoption in Classroom in Context of Technology Acceptance Model (TAM) and the Innovation Diffusion Theory (IDT). arXiv.10.1109/IETC61393.2024.10564292 ↗
41. User Acceptance of Information Technology: Toward a Unified View10.2307/30036540 ↗
42. Steinert, S., Avila, K.E., Ruzika, S., Kuhn, J., and Küchemann, S. (2023). Harnessing Large Language Models to Enhance Self-Regulated Learning via Formative Feedback. arXiv.10.1186/s40561-024-00354-1 ↗
43. Lin, Z., and Ng, Y.L. (2024). Unraveling Gratifications, Concerns, and Acceptance of Generative Artificial Intelligence. Int. J. Hum. Comput. Interact., 1–18.10.1080/10447318.2024.2436749 ↗
44. Clark, R.C., and Mayer, R.E. (2016). E-Learning and the Science of Instruction: Proven Guidelines for Consumers and Designers of Multimedia Learning, John Wiley & Sons.10.1002/9781119239086 ↗
45. Al-Abdullatif, A.M. (2024). Modeling Teachers’ Acceptance of Generative Artificial Intelligence Use in Higher Education: The Role of AI Literacy, Intelligent TPACK, and Perceived Trust. Educ. Sci., 14.10.3390/educsci14111209 ↗
46. Instructional Control of Cognitive Load in the Training of Complex Cognitive Tasks10.1007/BF02213420 ↗
47. Mayer, R.E. (2009). Multimedia Learning. Multimedia Learning, Cambridge University Press. [2nd ed.].10.1017/CBO9780511811678 ↗
48. da G.C. Effects of Interactive Video Annotations on Students’ Browsing Behavior and Perceived Workload10.1145/3357385.3357389 ↗
49. Failure Rates in Introductory Programming10.1145/1272848.1272879 ↗
50. Suzuki, R., Kato, J., and Yatani, K. (2020). ClassCode: An Interactive Teaching and Learning Environment for Programming Education in Classrooms. arXiv.
51. Yang, C., Thung, F., and Lo, D. (2022, January 15–18). Efficient Search of Live-Coding Screencasts from Online Videos. Proceedings of the 2022 IEEE International Conference on Software Analysis, Evolution and Reengineering, SANER, Honolulu, HI, USA.10.1109/SANER53432.2022.00021 ↗
52. Khandwala, K., and Guo, P.J. (2018, January 26–28). Codemotion: Expanding the Design Space of Learner Interactions with Computer Programming Tutorial Videos. Proceedings of the 5th Annual ACM Conference on Learning at Scale, New York, NY, USA.10.1145/3231644.3231652 ↗
53. Li, W., Pea, R., Haber, N., and Subramonyam, H. (2024). Tutorly: Turning Programming Videos Into Apprenticeship Learning Environments with LLMs. arXiv.
54. Buffardi, K., and Wang, R. (2022, January 8–13). Integrating Videos with Programming Practice. Proceedings of the Annual Conference on Innovation and Technology in Computer Science Education, ITiCSE, Dublin, Ireland.
55. McGowan, A., Anderson, N., Trombino, G., Sage, P., Adhikari, J., and Galway, L. (2023, January 13–15). Learning to Code—Investigating the Adoption of Video-Based Lab Solutions for University-Level Novice Programmers. Proceedings of the ICERI2023 Proceedings, Seville, Spain.10.21125/iceri.2023.0380 ↗
56. Gamification Improves Learning: Experience in a Training Activity of Computer Programming in Higher Education10.1111/jcal.13000 ↗
57. Investigation of Continuous Assessment of Correctness in Introductory Programming
58. Kadar, R., Mahlan, S.B., Shamsuddin, M., Othman, J., and Wahab, N.A. (2022, January 21–22). Analysis of Factors Contributing to the Difficulties in Learning Computer Programming among Non-Computer Science Students. Proceedings of the 2022 12th IEEE Symposium on Computer Applications and Industrial Electronics, ISCAIE, Penang, Malaysia.10.1109/ISCAIE54458.2022.9794546 ↗
59. Esche, S., and Weihe, K. (2023, January 7–12). Choosing a Didactic Basis for an Instructional Video: What Are the Implications for Novice Programmers?. Proceedings of the Annual Conference on Innovation and Technology in Computer Science Education, ITiCSE, Turku, Finland.10.1145/3587102.3588795 ↗
60. Nguyen-Duc, A., Cabrero-Daniel, B., Przybylek, A., Arora, C., Khanna, D., Herda, T., Rafiq, U., Melegati, J., Guerra, E., and Kemell, K.-K. (2023). Generative Artificial Intelligence for Software Engineering—A Research Agenda. arXiv.10.2139/ssrn.4622517 ↗
61. Generative AI for Software Practitioners10.1109/MS.2023.3265877 ↗
62. Accelerating Innovation With Generative AI: AI-Augmented Digital Prototyping and Innovation Methods10.1109/EMR.2023.3272799 ↗
63. Ulfsnes, R., Moe, N.B., Stray, V., and Skarpen, M. (2024). Transforming Software Development with Generative AI: Empirical Insights on Collaboration and Workflow. Generative AI for Effective Software Development, Springer.10.1007/978-3-031-55642-5_10 ↗
64. Calegario, F., Burégio, V., Erivaldo, F., Moraes, D., Andrade, C., Felix, K., Barbosa, N., Lucas Da, P., Lucena, S., and França, C. (2023). Exploring the Intersection of Generative AI and Software Development. arXiv.
65. Cypress Copilot: Development of an AI Assistant for Boosting Productivity and Transforming Web Application Testing10.1109/ACCESS.2024.3521407 ↗
66. Mahadevappa, P., Muzammal, S.M., Tayyab, M., Mahadevappa, P., Muzammal, S.M., and Tayyab, M. (2025). Introduction to Generative AI in Web Engineering: Concepts and Applications, IGI Global.10.4018/979-8-3693-3703-5.ch015 ↗
67. Ho, C.L., Liu, X.Y., Qiu, Y.W., and Yang, S.Y. (2024, January 22–24). Research on Innovative Applications and Impacts of Using Generative AI for User Interface Design in Programming Courses. Proceedings of the International Conference on Information Technology, Data Science, and Optimization, Taipei, Taiwan.10.1145/3658549.3658566 ↗
68. Jayachandran, D., Maldikar, P., Love, T.S., and Blum, J.J. (2024, January 25–27). Leveraging Generative Artificial Intelligence to Broaden Participation in Computer Science. Proceedings of the 2024 AAAI Spring Symposium Series, Stanford, CA, USA.
69. Keuning, H., Alpizar-Chacon, I., Lykourentzou, I., Beehler, L., Köppe, C., de Jong, I., and Sosnovsky, S. (2024, January 12–17). Students’ Perceptions and Use of Generative AI Tools for Programming Across Different Computing Courses. Proceedings of the 24th Koli Calling International Conference on Computing Education Research, Koli, Finland.10.1145/3699538.3699546 ↗
70. The Effect of Generative Artificial Intelligence (AI)-Based Tool Use on Students’ Computational Thinking Skills, Programming Self-Efficacy and Motivation10.1016/j.caeai.2023.100147 ↗
71. Wilson, S.E., and Nishimoto, M. (2024). Assessing Learning of Computer Programing Skills in the Age of Generative Artificial Intelligence. J. Biomech. Eng., 146.10.1115/1.4064364 ↗
72. Empowering Learners with AI-Generated Content for Programming Learning and Computational Thinking: The Lens of Extended Effective Use Theory10.1111/jcal.12996 ↗
73. Integrating Generative AI into the Software Development Lifecycle: Impacts on Code Quality and Maintenance10.30574/ijsra.2024.13.1.1837 ↗
74. Yehia, E. (2024). Developments on Generative AI. AI and Emerging Technologies: Automated Decision-Making, Digital Forensics, and Ethical Considerations, CRC Press.10.1201/9781003501152-9 ↗
75. Toward Artificial Intelligence-Human Paired Programming: A Review of the Educational Applications and Research on Artificial Intelligence Code-Generation Tools10.1177/07356331241240460 ↗
76. Boguslawski, S., Deer, R., and Dawson, M.G. (Inf. Learn. Sci., 2024). Programming Education and Learner Motivation in the Age of Generative AI: Student and Educator Perspectives, Inf. Learn. Sci., ahead-of-print.10.1108/ILS-10-2023-0163 ↗
77. Generative Artificial Intelligence in Computer Programming: Does It Enhance Learning, Motivation, and the Learning Environment?10.1109/ACCESS.2025.3532883 ↗
78. AI Chatbots in Programming Education: Students’ Use in a Scientific Computing Course and Consequences for Learning10.1016/j.caeai.2024.100290 ↗
79. Pesovski, I., Santos, R., Henriques, R., and Trajkovik, V. (2024). Generative AI for Customizable Learning Experiences. Sustainability, 16.10.3390/su16073034 ↗
80. Frankford, E., Sauerwein, C., Bassner, P., Krusche, S., and Breu, R. (2024, January 14–20). AI-Tutoring in Software Engineering Education Experiences with Large Language Models in Programming Assessments. Proceedings of the 46th International Conference on Software Engineering, Lisbon, Portugal.10.1145/3639474.3640061 ↗
81. Li, H., Xu, T., Zhang, C., Chen, E., Liang, J., Fan, X., Li, H., Tang, J., and Wen, Q. (2024). Bringing Generative AI to Adaptive Learning in Education. arXiv.
82. Advancing Students’ Academic Excellence in Distance Education: Exploring the Potential of Generative AI Integration to Improve Academic Writing Skills10.55982/openpraxis.16.2.649 ↗
83. Ko, S., Chan, S.C.H., Ko, S., and Chan, S.C.H. (2024). A Framework for the Responsible Integration of Generative AI Tools in Learning, IGI Global.10.4018/979-8-3373-1017-6.ch006 ↗
84. Lakens, D. (2013). Calculating and Reporting Effect Sizes to Facilitate Cumulative Science: A Practical Primer for t-Tests and ANOVAs. Front. Psychol., 4.10.3389/fpsyg.2013.00863 ↗
85. Cognitive Load during Problem Solving: Effects on Learning10.1207/s15516709cog1202_4 ↗
86. Du, X., Liu, M., Wang, K., Wang, H., Liu, J., Chen, Y., Feng, J., Sha, C., Peng, X., and Lou, Y. (2024, January 14–20). Evaluating Large Language Models in Class-Level Code Generation. Proceedings of the 46th International Conference on Software Engineering, Lisbon, Portugal.10.1145/3597503.3639219 ↗
87. Vaithilingam, P., Zhang, T., and Glassman, E.L. (2022, January 29). Expectation vs. Experience: Evaluating the Usability of Code Generation Tools Powered by Large Language Models. Proceedings of the CHI Conference on Human Factors in Computing Systems, New Orleans, LA, USA.10.1145/3491101.3519665 ↗
88. The Effects of Type I Error Rate and Power of the ANCOVA F-Test and Selected Alternatives under Non-Normality and Variance Heterogeneity10.1080/00220970109599493 ↗
89. Ch, K.P., Poremba, K.D., and Rowell, R.K. (1997). Testing for Homogeneity of Slopes in Analysis of Covariance: A Tutorial, ERIC Institute of Education Science.
90. Robustness of Analysis of Covariance (Ancova) under the Distributions Assumptions and Variance Homogeneity10.15312/EurasianJVetSci.2020.260 ↗
91. The Effectiveness of a Meaningful Learning-Based Evaluation Model for Context-Aware Mobile Learning10.1111/bjet.12147 ↗
92. Kharrufa, A., and Johnson, I. (2024, January 5–7). The Potential and Implications of Generative AI on HCI Education. Proceedings of the 6th Annual Symposium on HCI Education, New York, NY, USA.10.1145/3658619.3658627 ↗
93. Hou, I., Mettille, S., Li, Z., Man, O., Zastudil, C., and MacNeil, S. (2024, January 29). The Effects of Generative AI on Computing Students’ Help-Seeking Preferences. Proceedings of the 26th Australian Computing Education Conference, Sydney, Australia.10.1145/3636243.3636248 ↗
94. Chen, E., Lee, J.E., Lin, J., and Koedinger, K. (2024, January 18–20). GPTutor: Great Personalized Tutor with Large Language Models for Personalized Learning Content Generation. Proceedings of the L@S 2024—11th ACM Conference on Learning @ Scale, Atlanta, GA, USA.10.1145/3657604.3664718 ↗
95. Closing Achievement Gaps: Revisiting Benjamin S. Bloom’s “Learning for Mastery”
96. Review of A Taxonomy for Learning, Teaching, and Assessing: A Revision of Bloom’s Taxonomy of Educational Objectives Complete Edition
97. Learning for Mastery. Instruction and Curriculum. Regional Education Laboratory for the Carolinas and Virginia, Topical Papers and Reprints, Number 1