journal article·2026-05-14Open access

AI as a Procedural Equalizer: Performance Comparison in Programming-Based Engineering Coursework Following the Emergence of Generative AI

Ghazal Barari, Jorge Ortega-Moody, Kouroush Jenab, Tyler Ward, Karl Siebold

Applied Sciences· Volume 16· Issue 10

Abstract

Generative artificial intelligence (AI), especially large language models (LLMs) that can write and debug code, is changing how students approach programming work in engineering education. Unlike more open-ended conceptual or modeling tasks, programming fits closely with what these systems do well: generating syntax, fixing errors, building procedural logic, and completing code structures. Hence, programming coursework may be one of the areas in which AI changes performance patterns in a measurable way. This study examines whether that shift appears in actual student outcomes. Using a retrospective pre/post design, it compares results from a pre-AI period (2021–2022) with results from a post-AI period (2023–2025), when generative AI tools became widely available to students. The focal assessment is a comprehensive programming project graded with the same rubric across multiple sections and terms. Performance is evaluated through descriptive statistics, distributional comparisons, and mastery thresholds (≥80%). The post-AI period shows a rise in overall scores, along with strong clustering near the top of the scale. Lower- and middle-range scores become much less common, most students fall in the highest score band, and overall variability declines. These results suggest that generative AI acts as a procedural equalizer in programming contexts, referring to the role of generative AI in reducing performance differences by assisting with rule-based, syntax-driven, and execution-oriented aspects of tasks, thereby raising baseline outcomes while compressing variation among students. It appears to raise lower-end performance and make outcomes more consistent, but it also narrows the spread among stronger students and creates a ceiling effect. That pattern raises questions about assessment validity, skill differentiation, and what “mastery” means when AI can handle much of the procedural work. Using multi-term data from authentic online courses, this study adds empirical evidence to the growing literature on AI in engineering education and identifies programming coursework as a setting where generative AI may have already changed performance dynamics in a structural way.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: Teaching and Learning Programming · Artificial Intelligence in Healthcare and Education · Ethics and Social Impacts of AI

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

School of Engineering, Embry-Riddle Aeronautical University, Daytona, FL 32114, USA

Department of Engineering Sciences, Morehead State University, Morehead, KY 40351, USA

Department of Computer Science, University of Kentucky, Lexington, KY 40506, USA

References (51)

Show source-provided references
1. ChatGPT for good? On opportunities and challenges of large language models for education10.1016/j.lindif.2023.102274 ↗
2. What if the devil is my guardian angel: ChatGPT as a case study of using chatbots in education10.1186/s40561-023-00237-x ↗
3. The impact of generative AI on higher education learning and teaching: A study of educators’ perspectives10.1016/j.caeai.2024.100221 ↗
4. “ChatGPT seems too good to be true”: College students’ use and perceptions of generative AI10.1016/j.caeai.2024.100294 ↗
5. Generative AI in higher education: Seeing ChatGPT through universities’ policies, resources, and guidelines10.1016/j.caeai.2024.100326 ↗
6. Finnie-Ansley, J., Denny, P., Becker, B.A., Luxton-Reilly, A., and Prather, J. (2022). The robots are coming: Exploring the implications of OpenAI Codex on introductory programming. Australasian Computing Education Conference (ACE ’22), Association for Computing Machinery.10.1145/3511861.3511863 ↗
7. Nguyen, N., and Nadi, S. (2022). An empirical evaluation of GitHub Copilot’s code suggestions. Proceedings of the 19th International Conference on Mining Software Repositories (MSR ’22), Association for Computing Machinery.10.1145/3524842.3528470 ↗
8. Yetistiren, B., Ozsoy, I., and Tüzün, E. (2022). Assessing the quality of GitHub Copilot’s code generation. Proceedings of the 18th International Conference on Predictive Models and Data Analytics in Software Engineering (PROMISE ’22), Association for Computing Machinery.10.1145/3558489.3559072 ↗
9. “It’s weird that it knows what I want”: Usability and interactions with Copilot for novice programmers10.1145/3617367 ↗
10. “Will I be replaced?” Assessing ChatGPT’s effect on software development and programmer perceptions of AI tools10.1016/j.scico.2024.103111 ↗
11. Assessing and analyzing the correctness of GitHub Copilot’s code suggestions10.1145/3715108 ↗
12. Chatting and cheating: Ensuring academic integrity in the era of ChatGPT10.1080/14703297.2023.2190148 ↗
13. Chen, B., Lewis, C.M., West, M., and Zilles, C.B. (2024). Plagiarism in the age of generative AI: Cheating method change and learning loss in an intro to CS course. Eleventh ACM Conference on Learning @ Scale (L@S ’24), Atlanta, GA, USA, 18–20 July 2024, Association for Computing Machinery.10.1145/3657604.3662046 ↗
14. ChatGPT unveiled: Understanding perceptions of academic integrity in higher education: A qualitative approach10.1007/s10805-024-09543-6 ↗
15. Chat or cheat? Academic dishonesty, risk perceptions, and ChatGPT usage in higher education students10.1007/s11218-025-10080-2 ↗
16. To use or not to use ChatGPT in higher education? A study of students’ acceptance and use of technology10.1080/10494820.2023.2209881 ↗
17. Generative AI in higher education assessments: Examining risk and tech-savviness on student’s adoption10.1177/02734753241302459 ↗
18. ChatGPT in academia: University students’ attitudes towards the use of ChatGPT and plagiarism10.1007/s10805-025-09603-5 ↗
19. Batista, J., Mesquita, A., and Carnaz, G. (2024). Generative AI and higher education: Trends, challenges, and future directions from a systematic literature review. Information, 15.10.20944/preprints202409.1496.v1 ↗
20. ChatGPT in higher education: A synthesis of the literature and a future research agenda10.1007/s10639-024-12723-x ↗
21. Ogunleye, B., Adepoju, O., and Chen, H. (2024). A systematic review of generative AI for teaching and learning in higher education. Educ. Sci., 14.10.3390/educsci14060636 ↗
22. A systematic literature review of empirical research on applying generative artificial intelligence in education10.1007/s44366-024-0028-5 ↗
23. Ghosh, S. (2024, January 23–26). ChatGPT as a tool for equitable education in engineering classes. Proceedings of the 2024 ASEE Annual Conference Exposition, Portland, OR, USA.
24. Augmented intelligence in programming learning: Examining student views on the use of ChatGPT for programming learning10.1016/j.chbah.2023.100005 ↗
25. Would ChatGPT-facilitated programming mode impact college students’ programming behaviors, performances, and perceptions? An empirical study10.1186/s41239-024-00446-5 ↗
26. AI chatbots in programming education: Students’ use in a scientific computing course and consequences for learning10.1016/j.caeai.2024.100290 ↗
27. Teaching and learning computer programming using ChatGPT: A rapid review of literature amid the rise of generative AI technologies10.1007/s10639-025-13452-5 ↗
28. AI in the classroom: Exploring students’ interaction with ChatGPT in programming learning10.1007/s10639-025-13337-7 ↗
29. Programming education with ChatGPT: Outcomes for beginners and intermediate students10.1007/s10639-025-13542-4 ↗
30. Leveraging ChatGPT for personalized reflective learning in programming education: Effects on self-efficacy, higher-order thinking, and project implementation skills10.1007/s10639-025-13733-z ↗
31. Assessment and learning outcomes for generative AI in higher education: A scoping review on current research status and trends
32. Implementing generative AI (GenAI) in higher education: A systematic review of case studies10.1016/j.caeai.2025.100407 ↗
33. Artificial intelligence in computer programming education: A systematic literature review10.1016/j.caeai.2025.100403 ↗
34. Code suggestions and explanations in programming learning: Use of ChatGPT and performance
35. Does generative AI help in learning programming: Students’ perceptions, reported use and relation to performance10.1016/j.chbr.2025.100642 ↗
36. The influence of ChatGPT on student engagement: A systematic review and future research agenda10.1016/j.compedu.2024.105100 ↗
37. Does ChatGPT enhance student learning? A systematic review and meta-analysis of experimental studies10.1016/j.compedu.2024.105224 ↗
38. Barari, G., and Sanders, B. (2024, January 18–21). The effectiveness of virtual environments for increasing engagement in engineering technology courses. Proceedings of the 2024 Conference for Industry and Education Collaboration, Garden Grove, CA, USA.
39. Barari, G., and Sanders, B. (2025, January 22–25). Application of FLASH to Forecast Student Engagement in Online Engineering Courses. Proceedings of the 2025 ASEE Annual Conference & Exposition, Montreal, QC, Canada.
40. Ward, T., Jenab, K., Ortega-Moody, J., Barari, G., and Molina Acosta, L.D.C. (2025). Virtual classrooms, real impact: A framework for introducing virtual reality to K–12 STEM learning based on best practices. Appl. Sci., 15.10.3390/app152111356 ↗
41. Siebold, K., and Barari, G. (2026, January 4). Enhancing Undergraduate Engineering Education through HLA-Based Simulation Exploration Experience (SEE). Proceedings of the 2026 Computer-Utilized Education Society (CIEC), New Orleans, LA, USA.
42. Integrating T-shaped competency models in interdisciplinary aviation and engineering education
43. Barari, G., and Siebold, S. (2026, January 4). Curriculum Alignment through the Industry Pulse Platform. Proceedings of the 2026 Computer-Utilized Education Society (CIEC), New Orleans, LA, USA.
44. Janke, C., Lin, Y., and Barari, G. (2025). BOARD# 15: From drones to airplanes, Lessons learned from uncrewed aviation for teaching innovative and sustainable electric aviation propulsion. Proceedings of the 2025 ASEE Annual Conference & Exposition, Montreal, QC, Canada, 22–25 June 2025, American Society for Engineering Education.10.18260/1-2--55521 ↗
45. Generative AI in higher education: A global perspective of institutional adoption policies and guidelines10.1016/j.caeai.2024.100348 ↗
46. Barari, N., and Barari, G. (2025). Continual learning models in aviation systems. Coll. Aviat. Rev. Int., 43, Available online: https://ojs.library.okstate.edu/osu/index.php/CARI/article/view/10310.
47. Barari, N., and Kim, E. (2021). Linking sparse coding dictionaries for representation learning. 2021 International Conference on Rebooting Computing (ICRC), IEEE.10.1109/ICRC53822.2021.00022 ↗
48. Barari, N., Lian, X., and MacLellan, C.J. (2024). Incremental concept formation over visual images without catastrophic forgetting. arXiv.
49. Barari, N., Kim, E., and MacLellan, C. (2025). Explaining Robustness to Catastrophic Forgetting Through Incremental Concept Formation. arXiv.
50. Barari, G., Basu, D., Lin, Y., and Ortega-Moody, J. (2026). Evaluating Large Language Models in Engineering Education: A Multi-Course Assessment Across Bloom’s Taxonomy, IEEE Access.10.1109/ACCESS.2026.3686339 ↗
51. Shah, A., Chernova, A., Tomson, E., Porter, L., Griswold, W.G., and Soosai Raj, A.G. (March, January 26). Students’ use of GitHub Copilot for working with large code bases. Proceedings of the 56th ACM Technical Symposium on Computer Science Education V. 1 (SIGCSE TS ’25), Pittsburgh, PA, USA.