AI Competency Assessment and Ranking: A Framework for Higher Education
Source abstract
Abstract
Generative artificial intelligence is reshaping students’ learning strategies, yet most definitions of “AI competency” rely on self-reports and prescriptive checklists. We analyzed data from N = 686 university students in Spain to uncover behavioral patterns of AI use and translate them into practical guidance for teaching and policy. Using unsupervised clustering (k-means) complemented with a topological summary (Mapper), we identified four coherent profiles along a continuum of engagement and self-regulation: Low-Engagement, Active–Cautious, Balanced–Confident, and High-Use–Vigilant. The profiles differ in how often students use AI, how they revise outputs, their reliance on AI, and their ethical awareness, and they show distinct emotional patterns (e.g., curiosity, motivation, stress). A continuous structure links profiles rather than separating them rigidly. Self-rated digital competence and studying in STEM fields were associated with higher-level profiles. Overall, the results support a layered, data-informed view of AI competency that prioritizes observed practices over single summary scores. We introduce AI CAR as a formative framework to help institutions (i) locate students on this continuum and (ii) design targeted supports that strengthen revision habits, ethical reflection, and self-regulation across the curriculum.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Ethics and Social Impacts of AI · Artificial Intelligence in Healthcare and Education · Explainable Artificial Intelligence (XAI)
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Departamento de Matemática Aplicada, Universitat Politècnica de València, Camino de Vera s/n, 46022 Valencia, Spain
Departamento de Lingüística Aplicada, Universitat Politècnica de València, Camino de Vera s/n, 46022 Valencia, Spain
Instituto de Diseño y Fabricación, Universitat Politècnica de València, Camino de Vera s/n, 46022 Valencia, Spain
References (35)
Show source-provided references
1. AI literacy for users—A comprehensive review and future research directions of learning methods, components, and effects10.1016/j.chbah.2024.100062 ↗
2. ED-AI lit: An interdisciplinary framework for AI literacy in education10.1177/23727322231220339 ↗
3. Digital interaction literacy model: Conceptualizing competencies for literate interactions with voice-based AI systems10.1016/j.caeai.2022.100114 ↗
4. UNESCO (2025, November 15). Beijing Consensus on Artificial Intelligence and Education. Available online: https://unesdoc.unesco.org/ark:/48223/pf0000368303.
5. UNESCO (2025, November 15). AI Competency Framework for Teachers. Available online: https://unesdoc.unesco.org/ark:/48223/pf0000391104.
6. UNESCO (2025, November 15). AI Competency Framework for Students. Available online: https://unesdoc.unesco.org/ark:/48223/pf0000391105.
7. UNESCO (2025, November 15). AI and Education: Guidance for Policy-Makers. Available online: https://unesdoc.unesco.org/ark:/48223/pf0000376709.
8. European Commission (2018). The Digital Competency Framework for Citizens (DigComp), Joint Research Centre. Available online: https://joint-research-centre.ec.europa.eu/projects-and-activities/education-and-training/digital-transformation-education/digital-competence-framework-citizens-digcomp_en.
9. OECD (2019). Future of Education and Skills 2030, Organisation for Economic Co-Operation and Development. Available online: https://www.oecd.org/en/about/projects/future-of-education-and-skills-2030.html.
10. A comprehensive exploration of artificial intelligence competency frameworks for educators: A critical review10.1111/ejed.12663 ↗
11. Conceptualizing AI Literacy: An Exploratory Review10.1016/j.caeai.2021.100041 ↗
12. Topology and Data10.1090/S0273-0979-09-01249-X ↗
13. Singh, G., Mémoli, F., and Carlsson, G.E. (2007, January 2–3). Topological Methods for the Analysis of High Dimensional Data Sets and 3D Object Recognition. Proceedings of the Eurographics Symposium on Point-Based Graphics, Prague, Czech Republic.
14. Least squares quantization in PCM10.1109/TIT.1982.1056489 ↗
15. MAILS—Meta AI literacy scale: Development and testing of an AI literacy questionnaire based on well-founded competency models and psychological change- and meta-competencies10.1016/j.chbah.2023.100014 ↗
16. Delphi study for the development and preliminary validation of an item set for the assessment of non-experts’ AI literacy10.1016/j.caeai.2023.100126 ↗
17. Artificial intelligence in the L2 classroom: Implications and challenges on ethics and equity in higher education: A 21st century Pandora’s box10.1016/j.caeai.2023.100179 ↗
18. Ethical principles for artificial intelligence in K-12 education10.1016/j.caeai.2023.100131 ↗
19. Measuring user competency in using artificial intelligence: Validity and reliability of artificial intelligence literacy scale10.1080/0144929X.2022.2072768 ↗
20. Pinski, M., and Benlian, A. (2023, January 3–6). AI literacy: Towards measuring human competency in artificial intelligence. Proceedings of the 56th Hawaii International Conference on System Sciences, Maui, HI, USA.10.24251/HICSS.2023.021 ↗
21. Explicating AI literacy of employees at digital workplaces10.1109/TEM.2021.3138503 ↗
22. Long, D., and Magerko, B. (2020, January 25–30). What is AI Literacy? Competencies and Design Considerations. Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (CHI ’20), Honolulu, HI, USA.10.1145/3313831.3376727 ↗
23. Navigating the landscape of AI literacy education: Insights from a decade of research (2014–2024)10.1057/s41599-025-04583-8 ↗
24. Evaluating an artificial intelligence literacy programme for secondary students: Conceptual learning, literacy and ethical awareness10.1007/s10639-022-11408-7 ↗
25. Evaluating an artificial intelligence literacy programme for developing university students’ conceptual understanding, literacy, empowerment and ethical awareness
26. Evaluation of an artificial intelligence literacy course for university students with diverse study backgrounds10.1016/j.caeai.2021.100026 ↗
27. Exploring the determinants of artificial intelligence (AI) literacy: Digital divide, computational thinking, cognitive absorption10.1016/j.tele.2023.102026 ↗
28. The challenges and opportunities of AI-assisted writing: Developing AI literacy for the AI age
29. What ChatGPT means for universities: Perceptions of scholars and students
30. Combining human and artificial intelligence for enhanced AI literacy in higher education10.1016/j.caeo.2024.100184 ↗
31. The AI generation gap: Are Gen Z students more interested in adopting generative AI such as ChatGPT in teaching and learning than their teachers?10.1186/s40561-023-00269-3 ↗
32. Navigating the AI-enhanced STEM education landscape: A decade of insights, trends, and opportunities10.1080/02635143.2024.2370764 ↗
33. Developing a model for AI across the curriculum: Transforming the higher education landscape via innovation in AI literacy10.1016/j.caeai.2023.100127 ↗
34. AI literacy in K-12: A systematic literature review10.1186/s40594-023-00418-7 ↗
35. Artificial intelligence (AI) literacy education in secondary schools: A review10.1080/10494820.2023.2255228 ↗