Assessing the Selection of Digital Learning Materials: A Facet of Pre-Service Teachers’ Digital Competence
Source abstract
Abstract
Given the increasing digitalization of education and the variety of available digital learning materials (dLMs) of differing quality, (pre-service) teachers must develop the ability to select appropriate dLMs. Objective, reliable, and valid assessment instruments are necessary to evaluate the effectiveness of that development. This study conceptualized and designed an economical four-item instrument for assessing “selecting dLMs” based on accepted frameworks and competence models. The scientific quality of the instrument was evaluated in Study 1 (n = 164) with four dLMs and empirically investigated in a subsequent Study 2 (n = 395) with pre-service mathematics teachers from two universities. The empirical results indicate that the instrument could objectively and reliably gauge different levels of “selecting dLMs”. Furthermore, the results are consistent with the widely accepted notion that the competence of “selecting dLMs” depends on (content) knowledge; however, that relation was not strong. In addition, the results for objectively assessing “selecting dLMs” paralleled the results of self-assessed TPACK in terms of the academic progression of participants. The proposed approach allows for variations and integration of diverse dLMs, and it has the potential to be adapted in other subject areas and contexts.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Digital literacy in education · Technology-Enhanced Education Studies · Educational Technology and E-Learning
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Institute for Didactics of Mathematics, University of Cologne, 50931 Cologne, Germany
Department of Mathematics Education, University College of Education Upper Austria, 4020 Linz, Austria
Department of Psychology, Faculty of Human Sciences, University of Cologne, 50931 Cologne, Germany
Funding
DEAL
References (87)
Show source-provided references
1. The Role of Digital Technologies in Mathematics Education: Purposes and Perspectives10.1007/s11858-023-01535-x ↗
2. Roblyer, M.D., and Hughes, J.E. (2019). Integrating Educational Technology into Teaching: Transforming Learning Across Disciplines, Pearson Education, Inc.. [8th ed.].
3. Integrating Technologies Into Teaching and Learning Mathematics at the Beginning of Secondary Education in Austria10.29333/ejmste/11428 ↗
4. Transformation of the Mathematics Classroom with the Internet10.1007/s11858-020-01176-4 ↗
5. (2018). OECD The Future of Education and Skills Education 2030, OECD.
6. Mathematics Teaching, Learning, and Assessment in the Digital Age10.1007/s11858-024-01612-9 ↗
7. Digital Resources as an Aspect of Teacher Professional Digital Competence: One Term, Different Definitions—A Systematic Review10.1007/s10639-022-11321-z ↗
8. Teaching with Digital Technology10.1007/s11858-020-01196-0 ↗
9. Gonscherowski, P., and Rott, B. (2022). How Do Pre-/In-Service Mathematics Teachers Reason for or against the Use of Digital Technology in Teaching?. Mathematics, 10.10.3390/math10132345 ↗
10. Enhancing Mathematics Education through Collaborative Digital Material Design: Lessons from a National Project10.30935/scimath/14323 ↗
11. Selecting Digital Technology: A Review of TPACK Instruments
12. Teacher Education Effectiveness as an Emerging Research Paradigm: A Synthesis of Reviews of Empirical Studies Published over Three Decades (1993–2023)
13. Achieving Excellence and Equality in Mathematics: Two Degrees of Freedom?10.1080/00220272.2022.2098536 ↗
14. Characterising the Perceived Value of Mathematics Educational Apps in Preservice Teachers10.1007/s13394-015-0160-0 ↗
15. Fresh Perspectives on TPACK: Pre-Service Teachers’ Own Appraisal of Their Challenging and Confident TPACK Areas10.1007/s10639-019-10092-4 ↗
16. Technological Pedagogical Content Knowledge: A Framework for Teacher Knowledge10.1111/j.1467-9620.2006.00684.x ↗
17. Ronau, R.N., Rakes, C.R., and Niess, M.L. (2011). How Do We Measure TPACK? Let Me Count the Ways. A Research Handbook on Frameworks and Approaches, IGI Global.
18. Running in Circles: A Systematic Review of Reviews on Technological Pedagogical Content Knowledge (TPACK)10.1016/j.compedu.2024.105024 ↗
19. A Study on the Technological Pedagogical Contextual Knowledge of Science Teacher Candidates across Different Years of Study10.1007/s10639-018-9845-9 ↗
20. Beliefs, Self-Reported or Performance-Assessed TPACK: What Can Predict the Quality of Technology-Enhanced Biology Lesson Plans?10.1007/s10956-022-09974-z ↗
21. Unraveling TPACK: Investigating the Inherent Structure of TPACK from a Subject-Specific Angle Using Test-Based Instruments10.1016/j.compedu.2024.105040 ↗
22. Fostering Pre-Service Teachers’ Technological Pedagogical Content Knowledge (TPACK): A Quasi-Experimental Field Study10.1016/j.compedu.2021.104304 ↗
23. Examining Preservice Teachers’ TPACK, Attitudes, Self-Efficacy, and Perceptions of Teamwork in a Stand-Alone Educational Technology Course Using Flipped Classroom or Flipped Team-Based Learning Pedagogies10.1080/21532974.2020.1752335 ↗
24. A Longitudinal Examination of Preservice Teachers’ Technological Pedagogical Content Knowledge in the Context of Undergraduate Teacher Education10.1080/01626620.2016.1248301 ↗
25. Technology Use: Analysis of Lesson Plans on Fractions in an Online Laboratory School
26. Theoretically Framing the Pedagogy of Learning to Teach Mathematics with Technology
27. Fostering EFL Teachers’ CALL Competencies through Project-Based Learning
28. Revuelta-Domínguez, F.-I., Guerra-Antequera, J., González-Pérez, A., Pedrera-Rodríguez, M.-I., and González-Fernández, A. (2022). Digital Teaching Competence: A Systematic Review. Sustainability, 14.10.3390/su14116428 ↗
29. Developing and Validating Technological Pedagogical Content Knowledge—Practical TPACK through the Delphi Survey Technique10.1111/bjet.12078 ↗
30. Sheth, J., and Malhotra, N. (2010). Social Desirability Bias. Wiley International Encyclopedia of Marketing, Wiley.10.1002/9781444316568 ↗
31. (2020). Safrudiannur Measuring Teachers’ Beliefs Quantitatively: Criticizing the Use of Likert Scale and Offering a New Approach, Springer Fachmedien Wiesbaden.
32. Self-Reported Technological Pedagogical Content Knowledge (TPACK) of Pre-Service Teachers in Relation to Digital Technology Use in Lesson Plans10.1016/j.chb.2020.106586 ↗
33. Competence Measurement in (Mathematics) Teacher Education and Beyond: Implications for Policy10.1057/s41307-019-00139-z ↗
34. Beyond Dichotomies: Competence Viewed as a Continuum
35. Powerful Knowledge, Educational Potential and Knowledge-Rich Curriculum: Pushing the Boundaries10.1080/00220272.2022.2089538 ↗
36. Yang, X., Deng, J., Sun, X., and Kaiser, G. (2024). The Relationship between Opportunities to Learn in Teacher Education and Chinese Preservice Teachers’ Professional Competence. J. Curric. Stud., 1–19.10.1080/00220272.2024.2334944 ↗
37. Schleicher, A. (2023). PISA 2022 Insights and Interpretations, OECD.
38. Hattie, J. (2023). Visible Learning, the Sequel: A Synthesis of over 2,100 Meta-Analyses Relating to Achievement, Routledge. [1st ed.].10.4324/9781003380542 ↗
39. What Is Technological Pedagogical Content Knowledge (TPACK)?10.1177/002205741319300303 ↗
40. Those Who Understand: Knowledge Growth in Teaching10.3102/0013189X015002004 ↗
41. Learning Mechanisms Explaining Learning with Digital Tools in Educational Settings: A Cognitive Process Framework10.1007/s10648-024-09845-6 ↗
42. Contextualizing TPACK within Systems and Cultures of Practice10.1016/j.chb.2020.106673 ↗
43. Ben-David Kolikant, Y., Martinovic, D., and Milner-Bolotin, M. (2020). Teaching Mathematics in the Digital Era: Standards and Beyond. STEM Teachers and Teaching in the Digital Era, Springer International Publishing.10.1007/978-3-030-29396-3 ↗
44. Redecker, C., and Punie, Y. (2017). Digital Competence of Educators, Publications Office of the European Union.
45. Grossman, P.L. (2018). Teaching Core Practices in Teacher Education, Harvard Education Press.
46. TPACK in Special Education: Preservice Teacher Decision Making While Integrating iPads into Instruction
47. Investigating Prospective Teachers’ TPACK and Their Use of Mathematical Action Technologies as They Create Screencast Video Lessons on iPads
48. The Potential of Digital Tools to Enhance Mathematics and Science Learning in Secondary Schools: A Context-Specific Meta-Analysis10.1016/j.compedu.2020.103897 ↗
49. Digital Artefacts as Representations: Forging Connections between a Constructionist and a Social Semiotic Perspective10.1007/s10649-013-9523-1 ↗
50. From Static to Dynamic Mathematics: Historical and Representational Perspectives10.1007/s10649-008-9116-6 ↗
51. Cognitive Architecture and Instructional Design: 20 Years Later10.1007/s10648-019-09465-5 ↗
52. Developing Technological Pedagogical Content Knowledge (TPACK) in Animal Physiology10.1088/1742-6596/895/1/012052 ↗
53. How Can GeoGebra Support Physics Education in Upper-Secondary School—A Review10.1088/1361-6552/ac03fb ↗
54. The Use of Immersive Technologies in Distance Education: A Systematic Review10.1007/s10639-023-11849-8 ↗
55. Rüth, M., Breuer, J., Zimmermann, D., and Kaspar, K. (2021). The Effects of Different Feedback Types on Learning with Mobile Quiz Apps. Front. Psychol., 12.10.3389/fpsyg.2021.665144 ↗
56. Cho, S.J. (2015). Digital Technology in Mathematics Education: Why It Works (Or Doesn’t). Selected Regular Lectures from the 12th International Congress on Mathematical Education, Springer International Publishing.
57. Opportunities to Learn, Technological Pedagogical Knowledge, and Personal Factors of Pre-Service Teachers: Understanding the Link between Teacher Education Program Characteristics and Student Teacher Learning Outcomes in Times of Digitalization10.1007/s11618-023-01162-y ↗
58. Kaiser, G. (2016). Uses of Technology in Lower Secondary Mathematics Education: A Concise Topical Survey, Springer International Publishing. ICME-13 Topical Surveys.
59. Metacognitive Scaffolding in an Innovative Learning Arrangement10.1007/s11251-010-9154-1 ↗
60. Inductive and Deductive Justification of Knowledge: Epistemological Beliefs and Critical Thinking at the Beginning of Studying Mathematics10.1007/s10649-020-10004-1 ↗
61. Analysis of Teachers’ Pedagogical Digital Competence: Identification of Factors Predicting Their Acquisition10.1007/s10758-019-09432-7 ↗
62. Gonscherowski, P., and Rott, B. (2021, January 19–22). Measuring Digital Competencies of Pre-Service Teachers-a Pilot Study. Proceedings of the 44th Conference of the International Group for the Psychology of Mathematics Education, Khon Kaen, Thailand.
63. Instrument to Assess the Knowledge and the Skills of Mathematics Educators’ Regarding Digital Technology
64. (2023, May 23). GeoGebra Team Classroom Resources. Available online: https://www.geogebra.org/materials.
65. (2021, July 10). Flink Maxi und der Baum—GeoGebra. Available online: https://www.geogebra.org/m/a4pppe7a.
66. Schüngel, M. (2022, February 28). Bewege Den Schmetterling—GeoGebra. Available online: https://www.geogebra.org/m/zrj2zcam.
67. (2023, February 28). FLINK Lieblingssport—GeoGebra. Available online: https://www.geogebra.org/m/v4xuvmhf.
68. (2023, February 28). FLINK Welche Zahl Fehlt?—GeoGebra. Available online: https://www.geogebra.org/m/qqv3kxt6.
69. Quantitative Methods in Psychology: A Power Primer10.1037/0033-2909.112.1.155 ↗
70. Guidelines, Criteria, and Rules of Thumb for Evaluating Normed and Standardized Assessment Instruments in Psychology10.1037/1040-3590.6.4.284 ↗
71. The Measurement of Observer Agreement for Categorical Data10.2307/2529310 ↗
72. The Use of Cronbach’s Alpha When Developing and Reporting Research Instruments in Science Education10.1007/s11165-016-9602-2 ↗
73. Technological and Pedagogical Support for Pre-Service Teachers’ Lesson Planning10.1080/1475939X.2019.1569554 ↗
74. ICT Integration in Teachers’ Lesson Plans: A Scoping Review of Empirical Studies
75. Cohen, R.J. (2025). Uncovering Relations between Self-Reported TPACK and Objective Measures: Accounting for Experience. Proceedings of Society for Information Technology & Teacher Education International Conference, Association for the Advancement of Computing in Education (AACE).
76. Teaching Mathematics with Technology: A Multidimensional Analysis of Teacher Beliefs10.1007/s10649-021-10072-x ↗
77. The Spiral Model of Collaborative Knowledge Improvement: An Exploratory Study of a Networked Collaborative Classroom10.1007/s11412-021-09338-6 ↗
78. TPACK Development of Prospective Physics Teachers to Ease the Achievement of Learning Objectives: A Case Study at the State University of Malang, Indonesia10.1088/1742-6596/1185/1/012042 ↗
79. Methods for the Detection of Carelessly Invalid Responses in Survey Data10.1016/j.jesp.2015.07.006 ↗
80. Behavioural Intention and Pre-Service Mathematics Teachers’ Technological Pedagogical Content Knowledge
81. Beyond Knowledge: Measuring Primary Teachers’ Subject-Specific Competences in and for Teaching Mathematics with Items Based on Video Vignettes10.1007/s10763-014-9608-z ↗
82. Measuring Pre-Service Teachers’ Decision-Making in Classroom Management: A Video-Based Assessment Approach10.1016/j.tate.2023.104426 ↗
83. Baumanns, L., and Pohl, M. (2024, January 3–6). Leveraging ChatGPT for Problem Posing: An Exploratory Study of Pre-Service Teachers’ Professional Use of AI. Proceedings of the Mathematics Education in the Digital Age 4 (MEDA4), Bari, Italy.
84. On Understanding Mathematical Problem-Posing Processes10.1007/s11858-023-01536-w ↗
85. Trixa, J., and Kaspar, K. (2024). Information Literacy in the Digital Age: Information Sources, Evaluation Strategies, and Perceived Teaching Competences of Pre-Service Teachers. Front. Psychol., 15.10.3389/fpsyg.2024.1336436 ↗
86. Seeking Traces of Filters and Amplifiers as Pre-Service Teachers Perform Their Pedagogical Content Knowledge10.33828/sei.v34.i1.8 ↗
87. Brungs, C.L., Buchholtz, N., Streit, H., Theile, Y., and Rott, B. (2025). Empirical Reconstruction of Mathematics Teaching Practices in Problem-Solving Lessons: A Multi-Method Case Study. Front. Educ., 10.10.3389/feduc.2025.1555763 ↗