A Study on the Consistency and Efficiency of Student Performance Evaluation Methods: A Mathematical Framework and Comparative Simulation Results
Source abstract
Abstract
Background: Consistent evaluation methods foster fairness, reduce bias, and enhance student understanding and motivation. Notably, mathematical inconsistencies, such as improper weighting, flawed averaging, and unsound scaling, can undermine the accuracy and reliability of assigned grades. This paper addresses the critical need for consistent student evaluation methods, with a primary focus on ensuring mathematical consistency within grading systems. Methods: We propose a scheme aimed at identifying inconsistencies in student evaluation related to the mathematical framework of the used grading method. To explain the functioning of our construct, we provide mathematical representation of conventional grading methods, including summative assessments, rubrics, and the Systematic Task-Based Assessment Method (STBAM) that we have recently developed, which incorporates both traditional and fuzzy logic-based grading modules. We introduce a Consistency Index (CIM) depending on the Mean Absolute Deviation of assigned scores (MADM) and a method’s Complexity Pointer (CPM). We also propose a method’s Efficiency Index (βM) expressed in terms of the Consistency Index and the latter indicator. We compared the mathematical consistency and efficiency of the methods addressed in this study through simulation runs. Results: We demonstrated how the proposed indices can reveal the strengths and weaknesses of each grading scheme analysed. Conclusions: The fuzzy logic-based modulus of the STBAM yielded the highest values of CIM and βM. However, performing a pending analysis of scalability, teacher training, and cultural adaptability would be essential to strengthen the potential of the STBAM to be adopted as a reliable grading alternative to conventional grading approaches. In the meantime, our approach could provide a clear, logical, and defensible framework for testing the mathematical consistency of student assessment methods.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Student Assessment and Feedback · Evaluation and Performance Assessment · Higher Education Learning Practices
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Centro de Investigación Científica y de Estudios Superiores de Ensenada, Carretera Ensenada-Tijuana No. 3918, Zona Playitas, Ensenada 22860, Baja California, Mexico
Centro de Investigación en Matemáticas, A.C. Jalisco s/n, Mineral Valenciana, Guanajuato 36240, Guanajuato, Mexico
Instituto Mexicano de Investigación en Pesca y Acuacultura Sustentables (IMIPAS), Km 97.5 Carretera Tijuana-Ensenada, s/n, El Sauzal de Rodríguez, Ensenada 22760, Baja California, Mexico
Funding
Center for Scientific Research and Higher Education at Ensenada
References (61)
Show source-provided references
1. CSAI (2025, March 24). Valid and Reliable Assessments. Update, CSAI, The Center on Standards and Assessment Implementation, Available online: https://files.eric.ed.gov/fulltext/ED588476.pdf.
2. Palmer, N. (2022, January 7–9). Tools to Improve Efficiency and Consistency in Assessment Practices Whilst Delivering Meaningful Feedback. Proceedings of the ICERI2022 Proceedings, Seville, Spain.10.21125/iceri.2022.0296 ↗
3. Fair and equitable assessment practices for all students
4. Student anxiety and evaluation10.22329/celt.v12i0.5409 ↗
5. Impact of evaluation method shifts on student performance: An analysis of irregular improvement in passing percentages during COVID-19 at an Ecuadorian institution10.1007/s40979-024-00179-y ↗
6. Glaser, R., Chudowsky, N., and Pellegrino, J.W. (2001). Knowing What Students Know: The Science and Design of Educational Assessment, National Academies Press.
7. Improving Assessment and Feedback in the Learning Process: Directions and Best Practices10.56177/red.8.2023.art.3 ↗
8. Formative and summative evaluation techniques for improvement of learning process
9. Addressing inconsistencies in grading practices10.1177/00317217241251883 ↗
10. Cambridge International Examinations (2016). Developing Your School with Cambridge: A Guide for School Leaders, Director Education.
11. The Value and Effectiveness of Feedback in Improving Students’ Learning and Professionalizing Teaching in Higher Education
12. Disparate methods of combining test and assignment scores into course grades
13. Bias in grading: A meta-analysis of experimental research findings10.1177/0004944116664618 ↗
14. Newman, D., and Lazarev, V. (2015). How Teacher Evaluation is affected by Class Characteristics: Are Observations Biased. Empower. Educ. Evid.-Based Decis., 1–11.10.2139/ssrn.2574897 ↗
15. A Critique of Grading: Policies, Practices, and Technical Matters10.14507/epaa.26.3814 ↗
16. Do test score gaps grow before, during, or between the school years? Measurement artifacts and what we can know in spite of them10.15195/v6.a3 ↗
17. Lauck, L.V. (2019). Grade Distribution and Perceptions of School Culture and Climate In a New Secondary School. [Ph.D. Thesis, Rockhurst University].
18. The Perils of Prescribed Grade Distributions: What Every Medical Educator Should Know
19. Let’s Talk About Grading, Maybe: Using Transparency About the Grading Process To Aid in Student Learning
20. Dubois, P., and Lhotte, R. (2023). Consistency and Reproducibility of Grades in Higher Education: A Case Study in Deep Learning. arXiv, Available online: https://arxiv.org/abs/2305.07492.
21. Using student performance data to identify effective classroom practices10.1257/aer.100.2.256 ↗
22. Standards-based grading and reporting will improve education10.1177/0031721715579043 ↗
23. The benefits of standards-based grading: A critical evaluation of modern grading practices
24. Leal-Ramírez, C., and Echavarría-Heras, H.A. (2024). An Integrated Instruction and a Dynamic Fuzzy Inference System for Evaluating the Acquirement of Skills through Learning Activities by Higher Middle Education Students in Mexico. Mathematics, 12.10.3390/math12071015 ↗
25. Gronlund, N.E. (1998). Assessment of Student Achievement, Allyn & Bacon Publishing, Longwood Division. [8th ed.].
26. Yambi, T.D.A.C., and Yambi, C. (2018). Assessment and Evaluation in Education, University Federal do Rio de Janeiro.
27. Theoretical Framework for Educational Assessment: A Synoptic Review
28. Students’ perceptions about evaluation and assessment in higher education: A review10.1080/02602930500099102 ↗
29. Does mathematical precision ensure valid grades? What every veterinary medical educator should know10.3138/jvme.0115-005R1 ↗
30. Sikora, A.S. (2025, March 24). Mathematical Theory of Student Assessment Through Grading. Available online: https://citeseerx.ist.psu.edu/document?repid=rep1&type=pdf&doi=085fd7628dc0d7e7c0e22dc458c81d4e50c44dee.
31. The reliability of setting grade boundaries using comparative judgement10.1080/02671522.2015.1027723 ↗
32. Hanania, M.I. (1947). Mathematical Methods Applicable to the Standardization of Examinations. [Master’s Thesis, American University of Beirut].
33. Fitzpatrick, J.L., Sanders, J.R., Worthen, B.R., and Wingate, L.A. (2012). Program Evaluation: Alternative Approaches and Practical Guidelines, Pearson.
34. The role of assessment in a learning culture10.3102/0013189X029007004 ↗
35. Stufflebeam, D.L., and Coryn, C.L. (2014). Evaluation Theory, Models, and Applications, John Wiley & Sons.
36. Assessing Assessment Theories
37. Astin, A.W. (2012). Assessment for Excellence: The Philosophy and Practice of Assessment and Evaluation in Higher Education, Rowman & Littlefield Publishers.
38. Chen, H.T. (2012). Theory-Driven Evaluation: Conceptual Framework, Application and Advancement, Springer Fachmedien Wiesbaden.10.1007/978-3-531-19009-9_2 ↗
39. Gipps, C. (2011). Beyond Testing (Classic Edition): Towards a Theory of Educational Assessment, Routledge.10.4324/9780203182437 ↗
40. Wiliam, D. (2000). Integrating formative and summative functions of assessment. Working Group, King’s College London.
41. Assessment and evaluation methods for 21st century education: Measuring what matters10.57125/FED.2024.12.25.01 ↗
42. Galamison, T.J. (2014). Benchmarking: A Study of the Perceptions Surrounding Accountability, Instructional Leadership, School Culture, Formative Assessments and Student Success. [Ph.D. Thesis, University of Houston].
43. Goldberger, S., Keough, R., and Almeida, C. (2000). Benchmarks for Success in High School Education. Putting Data to Work in School to Career Education Reform, LAB at Brown University.
44. Dunn, D.S., McCarthy, M.A., Baker, S.C., and Halonen, J.S. (2010). Using Quality Benchmarks for Assessing and Developing Undergraduate Programs, John Wiley & Sons.
45. Summative Assessment in Higher Education: Practices in Disarray10.1080/03075070220000662 ↗
46. Summative Assessment: Dealing with the ‘Measurement Fallacy’10.1080/03075070903545082 ↗
47. Harlen, W., Crick, R.D., Broadfoot, P., Daugherty, R., Gardner, J., James, M., and Stobart, G. (2002). A Systematic Review of the Impact of Summative Assessment and Tests on Students’ Motivation for Learning, University of Bristol, Evidence-Based Practice Unit.
48. Exploring Summative Assessment and Effects: Primary to Higher Education
49. Bijsterbosch, H. (2018). Professional Development of Geography Teachers with Regard to Summative Assessment Practices. [Ph.D. Thesis, Utrecht University].
50. Ekwue, U.N. (2015). A Hybrid Exploration of the Impact of Summative Assessment on A-level Students’ Motivation and Depth of Learning and the Extent to Which This Is a Reflection of the Self. [Ph.D. Thesis, King’s College London].
51. Stevens, D.D. (2023). Introduction to Rubrics: An Assessment Tool to Save Grading Time, Convey Effective Feedback, and Promote Student Learning, Routledge.
52. The sensitivity of teacher performance ratings to the design of teacher evaluation systems10.3102/0013189X17726752 ↗
53. The effect of rubric rating scale on the evaluation of engineering design projects
54. Common structural design features of rubrics may represent a threat to validity10.3102/0013189X14542154 ↗
55. Indeterminacy in the use of preset criteria for assessment and grading10.1080/02602930801956059 ↗
56. The quality and effectiveness of descriptive rubrics10.1080/00131911.2014.929565 ↗
57. Fuzzy sets10.1016/S0019-9958(65)90241-X ↗
58. Mousse, M.A., Almufti, S.M., García, D.S., Jebbor, I., Aljarbouh, A., and Tsarev, R. (2023). Application of fuzzy logic for evaluating student learning outcomes in E-learning. Proceedings of the Computational Methods in Systems and Software, Springer.10.1007/978-3-031-54820-8_15 ↗
59. Lógica difusa: Una nueva epistemología para las Ciencias de la Educación
60. Sripan, R., and Suksawat, B. (2010, January 27–30). Propose of fuzzy logic-based students’ learning assessment. Proceedings of the ICCAS 2010, Gyeonggi-do, Republic of Korea.10.1109/ICCAS.2010.5669786 ↗
61. Hegazi, M.O., Almaslukh, B., and Siddig, K. (2023). A fuzzy model for reasoning and predicting student’s academic performance. Appl. Sci., 13.10.3390/app13085140 ↗