A New Measure for Serious Games Evaluation: Gaming Educational Balanced (GEB) Model
Source abstract
Abstract
Serious games have to meet certain characteristics relating to gameplay and educational content to be effective as educational tools. There are some models that evaluate these aspects, but they usually lack a good balance between both ludic and learning requirements, and provide no guide for the design of new games. This study develops the Gaming Educational Balanced (GEB) Model which addresses these two limitations. GEB is based on the Mechanics, Dynamics and Aesthetics framework and the Four Pillars of Educational Games theory. This model defines a metric to evaluate serious games, which can also be followed to guide their subsequent development. This rubric is tested with three indie serious games developed using different genres to raise awareness of mental illnesses. This evaluation revealed two main issues: the three games returned good results for gameplay, but the application of educational content was deficient, due in all likelihood to the lack of expert educators participating in their development. A statistical and machine learning validation of the results is also performed to ensure that the GEB metric features are clearly explained and the players are able to evaluate them correctly. These results underline the usefulness of the new metric tool for identifying game design strengths and weaknesses. Future works will apply this metric to more serious games to further test its effectiveness and to guide the design of new serious games.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Educational Games and Gamification · Flow Experience in Various Fields · Digital Games and Media
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Department of History, Geography and Communication, Universidad de Burgos, 09001 Burgos, Spain
Department of Computer Engineering, Universidad de Burgos, 09006 Burgos, Spain
Funding
Consejería de Empleo of the Junta de Castilla y León (Spain) · Ministry of Science, Innovation and Universities
References (100+)
Show source-provided references
1. A review of immersive virtual reality serious games to enhance learning and training10.1007/s11042-019-08348-9 ↗
2. Motivation, engagement, and performance across multiple virtual reality sessions and levels of immersion10.1111/jcal.12520 ↗
3. Success factors for serious games to enhance learning: A systematic review10.1007/s10055-016-0298-4 ↗
4. Evaluation of technology use in education: Findings from a critical analysis of systematic literature reviews10.1111/jcal.12412 ↗
5. Design requirements for educational serious games with focus on player enjoyment10.1016/j.entcom.2021.100413 ↗
6. Mitgutsch, K., and Alvarado, N. (June, January 29). Purposeful by design? A Serious Game Design Assessment Framework. Proceedings of the International Conference on the Foundations of Digital Games—FDG’12, Raleigh, NC, USA.
7. Towards a Comprehensive Methodology for the Research and Evaluation of Serious Games10.1016/j.procs.2012.10.075 ↗
8. Classifying games in container terminal logistics field: A systematic review10.1016/j.entcom.2021.100465 ↗
9. Hunicke, R., Leblanc, M., and Zubek, R. (2004, January 25–29). MDA: A formal approach to game design and game research. Proceedings of the AAAI Workshop on Challenges in Game AI, San José, CA, USA.
10. Becker, K. (2017). Choosing and Using Digital Games in the Classroom: A Practical Guide, Springer International Publishing. Advances in Game-Based Learning.10.1007/978-3-319-12223-6_9 ↗
11. Awareness, Prevention, Detection, and Therapy Applications for Depression and Anxiety in Serious Games for Children and Adolescents: Systematic Review10.2196/30482 ↗
12. Dankov, Y., Antonova, A., and Bontchev, B. (2021). Adopting User-Centered Design to Identify Assessment Metrics for Adaptive Video Games for Education. Interaction, Emerging Technologies and Future Systems V, Springer.10.1007/978-3-030-85540-6_37 ↗
13. Suryapranata, L.K.P., Soewito, B., Kusuma, G.P., Gaol, F.L., and Warnars, H.L.H.S. (2017, January 21–23). Quality measurement for serious games. Proceedings of the 2017 International Conference on Applied Computer and Communication Technologies (ComCom), Tumkur, Karnataka, India.10.1109/COMCOM.2017.8167098 ↗
14. Becker, K. (2007, January 15–17). Instructional ethology: Reverse engineering for serious design of educational games. Proceedings of the Future Play, The International Conference on the Future of Game Design and Technology, Toronto, ON, Canada.10.1145/1328202.1328224 ↗
15. Junior, R., and Silva, F. (2021). Redefining the MDA Framework—The Pursuit of a Game Design Ontology. Information, 12.10.3390/info12100395 ↗
16. Dynamical model for gamification of learning (DMGL)10.1007/s11042-013-1612-8 ↗
17. Winn, B.M. (2009). The Design, Play, and Experience Framework. Handbook of Research on Effective Electronic Gaming in Education, IGI Global.10.4018/978-1-59904-808-6.ch058 ↗
18. A psychologically “embedded approach to designing games for prosocial causes10.5817/CP2015-3-5 ↗
19. Pereira, M., Winn, B., Cerazotto, M., Luiz, A., and Varella, P. (2018, January 15–20). Educational Digital Games: A theoretical framework about design models, learning theories and user experience. Proceedings of the 7th International Conference, Design, User Experience and Usability, Theory and Practice, Las Vegas, NV, USA.
20. Quality Criteria for Serious Games: Serious Part, Game Part, and Balance10.2196/19037 ↗
21. Self-directed learning in video games, affordances and pedagogical implications for teaching and learning10.1016/j.compedu.2020.103912 ↗
22. Are They Learning or Playing? Students’ Perception Traits and Their Learning Self-Efficacy in a Game-Based Learning Environment10.1177/0735633118820684 ↗
23. Digital game elements, user experience and learning: A conceptual framework10.1007/s10639-018-9730-6 ↗
24. Fun and games in higher education: An analysis of UK student perspectives10.1080/13562517.2018.1541885 ↗
25. Potential and limits of game-based learning10.1504/IJTEL.2020.110047 ↗
26. Communication, Reasoning, and Planned Behaviors: Unveiling the Effect of Interactive Communication in an Anti-Smoking Social Media Campaign10.1080/10410236.2015.1099501 ↗
27. An investigation of game behavior in the context of digital game-based learning: An individual difference perspective10.1016/j.chb.2020.106432 ↗
28. A systematic literature review of empirical evidence on computer games and serious games10.1016/j.compedu.2012.03.004 ↗
29. Kowert, R., and Quandt, T. (2020). Serious games: Play for change. The Video Game Debate 2, Routledge.10.4324/9780429351815 ↗
30. Balancing learning and enjoyment in serious games: Kerbal Space Program and the communication mediation model10.1016/j.compedu.2022.104480 ↗
31. The Gameplay Loop Methodology as a Tool for Educational Game Design10.34190/JEL.17.3.004 ↗
32. An approach to evaluating the user experience of serious games10.1016/j.compedu.2019.04.006 ↗
33. Challenging games help students learn: An empirical study on engagement, flow and immersion in game-based learning10.1016/j.chb.2015.07.045 ↗
34. Understanding continuance intention to play online games: Roles of self-expressiveness, self-congruity, self-efficacy, and perceived risk10.1080/0144929X.2020.1811770 ↗
35. The Effect of Digital Game-Based Learning on Learning Motivation and Performance under Social Cognitive Theory and Entrepreneurial Thinking10.3389/fpsyg.2021.750711 ↗
36. Defining motivation in video game-based training: Exploring the differences between measures of motivation10.1111/ijtd.12233 ↗
37. Benton, L., Vasalou, A., Barendregt, W., Bunting, L., and Révész, A. (2019, January 4–9). What’s Missing: The Role of Instructional Design in Children’s Games-Based Learning. Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, Glasgow, UK.10.1145/3290605.3300812 ↗
38. Alves, F. (2015). Gamification: Como Criar Experiências de Aprendizagem Engajadoras: Um Guía Completo: Do conceito à Prática, DVS Editora. [2nd ed.].
39. Buley, L. (2013). The User Experience Team of One, Roselfeld.
40. Garrett, J.J. (2011). The Elements of User Experience: User-Centered Design for the Web and Beyond, New Riders. [2nd ed.].
41. Malone, T.W. (1980). What Makes Things Fun to Learn? A Study of Intrinsically Motivating Computer Games, Palo Alto Research Center (Xerox).10.1145/800088.802839 ↗
42. Snow, R.E., and Farr, M.J. (1987). Making learning fun: A taxonomy of intrinsic motivations for learning. Aptitude, Learning and Instruction III: Conative and Affective Process Analyses, Erlbaum.
43. Biles, M.L., Plass, J.L., and Homer, B.D. (2020). Designing Digital Badges for Educational Games. Learning and Performance Assessment: Concepts, Methodologies, Tools, and Applications, IGI Global.10.4018/978-1-7998-0420-8.ch062 ↗
44. Ferrara, J. (2012). Playful Design: Creating Game Experiences in Everyday Interfaces, Rosenfeld Media.
45. Meta-emotions and the complexity of human emotional experience10.1016/j.newideapsych.2019.05.001 ↗
46. Meta-emotions in daily life: Associations with emotional awareness and depression10.1037/emo0000488 ↗
47. Factors Influencing the Subjective Learning Effectiveness of Serious Games
48. Menezes, F.M., Silva, I.C.S., and Frosi, F.O. (2017, January 2–4). Game User Experience (UX): Explorando a Teoria da Diegese. Proceedings of the SBGames 2017 (XVI SBGames (SBC)), São Paulo, SP, Brazil.
49. Exploring students’ flow experiences in business simulation games10.1111/jcal.12237 ↗
50. Design and Evaluation of a «Gamified» System for Improving Career Knowledge in Computing Sciences
51. An educational game for learning human immunology: What do students learn and how do they perceive?10.1111/bjet.12098 ↗
52. Design and evaluation of history Digital Game Based Learning (DGBL) software
53. Thinking technology: Toward a constructivist design model
54. Game-based curriculum and transformational play: Designing to meaningfully positioning person, content, and context10.1016/j.compedu.2011.08.001 ↗
55. A comparative study of the learning effectiveness of a blended and embodied interactive video game for kindergarten students10.1080/10494820.2010.542760 ↗
56. An online game approach for improving students’ learning performance in web-based problem-solving activities10.1016/j.compedu.2012.05.009 ↗
57. On the nature of reactance and its role in persuasive health communication10.1080/03637750500111815 ↗
58. Adolescent reactance and anti-smoking campaigns: A theoretical approach10.1207/S15327027HC1503_6 ↗
59. Examining the use of forceful language when designing exercise persuasive messages for adults: A test of conceptualizing reactance arousal as a two-step process10.1080/10410230802342150 ↗
60. Kaufman, G., Flanagan, M., and Seidman, M. (2015, January 14–17). Creating stealth game interventions for attitude and behavior change: An “Embedded Design” model. Proceedings of the Digital Games Research Association (DiGRA) Conference, Lüneburg, Germany.10.26503/todigra.v2i3.57 ↗
61. Lost in translation: Comparing the impact of an analog and digital version of a public health game on players’ perceptions, attitudes, and cognitions10.4018/jgcms.2013070101 ↗
62. Strategies for stimulating the motivation to learn
63. Maximizing learning without sacrificing the fun: Stealth assessment, adaptivity and learning supports in educational games10.1111/jcal.12473 ↗
64. Associations among scaffold presentation, reward mechanisms and problem-solving behaviors in game play10.1016/j.compedu.2018.01.001 ↗
65. Design and evaluation of a computer game for the learning of Information and Communication Technologies (ICT) concepts by physical education and sport science students10.1007/s10639-011-9179-3 ↗
66. Designing and evaluating collaboration in a virtual game environment for vocational learning10.1016/j.compedu.2006.04.001 ↗
67. Games for engaged learning of middle school children with special learning needs10.1111/j.1467-8535.2012.01326.x ↗
68. Implementing digital game-based learning in schools: Augmented learning environment of ‘Europe 2045’10.1007/s00530-009-0174-0 ↗
69. A concept map-embedded educational computer game for improving students’ learning performance in natural science courses10.1016/j.compedu.2013.07.008 ↗
70. Students’ learning outcomes and learning experiences through playing a serious educational game10.1080/00219266.2012.688848 ↗
71. A case study of computer gaming for math: Engaged learning from gameplay?10.1016/j.compedu.2008.03.003 ↗
72. Content creation challenges and flow experience in educational games: The IT-Emperor case10.1016/j.iheduc.2005.06.001 ↗
73. Micro-adaptivity: Protecting immersion in didactically adaptive digital educational games10.1111/j.1365-2729.2009.00332.x ↗
74. A quest for meta-learning gains in a physics serious game10.1007/s10639-012-9219-7 ↗
75. Why and How Serious Games can Become Far More Effective: Accommodating Productive Learning Experiences, Learner Motivation and the Monitoring of Learning Gains
76. Enhancing the educational value of video games10.1145/1541895.1541903 ↗
77. Investigating the impact of video games on high school students’ engagement and learning about genetics10.1016/j.compedu.2008.12.020 ↗
78. Good video games and good learning
79. Designing social videogames for educational uses10.1016/j.compedu.2011.08.014 ↗
80. Applying the self-explanation principle to multimedia learning in a computer-based game-like environment10.1016/j.chb.2010.03.025 ↗
81. The effect of contextual pedagogical advisement and competition on middle-school students’ attitude toward mathematics and mathematics instruction using a computer-based simulation game
82. Russell, S.J., and Norvig, P. (2014). Artificial Intelligence: A Modern Approach, Pearson New International Edition. [3rd ed.].
83. Learning through playing Virtual Age: Exploring the interactions among student concept learning, gaming performance, in-game behaviors, and the use of in-game characters10.1016/j.compedu.2015.03.007 ↗
84. Design and analysis of collaborative interactions in social educational videogames10.1016/j.chb.2013.06.039 ↗
85. An application of adaptive games-based learning based on learning style to teach SQL10.1016/j.compedu.2015.03.015 ↗
86. The effectiveness of adaptive versus non-adaptive learning with digital educational games10.1111/jcal.12416 ↗
87. Playing style recognition through an adaptive video game10.1016/j.chb.2017.12.040 ↗
88. Examining competitive, collaborative and adaptive gamification in young learners’ math learning10.1016/j.compedu.2018.06.022 ↗
89. Serious video games for health: How behavioral science guided the development of a serious video game10.1177/1046878108328087 ↗
90. A serious game model for cultural heritage10.1145/2399180.2399185 ↗
91. Conceptual change takes time: Game based learning cannot be only supplementary amusement
92. Combining software games with education: Evaluation of its educational effectiveness
93. Introducing surprising events can stimulate deep learning in a serious game10.1111/j.1467-8535.2011.01282.x ↗
94. A contextual game-based learning approach to improving students’ inquiry-based learning performance in social studies courses10.1016/j.compedu.2014.09.006 ↗
95. Assessing knowledge retention of an immersive serious game vs. a traditional education method in aviation safety10.1109/TVCG.2015.2391853 ↗
96. Serious gaming technology in major incident triage training: A pragmatic controlled trial10.1016/j.resuscitation.2010.03.042 ↗
97. Ifenthaler, D., and Kim, Y.J. (2019). Emerging practices in game-based assessment. Game-Based Assessment Revisited, Springer International Publishing.10.1007/978-3-030-15569-8 ↗
98. Using a game environment to foster collaborative learning: A design-based study Technology10.1080/1475939X.2011.554010 ↗
99. A knowledge engineering approach to developing educational computer games for improving students’ differentiating knowledge10.1111/j.1467-8535.2012.01285.x ↗
100. Csikszentmihalyi, M. (2008). Flow: The Psychology of Optimal Experience, Harper Perennial. [2nd ed.].