The Novice, the Expert, and the Algorithm: A Comparative Analysis of Human Expertise Transfer and AI Performance in Audio-Only Gaming Environments
Source abstract
Abstract
This study provides a symmetrical, cross-genre comparison of human expertise transfer and “blind” artificial intelligence (AI) performance in audio-only gaming environments. Although previous research has focused on human performance in audio games and the feasibility of blind agents trained on auditory inputs separately, a direct comparison of these two forms of expertise is missing. We fill this gap with a robust experimental design, involving 37 human players (aged 18–44), grouped by gaming experience and specialized blind AI agents. We measured key performance variables, including win ratios, health differences, and task completion times across two genres: a fighting game (DareFightingICE) and a first-person shooter (SonicDoom). Our findings show a complex, task-dependent relationship. In DareFightingICE, expert humans (73.0% win ratio) significantly outperformed the AI (54.0% win ratio), demonstrating effective cognitive transfer. Meanwhile, the AI’s performance matched the overall human average (54.0% vs. 53.0%). Conversely, in SonicDoom, AI achieved superhuman speed in simple tasks (1.55 s vs. 5.35 s) but underperformed compared to expert humans in complex scenarios, highlighting that the AI’s proficiency is specialized but fragile, whereas human expertise is more robust and adaptable. The results provide practical insights for audio-rich game design and highlight the crucial need for AI models beyond reactive policies.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: AI in Service Interactions · Social Robot Interaction and HRI · Tactile and Sensory Interactions
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Graduate School of Information Science and Engineering, Ritsumeikan University, Ibaraki 567-8570, Japan
Laboratory of Agent Based Social Simulation, Institute of Cognitive Science and Technology, Consiglio Nazionale delle Ricerche Roma, 00185 Roma, Italy
College of Information Science and Engineering, Ritsumeikan University, Ibaraki 567-8570, Japan
References (64)
Show source-provided references
1. Collins, K. (2017). From Pac-Man to Pop Music: Interactive Audio in Games and New Media, Routledge.10.4324/9781351217743 ↗
2. Gursesli, M.C., Tarchi, P., Guazzini, A., Duradoni, M., Yaras, A.T., Akinci, E., Yurdakul, U., Calà, F., Tonacci, A., and Vilone, D. (2025, January 16–18). Silent vs. Sound: The Impact of Uniform Auditory Stimuli on Eye Movements and Game Performance. Proceedings of the 2025 IEEE Gaming, Entertainment, and Media Conference (GEM), Kaohsiung, Taiwan.10.1109/GEM66882.2025.11155778 ↗
3. Andrade, R., Rogerson, M.J., Waycott, J., Baker, S., and Vetere, F. (2019, January 4–9). Playing Blind: Revealing the World of Gamers with Visual Impairment. Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, Glasgow, UK.10.1145/3290605.3300346 ↗
4. Video game players show more precise multisensory temporal processing abilities10.3758/APP.72.4.1120 ↗
5. Massiceti, D., Hicks, S.L., and van Rheede, J.J. (2018). Stereosonic vision: Exploring visual-to-auditory sensory substitution mappings in an immersive virtual reality navigation paradigm. PLoS ONE, 13.10.1371/journal.pone.0199389 ↗
6. Connors, E.C., Yazzolino, L.A., Sánchez, J., and Merabet, L.B. (2013). Development of an audio-based virtual gaming environment to assist with navigation skills in the blind. J. Vis. Exp. JoVE.10.3791/50272 ↗
7. Human-level control through deep reinforcement learning10.1038/nature14236 ↗
8. Grandmaster level in StarCraft II using multi-agent reinforcement learning10.1038/s41586-019-1724-z ↗
9. Van Nguyen, T., Dai, X., Khan, I., Thawonmas, R., and Pham, H.V. (2022, January 21–24). A deep reinforcement learning blind AI in DareFightingICE. Proceedings of the 2022 IEEE Conference on Games (CoG), Beijing, China.10.1109/CoG51982.2022.9893718 ↗
10. Lake, B.M., Ullman, T.D., Tenenbaum, J.B., and Gershman, S.J. (2017). Building machines that learn and think like people. Behav. Brain Sci., 40.10.1017/S0140525X16001837 ↗
11. Facial emotion recognition (FER) through custom lightweight CNN model: Performance evaluation in public datasets10.1109/ACCESS.2024.3380847 ↗
12. Khan, I., Van Nguyen, T., Dai, X., and Thawonmas, R. (2022, January 21–24). Darefightingice competition: A fighting game sound design and ai competition. Proceedings of the 2022 IEEE Conference on Games (CoG), Beijing, China.10.1109/CoG51982.2022.9893624 ↗
13. Kempka, M., Wydmuch, M., Runc, G., Toczek, J., and Jaśkowski, W. (2016, January 20–23). Vizdoom: A doom-based ai research platform for visual reinforcement learning. Proceedings of the 2016 IEEE Conference on Computational Intelligence and Games (CIG), Santorini, Greece.10.1109/CIG.2016.7860433 ↗
14. Smuts, A. (2005). Are video games art?. Contemp. Aesthet. (J. Arch.), 3.
15. Development and validation of the psychological motivations for playing video games scale (PMPVGs)10.1177/10468781241260861 ↗
16. Gursesli, M.C., Guazzini, A., Thawonmas, R., Valenti, C., Duradoni, M., and Thawonmas, R. (2025). Internet Gaming Disorder and Psychological Distress: A PRISMA systematic review. Heliyon, 11.10.1016/j.heliyon.2025.e43518 ↗
17. Player motivations: A psychological perspective
18. A motivational model of video game engagement10.1037/a0019440 ↗
19. Thiparpakul, P., Mokekhaow, S., and Supabanpot, K. (2021, January 27–29). How can video game atmosphere affect audience emotion with sound. Proceedings of the 2021 9th International Conference on Information and Education Technology (ICIET), Okayama, Japan.10.1109/ICIET51873.2021.9419652 ↗
20. Guillen, G., Jylha, H., and Hassan, L. (2021, January 1–3). The role sound plays in games: A thematic literature study on immersion, inclusivity and accessibility in game sound research. Proceedings of the 24th International Academic Mindtrek Conference, Virtual, Finland.10.1145/3464327.3464365 ↗
21. Paterson, N., Naliuka, K., Jensen, S.K., Carrigy, T., Haahr, M., and Conway, F. (2010, January 8–10). Spatial audio and reverberation in an augmented reality game sound design. Proceedings of the 40th AES Conference: Spatial Audio, Tokyo, Japan.
22. Haehn, L., Schlittmeier, S.J., and Böffel, C. (2024). Exploring the impact of ambient and character sounds on player experience in video games. Appl. Sci., 14.10.3390/app14020583 ↗
23. More than a feeling: Measurement of sonic user experience and psychophysiology in a first-person shooter game10.1016/j.intcom.2010.04.005 ↗
24. Broderick, J., Duggan, J., and Redfern, S. (2018, January 15–17). The importance of spatial audio in modern games and virtual environments. Proceedings of the 2018 IEEE Games, Entertainment, Media Conference (GEM), Galway, Ireland.10.1109/GEM.2018.8516445 ↗
25. Semionov, K., and McGregor, I. (2020, January 15–17). Effect of various spatial auditory cues on the perception of threat in a first-person shooter video game. Proceedings of the 15th International Audio Mostly Conference, Graz, Austria.10.1145/3411109.3411119 ↗
26. Sadowska, D., Sacewicz, T., Rębiś, K., Kowalski, T., and Krzepota, J. (2023). Examining physiological changes during counter-strike: Global offensive (CS: GO) performance in recreational male esports players. Appl. Sci., 13.10.3390/app132011526 ↗
27. Ng, P., Nesbitt, K., and Blackmore, K. (2015, January 5–7). Sound improves player performance in a multiplayer online battle arena game. Proceedings of the Australasian Conference on Artificial Life and Computational Intelligence, Newcastle, NSW, Australia.10.1007/978-3-319-14803-8_13 ↗
28. Ng, P., and Nesbitt, K. (October, January 30). Informative sound design in video games. Proceedings of the 9th Australasian Conference on Interactive Entertainment: Matters of Life and Death, Melbourne, Australia.
29. Schaffert, N., Mattes, K., and Effenberg, A.O. (2009). A sound design for acoustic feedback in elite sports. International Symposium on Computer Music Modeling and Retrieval, Proceedings of the 6th International Symposium, CMMR/ICAD 2009, Copenhagen, Denmark, 18–22 May 2009, Springer.
30. Bandai Namco Entertainment (Tekken 7, 2015). Tekken 7.
31. NetherRealm Studios (Mortal Kombat 11, 2019). Mortal Kombat 11.
32. Double Helix Games (Killer Instinct, 2013). Killer Instinct.
33. Khan, I., Van Nguyen, T., and Thawonmas, R. (2025). Fighting to the beat: Multi-instrumental adaptive background music approach. Entertain. Comput., 55.10.1016/j.entcom.2025.100985 ↗
34. Game accessibility for visually impaired people: A review10.1007/s00500-024-09827-4 ↗
35. Games, G. (2025, October 27). Shades of Doom. Available online: https://www.gmagames.com/sod.html.
36. AGRIP (2025, October 27). AudioQuake. Available online: https://www.igdb.com/games/audioquake.
37. The Terraformers Team (2025, October 27). Terraformers. Available online: https://terraformers.nu/.
38. Nair, V., Karp, J.L., Silverman, S., Kalra, M., Lehv, H., Jamil, F., and Smith, B.A. (2021, January 10–14). Navstick: Making video games blind-accessible via the ability to look around. Proceedings of the 34th Annual ACM Symposium on User Interface Software and Technology, Virtual.10.1145/3472749.3474768 ↗
39. Nair, V., Zhu, H., Song, P., Wang, J., and Smith, B.A. (2024, January 11–16). Surveyor: Facilitating Discovery Within Video Games for Blind and Low Vision Players. Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems, Honolulu, HI, USA.10.1145/3613904.3642615 ↗
40. Marples, D., Gledhill, D., and Carter, P. (2020, January 5–7). The effect of lighting, landmarks and auditory cues on human performance in navigating a virtual maze. Proceedings of the Symposium on Interactive 3D Graphics and Games, San Francisco, CA, USA.10.1145/3384382.3384527 ↗
41. Tan, S.L., Baxa, J., and Spackman, M. (2010). The role of music in video games. Playing Video Games: Motives, Responses, and Consequences, Routledge.
42. Nacke, L.E., Grimshaw, M.N., and Lindley, C.A. (2010, January 15–17). The effects of sound and graphics on immersion in video games. Proceedings of the Audio Mostly Conference, Piteå, Sweden.
43. Mirza, B., Hoque, M.E., and Mahmud, H. (2019). The impact of audio-visual stimuli on gaming performance. Entertain. Comput., 31.
44. Lipscomb, S.D., and Zehnder, S.M. (2005, January 4–10). Interactive music: Crossing boundaries. Proceedings of the 2005 International Computer Music Conference, Barcelona, Spain.
45. Khan, I., Van Nguyen, T., Gursesli, M.C., and Thawonmas, R. (2025, January 26–29). Sonic Doom: Enhanced Sound Design and Accessibility in a First-Person Shooter Game. Proceedings of the 2025 IEEE Conference on Games (CoG), Lisbon, Portugal.10.1109/CoG64752.2025.11114112 ↗
46. Ishihara, M., Miyazaki, T., Chu, C.Y., Harada, T., and Thawonmas, R. (2016, January 9–12). Applying and improving Monte-Carlo Tree Search in a fighting game AI. Proceedings of the 13th International Conference on Advances in Computer Entertainment Technology, Osaka, Japan.10.1145/3001773.3001797 ↗
47. Organizers, D.C. (2025, September 09). The DareFightingICE Sound Design and AI Competition Winners Details, R22. Results of a Competition Held at the IEEE Conference on Games (CoG) 2022. Available online: https://www.ice.ci.ritsumei.ac.jp/~ftgaic/index-R22.html.
48. Schulman, J., Wolski, F., Dhariwal, P., Radford, A., and Klimov, O. (2017). Proximal Policy Optimization Algorithms. arXiv.
49. Petrenko, A., Huang, Z., Kumar, T., Sukhatme, G., and Koltun, V. (2020, January 13–18). Sample factory: Egocentric 3d control from pixels at 100000 fps with asynchronous reinforcement learning. Proceedings of the International Conference on Machine Learning. PMLR, Virtual.
50. Cheiran, J.F., Nedel, L., and Pimenta, M.S. (2011, January 7–9). Inclusive games: A multimodal experience for blind players. Proceedings of the 2011 Brazilian Symposium on Games and Digital Entertainment, Salvador, Brazil.10.1109/SBGAMES.2011.24 ↗
51. Smith, B.A., and Nayar, S.K. (2018, January 21–26). The RAD: Making racing games equivalently accessible to people who are blind. Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, Montreal, QC, Canada.10.1145/3173574.3174090 ↗
52. Zeng, L., and Weber, G. (2015, January 24–27). A pilot study of collaborative accessibility: How blind people find an entrance. Proceedings of the 17th International Conference on Human-Computer Interaction with Mobile Devices and Services, Copenhagen, Denmark.10.1145/2785830.2785875 ↗
53. The development and validation of the game user experience satisfaction scale (GUESS)10.1177/0018720816669646 ↗
54. Time for some a priori thinking about post hoc testing10.1093/beheco/arn020 ↗
55. Evolving knowledge management: Artificial intelligence and the dynamics of social interactions10.1109/EMR.2024.3519342 ↗
56. Whatever next? Predictive brains, situated agents, and the future of cognitive science10.1017/S0140525X12000477 ↗
57. Soundspaces 2.0: A simulation platform for visual-acoustic learning
58. Ha, D., and Schmidhuber, J. (2018). World models. arXiv.
59. Cheng, L., Wei, X., Li, M., Tan, C., Yin, M., Shen, T., and Zou, T. (2024). Integrating Evolutionary Game-Theoretical Methods and Deep Reinforcement Learning for Adaptive Strategy Optimization in User-Side Electricity Markets: A Comprehensive Review. Mathematics, 12.10.3390/math12203241 ↗
60. EEG-triggered dynamic difficulty adjustment for multiplayer games10.1016/j.entcom.2017.11.003 ↗
61. Soares, R.T., Sarmanho, E., Miura, M., Barros, T., Jacobi, R., and Castanho, C. (2017, January 2–4). Biofeedback sensors in electronic games: A practical evaluation. Proceedings of the 2017 16th Brazilian symposium on computer games and digital entertainment (SBGames), Curitiba, Brazil.
62. Hunicke, R. (2005, January 15–17). The case for dynamic difficulty adjustment in games. Proceedings of the 2005 ACM SIGCHI International Conference on Advances in Computer Entertainment Technology, Valencia, Spain.10.1145/1178477.1178573 ↗
63. Gursesli, M.C., Selek, M.E., Samur, M.O., Duradoni, M., Park, K., Guazzini, A., and Lanatà, A. (2023). Design of cloud-based real-time eye-tracking monitoring and storage system. Algorithms, 16.10.20944/preprints202306.1589.v1 ↗
64. Gursesli, M.C., Calà, F., Tarchi, P., Frassineti, L., Guazzini, A., and Lanatà, A. (2023, January 19–22). Eyetracking correlated in the matching pairs game. Proceedings of the 2023 IEEE Gaming, Entertainment, and Media Conference (GEM), Bridgetown, Barbados.10.1109/GEM59776.2023.10390418 ↗