Evaluation of the Human Capacity to Detect Spanish Deepfake Audios with a Paraguayan Accent
Source abstract
Abstract
Deepfakes, synthetic multimedia files generated by artificial intelligence, are drastically undermining digital credibility. Their ability to manipulate our perception of reality has created a new and complex battleground for disinformation, posing a critical threat to non-English-speaking audio with distinctive accents. Consequently, the objective of this study is to determine the human capacity to detect deepfake audio in Spanish with a Paraguayan accent through an experiment conducted with an Android application called ReFake (developed specifically for this research). In this experiment, 450 participants, aged 16–72, evaluated 10 audio samples of up to 15 s each, classifying them as authentic (belonging to Paraguayan journalists) or fake (generated with ElevenLabs). The findings suggests that human ear is more accurate than artificial intelligence (AI) at detecting vocal ‘naturalness’. This ability is influenced by generational age and educational level, with younger people and those with postgraduate degrees demonstrating greater performance. Conversely, gender and nationality do not influence detection, although the high prosodic quality of deepfakes still leads to errors in human judgment. Given these results, it is crucial to adapt and develop new strategies for a secure and resilient online ecosystem.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Digital Media Forensic Detection · Emotion and Mood Recognition · Multisensory perception and integration
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Facultad Politécnica, Universidad Nacional de Asunción, San Lorenzo 111421, Paraguay
Escuela de Ingeniería Eléctrica, Pontificia Universidad Católica de Valparaíso, Av. Brasil 2950, Valparaíso 2340000, Chile
Facultad de Ciencias de la Salud, Universidad Sudamericana, Pedro Juan Caballero 1206, Paraguay
Funding
ANID Doctorado Nacional
References (27)
Show source-provided references
1. Collins, A. (2019). Forged Authenticity: Governing Deepfake Risks, EPFL International Risk Governance Center (IRGC). Available online: https://scholar.google.com/scholar_lookup?title=Forged+Authenticity:+Governing+Deepfake+Risks&author=Collins,+A.&publication_year=2019.
2. Gaur, L. (2023). Deepfakes: Creation, Detection, and Impact, CRC Press. [1st ed.].10.1201/9781003231493 ↗
3. Lyon, B., and Tora, M. (2023). Exploring Deepfakes: Deploy Powerful AI Techniques for Face Replacement and More with This Comprehensive Guide, Packt Publishing.
4. Deepfakes: Current and future trends10.1007/s10462-023-10679-x ↗
5. Human detection of political speech deepfakes across transcripts, audio, and video10.1038/s41467-024-51998-z ↗
6. Testing human ability to detect ‘deepfake’ images of human faces10.1093/cybsec/tyad011 ↗
7. Müller, N.M., Pizzi, K., and Williams, J. (2022, January 14). Human perception of audio deepfakes. Proceedings of the 1st International Workshop on Deepfake Detection for Audio Multimedia, Lisboa, Portugal.10.1145/3552466.3556531 ↗
8. A systematic literature review on the effectiveness of deepfake detection techniques10.1080/23742917.2023.2192888 ↗
9. Human performance in detecting deepfakes: A systematic review and meta-analysis of 56 papers10.1016/j.chbr.2024.100538 ↗
10. Frank, J., Herbert, F., Ricker, J., Schönherr, L., Eisenhofer, T., Fischer, A., Dürmuth, M., and Holz, T. (2024). A Representative Study on Human Detection of Artificially Generated Media Across Countries. 2024 IEEE Symposium on Security and Privacy (SP), San Francisco, CA, USA, 2024, IEEE.10.1109/SP54263.2024.00159 ↗
11. Yi, J., Wang, C., Tao, J., Zhang, X., Zhang, C.Y., and Zhao, Y. (2023). Audio deepfake detection: A survey. arXiv.
12. Detecting fake audio of arabic speakers using self-supervised deep learning10.1109/ACCESS.2023.3286864 ↗
13. Review of audio deepfake detection techniques: Issues and prospects10.1111/exsy.13322 ↗
14. The effect of deep learning methods on deepfake audio detection for digital investigation10.1016/j.procs.2023.01.283 ↗
15. Xie, Z., Li, B., Xu, X., Liang, Z., Yu, K., and Wu, M. (2024). FakeSound: Deepfake general audio detection. arXiv.10.21437/Interspeech.2024-1703 ↗
16. Video and Audio Deepfake Datasets and Open Issues in Deepfake Technology: Being Ahead of the Curve10.3390/forensicsci4030021 ↗
17. Audio-deepfake detection: Adversarial attacks and countermeasures10.1016/j.eswa.2024.123941 ↗
18. Alali, A., and Theodorakopoulos, G. (2025). Partial fake speech attacks in the real world using deepfake audio. J. Cybersecur. Priv., 5.10.3390/jcp5010006 ↗
19. Ouajdi, H., Hadder, O., Tailleur, M., Lagrange, M., and Heller, L.M. (2024). Detection of deepfake environmental audio. 2024 32nd European Signal Processing Conference (EUSIPCO), IEEE.10.23919/EUSIPCO63174.2024.10715076 ↗
20. Mai, K.T., Bray, S., Davies, T., and Griffin, L.D. (2023). Warning: Humans cannot reliably detect speech deepfakes. PLoS ONE, 18.10.1371/journal.pone.0285333 ↗
21. Rathgeb, C., Tolosana, R., Vera-Rodriguez, R., and Busch, C. (2022). Handbook of Digital Face Manipulation and Detection From DeepFakes to Morphing Attacks, Springer.10.1007/978-3-030-87664-7 ↗
22. Babaei, R., Cheng, S., Duan, R., and Zhao, S. (2025). Generative Artificial Intelligence and the Evolving Challenge of Deepfake Detection: A Systematic Analysis. J. Sens. Actuator Netw., 14.10.3390/jsan14010017 ↗
23. Han, C., Mitra, P., and Billah, S.M. (2024). Uncovering human traits in determining real and spoofed audio: Insights from blind and sighted individuals. Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems, ACM.10.1145/3613904.3642817 ↗
24. Cooke, D., Edwards, A., Barkoff, S., and Kelly, K. (2024). As Good As A Coin Toss: Human detection of AI-generated images, videos, audio, and audiovisual stimuli. arXiv.10.1145/3729417 ↗
25. Classification Assessment Methods10.1016/j.aci.2018.08.003 ↗
26. wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations
27. An Introduction to ROC Analysis10.1016/j.patrec.2005.10.010 ↗