Detecting Deepfake Voice Using Explainable Deep Learning Techniques
Source abstract
Abstract
Fake media, generated by methods such as deepfakes, have become indistinguishable from real media, but their detection has not improved at the same pace. Furthermore, the absence of interpretability on deepfake detection models makes their reliability questionable. In this paper, we present a human perception level of interpretability for deepfake audio detection. Based on their characteristics, we implement several explainable artificial intelligence (XAI) methods used for image classification on an audio-related task. In addition, by examining the human cognitive process of XAI on image classification, we suggest the use of a corresponding data format for providing interpretability. Using this novel concept, a fresh interpretation using attribution scores can be provided.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Digital Media Forensic Detection · Music and Audio Processing · Anomaly Detection Techniques and Applications
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Department of Artificial Intelligence, Hanyang University, Seoul 04763, Korea
Division of Nanotechnology, Daegu Gyeongbuk Institute of Science & Technology (DGIST), Deagu 42988, Korea
Funding
Korea government(MSIT) · Korean government (MSIT) · Ministry of Science and ICT of KOREA
References (22)
Show source-provided references
1. Dolhansky, B., Bitton, J., Pflaum, B., Lu, J., Howes, R., Wang, M., and Ferrer, C.C. (2020). The deepfake detection challenge (dfdc) dataset. arXiv.
2. Malolan, B., Parekh, A., and Kazi, F. (2020, January 9–12). Explainable deep-fake detection using visual interpretability methods. Proceedings of the 2020 3rd International Conference on Information and Computer Technologies (ICICT), San Jose, CA, USA.10.1109/ICICT50521.2020.00051 ↗
3. Baevski, A., Zhou, H., Mohamed, A., and Auli, M. (2020). wav2vec 2.0: A framework for self-supervised learning of speech representations. arXiv.
4. Dong, L., Xu, S., and Xu, B. (2018, January 15–20). Speech-transformer: A no-recurrence sequence-to-sequence model for speech recognition. Proceedings of the 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Calgary, AB, Canada.10.1109/ICASSP.2018.8462506 ↗
5. Gulati, A., Qin, J., Chiu, C.C., Parmar, N., Zhang, Y., Yu, J., Han, W., Wang, S., Zhang, Z., and Wu, Y. (2020). Conformer: Convolution-augmented transformer for speech recognition. arXiv.10.21437/Interspeech.2020-3015 ↗
6. Kanagasundaram, A., Vogt, R., Dean, D., Sridharan, S., and Mason, M. (2011, January 27–31). I-vector based speaker recognition on short utterances. Proceedings of the 12th Annual Conference of the International Speech Communication Association, Florence, Italy.10.21437/Interspeech.2011-58 ↗
7. Snyder, D., Garcia-Romero, D., Sell, G., Povey, D., and Khudanpur, S. (2018, January 15–20). X-vectors: Robust dnn embeddings for speaker recognition. Proceedings of the 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Calgary, AB, Canada.10.1109/ICASSP.2018.8461375 ↗
8. Variani, E., Lei, X., McDermott, E., Moreno, I.L., and Gonzalez-Dominguez, J. (2014, January 4–9). Deep neural networks for small footprint text-dependent speaker verification. Proceedings of the 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Florence, Italy.10.1109/ICASSP.2014.6854363 ↗
9. Ito, K., and Johnson, L. (2021, June 15). The LJ Speech Dataset. Available online: https://keithito.com/LJ-Speech-Dataset/.
10. ASVspoof 2019: A large-scale public database of synthesized, converted and replayed speech10.1016/j.csl.2020.101114 ↗
11. Bharadhwaj, H. (2018, January 6–8). Layer-wise relevance propagation for explainable deep learning based speech recognition. Proceedings of the 2018 IEEE International Symposium on Signal Processing and Information Technology (ISSPIT), Louisville, KY, USA.10.1109/ISSPIT.2018.8642691 ↗
12. Explaining nonlinear classification decisions with deep taylor decomposition10.1016/j.patcog.2016.11.008 ↗
13. Sundararajan, M., Taly, A., and Yan, Q. (2017). Axiomatic attribution for deep networks. arXiv.
14. Bracewell, R.N., and Bracewell, R.N. (1986). The Fourier Transform and Its Applications, McGraw-Hill.
15. Speech emotion recognition from 3D log-mel spectrograms with deep learning network10.1109/ACCESS.2019.2938007 ↗
16. A practical comparison of manual and autonomous methods for acoustic monitoring10.1111/2041-210X.12060 ↗
17. Zhou, Y., and Lim, S.N. (2021, January 11–17). Joint Audio-Visual Deepfake Detection. Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), Montreal, BC, Canada.10.1109/ICCV48922.2021.01453 ↗
18. Becker, S., Ackermann, M., Lapuschkin, S., Müller, K.R., and Samek, W. (2018). Interpreting and explaining deep neural networks for classification of audio signals. arXiv.
19. Explaining CNN and RNN Using Selective Layer-Wise Relevance Propagation10.1109/ACCESS.2021.3051171 ↗
20. Shen, J., Pang, R., Weiss, R.J., Schuster, M., Jaitly, N., Yang, Z., Chen, Z., Zhang, Y., Wang, Y., and Skerrv-Ryan, R. (2018, January 15–20). Natural tts synthesis by conditioning wavenet on mel spectrogram predictions. Proceedings of the 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Calgary, AB, Canada.10.1109/ICASSP.2018.8461368 ↗
21. Nam, W.J., Gur, S., Choi, J., Wolf, L., and Lee, S.W. (2020, January 7–12). Relative attributing propagation: Interpreting the comparative contributions of individual units in deep neural networks. Proceedings of the AAAI Conference on Artificial Intelligence, New York, NY, USA.10.1609/aaai.v34i03.5632 ↗
22. Signal estimation from modified short-time Fourier transform10.1109/TASSP.1984.1164317 ↗