Deepfake Face Detection and Adversarial Attack Defense Method Based on Multi-Feature Decision Fusion
Source abstract
Abstract
The rapid advancement in deep forgery technology in recent years has created highly deceptive face video content, posing significant security risks. Detecting these fakes is increasingly urgent and challenging. To improve the accuracy of deepfake face detection models and strengthen their resistance to adversarial attacks, this manuscript introduces a method for detecting forged faces and defending against adversarial attacks based on a multi-feature decision fusion. This approach allows for rapid detection of fake faces while effectively countering adversarial attacks. Firstly, an improved IMTCCN network was employed to precisely extract facial features, complemented by a diffusion model for noise reduction and artifact removal. Subsequently, the FG-TEFusionNet (Facial-geometry and Texture enhancement fusion-Net) model was developed for deepfake face detection and assessment. This model comprises two key modules: one for extracting temporal features between video frames and another for spatial features within frames. Initially, a facial geometry landmark calibration module based on the LRNet baseline framework ensured an accurate representation of facial geometry. A SENet attention mechanism was then integrated into the dual-stream RNN to enhance the model’s capability to extract inter-frame information and derive preliminary assessment results based on inter-frame relationships. Additionally, a Gram image texture feature module was designed and integrated into EfficientNet and the attention maps of WSDAN (Weakly Supervised Data Augmentation Network). This module aims to extract deep-level feature information from the texture structure of image frames, addressing the limitations of purely geometric features. The final decisions from both modules were integrated using a voting method, completing the deepfake face detection process. Ultimately, the model’s robustness was validated by generating adversarial samples using the I-FGSM algorithm and optimizing model performance through adversarial training. Extensive experiments demonstrated the superior performance and effectiveness of the proposed method across four subsets of FaceForensics++ and the Celeb-DF dataset.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Digital Media Forensic Detection · Anomaly Detection Techniques and Applications · Adversarial Robustness in Machine Learning
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
College of Information Engineering, Xizang Minzu University, No. 6, East Section of Wenhui Road, Weicheng District, Xianyang 712082, China
Funding
Scientific Research Program · Innovation Practice Project · Xizang Minzu University · National Natural Science Foundation of China · Special Scientific Research Program Project of the Shaanxi Provincial Department of Education
References (47)
Show source-provided references
1. Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., and Bengio, Y. (2014). Generative Adversarial Nets, MIT Press.
2. Denoising diffusion probabilistic models
3. Deep forgery video tampering detection based on 3D CNNS
4. Fu, X., Yan, Z., Yao, T., Fu, X., and Yan, Z. (2025). Exploring Unbiased Deepfake Detection via Token-Level Shuffling and Mixing. arXiv.10.1609/aaai.v39i3.32312 ↗
5. Enhanced deepfake detection with DenseNet and Cross-ViT10.1016/j.eswa.2024.126150 ↗
6. Frank, J., Eisenhofer, T., Schönherr, L., Fischer, A., Kolossa, D., and Holz, T. (2020, January 13–18). Leveraging frequency analysis for deep fake image recognition. Proceedings of the International Conference on Machine Learning, PMLR, Virtual.
7. Recurrent convolutional strategies for face manipulation detection in videos
8. Gu, Z., Chen, Y., Yao, T., Ding, S., Li, J., Huang, F., and Ma, L. (2021, January 20–24). Spatiotemporal inconsistency learning for deepfake video detection. Proceedings of the 29th ACM International Conference on Multimedia, Virtual.10.1145/3474085.3475508 ↗
9. Kroiß, L., and Reschke, J. (2025). Deepfake Detection of Face Images based on a Convolutional Neural Network. arXiv.
10. Kundu, R., Balachandran, A., and Roy-Chowdhury, A.K. (2025). TruthLens: Explainable DeepFake Detection for Face Manipulated and Fully Synthetic Data. arXiv.
11. Cheng, J., Yan, Z., Zhang, Y., Luo, Y., Wang, Z., and Li, C. (2024). Can We Leave Deepfake Data Behind in Training Deepfake Detector?. arXiv.
12. Choi, J., Kim, T., Jeong, Y., Baek, S., and Choi, J. (2024, January 16–22). Exploiting style latent flows for generalizing deepfake video detection. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.10.1109/CVPR52733.2024.00114 ↗
13. Cozzolino, D., Poggi, G., Corvi, R., Nießner, M., and Verdoliva, L. (2024, January 16–22). Raising the Bar of AI-generated Image Detection with CLIP. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.10.1109/CVPRW63382.2024.00439 ↗
14. Yang, T., Wu, J., Liu, L., Chang, X., and Feng, G. (2020, January 27–29). VTD-Net: Depth face forgery oriented video tampering detection based on convolutional neural network. Proceedings of the 2020 39th Chinese Control Conference (CCC), Shenyang, China.10.23919/CCC50068.2020.9188580 ↗
15. Chinchalkar, R., Sinha, R., Kumar, M., Chauhan, N., Deokar, S., and Gonge, S. (2023, January 23–25). Detecting Deepfakes using CNN and LSTM. Proceedings of the 2023 Second International Conference on Informatics (ICI), Noida, India.10.1109/ICI60088.2023.10421656 ↗
16. Coccomini, D.A., Messina, N., Gennaro, C., and Falchi, F. (2022). Combining efficientnet and vision transformers for video deepfake detection. Image Analysis and Processing–ICIAP 2022: 21st International Conference, Lecce, Italy, 23–27 May 2022, Springer. Proceedings, Part III.10.1007/978-3-031-06433-3_19 ↗
17. Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., and Gelly, S. (2020). An image is worth 16x16 words: Transformers for image recognition at scale. arXiv.
18. Diffusionfake: Enhancing generalization in deepfake detection via guided stable diffusion
19. ISTVT: Interpretable spatial-temporal video transformer for deepfake detection10.1109/TIFS.2023.3239223 ↗
20. Afchar, D., Nozick, V., Yamagishi, J., and Echizen, I. (2018, January 11–13). Mesonet: A compact facial video forgery detection network. Proceedings of the 2018 IEEE International Workshop on Information Forensics and Security (WIFS), Hong Kong.10.1109/WIFS.2018.8630761 ↗
21. Li, L., Bao, J., Zhang, T., Yang, H., Chen, D., Wen, F., and Guo, B. (2020, January 14–19). Face x-ray for more general face forgery detection. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.10.1109/CVPR42600.2020.00505 ↗
22. Liu, H., Li, X., Zhou, W., Chen, Y., He, Y., Xue, H., Zhang, W., and Yu, N. (2021, January 19–25). Spatial-phase shallow learning: Rethinking face forgery detection in frequency domain. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Virtual.10.1109/CVPR46437.2021.00083 ↗
23. Wang, J., Wu, Z., Ouyang, W., Han, X., Chen, J., Lim, S.-N., and Jiang, Y.-G. (2022, January 27–30). M2tr: Multi-modal multi-scale transformers for deepfake detection. Proceedings of the 2022 International Conference on Multimedia Retrieval, Newark, NJ, USA.10.1145/3512527.3531415 ↗
24. Detection of deepfake videos using long-distance attention10.1109/TNNLS.2022.3233063 ↗
25. Rossler, A., Cozzolino, D., Verdoliva, L., Riess, C., Thies, J., and Nießner, M. (November, January 27). Faceforensics++: Learning to detect manipulated facial images. Proceedings of the IEEE/CVF International Conference on Computer Vision, Seoul, Republic of Korea.
26. Zhao, H., Zhou, W., Chen, D., Wei, T., Zhang, W., and Yu, N. (2021, January 20–25). Multi-attentional deepfake detection. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA.10.1109/CVPR46437.2021.00222 ↗
27. A two-branch deep face replacement detection method based on multi-task learning and attention mechanism
28. Li, J., Xie, H., Li, J., Wang, Z., and Zhang, Y. (2021, January 20–25). Frequency-aware discriminative feature learning supervised by single-center loss for face forgery detection. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA.10.1109/CVPR46437.2021.00639 ↗
29. Zheng, Y., Bao, J., Chen, D., Zeng, M., and Wen, F. (2021, January 11–17). Exploring temporal coherence for more general video face forgery detection. Proceedings of the IEEE/CVF International Conference on Computer Vision, Virtual.10.1109/ICCV48922.2021.01477 ↗
30. Shao, R., Wu, T., and Liu, Z. (2023, January 17–24). Detecting and grounding multi-modal media manipulation. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, BC, Canada.10.1109/CVPR52729.2023.00667 ↗
31. Hu, T., Qi, H., Huang, Q., and Lu, Y. (2019). See better before looking closer: Weakly supervised data augmentation network for fine-grained visual classification. arXiv.
32. Sun, Z., Han, Y., Hua, Z., Ruan, N., and Jia, W. (2021, January 20–25). Improving the efficiency and robustness of deepfakes detection through precise geometric features. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA.10.1109/CVPR46437.2021.00361 ↗
33. Texture synthesis using convolutional neural networks
34. Kurakin, A., Goodfellow, I.J., and Bengio, S. (2018). Adversarial examples in the physical world. Artificial Intelligence Safety and Security, Chapman and Hall/CRC.10.1201/9781351251389-8 ↗
35. Mandatory lane-changing modelling based on a game theoretic approach in traditional and connected environments10.1093/tse/tdac035 ↗
36. Intelligent ship route planning via an A∗ search model enhanced double-deep Q-network10.1016/j.oceaneng.2025.120956 ↗
37. Li, Y., Yang, X., Sun, P., Qi, H., and Lyu, S. (2020, January 13–19). Celeb-DF: A Large-Scale Challenging Dataset for DeepFake Forensics. Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.10.1109/CVPR42600.2020.00327 ↗
38. An introduction to ROC analysis10.1016/j.patrec.2005.10.010 ↗
39. The earth mover’s distance as a metric for image retrieval10.1023/A:1026543900054 ↗
40. Hore, A., and Ziou, D. (2010, January 23–26). Image quality metrics: PSNR vs. SSIM. Proceedings of the 2010 20th International Conference on Pattern Recognition, Istanbul, Turkey.10.1109/ICPR.2010.579 ↗
41. Zhang, R., Isola, P., Efros, A.A., Shechtman, E., and Wang, O. (2018, January 18–23). The unreasonable effectiveness of deep features as a perceptual metric. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00068 ↗
42. Madry, A., Makelov, A., Schmidt, L., Tsipras, D., and Vladu, A. (2017). Towards deep learning models resistant to adversarial attacks. arXiv.
43. Nguyen, H.H., Yamagishi, J., and Echizen, I. (2019, January 12–17). Capsule-forensics: Using capsule networks to detect forged images and videos. Proceedings of the ICASSP 2019—2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Brighton, UK.10.1109/ICASSP.2019.8682602 ↗
44. Qian, Y., Yin, G., Sheng, L., Chen, Z., and Shao, J. (2020, January 23–28). Thinking in frequency: Face forgery detection by mining frequency-aware clues. Proceedings of the Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK. Proceedings, Part XII.10.1007/978-3-030-58610-2_6 ↗
45. Haliassos, A., Vougioukas, K., Petridis, S., and Pantic, M. (2021, January 20–25). Lips don’t lie: A generalizable and robust approach to face forgery detection. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA.10.1109/CVPR46437.2021.00500 ↗
46. Zhou, P., Han, X., Morariu, V.I., and Davis, L.S. (2017, January 21–26). Two-Stream Neural Networks for Tampered Face Detection. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Honolulu, HI, USA.10.1109/CVPRW.2017.229 ↗
47. Masi, I., Killekar, A., Mascarenhas, R.M., Gurudatt, S.P., and AbdAlmageed, W. (2020, January 23–28). Two-branch recurrent network for isolating deepfakes in videos. Proceedings of the Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK. Proceedings, Part VII 16.10.1007/978-3-030-58571-6_39 ↗