Multi-Domain Perception Transformer for Generalized Forgery Image Detection
Source abstract
Abstract
With the rapid advancement of generative AI (AIGC) technology, synthetic images are increasingly approaching real pictures in terms of resolution and semantic consistency. Traditional detection methods face numerous challenges, such as insufficient cross-modal generalization capabilities and difficulty in identifying hidden generative traces. Existing solutions primarily design feature extractors for single generative models, struggling to address the complexity of multimodal forgeries. Therefore, we propose a multi-domain feature fusion Transformer network that integrates spatial, frequency, and wavelet transform features and introduce a cross-domain feature fusion module (CDAF) to detect subtle forgery traces in deepfake images. This model demonstrates superior detection performance on current forged images generated by generative adversarial networks (GANs) and diffusion models while exhibiting enhanced robustness.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Digital Media Forensic Detection · Generative Adversarial Networks and Image Synthesis · Advanced Image Processing Techniques
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Department of Computer Engineering, Gachon University, 1342 Seongnamdaero, Sujeong-gu, Seongnam-si 13120, Republic of Korea
References (40)
Show source-provided references
1. Generative adversarial networks10.1145/3422622 ↗
2. Kingma, D.P., and Welling, M. (2013). Auto-encoding variational bayes. arXiv.
3. Rombach, R., Blattmann, A., Lorenz, D., Esser, P., and Ommer, B. (2022, January 19–20). High-resolution image synthesis with latent diffusion models. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.10.1109/CVPR52688.2022.01042 ↗
4. Luo, Z., Chen, D., Zhang, Y., Huang, Y., Wang, L., Shen, Y., Zhao, D., Zhou, J., and Tan, T. (2023). Videofusion: Decomposed diffusion models for high-quality video generation. arXiv.10.1109/CVPR52729.2023.10308948 ↗
5. MiniNet: A concise CNN for image forgery detection10.1007/s12530-022-09446-0 ↗
6. Aggarwal, G., Srivastava, A.K., Jhajharia, K., Sharma, N.V., and Singh, G. (2023, January 1–3). Detection of deep fake images using convolutional neural networks. Proceedings of the 2023 3rd International Conference on Technological Advancements in Computational Sciences (ICTACS), Tashkent, Uzbekistan.10.1109/ICTACS59847.2023.10389973 ↗
7. Fake and propaganda images detection using automated adaptive gaining sharing knowledge algorithm with DenseNet12110.1007/s12652-024-04829-4 ↗
8. Detecting fake images by identifying potential texture difference10.1016/j.future.2021.06.043 ↗
9. Fake face images detection and identification of celebrities based on semantic segmentation10.1109/LSP.2022.3205481 ↗
10. Frank, J., Eisenhofer, T., Schönherr, L., Fischer, A., Kolossa, D., and Holz, T. (2020, January 12–18). Leveraging frequency analysis for deep fake image recognition. Proceedings of the International Conference on Machine Learning, Vienna, Austria.
11. Gu, Q., Chen, S., Yao, T., Chen, Y., Ding, S., and Yi, R. (March, January 22). Exploiting fine-grained face forgery clues via progressive enhancement learning. Proceedings of the AAAI Conference on Artificial Intelligence, Virtual.
12. Jeong, Y., Kim, D., Ro, Y., and Choi, J. (March, January 22). Frepgan: Robust deepfake detection using frequency-level perturbations. Proceedings of the AAAI Conference on Artificial Intelligence, Virtual.
13. Shiohara, K., and Yamasaki, T. (2022, January 18–24). Detecting deepfakes with self-blended images. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.10.1109/CVPR52688.2022.01816 ↗
14. Real face foundation representation learning for generalized deepfake detection10.1016/j.patcog.2024.111299 ↗
15. Li, Y., Chang, M.-C., and Lyu, S. (2018). In ictu oculi: Exposing ai generated fake face videos by detecting eye blinking. arXiv.10.1109/WIFS.2018.8630787 ↗
16. Yang, X., Li, Y., and Lyu, S. (2019, January 12–17). Exposing deep fakes using inconsistent head poses. Proceedings of the ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Brighton, UK.10.1109/ICASSP.2019.8683164 ↗
17. Rossler, A., Cozzolino, D., Verdoliva, L., Riess, C., Thies, J., and Nießner, M. (November, January 27). Faceforensics++: Learning to detect manipulated facial images. Proceedings of the IEEE/CVF International Conference on Computer Vision, Seoul, Republic of Korea.
18. Chollet, F. (2017, January 21–26). Xception: Deep learning with depthwise separable convolutions. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.10.1109/CVPR.2017.195 ↗
19. He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 11–14). Identity mappings in deep residual networks. Proceedings of the European Conference on Computer Vision, Amsterdam, The Netherlands.10.1007/978-3-319-46493-0_38 ↗
20. Nguyen, H.H., Yamagishi, J., and Echizen, I. (2022). Capsule-forensics networks for deepfake detection. Handbook of Digital Face Manipulation and Detection: From DeepFakes to Morphing Attacks, Springer International Publishing.10.1007/978-3-030-87664-7_13 ↗
21. Chen, L., Zhang, Y., Song, Y., Liu, L., and Wang, J. (2022, January 19–20). Self-supervised learning of adversarial example: Towards good generalizations for deepfake detection. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.10.1109/CVPR52688.2022.01815 ↗
22. Ren, H., Yan, A., Ren, X., Ye, P.-G., Gao, C.-Z., Zhou, Z., and Li, J. (2023). Ganfinger: Gan-based fingerprint generation for deep neural network ownership verification. arXiv.
23. Zhao, H., Zhou, W., Chen, D., Wei, T., Zhang, W., and Yu, N. (2021, January 20–25). Multi-attentional deepfake detection. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA.10.1109/CVPR46437.2021.00222 ↗
24. Kim, M., Tariq, S., and Woo, S.S. (2021, January 20–25). Fretal: Generalizing deepfake detection using knowledge distillation and representation learning. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA.10.1109/CVPRW53098.2021.00111 ↗
25. Aneja, S., and Nießner, M. (2020). Generalized zero and few-shot transfer for facial forgery detection. arXiv.
26. Wang, C., and Deng, W. (2021, January 20–25). Representative forgery mining for fake face detection. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA.10.1109/CVPR46437.2021.01468 ↗
27. Dosovitskiy, A. (2020). An image is worth 16 × 16 words: Transformers for image recognition at scale. arXiv.
28. Wodajo, D., and Atnafu, S. (2021). Deepfake video detection using convolutional vision transformer. arXiv.
29. Heo, Y.-J., Choi, Y.-J., Lee, Y.-W., and Kim, B.-G. (2021). Deepfake detection scheme based on vision transformer and distillation. arXiv.
30. Park, J. (2024, January 9). Using the Swin-Transformer for Real & Fake Data Recognition in PC-Model. Proceedings of the 2024 IEEE Integrated STEM Education Conference (ISEC), Princeton, NJ, USA.10.1109/ISEC61299.2024.10664887 ↗
31. Deepfake detection algorithm based on improved vision transformer10.1007/s10489-022-03867-9 ↗
32. Dolhansky, B., Howes, R., Pflaum, B., Baram, N., and Ferrer, C.C. (2019). The deepfake detection challenge (dfdc) preview dataset. arXiv.
33. Li, Y., Yang, X., Sun, P., Qi, H., and Lyu, S. (2020, January 13–19). Celeb-df: A large-scale challenging dataset for deepfake forensics. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.10.1109/CVPR42600.2020.00327 ↗
34. Wang, S.-Y., Wang, O., Zhang, R., Owens, A., and Efros, A.A. (2020, January 13–19). CNN-generated images are surprisingly easy to spot… for now. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.10.1109/CVPR42600.2020.00872 ↗
35. Liu, L., Ren, Y., Lin, Z., and Zhao, Z. (2022). Pseudo numerical methods for diffusion models on manifolds. arXiv.
36. Diffusion models beat gans on image synthesis
37. Ramesh, A., Pavlov, M., Goh, G., Gray, S., Voss, C., Radford, A., Chen, M., and Sutskever, I. (2021, January 18–24). Zero-shot text-to-image generation. Proceedings of the International Conference on Machine Learning, Virtual.
38. Gu, S., Chen, D., Bao, J., Wen, F., Zhang, B., Chen, D., Yuan, L., and Guo, B. (2022, January 19–20). Vector quantized diffusion model for text-to-image synthesis. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.10.1109/CVPR52688.2022.01043 ↗
39. TAN-GFD: Generalizing face forgery detection based on texture information and adaptive noise mining10.1007/s10489-023-04462-2 ↗
40. Peng, S., Zhang, T., Gao, L., Zhu, X., Zhang, H., Pang, K., and Lei, Z. (2025, January 27–31). Wmamba: Wavelet-based mamba for face forgery detection. Proceedings of the 33rd ACM International Conference on Multimedia, Dublin, Ireland.10.1145/3746027.3755592 ↗