journal article·2023-11-05Open access

GP-Net: Image Manipulation Detection and Localization via Long-Range Modeling and Transformers

Jin Peng, Chengming Liu, Haibo Pang, Xiaomeng Gao, Guozhen Cheng, Bing Hao

Applied Sciences· Volume 13· Issue 21

Abstract

With the rise of image manipulation techniques, an increasing number of individuals find it easy to manipulate image content. Undoubtedly, this presents a significant challenge to the integrity of multimedia data, thereby fueling the advancement of image forgery detection research. A majority of current methods employ convolutional neural networks (CNNs) for image manipulation localization, yielding promising outcomes. Nevertheless, CNN-based approaches possess limitations in establishing explicit long-range relationships. Consequently, addressing the image manipulation localization task necessitates a solution that adeptly builds global context while preserving a robust grasp of low-level details. In this paper, we propose GPNet to address this challenge. GPNet combines Transformer and CNN in parallel which can build global dependency and capture low-level details efficiently. Additionally, we devise an effective fusion module referred to as TcFusion, which proficiently amalgamates feature maps generated by both branches. Thorough extensive experiments conducted on diverse datasets showcase that our network outperforms prevailing state-of-the-art manipulation detection and localization approaches.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: Digital Media Forensic Detection · Adversarial Robustness in Machine Learning · Image Processing Techniques and Applications

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

School of Cyber Science and Engineering, Zhengzhou University, No. 97, Wenhua Road, Zhengzhou 450002, China

Institute of Information Technology, Information Engineering University, Zhengzhou 450002, China

Songshan Laboratory, Zhengzhou 450002, China

Funding

Nature Science Foundation of China · Key science and technology project of Henan Province · technological research projects in Henan province

References (31)

Show source-provided references
1. Razavi, A., Oord, A., and Vinyals, O. (2019, January 8–14). Generating diverse high-fidelity images with vq-vae-2. Proceedings of the Neural Information Processing Systems, Vancouver, BC, Canada.
2. Goodfellow, I., Pouget-Abadie, J., and Mirza, M. (2014, January 8–13). Generative adversarial nets. Proceedings of the Neural Information Processing Systems, Montreal, QC, Canada.
3. Park, T., Zhu, J.-Y., and Wang, O. (2020, January 6–12). Swapping autoencoder for deep image manipulation. Proceedings of the Neural Information Processing Systems, Online.
4. Dhamo, H., Farshad, A., and Laina, I. (2020, January 13–19). Semantic image manipulation using scene graphs. Proceedings of the Computer Vision and Pattern Recognition, Seattle, WA, USA.10.1109/CVPR42600.2020.00526 ↗
5. Li, B., Qi, X., and Lukasiewicz, T. (2020, January 13–19). Manigan: Text-guided image manipulation. Proceedings of the Computer Vision and Pattern Recognition, Seattle, WA, USA.10.1109/CVPR42600.2020.00790 ↗
6. Liu, B., and Pun, C.-M. (2018, January 8–14). Deep fusion network for splicing forgery localization. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.10.1007/978-3-030-11012-3_21 ↗
7. Bi, X., Wei, Y., Xiao, B., and Li, W. (2019, January 16–17). The Ringed Residual U-Net for Image Splicing Forgery Detection. Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, Long Beach, CA, USA.10.1109/CVPRW.2019.00010 ↗
8. Bi, X., Zhang, Z., and Liu, Y. (2021, January 5–9). Multi-task wavelet corrected network for image splicing forgery detection and localization. Proceedings of the 2021 IEEE International Conference on Multimedia and Expo (ICME), Shenzhen, China.10.1109/ICME51207.2021.9428466 ↗
9. Kwon, M.J., Yu, I.J., and Nam, S.H.K. (2021, January 3–8). CAT-Net: Compression artifact tracing network for detection and localization of image splicing. Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision 2021, Waikoloa, HI, USA.10.1109/WACV48630.2021.00042 ↗
10. Image splicing forgery detection combining coarse to refined convolutional neural network and adaptive clustering10.1016/j.ins.2019.09.038 ↗
11. Wu, Y., Abd-Almageed, W., and Natarajan, P. (2018, January 8–14). Busternet: Detecting copy-move image forgery with source/target localization. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.10.1007/978-3-030-01231-1_11 ↗
12. AR-Net: Adaptive attention and residual refinement network for copy-move forgery detection10.1109/TII.2020.2982705 ↗
13. CNN-Transformer Based Generative Adversarial Network for Copy-Move Source/Target Distinguishment10.1109/TCSVT.2022.3220630 ↗
14. A deep learning approach to patch-based image inpainting forensics10.1016/j.image.2018.05.015 ↗
15. Li, H., and Huang, J. (November, January 27). Localization of deep inpainting using high-pass fully convolutional network. Proceedings of the IEEE/CVF International Conference on Computer Vision, Seoul, Republic of Korea.
16. Zhou, P., Han, X.T., and Morariu, V.I. (2018, January 18–23). Learning rich features for image manipulation detection. Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00116 ↗
17. Zhou, P., Chen, B.C., and Han, X. (2020, January 7–12). Generate, segment, and refine: Towards generic manipulation segmentation. Proceedings of the AAAI Conference on Artificial Intelligence, Hilton New York Midtown, NY, USA.
18. Chen, X., Dong, C., Ji, J., and Cao, J. (2021, January 10–17). Image manipulation detection by multi-view multi-scale supervision. Proceedings of the 2021 IEEE/CVF Conference on Computer Vision, Montreal, QC, Canada.10.1109/ICCV48922.2021.01392 ↗
19. Hu, X., Zhang, Z., and Jiang, Z. (2020, January 23–28). Span: Spatial pyramid attention network forimage manipulation localization. Proceedings of the European Conference on Computer Vision, ECCV 2020, Glasgow, UK.10.1007/978-3-030-58589-1_19 ↗
20. PSCC-Net: Progressive spatio-channel correlation network for image manipulation detection and localization10.1109/TCSVT.2022.3189545 ↗
21. Vaswani, A., Shazeer, N., and Parmar, N. (2017). Advances in Neural Information Processing Systems 30 (NIPS 2017), NIPS Foundation.
22. Wang, J., Wu, Z., and Chen, J. (2022, January 18–24). ObjectFormer for Image Manipulation Detection and Localization. Proceedings of the 2022 IEEE Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.10.1109/CVPR52688.2022.00240 ↗
23. ET: Edge-enhanced Transformer for Image Splicing Detection10.1109/LSP.2022.3172617 ↗
24. Huh, M., Liu, A., Owens, A., and Efros, A.A. (2018, January 8–14). Fighting fake news: Image splice detection via learned self-consistency. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.10.1007/978-3-030-01252-6_7 ↗
25. Rich models for steganalysis of digital images10.1109/TIFS.2012.2190402 ↗
26. He, K., Zhang, X., and Ren, S. (2016, January 27–30). Deep Residual Learning for Image Recognition. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.10.1109/CVPR.2016.90 ↗
27. Dosovitskiy, A., Beyer, L., and Kolesnikov, A. (2020). An image is worth 16 × 16 words: Transformers for image recognition at scale. arXiv.
28. Dong, J., Wang, W., and Tan, T.N. (2013, January 6–10). CASIA image tampering detection evaluation database. Proceedings of the 2013 IEEE China Summit and International Conference on Signal and Information Processing, Beijing, China.10.1109/ChinaSIP.2013.6625374 ↗
29. Nist (2016, February 05). Nimble 2016 Datasets, Available online: https://www.nist.gov/itl/iad/mig/nimble-challenge-2017-evaluation.
30. Wen, B., Zhu, Y., and Subramanian, R. (2016, January 25–28). Coverage—A novel database for copy-move forgery detection. Proceedings of the 2016 IEEE International Conference on Image Processing (ICIP), Phoenix, AZ, USA.10.1109/ICIP.2016.7532339 ↗
31. Korus, P., and Huang, J. (2016, January 4–7). Evaluation of random field models in multi-modal unsupervised tampering localization. Proceedings of the IEEE International Workshop on Information Forensics and Security, Abu Dhabi, United Arab Emirates.10.1109/WIFS.2016.7823898 ↗