A Brief, In-Depth Survey of Deep Learning-Based Image Watermarking
Source abstract
Abstract
This paper presents a comprehensive survey of deep learning-based image watermarking; this technique entails the invisible embedding and extraction of watermarks within a cover image, aiming for a seamless combination of robustness and adaptability. We navigate the complex landscape of this interdisciplinary domain, linking historical foundations, current innovations, and prospective developments. Unlike existing literature, our study concentrates exclusively on image watermarking with deep learning, delivering an in-depth, yet brief analysis enriched by three fundamental contributions. First, we introduce a refined categorization, segmenting the field into embedder–extractor, deep networks for feature transformation, and hybrid methods. This taxonomy, inspired by the varied roles of deep learning across studies, is designed to infuse clarity, offering readers technical insights and directional guidance. Second, our exploration dives into representative methodologies, encapsulating the diverse research directions and inherent challenges within each category to provide a consolidated perspective. Lastly, we venture beyond established boundaries, outlining emerging frontiers and providing detailed insights into prospective research avenues.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Advanced Steganography and Watermarking Techniques · Digital Media Forensic Detection · Generative Adversarial Networks and Image Synthesis
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Department of Computer Science, University of Nebraska Omaha, Omaha, NE 68182, USA
Funding
National Science Foundation (NSF)
References (80)
Show source-provided references
1. (2023, September 01). Number of Photos (2023): Statistics, Facts, & Predictions. Available online: https://photutorial.com/photos-statistics/.
2. Cox, I., Miller, M., Bloom, J., Fridrich, J., and Kalker, T. (2007). Digital Watermarking and Steganography, Morgan Kaufmann.10.1016/B978-012372585-1.50015-2 ↗
3. Barni, M., and Bartolini, F. (2004). Watermarking Systems Engineering: Enabling Digital Assets security and Other Applications, CRC Press.10.1201/9780203913512 ↗
4. Mastorakis, S., Zhong, X., Huang, P.C., and Tourani, R. (2021, January 9–12). Dlwiot: Deep learning-based watermarking for authorized iot onboarding. Proceedings of the 2021 IEEE 18th Annual Consumer Communications & Networking Conference (CCNC), Las Vegas, NV, USA.10.1109/CCNC49032.2021.9369515 ↗
5. Survey on watermarking methods in the artificial intelligence domain and beyond10.1016/j.comcom.2022.02.023 ↗
6. Shih, F.Y. (2017). Digital Watermarking and Steganography: Fundamentals and Techniques, CRC Press.10.1201/9781315219783 ↗
7. Zhu, J., Kaplan, R., Johnson, J., and Fei-Fei, L. (2018, January 8–14). Hidden: Hiding data with deep networks. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.10.1007/978-3-030-01267-0_40 ↗
8. Baluja, S. (2017, January 4–9). Hiding images in plain sight: Deep steganography. Proceedings of the 31st Conference on Neural Information Processing Systems (NIPS 2017), Long Beach, CA, USA.
9. High-capacity multiple regions of interest watermarking for medical images10.1016/j.ins.2016.07.015 ↗
10. Quantization index modulation: A class of provably good methods for digital watermarking and information embedding10.1109/18.923725 ↗
11. Efficient general print-scanning resilient data hiding based on uniform log-polar mapping10.1109/TIFS.2009.2039604 ↗
12. Goodfellow, I., Bengio, Y., and Courville, A. (2016). Deep Learning, MIT Press.
13. A survey of deep neural network watermarking techniques10.1016/j.neucom.2021.07.051 ↗
14. Gupta, M., and Kishore, R.R. (2021, January 19–20). A survey of watermarking technique using deep neural network architecture. Proceedings of the 2021 International Conference on Computing, Communication, and Intelligent Systems (ICCCIS), Greater Noida, India.10.1109/ICCCIS51004.2021.9397226 ↗
15. Zhang, C., Lin, C., Benz, P., Chen, K., Zhang, W., and Kweon, I.S. (2021). A brief survey on deep learning based data hiding. arXiv.
16. Byrnes, O., La, W., Wang, H., Ma, C., Xue, M., and Wu, Q. (2021). Data hiding with deep learning: A survey unifying digital watermarking and steganography. arXiv.
17. Comprehensive review of watermarking techniques in deep-learning environments
18. Secure spread spectrum watermarking for multimedia10.1109/83.650120 ↗
19. Advanced encryption standard
20. Zhou, X., and Tang, X. (2011, January 22–24). Research and implementation of RSA algorithm for encryption and decryption. Proceedings of the 2011 6th International Forum on Strategic Technology, Harbin, China.
21. Peterson, W.W., and Weldon, E.J. (1972). Error-Correcting Codes, MIT Press.
22. Wicker, S.B., and Bhargava, V.K. (1999). Reed-Solomon Codes and Their Applications, John Wiley & Sons.10.1109/9780470546345 ↗
23. Johannesson, R., and Zigangirov, K.S. (2015). Fundamentals of Convolutional Coding, John Wiley & Sons.10.1002/9781119098799 ↗
24. Image quality assessment: From error visibility to structural similarity10.1109/TIP.2003.819861 ↗
25. Begum, M., and Uddin, M.S. (2020). Digital image watermarking techniques: A review. Information, 11.10.3390/info11020110 ↗
26. Fierro-Radilla, A., Nakano-Miyatake, M., Cedillo-Hernandez, M., Cleofas-Sanchez, L., and Perez-Meana, H. (2019, January 2–3). A robust image zero-watermarking using convolutional neural networks. Proceedings of the 2019 7th International Workshop on Biometrics and Forensics (IWBF), Cancun, Mexico.10.1109/IWBF.2019.8739245 ↗
27. Dong, F., Li, J., Bhatti, U.A., Liu, J., Chen, Y.W., and Li, D. (2023). Robust Zero Watermarking Algorithm for Medical Images Based on Improved NasNet-Mobile and DCT. Electronics, 12.10.3390/electronics12163444 ↗
28. Zhang, C., Benz, P., Karjauv, A., Sun, G., and Kweon, I.S. (2020, January 6–12). UDH: Universal deep hiding for steganography, watermarking, and light field messaging. Proceedings of the 34th International Conference on Neural Information Processing Systems, Online.
29. Liu, Y., Guo, M., Zhang, J., Zhu, Y., and Xie, X. (2019, January 21–25). A novel two-stage separable deep learning framework for practical blind watermarking. Proceedings of the 27th ACM International Conference on Multimedia, Nice, France.10.1145/3343031.3351025 ↗
30. JSNet: A simulation network of JPEG lossy compression and restoration for robust image watermarking against JPEG attack10.1016/j.cviu.2020.103015 ↗
31. Jia, Z., Fang, H., and Zhang, W. (2021, January 20–24). MBRS: Enhancing robustness of dnn-based watermarking by mini-batch of real and simulated jpeg compression. Proceedings of the 29th ACM International Conference on Multimedia, Online.10.1145/3474085.3475324 ↗
32. Zhang, C., Karjauv, A., Benz, P., and Kweon, I.S. (2021, January 20–24). Towards robust deep hiding under non-differentiable distortions for practical blind watermarking. Proceedings of the 29th ACM International Conference on Multimedia, Online.10.1145/3474085.3475628 ↗
33. Fang, H., Jia, Z., Ma, Z., Chang, E.C., and Zhang, W. (2022, January 10–14). PIMoG: An Effective Screen-shooting Noise-Layer Simulation for Deep-Learning-Based Watermarking Network. Proceedings of the 30th ACM International Conference on Multimedia, Lisbon, Portugal.10.1145/3503161.3548049 ↗
34. Gu, W., Chang, C.C., Bai, Y., Fan, Y., Tao, L., and Li, L. (2023). Anti-Screenshot Watermarking Algorithm for Archival Image Based on Deep Learning Model. Entropy, 25.10.3390/e25020288 ↗
35. An automated and robust image watermarking scheme based on deep neural networks10.1109/TMM.2020.3006415 ↗
36. Luo, X., Zhan, R., Chang, H., Yang, F., and Milanfar, P. (2020, January 14–19). Distortion agnostic deep watermarking. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.10.1109/CVPR42600.2020.01356 ↗
37. ReDMark: Framework for residual diffusion watermarking based on deep networks10.1016/j.eswa.2019.113157 ↗
38. Plata, M., and Syga, P. (January, January 29). Robust spatial-spread deep neural image watermarking. Proceedings of the 2020 IEEE 19th International Conference on Trust, Security and Privacy in Computing and Communications (TrustCom), Guangzhou, China.10.1109/TrustCom50675.2020.00022 ↗
39. Zhang, L., Li, W., and Ye, H. (2021, January 20–22). A blind watermarking system based on deep learning model. Proceedings of the 2021 IEEE 20th International Conference on Trust, Security and Privacy in Computing and Communications (TrustCom), Shenyang, China.10.1109/TrustCom53373.2021.00164 ↗
40. Robust image watermarking based on generative adversarial network10.23919/JCC.2020.11.012 ↗
41. Xu, H.B., Wang, R., Wei, J., and Lu, S.P. (2021). A Compact Neural Network-based Algorithm for Robust Image Watermarking. arXiv.
42. Autoencoder-convolutional neural network-based embedding and extraction model for image watermarking
43. Zhao, Y., Wang, C., Zhou, X., and Qin, Z. (2022). DARI-Mark: Deep Learning and Attention Network for Robust Image Watermarking. Mathematics, 11.10.3390/math11010209 ↗
44. Ying, Q., Zhou, H., Zeng, X., Xu, H., Qian, Z., and Zhang, X. (2022, January 16–19). Hiding Images into Images with Real-world Robustness. Proceedings of the 2022 IEEE International Conference on Image Processing (ICIP), Bordeaux, France.10.1109/ICIP46576.2022.9897931 ↗
45. Fang, H., Jia, Z., Qiu, Y., Zhang, J., Zhang, W., and Chang, E.C. (2022). De-END: Decoder-driven Watermarking Network. arXiv.10.1109/TMM.2022.3223559 ↗
46. Finding robust domain from attacks: A learning framework for blind watermarking10.1016/j.neucom.2019.01.067 ↗
47. Aloysius, N., and Geetha, M. (2017, January 6–8). A review on deep convolutional neural networks. Proceedings of the 2017 International Conference on Communication and Signal Processing (ICCSP), Chennai, India.10.1109/ICCSP.2017.8286426 ↗
48. Das, A., and Zhong, X. (2021, January 5–8). A Deep Learning-based Audio-in-Image Watermarking Scheme. Proceedings of the 2021 International Conference on Visual Communications and Image Processing (VCIP), Munich, Germany.10.1109/VCIP53242.2021.9675375 ↗
49. A robust document image watermarking scheme using deep neural network10.1007/s11042-023-15048-y ↗
50. GIFMarking: The robust watermarking for animated GIF based deep learning10.1016/j.jvcir.2021.103244 ↗
51. He, L., He, Z., Luo, T., and Song, Y. (2023). Shrinkage and Redundant Feature Elimination Network-Based Robust Image Zero-Watermarking. Symmetry, 15.10.3390/sym15050964 ↗
52. Han, B., Wang, H., Qiao, D., Xu, J., and Yan, T. (2023). Application of Zero-Watermarking Scheme Based on Swin Transformer for Securing the Metaverse Healthcare Data. IEEE J. Biomed. Health Inform., early access.10.1109/JBHI.2023.3257340 ↗
53. Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., and Guo, B. (2021, January 11–17). Swin transformer: Hierarchical vision transformer using shifted windows. Proceedings of the IEEE/CVF International Conference on Computer Vision, Montreal, BC, Canada.10.1109/ICCV48922.2021.00986 ↗
54. Vukotić, V., Chappelier, V., and Furon, T. (2020). Are classification deep neural networks good for blind image watermarking?. Entropy, 22.10.3390/e22020198 ↗
55. Fernandez, P., Sablayrolles, A., Furon, T., Jégou, H., and Douze, M. (2022, January 22–27). Watermarking images in self-supervised latent spaces. Proceedings of the ICASSP 2022–2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Singapore.10.1109/ICASSP43922.2022.9746058 ↗
56. Exploring the learning capabilities of convolutional neural networks for robust image watermarking10.1016/j.cose.2016.11.016 ↗
57. Ferdowsi, A., and Saad, W. (2018, January 20–24). Deep learning-based dynamic watermarking for secure signal authentication in the Internet of Things. Proceedings of the 2018 IEEE International Conference on Communications (ICC), IEEE, Kansas City, MO, USA.10.1109/ICC.2018.8422728 ↗
58. A novel CNN based security guaranteed image watermarking generation scenario for smart city applications10.1016/j.ins.2018.02.060 ↗
59. A fast and efficient image watermarking scheme based on deep neural network10.1016/j.patrec.2021.08.015 ↗
60. Sweldens, W. (1995, January 12–14). Lifting scheme: A new philosophy in biorthogonal wavelet constructions. Proceedings of the Wavelet Applications in Signal and Image Processing III, SPIE, San Diego, CA, USA.10.1117/12.217619 ↗
61. Robust location-secured high-definition image watermarking based on key-point detection and deep learning10.1016/j.ijleo.2021.168194 ↗
62. Bay, H., Tuytelaars, T., and Van Gool, L. (2006, January 7–13). Surf: Speeded up robust features. Proceedings of the Computer Vision–ECCV 2006: 9th European Conference on Computer Vision, Graz, Austria. Proceedings, Part I 9.10.1007/11744023_32 ↗
63. Deep learning-based robust medical image watermarking exploiting DCT and Harris hawks optimization10.1002/int.22742 ↗
64. Deep template-based watermarking10.1109/TCSVT.2020.3009349 ↗
65. Kim, W.H., Kang, J., Mun, S.M., and Hou, J.U. (2020). Convolutional neural network architecture for recovering watermark synchronization. Sensors, 20.10.3390/s20185427 ↗
66. Chen, Y.P., Fan, T.Y., and Chao, H.C. (2021). Wmnet: A lossless watermarking technique using deep learning for medical image authentication. Electronics, 10.10.3390/electronics10080932 ↗
67. Online learning: A comprehensive survey10.1016/j.neucom.2021.04.112 ↗
68. Finn, C., Abbeel, P., and Levine, S. (2017, January 6–11). Model-agnostic meta-learning for fast adaptation of deep networks. Proceedings of the International Conference on Machine Learning, PMLR, Sydney, Australia.
69. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., and Polosukhin, I. (2017, January 4–9). Attention is all you need. Proceedings of the Advances in Neural Information Processing Systems 30 (NIPS 2017), Long Beach, CA, USA.
70. Quantization Robust Pruning With Knowledge Distillation10.1109/ACCESS.2023.3257864 ↗
71. He, K., Chen, X., Xie, S., Li, Y., Dollár, P., and Girshick, R. (2022, January 18–24). Masked autoencoders are scalable vision learners. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.10.1109/CVPR52688.2022.01553 ↗
72. Chen, T., Kornblith, S., Norouzi, M., and Hinton, G. (2020, January 13–18). A simple framework for contrastive learning of visual representations. Proceedings of the International Conference on Machine Learning, PMLR, Online.
73. Caron, M., Touvron, H., Misra, I., Jégou, H., Mairal, J., Bojanowski, P., and Joulin, A. (2021, January 11–17). Emerging properties in self-supervised vision transformers. Proceedings of the IEEE/CVF International Conference on Computer Vision, Montreal, BC, Canada.10.1109/ICCV48922.2021.00951 ↗
74. Solo-learn: A library of self-supervised methods for visual representation learning
75. Bardes, A., Ponce, J., and LeCun, Y. (2021). Vicreg: Variance-invariance-covariance regularization for self-supervised learning. arXiv.
76. Uchida, Y., Nagai, Y., Sakazawa, S., and Satoh, S. (2017, January 6–9). Embedding watermarks into deep neural networks. Proceedings of the 2017 ACM on International Conference on Multimedia Retrieval, Bucharest, Romania.10.1145/3078971.3078974 ↗
77. Guo, J., and Potkonjak, M. (2018, January 5–8). Watermarking deep neural networks for embedded systems. Proceedings of the 2018 IEEE/ACM International Conference on Computer-Aided Design (ICCAD), San Diego, CA, USA.10.1145/3240765.3240862 ↗
78. Watermarking neural networks with watermarked images10.1109/TCSVT.2020.3030671 ↗
79. Jiang, H., Yang, J., Hua, G., Li, L., Wang, Y., Tu, S., and Xia, S. (2021). Fawa: Fast adversarial watermark attack. IEEE Trans. Comput., early access.10.1109/TC.2021.3065172 ↗
80. Apostolidis, K.D., and Papakostas, G.A. (2022). Digital watermarking as an adversarial attack on medical image analysis with deep learning. J. Imaging, 8.10.3390/jimaging8060155 ↗