Progressive generative mural image restoration based on adversarial structure learning
Source abstract
Abstract
As natural aging and human activities continue to pose significant challenges to the preservation of the structural consistency and color fidelity of the ancient murals in China’s Dunhuang Grottoes, mural restoration plays a vital role in safeguarding this invaluable cultural and artistic treasure. This paper presents a progressive image inpainting method grounded in Generative Adversarial Networks, effectively addressing the limitations of traditional methods in handling local distortion and incomplete details. Featuring a deformable convolution-based structure generator and global stabilization mechanisms, the proposed framework includes a gated gradient penalty for model stability and an adaptive loss function. Furthermore, the Mural Image Dataset was constructed using 3264 mural images, with data augmentation applied to enhance model robustness. The experimental results demonstrate 22.21% SSIM, 2.06% PSNR, and 12.69% FID improvements over existing methods, with further gains of 52.43%, 6.79%, and 13.38%, respectively, versus the baseline. These consistent metric advancements confirm the method’s effectiveness.
Source: openalex_inverted_index · Quality: baseline
Research context
OpenAlex topics: Advanced Image Processing Techniques · Generative Adversarial Networks and Image Synthesis · Image and Signal Denoising Methods
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
References (37)
Show source-provided references
1. Ren, H., Sun, K., Zhao, F. & Zhu, X. Dunhuang murals image restoration method based on generative adversarial network. Herit. Sci. 12, 39 (2024).10.1186/s40494-024-01159-8 ↗
2. Kumar, L., Kapoor, N. & Tiwari, A. Biotechnology to restoration and conservation. in Microbial Biotechnology Approaches to Monuments of Cultural Heritage(Springer Nature), 151–175 (2020).10.1007/978-981-15-3401-0_9 ↗
3. Elharrouss, O., Almaadeed, N., Al-Maadeed, S. & Akbari, Y. Image inpainting: a review. Neural Process Lett. 51, 2007–2028 (2020).10.1007/s11063-019-10163-0 ↗
4. Li, X. et al. Diffusion models for image restoration and enhancement-a comprehensive survey. Preprint at https://doi.org/10.48550/arXiv.2308.09388 (2023).10.48550/arXiv.2308.09388 ↗
5. Goodfellow, I. et al. Generative adversarial networks. Commun. ACM 63, 139–144 (2020).10.1145/3422622 ↗
6. Radford, A., Metz, L. & Chintala, S.: Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks. In: International Conference on Learning Representations (ICLR). (2016). Available at: https://openreview.net/forum?id=B1QRgziT-
7. Karras, T. A style-based generator architecture for generative adversarial networks. Preprint at https://arxiv.org/abs/1812.04948 (2019).10.1109/CVPR.2019.00453 ↗
8. Kupyn, O. et al. Deblurgan: blind motion deblurring using conditional adversarial networks. in Proc. IEEE Conference on Computer Vision and Pattern Recognition(CVPR) (2018).10.1109/CVPR.2018.00854 ↗
9. Saxena, D. & Cao, J. Generative adversarial networks (GANs) challenges, solutions, and future directions. ACM Comput. Surv. 54, 1–42 (2021).10.1145/3446374 ↗
10. Nazeri, K., Ng, E., Joseph, T., Qureshi, F. & Ebrahimi, M. Edgeconnect: structure guided image inpainting using edge prediction. in Proc. IEEE/CVF International Conference on Computer Vision Workshops(ICCVW) (2019).10.1109/ICCVW.2019.00408 ↗
11. Wan, Z., Zhang, J., Chen, D. & Liao, J. High-fidelity pluralistic image completion with transformers. in Proc. IEEE/CVF International Conference on Computer Vision(ICCV) (2021).10.1109/ICCV48922.2021.00465 ↗
12. Yu, J. et al. Free-form image inpainting with gated convolution. in Proc. IEEE/CVF International Conference on Computer Vision(ICCV) (2019).10.1109/ICCV.2019.00457 ↗
13. Zhu, S. et al. Text Image inpainting via global structure-guided diffusion models. in Proc. AAAI Conference on Artificial Intelligence(AAAI) (2024).10.1609/aaai.v38i7.28612 ↗
14. Yu, Tao, et al. Region normalization for image inpainting. in Proc. AAAI Conference on Artificial Intelligence(AAAI), Vol. 34 (2020).10.1609/aaai.v34i07.6967 ↗
15. Jabbar, A., Li, X. & Omar, B. A survey on generative adversarial networks: variants, applications, and training. ACM Comput. Surv. 54, 1–49 (2021).10.1145/3463475 ↗
16. Arjovsky, M., Chintala, S. & Bottou, L. Wasserstein generative adversarial networks. in International Conference on Machine Learning(ICML), PMLR (2017).
17. Li, Z. et al. A systematic survey of regularization and normalization in GANs. ACM Comput. Surv. 55, 1–37 (2023).
18. Gulrajani, I., Ahmed, F., Arjovsky, M., Dumoulin, V. & Courville, A. C. Improved training of wasserstein gans. Adv. Neural Inform. Process. Syst. 30 (2017).
19. Tirel, L., Ali, A. M. & Hashim, H. A. Novel hybrid integrated Pix2Pix and WGAN model with gradient penalty for binary images denoising. Syst. Soft Comput. 6, 200122 (2024).10.1016/j.sasc.2024.200122 ↗
20. Miyato, T., Kataoka, T., Koyama, M. & Yoshida, Y.: Spectral Normalization for Generative Adversarial Networks. In: International Conference on Learning Representations (ICLR). (2018). Available at: https://openreview.net/forum?id=B1QRgziT- (Accessed: 2023-10-27).
21. Wang, G., Yang, Z., Sun, H., Zhou, Q. & Yang, Z. A. C. - SNGAN: multi-class data augmentation for damage detection of conveyor belt surface using improved ACGAN. Measurement 224, 113814 (2024).10.1016/j.measurement.2023.113814 ↗
22. Singh, R. et al. CVAD-GAN: constrained video anomaly detection via generative adversarial network. Image Vis. Comput. 143, 104950 (2024).10.1016/j.imavis.2024.104950 ↗
23. Quan, D. et al. WVD-GAN: a Wigner-Ville distribution enhancement method based on generative adversarial network. IET Radar Sonar Nav. 18, 849–865 (2024).10.1049/rsn2.12532 ↗
24. Yang, J. & Ruhaiyem, N. I. R. Review of deep learning-based image inpainting techniques. IEEE Access 12, 138441–138482 (2024).10.1109/ACCESS.2024.3461782 ↗
25. Xiong, Y. et al. Efficient deformable convnets: rethinking dynamic and sparse operator for vision applications. in Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR) (2024).10.1109/CVPR52733.2024.00540 ↗
26. Wang, S. et al. Exploring DCN-like architecture for fast image generation with arbitrary resolution. Advances in Neural Information Processing Systems 37, 87959-87977 (2024).
27. Jin, Z., Qiu, Y., Zhang, K., Li, H. & Luo, W. MB-TaylorFormer V2: improved multi-branch linear transformer expanded by Taylor formula for image restoration. IEEE Trans. Pattern Anal. Mach. Intell. 47, 5990–6005 (2025).10.1109/TPAMI.2025.3559891 ↗
28. Zhang, K. et al. Deep dense multi-scale network for snow removal using semantic and depth priors. IEEE Trans. Image Process. 30, 7419–7431 (2021).10.1109/TIP.2021.3104166 ↗
29. Zhang, K. et al. Enhanced spatio-temporal interaction learning for video deraining: faster and better. IEEE Trans. Pattern Anal. Mach. Intell. 45, 1287–1293 (2022).10.1109/TPAMI.2022.3148707 ↗
30. Wang, T. et al. Gridformer: residual dense transformer with grid structure for image restoration in adverse weather conditions. Int. J. Comput. Vis. 132, 4541–4563 (2024).10.1007/s11263-024-02056-0 ↗
31. Guo, Z., Chen, Z., Yu, T., Chen, J. & Liu, S. Progressive image inpainting with full-resolution residual network. in Proc. 27th ACM International Conference on Multimedia(ACM MM) (2019).10.1145/3343031.3351022 ↗
32. Wei, G. et al. A three-stage GAN model based on edge and color prediction for image outpainting. Expert Syst. Appl. 214, 119136 (2023).10.1016/j.eswa.2022.119136 ↗
33. Karnewar, A. & Wang, O. Msg-gan: Multi-scale gradients for generative adversarial networks. in Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR) (2020).10.1109/CVPR42600.2020.00782 ↗
34. Cai, J., Li, C., Tao, X. & Tai, Y. W. Image multi-inpainting via progressive generative adversarial networks. in Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR) (2022).10.1109/CVPRW56347.2022.00111 ↗
35. Li, L. et al. Line drawing guided progressive inpainting of mural damages. Preprint at https://arxiv.org/abs/2211.06649 (2022).
36. Ho, J., Jain, A. & Abbeel, P. Denoising diffusion probabilistic models. Adv. Neural Inf. Process. Syst. 33, 6840–6851 (2020).
37. He, C. et al. Diffusion models in low-level vision: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence (IEEE, 2025).10.1109/TPAMI.2025.3545047 ↗