Mural inpainting for structural distortion and color deviation: edge-constrained diffusion and adversarial optimization strategy
Source abstract
Abstract
Murals are an essential component of cultural heritage; however, their digital restoration has long been constrained by severe structural fragmentation and chromatic degradation caused by material aging, environmental erosion, and historical damage. To address these challenges, this paper proposes a heritage-oriented digital mural restoration framework. The method first introduces an attention-guided edge prediction module to provide strong spatial supervision for subsequent generation. Then, under structural priors, a diffusion-based image inpainting framework is constructed and combined with a non-local color optimization module to ensure overall layout consistency, line continuity, and global color consistency. Experimental results on multiple mural datasets demonstrate that the proposed method significantly outperforms state-of-the-art approaches in terms of structural continuity and color consistency. Overall, the framework provides a controllable and interpretable technical pathway for noninvasive digital mural restoration, effectively supporting the digital documentation, scientific analysis, and long-term preservation of mural heritage.
Source: openalex_inverted_index · Quality: baseline
Research context
OpenAlex topics: Generative Adversarial Networks and Image Synthesis · Public Spaces through Art · Image Enhancement Techniques
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
References (34)
Show source-provided references
1. Deng, X. & Yu, Y. Ancient mural inpainting via structure information guided two-branch model. Herit. Sci. 11, 131 (2023).10.1186/s40494-023-00972-x ↗
2. Lyu, Q., Zhao, N., Song, J., Yang, Y. & Gong, Y. Mural inpainting via two-stage generative adversarial network. Npj Herit. Sci. 13, 188 (2025).10.1038/s40494-025-01710-1 ↗
3. Zhang, J., Bai, S., Zeng, X., Liu, K. & Yuan, H. Supporting historic mural image inpainting by using coordinate attention aggregated transformations with U-Net-based discriminator. Npj Herit. Sci. 13, 1–12 (2025).
4. Xu, Z., Zhang, C. & Wu, Y. Digital inpainting of mural images based on DC-CycleGAN. Herit. Sci. 11, 169 (2023).10.1186/s40494-023-01015-1 ↗
5. Mol, V. R. & Maheswari, P. U. The digital reconstruction of degraded ancient temple murals using dynamic mask generation and an extended exemplar-based region-filling algorithm. Herit. Sci. 9, 1–18 (2021).10.1186/s40494-021-00604-2 ↗
6. Dubey, S. R. & Singh, S. K. Transformer-based generative adversarial networks in computer vision: a comprehensive survey. IEEE Trans. Artif. Intell. 5, 4851–4867 (2024).10.1109/TAI.2024.3404910 ↗
7. Qin, Z., Zeng, Q., Zong, Y. & Xu, F. Image inpainting based on deep learning: a review. Displays 69, 102028 (2021).10.1016/j.displa.2021.102028 ↗
8. Wang, Q., Chen, Y., Zhang, N. & Gu, Y. Medical image inpainting with edge and structure priors. Measurement 185, 110027 (2021).10.1016/j.measurement.2021.110027 ↗
9. Ren, H., Zhang, C., Li, Z. & Su, Z. EdgeConnect model based on Fourier convolution for Dunhuang mural costume image restoration. Npj Herit. Sci. 13, 592 (2025).10.1038/s40494-025-02138-3 ↗
10. Wang, Z., Zhao, H., Zhao, C., Zhang, T. & Qiao, S. An enhanced edge information network for image denoising via feature multi-modulation attention. IEEE Access 12, 136946–136961 (2024).10.1109/ACCESS.2024.3415420 ↗
11. Li, Z. et al. Image inpainting based on CNN-transformer framework via structure and texture restoration. Appl. Soft Comput. 170, 112671 (2025).10.1016/j.asoc.2024.112671 ↗
12. Liao, L., Xiao, J., Wang, Z., Lin, C.-W. & Satoh, S. Guidance and evaluation: Semantic-aware image inpainting for mixed scenes. In Proc. European Conference on Computer Vision, 683–700 (Springer, 2020).10.1007/978-3-030-58583-9_41 ↗
13. Hu, Q. et al. SGRGAN: Sketch-guided restoration for traditional Chinese landscape paintings. Herit. Sci. 12, 163 (2024).10.1186/s40494-024-01253-x ↗
14. Shamsolmoali, P., Zareapoor, M. & Granger, E. TransInpaint: transformer-based image inpainting with context adaptation. In Proc. IEEE/CVF International Conference on Computer Vision Workshops, 849–858 (IEEE, 2023).10.1109/ICCVW60793.2023.00092 ↗
15. Dosovitskiy, A. et al. An image is worth 16×16 words: transformers for image recognition at scale. In Proc. International Conference on Learning Representations (ICLR), Virtual Event, Austria, (ICLR, 2021).
16. Hayat, M. & Aramvith, S. Transformer’s role in brain MRI: a scoping review. IEEE Access 12, 108876–108896 (2024).10.1109/ACCESS.2024.3434714 ↗
17. Zhang, C., Yang, W., Li, X. & Han, H. MMGInpainting: multi-modality guided image inpainting based on diffusion models. IEEE Trans. Multimed. 26, 8811–8823 (2024).10.1109/TMM.2024.3382484 ↗
18. Reinhardt, C. N., Anderson, C. & Schenck, E. V2IR-CnLDM: a generative visible-to-infrared image translation using ControlNet-guided conditional latent diffusion model. Opt. Eng. 64, 092206 (2025).10.1117/1.OE.64.9.092206 ↗
19. Zhao, F., Ren, H., Su, Z., Zhu, X. & Zhang, C. Diffusion-based heterogeneous network for ancient mural restoration. Npj Herit. Sci. 13, 206 (2025).10.1038/s40494-025-01719-6 ↗
20. Corneanu, C., Gadde, R. & Martinez, A. M. LatentPaint: image inpainting in latent space with diffusion models. In Proc. IEEE/CVF Winter Conference on Applications of Computer Vision, 4334–4343 (IEEE, 2024).10.1109/WACV57701.2024.00428 ↗
21. Han, P. et al. DiffuMural: a diffusion model for Dunhuang murals restoration via multi-scale convergence and cooperative guidance. Npj Herit. Sci. 14, 44 (2026).10.1038/s40494-025-02274-w ↗
22. Rezvani, S., Fateh, M. & Khosravi, H. ABANet: attention boundary-aware network for image segmentation. Expert Syst. 41, e13625 (2024).10.1111/exsy.13625 ↗
23. Nazeri, K., Ng, E., Joseph, T., Qureshi, F. & Ebrahimi, M. EdgeConnect: structure guided image inpainting using edge prediction. In Proc. IEEE/CVF International Conference on Computer Vision Workshops (IEEE, 2019).10.1109/ICCVW.2019.00408 ↗
24. Li, L. et al. Line drawing-guided progressive inpainting for mural damage. ACM J. Comput. Cult. Herit. 18, 1–20 (2025).
25. Li, C., He, K., Liu, K. & Ma, X. Image inpainting using two-stage loss function and global and local Markovian discriminators. Sensors 20, 6193 (2020).10.3390/s20216193 ↗
26. Wang, N., Wang, W., Hu, W., Fenster, A. & Li, S. Damage sensitive and original restoration driven Thanka mural inpainting. In Proc. Chinese Conference on Pattern Recognition and Computer Vision (PRCV), 142–154 (Springer, 2020).10.1007/978-3-030-60633-6_12 ↗
27. Du, Y., Liu, H. & Chen, S. Collaborative contrastive learning-based generative model for image inpainting. IEEE Access 10, 106641–106654 (2022).10.1109/ACCESS.2022.3211961 ↗
28. Hsieh, T. et al. Text and edge guided Thangka image inpainting with diffusion model. In Proc. IEEE International Conference on Multimedia and Expo (ICME), 1–10 (IEEE, 2024).10.1109/ICME57554.2024.10687698 ↗
29. Wang, D., Xie, C., Liu, S., Niu, Z. & Zuo, W. Image inpainting with edge-guided learnable bidirectional attention maps. Preprint at https://doi.org/10.48550/arXiv.2104.12087 (2021).10.48550/arXiv.2104.12087 ↗
30. Bakurov, I., Buzzelli, M., Schettini, R., Castelli, M. & Vanneschi, L. Structural similarity index (SSIM) revisited: a data-driven approach. Expert Syst. Appl. 189, 116087 (2022).10.1016/j.eswa.2021.116087 ↗
31. Poobathy, D. & Chezian, R. M. Edge detection operators: peak signal to noise ratio based comparison. Int. J. Image Graph. Signal Process. 10, 55–61 (2014).
32. Yu, Y., Zhang, W. & Deng, Y. Frechet inception distance (FID) for evaluating GANs. China Univ. Min. Technol. Beijing Grad. Sch. 3, 11 (2021).
33. Gómez-Polo, C. et al. Comparison of the CIELab and CIEDE2000 color difference formulas. J. Prosthet. Dent. 115, 65–70 (2016).10.1016/j.prosdent.2015.07.001 ↗
34. Hayat, M. & Aramvith, S. Superpixel-guided graph-attention boundary GAN for adaptive feature refinement in scribble-supervised medical image segmentation. IEEE Access 13, 196654–196668 (2025).10.1109/ACCESS.2025.3634156 ↗