Digital restoration of ancient Jiangnan murals via proxy learning and structural guidance
Source abstract
Abstract
This article proposes a structure guided proxy restoration (SGPR) workflow to address the problem of the lack of real paired data in the digital restoration of ancient murals in Jiangnan. This method adopts a dual track strategy, training a generative model (ArtBooth) on a Chinese classical painting dataset through proxy learning to master artistic styles, and combining it with a non training selective feature extraction (SFE) algorithm to directly extract structural information from damaged murals. The two are integrated through an optimized multi-condition control network (OptCtrl) to achieve high fidelity repair. Experiments on simulated and real murals show that this method outperforms existing mainstream methods in both objective indicators and expert blind evaluations. This study provides a data-efficient new paradigm for intelligent restoration of cultural heritage, breaking through the limitations of traditional reliance on professional training data.
Source: openalex_inverted_index · Quality: baseline
Research context
OpenAlex topics: Generative Adversarial Networks and Image Synthesis · Aesthetic Perception and Analysis · Image Processing and 3D Reconstruction
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
References (32)
Show source-provided references
1. Huang, M. & Yang, J. A study on the conservation and restoration of murals in traditional Jiangnan architecture: Taking the practice of rectifying the screen wall and restoring murals at Chen Wangdao’s former residence as an example. Oriental Miscellany 112–120 (2022).
2. Xu, T. Evaluation of Surface Reinforcement And Crack Grouting Materials for Murals in Humid Southern Regions (Zhejiang University, 2019).
3. Sheng, X. A Preliminary Study on Novel Adhesive Agents for Murals—Targeting Humid Southern Regions (China Academy of Art, 2017).
4. Deng, X. & Yu, Y. Ancient mural inpainting via structure information guided two-branch model. Herit. Sci. 11, 131 (2023).10.1186/s40494-023-00972-x ↗
5. Lv, C., Li, Z., Shen, Y., Li, J. & Zheng, J. Separafill: two generators connected mural image restoration based on generative adversarial network with skip connect. Herit. Sci. 10, 135 (2022).10.1186/s40494-022-00771-w ↗
6. Lyu, Q., Zhao, N., Song, J., Yang, Y. & Gong, Y. Mural inpainting via two-stage generative adversarial network. npj Herit. Sci. 13, 188 (2025).10.1038/s40494-025-01710-1 ↗
7. Li, Y. et al. Efficient and explicit modelling of image hierarchies for image restoration. In Proc. 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Vancouver, BC, Canada. 18278–18289 (IEEE, 2023).10.1109/CVPR52729.2023.01753 ↗
8. Zeng, Y., Fu, J., Chao, H. & Guo, B. Aggregated contextual transformations for high-resolution image inpainting. IEEE Trans. Visualiz. Comput. Graph. 29, 3266–3280 (2023).10.1109/TVCG.2022.3156949 ↗
9. Yang, J., Ruhaiyem, N. I. R. & Zhou, C. A 3M-Hybrid model for the restoration of unique giant murals: a case study on the murals of Yongle Palace. IEEE Access 13, 38809–38824 (2025).10.1109/ACCESS.2025.3542320 ↗
10. Zheng, C., Cham, T. J., Cai, J. & Phung, D. Bridging global context interactions for high-fidelity image completion. In Proc. 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 11502–11512 (IEEE, 2022).10.1109/CVPR52688.2022.01122 ↗
11. Quan, W. et al. Image Inpainting with Local and Global Refinement. IEEE Trans. Image Process. 31, 2405–2420 (2022).10.1109/TIP.2022.3152624 ↗
12. Li, Y., Zhang, C., Li, Y., Sui, D. & Guo, M. An improved mural image restoration method based on diffusion model. Herit. Sci. 13, 347 (2025).10.1038/s40494-025-01914-5 ↗
13. Tian, B. et al. HASM: restoration of Dunhuang murals guided by spatial structure prediction combined with line drawing assistance. J. Univ. Electron. Sci. Technol. China 54, 582–591 (2025).
14. Xu, H., Kang, J. & Zhang, J. Feature-aware based digital mural restoration method. Comput. Sci. 49, 217–223 (2022).
15. Cao, J. et al. Two-stage mural image restoration using an edge-constrained attention mechanism. PLoS ONE 19 (2024).10.1371/journal.pone.0307811 ↗
16. Shi, J. et al. Image inpainting guided by image smoothing structure. J. Front. Comput. Sci. Technol. 19, 2149–2160 (2025).
17. Qiang, Z. et al. A review of deep learning-based image inpainting methods. J. Image Graph. 24, 447–463 (2019).
18. Bertalmio, M., Sapiro, G., Caselles, V. & Ballester, C. Image inpainting. In Proc. 27th Annual Conference on Computer Graphics and Interactive Techniques (417–424) (ACM Press/Addison-Wesley Publishing Co., USA, 2000).10.1145/344779.344972 ↗
19. Criminisi, A., Pérez, P. & Toyama, K. Region filling and object removal by exemplar-based image inpainting. IEEE Trans. Image Process. 13, 1200–1212 (2004).10.1109/TIP.2004.833105 ↗
20. Yu, J. et al. Generative image inpainting with contextual attention. In Proc. IEEE Conference on Computer Vision and Pattern Recognition, (5505–5514) (Computer Vision Foundation/IEEE Computer Society, Salt Lake City, UT, USA, 2018).10.1109/CVPR.2018.00577 ↗
21. Nazeri, K., Ng, E., Joseph, T., Qureshi, F. Z. & Ebrahimi, M. EdgeConnect: structure guided image inpainting using edge prediction. In Proc. 2019 IEEE/CVF International Conference on Computer Vision Workshops (3265–3274) (IEEE, Seoul, Korea (South), 2019).10.1109/ICCVW.2019.00408 ↗
22. Ho, J., Jain, A. & Abbeel, P. Denoising diffusion probabilistic models. Adv. Neural Inf. Process. Syst. 33, 6840–6851 (2020).
23. Rombach, R., Blattmann, A., Lorenz, D., Esser, P. & Ommer, B. High-resolution image synthesis with latent diffusion models. In Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition (10674–10685) (IEEE, New Orleans, LA, USA, 2022).10.1109/CVPR52688.2022.01042 ↗
24. Wang, Z., Zhang, J., Ji, Z., Bai, J. & Shan, S. CCLAP: controllable Chinese landscape painting generation via latent diffusion model. In Proc. IEEE International Conference on Multimedia and Expo (2117–2122) (IEEE, Brisbane, Australia, 2023).10.1109/ICME55011.2023.00362 ↗
25. Xu, Z. et al. MuralDiff: diffusion for ancient murals restoration on large-scale pre-training. IEEE Trans. Emerg. Top. Comput. Intell. 8, 2169–2181 (2024).10.1109/TETCI.2024.3359038 ↗
26. Ruiz, N. et al. DreamBooth: fine tuning text-to-image diffusion models for subject-driven generation. In Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition (22500–22510) (IEEE, Vancouver, BC, Canada, 2023).10.1109/CVPR52729.2023.02155 ↗
27. Gatys, L. A., Ecker, A. S. & Bethge, M. Image style transfer using convolutional neural networks. In Proc. 2016 IEEE Conference on Computer Vision and Pattern Recognition (2414–2423) (IEEE Computer Society, Las Vegas, NV, USA, 2016).10.1109/CVPR.2016.265 ↗
28. Hu, E. J. et al. LoRA: low-rank adaptation of large language models. In Proc. Tenth International Conference on Learning Representations (OpenReview.net, Virtual Event, 2022).
29. Gu, Y. et al. Mix-of-Show: decentralized low-rank adaptation for multi-concept customization of diffusion models. in Advances in Neural Information Processing Systems (eds Oh, A., Naumann, T., Globerson, A., Saenko, K., Hardt, M. & Levine, S.) Vol. 36, pp. 1–12 (NeurIPS, New Orleans, LA, USA, 2023).
30. Zhang, L., Rao, A. & Agrawala, M. Adding conditional control to text-to-image diffusion models. In Proc. IEEE/CVF International Conference on Computer Vision (3813–3824) (IEEE, Paris, France, 2023).10.1109/ICCV51070.2023.00355 ↗
31. Zhang, R., Isola, P., Efros, A. A., Shechtman, E. & Wang, O. The unreasonable effectiveness of deep features as a perceptual metric. In Proc. IEEE Conference on Computer Vision and Pattern Recognition (586-595) (Computer Vision Foundation/IEEE Computer Society, Salt Lake City, UT, USA, 2018).10.1109/CVPR.2018.00068 ↗
32. Zhuang, J., Zeng, Y., Liu, W., Yuan, C. & Chen, K. A task is worth one word: learning with task prompts for high-quality versatile image inpainting. in Computer Vision - ECCV 2024 (eds Leonardis, A., Ricci, E., Roth, S., Russakovsky, O., Sattler, T. & Varol, G.) Vol. 15116, 195–211 (Lecture Notes in Computer Science, Springer, Milan, Italy, 2024).10.1007/978-3-031-73636-0_12 ↗