DCADif: decoupled conditional adaptive time-dynamic fusion diffusion inpainting of traditional Chinese mural paintings
Source abstract
Abstract
Digital inpainting of traditional Chinese murals is challenged by the difficulty of disentangling intricate structures from unique artistic styles, often leading to artifacts. To address this, we propose DCADif, a novel diffusion model for high-fidelity mural restoration. DCADif’s core innovation is a Decoupled Conditional Encoder that uses parallel pathways a pre-trained CLIP for structural line art and a new SwinStyle Encoder for stylistic features to achieve independent control. Furthermore, a Time-Adaptive Feature Fusion (TAFF) module dynamically adjusts the influence of these features during denoising, prioritizing structure in early stages and style in later ones, mimicking an expert’s coarse-to-fine workflow. Evaluated on our new large-scale MuralVerse-S dataset, DCADif significantly outperforms state-of-the-art methods across all degradation levels. It establishes a new benchmark for digital cultural heritage preservation by effectively balancing structural accuracy with artistic authenticity. The dataset and code are publicly available.The dataset and code are available at https://github.com/LPDLG/DCADif .
Source: openalex_inverted_index · Quality: baseline
Research context
OpenAlex topics: Generative Adversarial Networks and Image Synthesis · Aesthetic Perception and Analysis · Computer Graphics and Visualization Techniques
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
References (38)
Show source-provided references
1. He, K. et al. Masked autoencoders are scalable vision learners. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 16000–16009 (2022).10.1109/CVPR52688.2022.01553 ↗
2. Yu, J. et al. Free-form image inpainting with gated convolution. In Proceedings of the IEEE/CVF international conference on computer vision, 4471–4480 (2019).10.1109/ICCV.2019.00457 ↗
3. Rombach, R., Blattmann, A., Lorenz, D., Esser, P. & Ommer, B. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 10684–10695 (2022).10.1109/CVPR52688.2022.01042 ↗
4. Lugmayr, A. et al. Repaint: Inpainting using denoising diffusion probabilistic models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 11461–11471 (2022).10.1109/CVPR52688.2022.01117 ↗
5. Zeng, Y., Fu, Z., Chao, H.-Y. & Zhang, L. CR-Fill: Generative image inpainting with auxiliary contextual reconstruction. In European Conference on Computer Vision (ECCV), 151–167 (Springer, 2022).
6. Liu, Z. et al. Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF international conference on computer vision, 10012–10022 (2021).10.1109/ICCV48922.2021.00986 ↗
7. Zhu, L. et al. Enlightening low-light images with dynamic guidance for context enrichment. IEEE Transactions on Circuits and Systems for Video Technology 32, 5068–5079 (2022).10.1109/TCSVT.2022.3146731 ↗
8. Barnes, C., Shechtman, E., Finkelstein, A. & Goldman, D. B. Patchmatch: A randomized correspondence algorithm for structural image editing. ACM Trans. Graph. 28, 24 (2009).10.1145/1531326.1531330 ↗
9. Suvorov, R. et al. Resolution-robust large mask inpainting with fourier convolutions. In Proceedings of the IEEE/CVF winter conference on applications of computer vision, 2149–2159 (2022).10.1109/WACV51458.2022.00323 ↗
10. Zhang, L., Rao, A. & Agrawala, M. Adding conditional control to text-to-image diffusion models. In Proceedings of the IEEE/CVF international conference on computer vision, 3836–3847 (2023).10.1109/ICCV51070.2023.00355 ↗
11. Tumanyan, N., Geyer, M., Bagon, S. & Dekel, T. Plug-and-play diffusion features for text-driven image-to-image translation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 1921–1930 (2023).10.1109/CVPR52729.2023.00191 ↗
12. Brooks, T., Holynski, A. & Efros, A. A. Instructpix2pix: Learning to follow image editing instructions. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 18392–18402 (2023).10.1109/CVPR52729.2023.01764 ↗
13. Couairon, G., Verbeek, J., Schwenk, H. & Cord, M. Diffedit: Diffusion-based semantic image editing with mask guidance. arXiv preprint arXiv:2210.11427 (2022).
14. Radford, A. et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, 8748–8763 (PmLR, 2021).
15. Avrahami, O., Lischinski, D. & Fried, O. Blended diffusion for text-driven editing of natural images. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 18208–18218 (2022).10.1109/CVPR52688.2022.01767 ↗
16. Sharma, D., Dhiman, C. & Kumar, D. Control with style: Style embedding-based variational autoencoder for controlled stylized caption generation framework. IEEE Transactions on Cognitive and Developmental Systems (2024).10.1109/TCDS.2024.3405573 ↗
17. Sharma, D., Dhiman, C. & Kumar, D. Evolution of visual data captioning methods, datasets, and evaluation metrics: A comprehensive survey. Expert Systems with Applications 221, 119773 (2023).10.1016/j.eswa.2023.119773 ↗
18. Sharma, D., Dhiman, C. & Kumar, D. A review of stylized image captioning techniques, evaluation parameters, and datasets. In 2022 4th International Conference on Artificial Intelligence and Speech Technology (AIST), 1–5 (IEEE, 2022).10.1109/AIST55798.2022.10064842 ↗
19. Gatys, L. A., Ecker, A. S. & Bethge, M. Image style transfer using convolutional neural networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, 2414–2423 (2016).10.1109/CVPR.2016.265 ↗
20. Deng, Y. et al. Stytr2: Image style transfer with transformers. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 11326–11336 (2022).10.1109/CVPR52688.2022.01104 ↗
21. Sharma, D., Dhiman, C. & Kumar, D. Unma-capsumt: unified and multi-head attention-driven caption summarization transformer. arXiv preprint arXiv:2412.11836 (2024).10.1016/j.jvcir.2025.104600 ↗
22. Wang, Z. et al. Uformer: A general u-shaped transformer for image restoration. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 17683–17693 (2022).10.1109/CVPR52688.2022.01716 ↗
23. Rezvani, S., Fateh, M. & Khosravi, H. Abanet: attention boundary-aware network for image segmentation. Expert Systems 41, e13625 (2024).10.1111/exsy.13625 ↗
24. Rezvani, S., Fateh, M., Jalali, Y. & Fateh, A. Fusionlungnet: multi-scale fusion convolution with refinement network for lung ct image segmentation. Biomedical Signal Processing and Control 107, 107858 (2025).10.1016/j.bspc.2025.107858 ↗
25. Vaswani, A. et al. Attention is all you need. In Advances in Neural Information Processing Systems (NeurIPS), vol. 30 (2017).
26. Wang, Z. & Bovik, A. C. Mean squared error: Love it or leave it? a new look at signal fidelity measures. IEEE signal processing magazine 26, 98–117 (2009).10.1109/MSP.2008.930649 ↗
27. Wang, Z., Bovik, A. C., Sheikh, H. R. & Simoncelli, E. P. Image quality assessment: from error visibility to structural similarity. IEEE transactions on image processing 13, 600–612 (2004).10.1109/TIP.2003.819861 ↗
28. Zhang, R., Isola, P., Efros, A. A., Shechtman, E. & Wang, O. The unreasonable effectiveness of deep features as a perceptual metric. In Proceedings of the IEEE conference on computer vision and pattern recognition, 586–595 (2018).10.1109/CVPR.2018.00068 ↗
29. Yang, K.-F. et al. Adair: Adaptive all-in-one image restoration. In European Conference on Computer Vision, 435–452 (Springer, 2022).
30. He, H.-J., Chen, Q.-G., Guo, M.-H., Ouyang, B.-W. & Yan, D.-M. Coupled texture-structure-aware decomposition and guided restoration for image inpainting. IEEE Transactions on Multimedia (2023).
31. Li, J., Wang, N., Zhang, L., Du, B. & Tao, D. Recurrent feature reasoning for image inpainting. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 7760–7768 (2020).10.1109/CVPR42600.2020.00778 ↗
32. Zhang, L., Zhu, Y., Li, X., Luan, F. & Li, T.-T. Diffusionart: Training-free line art colorization via convolutional adaptive instance normalization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 19523–19532 (2023).
33. Li, W. et al. Mask-aware transformer for large-hole image inpainting. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 11172–11181 (2022).10.1109/CVPR52688.2022.01049 ↗
34. Potlapalli, V., Kulkarni, K. A., Nagaraj, T. & Balasubramanian, V. N. Promptir: Prompting for all-in-one blind image restoration. In NeurIPS (2023).
35. Chen, Z. et al. Strdiffusion: Structured momentum diffusion for accelerated image inpainting. In CVPR (2024).
36. Song, Y. et al. Score-based generative modeling through stochastic differential equations. In ICLR (2021).
37. Lugmayr, A., Danelljan, M., Romero, A., Yu, F. & Timofte, R. Repaint: Inpainting using denoising diffusion probabilistic models. In CVPR (2022).10.1109/CVPR52688.2022.01117 ↗
38. Zhu, L., Zeng, X., Chen, B., Liu, S. & Li, P. Leveraging diffusion knowledge for generative image compression with fractal frequency-aware band learning. arXiv preprint arXiv:2503.11321 (2025).