LFMDiff: generation of Chinese traditional landscape paintings based on diffusion model
Source abstract
Abstract
Abstract To address the challenges in generating traditional Chinese landscape paintings—such as complex hierarchical composition, difficulty in capturing brushwork details, and insufficient stylistic coherence—this study proposes LFMDiff, a diffusion-based generative framework. The model employs a hierarchical local LoRA mechanism to preserve global composition while enhancing local hierarchical and textural details. It further introduces dynamic rank and dynamic alpha mechanisms to adaptively allocate model capacity between easily learned and complex features. In addition, an adversarial adaptive flow and a multi-scale maximum mean discrepancy constraint are integrated to improve latent space alignment and stylistic consistency across multiple granularities. A dedicated dataset, CTLPD, is also constructed for model training and evaluation. Experimental results demonstrate that LFMDiff achieves superior performance in global composition, brushstroke hierarchy, and fine-grained detail generation, offering new insights into the integration of artificial intelligence and traditional painting styles.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Generative Adversarial Networks and Image Synthesis · Aesthetic Perception and Analysis · Computer Graphics and Visualization Techniques
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
References (37)
Show source-provided references
1. Duan, X. A study of chinese and western landscape painting from the comparative perspective. Front. Art Res. 5, 1–7 (2023).
2. Mim, N. J., Nandi, D., Khan, S. S., Dey, A. & Ahmed, S. I. In-between visuals and visible: the impacts of text-to-image generative AI tools on digital image-making practices in the global south. In Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems, 1–18 (Association for Computing Machinery, 2024).10.1145/3613904.3641951 ↗
3. Windhager, F. & Mayr, E. Digital humanities and distributed cognition: from a lack of theory to its visual augmentation. J. Cult. Anal. 7, 1–4 (2024).
4. Li, W., Tang, Z., Zhang, P. & Wang, J. Collaborative data behaviors in digital humanities research teams. In Proceedings of the 24th ACM/IEEE Joint Conference on Digital Libraries, 1–5 (2024).10.1145/3677389.3702537 ↗
5. Hwang, H., Park, C.-W., Kim, H.-K. & Lee, J.-H. Cats: cultural-heritage classification using llms and distribute model. npj Herit. Sci. 13, 76 (2025).10.1038/s40494-025-01621-1 ↗
6. Saharia, C. et al. Photorealistic text-to-image diffusion models with deep language understanding. Adv. Neural Inf. Process. Syst. 35, 36479–36494 (2022).
7. Rombach, R., Blattmann, A., Lorenz, D., Esser, P. & Ommer, B. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 10684–10695 (2022).10.1109/CVPR52688.2022.01042 ↗
8. Mou, C. et al. T2i-adapter: Learning adapters to dig out more controllable ability for text-to-image diffusion models. Proc. AAAI Conf. Artif. Intell. 38, 4296–4304 (2024).
9. Zhang, L., Rao, A. & Agrawala, M. Adding conditional control to text-to-image diffusion models. In Proceedings of the IEEE/CVF international conference on computer vision, 3836–3847 (2023).10.1109/ICCV51070.2023.00355 ↗
10. Hertz, A. et al. Prompt-to-prompt image editing with cross-attention control. In Proceedings of the International Conference on Learning Representations, 1–36 (2023).
11. Brooks, T., Holynski, A. & Efros, A. A. Instructpix2pix: Learning to follow image editing instructions. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 18392–18402 (2023).10.1109/CVPR52729.2023.01764 ↗
12. Meng, C. et al. Sdedit: Guided image synthesis and editing with stochastic differential equations. In Proceedings of the International Conference on Learning Representations, 1–33 (2021).
13. Chai, Y. et al. ERNIE-code: beyond English-centric cross-lingual pretraining for programming languages. In Proceedings of the Association for Computational Linguistics, 10628–10650 (Association for Computational Linguistics, 2023).10.18653/v1/2023.findings-acl.676 ↗
14. Xue, Z. et al. Raphael: Text-to-image generation via large mixture of diffusion paths. Adv. Neural Inf. Process. Syst. 36, 41693–41706 (2023).
15. Jiang, J. et al. Tencent text-video retrieval: hierarchical cross-modal interactions with multi-level representations. IEEE Access. 13, 122453–122463 (2022).10.1109/ACCESS.2022.3227973 ↗
16. Yang, L. et al. Diffusion models: a comprehensive survey of methods and applications. ACM Comput. Surv. 56, 1–39 (2023).10.1145/3554729 ↗
17. Qiao, T., Zhang, W., Zhang, M., Ma, Z. & Xu, D. Ancient painting to natural image: a new solution for painting processing. In Proceedings of the IEEE Winter Conference on applications of computer vision, 521–530 (IEEE, 2019).10.1109/WACV.2019.00061 ↗
18. Galerne, B., Raad, L., Lezama, J. & Morel, J.-M. Scaling painting style transfer. Comput. Graph. Forum 43, e15155 (2024).10.1111/cgf.15155 ↗
19. Gui, X., Zhang, B., Li, L. & Yang, Y. Dlp-gan: learning to draw modern chinese landscape photos with generative adversarial network. Neural Comput. Appl. 36, 5267–5284 (2024).10.1007/s00521-023-09345-8 ↗
20. Yuan, S. et al. Learning to generate poetic chinese landscape painting with calligraphy. In Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence, 5027–5033 (International Joint Conferences on Artificial Intelligence Organization, 2022).10.24963/ijcai.2022/696 ↗
21. Sun, Z. et al. Text-to-chinese-painting method based on multi-domain vqgan. Int. J. Softw. Informatics 13, 197–219 (2023).10.21655/ijsi.1673-7288.00314 ↗
22. Peng, X. et al. Fhs-adapter: fine-grained hierarchical semantic adapter for Chinese landscape paintings generation. Herit. Sci. 12, 265 (2024).10.1186/s40494-024-01370-7 ↗
23. Jia, Z., Wang, A., Qu, X., Yang, X. & Wang, J. Hierarchical-task-aware multi-modal mixture of incremental LoRA experts for embodied continual learning. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics, 28415–28427 (Association for Computational Linguistics, 2025).10.18653/v1/2025.acl-long.1379 ↗
24. Mao, Y. et al. DoRA: Enhancing parameter-efficient fine-tuning with dynamic rank distribution. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics, 11662–11675 (Association for Computational Linguistics, 2024).10.18653/v1/2024.acl-long.626 ↗
25. Li, H., Qiao, R., Yu, P., Li, H. & Tan, M. Cthd-net: Cnn-transformer hybrid dehazing network via residual global attention and gated boosting strategy. J. Vis. Commun. Image Represent. 99, 104066 (2024).10.1016/j.jvcir.2024.104066 ↗
26. Schopmans, H. & Friederich, P. Conditional normalizing flows for active learning of coarse-grained molecular representations. In Proceedings of the 41st International Conference on Machine Learning, vol. 235 of Proceedings of Machine Learning Research, 43804–43827 (PMLR, 2024).
27. Quadrianto, N., Petterson, J. & Smola, A. Distribution matching for transduction. Adv. Neural Inf. Process. Syst. 22, 1–9 (2009).
28. Xue, A. End-to-end chinese landscape painting creation using generative adversarial networks. In Proceedings of the IEEE/CVF Winter conference on applications of computer vision, 3863–3871 (2021).10.1109/WACV48630.2021.00391 ↗
29. Radford, A. et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, 8748–8763 (PmLR, 2021).
30. Zhang, R., Isola, P., Efros, A. A., Shechtman, E. & Wang, O. The unreasonable effectiveness of deep features as a perceptual metric. In Proceedings of the IEEE conference on computer vision and pattern recognition, 586–595 (2018).10.1109/CVPR.2018.00068 ↗
31. Heusel, M., Ramsauer, H., Unterthiner, T., Nessler, B. & Hochreiter, S. Gans trained by a two time-scale update rule converge to a local nash equilibrium. Adv. Neural Inf. Process. Syst. 30, 1–12 (2017).
32. Podell, D. et al. Sdxl: improving latent diffusion models for high-resolution image synthesis. In Proceedings of the International Conference on Representation Learning, 1862–1874 (2024).
33. Nichol, A. et al. Glide: towards photorealistic image generation and editing with text-guided diffusion models. In Proceedings of the International Conference on Machine Learning, 1–20 (2021).
34. Luo, L. et al. Taiyi: a bilingual fine-tuned large language model for diverse biomedical tasks. J. Am. Med. Inform. Assoc. 31, 1865–1874 (2024).10.1093/jamia/ocae037 ↗
35. Gal, R. et al. Lcm-lookahead for encoder-based text-to-image personalization. In Proceedings of the European Conference on Computer Vision, 322–340 (Springer Nature Switzerland, 2025).10.1007/978-3-031-72630-9_19 ↗
36. Wang, Z., Zhang, J., Ji, Z., Bai, J. & Shan, S. Cclap: controllable chinese landscape painting generation via latent diffusion model. In Proceedings of the IEEE International Conference on Multimedia and Expo, 2117–2122 (IEEE, 2023).10.1109/ICME55011.2023.00362 ↗
37. Hu, T. et al. Improving autoregressive visual generation with cluster-oriented token prediction. In Proceedings of the Computer Vision and Pattern Recognition Conference, 9351–9360 (2025).10.1109/CVPR52734.2025.00873 ↗