Multimodal intelligent reconstruction-driven digital regeneration of cultural heritage in Qing Dynasty tea plantation landscapes
Source abstract
Abstract
Advances in AI, particularly deep learning, are transforming the protection and utilization of historical and cultural landscapes. This study proposes a comprehensive digital regeneration workflow for fragmented historical images that systematically integrates image analysis, including segmentation and color extraction, with conditional generation and dynamic presentation. Based on 104 Qing Dynasty export paintings, we develop a hybrid “Stable Diffusion + ControlNet” architecture that incorporates semantic segmentation and HSV color analysis to achieve fine-grained scene generation. The research establishes a complete multimodal technical process of “intelligent reconstruction–dynamic regeneration”, extending beyond the mere application of generative models. The final output transforms fragmented historical imagery into narrative-driven multimedia spatial demonstrations, offering a reusable operational framework and methodology for immersive heritage experiences and sustainable cultural preservation.
Source: openalex_inverted_index · Quality: baseline
Research context
OpenAlex topics: Generative Adversarial Networks and Image Synthesis · 3D Surveying and Cultural Heritage · Aesthetic Perception and Analysis
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
References (50)
Show source-provided references
1. Chen, Y. The research on Chinese export paintings: review and prospect. Acad. Res. 7, 154–161 (2018).
2. van der Poel, R. Made for Trade-Made in China. Chinese Export Paintings in Dutch Collections: Art and Commodity (Leiden University, 2016).
3. Fraser, S. E., Wakita, M. & Wang, L. Women Cross Media: East Asian Photography, Prints, and Porcelain from the Staatliche Kunstsammlungen Dresden 40–85 (arthistoricum.net, 2022).
4. Liu, H., Fu, C. & Li, W. Silk road heritage: the artistic representation of port trading culture in the images of characters in Qing Dynasty Guangzhou export paintings. PLoS ONE 19, e0308309 (2024).10.1371/journal.pone.0308309 ↗
5. Zhang, F. From commodities to exhibits: an analysis of Shipping in the Pearl River off Canton and Shipping in the Pearl River off Honam Island, Canton in the National Maritime Museum in the UK. Art Soc. 1, 49–58 (2022).10.56397/AS.2022.08.07 ↗
6. Koo, L. S. Export paintings as art and agency: Re-assessing export paintings of China and Korea. Athanor 39, 95–110 (2022).10.33009/FSU_athanor131145 ↗
7. Shin, K.-Y. I. Making “Chinese Art”: Knowledge and Authority in the Transpacific Progressive Era 212 (Columbia University, 2016).
8. Lu, D. & Pan, Y. Digital Preservation for Heritages: Technologies and Applications (Springer Science & Business Media, 2011).
9. Lian, Y. & Xie, J. The evolution of digital cultural heritage research: identifying key trends, hotspots, and challenges through bibliometric analysis. Sustainability 16, 7125 (2024).10.3390/su16167125 ↗
10. Buragohain, D., Meng, Y., Deng, C., Li, Q. & Chaudhary, S. Digitalizing cultural heritage through metaverse applications: challenges, opportunities, and strategies. Herit. Sci. 12, 295 (2024).10.1186/s40494-024-01403-1 ↗
11. Barman, R., Ehrmann, M., Clematide, S., Oliveira, S. A. & Kaplan, F. Combining visual and textual features for semantic segmentation of historical newspapers. J. Data Min. Digit. Humanit. 2021, 6107 (2021).10.46298/jdmdh.6107 ↗
12. Li, Y. et al. A review of the tools and techniques used in the digital preservation of architectural heritage within disaster cycles. Herit. Sci. 11, 199 (2023).10.1186/s40494-023-01035-x ↗
13. Mazzetto, S. Integrating emerging technologies with digital twins for heritage building conservation: an interdisciplinary approach with expert insights and bibliometric analysis. Heritage 7, 6432–6479 (2024).10.3390/heritage7110300 ↗
14. Koller, D., Frischer, B. & Humphreys, G. Research challenges for digital archives of 3D cultural heritage models. J. Comput. Cult. Herit. 2, 1–17 (2010).10.1145/1658346.1658347 ↗
15. Barsanti, S. G., Remondino, F., Fenández-Palacios, B. J. & Visintini, D. Critical factors and guidelines for 3D surveying and modelling in cultural heritage. Int. J. Herit. Digit. Era 3, 141–158 (2014).10.1260/2047-4970.3.1.141 ↗
16. Llamas, J., M. Lerones, P., Medina, R., Zalama, E. & Gómez-García-Bermejo, J. Classification of architectural heritage images using deep learning techniques. Appl. Sci. 7, 992 (2017).10.3390/app7100992 ↗
17. Grilli, E. & Remondino, F. Classification of 3D digital heritage. Remote Sens. 11, 847 (2019).10.3390/rs11070847 ↗
18. Kumar, P., Ofli, F., Imran, M. & Castillo, C. Detection of disaster-affected cultural heritage sites from social media images using deep learning techniques. J. Comput. Cult. Herit. 13, 1–31 (2020).10.1145/3383314 ↗
19. Shao, L. & Sun, J. Identification of historic building “genes” based on deep learning: a case study on Chinese baroque architecture in Harbin, China. Herit. Sci. 11, 241 (2023).10.1186/s40494-023-01091-3 ↗
20. Bruno, S., Galantucci, R. A. & Musicco, A. Decay detection in historic buildings through image-based deep learning. VITRUVIO Int. J. Archit. Technol. Sustain. 8, 6–17 (2023).
21. Luo, S. & Wang, H. Digital twin research on masonry–timber architectural heritage pathology cracks using 3D laser scanning and deep learning model. Buildings 14, 1129 (2024).10.3390/buildings14041129 ↗
22. Tejedor, B., Lucchi, E., Bienvenido-Huertas, D. & Nardi, I. Non-destructive techniques (NDT) for the diagnosis of heritage buildings: traditional procedures and futures perspectives. Energy Build. 263, 112029 (2022).10.1016/j.enbuild.2022.112029 ↗
23. Pan, X. et al. Deep learning based approaches from semantic point clouds to semantic BIM models for heritage digital twin. Herit. Sci. 12, 65 (2024).10.1186/s40494-024-01179-4 ↗
24. Csurka, G. & Perronnin, F. An efficient approach to semantic segmentation. Int. J. Comput. Vis. 95, 198–212 (2010).10.1007/s11263-010-0344-8 ↗
25. Gupta, V., Sambyal, N., Sharma, A. & Kumar, P. Restoration of artwork using deep neural networks. Evol. Syst. 12, 439–446 (2019).10.1007/s12530-019-09303-7 ↗
26. Tonazzini, A. et al. Analytical and mathematical methods for revealing hidden details in ancient manuscripts and paintings: a review. J. Adv. Res. 17, 31–42 (2019).10.1016/j.jare.2019.01.003 ↗
27. Zhang, C. et al. A comprehensive survey on segment anything model for vision and beyond. Preprint at arXiv https://doi.org/10.48550/arXiv.2305.08196 (2023).10.48550/arXiv.2305.08196 ↗
28. He, K., Gkioxari, G., Dollar, P. & Girshick, R. Mask R-CNN. In Proc. IEEE International Conference on Computer Vision (ICCV), 2961–2969 (IEEE, 2017).10.1109/ICCV.2017.322 ↗
29. Chen, L.-C., Zhu, Y., Papandreou, G., Schroff, F. & Adam, H. Encoder-decoder with atrous separable convolution for semantic image segmentation. In Proc. European Conference on Computer Vision (ECCV), 834–851 (Springer, 2018).10.1007/978-3-030-01234-2_49 ↗
30. Oršić, M. & Šegvić, S. Efficient semantic segmentation with pyramidal fusion. Pattern Recognit. 110, 107611 (2021).10.1016/j.patcog.2020.107611 ↗
31. Shen, J. et al. An algorithm based on lightweight semantic features for ancient mural element object detection. npj Herit. Sci. 13, 70 (2025).10.1038/s40494-025-01565-6 ↗
32. Zhang, Q. et al. A Mamba based vision transformer for fine grained image segmentation of mural figures. npj Herit. Sci. 13, 204 (2025).10.1038/s40494-025-01682-2 ↗
33. Xie, Z. et al. Cross modal networks for point cloud semantic segmentation of Chinese ancient buildings. npj Herit. Sci. 13, 131 (2025).10.1038/s40494-025-01701-2 ↗
34. Zhang, L., Rao, A. & Agrawala, M. Adding conditional control to text-to-image diffusion models. In 2023 IEEE/CVF International Conference on Computer Vision (ICCV), 3836–3847 (IEEE, 2023).10.1109/ICCV51070.2023.00355 ↗
35. Wen, B., Xie, H., Chen, Z., Hong, F. & Liu, Z. 3D scene generation: a survey. Preprint at arXiv https://doi.org/10.48550/arXiv.2505.05474 (2025).10.48550/arXiv.2505.05474 ↗
36. Zhou, J., Liu, Y. S. & Han, Z. Zero-shot scene reconstruction from single images with deep prior assembly. Advances in Neural Information Processing Systems, Vol. 37, 39104–39127 (2024)..10.52202/079017-1234 ↗
37. Kirillov, A. et al. Segment anything. In 2023 IEEE/CVF International Conference on Computer Vision (ICCV), 4015–4026 (IEEE, 2023).10.1109/ICCV51070.2023.00371 ↗
38. Liu, Z. et al. Swin transformer: Hierarchical vision transformer using shifted windows. In 2021 IEEE/CVF International Conference on Computer Vision (ICCV), 10012–10022 (IEEE, 2021).10.1109/ICCV48922.2021.00986 ↗
39. Zhao, F., Ren, H., Sun, K. & Zhu, X. GAN-based heterogeneous network for ancient mural restoration. Herit. Sci. 12, 418 (2024).10.1186/s40494-024-01517-6 ↗
40. Chen, Y., Zhang, A., Gao, F. & Guo, J. Ancient mural super-resolution reconstruction based on conditional diffusion model for enhanced visual information. npj Herit. Sci. 13, 205 (2025).10.1038/s40494-025-01718-7 ↗
41. Zhang, R., Zhang, G. & Yin, J. Semantic segmentation method of 3D scenes using dynamic graph CNN for complex city. Acta Geod. Cartogr. Sin. 52, 1703–1713 (2023).
42. Rombach, R., Blattmann, A., Lorenz, D., Esser, P. & Ommer, B. High-resolution image synthesis with latent diffusion models. In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 10684–10695 (IEEE, 2022).10.1109/CVPR52688.2022.01042 ↗
43. Goodfellow, I. J. et al. Generative adversarial nets. Adv. Neural Inf. Process. Syst. 2, 2672–2680 (2014).
44. Yu, J. et al. Free-form image inpainting with gated convolution. In 2019 IEEE/CVF International Conference on Computer Vision (ICCV), 4471–4480 (IEEE, 2019).10.1109/ICCV.2019.00457 ↗
45. Zhu, J.-Y., Park, T., Isola, P. & Efros, A. A. Unpaired image-to-image translation using cycle-consistent adversarial networks. In 2017 IEEE International Conference on Computer Vision (ICCV), 2223–2232 (IEEE, 2017).10.1109/ICCV.2017.244 ↗
46. Zhang, D. Logic of information organizing in information design. Design 4, 116–117 (2016).
47. Tenner, E. The design of everyday things by Donald Norman. Technol. Cult. 56, 785–787 (2015).10.1353/tech.2015.0104 ↗
48. Venkatesh, V., Morris, M. G., Davis, G. B. & Davis, F. D. User acceptance of information technology: toward a unified view. MIS Q. 425–478 (2003).10.2307/30036540 ↗
49. Hovland, C. I., Janis, I. L. & Kelley, H. H. Communication and Persuasion (Greenwood Press Reprint, 1953).
50. Norman, D. Emotional Design: Why We Love (or Hate) Everyday Things (Basic Books, 2007).