High-fidelity 3D Buddhist sculpture reconstruction from single images using domain-adaptive diffusion
Source abstract
Abstract
Buddhist sculptures face preservation challenges due to physical degradation and limited accessibility. We present a framework for high-fidelity 3D reconstruction from single 2D images, integrating modified Stable Diffusion architecture with a novel Instant and Consistent Mesh Reconstruction (ISOMER) algorithm. Our method employs multi-stage processing: multi-view image generation, high-resolution refinement via ControlNet and Real-ESRGAN, and normal map prediction. Enhanced by a curated dataset of 672 Buddhist sculptures, we introduce Explicit Target optimization in ISOMER, significantly improving fine iconographic detail preservation. Experimental results demonstrate superior performance over state-of-the-art methods, achieving 7.1% reduction in Chamfer Distance and 2.2% increase in F-Score. The framework captures intricate surface textures and stylistic elements crucial for art historical research and digital conservation, though limitations persist in reconstructing highly reflective or translucent materials.
Source: openalex_inverted_index · Quality: baseline
Research context
OpenAlex topics: Generative Adversarial Networks and Image Synthesis · Image Processing and 3D Reconstruction · Aesthetic Perception and Analysis
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
References (37)
Show source-provided references
1. Windhager, F. et al. Visualization of cultural heritage collection data: state of the art and future challenges. IEEE Trans. Vis. Comput. Graph. 25, 2311–2330 (2018).10.1109/TVCG.2018.2830759 ↗
2. Lu, Z., Wang, W., Guo, T. & Wang, F. Single-image reflection removal via self-supervised diffusion models. J. Supercomput. 81, 338 (2025).10.1007/s11227-024-06837-9 ↗
3. Remondino, F. Heritage recording and 3d modeling with photogrammetry and 3d scanning. Remote Sens. 3, 1104–1138 (2011).10.3390/rs3061104 ↗
4. Szymanowicz, S., Rupprecht, C. & Vedaldi, A. Splatter image: ultra-fast single-view 3D reconstruction. In Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition 10208–10217 (IEEE, 2024).10.1109/CVPR52733.2024.00972 ↗
5. Lu, Z. & Chen, Y. Self-supervised monocular depth estimation via multiple bilateral consistency. Multimed. Syst. 31, 111 (2025).10.1007/s00530-025-01700-0 ↗
6. Zhu, J.-Y., Park, T., Isola, P. & Efros, A. A. Unpaired image-to-image translation using cycle-consistent adversarial networks. In Proc. 16th IEEE International Conference on Computer Vision (ICCV) 2242–2251 (IEEE, Venice, ITALY, 2017).10.1109/ICCV.2017.244 ↗
7. Wu, J., Zhang, C., Xue, T., Freeman, W. T. & Tenenbaum, J. B. Learning a probabilistic latent space of object shapes via 3d generative-adversarial modeling. vol. 29 of Advances in Neural Information Processing Systems (2016). 30th Conference on Neural Information Processing Systems (NIPS), Barcelona, SPAIN, (Curran Associates Inc., 2016).
8. Hoiem, D., Efros, A. A. & Hebert, M. Automatic photo pop-up. ACM Trans. Graph. 24, 577–584 (2005).10.1145/1073204.1073232 ↗
9. Eigen, D., Puhrsch, C. & Fergus, R. Depth map prediction from a single image using a multi-scale deep network. vol. 27 of Advances in Neural Information Processing Systems (2014). 28th Conference on Neural Information Processing Systems (NIPS), Montreal, CANADA, (Curran Associates Inc., 2014).
10. Choy, C. B., Xu, D., Gwak, J. Y., Chen, K. & Savarese, S. 3d-r2n2: A unified approach for single and multi-view 3d object reconstruction. In COMPUTER VISION—ECCV 2016, PT VIII Vol. 9912 (eds Leibe, B., Matas, J., Sebe, N. & Welling, M.) 628–644 (Springer International Publishing, Amsterdam, Netherlands, 2016).10.1007/978-3-319-46484-8_38 ↗
11. Wang, N. et al. Pixel2mesh: 3d mesh model generation via image guided deformation. IEEE Trans. Pattern Anal. Mach. Intell. 43, 3600–3613 (2021).10.1109/TPAMI.2020.2984232 ↗
12. Liu, S., Saito, S., Chen, W. & Li, H. Learning to infer implicit surfaces without 3d supervision. In Proc. 33rd International Conference on Neural Information Processing Systems (NIPS) 8295–8306 (Vancouver, Canada, 2019).
13. Mildenhall, B. et al. Nerf: representing scenes as neural radiance fields for view synthesis. Commun. ACM 65, 99–106 (2021).10.1145/3503250 ↗
14. Long, X. et al. Wonder3D: single image to 3D using cross-domain diffusion. Proc. of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 9970–9980 (IEEE, 2024).10.1109/CVPR52733.2024.00951 ↗
15. He, Z. & Wang, T. Openlrm: open-source large reconstruction models. https://github.com/3DTopia/OpenLRM (2023).
16. Xu, J. et al. Instantmesh: efficient 3D mesh generation from a single image with sparse-view large reconstruction models. arXiv preprint https://doi.org/10.48550/arXiv.2404.07191 (2024).10.48550/arXiv.2404.07191 ↗
17. Xu, Y. et al. Grm: large gaussian reconstruction model for efficient 3D reconstruction and generation. In European Conference on Computer Vision 1–20 (Springer, 2024).10.1007/978-3-031-72633-0_1 ↗
18. Wang, Z. et al. Crm: single image to 3D textured mesh with convolutional reconstruction model. In European Conference on Computer Vision 57–74 (Springer, 2024).10.1007/978-3-031-72751-1_4 ↗
19. Beijbom, O., Joshi, N., Morris, D., Saponas, S. & Khullar, S. Menu-match: restaurant-specific food logging from images. In 2015 IEEE Winter Conference on Applications of Computer Vision, 844–851 (IEEE, 2015).10.1109/WACV.2015.117 ↗
20. Reese, K. M. Deep learning artificial neural networks for non-destructive archaeological site dating. J. Archaeol. Sci. 132, 105413 (2021).10.1016/j.jas.2021.105413 ↗
21. Grilli, E. & Remondino, F. Classification of 3D digital heritage. Remote Sens. 11, 847 (2019).10.3390/rs11070847 ↗
22. Hermoza, R. & Sipiran, I. 3D reconstruction of incomplete archaeological objects using a generative adversarial network. In Proc. 35th Computer Graphics International (CGI) 2018, 5–11 (Association for Computing Machinery, Bintan Island, Indonesia, 2018).10.1145/3208159.3208173 ↗
23. Elgammal, A., Kang, Y. & Den Leeuw, M. Automated analysis of drawings at the stroke level for attribution and authentication using artificial intelligence. In Analytical Chemistry for the Study of Paintings and the Detection of Forgeries (eds Colombini, M. P., Degano, I. & Nevin, A.) 133–154 (Springer International Publishing, Cham, 2022).10.1007/978-3-030-86865-9_5 ↗
24. Johnson, C. R. et al. Image processing for artist identification. IEEE Signal Process. Mag. 25, 37–48 (2008).10.1109/MSP.2008.923513 ↗
25. Rombach, R., Blattmann, A., Lorenz, D., Esser, P. & Ommer, B. High-resolution image synthesis with latent diffusion models. In Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 10684–10695 (IEEE, New Orleans, USA, 2022).10.1109/CVPR52688.2022.01042 ↗
26. Paszke, A. Pytorch: an imperative style, high-performance deep learning library. arXiv preprint https://doi.org/10.48550/arXiv.1912.01703 (2019).10.48550/arXiv.1912.01703 ↗
27. Wu, K. et al. Unique3D: high-quality and efficient 3D mesh generation from a single image. In Proc. of the 38th International Conference on Neural Information Processing Systems, 125116-125141 (Curran Associates Inc., Red Hook, NY, USA 2024). https://doi.org/10.5555/3737916.3741890.10.5555/3737916.3741890 ↗
28. Bekele, M. K., Pierdicca, R., Frontoni, E., Malinverni, E. S. & Gain, J. A survey of augmented, virtual, and mixed reality for cultural heritage. J. Comput. Cult. Herit. 11, 1–36 (2018).10.1145/3145534 ↗
29. Pavlidis, G., Koutsoudis, A., Arnaoutoglou, F., Tsioukas, V. & Chamzas, C. Methods for 3D digitization of cultural heritage. J. Cult. Herit. 8, 93–98 (2007).10.1016/j.culher.2006.10.007 ↗
30. Cameron, F., Kenderdine, S. Theorizing Digital Cultural Heritage: A Critical Discourse. The MIT Press, 2007.10.7551/mitpress/9780262033534.001.0001 ↗
31. Scopigno, R., Cignoni, P., Pietroni, N., Callieri, M. & Dellepiane, M. Digital fabrication techniques for cultural heritage: a survey. In Computer Graphics Forum Vol. 36, 6–21 (Wiley Online Library, 2017).10.1111/cgf.12781 ↗
32. Yastikli, N. Documentation of cultural heritage using digital photogrammetry and laser scanning. J. Cult. Herit. 8, 423–427 (2007).10.1016/j.culher.2007.06.003 ↗
33. Wu, S., Rupprecht, C. & Vedaldi, A. Unsupervised learning of probably symmetric deformable 3d objects from images in the wild. In Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 1–10 (IEEE, Seattle, USA, 2020).10.1109/CVPR42600.2020.00008 ↗
34. Kanazawa, A., Tulsiani, S., Efros, A. A. & Malik, J. Learning category-specific mesh reconstruction from image collections. In Proc. 15th European Conference on Computer Vision (ECCV) 371–386 (Springer, Munich, Germany, 2018).10.1007/978-3-030-01267-0_23 ↗
35. Lu, Z. & Chen, Y. Self-supervised monocular depth estimation on water scenes via specular reflection prior. Digit. Signal Process. 149, 104496 (2024).10.1016/j.dsp.2024.104496 ↗
36. Zhang, K., Luan, F., Wang, Q., Bala, K. & Snavely, N. Physg: inverse rendering with spherical gaussians for physics-based material editing and relighting. In Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 5453–5462 (IEEE, Nashville, USA, 2021).10.1109/CVPR46437.2021.00541 ↗
37. Lu, Z. & Chen, Y. Joint self-supervised depth and optical flow estimation towards dynamic objects. Neural Process. Lett. 55, 10235–10249 (2023).10.1007/s11063-023-11325-x ↗