Multiscale voxel feature fusion network for large scale noisy point cloud completion in cultural heritage restoration
Source abstract
Abstract
The high-fidelity digitization of large-scale cultural heritage via point clouds is often hindered by occlusions and environmental noise during scanning, compromising morphological analysis and restoration. To address challenges such as noise interference and inadequate detail reconstruction, we propose a novel three-stage framework. It incorporates a Multistage Filtering Module to cleanse raw point clouds by balancing noise suppression with geometric preservation. A Multiscale Voxel Feature Fusion Framework hierarchically extracts and fuses features at varying voxel granularities, enhancing recovery of both global structures and local details. Additionally, a Curvature-guided Feature Enhancement Module sharpens reconstruction in high-curvature areas during skeleton point prediction. Experiments show our method outperforms existing approaches by 16% on the ShapeNet-55 and 12% on a real-world cultural heritage roof dataset. The application to Tamaki-jinja Shrine data effectively completed the missing roof regionsc and transparent visualization results confirmed improved perceptual clarity and structural visibility, thereby validating its utility for digital preservation.
Source: openalex_inverted_index · Quality: baseline
Research context
OpenAlex topics: 3D Shape Modeling and Analysis · 3D Surveying and Cultural Heritage · Image Processing and 3D Reconstruction
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
References (53)
Show source-provided references
1. Yang, S., Xu, S. & Huang, W. 3d point cloud for cultural heritage: a scientometric survey. Remote Sens. 14, 5542 (2022).10.3390/rs14215542 ↗
2. Xue, F., Lu, W., Chen, Z. & Webster, C. J. From lidar point cloud towards digital twin city: clustering city objects based on gestalt principles. ISPRS J. Photogramm. Remote Sens. 167, 418–431 (2020).10.1016/j.isprsjprs.2020.07.020 ↗
3. Cui, Y. et al. Deep learning for image and point cloud fusion in autonomous driving: A review. IEEE Trans. Intell. Transp. Syst. 23, 722–739 (2021).10.1109/TITS.2020.3023541 ↗
4. Yuan, W., Khot, T., Held, D., Mertz, C. & Hebert, M. Pcn: Point completion network. In 2018 international conference on 3D vision (3DV), 728–737 (IEEE, 2018).10.1109/3DV.2018.00088 ↗
5. Huang, Z., Yu, Y., Xu, J., Ni, F. & Le, X. Pf-net: Point fractal network for 3d point cloud completion. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 7662–7670 (IEEE, 2020).10.1109/CVPR42600.2020.00768 ↗
6. Hackel, T., Savinov, N., Ladicky, L., Wegner, J. D., Schindler, K. & Pollefeys, M. SEMANTIC3D.NET: A new large-scale point cloud classification benchmark. ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, IV-1/W1, 91–98 (2017).10.5194/isprs-annals-IV-1-W1-91-2017 ↗
7. Li, W., Pan, J., Hasegawa, K., Li, L. & Tanaka, S. Missing region completion network for large-scale laser-scanned point clouds: application to transparent visualization of cultural heritage. Remote Sens. 16, 2758 (2024).10.3390/rs16152758 ↗
8. Zhou, S., Liu, X., Wang, C. & Yang, B. Non-iterative denoising algorithm based on a dual threshold for a 3d point cloud. Opt. Lasers Eng. 126, 105921 (2020).10.1016/j.optlaseng.2019.105921 ↗
9. Leal, E., Sanchez-Torres, G. & Branch, J. W. Sparse regularization-based approach for point cloud denoising and sharp features enhancement. Sensors 20, 3206 (2020).10.3390/s20113206 ↗
10. Zhu, D. et al. Nonlocal low-rank point cloud denoising for 3-D measurement surfaces. IEEE Trans. Instrum. Meas. 71, 1–14 (2022).
11. Lu, X. et al. Gpf: Gmm-inspired feature-preserving point set filtering. IEEE Trans. Vis. Comp. Graph. 24, 2315–2326 (2017).10.1109/TVCG.2017.2725948 ↗
12. Luo, S. & Hu, W. Score-based point cloud denoising. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 4583–4592 (IEEE, 2021).10.1109/ICCV48922.2021.00454 ↗
13. Kalogerakis, E., Chaudhuri, S., Koller, D. & Koltun, V. A probabilistic model for component-based shape synthesis. ACM Trans. Graph. 31, 1–11 (2012).
14. Berger, M. et al. State of the art in surface reconstruction from point clouds. In 35th Annual Conference of the European Association for Computer Graphics, Eurographics 2014-State of the Art Reports (The Eurographics Association, 2014).
15. Hu, W., Fu, Z. & Guo, Z. Local frequency interpretation and non-local self-similarity on graph for point cloud inpainting. IEEE Trans. Image Process. 28, 4087–4100 (2019).10.1109/TIP.2019.2906554 ↗
16. Xie, H. et al. Grnet: Gridding residual network for dense point cloud completion. In European conference on computer vision, 365–381 (Springer, 2020).10.1007/978-3-030-58545-7_21 ↗
17. Kim, J., Kwon, H., Yang, Y. & Yoon, K.-J. Learning point cloud completion without complete point clouds: a pose-aware approach. In ICCV, 14157–14167 (ICCV, 2023).10.1109/ICCV51070.2023.01306 ↗
18. Gong, B., Nie, Y., Lin, Y., Han, X. & Yu, Y. Me-pcn: Point completion conditioned on mask emptiness. In Proceedings of the IEEE/CVF international conference on computer vision, 12488–12497 (IEEE, 2021).10.1109/ICCV48922.2021.01226 ↗
19. Hu, T., Han, Z., Shrivastava, A. & Zwicker, M. Render4completion: Synthesizing multi-view depth maps for 3d shape completion. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) Workshops (IEEE, 2019).10.1109/ICCVW.2019.00506 ↗
20. Wang, Y., Tan, D. J., Navab, N. & Tombari, F. Softpoolnet: Shape descriptor for point cloud completion and classification. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part III 16, 70–85 (Springer, 2020).10.1007/978-3-030-58580-8_5 ↗
21. Zhang, W., Yan, Q. & Xiao, C. Detail preserved point cloud completion via separated feature aggregation. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XXV 16, 512–528 (Springer, 2020).10.1007/978-3-030-58595-2_31 ↗
22. Vaswani, A. et al. Attention is all you need. Advances in Neural Information Processing Systems30 (NIPS, 2017).
23. Zhao, H., Jiang, L., Jia, J., Torr, P. H. & Koltun, V. Point transformer. In Proceedings of the IEEE/CVF international conference on computer vision, 16259–16268 (2021).10.1109/ICCV48922.2021.01595 ↗
24. Guo, M.-H. et al. PCT: point cloud transformer. Comput. Vis. Media 7, 187–199 (2021).10.1007/s41095-021-0229-5 ↗
25. Yu, X. et al. Pointr: Diverse point cloud completion with geometry-aware transformers. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 12498–12507 (IEEE, 2021).10.1109/ICCV48922.2021.01227 ↗
26. Yang, Y., Feng, C., Shen, Y. & Tian, D. Foldingnet: Point cloud auto-encoder via deep grid deformation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 206–215 (IEEE, 2018).10.1109/CVPR.2018.00029 ↗
27. Wen, X. et al. Pmp-net: Point cloud completion by learning multi-step point moving paths. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 7443–7452 (IEEE, 2021).10.1109/CVPR46437.2021.00736 ↗
28. Gross, M. & Pfister, H.Point-based graphics (Elsevier, 2011).
29. Kersten, T. P., Keller, F., Saenger, J. & Schiewe, J. Automated generation of an historic 4d city model of hamburg and its visualisation with the ge engine. In Progress in Cultural Heritage Preservation: 4th International Conference, EuroMed 2012, Limassol, Cyprus, October 29–November 3, 2012. Proceedings 4, 55–65 (Springer, 2012).10.1007/978-3-642-34234-9_6 ↗
30. Aicardi, I., Chiabrando, F., Lingua, A. M. & Noardo, F. Recent trends in cultural heritage 3d survey: The photogrammetric computer vision approach. J. Cult. Herit. 32, 257–266 (2018).10.1016/j.culher.2017.11.006 ↗
31. Zwicker, M., Pfister, H., Van Baar, J. & Gross, M. Ewa splatting. IEEE Trans. Vis. Computer Graph. 8, 223–238 (2002).10.1109/TVCG.2002.1021576 ↗
32. Seemann, P., Palma, G., Dellepiane, M., Cignoni, P. & Goesele, M. Soft transparency for point cloud rendering. In EGSR (EI&I), 95–106 (EGSR, 2018).
33. Zhang, Y. & Pajarola, R. Deferred blending: Image composition for single-pass point rendering. Comp. Graph. 31, 175–189 (2007).10.1016/j.cag.2006.11.012 ↗
34. Tanaka, S. et al. Particle-based transparent rendering of implicit surfaces and its application to fused visualization. In EuroVis (Short Papers), 35–29 (EG, 2012).
35. Tanaka, S. et al. See-through imaging of laser-scanned 3d cultural heritage objects based on stochastic rendering of large-scale point clouds. ISPRS Ann. Photogramm. Remote Sens. Spat. Inf. Sci. 3, 73–80 (2016).
36. Uchida, T. et al. Noise-robust transparent visualization of large-scale point clouds acquired by laser scanning. ISPRS J. Photogramm. Remote Sens. 161, 124–134 (2020).10.1016/j.isprsjprs.2020.01.004 ↗
37. Li, W., Hasegawa, K., Li, L., Tsukamoto, A. & Tanaka, S. Deep learning-based point upsampling for edge enhancement of 3d-scanned data and its application to transparent visualization. Remote Sens. 13, 2526 (2021).10.3390/rs13132526 ↗
38. Lipman, Y., Cohen-Or, D., Levin, D. & Tal-Ezer, H. Parameterization-free projection for geometry reconstruction. ACM Trans. Graph. 26, 22–es (2007).10.1145/1276377.1276405 ↗
39. Huang, H., Li, D., Zhang, H., Ascher, U. & Cohen-Or, D. Consolidation of unorganized point clouds for surface reconstruction. ACM Trans. Graph. 28, 1–7 (2009).10.1145/1618452.1618522 ↗
40. Lang, A. H. et al. Pointpillars: Fast encoders for object detection from point clouds. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 12697–12705 (IEEE, 2019).10.1109/CVPR.2019.01298 ↗
41. Choy, C., Gwak, J. & Savarese, S. 4d spatio-temporal convnets: Minkowski convolutional neural networks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 3075–3084 (IEEE, 2019).10.1109/CVPR.2019.00319 ↗
42. Hu, J., Shen, L. & Sun, G. Squeeze-and-excitation networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, 7132–7141 (IEEE2018).10.1109/CVPR.2018.00745 ↗
43. Kawakami, K. et al. Opacity-based edge highlighting for transparent visualization of 3d scanned point clouds. ISPRS Ann. Photogramm., Remote Sens. Spat. Inf. Sci. 2, 373–380 (2020).
44. Li, R., Li, X., Fu, C.-W., Cohen-Or, D. & Heng, P.-A. Pu-gan: a point cloud upsampling adversarial network. In Proceedings of the IEEE/CVF international conference on computer vision, 7203–7212 (IEEE, 2019).10.1109/ICCV.2019.00730 ↗
45. Fan, H., Su, H. & Guibas, L. J. A point set generation network for 3d object reconstruction from a single image. In Proceedings of the IEEE conference on computer vision and pattern recognition, 605–613 (IEEE, 2017).10.1109/CVPR.2017.264 ↗
46. Yu, L., Li, X., Fu, C.-W., Cohen-Or, D. & Heng, P.-A. Pu-net: Point cloud upsampling network. In Proceedings of the IEEE conference on computer vision and pattern recognition, 2790–2799 (IEEE, 2018).10.1109/CVPR.2018.00295 ↗
47. Chang, A. X. et al. Shapenet: an information-rich 3D model repository. arXiv preprint arXiv:1512.03012 (2015).
48. Kingma, D. P. & Ba, J. Adam: A method for stochastic optimization. In 3rd International Conference on Learning Representations, Conference Track Proceedings (ICLR, 2015).
49. Tchapmi, L. P., Kosaraju, V., Rezatofighi, H., Reid, I. & Savarese, S. Topnet: Structural point cloud decoder. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 383–392 (IEEE, 2019).10.1109/CVPR.2019.00047 ↗
50. Yan, W. et al. Vaccine-style-net: Point cloud completion in implicit continuous function space. In Proceedings of the 28th ACM international conference on multimedia, 2067–2075 (ACM, 2020).10.1145/3394171.3413648 ↗
51. Berger, M., Levine, J. A., Nonato, L. G., Taubin, G. & Silva, C. T. A benchmark for surface reconstruction. ACM Trans. Graph. 32, 1–17 (2013).
52. Lague, D., Brodu, N. & Leroux, J. Accurate 3d comparison of complex topography with terrestrial laser scanner: Application to the rangitikei canyon (nz). ISPRS J. Photogramm. Remote Sens. 82, 10–26 (2013).10.1016/j.isprsjprs.2013.04.009 ↗
53. Mescheder, L., Oechsle, M., Niemeyer, M., Nowozin, S. & Geiger, A. Occupancy networks: Learning 3d reconstruction in function space. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 4460–4470 (IEEE, 2019).10.1109/CVPR.2019.00459 ↗