Implicit 3D human body reconstruction method from geometric diffusion of dense surface correspondences
Source abstract
Abstract
To address the limitations of insufficient topological constraints and feature redundancy in existing high-resolution implicit geometric representation methods for 3D human body reconstruction, we propose an implicit reconstruction framework based on geometric diffusion of dense surface correspondences. The framework consists of three-stage. First, PoseCorr-Net nonlinearly maps clothed human scan point cloud into the canonical space, mitigating pose distortion and self-occlusion. Second, a bidirectional diffusion mechanism is introduced to enhance the hybrid point cloud representation, generating a balanced point set that preserves high-frequency surface details, maintains continuous internal structures, and reduces redundant sampling. In BodyOcc-Net, the part-aware feature aggregation module is embedded to infer occupancy field, enhancing the geometric representation of the human body. Finally, a SMPL-D-guided differentiable inverse mapping ensures topology-consistent reconstruction by transforming body point cloud from canonical space back to their the original pose space. Extensive quantitative and qualitative evaluations demonstrate that the proposed method achieves high-fidelity 3D human body reconstruction with strong computational efficiency.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: 3D Shape Modeling and Analysis · Human Pose and Action Recognition · Robotics and Sensor-Based Localization
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Shanghai Key Laboratory of Intelligent Manufacturing and Robotics, Shanghai University, Shanghai, China
College of Mechanical and Automotive Engineering, Ningbo University of Technology, Zhejiang Province, China
School of Mechanical and Electrical Engineering, Henan Institute of Science and Technology, Henan Province, China
Funding
The Key Science and Technology Program of Henan Province
References (38)
Show source-provided references
1. Liang JB, Lin MC. Shape-aware human pose and shape reconstruction using multi-view images. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, 2019, pp. 4352-4362.
2. Pavlakos G, Choutas V, Ghorbani N, et al. Expressive body capture: 3D hands, face, and body from a single image. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019, pp. 10975-10985.
3. 10.1007/s00371-023-03122-310.1007/s00371-023-03122-3 ↗
4. 10.1177/0040517522109366310.1177/00405175221093663 ↗
5. Cao YK, Han K, Wong KYK. SESDF: self-evolved signed distance field for implicit 3D clothed human reconstruction. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 4647-4657.
6. Chan KY, Liu FY, Lin GS, Foo CS, Lin WS. 3DFG-PIFu: 3D feature grids for human digitization from sparse views. In: Proceedings Computer Vision - ECCV 2024: 18th European Conference on Computer Vision, Milan, Italy, 29 Sepember-4 October 2024, Part XXIV (15082). Berlin: Springer, 2024, pp. 38-54.
7. 10.1016/j.patcog.2024.11075810.1016/j.patcog.2024.110758 ↗
8. Ma BR, Liu YS, Han ZZ. Reconstructing surfaces for sparse point clouds with on-surface priors. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 6315-6325.
9. Park JJ, Florence P, Straub J, Newcombe R, Lovegrove S. DeepSDF: Learning continuous signed distance functions for shape representation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019, pp. 165-174.
10. Lin CH, Wang C, Lucey S. SDF-SRN: Learning signed distance 3D object reconstruction from static images. Adv Neur Inf Process Syst 2020; 33: 11453-11464.
11. Jiang Y, Ji DT, Han ZZ, Zwicker M. SDFDiff: differentiable rendering of signed distance fields for 3D shape optimization. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, pp. 1251-1261.
12. Mescheder L, Oechsle M, Niemeyer M, Nowozin S, Geiger A. Occupancy networks: Learning 3D reconstruction in function space. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019, pp. 4460-4470.
13. Saito S, Huang Z, Natsume R, Morishima S, Kanazawa A, Li H. PIFu: Pixel-aligned implicit function for high-resolution clothed human digitization. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019, pp. 2304-2314.
14. Saito S, Simon T, Saragih J, Joo H. PIFuHD: Multi-level pixel-aligned implicit function for high-resolution 3D human digitization. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, pp. 84-93.
15. Chibane J, Alldieck T, Pons-Moll G. Implicit functions in feature space for 3D shape reconstruction and completion. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, pp. 6970-6981.
16. 10.1007/978-3-030-58536-5_1910.1007/978-3-030-58536-5_19 ↗
17. 10.1007/978-3-030-01219-9_810.1007/978-3-030-01219-9_8 ↗
18. Neverova N, Novotny D, Khalidov V, Szafraniec M, Labatut P, Vedaldi A. Continuous surface embeddings. Adv Neur Inf Process Syst 2020; 33: 17258-17270.
19. Tan F, Tang D, Dou M, et al. HumanGPS: Geodesic preserving feature for dense human correspondences. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021, pp. 1820-1830.
20. 10.1007/978-3-030-58526-6_1410.1007/978-3-030-58526-6_14 ↗
21. Donati N, Sharma A, Ovsjanikov M. Deep geometric functional maps: Robust feature learning for shape correspondence. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, pp. 8589-8598.
22. 10.1007/978-3-030-58558-7_310.1007/978-3-030-58558-7_3 ↗
23. Halimi O, Litany O, Rodola E, Bronstein A, Kimmet R. Unsupervised learning of dense shape correspondence. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019, pp. 4365-4374.
24. Roufosse JM, Sharma A, Ovsjanikov M. Unsupervised deep learning for structured shape matching. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019, pp. 1617-1627.
25. Bhatnagar BL, Sminchisescu C, Theobalt C, Pons-Moll G. LoopReg: Self-supervised learning of implicit surface correspondences, pose and shape for 3D human mesh registration. Adv Neur Inf Process Syst 2020; 33: 12909-12922.
26. Kanazawa A, Block MJ, Jacobs DW, Malik J. End-to-end recovery of human shape and pose. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2018, pp. 7122-7131.
27. Chibane J, Alldieck T, Pons-Moll G. Implicit functions in feature space for 3D shape reconstruction and completion. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, pp. 6968-6979.
28. 10.1145/3072959.307371110.1145/3072959.3073711 ↗
29. Wang SF, Geiger A, Tang SY. Locally aware piecewise transformation fields for 3D human mesh registration. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021, pp. 7635-7644.
30. 10.1109/TVCG.2024.347541410.1109/TVCG.2024.3475414 ↗
31. 10.1109/3DV.2019.0007610.1109/3DV.2019.00076 ↗
32. Alldieck T, Magnor M, Bhatnagar BL, Theobalt C, Pons-Moll G. Learning to reconstruct people in clothing from a single RGB camera. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019, pp. 1175-1186.
33. Feng HW, Kulits P, Liu SC, Black MJ, Abrevaya V. Generalizing neural human fitting to unseen poses with articulated SE(3) equivariance. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 7943-7954.
34. Jiang HY, Cai JF, Zheng JM. Skeleton-aware 3D human shape reconstruction from point clouds. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019, pp. 5430-5440.
35. Ma QL, Yang JL, Ranjan A, et al. Learning to dress 3D people in generative clothing. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, pp. 6468-6477.
36. Yu T, Zheng ZR, Guo KW, Liu PP, Dai QH, Liu YB. Function4D: Real-time human volumetric capture from very sparse consumer RGBD sensors. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021, pp. 5742-5752.
37. Zhang C, Pujades S, Black M, Pons-Moll G. Detailed, accurate, human shape estimation from clothed 3D scan sequences. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2017, pp. 5484-5493.
38. Bhatnagar BL, Tiwari G, Theobalt C, Pons-Moll G. Multi-Garment Net: Learning to dress 3D people from images. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, 2019, pp. 5419-5429.