Part-Whole Relational Few-Shot 3D Point Cloud Semantic Segmentation
Source abstract
Abstract
This paper focuses on the task of few-shot 3D point cloud semantic segmentation. Despite some progress, this task still encounters many issues due to the insufficient samples given, e.g., incomplete object segmentation and inaccurate semantic discrimination. To tackle these issues, we first leverage part-whole relationships into the task of 3D point cloud semantic segmentation to capture semantic integrity, which is empowered by the dynamic capsule routing with the module of 3D Capsule Networks (CapsNets) in the embedding network. Concretely, the dynamic routing amalgamates geometric information of the 3D point cloud data to construct higher-level feature representations, which capture the relationships between object parts and their wholes. Secondly, we designed a multi-prototype enhancement module to enhance the prototype discriminability. Specifically, the single-prototype enhancement mechanism is expanded to the multi-prototype enhancement version for capturing rich semantics. Besides, the shot-correlation within the category is calculated via the interaction of different samples to enhance the intra-category similarity. Ablation studies prove that the involved part-whole relations and proposed multi-prototype enhancement module help to achieve complete object segmentation and improve semantic discrimination. Moreover, under the integration of these two modules, quantitative and qualitative experiments on two public benchmarks, including S3DIS and ScanNet, indicate the superior performance of the proposed framework on the task of 3D point cloud semantic segmentation, compared to some state-of-the-art methods.
Source: openalex_inverted_index · Quality: baseline
Research context
OpenAlex topics: 3D Shape Modeling and Analysis · 3D Surveying and Cultural Heritage · Remote Sensing and LiDAR Applications
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
References (44)
Show source-provided references
1. JSNet++: Dynamic filters and pointwise correlation for 3D point cloud instance and semantic segmentation10.1109/TCSVT.2022.3218076 ↗
2. Learning hybrid semantic affinity for point cloud segmentation10.1109/TCSVT.2021.3132047 ↗
3. DCNet: Large-scale point cloud semantic segmentation with discriminative and efficient feature aggregation10.1109/TCSVT.2023.3239541 ↗
4. Joint global and dynamic pseudo labeling for semi-supervised point cloud sequence segmentation10.1109/TCSVT.2023.3253210 ↗
5. A multi-phase camera-lidar fusion network for 3D semantic segmentation with weak supervision10.1109/TCSVT.2023.3241641 ↗
6. Point cloud based semantic segmentation method for unmanned shuttle bus10.32604/iasc.2023.038948 ↗
7. ConvPoint: Continuous convolutions for point cloud processing10.1016/j.cag.2020.02.005 ↗
8. 3D recurrent neural networks with context fusion for point cloud semantic segmentation
9. RandLA-Net: Efficient semantic segmentation of large-scale point clouds
10. Recurrent slice networks for 3D segmentation of point clouds
11. PointNet: Deep learning on point sets for 3D classification and segmentation
12. Vinyals · Advances in Neural Information Processing Systems · 2016 · 29
13. Few-shot 3D point cloud semantic segmentation
14. Tackling background ambiguities in multi-class few-shot point cloud semantic segmentation10.1016/j.knosys.2022.109508 ↗
15. Y. Mao, Z. Guo, X. Lu, Z. Yuan and H. Guo, “Bidirectional feature globalization for few-shot semantic segmentation of 3D point cloud scenes,” arXiv preprint arXiv:2208.06671, 2022.10.1109/3DV57658.2022.00062 ↗
16. 3D semantic parsing of large-scale indoor spaces
17. Sabour · Advances in Neural Information Processing Systems · 2017 · 30
18. 3D point capsule networks
19. What makes for effective few-shot point cloud classification?
20. ScanNet: Richly-annotated 3D reconstructions of indoor scenes
21. PointNet++: Deep hierarchical feature learning on point sets in a metric space
22. Dynamic graph CNN for learning on point clouds
23. K. Zhang, M. Hao, J. Wang, C. W. de Silva and C. Fu, “Linked dynamic graph CNN: Learning on point cloud via linking hierarchical features,” arXiv preprint arXiv:1904.10014, 2019.
24. Siamese neural networks for one-shot image recognition
25. Snell · Advances in Neural Information Processing Systems · 2017 · 30
26. Few-shot semantic segmentation with prototype learning
27. Transforming auto-encoders
28. Hinton · Advances in Neural Information Processing Systems · 2018
29. Kosiorek · Advances in Neural Information Processing Systems · 2019 · 32
30. Part-object relational visual saliency
31. Disentangled capsule routing for fast part-object relational saliency10.1109/TIP.2022.3215887 ↗
32. Integrating part-object relationship and contrast for camouflaged object detection10.1109/TIFS.2021.3124734 ↗
33. Deep unsupervised part-whole relational visual saliency10.1016/j.neucom.2023.126916 ↗
34. Employing deep part-object relationships for salient object detection
35. N. Srivastava, H. Goh and R. Salakhutdinov, “Geometric capsule autoencoders for 3D point clouds,” arXiv preprint arXiv:1912.03310, 2019.
36. Quaternion equivariant capsule networks for 3D point clouds
37. Attention is all you need
38. Evolutionary multiform optimization with two-stage bidirectional knowledge transfer strategy for point cloud registration
39. Channel interaction networks for fine-grained image categorization
40. J. Yosinski, J. Clune, A. Nguyen, T. Fuchs and H. Lipson, “Understanding neural networks through deep visualization,” arXiv preprint arXiv:1506.06579, 2015.
41. X. J. Zhu and Z. Ghahramani, “Learning from labeled and unlabeled data with label propagation,” Technical Report CMU-CALD-02–107, Carnegie Mellon University, 2002.
42. Label propagation for deep semi-supervised learning
43. Crossmodal few-shot 3D point cloud semantic segmentation
44. X. L. Li and P. Liang, “Prefix-tuning: Optimizing continuous prompts for generation,” arXiv preprint arXiv:2101.00190, 2021.