SemABC: Semantic-Guided Adaptive Bias Calibration for Generative Zero-Shot Point Cloud Segmentation
Source abstract
Abstract
Due to the limited quantity and high cost of high-quality three-dimensional annotations, generalized zero-shot point cloud segmentation aims to transfer the knowledge of seen to unseen classes by leveraging semantic correlations to achieve generalization purposes. Existing generative point cloud semantic segmentation approaches rely on generators trained on seen classes to synthesize visual features for unseen classes in order to help the segmentation model gain the ability of generalization, but this often leads to a bias toward seen classes. To address this issue, we propose a semantic-guided adaptive bias calibration approach with a dual-branch network architecture. This network consists of a novel visual–semantic fusion branch alongside the primary segmentation branch to suppress the bias toward seen classes. Specifically, the visual–semantic branch exploits the visual–semantic relevance of the synthetic features of unseen classes to provide auxiliary predictions. Furthermore, we introduce an adaptive bias calibration module that dynamically integrates the predictions from both the main and auxiliary branches to achieve unbiased segmentation results. Extensive experiments conducted on standard benchmarks demonstrate that our approach significantly outperforms state-of-the-art methods on both seen and unseen classes, thereby validating the effectiveness of our approach.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: 3D Shape Modeling and Analysis · 3D Surveying and Cultural Heritage · Image Processing and 3D Reconstruction
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
School of Information Science and Engineering, Shandong Normal University, Jinan 250358, China
Funding
National Natural Science Foundation of China · Joint Funds of Natural Science Foundation of Shandong Province
References (48)
Show source-provided references
1. A framework for robotic grasping of 3D objects in a tabletop environment10.1007/s11042-024-20178-y ↗
2. Robot-assisted mobile scanning for automated 3D reconstruction and point cloud semantic segmentation of building interiors10.1016/j.autcon.2023.104949 ↗
3. Huang, C., Mees, O., Zeng, A., and Burgard, W. (June, January 29). Visual language maps for robot navigation. Proceedings of the 2023 IEEE International Conference on Robotics and Automation (ICRA), London, UK.10.1109/ICRA48891.2023.10160969 ↗
4. Motion planning and control for mobile robot navigation using machine learning: A survey10.1007/s10514-022-10039-8 ↗
5. Arena, F., Collotta, M., Pau, G., and Termine, F. (2022). An overview of augmented reality. Computers, 11.10.3390/computers11020028 ↗
6. Active self-training for weakly supervised 3D scene semantic segmentation10.1007/s41095-022-0311-7 ↗
7. Exploring high-contrast areas context for 3D point cloud segmentation via MLP-driven Discrepancy mechanism10.1016/j.cag.2025.104222 ↗
8. Semantic segmentation-assisted instance feature fusion for multi-level 3D part instance segmentation10.1007/s41095-022-0300-x ↗
9. Zero-shot learning on 3d point cloud objects and beyond10.1007/s11263-022-01650-4 ↗
10. Xu, W., Xu, R., Wang, C., Xu, S., Guo, L., Zhang, M., and Zhang, X. (2024, January 26–27). Spectral prompt tuning: Unveiling unseen classes for zero-shot semantic segmentation. Proceedings of the AAAI Conference on Artificial Intelligence, Vancouver, BC, Canada.10.1609/aaai.v38i6.28456 ↗
11. Zhang, Y., Guo, M.H., Wang, M., and Hu, S.M. (2024, January 16–22). Exploring regional clues in CLIP for zero-shot semantic segmentation. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.10.1109/CVPR52733.2024.00315 ↗
12. Transductive learning with prior knowledge for generalized zero-shot action recognition10.1109/TCSVT.2023.3284977 ↗
13. Ge, C., Wang, J., Qi, Q., Sun, H., Xu, T., and Liao, J. (2023, January 8–10). Semi-transductive learning for generalized zero-shot sketch-based image retrieval. Proceedings of the AAAI Conference on Artificial Intelligence, Montréal, QC, Canada.10.1609/aaai.v37i6.25931 ↗
14. Transductive zero-shot learning with generative model-driven structure alignment10.1016/j.patcog.2024.110561 ↗
15. ISAnWin: Inductive generalized zero-shot learning using deep CNN for malware detection across windows and android platforms10.7717/peerj-cs.2604 ↗
16. Chen, S., Wang, W., Xia, B., Peng, Q., You, X., Zheng, F., and Shao, L. (2021, January 11–17). Free: Feature refinement for generalized zero-shot learning. Proceedings of the IEEE/CVF International Conference on Computer Vision, Montreal, BC, Canada.10.1109/ICCV48922.2021.00019 ↗
17. Prototype-augmented self-supervised generative network for generalized zero-shot learning10.1109/TIP.2024.3351439 ↗
18. Michele, B., Boulch, A., Puy, G., Bucher, M., and Marlet, R. (2021, January 1–3). Generative zero-shot learning for semantic segmentation of 3d point clouds. Proceedings of the 2021 International Conference on 3D Vision (3DV), London, UK.10.1109/3DV53792.2021.00107 ↗
19. Yang, Y., Hayat, M., Jin, Z., Zhu, H., and Lei, Y. (2023, January 1–6). Zero-shot point cloud segmentation by semantic-visual aware synthesis. Proceedings of the IEEE/CVF International Conference on Computer Vision, Paris, France.10.1109/ICCV51070.2023.01064 ↗
20. Li, Y., Swersky, K., and Zemel, R. (2015, January 7–9). Generative moment matching networks. Proceedings of the International Conference on Machine Learning. PMLR, Lille, France.
21. Odena, A., Olah, C., and Shlens, J. (2017, January 6–11). Conditional image synthesis with auxiliary classifier gans. Proceedings of the International Conference on Machine Learning. PMLR, Sydney, Australia.
22. Wei, Z., Chen, P., Yu, X., Li, G., Jiao, J., and Han, Z. (2024, January 16–22). Semantic-aware SAM for point-prompted instance segmentation. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.10.1109/CVPR52733.2024.00344 ↗
23. Zheng, X., Lyu, Y., and Wang, L. Learning modality-agnostic representation for semantic segmentation from any modalities. Proceedings of the European Conference on Computer Vision.
24. Li, H., Zhang, D., Dai, Y., Liu, N., Cheng, L., Li, J., Wang, J., and Han, J. (2024, January 16–22). Gp-nerf: Generalized perception nerf for context-aware 3d scene understanding. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.10.1109/CVPR52733.2024.02051 ↗
25. Zhai, X., Han, W., Li, X., and Huang, S. (2024, January 19–21). PLGCA: A Progressive Local-Global Context-Aware Semantic Segmentation Network for Crop Remote Sensing Mapping. Proceedings of the 2024 6th International Conference on Electronics and Communication, Network and Computer Technology (ECNCT), Guangzhou, China.10.1109/ECNCT63103.2024.10704428 ↗
26. Qi, C.R., Su, H., Mo, K., and Guibas, L.J. (2017, January 21–26). Pointnet: Deep learning on point sets for 3d classification and segmentation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.
27. Pointnet++: Deep hierarchical feature learning on point sets in a metric space
28. Thomas, H., Qi, C.R., Deschaud, J.E., Marcotegui, B., Goulette, F., and Guibas, L.J. (November, January 27). Kpconv: Flexible and deformable convolution for point clouds. Proceedings of the IEEE/CVF International Conference on Computer Vision, Seoul, Republic of Korea.
29. Hu, Q., Yang, B., Xie, L., Rosa, S., Guo, Y., Wang, Z., Trigoni, N., and Markham, A. (2020, January 13–19). Randla-net: Efficient semantic segmentation of large-scale point clouds. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.10.1109/CVPR42600.2020.01112 ↗
30. Zhou, Z., Lei, Y., Zhang, B., Liu, L., and Liu, Y. (2023, January 17–24). Zegclip: Towards adapting clip for zero-shot semantic segmentation. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, BC, Canada.10.1109/CVPR52729.2023.01075 ↗
31. Tian, J., Aggarwal, L., Colaco, A., Kira, Z., and Gonzalez-Franco, M. (2024, January 16–22). Diffuse attend and segment: Unsupervised zero-shot segmentation using stable diffusion. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.10.1109/CVPR52733.2024.00341 ↗
32. Cheraghian, A., Rahman, S., and Petersson, L. (2019, January 27–31). Zero-shot learning of 3d point cloud objects. Proceedings of the 2019 16th International Conference on Machine Vision Applications (MVA), Tokyo, Japan.10.23919/MVA.2019.8758063 ↗
33. Cheraghian, A., Rahman, S., Campbell, D., and Petersson, L. (2019). Mitigating the hubness problem for zero-shot learning of 3d objects. arXiv.10.23919/MVA.2019.8758063 ↗
34. Cheraghian, A., Rahman, S., Campbell, D., and Petersson, L. (2020, January 1–5). Transductive zero-shot learning for 3d point cloud classification. Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, Snowmass, CO, USA.10.1109/WACV45572.2020.9093545 ↗
35. Chao, W.L., Changpinyo, S., Gong, B., and Sha, F. (2016, January 11–14). An empirical study and analysis of generalized zero-shot learning for object recognition in the wild. Proceedings of the Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands. Proceedings, Part II 14.10.1007/978-3-319-46475-6_4 ↗
36. Multi-domain feature-enhanced attribute updater for generalized zero-shot learning10.1007/s00521-025-11005-y ↗
37. Pennington, J., Socher, R., and Manning, C.D. (2014, January 25–29). Glove: Global vectors for word representation. Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), Doha, Qatar.10.3115/v1/D14-1162 ↗
38. Distributed representations of words and phrases and their compositionality
39. Dai, A., Chang, A.X., Savva, M., Halber, M., Funkhouser, T., and Nießner, M. (2017, January 21–26). Scannet: Richly-annotated 3d reconstructions of indoor scenes. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.10.1109/CVPR.2017.261 ↗
40. Armeni, I., Sener, O., Zamir, A.R., Jiang, H., Brilakis, I., Fischer, M., and Savarese, S. (2016, January 27–30). 3d semantic parsing of large-scale indoor spaces. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.10.1109/CVPR.2016.170 ↗
41. Behley, J., Garbade, M., Milioto, A., Quenzel, J., Behnke, S., Stachniss, C., and Gall, J. (November, January 27). Semantickitti: A dataset for semantic scene understanding of lidar sequences. Proceedings of the IEEE/CVF International Conference on Computer Vision, Seoul, Republic of Korea.
42. Devise: A deep visual-semantic embedding model
43. Boulch, A., Puy, G., and Marlet, R. (4, January 30). FKAConv: Feature-kernel alignment for point cloud convolution. Proceedings of the Asian Conference on Computer Vision, Kyoto, Japan.
44. ConvPoint: Continuous convolutions for point cloud processing10.1016/j.cag.2020.02.005 ↗
45. Predictive uncertainty estimation via prior networks
46. Energy-based out-of-distribution detection
47. Seo, S., Seo, P.H., and Han, B. (2019, January 15–20). Learning for single-shot confidence calibration in deep neural networks through stochastic inferences. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.10.1109/CVPR.2019.00924 ↗
48. Tang, Y., Lin, Z., Wang, Q., Zhu, P., and Hu, Q. (2024, January 16–22). Amu-tuning: Effective logit bias for clip-based few-shot learning. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.10.1109/CVPR52733.2024.02201 ↗