Sem-SLAM: Semantic-Integrated SLAM Approach for 3D Reconstruction
Source abstract
Abstract
Under the upsurge of research on the integration of Simultaneous Localization and Mapping (SLAM) and neural implicit representation, existing methods exhibit obvious limitations in terms of environmental semantic parsing and scene understanding capabilities. In response to this, this paper proposes a SLAM system that integrates a full attention mechanism and a multi-scale information extractor. This system constructs a more accurate 3D environmental model by fusing semantic, shape, and geometric orientation features. Meanwhile, to deeply excavate the semantic information in images, a pre-trained frozen 2D segmentation algorithm is employed to extract semantic features, providing a powerful support for 3D environmental reconstruction. Furthermore, a multi-layer perceptron and interpolation techniques are utilized to extract multi-scale features, distinguishing information at different scales. This enables the effective decoding of semantic, RGB, and Truncated Signed Distance Field (TSDF) values from the fused features, achieving high-quality information rendering. Experimental results demonstrate that this method significantly outperforms the baseline-based methods in terms of mapping and tracking accuracy on the Replica and ScanNet datasets. It also shows superior performance in semantic segmentation and real-time semantic mapping tasks, offering a new direction for the development of SLAM technology.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Robotics and Sensor-Based Localization · 3D Shape Modeling and Analysis · 3D Surveying and Cultural Heritage
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Key Laboratory of IoT Monitoring and Early Warning, Ministry of Emergency Management, School of Intelligent Engieering and Automation, Beijing University of Posts and Telecommunications, Beijing 100876, China
Funding
National Key R&D Program of China
References (26)
Show source-provided references
1. Li, M., Ma, Y., and Qiu, Q. (2023, January 5–8). SemanticSLAM: Learning based Semantic Map Construction and Robust Camera Localization. Proceedings of the 2023 IEEE Symposium Series on Computational Intelligence (SSCI), Orlando, FL, USA.10.1109/SSCI52147.2023.10371814 ↗
2. Sha, Y., Zhu, S., Guo, H., Wang, Z., and Wang, H. (2024). Towards Autonomous Indoor Parking: A Globally Consistent Semantic SLAM System and A Semantic Localization Subsystem. arXiv.
3. Nerf: Representing scenes as neural radiance fields for view synthesis10.1145/3503250 ↗
4. Mao, Y., Yu, X., Wang, K., Wang, Y., Xiong, R., and Liao, Y. (2023). NGEL-SLAM: Neural Implicit Representation-based Global Consistent Low-Latency SLAM System. arXiv.10.1109/ICRA57147.2024.10611269 ↗
5. Tao, Y., Bhalgat, Y., Fu, L.F.T., Mattamala, M., Chebrolu, N., and Fallon, M.F. (2024, January 13–17). SiLVR: Scalable Lidar-Visual Reconstruction with Neural Radiance Fields for Robotic Inspection. Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), Yokohama, Japan.10.1109/ICRA57147.2024.10611278 ↗
6. Straub, J., Whelan, T., Ma, L., Chen, Y., Wijmans, E., Green, S., Engel, J.J., Mur-Artal, R., Ren, C., and Verma, S. (2019). The Replica dataset: A digital replica of indoor spaces. arXiv.
7. Dai, A., Chang, A.X., Savva, M., Halber, M., Funkhouser, T., and Nießner, M. (2017, January 21–26). Scannet: Richly-annotated 3d reconstructions of indoor scenes. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.10.1109/CVPR.2017.261 ↗
8. Ming, Y., Ye, W., and Calway, A. (2022). iDF-SLAM: End-to-End RGB-D SLAM with Neural Implicit Mapping and Deep Feature Tracking. arXiv.
9. Yang, F., Wang, Y., Tan, L., Li, M., Shan, H., and Liao, P. (2025, January 25–28). DNIV-SLAM: Dynamic Neural Implicit Volumetric SLAM. Proceedings of the Chinese Conference on Pattern Recognition and Computer Vision (PRCV), Singapore.
10. Xin, Z., Yue, Y., Zhang, L., and Wu, C. (2024, January 13–17). HERO-SLAM: Hybrid Enhanced Robust Optimization of Neural SLAM. Proceedings of the 2024 IEEE International Conference on Robotics and Automation (ICRA), Yokohama, Japan.10.1109/ICRA57147.2024.10610000 ↗
11. Li, H., Gu, X., Yuan, W., Yang, L., Dong, Z., and Tan, P. (2023, January 1–5). Dense RGB SLAM With Neural Implicit Maps. Proceedings of the International Conference on Learning Representations (ICLR), Kigali, Rwanda.
12. Wei, W., Wang, J., Deng, S., and Liu, J. (2024). DF-SLAM: Dictionary Factors Representation for High-Fidelity Neural Implicit Dense Visual SLAM System. arXiv.
13. Zhu, Z., Peng, S., Larsson, V., Xu, W., Bao, H., Cui, Z., Oswald, M.R., and Pollefeys, M. (2022, January 18–24). Nice-slam: Neural implicit scalable encoding for slam. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA.10.1109/CVPR52688.2022.01245 ↗
14. Zhang, Y., Tosi, F., Mattoccia, S., and Poggi, M. (2023, January 2–6). GO-SLAM: Global Optimization for Consistent 3D Instant Reconstruction. Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), Paris, France.10.1109/ICCV51070.2023.00345 ↗
15. Qu, D., Yan, C., Wang, D., Yin, J., Chen, Q., Xu, D., Zhang, Y., Zhao, B., and Li, X. (2024, January 17–21). Implicit Event-RGBD Neural SLAM. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.10.1109/CVPR52733.2024.01852 ↗
16. Li, M., Zhou, Y., Jiang, G., Deng, T., Wang, Y., and Wang, H. (2024). DDN-SLAM: Real-time Dense Dynamic Neural Implicit SLAM. arXiv.10.1109/LRA.2025.3546130 ↗
17. Zhu, S., Wang, G., Blum, H., Liu, J., Song, L., Pollefeys, M., and Wang, H. (2024, January 17–21). SNI-SLAM: Semantic Neural Implicit SLAM. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.10.1109/CVPR52733.2024.02000 ↗
18. Li, G., Chen, Q., Yan, Y., and Pu, J. (2024). EC-SLAM: Real-time Dense Neural RGB-D SLAM System with Effectively Constrained Global Bundle Adjustment. arXiv.
19. Cartillier, V., Schindler, G., and Essa, I. (2024, January 17–21). SLAIM: Robust Dense Neural SLAM for Online Tracking and Mapping. Proceedings of the 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Seattle, WA, USA.10.1109/CVPRW63382.2024.00292 ↗
20. Hu, J., Chen, X., Feng, B., Li, G., Yang, L., Bao, H., Zhang, G., and Cui, Z. (2025, January 21–25). Cg-slam: Efficient dense rgb-d slam in a consistent uncertainty-aware 3d gaussian field. Proceedings of the European Conference on Computer Vision (ECCV), Zurich, Switzerland.10.1007/978-3-031-72698-9_6 ↗
21. Automated fault diagnosis detection of air handling units using real operational labelled data and transformer-based methods at 24-hour operation hospital10.1016/j.buildenv.2025.113257 ↗
22. Oquab, M., Darcet, T., Moutakanni, T., Vo, H., Szafraniec, M., Khalidov, V., Fernandez, P., Haziza, D., Massa, F., and El-Nouby, A. (2023). Dinov2: Learning robust visual features without supervision. arXiv.
23. Johari, M.M., Carta, C., and Fleuret, F. (2023, January 18–22). Eslam: Efficient dense slam system based on hybrid representation of signed distance fields. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Vancouver, BC, Canada.10.1109/CVPR52729.2023.01670 ↗
24. Yang, X., Li, H., Zhai, H., Ming, Y., Liu, Y., and Zhang, G. (2022, January 17–21). Vox-Fusion: Dense tracking and mapping with voxel-based neural implicit representation. Proceedings of the IEEE International Symposium on Mixed and Augmented Reality (ISMAR), Singapore.10.1109/ISMAR55827.2022.00066 ↗
25. NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstruction
26. Haghighi, Y., Kumar, S., Thiran, J.P., and Gool, L.V. (2023). Neural Implicit Dense Semantic SLAM. arXiv.