journal article·2023-02-20Open access

An Improved Visual SLAM Based on Map Point Reliability under Dynamic Environments

Jianjun Ni, Li Wang, Xiaotian Wang, Guangyi Tang

Applied Sciences· Volume 13· Issue 4

Abstract

The visual simultaneous localization and mapping (SLAM) method under dynamic environments is a hot and challenging issue in the robotic field. The oriented FAST and Rotated BRIEF (ORB) SLAM algorithm is one of the most effective methods. However, the traditional ORB-SLAM algorithm cannot perform well in dynamic environments due to the feature points of dynamic map points at different timestamps being incorrectly matched. To deal with this problem, an improved visual SLAM method built on ORB-SLAM3 is proposed in this paper. In the proposed method, an improved new map points screening strategy and the repeated exiting map points elimination strategy are presented and combined to identify obvious dynamic map points. Then, a concept of map point reliability is introduced in the ORB-SLAM3 framework. Based on the proposed reliability calculation of the map points, a multi-period check strategy is used to identify the unobvious dynamic map points, which can further deal with the dynamic problem in visual SLAM, for those unobvious dynamic objects. Finally, various experiments are conducted on the challenging dynamic sequences of the TUM RGB-D dataset to evaluate the performance of our visual SLAM method. The experimental results demonstrate that our SLAM method can run at an average time of 17.51 ms per frame. Compared with ORB-SLAM3, the average RMSE of the absolute trajectory error (ATE) of the proposed method in nine dynamic sequences of the TUM RGB-D dataset can be reduced by 63.31%. Compared with the real-time dynamic SLAM methods, the proposed method can obtain state-of-the-art performance. The results prove that the proposed method is a real-time visual SLAM, which is effective in dynamic environments.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: Robotics and Sensor-Based Localization · Robotic Path Planning Algorithms · Advanced Image and Video Retrieval Techniques

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

College of Internet of Things Engineering, Hohai University, Changzhou 213022, China

School of Artificial Intelligence, Hohai University, Changzhou 213022, China

Funding

National Natural Science Foundation of China · Science and Technology Support Program of Changzhou

References (52)

Show source-provided references
1. Improved visual SLAM: A novel approach to mapping and localization using visual landmarks in consecutive frames10.1007/s11042-017-4694-x ↗
2. Visual simultaneous localization and mapping: A survey10.1007/s10462-012-9365-8 ↗
3. An improved vision-based SLAM approach inspired from animal spatial cognition
4. Dynamic EKF-based SLAM for autonomous mobile convergence platforms10.1007/s11042-014-2093-0 ↗
5. A bioinspired neural model based extended Kalman filter for robot SLAM10.1155/2014/905826 ↗
6. Research on SLAM Road Sign Observation Based on Particle Filter
7. SBA: A software package for generic sparse bundle adjustment10.1145/1486525.1486527 ↗
8. Visual SLAM algorithms: A survey from 2010 to 201610.1186/s41074-017-0027-2 ↗
9. Local optimized and scalable frame-to-model SLAM10.1007/s11042-015-2780-5 ↗
10. Gaze Control-Based Navigation Architecture With a Situation-Specific Preference Approach for Humanoid Robots10.1109/TMECH.2014.2382633 ↗
11. Ni, J., Chen, Y., Chen, Y., Zhu, J., Ali, D., and Cao, W. (2020). A Survey on Theories and Applications for Self-Driving Cars Based on Deep Learning Methods. Appl. Sci., 10.10.3390/app10082749 ↗
12. Robust and fast collaborative augmented reality framework based on monocular SLAM10.5573/IEIESPC.2020.9.4.325 ↗
13. SVO: Semidirect visual odometry for monocular and multicamera systems10.1109/TRO.2016.2623335 ↗
14. An Ensemble of Spatial Clustering and Temporal Error Profile Based Dynamic Point Removal for visual Odometry10.1007/s11042-022-12063-3 ↗
15. Kümmerle, R., Grisetti, G., Strasdat, H., Konolige, K., and Burgard, W. (2011, January 9–13). G2o: A general framework for graph optimization. Proceedings of the 2011 IEEE International Conference on Robotics and Automation, Shanghai, China.
16. ORB-SLAM3: An Accurate Open-Source Library for Visual, Visual–Inertial, and Multimap SLAM10.1109/TRO.2021.3075644 ↗
17. An Improved Deep Residual Network-Based Semantic Simultaneous Localization and Mapping Method for Monocular Vision Robot10.1155/2020/7490840 ↗
18. Engel, J., Schöps, T., and Cremers, D. (2014, January 6–12). LSD-SLAM: Large-scale direct monocular SLAM. Proceedings of the 13th European Conference on Computer Vision, ECCV 2014, Zurich, Switzerland.10.1007/978-3-319-10605-2_54 ↗
19. Gao, X., Wang, R., Demmel, N., and Cremers, D. (2018, January 1–5). LDSO: Direct sparse odometry with loop closure. Proceedings of the 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2018, Madrid, Spain.10.1109/IROS.2018.8593376 ↗
20. A Variable Radius Side Window Direct SLAM Method Based on Semantic Information
21. Direct sparse odometry10.1109/TPAMI.2017.2658577 ↗
22. GMSK-SLAM: A new RGB-D SLAM method with dynamic areas detection towards dynamic environments10.1007/s11042-021-11168-5 ↗
23. RDS-SLAM: Real-Time Dynamic SLAM Using Semantic Segmentation Methods10.1109/ACCESS.2021.3050617 ↗
24. He, K., Gkioxari, G., Dollár, P., and Girshick, R. (2017, January 22–29). Mask R-CNN. Proceedings of the 2017 IEEE International Conference on Computer Vision (ICCV), Venice, Italy.10.1109/ICCV.2017.322 ↗
25. SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation10.1109/TPAMI.2016.2644615 ↗
26. An Improved Deep Network-Based Scene Classification Method for Self-Driving Cars10.1109/TIM.2022.3146923 ↗
27. Kim, J.H., Kim, N., Park, Y.W., and Won, C.S. (2022). Object Detection and Classification Based on YOLO-V5 with Improved Maritime Dataset. J. Mar. Sci. Eng., 10.10.3390/jmse10030377 ↗
28. DynaSLAM: Tracking, Mapping, and Inpainting in Dynamic Scenes10.1109/LRA.2018.2860039 ↗
29. Strecke, M., and Stueckler, J. (November, January 27). EM-Fusion: Dynamic Object-Level SLAM With Probabilistic Data Association. Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision (ICCV), Seoul, Republic of Korea.10.1109/ICCV.2019.00596 ↗
30. Yu, C., Liu, Z., Liu, X.J., Xie, F., Yang, Y., Wei, Q., and Fei, Q. (2018, January 1–5). DS-SLAM: A Semantic Visual SLAM towards Dynamic Environments. Proceedings of the 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Madrid, Spain.10.1109/IROS.2018.8593691 ↗
31. Zhong, F., Wang, S., Zhang, Z., Chen, C., and Wang, Y. (2018, January 12–15). Detect-SLAM: Making object detection and SLAM mutually beneficial. Proceedings of the 2018 IEEE Winter Conference on Applications of Computer Vision (WACV), Lake Tahoe, NV, USA.10.1109/WACV.2018.00115 ↗
32. Huang, J., Yang, S., Mu, T.J., and Hu, S.M. (2020, January 13–19). ClusterVO: Clustering Moving Instances and Estimating Visual Odometry for Self and Surroundings. Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.10.1109/CVPR42600.2020.00224 ↗
33. YOLO-SLAM: A semantic SLAM system towards dynamic environment with geometric constraint10.1007/s00521-021-06764-3 ↗
34. Runz, M., Buffier, M., and Agapito, L. (2018, January 16–20). MaskFusion: Real-Time Recognition, Tracking and Reconstruction of Multiple Moving Objects. Proceedings of the 2018 IEEE International Symposium on Mixed and Augmented Reality (ISMAR), Munich, Germany.10.1109/ISMAR.2018.00024 ↗
35. Xu, B., Li, W., Tzoumanikas, D., Bloesch, M., Davison, A., and Leutenegger, S. (2019, January 20–24). MID-Fusion: Octree-based Object-Level Multi-Instance Dynamic SLAM. Proceedings of the 2019 International Conference on Robotics and Automation (ICRA), Montreal, QC, Canada.10.1109/ICRA.2019.8794371 ↗
36. CubeSLAM: Monocular 3-D Object SLAM10.1109/TRO.2019.2909168 ↗
37. Alcantarilla, P.F., Yebes, J.J., Almazán, J., and Bergasa, L.M. (2012, January 14–18). On combining visual SLAM and dense scene flow to increase the robustness of localization and mapping in dynamic environments. Proceedings of the 2012 IEEE International Conference on Robotics and Automation, Saint Paul, MN, USA.10.1109/ICRA.2012.6224690 ↗
38. RGB-D SLAM in Dynamic Environments Using Static Point Weighting10.1109/LRA.2017.2724759 ↗
39. Kerl, C., Sturm, J., and Cremers, D. (2013, January 6–10). Robust odometry estimation for RGB-D cameras. Proceedings of the 2013 IEEE International Conference on Robotics and Automation, Karlsruhe, Germany.10.1109/ICRA.2013.6631104 ↗
40. Effective Background Model-Based RGB-D Dense Visual Odometry in a Dynamic Environment10.1109/TRO.2016.2609395 ↗
41. Improving RGB-D SLAM in dynamic environments: A motion removal approach10.1016/j.robot.2016.11.012 ↗
42. RGB-D SLAM in Dynamic Environments Using Point Correlations10.1109/TPAMI.2020.3010942 ↗
43. CoSLAM: Collaborative Visual SLAM in Dynamic Environments10.1109/TPAMI.2012.104 ↗
44. Zhang, T., Zhang, H., Li, Y., Nakamura, Y., and Zhang, L. (August, January 31). FlowFusion: Dynamic Dense RGB-D SLAM Based on Optical Flow. Proceedings of the 2020 IEEE International Conference on Robotics and Automation (ICRA), Paris, France.10.1109/ICRA40945.2020.9197349 ↗
45. Palazzolo, E., Behley, J., Lottes, P., Giguère, P., and Stachniss, C. (2019, January 3–8). ReFusion: 3D Reconstruction in Dynamic Environments for RGB-D Cameras Exploiting Residuals. Proceedings of the 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Macau, China.10.1109/IROS40897.2019.8967590 ↗
46. A Point-Line VIO System with Novel Feature Hybrids and with Novel Line Predicting-Matching10.1109/LRA.2021.3113987 ↗
47. Bags of Binary Words for Fast Place Recognition in Image Sequences10.1109/TRO.2012.2197158 ↗
48. Sturm, J., Engelhard, N., Endres, F., Burgard, W., and Cremers, D. (2012, January 7–12). A benchmark for the evaluation of RGB-D SLAM systems. Proceedings of the 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems, Vilamoura-Algarve, Portugal.10.1109/IROS.2012.6385773 ↗
49. Sun, H., Wang, P., Ni, C., and Li, J.m. (2022). Loop closure detection based on image semantic feature and bag-of-words. Multimed. Tools Appl.10.1007/s11042-022-13353-6 ↗
50. Afifi, A., Takada, C., Yoshimura, Y., and Nakaguchi, T. (2021). Real-time expanded field-of-view for minimally invasive surgery using multi-camera visual simultaneous localization and mapping. Sensors, 21.10.3390/s21062106 ↗
51. Scona, R., Jaimez, M., Petillot, Y.R., Fallon, M., and Cremers, D. (2018, January 21–25). StaticFusion: Background Reconstruction for Dense RGB-D SLAM in Dynamic Environments. Proceedings of the 2018 IEEE International Conference on Robotics and Automation (ICRA), Brisbane, QLD, Australia.10.1109/ICRA.2018.8460681 ↗
52. Rünz, M., and Agapito, L. (June, January 29). Co-fusion: Real-time segmentation, tracking and fusion of multiple objects. Proceedings of the 2017 IEEE International Conference on Robotics and Automation (ICRA), Singapore.10.1109/ICRA.2017.7989518 ↗