Dynamic SLAM Visual Odometry Based on Instance Segmentation: A Comprehensive Review
Source abstract
Abstract
Dynamic Simultaneous Localization and Mapping (SLAM) in visual scenes is currently a major research area in fields such as robot navigation and autonomous driving. However, in the face of complex real-world environments, current dynamic SLAM systems struggle to achieve precise localization and map construction. With the advancement of deep learning, there has been increasing interest in the development of deep learning-based dynamic SLAM visual odometry in recent years, and more researchers are turning to deep learning techniques to address the challenges of dynamic SLAM. Compared to dynamic SLAM systems based on deep learning methods such as object detection and semantic segmentation, dynamic SLAM systems based on instance segmentation can not only detect dynamic objects in the scene but also distinguish different instances of the same type of object, thereby reducing the impact of dynamic objects on the SLAM system’s positioning. This article not only introduces traditional dynamic SLAM systems based on mathematical models but also provides a comprehensive analysis of existing instance segmentation algorithms and dynamic SLAM systems based on instance segmentation, comparing and summarizing their advantages and disadvantages. Through comparisons on datasets, it is found that instance segmentation-based methods have significant advantages in accuracy and robustness in dynamic environments. However, the real-time performance of instance segmentation algorithms hinders the widespread application of dynamic SLAM systems. In recent years, the rapid development of single-stage instance segmentation methods has brought hope for the widespread application of dynamic SLAM systems based on instance segmentation. Finally, possible future research directions and improvement measures are discussed for reference by relevant professionals.
Source: openalex_inverted_index · Quality: baseline
Research context
OpenAlex topics: Robotics and Sensor-Based Localization · Advanced Image and Video Retrieval Techniques · Image and Object Detection Techniques
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
References (100+)
Show source-provided references
1. A comprehensive survey of visual SLAM algorithms10.3390/robotics11010024 ↗
2. An overview on visual SLAM: From tradition to semantic10.3390/rs14133010 ↗
3. Visual SLAM algorithms: A survey from 2010 to 201610.1186/s41074-017-0027-2 ↗
4. Visual simultaneous localization and mapping: A survey10.1007/s10462-012-9365-8 ↗
5. Keyframe-based monocular SLAM: Design, survey, and future directions10.1016/j.robot.2017.09.010 ↗
6. ORB-SLAM: A versatile and accurate monocular SLAM system10.1109/TRO.2015.2463671 ↗
7. ORB-SLAM2: An open-source SLAM system for monocular, stereo, and RGB-D cameras10.1109/TRO.2017.2705103 ↗
8. ORB-SLAM3: An accurate open-source library for visual, visual-inertial, and multimap SLAM10.1109/TRO.2021.3075644 ↗
9. Direct sparse odometry10.1109/TPAMI.2017.2658577 ↗
10. Visual SLAM and structure from motion in dynamic environments: A survey10.1145/3177853 ↗
11. Visual tracking via hierarchical deep reinforcement learning10.1609/aaai.v35i4.16443 ↗
12. A unified framework for mutual improvement of SLAM and semantic segmentation
13. A comprehensive overview of dynamic visual SLAM and deep learning: Concepts, methods and challenges10.1007/s00138-022-01306-w ↗
14. Lin · Lecture Notes in Computer Science · 2014 · 8693 · 74010.1007/978-3-319-10602-1_48 ↗
15. A benchmark for the evaluation of RGB-D SLAM systems
16. Vision meets robotics: The KITTI dataset10.1177/0278364913491297 ↗
17. On the representation and estimation of spatial uncertainty10.1177/027836498600500404 ↗
18. A combined corner and edge detector
19. Machine learning for high-speed corner detection
20. Good features to track
21. Distinctive image features from scale-invariant keypoints10.1023/B:VISI.0000029664.99615.94 ↗
22. A comparison of SIFT, PCA-SIFT and SURF
23. ORB: An efficient alternative to SIFT or SURF
24. BRISK: Binary Robust invariant scalable keypoints
25. Efficient variants of the ICP algorithm
26. Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography10.1145/358669.358692 ↗
27. Lepetit · International Journal of Computer Vision · 2009 · 81 · 15510.1007/s11263-008-0152-6 ↗
28. Complete solution classification for the perspective-three-point problem10.1109/TPAMI.2003.1217599 ↗
29. DeepSFM: Structure from motion via deep bundle adjustment
30. Parallel tracking and mapping for small AR workspaces
31. DTAM: Dense tracking and mapping in real-time
32. LSD-SLAM: Large-scale direct monocular SLAM
33. SVO: Fast semi-direct monocular visual odometry
34. Moving object detection by multi-view geometric techniques from a single camera mounted robot
35. Dynamic RGB-D SLAM algorithm based on reprojection depth difference cumulative map and static probability
36. Real-time 3D reconstruction in dynamic scenes using point-based fusion
37. KinectFusion: Real-time 3D reconstruction and interaction using a moving depth camera
38. CoSLAM: Collaborative visual SLAM in dynamic environments10.1109/TPAMI.2012.104 ↗
39. Robust monocular SLAM in dynamic environments
40. Improving RGB-D SLAM in dynamic environments: A motion removal approach10.1016/j.robot.2016.11.012 ↗
41. Determining optical flow10.1016/0004-3702(81)90024-2 ↗
42. Moving object segmentation using optical flow and depth information
43. Robust odometry estimation for RGB-D cameras
44. Moving object detection in real-time using stereo from a mobile platform10.1142/S2301385015400026 ↗
45. Segmentation of moving objects by long term video analysis10.1109/TPAMI.2013.242 ↗
46. Towards dense moving object segmentation based robust dense RGB-D SLAM in dynamic scenarios
47. FlowFusion: Dynamic dense RGB-D SLAM based on optical flow
48. Deep semantic segmentation of natural and medical images: A review10.1007/s10462-020-09854-1 ↗
49. Learning to segment object candidates
50. Mask R-CNN
51. Faster R-CNN: Towards real-time object detection with region proposal networks10.1109/TPAMI.2016.2577031 ↗
52. Cascade R-CNN: High quality object detection and instance segmentation10.1109/TPAMI.2019.2956516 ↗
53. Cascade R-CNN: Delving into high quality object detection
54. Hybrid task cascade for instance segmentation
55. Path aggregation network for instance segmentation
56. Mask scoring RCNN
57. B. D. Brabandere, D. Neven and L. V. Gool, “Semantic instance segmentation with a discriminative loss function,” arXiv:1708.02551, 2017.10.1109/CVPRW.2017.66 ↗
58. A. Fathi, Z. Wojna, V. Rathod, P. Wang and H. Song, “Semantic instance segmentation via deep metric learning,” arXiv:1703.10277, 2017.
59. Recurrent pixel embedding for instance grouping
60. Instance-sensitive fully convolutional networks
61. Fully convolutional instance-aware semantic segmentation
62. Yolact: Real-time instance segmentation
63. BlendMask: Top-down meets bottom-up for instance segmentation
64. H. Ying, Z. Huang, S. Liu, T. Shao and K. Zhou, “Embedmask: Embedding coupling for one-stage instance segmentation,” arXiv:1912.01954, 2019.
65. Conditional convolutions for instance segmentation
66. YOLACT++ better real-time instance segmentation10.1109/TPAMI.2020.3014297 ↗
67. FCOS: Fully convolutional one-stage object detection
68. SOLO: Segmenting objects by locations
69. You only look once: Unified, real-time object detection
70. Multiscale combinatorial grouping
71. SOLOv2: Dynamic and fast instance segmentation
72. PointRend: Image segmentation as rendering
73. Y. Fang, S. Yang, X. Wang, Y. Li, C. Fang et al., “QueryInst: Parallelly supervised mask query for instance segmentation,” arXiv:2105.01928, 2021.
74. Bottom-up object detection by grouping extreme and center points
75. Explicit shape encoding for real-time instance segmentation
76. PolarMask: Single shot instance segmentation with polar representation
77. FourierNet: Compact mask representation for instance segmentation using differentiable shape decoders
78. Poly-YOLO: Higher speed, more precise detection and instance segmentation for YOLOv310.1007/s00521-021-05978-9 ↗
79. ContrastMask: Contrastive learning to segment every thing
80. E2EC: An end-to-end contour-based method for high-quality high-speed instance segmentation
81. Carion · 2020 · 213
82. M. Jaderberg, K. Simonyan and A. Zisserman, “Spatial transformer networks,” arXiv:1506.02025, 2015.
83. Per-pixel classification is not all you need for semantic segmentation
84. Masked-attention mask transformer for universal image segmentation
85. A. Kirillov, E. Mintun and N. Ravi, “Segment anything,” arXiv:2304.02643, 2023.10.1109/ICCV51070.2023.00371 ↗
86. Li · 2022 · 280
87. DS-SLAM: A semantic visual SLAM towards dynamic environments
88. SegNet: A deep convolutional encoder-decoder architecture for image segmentation10.1109/TPAMI.2016.2644615 ↗
89. DynaSLAM: Tracking, mapping, and inpainting in dynamic scenes10.1109/LRA.2018.2860039 ↗
90. The method of static semantic map construction based on instance segmentation and dynamic point elimination10.3390/electronics10161883 ↗
91. RDS-SLAM: Real-time dynamic SLAM using semantic segmentation methods10.1109/ACCESS.2021.3050617 ↗
92. MaskFusion: Real-time recognition, tracking and reconstruction of multiple moving objects
93. J. Zhang, M. Henein, R. Mahony and V. Ila, “VDO-SLAM: A visual dynamic object-aware SLAM system,” arXiv:2005.11052, 2020.
94. A real-time dynamic object segmentation framework for SLAM system in dynamic scenes
95. A visual SLAM approach based on optical flow and instance segmentation in dynamic scenes
96. MaskFlownet: Asymmetric feature matching with learnable occlusion mask
97. An enhanced segmentation-based RGB-D SLAM method for dynamic environments
98. The optimized K-means algorithms for improving randomly-initialed midpoints
99. VSLAM based on instance segmentation
100. SOLO-SLAM: A parallel semantic SLAM algorithm for dynamic scenes10.3390/s22186977 ↗