Visual SLAM Based on Object Detection Network: A Review
Source abstract
Abstract
Visual simultaneous localization and mapping (SLAM) is crucial in robotics and autonomous driving. However, traditional visual SLAM faces challenges in dynamic environments. To address this issue, researchers have proposed semantic SLAM, which combines object detection, semantic segmentation, instance segmentation, and visual SLAM. Despite the growing body of literature on semantic SLAM, there is currently a lack of comprehensive research on the integration of object detection and visual SLAM. Therefore, this study aims to gather information from multiple databases and review relevant literature using specific keywords. It focuses on visual SLAM based on object detection, covering different aspects. Firstly, it discusses the current research status and challenges in this field, highlighting methods for incorporating semantic information from object detection networks into mileage measurement, closed-loop detection, and map construction. It also compares the characteristics and performance of various visual SLAM object detection algorithms. Lastly, it provides an outlook on future research directions and emerging trends in visual SLAM. Research has shown that visual SLAM based on object detection has significant improvements compared to traditional SLAM in dynamic point removal, data association, point cloud segmentation, and other technologies. It can improve the robustness and accuracy of the entire SLAM system and can run in real time. With the continuous optimization of algorithms and the improvement of hardware level, object visual SLAM has great potential for development.
Source: openalex_inverted_index · Quality: baseline
Research context
OpenAlex topics: Robotics and Sensor-Based Localization · Robotic Path Planning Algorithms · 3D Surveying and Cultural Heritage
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
References (98)
Show source-provided references
1. SLAM-based augmented reality system in interactive exhibition
2. Real-time visual–Inertial SLAM based on adaptive keyframe selection for mobile AR applications10.1109/TMM.2019.2913324 ↗
3. Stereo vision-based semantic 3D object and ego-motion tracking for autonomous driving
4. An intelligent logistics navigation device based on SLAM dual radar
5. Visual SLAM for handheld monocular endoscope10.1109/TMI.2013.2282997 ↗
6. M. Turan, Y. Almalioglu, H. Gilbert, H. Araujo, E. Konukoglu et al., “Magnetic-visual sensor fusion based medical SLAM for endoscopic capsule robot,” 2017. [Online]. Available: https://arxiv.org/abs/1705.06196 (accessed on 05/04/2023).
7. Target localization in local densemapping using RGBD SLAM and object detection10.1002/cpe.6655 ↗
8. Optimization and path planning of simultaneous localization and mapping construction based on binocular stereo vision10.18494/SAM3474 ↗
9. Fusion of IMU and vision for absolute scale estimation in monocular SLAM10.1007/s10846-010-9490-z ↗
10. Rich feature hierarchies for accurate object detection and semantic segmentation
11. Fast R-CNN
12. Faster R-CNN: Towards real-time object detection with region proposal networks10.1109/TPAMI.2016.2577031 ↗
13. U-Net: Convolutional networks for biomedical image segmentation
14. A. Kendal, V. Badrinarayanan and R. Cipolla, “Bayesian SegNet: Model uncertainty in deep convolutional encoder-decoder architectures for scene understanding,” 2015. [Online]. Available: https://arxiv.org/abs/1511.02680 (accessed on 05/04/2023).
15. Mask R-CNN
16. Moving object detection for camera pose estimation in dynamic environments
17. YOLOv3: An incremental improvement
18. Dynamic scene semantics SLAM based on semantic segmentation10.1109/ACCESS.2020.2977684 ↗
19. J. Y. Ko, H. Wang and L. Xie, “Multi-modal semantic SLAM for complex dynamic environments,” 2022. [Online]. Available: https://arxiv.org/abs/2205.04300 (accessed on 05/04/2023).
20. A survey on visual SLAM based on deep learning
21. K. Chen, J. Zhang, J. Liu, Q. Tong, R. Liu et al., “Semantic visual simultaneous localization and mapping: A survey,” 2022. [Online]. Available: https://arxiv.org/abs/2209.06428 (accessed on 05/04/2023).
22. A survey of image semantics-based visual simultaneous localization and mapping: Application-oriented solutions to autonomous navigation of mobile robots10.1177/1729881420919185 ↗
23. Survey of simultaneous localization and mapping based on environmental semantic information
24. Advances in visual SLAM research
25. ORB-SLAM2: An open-source SLAM system for monocular, stereo, and RGB-D cameras10.1109/TRO.2017.2705103 ↗
26. DM-VIO: Delayed marginalization visual-inertial odometry10.1109/LRA.2021.3140129 ↗
27. Direct sparse odometry10.1109/TPAMI.2017.2658577 ↗
28. ORB-SLAM3: An accurate open-source library for visual, visual-inertial, and multimap SLAM10.1109/TRO.2021.3075644 ↗
29. VINS-Mono: A robust and versatile monocular visual-inertial state estimator10.1109/TRO.2018.2853729 ↗
30. Tightly coupled SLAM system based on multi-sensor fusion
31. S. Urban, J. Leitloff and S. Hinz, “Mlpnp-a real-time maximum likelihood solution to the perspective-n-point problem,” 2016. [Online]. Available: https://arxiv.org/abs/1607.08112 (accessed on 05/04/2023).
32. You only look once: Unifified, real-time object detection
33. YOLO9000: Better, faster, stronger
34. C. Li, L. Li, H. Jiang, K. Weng, Y. Geng et al., “YOLOv6: A single-stage object detection framework for industrial applications,” 2022. [Online]. Available: https://arxiv.org/abs/2209.02976 (accessed on 10/04/2023).
35. SSD: Single shot multibox detector
36. DetectorNet: Transformer-enhanced spatial temporal graph neural network for traffic prediction
37. C. Wang, A. Bochkovskiy and H. Liao, “YOLOv7: Trainable bag-offreebies sets new state-of-the-art for real-time object detectors,” 2022. [Online]. Available: https://arxiv.org/abs/2207.02696 (accessed on 10/04/2023).10.1109/CVPR52729.2023.00721 ↗
38. Detect-SLAM: Making object detection and SLAM mutually beneficial
39. Improved visual odometry based on SSD algorithm in dynamic environment
40. Object aware RGBD SLAM in dynamic environments
41. Study on SLAM algorithm based on object detection in dynamic scene
42. Visual SLAM based on dynamic object detection
43. ORB-SLAM: A versatile and accurate monocular SLAM system10.1109/TRO.2015.2463671 ↗
44. Visual SLAM with lightweight YOLOv5s in dynamic environment
45. Lucas-kanade 20 years on: A unifying framework10.1023/B:VISI.0000011205.11775.fd ↗
46. A visual odometry algorithm in dynamic scenes based on object detection
47. Robust SLAM in dynamic scenes based on deep learning and edge detection
48. Robust RGB-D SLAM for dynamic environments based on YOLOv4
49. Pixel-level segmentation algorithm combining depth map clustering and object detection
50. CFP-SLAM: A real-time visual SLAM based on coarse-to-fine probability in dynamic environments
51. DBSCAN revisited, revisited: Why and how you should (still) use DBSCAN10.1145/3068335 ↗
52. Semantic SLAM algorithm based on deep learning in dynamic environment
53. Visual SLAM algorithm based on target detection and semantic segmentation
54. Loop detection and semantic mapping algorithm fusing with semantic information
55. Loop closure detection for visual SLAM systems based on convolutional netural network
56. Visual SLAM optimization in based on object detection network
57. J. C. V. Soares, V. S. Medeiros, G. F. Abati, M. Becker, G. Caurin et al., “Visual localization and mapping in dynamic and changing environments,” 2022. [Online]. Available: https://arxiv.org/abs/2209.10710 (accessed on 13/04/2023).
58. Monocular visual inertial odometer based on convolutional neural network with extended Kalman filter
59. Enhancement of mobile robot relocalization based on environmental semantic information
60. Improving relocalization in visual SLAM by using object detection
61. OA-SLAM: Leveraging objects for camera relocalization in visual SLAM
62. SO-SLAM: Semantic object SLAM with scale proportional and symmetrical texture constraints10.1109/LRA.2022.3148465 ↗
63. FusionLane: Multi-sensor fusion for lane marking semantic segmentation using deep neural networks10.1109/TITS.2020.3030767 ↗
64. Multi-hierarchical semantic maps for mobile robotics
65. Semantic hazard labelling and risk assessment mapping during robot exploration10.1109/ACCESS.2022.3148544 ↗
66. SEMANTIC-RTAB-MAP (SRM): A semantic SLAM system with CNNs on depth images10.3934/mfc.2019003 ↗
67. Appearance-based loop closure detection for online large-scale and long-term operation10.1109/TRO.2013.2242375 ↗
68. EAO-SLAM: Monocular semi-dense object SLAM based on ensemble data association
69. Indoor room identify and mapping with virtual based SLAM using furnitures and household objects relationship based on CNNs
70. Meaningful maps with object-oriented semantic mapping
71. LSD-SLAM: Large-scale direct monocular SLAM
72. Semantic map construction via multi-sensor fusion
73. Improved techniques for grid mapping with rao-blackwellized particle filters10.1109/TRO.2006.889486 ↗
74. Semantic SLAM based on visual SLAM and object detection10.5768/JAO202142.0102002 ↗
75. Semantic map generation algorithm combined with YOLOv5
76. The new college vision and laser data set10.1177/0278364909103911 ↗
77. Are we ready for autonomous driving? The KITTI vision benchmark suite
78. A benchmark for the evaluation of RGB-D SLAM systems
79. Are we there yet? Challenging SeqSLAM on a 3000 km journey across all four seasons
80. The EuRoC micro aerial vehicle datasets10.1177/0278364915620033 ↗
81. 1 year, 1000 km: The Oxford robotcar dataset10.1177/0278364916679498 ↗
82. Drive on pedestrian walk. tuk campus dataset
83. MOD-SLAM: Visual SLAM with moving object detection in dynamic environments
84. A dense dynamic SLAM system based on motion element filtering
85. An adaptive ORB-SLAM3 system for outdoor dynamic environments10.3390/s23031359 ↗
86. Probabilistic data association for semantic SLAM at scale
87. ShuffleNet V2: Practical guidelines for efficient cnn architecture design
88. MobileNetV2 Inverted residuals and linear bottlenecks
89. Searching for MobileNetV3
90. GhostNet: More features from cheap operations
91. Pedestrian object detection based on lightweight SSD algorithm
92. Research on object detection algorithm based on improved YOLOv5
93. Real-time safety helmet-wearing detection based on improved YOLOv510.32604/csse.2022.028224 ↗
94. Improving feature-based visual SLAM by semantics
95. Research on autonomous navigation method of mobile robot based on semantic ORB-SLAM2 algorithm
96. Z. H. Li, “Design of UAV autonomous navigation system based on semantic VSLAM,” China National Knowledge Infrastructure, 2022. [Online]. Available: https://kns.cnki.net/kcms2/article/abstract?v=2F6201taHdfnfj4OhP6V362lrQXTcSoBu1q89BFighGztndCWyODbAmAEvPoC8cdsnPwb9oKdvAImmrQs8Hwh2DHQ-GhGkE4Y8HBuYZIxurivBIwriJhiJ7_Dg9zb72UmE_lbTE71=&uniplatform=NZKPT&language=CHS (accessed on 13/04/2023).
97. CubeSLAM: Monocular 3-D object SLAM10.1109/TRO.2019.2909168 ↗
98. Towards accurate loop closure detection in semantic SLAM with 3D semantic covisibility graphs10.1109/LRA.2022.3145066 ↗