journal article·2023-07-31Open access

A Comparison of Monocular Visual SLAM and Visual Odometry Methods Applied to 3D Reconstruction

Erick P. Herrera-Granda, Juan C. Torres-Cantero, Andrés Rosales, Diego H. Peluffo-Ordóñez

Applied Sciences· Volume 13· Issue 15

Abstract

Pure monocular 3D reconstruction is a complex problem that has attracted the research community’s interest due to the affordability and availability of RGB sensors. SLAM, VO, and SFM are disciplines formulated to solve the 3D reconstruction problem and estimate the camera’s ego-motion; so, many methods have been proposed. However, most of these methods have not been evaluated on large datasets and under various motion patterns, have not been tested under the same metrics, and most of them have not been evaluated following a taxonomy, making their comparison and selection difficult. In this research, we performed a comparison of ten publicly available SLAM and VO methods following a taxonomy, including one method for each category of the primary taxonomy, three machine-learning-based methods, and two updates of the best methods to identify the advantages and limitations of each category of the taxonomy and test whether the addition of machine learning or updates on those methods improved them significantly. Thus, we evaluated each algorithm using the TUM-Mono dataset and benchmark, and we performed an inferential statistical analysis to identify the significant differences through its metrics. The results determined that the sparse-direct methods significantly outperformed the rest of the taxonomy, and fusing them with machine learning techniques significantly enhanced the geometric-based methods’ performance from different perspectives.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: Robotics and Sensor-Based Localization · Advanced Vision and Imaging · Optical measurement and interference techniques

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

Unidad de Educación en Línea, Universidad de Otavalo, Otavalo 100202, Ecuador

Department of Computer Languages and Systems, University of Granada, 18071 Granada, Spain

SDAS Research Group, Ben Guerir 43150, Morocco

GIECAR, Departamento de Automatizacióny Control Industrial, Escuela Politécnica Nacional, Quito 170525, Ecuador

Universidad de Investigación de Tecnología Experimental Yachay, San Miguel de Urcuquí 100115, Ecuador

College of Computing, Mohammed VI Polytechnic University, Salé 43150, Morocco

Faculty of Engineering, Corporación Universitaria Autónoma de Nariño, Pasto 520001, Colombia

Funding

SDAS Research Group

References (96)

Show source-provided references
1. Real-time Depth Estimation Using Recurrent CNN with Sparse Depth Cues for SLAM System10.1007/s12555-019-0350-8 ↗
2. DeepFactors: Real-Time Probabilistic Dense Monocular SLAM10.1109/LRA.2020.2965415 ↗
3. Review of visual odometry: Types, approaches, challenges, and applications10.1186/s40064-016-3573-7 ↗
4. State of the Art on Monocular 3D Face Reconstruction, Tracking, and Applications10.1111/cgf.13382 ↗
5. A Tutorial: Mobile Robotics, SLAM, Bayesian Filter, Keyframe Bundle Adjustment and ROS Applications
6. Engel, J., Usenko, V., and Cremers, D. (2016). A Photometrically Calibrated Benchmark For Monocular Visual Odometry. arXiv.
7. Barros, A.M., Michel, M., Moline, Y., Corre, G., and Carrel, F. (2022). A Comprehensive Survey of Visual SLAM Algorithms. Robotics, 11.10.3390/robotics11010024 ↗
8. Handa, A., Whelan, T., McDonald, J., and Davison, A.J. (2014–7, January 31). A benchmark for RGB-D visual odometry, 3D reconstruction and SLAM. Proceedings of the 2014 IEEE International Conference on Robotics and Automation (ICRA), Hong Kong, China.10.1109/ICRA.2014.6907054 ↗
9. Herrera-Granda, E.P. (2023, June 16). An Extended Taxonomy for Monocular Visual SLAM, Visual Odometry, and Structure from Motion methods applied to 3D Reconstruction. GitHub Repository. Available online: https://github.com/erickherreraresearch/TaxonomyPureVisualMonocularSLAM/.10.2139/ssrn.4546921 ↗
10. Tateno, K., Tombari, F., Laina, I., and Navab, N. (2016, January 21–26). CNN-SLAM: Real-time dense monocular SLAM with learned depth prediction. Proceedings of the 30th IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.10.1109/CVPR.2017.695 ↗
11. Loo, S.Y., Amiri, A.J., Mashohor, S., Tang, S.H., and Zhang, H. (2019, January 20–24). CNN-SVO: Improving the Mapping in Semi-Direct Visual Odometry Using Single-Image Depth Prediction. Proceedings of the 2019 International Conference on Robotics and Automation (ICRA), Montreal, QC, Canada.10.1109/ICRA.2019.8794425 ↗
12. SVO: Semidirect Visual Odometry for Monocular and Multicamera Systems10.1109/TRO.2016.2623335 ↗
13. Forster, C., Pizzoli, M., and Scaramuzza, D. (June, January 31). SVO: Fast semi-direct monocular visual odometry. Proceedings of the 2014 IEEE International Conference on Robotics and Automation (ICRA), Hong Kong, China.10.1109/ICRA.2014.6906584 ↗
14. Wimbauer, F., Yang, N., von Stumberg, L., Zeller, N., and Cremers, D. (2021, January 20–25). MonoRec: Semi-Supervised Dense Reconstruction in Dynamic Environments from a Single Moving Camera. Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA.10.1109/CVPR46437.2021.00605 ↗
15. Muskie (2023, June 16). CNN-DSO: A Combination of Direct Sparse Odometry and CNN Depth Prediction. GitHub Repository. Available online: https://github.com/muskie82/CNN-DSO.
16. Wang, S. (2023, June 16). DF-ORB-SLAM. GitHub Repository. Available online: https://github.com/834810269/DF-ORB-SLAM.10.1299/jsmermd.2023.2P1-G22 ↗
17. Bescos, B. (2023, June 16). DynaSLAM. GitHub Repository. Available online: https://github.com/BertaBescos/DynaSLAM.
18. Sun, J., Wang, Y., and Shen, Y. (2019, January 19–22). Fully Scaled Monocular Direct Sparse Odometry with A Distance Constraint. Proceedings of the 2019 5th International Conference on Control, Automation and Robotics (ICCAR), Beijing, China.10.1109/ICCAR.2019.8813371 ↗
19. Sundar, A. (2023, June 16). CNN-SLAM. GitHub Repository. Available online: https://github.com/iitmcvg/CNN_SLAM.
20. Tang, C., and Tan, P. (2018). BA-Net: Dense Bundle Adjustment Network. arXiv.
21. Direct Sparse Odometry10.1109/TPAMI.2017.2658577 ↗
22. MonoSLAM: Real-Time Single Camera SLAM10.1109/TPAMI.2007.1049 ↗
23. Klein, G., and Murray, D. (2007, January 13–16). Parallel Tracking and Mapping for Small AR Workspaces. Proceedings of the 2007 6th IEEE and ACM International Symposium on Mixed and Augmented Reality, Nara, Japan.10.1109/ISMAR.2007.4538852 ↗
24. ORB-SLAM: A Versatile and Accurate Monocular SLAM System10.1109/TRO.2015.2463671 ↗
25. Dense versus Sparse Approaches for Estimating the Fundamental Matrix10.1007/s11263-011-0466-7 ↗
26. Ranftl, R., Vineet, V., Chen, Q., and Koltun, V. (2016, January 1–26). Dense Monocular Depth Estimation in Complex Dynamic Scenes. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.10.1109/CVPR.2016.440 ↗
27. Stühmer, J., Gumhold, S., and Cremers, D. (2010, January 22–24). Real-Time Dense Geometry from a Handheld Camera. Proceedings of the 32nd DAGM Symposium, Darmstadt, Germany.10.1007/978-3-642-15986-2_2 ↗
28. Pizzoli, M., Forster, C., and Scaramuzza, D. (June, January 31). REMODE: Probabilistic, monocular dense reconstruction in real time. Proceedings of the IEEE International Conference on Robotics and Automation, Hong Kong, China.
29. Engel, J., Schöps, T., and Cremers, D. (2014). European Conference on Computer Vision, Springer International Publishing.
30. Gao, X., Wang, R., Demmel, N., and Cremers, D. (2018, January 1–5). LDSO: Direct Sparse Odometry with Loop Closure. Proceedings of the 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Madrid, Spain.10.1109/IROS.2018.8593376 ↗
31. Direct Sparse Mapping10.1109/TRO.2020.2991614 ↗
32. DynaSLAM: Tracking, Mapping, and Inpainting in Dynamic Scenes10.1109/LRA.2018.2860039 ↗
33. Bloesch, M., Czarnowski, J., Clark, R., Leutenegger, S., and Davison, A.J. (2018, January 18–23). CodeSLAM—Learning a Compact, Optimisable Representation for Dense Visual SLAM. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00271 ↗
34. Min, Z., and Dunn, E. (June, January 30). VOLDOR+SLAM: For the Times When Feature-Based or Direct Methods Are Not Good Enough. Proceedings of the 2021 IEEE International Conference on Robotics and Automation, Xi’an, China.10.1109/ICRA48506.2021.9561230 ↗
35. DROID-SLAM: Deep Visual SLAM for Monocular, Stereo, and RGB-D Cameras
36. SDF-SLAM: A Deep Learning Based Highly Accurate SLAM Using Monocular Camera Aiming at Indoor Map Reconstruction with Semantic and Depth Fusion10.1109/ACCESS.2022.3144845 ↗
37. Lang, R., Fan, Y., and Chang, Q. (2023). SVR-Net: A Sparse Voxelized Recurrent Network for Robust Monocular SLAM with Direct TSDF Mapping. Sensors, 23.10.3390/s23083942 ↗
38. Challenges in Monocular Visual Odometry: Photometric Calibration, Motion Bias, and Rolling Shutter Effect10.1109/LRA.2018.2846813 ↗
39. Mingachev, E., Lavrenov, R., Magid, E., and Svinin, M. (2020, January 15–18). Comparative Analysis of Monocular SLAM Algorithms Using TUM and EuRoC Benchmarks. Proceedings of the 15th International Conference on Electromechanics and Robotics” Zavalishin’s Readings” ER (ZR) 2020, Ufa, Russia.10.1007/978-981-15-5580-0_28 ↗
40. Mingachev, E., Lavrenov, R., Tsoy, T., Matsuno, F., Svinin, M., Suthakorn, J., and Magid, E. (2020). Interactive Collaborative Robotics, Springer.
41. Visual and Visual-Inertial SLAM: State of the Art, Classification, and Experimental Benchmarking10.1155/2021/2054828 ↗
42. Wimbauer, F., and Yang, N. (2023, June 16). MonoRec. GitHub Repository. Available online: https://github.com/Brummi/MonoRec.
43. Czarnowski, J., and Kaneko, M. (2023, June 16). DeepFactors. GitHub Repository. Available online: https://github.com/jczarnowski/DeepFactors.
44. Zhang, S. (2023, June 16). DVSO: Deep Virtual Stereo Odometry. GitHub Repository. Available online: https://github.com/SenZHANG-GitHub/dvso.
45. Cheng, R. (2023, June 16). CNN-DVO. McGill Repository. McGill. Available online: http://www.cim.mcgill.ca/∼mrl/ran/crv2020.
46. Loo, S.Y. (2023, June 16). CNN-SVO. GitHub Repository. Available online: https://github.com/yan99033/CNN-SVO.
47. Steenbeek, A. (2023, June 16). Sparse-to-Dense: Depth Prediction from Sparse Depth Samples and a Single Image. GitHub Repository. Available online: https://github.com/annesteenbeek/sparse-to-dense-ros.
48. Ummenhofer, B. (2023, June 16). DeMoN: Depth and Motion Network. GitHub Repository. Available online: https://github.com/lmb-freiburg/demon.
49. Teed, Z., and Deng, J. (2023, June 16). DeepV2D. GitHub Repository. Available online: https://github.com/princeton-vl/DeepV2D.
50. Min, Z. (2023, June 16). VOLDOR: Visual Odometry from Log-logistic Dense Optical Flow Residual. GitHub Repository. Available online: https://github.com/htkseason/VOLDOR.
51. Teed, Z., and Deng, J. (2023, June 16). DROID-SLAM. GitHub Repository. Available online: https://github.com/princeton-vl/DROID-SLAM.
52. Zhou, H., Ummenhofer, B., and Brox, T. (2023, June 16). DeepTAM. GitHub Repository. Available online: https://github.com/lmb-freiburg/deeptam.
53. Troscot, S. (2023, June 16). CodeSLAM. GitHub Repository. Available online: https://github.com/silviutroscot/CodeSLAM.
54. ORB-SLAM2: An Open-Source SLAM System for Monocular, Stereo, and RGB-D Cameras10.1109/TRO.2017.2705103 ↗
55. Mur-Artal, R. (2023, June 16). ORB-SLAM2. GitHub Repository. Available online: https://github.com/raulmur/ORB_SLAM2.
56. Omnidirectional DSO: Direct Sparse Odometry with Fisheye Cameras10.1109/LRA.2018.2855443 ↗
57. Engel, J., Sturm, J., and Cremers, D. (2013, January 2–8). Semi-dense visual odometry for a monocular camera. Proceedings of the IEEE International Conference on Computer Vision, Sydney, Australia.10.1109/ICCV.2013.183 ↗
58. FAB-MAP: Probabilistic Localization and Mapping in the Space of Appearance10.1177/0278364908090961 ↗
59. Engel, J. (2023, June 16). LSD-SLAM: Large-Scale Direct Monocular SLAM. GitHub Repository. Available online: https://github.com/tum-vision/lsd_slam.
60. Engel, J. (2023, June 16). DSO: Direct Sparse Odometry. GitHub Repository. Available online: https://github.com/JakobEngel/dso.
61. Forster, C. (2023, June 16). Semi-Direct Monocular Visual Odometry. GitHub Repository. Available online: https://github.com/uzh-rpg/rpg_svo.
62. Demmel, N., Xiang, G., and Erkam, U. (2023, June 16). LDSO: Direct Sparse Odometry with Loop Closure. GitHub Repository. Available online: https://github.com/tum-vision/LDSO.
63. Zubizarreta, J. (2023, June 16). DSM: Direct Sparse Mapping. GitHub Repository. Available online: https://github.com/jzubizarreta/dsm.
64. He, K., Gkioxari, G., Dollár, P., and Girshick, R. (2017, January 27–29). Mask R-CNN. Proceedings of the 2017 IEEE International Conference on Computer Vision (ICCV), Venice, Italy.10.1109/ICCV.2017.322 ↗
65. Yang, N., von Stumberg, L., Wang, R., and Cremers, D. (2020, January 13–19). D3VO: Deep Depth, Deep Pose and Deep Uncertainty for Monocular Visual Odometry. Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.10.1109/CVPR42600.2020.00136 ↗
66. Deep Direct Visual Odometry10.1109/TITS.2021.3071886 ↗
67. Godard, C., Mac Aodha, O., and Brostow, G.J. (2017, January 21–26). Unsupervised Monocular Depth Estimation with Left-Right Consistency. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.10.1109/CVPR.2017.699 ↗
68. Loo, S.Y. (2023, June 16). MonoDepth CPP. Available online: https://github.com/yan99033/monodepth-cpp.
69. The YTU dataset and recurrent neural network based visual-inertial odometry10.1016/j.measurement.2021.109878 ↗
70. Chen, T., Pu, F., Chen, H., and Liu, Z. (2022). WHUVID: A Large-Scale Stereo-IMU Dataset for Visual-Inertial Odometry and Autonomous Driving in Chinese Urban Scenarios. Remote Sens., 14.10.3390/rs14092033 ↗
71. Unsupervised Depth Completion From Visual Inertial Odometry10.1109/LRA.2020.2969938 ↗
72. Cordts, M. (2016, January 27–30). The Cityscapes Dataset for Semantic Urban Scene Understanding. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.10.1109/CVPR.2016.350 ↗
73. Huang, X. (2018, January 18–22). The ApolloScape Dataset for Autonomous Driving. Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Salt Lake City, UT, USA.10.1109/CVPRW.2018.00141 ↗
74. Geiger, A., Lenz, P., and Urtasun, R. (2012, January 16–21). Are we ready for autonomous driving? The KITTI vision benchmark suite. Proceedings of the 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Providence, RI, USA.10.1109/CVPR.2012.6248074 ↗
75. The EuRoC micro aerial vehicle datasets10.1177/0278364915620033 ↗
76. Butler, D.J., Wulff, J., Stanley, G.B., and Black, M.J. (2012). European Conference on Computer Vision, Springer.
77. Zamir, A.R., Sax, A., Shen, W., Guibas, L., Malik, J., and Savarese, S. (2018, January 18–22). Taskonomy: Disentangling Task Transfer Learning. Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00391 ↗
78. Virtual Normal: Enforcing Geometric Constraints for Accurate and Robust Depth Prediction10.1109/TPAMI.2021.3097396 ↗
79. Cho, J., Min, D., Kim, Y., and Sohn, K. (2019). A Large RGB-D Dataset for Semi-supervised Monocular Depth Estimation. arXiv.
80. Xian, K. (2018, January 18–22). Monocular Relative Depth Perception with Web Stereo Data Supervision. Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00040 ↗
81. Sturm, J., Engelhard, N., Endres, F., Burgard, W., and Cremers, D. (2012, January 7–12). A benchmark for the evaluation of RGB-D SLAM systems. Proceedings of the 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems, Algarve, Portugal.10.1109/IROS.2012.6385773 ↗
82. Closed-form solution of absolute orientation using unit quaternions10.1364/JOSAA.4.000629 ↗
83. Sarabandi, S., and Thomas, F. (2018). Advances in Robot Kinematics, Springer.
84. Straight lines have to be straight10.1007/PL00013269 ↗
85. Castle, R. (2023, June 16). PTAM-GPL: Parallel Tracking and Mapping. GitHub Repository. Available online: https://github.com/Oxford-PTAM/PTAM-GPL.
86. ROS.org (2022, December 26). ROS Camera Calibration. Available online: http://wiki.ros.org/camera_calibration.
87. Vision, O.S.C. (2022, December 26). Camera Calibration with OpenCV. Available online: https://docs.opencv.org/4.1.1/d4/d94/tutorial_camera_calibration.html.
88. MAHALANOBIS DISTANCE AND ITS APPLICATION FOR DETECTING MULTIVARIATE OUTLIERS
89. Steenbeek, A., and Nex, F. (2022). CNN-Based Dense Monocular Visual SLAM for Real-Time UAV Exploration in Emergency Conditions. Drones, 6.10.3390/drones6030079 ↗
90. Improving Monocular Visual Odometry Using Learned Depth10.1109/TRO.2022.3164834 ↗
91. Ummenhofer, B., Zhou, H., Uhrig, J., Mayer, N., Ilg, E., Dosovitskiy, A., and Brox, T. (2017, January 21–26). DeMoN: Depth and Motion Network for Learning Monocular Stereo. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.10.1109/CVPR.2017.596 ↗
92. Teed, Z., and Deng, J. (2018). DeepV2D: Video to Depth with Differentiable Structure from Motion. arXiv.
93. DeepTAM: Deep Tracking and Mapping with Convolutional Neural Networks10.1007/s11263-019-01221-0 ↗
94. Laidlow, T., Czarnowski, J., and Leutenegger, S. (2019, January 20–24). DeepFusion: Real-Time Dense 3D Reconstruction for Monocular SLAM using Single-View Depth and Gradient Predictions. Proceedings of the 2019 International Conference on Robotics and Automation (ICRA), Montreal, QC, Canada.10.1109/ICRA.2019.8793527 ↗
95. Yang, N., Wang, R., Stückler, J., and Cremers, D. (2018). European Conference on Computer Vision, Springer.
96. Tang, C. (2022, December 26). BA-Net: Dense Bundle Adjustment Network. GitHub Repository. Available online: https://github.com/frobelbest/BANet.