Visual Odometry of a Low-Profile Pallet Robot Based on Ortho-Rectified Ground Plane Image from Fisheye Camera
Source abstract
Abstract
This study presents a visual-only odometry technique of a low-profile pallet robot using image feature tracking in ground plane images generated from a fisheye camera. The fisheye camera is commonly used in many robot vision applications because it provides a larger field of view (FoV) around a robot. However, because of the large radial distortion, the fisheye image is generally converted to a pinhole image for visual feature tracking or matching. Although the radial distortion can be eliminated via image undistortion with the lens calibration parameters, it causes several side effects, such as degraded image resolution and a significant reduction in the FoV. In this paper, instead of using the pinhole model, we propose to generate a ground plane image (GPI) from the fisheye image. GPI is a virtual top-view image that only contains the ground plane at the front of the robot. First, the original fisheye image is projected to several virtual pinhole images to generate a cubemap. Second, the front and bottom faces of the cubemap are projected to a GPI. Third, the GPI is homographically transformed again to further reduce image distortion. As a result, an accurate ortho-rectified ground plane image is obtained from the virtual top-view camera. For visual odometry using the ortho-rectified GPI, a number of 2D motion vectors are obtained using feature extraction and tracking between the previous and current frames in the GPI. By calculating a scaled motion vector, which is the measurement of the virtual wheel encoder of the mobile robot, we estimate the velocity and steering angle of the virtual wheel using the motion vector. Finally, we estimate the pose of the mobile robot by applying a kinematic model to the mobile robot.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Robotics and Sensor-Based Localization · Advanced Vision and Imaging · Advanced Image and Video Retrieval Techniques
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
School of Electronic and Electrical Engineering, Kyungpook National University, 80 Daehak-ro, Puk-gu, Daegu 41566, Republic of Korea
HL Klemove, 224 Harmony-ro, Yeonsu-gu, Incheon 22011, Republic of Korea
KLA Corporation, 830 Dongtansunwhan-daero, Hwaseong-si 18468, Republic of Korea
Funding
Ministry of Education
References (33)
Show source-provided references
1. Wang, Y., Cai, S., Li, S.J., Liu, Y., Guo, Y., Li, T., and Cheng, M.M. (2019). Proceedings of the Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), Springer International Publishing.
2. Performance of Optical Flow Techniques10.1007/BF01420984 ↗
3. Polack, P., Altche, F., DAndrea-Novel, B., and de La Fortelle, A. (2017, January 11–14). The Kinematic Bicycle Model: A Consistent Model for Planning Feasible Trajectories for Autonomous Vehicles?. Proceedings of the IEEE Intelligent Vehicles Symposium, Los Angeles, CA, USA.10.1109/IVS.2017.7995816 ↗
4. Unscented Filtering and Nonlinear Estimation10.1109/JPROC.2003.823141 ↗
5. (2023, August 04). Intel T265 Tracking Camera. Available online: https://www.intelrealsense.com/tracking-camera-t265/.
6. Flores, M., Valiente, D., Cebollada, S., Reinoso, Ó., and Payá, L. (2021, January 6–8). Evaluating the Influence of Feature Matching on the Performance of Visual Localization with Fisheye Images. Proceedings of the International Conference on Informatics in Control, Automation and Robotics, Paris, France.10.5220/0010555100002994 ↗
7. Rublee, E., Rabaud, V., Konolige, K., and Bradski, G. (2011, January 6–13). ORB: An Efficient Alternative to SIFT or SURF. Proceedings of the IEEE International Conference on Computer Vision, Barcelona, Spain.10.1109/ICCV.2011.6126544 ↗
8. Speeded-Up Robust Features (SURF)10.1016/j.cviu.2007.09.014 ↗
9. Rosten, E., and Drummond, T. (2006, January 7–13). Machine Learning for High-Speed Corner Detection. Proceedings of the Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), Graz, Austria.10.1007/11744023_34 ↗
10. Alcantarilla, P.F., Bartoli, A., and Davison, A.J. (2012). Proceedings of the European Conference on Computer Vision, Springer.
11. Zhang, Z., Rebecq, H., Forster, C., and Scaramuzza, D. (2016, January 16–21). Benefit of Large Field-of-View Cameras for Visual Odometry. Proceedings of the IEEE International Conference on Robotics and Automation, Stockholm, Sweden.
12. ORB-SLAM: A Versatile and Accurate Monocular SLAM System10.1109/TRO.2015.2463671 ↗
13. Cyganek, B., and Siebert, J.P. (2009). An Introduction to 3D Computer Vision Techniques and Algorithms, Wiley.10.1002/9780470699720 ↗
14. Swank, A.J. (2012). Localization Using Visual Odometry and a Single Downward-Pointing Camera, NASA.
15. Zhang, J., Singh, S., and Kantor, G. (2014). Proceedings of the Springer Tracts in Advanced Robotics, Springer.
16. Jordan, J., and Zell, A. (2017, January 6–8). Kinematic Model Based Visual Odometry for Differential Drive Vehicles. Proceedings of the 2017 European Conference on Mobile Robots, ECMR, Paris, France.10.1109/ECMR.2017.8098713 ↗
17. Ground-Plane-Based Absolute Scale Estimation for Monocular Visual Odometry10.1109/TITS.2019.2900330 ↗
18. Ground Plane Based Visual Odometry for RGBD-Cameras Using Orthogonal Projection10.1016/j.ifacol.2016.07.717 ↗
19. Malis, E. (May, January 26). Improving Vision-Based Control Using Efficient Second-Order Minimization Techniques. Proceedings of the IEEE International Conference on Robotics and Automation, New Orleans, LA, USA.
20. Visual-Gyroscope-Wheel Odometry with Ground Plane Constraint for Indoor Robots in Dynamic Environment10.1109/LSENS.2021.3057088 ↗
21. Chen, X., Vempati, A.S., and Beardsley, P. (2018, January 1–5). StreetMap—Mapping and Localization on Ground Planes Using a Downward Facing Camera. Proceedings of the IEEE International Conference on Intelligent Robots and Systems, Madrid, Spain.10.1109/IROS.2018.8594157 ↗
22. Lin, H.Y., Chung, Y.C., and Wang, M.L. (2021). Self-Localization of Mobile Robots Using a Single Catadioptric Camera with Line Feature Extraction. Sensors, 21.10.3390/s21144719 ↗
23. Visual odometry of a mobile palette robot using ground plane image from a fisheye camera10.5194/isprs-archives-XLIII-B1-2022-431-2022 ↗
24. Lucas, B.D., and Kanade, T. (1981). Iterative Image Registration Technique with an Application to Stereo Vision, Carnegie Mellon University.
25. An Enhanced Unified Camera Model10.1109/LRA.2015.2502921 ↗
26. Rehder, J., Nikolic, J., Schneider, T., Hinzmann, T., and Siegwart, R. (2016, January 16–21). Extending Kalibr: Calibrating the Extrinsics of Multiple IMUs and of Individual Axes. Proceedings of the IEEE International Conference on Robotics and Automation, Stockholm, Sweden.10.1109/ICRA.2016.7487628 ↗
27. Furgale, P., Rehder, J., and Siegwart, R. (2013, January 3–7). Unified Temporal and Spatial Calibration for Multi-Sensor Systems. Proceedings of the IEEE International Conference on Intelligent Robots and Systems, Tokyo, Japan.10.1109/IROS.2013.6696514 ↗
28. Continuous-Time Batch Trajectory Estimation Using Temporal Basis Functions10.1177/0278364915585860 ↗
29. Maye, J., Furgale, P., and Siegwart, R. (2013, January 23–26). Self-Supervised Calibration for Robotic Systems. Proceedings of the IEEE Intelligent Vehicles Symposium, Gold Coast, QLD, Australia.10.1109/IVS.2013.6629513 ↗
30. Oth, L., Furgale, P., Kneip, L., and Siegwart, R. (2013, January 23–28). Rolling Shutter Camera Calibration. Proceedings of the Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Portland, OR, USA.10.1109/CVPR.2013.179 ↗
31. Lee, U.G., Choi, K.J., and Park, S.Y. (2021, January 17–20). The Design and Implementation of Autonomous Driving Pallet Robot System Using ROS. Proceedings of the International Conference on Ubiquitous and Future Networks, ICUFN, Jeju Island, Republic of Korea.10.1109/ICUFN49451.2021.9528735 ↗
32. (2023, August 04). OptiTrack V120. Available online: https://optitrack.com/cameras/v120-trio/.
33. Zhang, Z., and Scaramuzza, D. (2018, January 1–5). A Tutorial on Quantitative Trajectory Evaluation for Visual(-Inertial) Odometry. Proceedings of the IEEE International Conference on Intelligent Robots and Systems, Madrid, Spain.10.1109/IROS.2018.8593941 ↗