Optical Flow Odometry with Panoramic Image Based on Spherical Congruence Projection
Source abstract
Abstract
Panoramic images provide distinct advantages in odometry applications, which are largely due to their extensive field of view and higher information density captured in a single frame. Traditional odometry methods often rely on mapping panoramic images onto the planar structure for feature tracking. However, this process introduces uneven distortions of features, which diminish the accuracy of feature tracking and odometry, particularly in scenarios involving large displacements. In this work, we address this challenge by introducing a novel approach, named spherical congruence projection (SCP), that maps panoramic images onto a spherical structure and projects the spherical pixels onto a two-dimensional data format while preserving the spherical pixel topology. SCP effectively eliminates the distortion across the panoramic image. Additionally, we present the optical flow odometry on the panoramic image in the spherical structure and integrate it with the proposed SCP method for the first time. The experimental results in public and custom-built datasets demonstrate that the proposed SCP-based odometry method reliably tracks features and maintains accurate odometry performance, even in fast-moving scenarios.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Image and Object Detection Techniques · Advanced Vision and Imaging · Robotics and Sensor-Based Localization
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
School of Mechatronic Engineering and Automation, Shanghai University, Shanghai 200444, China
Shanghai Key Laboratory of Intelligent Manufacturing and Robotics, Shanghai University, Shanghai 200444, China
Funding
National Natural Science Foundation of China
References (34)
Show source-provided references
1. PanoFlow: Learning 360° backslashcirc Optical Flow for Surrounding Temporal Understanding10.1109/TITS.2023.3241212 ↗
2. Kinzig, C., Miller, H., Lauer, M., and Stiller, C. (2024, January 2–5). Panoptic Segmentation from Stitched Panoramic View for Automated Driving. Proceedings of the 2024 IEEE Intelligent Vehicles Symposium (IV), Jeju Island, Republic of Korea.10.1109/IV55156.2024.10588453 ↗
3. Measuring visual walkability perception using panoramic street view images, virtual reality, and deep learning10.1016/j.scs.2022.104140 ↗
4. Review on panoramic imaging and its applications in scene understanding
5. Scaramuzza, D., Martinelli, A., and Siegwart, R. (2006, January 4–7). A flexible technique for accurate omnidirectional camera calibration and structure from motion. Proceedings of the Fourth IEEE International Conference on Computer Vision Systems (ICVS’06), New York, NY, USA.10.1109/ICVS.2006.3 ↗
6. PALVO: Visual odometry based on panoramic annular lens10.1364/OE.27.024481 ↗
7. Panoramic SLAM from a multiple fisheye camera rig10.1016/j.isprsjprs.2019.11.014 ↗
8. Wang, Y., Cai, S., Li, S.J., Liu, Y., Guo, Y., Li, T., and Cheng, M.M. (2018, January 2–6). CubemapSLAM: A piecewise-pinhole monocular fisheye SLAM system. Proceedings of the Asian Conference on Computer Vision, Perth, Australia.10.1007/978-3-030-20876-9_3 ↗
9. Pose estimation by omnidirectional visual-inertial odometry10.1016/j.robot.2018.03.007 ↗
10. Seok, H., and Lim, J. (2019, January 20–24). Rovo: Robust omnidirectional visual odometry for wide-baseline wide-fov camera systems. Proceedings of the 2019 International Conference on Robotics and Automation (ICRA), Montreal, QC, Canada.10.1109/ICRA.2019.8793758 ↗
11. Yoon, Y., Chung, I., Wang, L., and Yoon, K.J. (2022, January 21–24). Spheresr: 360deg image super-resolution with arbitrary projection via continuous spherical image representation. Proceedings of the EEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.10.1109/CVPR52688.2022.00559 ↗
12. Lee, Y., Jeong, J., Yun, J., Cho, W., and Yoon, K.J. (2019, January 15–20). Spherephd: Applying cnns on a spherical polyhedron representation of 360deg images. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.10.1109/CVPR.2019.00940 ↗
13. Gao, X., Shi, Y., Zhao, Y., Wang, Y., Wang, J., and Wu, G. (2025). CAPDepth: 360 Monocular Depth Estimation by Content-Aware Projection. Appl. Sci., 15.10.3390/app15020769 ↗
14. Aguiar, A., Santos, F., Santos, L., and Sousa, A. (2019, January 3–6). Monocular visual odometry using fisheye lens cameras. Proceedings of the Progress in Artificial Intelligence: 19th EPIA Conference on Artificial Intelligence, EPIA 2019, Vila Real, Portugal. Proceedings, Part II 19.10.1007/978-3-030-30244-3_27 ↗
15. Visual Odometry of a Mobile Palette Robot Using Ground Plane Image from a Fisheye Camera10.5194/isprs-archives-XLIII-B1-2022-431-2022 ↗
16. Eder, M., Shvets, M., Lim, J., and Frahm, J.M. (2020, January 13–19). Tangent images for mitigating spherical distortion. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.10.1109/CVPR42600.2020.01244 ↗
17. Huang, H., and Yeung, S.K. (2022, January 23—27). 360vo: Visual odometry using a single 360 camera. Proceedings of the 2022 International Conference on Robotics and Automation (ICRA), Philadelphia, PA, USA.10.1109/ICRA46639.2022.9812203 ↗
18. Kitamura, R., Li, S., and Nakanishi, I. (2015, January 2–5). Spherical FAST corner detector. Proceedings of the 2015 IEEE International Conference on Mechatronics and Automation (ICMA), Beijing, China.10.1109/ICMA.2015.7237896 ↗
19. Integration of the barotropic vorticity equation on a spherical geodesic grid10.3402/tellusa.v20i4.10044 ↗
20. SPHORB: A fast and robust binary feature on the sphere10.1007/s11263-014-0787-4 ↗
21. Guan, H., and Smith, W.A. (2017, January 21–26). BRISKS: Binary features for spherical images on a geodesic grid. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.10.1109/CVPR.2017.519 ↗
22. Zhang, Y., Song, J., Ding, Y., Yuan, Y., and Wei, H.L. (2021). FSD-BRIEF: A distorted BRIEF descriptor for fisheye image based on spherical perspective model. Sensors, 21.10.3390/s21051839 ↗
23. sRD-SIFT: Keypoint detection and matching in images with radial distortion10.1109/TRO.2012.2184952 ↗
24. Gava, C.C., Hengen, J.M., Taetz, B., and Stricker, D. (2013, January 29–31). Keypoint detection and matching on high resolution spherical images. Proceedings of the Advances in Visual Computing: 9th International Symposium, ISVC 2013, Rethymnon, Crete, Greece. Proceedings, Part I 9.
25. Scale invariant feature transform on the sphere: Theory and applications10.1007/s11263-011-0505-4 ↗
26. Chen, Z., Li, J., and Wu, K. (2024, January 3–8). 360ORB-SLAM: A Visual SLAM System for Panoramic Images with Depth Completion Network. Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), Waikoloa, HI, USA.10.1109/CSCWD61410.2024.10580875 ↗
27. LF-VISLAM: A SLAM Framework for Large Field-of-View Cameras With Negative Imaging Plane on Mobile Agents10.1109/TASE.2023.3324495 ↗
28. Yuan, H., Liu, T., Zhang, Y., and Jiang, Z. (2024, January 17–21). Panoramic Direct LiDAR-assisted Visual Odometry. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.
29. ROVINS: Robust omnidirectional visual inertial navigation system10.1109/LRA.2020.3010457 ↗
30. Wang, Z., Yang, K., Shi, H., Li, P., Gao, F., and Wang, K. (2022, January 23–27). LF-VIO: A visual-inertial-odometry framework for large field-of-view cameras with negative plane. Proceedings of the 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Kyoto, Japan.10.1109/IROS47612.2022.9981217 ↗
31. PAL-SLAM2: Visual and visual–inertial monocular SLAM for panoramic annular lens10.1016/j.isprsjprs.2024.03.016 ↗
32. Scaramuzza, D., Martinelli, A., and Siegwart, R. (2006, January 9–15). A toolbox for easily calibrating omnidirectional cameras. Proceedings of the 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems, Beijing, China.10.1109/IROS.2006.282372 ↗
33. UVS-CNNs: Constructing general convolutional neural networks on quasi-uniform spherical images10.1016/j.cag.2024.103973 ↗
34. Vins-mono: A robust and versatile monocular visual-inertial state estimator10.1109/TRO.2018.2853729 ↗