A Method for the Calibration of a LiDAR and Fisheye Camera System
Source abstract
Abstract
LiDAR and camera systems are frequently used together to gain a more complete understanding of the environment in different fields, such as mobile robotics, autonomous driving, or intelligent surveillance. Accurately calibrating the extrinsic parameters is crucial for the accurate fusion of the data captured by both systems, which is equivalent to finding the transformation between the reference systems of both sensors. Traditional calibration methods for LiDAR and camera systems are developed for pinhole cameras and are not directly applicable to fisheye cameras. This work proposes a target-based calibration method for LiDAR and fisheye camera systems that avoids the need to transform images to a pinhole camera model, reducing the computation time. Instead, the method uses the spherical projection of the image, obtained with the intrinsic calibration parameters and the corresponding point cloud for LiDAR–fisheye calibration. Thus, unlike a pinhole-camera-based system, a wider field of view is provided, adding more information, which will lead to a better understanding of the environment itself, as well as enabling using fewer image sensors to cover a wider area.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Robotics and Sensor-Based Localization · Remote Sensing and LiDAR Applications · 3D Surveying and Cultural Heritage
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Engineering Research Institute of Elche (I3E), Miguel Hernandez University, Avinguda de la Universidad d’Elx, s/n, 03202 Elche, Spain
Valencian Graduate School and Research Network of Artificial Intelligence (ValgrAI), Camí de Vera S/N, Edificio 3Q, 46022 Valencia, Spain
Funding
MICIU/AEI/10.13039/501100011033 · European Union NextGenerationEU/PRTR · FEDER, UE
References (38)
Show source-provided references
1. Wang, Z., Huang, Z., Gao, Y., Wang, N., and Liu, S. (2024). MV2DFusion: Leveraging Modality-Specific Object Semantics for Multi-Modal 3D Detection. arXiv.
2. Zhang, H., Liang, L., Zeng, P., Song, X., and Wang, Z. (October, January 29). SparseLIF: High-performance sparse LiDAR–camera fusion for 3D object detection. Proceedings of the European Conference on Computer Vision, Milan, Italy.
3. Yang, X.S., Sherratt, S., Dey, N., and Joshi, A. (2022, January 21–24). Camera and LiDAR Fusion for Point Cloud Semantic Segmentation. Proceedings of the Seventh International Congress on Information and Communication Technology, London, UK.
4. Wu, H., Wen, C., Shi, S., Li, X., and Wang, C. (2023, January 18–23). Virtual Sparse Convolution for Multimodal 3D Object Detection. Proceedings of the 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Vancouver, BC, Canada.10.1109/CVPR52729.2023.02074 ↗
5. Tan, Z., Zhang, X., Teng, S., Wang, L., and Gao, F. (2024). A Review of Deep Learning-Based LiDAR and Camera Extrinsic Calibration. Sensors, 24.10.3390/s24123878 ↗
6. A flexible new technique for camera calibration10.1109/34.888718 ↗
7. Zhang, Z., and Tang, Q. (2016, January 6–8). Camera Self-Calibration Based on Multiple View Images. Proceedings of the 2016 Nicograph International (NicoInt), Hanzhou, China.10.1109/NicoInt.2016.16 ↗
8. Conic and circular points based camera linear calibration
9. Ishikawa, R., Oishi, T., and Ikeuchi, K. (2018, January 1–5). LiDAR and Camera Calibration Using Motions Estimated by Sensor Fusion Odometry. Proceedings of the 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Madrid, Spain.10.1109/IROS.2018.8593360 ↗
10. Lv, X., Wang, B., Dou, Z., Ye, D., and Wang, S. (2021, January 20–25). LCCNet: LiDAR and Camera Self-Calibration Using Cost Volume Network. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, Nashville, TN, USA.10.1109/CVPRW53098.2021.00324 ↗
11. Luo, Z., Yan, G., and Li, Y. (2023). Calib-Anything: Zero-training LiDAR-Camera Extrinsic Calibration Method Using Segment Anything. arXiv.10.1109/ICRA57147.2024.10610983 ↗
12. Random Sample Consensus: A Paradigm for Model Fitting with Applications to Image Analysis and Automated Cartography10.1145/358669.358692 ↗
13. Visual odometry for ground vehicle applications10.1002/rob.20103 ↗
14. Burghardt, T., Damen, D., Mayol-Cuevas, W.W., and Mirmehdi, M. (2013, January 9–13). Fast Explicit Diffusion for Accelerated Features in Nonlinear Scale Spaces. Proceedings of the British Machine Vision Conference, BMVC 2013, Bristol, UK.10.5244/C.27.13 ↗
15. Schneider, N., Piewak, F., Stiller, C., and Franke, U. (2017, January 11–14). RegNet: Multimodal sensor registration using deep neural networks. Proceedings of the 2017 IEEE Intelligent Vehicles Symposium (IV), Los Angeles, CA, USA.10.1109/IVS.2017.7995968 ↗
16. Iyer, G., Ram, R.K., Murthy, J.K., and Krishna, K.M. (2018, January 1–5). CalibNet: Geometrically Supervised Extrinsic Calibration using 3D Spatial Transformer Networks. Proceedings of the 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Madrid, Spain.10.1109/IROS.2018.8593693 ↗
17. Ren, F., Liu, H., and Wang, H. (2024). A LiDAR-Camera Joint Calibration Algorithm Based on Deep Learning. Sensors, 24.10.3390/s24186033 ↗
18. Pandey, G., McBride, J., Savarese, S., and Eustice, R. (2012, January 22–26). Automatic Targetless Extrinsic Calibration of a 3D LiDAR and Camera by Maximizing Mutual Information. Proceedings of the Twenty-Sixth AAAI Conference on Artificial Intelligence, Toronto, ON, Canada.10.1609/aaai.v26i1.8379 ↗
19. A Computational Approach to Edge Detection10.1109/TPAMI.1986.4767851 ↗
20. Pixel-Level Extrinsic Self Calibration of High Resolution LiDAR and Camera in Targetless Environments10.1109/LRA.2021.3098923 ↗
21. Targetless Extrinsic Calibration of Multiple Small FoV LiDARs and Cameras Using Adaptive Voxelization10.1109/TIM.2022.3219307 ↗
22. Ma, T., Liu, Z., Yan, G., and Li, Y. (2021). CRLF: Automatic Calibration and Refinement based on Line Feature for LiDAR and Camera in Road Scenes. arXiv.
23. Bai, Z., Jiang, G., and Xu, A. (2020). LiDAR-Camera Calibration Using Line Correspondences. Sensors, 20.10.3390/s20216319 ↗
24. Koide, K., Oishi, S., Yokozuka, M., and Banno, A. (June, January 29). General, Single-shot, Target-less, and Automatic LiDAR-Camera Extrinsic Calibration Toolbox. Proceedings of the 2023 IEEE International Conference on Robotics and Automation (ICRA), London, UK.10.1109/ICRA48891.2023.10160691 ↗
25. Kirillov, A., Mintun, E., Ravi, N., Mao, H., Rolland, C., Gustafson, L., Xiao, T., Whitehead, S., Berg, A.C., and Lo, W.Y. (2023, January 4–6). Segment Anything. Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), Paris, France.10.1109/ICCV51070.2023.00371 ↗
26. Speeded Up Detection of Squared Fiducial Markers10.1016/j.imavis.2018.05.004 ↗
27. Cui, J., Niu, J., Ouyang, Z., He, Y., and Liu, D. (2020). ACSC: Automatic Calibration for Non-repetitive Scanning Solid-State LiDAR and Camera Systems. arXiv.
28. Wang, W., Sakurada, K., and Kawaguchi, N. (2017). Reflectance Intensity Assisted Automatic and Accurate Extrinsic Calibration of 3D LiDAR and Panoramic Camera Using a Printed Chessboard. Remote Sens., 9.10.3390/rs9080851 ↗
29. Dhall, A., Chelani, K., Radhakrishnan, V., and Krishna, K.M. (2017). LiDAR-Camera Calibration using 3D-3D Point correspondences. arXiv.
30. Yoo, J.H., Jung, G.B., Jung, H.G., and Suhr, J.K. (2024). Camera–LiDAR Calibration Using Iterative Random Sampling and Intersection Line-Based Quality Evaluation. Electronics, 13.10.3390/electronics13020249 ↗
31. LCE-Calib: Automatic LiDAR-Frame/Event Camera Extrinsic Calibration With a Globally Optimal Solution10.1109/TMECH.2023.3259444 ↗
32. Verma, S., Berrio, J.S., Worrall, S., and Nebot, E. (2019, January 27–30). Automatic extrinsic calibration between a camera and a 3D LiDAR using 3D point and plane correspondences. Proceedings of the 2019 IEEE Intelligent Transportation Systems Conference (ITSC), Auckland, New Zealand.10.1109/ITSC.2019.8917108 ↗
33. Zhou, L., Li, Z., and Kaess, M. (2018, January 1–5). Automatic Extrinsic Calibration of a Camera and a 3D LiDAR Using Line and Plane Correspondences. Proceedings of the 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Madrid, Spain.10.1109/IROS.2018.8593660 ↗
34. Xu, J., Han, D.W., Li, K., Li, J.J., and Ma, Z.Y. (2024). A Comprehensive Overview of Fish-Eye Camera Distortion Correction Methods. arXiv.
35. A solution for the best rotation to relate two sets of vectors10.1107/S0567739476001873 ↗
36. Least-squares estimation of transformation parameters between two point patterns10.1109/34.88573 ↗
37. Scaramuzza, D., Martinelli, A., and Siegwart, R. (2006, January 9–16). A Toolbox for Easily Calibrating Omnidirectional Cameras. Proceedings of the 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems, Beijing, China.10.1109/IROS.2006.282372 ↗
38. Yan, G., He, F., Shi, C., Wei, P., Cai, X., and Li, Y. (2, January 29). Joint Camera Intrinsic and LiDAR-Camera Extrinsic Calibration. Proceedings of the 2023 IEEE International Conference on Robotics and Automation (ICRA), London, UK.10.1109/ICRA48891.2023.10160542 ↗