Three-Dimensional Reconstruction of Indoor and Outdoor Environments Using a Stereo Catadioptric System
Source abstract
Abstract
In this work, we present a panoramic 3D stereo reconstruction system composed of two catadioptric cameras. Each one consists of a CCD camera and a parabolic convex mirror that allows the acquisition of catadioptric images. We describe the calibration approach and propose the improvement of existing deep feature matching methods with epipolar constraints. We show that the improved matching algorithm covers more of the scene than classic feature detectors, yielding broader and denser reconstructions for outdoor environments. Our system can also generate accurate measurements in the wild without large amounts of data used in deep learning-based systems. We demonstrate the system’s feasibility and effectiveness as a practical stereo sensor with real experiments in indoor and outdoor environments.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Advanced Vision and Imaging · Robotics and Sensor-Based Localization · Advanced Image and Video Retrieval Techniques
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Faculty of Informatics, Autonomous University of Queretaro, Av. de las Ciencias S/N, Campus Juriquilla, Queretaro C.P. 76230, Mexico
AiFi Inc., 2388 Walsh Av., Santa Clara, CA 95051, USA
Research Center in Optics A.C., Lomas del Bosque 115, Col. Lomas del Campestre, Leon C.P. 37150, Mexico
References (56)
Show source-provided references
1. Jiang, W., Okutomi, M., and Sugimoto, S. (2006, January 17–22). Panoramic 3D reconstruction using rotational stereo camera with simple epipolar constraints. Proceedings of the 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’06), New York, NY, USA.
2. Deng, X., Wu, F., Wu, Y., and Wan, C. (2008, January 25–27). Automatic spherical panorama generation with two fisheye images. Proceedings of the 2008 7th World Congress on Intelligent Control and Automation, Chongqing, China.
3. Sagawa, R., Kurita, N., Echigo, T., and Yagi, Y. (October, January 28). Compound catadioptric stereo sensor for omnidirectional object detection. Proceedings of the 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)(IEEE Cat. No. 04CH37566), Sendai, Japan.
4. Multi-stereo 3D reconstruction with a single-camera multi-mirror catadioptric system10.1088/1361-6501/ab3be4 ↗
5. Panoramic stereo reconstruction using non-SVP optics10.1016/j.cviu.2004.07.015 ↗
6. Jaramillo, C., Valenti, R.G., and Xiao, J. (2016, January 9–14). GUMS: A generalized unified model for stereo omnidirectional vision (demonstrated via a folded catadioptric system). Proceedings of the 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Daejeon, Korea.10.1109/IROS.2016.7759393 ↗
7. Lin, S.S., and Bajcsy, R. (2003, January 14–19). High resolution catadioptric omni-directional stereo sensor for robot vision. Proceedings of the 2003 IEEE International Conference on Robotics and Automation (Cat. No. 03CH37422), Taipei, Taiwan.
8. Ragot, N., Ertaud, J., Savatier, X., and Mazari, B. (2006, January 6–10). Calibration of a panoramic stereovision sensor: Analytical vs. interpolation-based methods. Proceedings of the Iecon 2006-32nd Annual Conference on Ieee Industrial Electronics, Paris, France.10.1109/IECON.2006.348062 ↗
9. Cabral, E.L., De Souza, J., and Hunold, M.C. (2004, January 26). Omnidirectional stereo vision with a hyperbolic double lobed mirror. Proceedings of the 17th International Conference on Pattern Recognition, Cambridge, UK.10.1109/ICPR.2004.1333989 ↗
10. Single-camera panoramic stereo system with single-viewpoint optics10.1364/OL.31.000041 ↗
11. Su, L., Luo, C., and Zhu, F. (2006, January 20–23). Obtaining obstacle information by an omnidirectional stereo vision system. Proceedings of the 2006 IEEE International Conference on Information Acquisition, Weihai, China.10.1109/ICIA.2006.305769 ↗
12. Caron, G., Marchand, E., and Mouaddib, E.M. (2009, January 10–15). 3D model based pose estimation for omnidirectional stereovision. Proceedings of the 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems, St. Louis, MO, USA.10.1109/IROS.2009.5353955 ↗
13. Yi, S., and Ahuja, N. (2006, January 20–24). An omnidirectional stereo vision system using a single camera. Proceedings of the 18th International Conference on Pattern Recognition (ICPR’06), Hong Kong, China.
14. Single-camera panoramic stereo imaging system with a fisheye lens and a convex mirror10.1364/OE.19.005855 ↗
15. 3D multi-directional sensor with pyramid mirror and structured light10.1016/j.optlaseng.2017.01.016 ↗
16. Multiview panoramic cameras using mirror pyramids10.1109/TPAMI.2004.33 ↗
17. Schönbein, M., Kitt, B., and Lauer, M. (2018, January 1). Environmental Perception for Intelligent Vehicles Using Catadioptric Stereo Vision Systems. Proceedings of the 5th European Conference on Mobile Robots (ECMR), Berlin, Germany.
18. Eliminating blind spots for assisted driving10.1109/TITS.2008.2006815 ↗
19. An omnidirectional 3D sensor with line laser scanning10.1016/j.optlaseng.2016.04.001 ↗
20. 3D-objecttracking with a mixed omnidirectional stereo camera system10.1016/j.mechatronics.2010.05.007 ↗
21. Jaramillo, C., Valenti, R., Guo, L., and Xiao, J. (2016). Design and Analysis of a Single—Camera Omnistereo Sensor for Quadrotor Micro Aerial Vehicles (MAVs). Sensors, 16.10.3390/s16020217 ↗
22. Visual odometry with a single-camera stereo omnidirectional system10.1007/s00138-019-01041-9 ↗
23. Fringe projection profilometry for panoramic 3D reconstruction10.1016/j.optlaseng.2015.10.004 ↗
24. A panoramic fringe projection system10.1016/j.optlaseng.2014.02.002 ↗
25. Kerkaou, Z., Alioua, N., El Ansari, M., and Masmoudi, L. (2018, January 2–4). A new dense omnidirectional stereo matching approach. Proceedings of the 2018 International Conference on Intelligent Systems and Computer Vision (ISCV), Fez, Morocco.10.1109/ISACV.2018.8354071 ↗
26. Ma, C., Shi, L., Huang, H., and Yan, M. (2015). 3D reconstruction from full-view fisheye camera. arXiv.
27. Song, M., Watanabe, H., and Hara, J. (2018, January 7–9). Robust 3D reconstruction with omni-directional camera based on structure from motion. Proceedings of the 2018 International Workshop on Advanced Image Technology (IWAIT), Chiang Mai, Thailand.10.1109/IWAIT.2018.8369715 ↗
28. Corke, P. (2017). Robotics, Vision and Control: Fundamental Algorithms in MATLAB® Second, Completely Revised, Springer.
29. Boutteau, R., Savatier, X., Ertaud, J.Y., and Mazari, B. (2008, January 17–18). An omnidirectional stereoscopic system for mobile robot navigation. Proceedings of the 2008 International Workshop on Robotic and Sensors Environments, Ottawa, ON, Canada.10.1109/ROSE.2008.4669195 ↗
30. Omnidirectional stereo vision sensor based on single camera and catoptric system10.1364/AO.55.006813 ↗
31. Jang, G., Kim, S., and Kweon, I. (2006, January 1). Single camera catadioptric stereo system. Proceedings of the 6th Workshop on Omnidirectional Vision, Camera Networks and Non-Classical Cameras, Beijing, China.
32. Ragot, N., Rossi, R., Savatier, X., Ertaud, J., and Mazari, B. (July, January 30). 3D volumetric reconstruction with a catadioptric stereovision sensor. Proceedings of the 2008 IEEE International Symposium on Industrial Electronics, Cambridge, UK.10.1109/ISIE.2008.4677128 ↗
33. Monocular depth estimation using multi-scale continuous CRFs as sequential deep networks
34. Xu, D., Ouyang, W., Wang, X., and Sebe, N. (2018, January 18–22). Pad-net: Multi-tasks guided prediction-and-distillation network for simultaneous depth estimation and scene parsing. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00077 ↗
35. Fu, H., Gong, M., Wang, C., Batmanghelich, K., and Tao, D. (2018, January 18–22). Deep ordinal regression network for monocular depth estimation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00214 ↗
36. Won, C., Ryu, J., and Lim, J. (November, January 27). Omnimvs: End-to-end learning for omnidirectional stereo matching. Proceedings of the IEEE International Conference on Computer Vision, Seoul, Korea.
37. Ma, W.C., Wang, S., Hu, R., Xiong, Y., and Urtasun, R. (2019, January 15–21). Deep rigid instance scene flow. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.10.1109/CVPR.2019.00373 ↗
38. Dosovitskiy, A., Fischer, P., Ilg, E., Hausser, P., Hazirbas, C., Golkov, V., Van Der Smagt, P., Cremers, D., and Brox, T. (2015, January 7–13). Flownet: Learning optical flow with convolutional networks. Proceedings of the IEEE International Conference on Computer Vision, Santiago, Chile.10.1109/ICCV.2015.316 ↗
39. Sarlin, P.E., DeTone, D., Malisiewicz, T., and Rabinovich, A. (2020, January 16–18). SuperGlue: Learning Feature Matching with Graph Neural Networks. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.10.1109/CVPR42600.2020.00499 ↗
40. Deepmatching: Hierarchical deformable dense matching10.1007/s11263-016-0908-3 ↗
41. A panoramic 3D reconstruction system based on the projection of patterns10.5772/58227 ↗
42. Scaramuzza, D., Martinelli, A., and Siegwart, R. (2006, January 9–15). A toolbox for easily calibrating omnidirectional cameras. Proceedings of the 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems, Beijing, China.10.1109/IROS.2006.282372 ↗
43. Gonzalez-Barbosa, J.J., and Lacroix, S. (2005, January 18–22). Fast dense panoramic stereovision. Proceedings of the 2005 IEEE International Conference on Robotics and Automation, Barcelona, Spain.
44. Epipolar geometry for central catadioptric cameras10.1023/A:1019869530073 ↗
45. Hu, Y., Song, R., and Li, Y. (2016, January 27–30). Efficient coarse-to-fine patchmatch for large displacement optical flow. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.10.1109/CVPR.2016.615 ↗
46. Harris, C.G., and Stephens, M. (September, January 31). A combined corner and edge detector. Proceedings of the Fourth Alvey Vision Conference, Manchester, UK.
47. Shi, J. (1994, January 21–23). Good features to track. Proceedings of the 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.
48. Distinctive image features from scale-invariant keypoints10.1023/B:VISI.0000029664.99615.94 ↗
49. Bay, H., Tuytelaars, T., and Van Gool, L. (2006). Surf: Speeded up robust features. European Conference on Computer Vision, Springer.10.1007/11744023_32 ↗
50. Leutenegger, S., Chli, M., and Siegwart, R. (2011, January 6–13). BRISK: Binary robust invariant scalable keypoints. Proceedings of the 2011 IEEE international conference on computer vision (ICCV), Barcelona, Spain.10.1109/ICCV.2011.6126542 ↗
51. Rosten, E., and Drummond, T. (2006). Machine learning for high-speed corner detection. European Conference on Computer Vision, Springer.10.1007/11744023_34 ↗
52. Alcantarilla, P.F., Bartoli, A., and Davison, A.J. (2012). KAZE features. European Conference on Computer Vision, Springer.10.1007/978-3-642-33783-3_16 ↗
53. Schönbein, M. (2015). Omnidirectional Stereo Vision for Autonomous Vehicles, KIT Scientific Publishing.
54. The Machine Vision Toolbox: A MATLAB toolbox for vision and vision-based control, Omnidirectional Stereo Vision for Autonomous Vehicles, IEEE Robot10.1109/MRA.2005.1577021 ↗
55. MagicLeap (2020, July 09). SuperGlue Inference and Evaluation Demo Script. Available online: https://github.com/magicleap/SuperGluePretrainedNetwork.
56. Revaud, J. (2020, April 12). DeepMatching: Deep Convolutional Matching. Available online: https://thoth.inrialpes.fr/src/deepmatching/.