Equal Baseline Camera Array—Calibration, Testbed and Applications
Source abstract
Abstract
This paper presents research on 3D scanning by taking advantage of a camera array consisting of up to five adjacent cameras. Such an array makes it possible to make a disparity map with a higher precision than a stereo camera, however it preserves the advantages of a stereo camera such as a possibility to operate in wide range of distances and in highly illuminated areas. In an outdoor environment, the array is a competitive alternative to other 3D imaging equipment such as Structured-light 3D scanners or Light Detection and Ranging (LIDAR). The considered kinds of arrays are called Equal Baseline Camera Array (EBCA). This paper presents a novel approach to calibrating the array based on the use of self-calibration methods. This paper also introduces a testbed which makes it possible to develop new algorithms for obtaining 3D data from images taken by the array. The testbed was released under open-source. Moreover, this paper shows new results of using these arrays with different stereo matching algorithms including an algorithm based on a convolutional neural network and deep learning technology.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Advanced Vision and Imaging · Optical measurement and interference techniques · Robotics and Sensor-Based Localization
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Faculty of Electronics, Telecommunications and Informatics, Gdansk University of Technology, ul. G. Narutowicza 11/12, 80-233 Gdansk, Poland
Center for Vision, Automation and Control, AIT Austrian Institute of Technology GmbH, Giefinggasse 4, 1210 Vienna, Austria
References (70)
Show source-provided references
1. Acquisition of sharp depth map from multiple cameras10.1016/S0923-5965(98)00025-3 ↗
2. 3D Vision System for a Robotic Arm Based on Equal Baseline Camera Array10.1007/s10846-019-01117-8 ↗
3. Stereo vision with Equal Baseline Multiple Camera Set (EBMCS) for obtaining depth maps of plants10.1016/j.compag.2016.11.022 ↗
4. Choraś, R.S. (2017). Influence of Aggregating Window Size on Disparity Maps Obtained from Equal Baseline Multiple Camera Set (EBMCS). Image Processing and Communications Challenges 8, Springer International Publishing.
5. Kaczmarek, A.L. (2017, January 7–9). Stereo camera upgraded to equal baseline multiple camera set (EBMCS). Proceedings of the 2017 3DTV Conference: The True Vision—Capture, Transmission and Display of 3D Video (3DTV-CON), Copenhagen, Denmark.10.1109/3DTV.2017.8280416 ↗
6. Adaptive Geometric Calibration Correction for Camera Array10.2352/ISSN.2470-1173.2016.13.IQSP-009 ↗
7. Accurate camera calibration from multi-view stereo and bundle adjustment10.1007/s11263-009-0232-2 ↗
8. Antensteiner, D., and Blaschitz, B. (2018, January 15–16). Multi-camera Array Calibration For Light Field Depth Estimation. Proceedings of the Austrian Association for Pattern Recognition Workshop (OAGM), Hall in Tirol, Austria.
9. Scharstein, D., Szeliski, R., and Zabih, R. (2001, January 9–10). A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. Proceedings of the IEEE Workshop on Stereo and Multi-Baseline Vision (SMBV 2001), Kauai, HI, USA.
10. A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms10.1023/A:1014573219977 ↗
11. Geiger, A., Lenz, P., and Urtasun, R. (2012, January 16–21). Are we ready for autonomous driving? The KITTI vision benchmark suite. Proceedings of the 2012 IEEE Conference on Computer Vision and Pattern Recognition, Providence, RI, USA.10.1109/CVPR.2012.6248074 ↗
12. Stereo matching by training a convolutional neural network to compare image patches
13. Experimental Comparison of Optical Inline 3D Measurement and Inspection Systems10.1109/ACCESS.2021.3070381 ↗
14. Structured-light stereo: Comparative analysis and integration of structured-light and active stereo for measuring dynamic shape10.1016/j.optlaseng.2013.05.001 ↗
15. Rasul, A., Seo, J., and Khajepour, A. (2021). Development of Sensing Algorithms for Object Tracking and Predictive Safety Evaluation of Autonomous Excavators. Appl. Sci., 11.10.3390/app11146366 ↗
16. Yim, J.H., Kim, S.Y., Kim, Y., Cho, S., Kim, J., and Ahn, Y.H. (2021). Rapid 3D-Imaging of Semiconductor Chips Using THz Time-of-Flight Technique. Appl. Sci., 11.10.3390/app11114770 ↗
17. Seitz, S.M., Curless, B., Diebel, J., Scharstein, D., and Szeliski, R. (2006, January 17–22). A Comparison and Evaluation of Multi-View Stereo Reconstruction Algorithms. Proceedings of the 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’06), New York, NY, USA.
18. High-precision 3d sensing with hybrid light field & photometric stereo approach in multi-line scan framework10.2352/ISSN.2470-1173.2017.9.IRIACV-268 ↗
19. A multiple-baseline stereo10.1109/34.206955 ↗
20. Wilburn, B., Joshi, N., Vaish, V., Talvala, E.V., Antunez, E., Barth, A., Adams, A., Horowitz, M., and Levoy, M. (2005). High Performance Imaging Using Large Camera Arrays, ACM. ACM SIGGRAPH 2005 Papers.10.1145/1186822.1073259 ↗
21. A Study on Camera Array and Its Applications10.1016/j.ifacol.2017.08.1662 ↗
22. Manta, A. (2008). Multiview Imaging and 3D TV. A Survey, Delft University of Technology, Information and Communication Theory Group.
23. Stereo Vision Depth Estimation Methods for Robotic Applications
24. Ackermann, M., Cox, D., McGraw, J., and Zimmer, P. (2016, January 20–23). Lens and Camera Arrays for Sky Surveys and Space Surveillance. Proceedings of the Advanced Maui Optical and Space Surveillance Technologies Conference, Wailea, HI, USA.10.2172/1561833 ↗
25. PiCam: An Ultra-thin High Performance Monolithic Camera Array10.1145/2508363.2508390 ↗
26. Depth Estimation Using a Sliding Camera10.1109/TIP.2015.2507984 ↗
27. Occlusion Removal Using Depth Mapping in Three-Dimensional Integral Imaging10.1109/JDT.2012.2199960 ↗
28. Hybrid Face Recognition Based on Real-Time Multi-camera Stereo-Matching
29. Trinocular stereo vision for robotics10.1109/34.67633 ↗
30. Mulligan, J., and Kaniilidis, K. (2000, January 3–7). Trinocular stereo for non-parallel configurations. Proceedings of the 15th International Conference on Pattern Recognition, ICPR-2000, Barcelona, Spain.
31. Trinocular Stereo: A Real-Time Algorithm and its Evaluation10.1023/A:1014525320885 ↗
32. Andersen, J.C., Andersen, N.A., and Ravn, O. (2004, January 5–7). Trinocular stereo vision for intelligent robot navigation. Proceedings of the IFAC/EURON Symposium on Intelligent Autonomous Vehicles, Lisbon, Portugal.10.1016/S1474-6670(17)32026-8 ↗
33. Williamson, T., and Thorpe, C. (1999, January 10–15). A trinocular stereo system for highway obstacle detection. Proceedings of the 1999 IEEE International Conference on Robotics and Automation (Cat. No.99CH36288C), Detroit, MI, USA.
34. Three-dimensional integral imaging with flexible sensing10.1364/OL.39.006855 ↗
35. Scharstein, D., Hirschmüller, H., Kitajima, Y., Krathwohl, G., Nešić, N., Wang, X., and Westling, P. (2014, January 2–5). High-Resolution Stereo Datasets with Subpixel-Accurate Ground Truth. Proceedings of the Pattern Recognition: 36th German Conference, GCPR 2014, Münster, Germany.10.1007/978-3-319-11752-2_3 ↗
36. Menze, M., Heipke, C., and Geiger, A. (2015). Joint 3D Estimation of Vehicles and Scene Flow. In Proceedings of the ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume II-3W5, La Grande Motte, France, 28 September–3 October 2015. ISPRS.10.5194/isprsannals-II-3-W5-427-2015 ↗
37. Silberman, N., Hoiem, D., Kohli, P., and Fergus, R. (2012). Indoor Segmentation and Support Inference from RGBD Images. Lecture Notes in Computer Science, Proceedings of the ECCV 2012, Florence, Italy, 7–13 October 2012, Springer.10.1007/978-3-642-33715-4_54 ↗
38. Schöps, T., Schönberger, J.L., Galliani, S., Sattler, T., Schindler, K., Pollefeys, M., and Geiger, A. (2017, January 21–26). A Multi-View Stereo Benchmark with High-Resolution Images and Multi-Camera Videos. Proceedings of the IEEE Proceedings of the Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.10.1109/CVPR.2017.272 ↗
39. Efficient Large-Scale Stereo Matching
40. Stereo Processing by Semiglobal Matching and Mutual Information10.1109/TPAMI.2007.1166 ↗
41. Bradski, D.G.R., and Kaehler, A. (2008). Learning Opencv, O’Reilly Media, Inc.. [1st ed.].
42. Fast approximate energy minimization via graph cuts10.1109/34.969114 ↗
43. On the Statistical Analysis of Dirty Pictures10.1111/j.2517-6161.1986.tb01412.x ↗
44. Goodfellow, I., Bengio, Y., and Courville, A. (2016). Deep Learning, MIT Press. Available online: http://www.deeplearningbook.org.
45. Hartley, R., and Zisserman, A. (2004). Multiple View Geometry in Computer Vision, Cambridge University Press. [2nd ed.].10.1017/CBO9780511811685 ↗
46. Blaschitz, B., Štolc, S., and Antensteiner, D. Geometric calibration and image rectification of a multi-line scan camera for accurate 3d reconstruction. In Proceedings of the IS&T International Symposium on Electronic Imaging, Burlingame, CA, USA, 28 January–1 February 2018.10.2352/ISSN.2470-1173.2018.09.IRIACV-240 ↗
47. Tao, J., Wang, Y., Cai, B., and Wang, K. (2018). Camera Calibration with Phase-Shifting Wedge Grating Array. Appl. Sci., 8.10.3390/app8040644 ↗
48. An efficient image rectification method for parallel multi-camera arrangement10.1109/TCE.2011.6018853 ↗
49. A multi-view image rectification algorithm for matrix camera arrangement
50. Uncalibrated three-view image rectification10.1016/S0262-8856(02)00157-9 ↗
51. Theory and Practice of Projective Rectification10.1023/A:1008115206617 ↗
52. Hosseininaveh, A., Serpico, M., Robson, S., Hess, M., Boehm, J., Pridden, I., and Amati, G. (2012, January 19–21). Automatic Image Selection in Photogrammetric Multi-view Stereo Methods. Proceedings of the 13th International Symposium on Virtual Reality, Archaeology and Cultural Heritage VAST, Brighton, UK.
53. Lowe, D. (1999, January 20–27). Object recognition from local scale-invariant features. Proceedings of the Seventh IEEE International Conference on Computer Vision, Kerkyra, Greece.10.1109/ICCV.1999.790410 ↗
54. Liu, R., Zhang, H., Liu, M., Xia, X., and Hu, T. (2009, January 11–12). Stereo Cameras Self-Calibration Based on SIFT. Proceedings of the 2009 International Conference on Measuring Technology and Mechatronics Automation, Zhangjiajie, China.10.1109/ICMTMA.2009.338 ↗
55. Speeded-Up Robust Features (SURF)10.1016/j.cviu.2007.09.014 ↗
56. Boukamcha, H., Atri, M., and Smach, F. (2017, January 8–10). Robust auto calibration technique for stereo camera. Proceedings of the 2017 International Conference on Engineering MIS (ICEMIS), Monastir, Tunisia.10.1109/ICEMIS.2017.8272992 ↗
57. Online stereo camera calibration for automotive vision based on HW-accelerated A-KAZE-feature extraction10.1016/j.sysarc.2018.11.003 ↗
58. Carrera, G., Angeli, A., and Davison, A.J. (2011, January 9–13). SLAM-based automatic extrinsic calibration of a multi-camera rig. Proceedings of the 2011 IEEE International Conference on Robotics and Automation, Shanghai, China.10.1109/ICRA.2011.5980294 ↗
59. Heng, L., Bürki, M., Lee, G.H., Furgale, P., Siegwart, R., and Pollefeys, M. (June, January 31). Infrastructure-based calibration of a multi-camera rig. Proceedings of the 2014 IEEE International Conference on Robotics and Automation (ICRA), Hong Kong, China.10.1109/ICRA.2014.6907579 ↗
60. Fehrman, B., and McGough, J. (2014, January 6–8). Depth mapping using a low-cost camera array. Proceedings of the 2014 Southwest Symposium on Image Analysis and Interpretation, San Diego, CA, USA.10.1109/SSIAI.2014.6806039 ↗
61. Fehrman, B., and McGough, J. (2014, January 6–8). Handling occlusion with an inexpensive array of cameras. Proceedings of the 2014 Southwest Symposium on Image Analysis and Interpretation, San Diego, CA, USA.10.1109/SSIAI.2014.6806040 ↗
62. Improving depth maps of plants by using a set of five cameras10.1117/1.JEI.24.2.023018 ↗
63. A flexible new technique for camera calibration10.1109/34.888718 ↗
64. Tadic, V., Odry, A., Burkus, E., Kecskes, I., Kiraly, Z., Klincsik, M., Sari, Z., Vizvari, Z., Toth, A., and Odry, P. (2021). Painting Path Planning for a Painting Robot with a RealSense Depth Sensor. Appl. Sci., 11.10.3390/app11041467 ↗
65. Distance measurement system for autonomous vehicles using stereo camera10.1016/j.array.2020.100016 ↗
66. Kopf, C., Pock, T., Blaschitz, B., and Štolc, S. (2021). Inline Double Layer Depth Estimation with Transparent Materials. Lecture Notes in Computer Science, Proceedings of the Pattern Recognition: 42nd DAGM German Conference, DAGM GCPR 2020, Tübingen, Germany, 28 September–1 October 2020, Springer International Publishing.10.1007/978-3-030-71278-5_30 ↗
67. Transparent and Specular Object Reconstruction10.1111/j.1467-8659.2010.01753.x ↗
68. Kaczmarek, A.L., Lebiedź, J., Jaroszewicz, J., and Świȩszkowski, W. (2021, January 17–20). 3D Scanning of Semitransparent Amber with and without Inclusions. Proceedings of the International Conference in Central Europe on Computer Graphics, Visualization and Computer Vision WSCG 2021, Pilsen, Czech Republic.10.24132/CSRN.2021.3101.16 ↗
69. Watson, S., Duecker, D.A., and Groves, K. (2020). Localisation of Unmanned Underwater Vehicles (UUVs) in Complex and Confined Environments: A Review. Sensors, 20.10.3390/s20216203 ↗
70. A Versatile Uav near real-time mapping solution for disaster response—Concept, ideas and implementation