Monocular Real Time Full Resolution Depth Estimation Arrangement with a Tunable Lens
Source abstract
Abstract
This work introduces a real-time full-resolution depth estimation device, which allows integral displays to be fed with a real-time light-field. The core principle of the technique is a high-speed focal stack acquisition method combined with an efficient implementation of the depth estimation algorithm, allowing the generation of real time, high resolution depth maps. As the procedure does not depend on any custom hardware, if the requirements are met, the described method can turn any high speed camera into a 3D camera with true depth output. The concept was tested with an experimental setup consisting of an electronically variable focus lens, a high-speed camera, and a GPU for processing, plus a control board for lens and image sensor synchronization. The comparison with other state of the art algorithms shows our advantages in computational time and precision.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Advanced Vision and Imaging · Image Processing Techniques and Applications · Optical measurement and interference techniques
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Wooptix S.L., 28005 Madrid, Spain
Industrial Engineering Department, University of La Laguna, 38200 Santa Cruz de Tenerife, Spain
References (40)
Show source-provided references
1. Photogrammetric measurement of 3D freeform millimetre-sized objects with micro features: An experimental validation of the close-range camera calibration model for narrow angles of view10.1088/0957-0233/26/9/095203 ↗
2. Using close range photogrammetry to measure the position of inaccessible geological features10.1111/j.1747-1567.2009.00583.x ↗
3. 3D modeling of Close-Range Objects: Photogrammetry or Laser Scanning10.1117/12.586294 ↗
4. Close-range photogrammetric tools for small 3D archeological objects
5. Lastilla, L., Ravanelli, R., and Ferrara, S. (20, January 8–10). 3D high-quality modeling of small and complex archaeological inscribed objects: Relevant issues and proposed methodology. Proceedings of the GEORES 2019—2nd International Conference of Geomatics and Restoratio, Milan, Italy.10.5194/isprs-archives-XLII-2-W11-699-2019 ↗
6. Design and experimental validation of novel 3D optical scanner with zoom lens unit10.1088/1361-6501/aa82a6 ↗
7. Microsoft Kinect Sensor and Its Effect10.1109/MMUL.2012.24 ↗
8. Christian, J.A., and Cryan, S.P. (2013, January 19–22). A survey of LIDAR technology and its use in spacecraft relative navigation. Proceedings of the AIAA Guidance, Navigation, and Control (GNC) Conference, Boston, MA, USA.10.2514/6.2013-4641 ↗
9. Keselman, L., Woodfill, J.I., Grunnet-Jepsen, A., and Bhowmik, A. (2017, January 21–26). Intel RealSense stereoscopic depth cameras. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, Honolulu, HI, USA.10.1109/CVPRW.2017.167 ↗
10. A efficient and practical 3D face scanner using near infrared and visible photometric stereo10.1016/j.procs.2010.11.003 ↗
11. Infrared system for 3D scanning of metallic surfaces10.1007/s00138-013-0487-z ↗
12. Wang, Y., Lai, Z., Huang, G., Wang, B.H., van der Maaten, L., Campbell, M., and Weinberger, K.Q. (2018). Anytime Stereo Image Depth Estimation on Mobile Devices. arXiv.10.1109/ICRA.2019.8794003 ↗
13. Stereo Processing by Semiglobal Matching and Mutual Information10.1109/TPAMI.2007.1166 ↗
14. Shirai, Y., and Hirose, S. (1998). Small Vision Systems: Hardware and Implementation, Springer. Robotics Research.10.1007/978-1-4471-1580-9 ↗
15. Nyimbili, P., Demirel, H., Seker, D., and Erden, T. (2016, January 27–30). Structure from Motion (SfM)—Approaches and applications. Proceedings of the International Scientific Conference on Applied Sciences, Antalya, Turkey.
16. Schönberger, J.L., and Frahm, J. (2016, January 27–30). Structure-from-Motion revisited. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.10.1109/CVPR.2016.445 ↗
17. A Survey on Structure from Motion10.1017/S096249291700006X ↗
18. Suwajanakorn, S., Hernandez, C., and Seitz, S.M. (2015, January 7–12). Depth from focus with your mobile phone. Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA.10.1109/CVPR.2015.7298972 ↗
19. Hazirbas, C., Soyer, S.G., Staab, M.C., Leal-Taixé, L., and Cremers, D. (2018, January 2–6). Deep depth from focus. Proceedings of the Asian Conference on Computer Vision, Perth, WA, Australia.10.1007/978-3-030-20893-6_33 ↗
20. Fu, H., Gong, M., Wang, C., Batmanghelich, K., and Tao, D. (2018). Deep Ordinal Regression Network for Monocular Depth Estimation. arXiv.10.1109/CVPR.2018.00214 ↗
21. Ranftl, R., Lasinger, K., Hafner, D., Schindler, K., and Koltun, V. (2020). Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfer. arXiv.
22. Real-Time Depth From Focus on a Programmable Focal Plane Processor10.1109/TCSI.2017.2753878 ↗
23. (2020, October 05). Flare 2MP. Available online: http://www.ioindustries.com/flare2mp.html.
24. (2020, October 05). Matrox Radient eV-CL. Available online: https://www.matrox.com/en/imaging/products/components/frame-grabbers/radient-ev-cl.
25. (2020, October 05). C-C-39N0-250. Available online: https://www.corning.com/cala/es/innovation/corning-emerging-innovations/corning-varioptic-lenses/auto-focus-lens-modules-c-c-series/varioptic-C-C-39N0-250.html.
26. Carbone, M., Domeneghetti, D., Cutolo, F., D’Amato, R., Cigna, E., Parchi, P.D., Gesi, M., Morelli, L., Ferrari, M., and Ferrari, V. (2021). Can Liquid Lenses Increase Depth of Field in Head Mounted Video See-Through Devices?. J. Imaging, 7.10.3390/jimaging7080138 ↗
27. Self-Assembled Asymmetric Microlenses for Four-Dimensional Visual Imaging10.1021/acsnano.9b07104 ↗
28. (2020, October 05). Arty Z7. Available online: https://reference.digilentinc.com/reference/programmable-logic/arty-z7/start.
29. Analysis of focus measure operators in shape-from-focus10.1016/j.patcog.2012.11.011 ↗
30. Barron, J.T., and Poole, B. (2016, January 8–16). The fast bilateral solver. Proceedings of the European Conference on Computer Vision, Amsterdam, The Netherlands.10.1007/978-3-319-46487-9_38 ↗
31. Barron, J.T., Adams, A., Shih, Y., and Hernández, C. (2015, January 7–12). Fast bilateral-space stereo for synthetic defocus. Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA.10.1109/CVPR.2015.7299076 ↗
32. Real-time edge-aware image processing with the bilateral grid10.1145/1276377.1276506 ↗
33. Scarano, V., Chiara, R.D., and Erra, U. (2020). MeshLab: An open-source mesh processing tool. Proceeding of the Italian Chapter Conference 2020—Smart Tools and Apps in Computer Graphics, STAG 2020, Virtual Event, Italy, 12–13 November 2020, The Eurographics Association.
34. Depth map sensor based on optical doped lens with multi-walled carbon nanotubes of liquid crystal10.1364/AO.55.000140 ↗
35. Deep Spatialfocal Network for Depth from Focus10.2352/J.ImagingSci.Technol.2021.65.4.040501 ↗
36. Variational Depth From Focus Reconstruction10.1109/TIP.2015.2479469 ↗
37. Relative multiscale deep depth from focus10.1016/j.image.2021.116417 ↗
38. Mousnier, A., Vural, E., and Guillemot, C. (2015). Partial light field tomographic reconstruction from a fixed-camera focal stack. arXiv.
39. Scharstein, D., and Pal, C. (2007, January 17–22). Learning conditional random fields for stereo. Proceedings of the 2007 IEEE Conference on Computer Vision and Pattern Recognition, Minneapolis, MN, USA.10.1109/CVPR.2007.383191 ↗
40. Lee, J., Lee, S., Cho, S., and Lee, S. (2019, January 15–20). Deep defocus map estimation using domain adaptation. Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA.10.1109/CVPR.2019.01250 ↗