All-in-Focus Three-Dimensional Reconstruction Based on Edge Matching for Artificial Compound Eye
Source abstract
Abstract
An artificial compound eye consists of multiple apertures that allow for a large field of view (FOV) while maintaining a small size. Each aperture captures a sub-image, and multiple sub-images are needed to reconstruct the full FOV. The reconstruction process is depth-related due to the parallax between adjacent apertures. This paper presents an all-in-focus 3D reconstruction method for a specific type of artificial compound eye called the electronic cluster eye (eCley). The proposed method uses edge matching to address the edge blur and large textureless areas existing in the sub-images. First, edges are extracted from each sub-image, and then a matching operator is applied to match the edges based on their shape context and intensity. This produces a sparse matching result that is then propagated to the whole image. Next, a depth consistency check and refinement method is performed to refine the depth of all sub-images. Finally, the sub-images and depth maps are merged to produce the final all-in-focus image and depth map. The experimental results and comparative analysis demonstrate the effectiveness of the proposed method.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Image Processing Techniques and Applications · Advanced Vision and Imaging · Optical measurement and interference techniques
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
School of Automation, Chengdu University of Information Technology, Chengdu 610225, China
Funding
National Natural Science Foundation of China · Sichuan Science and Technology Program · the Opening Project of Unmanned System Intelligent Perception Control Technology Engineering Laboratory of Sichuan Province · Key Laboratory of Lidar and Device · Scientific Research Foundation of CUIT
References (53)
Show source-provided references
1. Thin wafer-level camera lenses inspired by insect compound eyes10.1364/OE.18.024379 ↗
2. Development of ultra thin three-dimensional image capturing system
3. Artificial compound eye: A survey of the state-of-the-art10.1007/s10462-016-9513-7 ↗
4. Correcting distortion and braiding of micro-images from multi-aperture imaging systems
5. A taxonomy and evaluation of dense two-frame stereo correspondence algorithms10.1023/A:1014573219977 ↗
6. Stereo matching using belief propagation10.1109/TPAMI.2003.1206509 ↗
7. Stereo processing by semiglobal matching and mutual information10.1109/TPAMI.2007.1166 ↗
8. Fast cost-volume filtering for visual correspondence and beyond10.1109/TPAMI.2012.156 ↗
9. Local similarity pattern and cost self-reassembling for deep stereo matching networks
10. Shen, Z., Dai, Y., Song, X., Rao, Z., Zhou, D., and Zhang, L. (2022, January 23–27). PCW-Net: Pyramid combination and warping cost volume for stereo matching. Proceedings of the European Conference on Computer Vision, Tel Aviv, Israel.10.1007/978-3-031-19824-3_17 ↗
11. Xu, G., Wang, X., Ding, X., and Yang, X. (2023, January 17–24). Iterative geometry encoding volume for stereo matching. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, BC, Canada.10.1109/CVPR52729.2023.02099 ↗
12. A quasi-dense approach to surface reconstruction from uncalibrated images10.1109/TPAMI.2005.44 ↗
13. Zabih, R., and Woodfill, J. (1994, January 2–6). Non-parametric local transforms for computing visual correspondence. Proceedings of the European Conference on Computer Vision, Stockholm, Sweden.10.1007/BFb0028345 ↗
14. Shape matching and object recognition using shape contexts10.1109/34.993558 ↗
15. Thin observation module by bound optics (TOMBO): Concept and experimental verification10.1364/AO.40.001806 ↗
16. Artificial apposition compound eye fabricated by micro-optics technology10.1364/AO.43.004303 ↗
17. Duparré, J., and Wippermann, F. (2006). Micro-optical artificial compound eyes. Bioinspir. Biomim., 1.10.1088/1748-3182/1/1/R01 ↗
18. Artificial neural superposition eye10.1364/OE.15.011922 ↗
19. MULTICAM: A miniature cryogenic camera for infrared detection
20. Optical cluster eye fabricated on wafer-level10.1364/OE.19.017506 ↗
21. Biologically inspired artificial compound eyes10.1126/science.1123053 ↗
22. Digital cameras with designs inspired by the arthropod eye10.1038/nature12083 ↗
23. Miniature curved artificial compound eyes10.1073/pnas.1219068110 ↗
24. Bioinspired artificial eyes: Optic components, digital cameras, and visual prostheses10.1002/adfm.201705202 ↗
25. Cheng, Y., Cao, J., Zhang, Y., and Hao, Q. (2019). Review of state-of-the-art artificial compound eye imaging systems. Bioinspir. Biomim., 14.10.1088/1748-3190/aaffb5 ↗
26. Bio-inspired artificial vision and neuromorphic image processing devices10.1002/admt.202100144 ↗
27. Reconstruction of a high-resolution image on a compound-eye image-capturing system10.1364/AO.43.001719 ↗
28. Image reconstruction for thin observation module by bound optics by using the iterative backprojection method10.1364/AO.45.002893 ↗
29. Three-dimensional information acquisition using a compound imaging system10.1007/s10043-007-0347-z ↗
30. A thin and compact compound-eye imaging system incorporated with an image restoration considering color shift, brightness variation, and defocus10.1007/s10043-009-0045-0 ↗
31. Vision Tape—A flexible compound vision sensor for motion detection and proximity estimation10.1109/JSEN.2011.2166760 ↗
32. A multiaperture bioinspired sensor with hyperacuity10.1109/JSEN.2010.2099112 ↗
33. Motion detection: A biomimetic vision sensor versus a CCD camera sensor10.1109/JSEN.2010.2100039 ↗
34. Depth estimation based on adaptive support weight and SIFT for multi-lenslet cameras
35. Low-power, bio-inspired time-stamp-based 2-D optic flow sensor for artificial compound eyes of micro air vehicles10.1109/JSEN.2019.2938559 ↗
36. Edge detection algorithm for Musca-Domestica inspired vision system10.1109/JSEN.2019.2931661 ↗
37. Depth-estimation-enabled compound eyes10.1016/j.optcom.2017.12.009 ↗
38. Digital focusing and refocusing with thin multi-aperture cameras
39. Ziegler, M., Zilly, F., Schaefer, P., Keinert, J., Schöberl, M., and Foessel, S. (2014, January 27–30). Dense lightfield reconstruction from multi aperture cameras. Proceedings of the 2014 IEEE International Conference on Image Processing (ICIP), Paris, France.10.1109/ICIP.2014.7025388 ↗
40. Jiang, T., Zhu, M., Kuhnert, K.D., and Kuhnert, L. (2014, January 9–10). Distance measuring using calibrating subpixel distances of stereo pixel pairs in artificial compound eye. Proceedings of the 2014 International Conference on Informative and Cybernetics for Computational Social Systems (ICCSS), Qingdao, China.10.1109/ICCSS.2014.6961827 ↗
41. Geometry based three-dimensional image processing method for electronic cluster eye10.3233/ICA-180564 ↗
42. A multi-aperture optical flow estimation method for an artificial compound eye10.3233/ICA-180593 ↗
43. Roadmap on 3D integral imaging: Sensing, processing, and display10.1364/OE.402193 ↗
44. Light field image processing: An overview10.1109/JSTSP.2017.2747126 ↗
45. Bay, H., Tuytelaars, T., and Van Gool, L. (2006, January 7–13). Surf: Speeded up robust features. Proceedings of the European Conference on Computer Vision, Graz, Austria.10.1007/11744023_32 ↗
46. Rosten, E., and Drummond, T. (2005, January 17–21). Fusing points and lines for high performance tracking. Proceedings of the IEEE International Conference on Computer Vision, Beijing, China.10.1109/ICCV.2005.104 ↗
47. Harris, C., and Stephens, M. (September, January 31). A combined corner and edge detector. Proceedings of the Alvey Vision Conference, Manchester, UK.
48. Leutenegger, S., Chli, M., and Siegwart, R.Y. (2011, January 6–13). BRISK: Binary robust invariant scalable keypoints. Proceedings of the IEEE International Conference on Computer Vision, Barcelona, Spain.10.1109/ICCV.2011.6126542 ↗
49. A comparison of affine region detectors10.1007/s11263-005-3848-x ↗
50. A computational approach to edge detection10.1109/TPAMI.1986.4767851 ↗
51. Mean shift: A robust approach toward feature space analysis10.1109/34.1000236 ↗
52. A closed-form solution to natural image matting10.1109/TPAMI.2007.1177 ↗
53. Research of definition assessment based on no-reference digital image quality