Computational Integral Imaging Reconstruction Based on Generative Adversarial Network Super-Resolution
Source abstract
Abstract
To improve acquisition efficiency and achieve super high-resolution reconstruction, a computational integral imaging reconstruction (CIIR) method based on the generative adversarial network (GAN) network is proposed. Firstly, a sparse camera array is used to generate an elemental image array of the 3D object. Then, the elemental image array is mapped to a low-resolution sparse view image. Finally, a lite GAN super-resolution network is presented to up-sample the low-resolution 3D images to high-resolution 3D images with realistic image quality. By removing batch normalization (BN) layers, reducing basic blocks, and adding intra-block operations, better image details and faster generation of super high-resolution images can be achieved. Experimental results demonstrate that the proposed method can effectively enhance the image quality, with the structural similarity (SSIM) reaching over 0.90, and can also reduce the training time by about 20%.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Advanced Optical Imaging Technologies · Digital Holography and Microscopy · Advanced Image Processing Techniques
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
College of Communication Engineering, Jilin University, No. 5372 Nanhu Road, Changchun 130012, China
College of Computer Science and Engineering, Changchun University of Technology, No. 2055 Yan’an Street, Changchun 130012, China
College of International Education, Changchun University of Technology, No. 2055 Yan’an Street, Changchun 130012, China
Funding
National Natural Science Foundation of China (NSFC) · Science and Technology Development Plan of Jilin Province
References (48)
Show source-provided references
1. Integral Imaging with Full Parallax Based on Mini LED Display Unit10.1109/ACCESS.2019.2903146 ↗
2. Light-field imaging and display systems
3. Interactive floating full-parallax digital three-dimensional light-field display based on wavefront recomposing10.1364/OE.26.008883 ↗
4. Large-scale elemental image array generation in integral imaging based on scale invariant feature transform and discrete viewpoint acquisition10.1016/j.displa.2021.102025 ↗
5. Performance metric and objective evaluation for displayed 3D images generated by different lenslet arrays10.1016/j.optcom.2018.05.068 ↗
6. Integral photography using hexagonal fly’s eye lens and fractional view10.1117/12.766247 ↗
7. Halle, M. (1998, January 19–24). Multiple viewpoint rendering. Proceedings of the 25th Annual Conference on Computer Graphics and Interactive Techniques, Orlando, FL, USA.10.1145/280814.280884 ↗
8. High-efficient computer-generated integral imaging based on the backward ray-tracing technique and optical reconstruction10.1364/OE.25.000330 ↗
9. Real-time optical 3D reconstruction based on Monte Carlo integration and recurrent CNNs denoising with the 3D light field display10.1364/OE.27.022198 ↗
10. Three-dimensional volumetric object reconstruction using computational integral imaging10.1364/OPEX.12.000483 ↗
11. Image quality enhancement in 3D computational integral imaging by use of interpolation methods10.1364/OE.15.012039 ↗
12. Integral three-dimensional imaging with digital reconstruction10.1364/OL.26.000157 ↗
13. Lee, E., and Cho, H. (2023). Computational Integral Imaging Reconstruction via Elemental Image Blending without Normalization. Sensors, 23.10.3390/s23125468 ↗
14. Analysis of the noise in back projection light field acquisition and its optimization10.1364/AO.56.000F20 ↗
15. Computational integral imaging reconstruction method of 3D images using pixel-to-pixel mapping and image interpolation10.1016/j.optcom.2009.04.008 ↗
16. Improved 3D integral imaging reconstruction with elemental image pixel rearrangement10.1088/2040-8986/aaa391 ↗
17. Computational reconstruction of three-dimensional integral imaging by rearrangement of elemental image pixels10.1109/JDT.2008.2004857 ↗
18. Visual quality enhancement of integral imaging by using pixel rearrangement technique with convolution operator (CPERTS)10.1016/j.optlaseng.2018.08.010 ↗
19. Resolution-enhanced light field displays by recombining subpixels across elemental images10.1364/OL.44.002438 ↗
20. Scale-variant magnification for computational integral imaging and its application to 3D object correlator10.1364/OE.16.008855 ↗
21. Artifact analysis and image enhancement in three-dimensional computational integral imaging using smooth windowing technique10.1364/OL.36.002107 ↗
22. Improved analysis on the signal property of computational integral imaging system10.1364/OE.15.014107 ↗
23. Improved 3-D image reconstruction using the convolution property of periodic functions in curved integral-imaging10.1016/j.optlaseng.2013.09.011 ↗
24. Free-depths reconstruction with synthetic impulse response in integral imaging10.1364/OE.23.030127 ↗
25. Optical three-dimensional refocusing from elemental images based on a sifting property of the periodic δ-function array in integral imaging10.1364/OE.22.001533 ↗
26. Optical arbitrary-depth refocusing for large-depth scene in integral imaging display based on reprojected parallax image10.1016/j.optcom.2018.10.022 ↗
27. Optical full-depth refocusing of 3-D objects based on subdivided-elemental images and local periodic δ-functions in integral imaging10.1364/OE.24.010359 ↗
28. Depth extraction by using the correlation of the periodic function with an elemental image in integral imaging10.1364/AO.51.003279 ↗
29. Intermediate elemental image reconstruction for refocused three-dimensional images in integral imaging by convolution with -function sequences10.1016/j.optlaseng.2017.04.015 ↗
30. Refocusing-range and image-quality enhanced optical reconstruction of 3-D objects from integral images using a principal periodic δ-function array10.1016/j.optcom.2017.11.032 ↗
31. Nadeem, M.I., and Ahmed, K. (2023). SHO-CNN: A Metaheuristic Optimization of a Convolutional Neural Network for Multi-Label News Classification. Electronics, 12.10.3390/electronics12010113 ↗
32. Light Field Synthesis by Training Deep Network in the Refocused Image Domain10.1109/TIP.2020.2992354 ↗
33. Wang, Y., and Ying, X. (2021, January 20–25). Symmetric parallax attention for stereo image super resolution. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA.10.1109/CVPRW53098.2021.00086 ↗
34. Characterization of abnormalities in breast cancer images using nature-inspired metaheuristic optimized convolutional neural networks model10.1002/cpe.6629 ↗
35. Zivkovic, M., and Bacanin, N. (2022). Hybrid CNN and XGBoost Model Tuned by Modified Arithmetic Optimization Algorithm for COVID-19 Early Diagnostics from X-ray Images. Electronics, 11.10.3390/electronics11223798 ↗
36. A deep journey into super-resolution: A survey10.1145/3390462 ↗
37. Image super-resolution using deep convolutional networks10.1109/TPAMI.2015.2439281 ↗
38. Kim, J., and Lee, J.K. (July, January 26). Accurate image super-resolution using very deep convolutional networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.
39. Zhang, Y., and Tian, Y. (2018, January 18–23). Residual dense network for image super-resolution. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00262 ↗
40. Ledig, C., and Theis, L. (2017, January 21–26). Photo-realistic single image super-resolution using a generative adversarial network. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.10.1109/CVPR.2017.19 ↗
41. Rakotonirina, N.C., and Rasoanaivo, A. (2020, January 4–8). ESRGAN+: Further improving enhanced super-resolution generative adversarial network. Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Barcelona, Spain.10.1109/ICASSP40776.2020.9054071 ↗
42. Dong, C., and Loy, C.C. (2016, January 11–14). Accelerating the super-resolution convolutional neural network. Proceedings of the Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands.10.1007/978-3-319-46475-6_25 ↗
43. Super-multiview integral imaging scheme based on sparse camera array and CNN super-resolution10.1364/AO.58.00A190 ↗
44. Real-time optical reconstruction for a three-dimensional light-field display based on path-tracing and CNN super-resolution10.1364/OE.441714 ↗
45. Wang, X.T., and Yu, K. (2018, January 8–14). ESRGAN: Enhanced Super-Resolution Generative Adversarial Networks. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.10.1007/978-3-030-11021-5_5 ↗
46. Levoy, M., and Hanrahan, P. (1996). Seminal Graphics Papers: Pushing the Boundaries, ACM.
47. Recent progress in three-dimensional information processing based on integral imaging10.1364/AO.48.000H77 ↗
48. The PASCAL visual object classes challenge: A retrospective10.1007/s11263-014-0733-5 ↗