HoloForkNet: Digital Hologram Reconstruction via Multibranch Neural Network
Source abstract
Abstract
Reconstruction of 3D scenes from digital holograms is an important task in different areas of science, such as biology, medicine, ecology, etc. A lot of parameters, such as the object’s shape, number, position, rate and density, can be extracted. However, reconstruction of off-axis and especially inline holograms can be challenging due to the presence of optical noise, zero-order image and twin image. We have used a deep-multibranch neural network model, which we call HoloForkNet, to reconstruct different 2D sections of a 3D scene from a single inline hologram. This paper describes the proposed method and analyzes its performance for different types of objects. Both computer-generated and optically registered digital holograms with resolutions up to 2048 × 2048 pixels were reconstructed. High-quality image reconstruction for scenes consisting of up to eight planes was achieved. The average structural similarity index (SSIM) for 3D test scenes with eight object planes was 0.94. The HoloForkNet can be used to reconstruct 3D scenes consisting of micro- and macro-objects.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Digital Holography and Microscopy · Image Processing Techniques and Applications · Cell Image Analysis Techniques
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Laser Physics Department, Institute for Laser and Plasma Technologies, National Research Nuclear University MEPhI (Moscow Engineering Physics Institute), Kashirskoe Shosse 31, 115409 Moscow, Russia
Funding
Russian Science Foundation (RSF)
References (75)
Show source-provided references
1. Schnars, U., Falldorf, C., Watson, J., and Jüptner, W. (2015). Digital Holography and Wavefront Sensing, Springer.10.1007/978-3-662-44693-5 ↗
2. Roadmap on digital holography [Invited]10.1364/OE.435915 ↗
3. Quantitative assessment of cancer cell morphology and motility using telecentric digital holographic microscopy and machine learning10.1002/cyto.a.23316 ↗
4. Extended depth-of-field in holographic imaging using deep-learning-based autofocusing and phase recovery10.1364/OPTICA.5.000704 ↗
5. Li, H., He, G., Song, Q., Xia, H., Liu, Z., Liang, J., and Li, T. (2022). The Study of Tooth Erosion Tested by the Color Digital Holography (CDH) Detection System. Appl. Sci., 12.10.3390/app12178613 ↗
6. Balasubramani, V., Kujawińska, M., Allier, C., Anand, V., Cheng, C.-J., Depeursinge, C., Hai, N., Juodkazis, S., Kalkman, J., and Kuś, A. (2021). Roadmap on Digital Holography-Based Quantitative Phase Imaging. J. Imaging, 7.10.3390/jimaging7120252 ↗
7. Petrov, V., Pogoda, A., Sementin, V., Sevryugin, A., Shalymov, E., Venediktov, D., and Venediktov, V. (2022). Advances in Digital Holographic Interferometry. J. Imaging, 8.10.3390/jimaging8070196 ↗
8. Dyomin, V., Semiletov, I., Chernykh, D., Chertoprud, E., Davydova, A., Kirillov, N., Konovalova, O., Olshukov, A., Osadchiev, A., and Polovtsev, I. (2022). Study of Marine Particles Using Submersible Digital Holographic Camera during the Arctic Expedition. Appl. Sci., 12.10.3390/app122111266 ↗
9. Quantitative phase imaging of biological cells using lensless inline holographic microscopy through sparsity-assisted iterative phase retrieval algorithm10.1063/5.0123677 ↗
10. Microplastic pollution monitoring with holographic classification and deep learning10.1088/2515-7647/abf250 ↗
11. Accurate real-time monitoring of high particulate matter concentration based on holographic speckles and deep learning10.1016/j.jhazmat.2020.124637 ↗
12. Digital holographic microscopy for live cell applications and technical inspection10.1364/AO.47.000A52 ↗
13. Muslimov, E.R., Sakhabutdinov, A.Z., Morozov, O.G., Pavlycheva, N.K., Akhmetov, D.M., and Kharitonov, D.Y. (2022). Digital Holographic Positioning Sensor for a Small Deployable Space Telescope. Appl. Sci., 12.10.3390/app12094427 ↗
14. Digital recording and numerical reconstruction of holograms10.1088/0957-0233/13/9/201 ↗
15. Off-axis digital hologram reconstruction: Some practical considerations10.1364/AO.50.00H136 ↗
16. Numerical Manipulation of Digital Holograms for 3-D Imaging and Display: An Overview10.1109/JPROC.2016.2617892 ↗
17. Cheremkhin, P.A., Kurbatova, E.A., Evtikhiev, N.N., Krasnov, V.V., Rodin, V.G., and Starikov, R.S. (2022). Adaptive Digital Hologram Binarization Method Based on Local Thresholding, Block Division and Error Diffusion. J. Imaging, 8.10.3390/jimaging8020015 ↗
18. Machine learning methods for digital holography and diffractive optics10.1016/j.procs.2020.02.243 ↗
19. Deep learning for digital holography: A review10.1364/OE.443367 ↗
20. Deep holography10.37188/lam.2022.013 ↗
21. Deep learning speckle de-noising algorithms for coherent metrology: A review and a phase-shifted iterative scheme [Invited]10.1364/JOSAA.444951 ↗
22. Spatial filtering for zero-order and twin-image elimination in digital off-axis holography10.1364/AO.39.004070 ↗
23. Twin-image problem in digital holography—A survey (Invited Paper)10.3788/COL201412.060013 ↗
24. Tunable liquid crystal grating based holographic 3D display system with wide viewing angle and large size10.1038/s41377-022-00880-y ↗
25. An optical-digital method of noise suppression in digital holography10.1088/2040-8986/ac90d3 ↗
26. Curved hologram generation method for speckle noise suppression based on the stochastic gradient descent algorithm10.1364/OE.444321 ↗
27. Wang, D., Li, Z.-S., Zheng, Y.-W., Li, N.-N., Li, Y.-L., and Wang, Q.-H. (2022). High-Quality Holographic 3D Display System Based on Virtual Splicing of Spatial Light Modulator. ACS Photonics.10.1021/acsphotonics.2c01514 ↗
28. Phase-shifting digital holography10.1364/OL.22.001268 ↗
29. Wavefront Reconstruction with Diffused Illumination and Three-Dimensional Objects10.1364/JOSA.54.001295 ↗
30. Inline hologram reconstruction with sparsity constraints10.1364/OL.34.003475 ↗
31. From Fienup’s phase retrieval techniques to regularized inversion for in-line holography: Tutorial10.1364/JOSAA.36.000D62 ↗
32. Reconstruction of in-line holograms: Combining model-based and regularized inversion10.1364/OE.27.014951 ↗
33. Twin-Image-Free Holography: A Compressive Sensing Approach10.1103/PhysRevLett.121.093902 ↗
34. Fast phase retrieval in off-axis digital holographic microscopy through deep learning10.1364/OE.26.019388 ↗
35. Tahon, M., Montrésor, S., and Picart, P. (2022). Deep Learning Network for Speckle De-Noising in Severe Conditions. J. Imaging, 8.10.3390/jimaging8060165 ↗
36. Untrained deep network powered with explicit denoiser for phase recovery in inline holography10.1063/5.0144795 ↗
37. Zhang, K., Zuo, W., Gu, S., and Zhang, L. (2017, January 21–26). Learning Deep CNN Denoiser Prior for Image Restoration. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.10.1109/CVPR.2017.300 ↗
38. Automatic phase aberration compensation for digital holographic microscopy based on deep learning background detection10.1364/OE.25.015043 ↗
39. Stitching sub-aperture in digital holography based on machine learning10.1364/OE.387511 ↗
40. Zero-order term suppression in off-axis holography based on deep learning method10.1016/j.optcom.2023.129264 ↗
41. Neural-network-enabled holographic image reconstruction via amplitude and phase extraction10.1364/JOT.89.000511 ↗
42. Jiao, S., Jin, Z., Chang, C., Zhou, C., Zou, W., and Li, X. (2018). Compression of phase-only holograms with JPEG standard and deep learning. Appl. Sci., 8.10.3390/app8081258 ↗
43. Particle field positioning with a commercial microscope based on a developed CNN and the depth-from-defocus method10.1016/j.optlaseng.2022.106989 ↗
44. Machine learning holography for 3D particle field imaging10.1364/OE.379480 ↗
45. Terbe, D., Orzó, L., and Zarándy, A. (2022). Classification of Holograms with 3D-CNN. Sensors, 22.10.3390/s22218366 ↗
46. Focus prediction in digital holographic microscopy using deep convolutional neural networks10.1364/AO.58.00A202 ↗
47. Fast autofocusing using tiny transformer networks for digital holographic microscopy10.1364/OE.458948 ↗
48. HoloPhaseNet: Fully automated deep-learning-based hologram reconstruction using a conditional generative adversarial model10.1364/BOE.452645 ↗
49. Learning-based complex field recovery from digital hologram with various depth objects10.1364/OE.461782 ↗
50. Speeding up reconstruction of 3D tomograms in holographic flow cytometry via deep learning10.1039/D1LC01087E ↗
51. Y-Net: A one-to-two deep learning framework for digital holographic reconstruction10.1364/OL.44.004765 ↗
52. Phase recovery and holographic image reconstruction using deep learning in neural networks10.1038/lsa.2017.141 ↗
53. Fourier Imager Network (FIN): A deep neural network for hologram reconstruction with superior external generalization10.1038/s41377-022-00949-8 ↗
54. Deep DIH: Single-Shot Digital In-Line Holography Reconstruction by Deep Learning10.1109/ACCESS.2020.3036380 ↗
55. Y4-Net: A deep learning solution to one-shot dual-wavelength digital holographic reconstruction10.1364/OL.395445 ↗
56. Holographic optical field recovery using a regularized untrained deep decoder network10.1038/s41598-021-90312-5 ↗
57. End-to-end deep learning framework for digital holographic reconstruction10.1117/1.AP.1.1.016004 ↗
58. Dense-U-net: Dense encoder–decoder network for holographic imaging of 3D particle fields10.1016/j.optcom.2021.126970 ↗
59. A Novel Elastomeric UNet for Medical Image Segmentation10.3389/fnagi.2022.841297 ↗
60. Neven, R., and Goedemé, T. (2021). A Multi-Branch U-Net for Steel Surface Defect Type and Severity Segmentation. Metals, 11.10.3390/met11060870 ↗
61. Convolutional neural network: A review of models, methodologies and applications to object detection10.1007/s13748-019-00203-0 ↗
62. U-Net: Convolutional Networks for Biomedical Image Segmentation
63. U-Net: Deep learning for cell counting, detection, and morphometry10.1038/s41592-018-0261-2 ↗
64. A review on deep learning techniques for cloud detection methodologies and challenges10.1007/s11760-021-01885-7 ↗
65. Transformers in medical image segmentation: A review10.1016/j.bspc.2023.104791 ↗
66. Hao, X., Yin, L., Li, X., Zhang, L., and Yang, R. (2023). A Multi-Objective Semantic Segmentation Algorithm Based on Improved U-Net Networks. Remote Sens., 15.10.3390/rs15071838 ↗
67. Wang, A., Togo, R., Ogawa, T., and Haseyama, M. (2022). Defect Detection of Subway Tunnels Using Advanced U-Net Network. Sensors, 22.10.3390/s22062330 ↗
68. He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27–30). Deep residual learning for image recognition. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.10.1109/CVPR.2016.90 ↗
69. Recent advances in convolutional neural networks10.1016/j.patcog.2017.10.013 ↗
70. Ding, B., Qian, H., and Zhou, J. (2018, January 9–11). Activation functions and their characteristics in deep neural networks. Proceedings of the 2018 Chinese Control And Decision Conference (CCDC), Shenyang, China.10.1109/CCDC.2018.8407425 ↗
71. Shi, W., Caballero, J., Huszár, F., Totz, J., Aitken, A.P., Bishop, R., Rueckert, D., and Wang, Z. (2016). Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network. arXiv.10.1109/CVPR.2016.207 ↗
72. Kingma, D.P., and Ba, J. (2014). Adam: A method for stochastic optimization. arXiv.
73. (2021, March 01). Available online: http://yann.lecun.com/exdb/mnist/.
74. Gonzalez, R.C., and Woods, R.E. (2018). Digital Image Processing, Pearson.
75. Image Quality Assessment: From Error Visibility to Structural Similarity10.1109/TIP.2003.819861 ↗