Learning to see high-density random images long-term transmitted in multimode fiber

Xueqing Li; Binbin Song; Jixuan Wu; Wei Lin; Wei Huang; Bo Liu; Xinliang Gao

doi:10.1063/5.0191029

journal article·2024-04-01Open access

Learning to see high-density random images long-term transmitted in multimode fiber

Xueqing Li, Binbin Song, Jixuan Wu, Wei Lin, Wei Huang, Bo Liu, Xinliang Gao

AIP Advances· Volume 14· Issue 4

Publisher page Open full text

Source abstract

Abstract

An improved multi-channel symmetric network (MCSNet) is proposed to reconstruct high-channel-density random images after long-term transmission through multimode fibers (MMFs). Temporal correlation within a period of 25 minutes is calculated to investigate the time-varying characteristics of speckles. The results demonstrated that due to noise accumulation along the MMF path, the quality of speckles deteriorates significantly after long-term transmission. The MCSNet integrates U-Net and ConvNeXt Block, which enables to more fully extract the features of each channel within the entire speckle. After being trained by different random image datasets within the initial moment, tests on random images and realistic scenes of endoscopic surgery after 25 min of transmission are carried out, and all of them demonstrate a near-perfect reconstruction performance and superior scalability, which indicates that MCSNet is suitable for long-term imaging demodulation of endoscopes.

Source: crossref · Quality: baseline

Research context

OpenAlex topics: Random lasers and scattering media · Optical Coherence Tomography Applications · Image and Video Quality Assessment

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

The Engineering Research Center of Learning-Based Intelligent System (Ministry of Education), The Key Laboratory of Computer Vision and Systems (Ministry of Education), and The School of Computer Science and Engineering, Tianjin University of Technology 1 , Tianjin 300384, China

Tianjin Key Laboratory of Optoelectronic Detection Technology and Systems, School of Electronic and Information Engineering, Tiangong University 2 , Tianjin 300387, China

Institute of Modern Optics, Tianjin Key Laboratory of Optoelectronic Sensor and Sensing Network Technology, Nankai University 3 , Tianjin 300350, China

Institute of Modern Optics, Tianjin Key Laboratory of Micro-scale Optical Information Science and Technology, Nankai University 4 , Tianjin 300350, China

Funding

National Natural Science Foundation of China · Natural Science Foundation of Tianjin Municipality · Opening Foundation of Tianjin Key Laboratory of Optoelectronic Detection Technology and Systems · Tianjin Key Laboratory of Micro-scale Optical Information Science and Technology · Tianjin Research Innovation Project for Postgraduate Students

References (24)

Show source-provided references

1. High-fidelity multimode fibre-based endoscopy for deep brain in vivo imaging10.1038/s41377-018-0094-x ↗

2. 3d computational imaging with single-pixel detectors10.1126/science.1234454 ↗

3. Learning to transmit images through optical speckle of a multimode fiber with high fidelity10.1063/5.0099159 ↗

4. Single multimode fiber endoscope10.1364/oe.25.001656 ↗

5. Deep learning the high variability and randomness inside multimode fibers10.1364/oe.27.020241 ↗

6. Measuring the transmission matrix in optics: An approach to the study and control of light propagation in disordered media10.1103/physrevlett.104.100601 ↗

7. Characterization of the angular memory effect of scattered light in biological tissues10.1364/oe.23.013505 ↗

8. Rotational scanning and multiple-spot focusing through a multimode fiber based on digital optical phase conjugation10.7567/apex.11.062501 ↗

9. Anti-perturbation multimode fiber imaging based on the active measurement of the fiber configuration10.1021/acsphotonics.3c00390 ↗

10. Multimode optical fiber transmission with a deep learning network10.1038/s41377-018-0074-1 ↗

11. Multimode fiber-based greyscale image projector enabled by neural networks with high generalization ability10.1364/oe.482551 ↗

12. Binary amplitude-only image reconstruction through a MMF based on an AE-SNN combined deep learning model10.1364/oe.403316 ↗

13. Image transmission through a dynamically perturbed multimode fiber by deep learning10.1002/lpor.202000553 ↗

14. Learning enabled continuous transmission of spatially distributed information through multimode fibers10.1002/lpor.202000348 ↗

15. Intensity and phase imaging through scattering media via deep despeckle complex neural networks10.1016/j.optlaseng.2022.107196 ↗

16. High accuracy transmission and recognition of complex images through multimode fibers using deep learning10.1002/lpor.202200339 ↗

17. Self-supervised dynamic learning for long-term high-fidelity image transmission through unstabilized diffusive media10.1038/s41467-024-45745-7 ↗

18. U-net: Convolutional networks for biomedical image segmentation

19. A ConvNet for the 2020s

20. Image quality assessment: From error visibility to structural similarity10.1109/tip.2003.819861 ↗

21. Image quality metrics: PSNR vs. SSIM

22. Mean squared error: Love it or leave it? A new look at signal fidelity measures10.1109/msp.2008.930649 ↗

23. Swin transformer: Hierarchical vision transformer using shifted windows

24. MobileNetV2: Inverted residuals and linear bottlenecks