journal article·2022-10-21Open access

Series-Parallel Generative Adversarial Network Architecture for Translating from Fundus Structure Image to Fluorescence Angiography

Yiwei Chen, Yi He, Wanyue Li, Jing Wang, Ping Li, Lina Xing, Xin Zhang, Guohua Shi

Applied Sciences· Volume 12· Issue 20

Abstract

Although fundus fluorescein angiography (FFA) is a very effective retinal imaging tool for ophthalmic diagnosis, the requirement of intravenous injection of harmful fluorescein dye limits its application. As a screening diagnostic method that reduces the frequency of intravenous injection, a series-parallel generative adversarial network (GAN) architecture for translating fundus structure image to FFA images is proposed herein, using deep learning-based software that only needs an intravenous injection for the training process. Firstly, the fundus structure image and the corresponding FFA images of three phases are collected. Secondly, our series-parallel GAN is trained to translate FFA images from fundus structure image with the supervision of FFA images. Thirdly, the trained series-parallel GAN model is used to translate FFA images by only using fundus structure image. By comparing the FFA images translated by our algorithm, Sequence GAN, pix2pix, and cycleGAN, we show the advancement of our algorithm. To further confirm the advancements of our algorithm, we evaluate the peak signal-to-noise ratio (PSNR), structural similarity (SSIM) index, and mean-squared error (MSE) of our algorithm, Sequence GAN, pix2pix, and cycleGAN. To demonstrate the performance of our method, we show some typical FFA images translated by our algorithm.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: Retinal Imaging and Analysis · Optical Coherence Tomography Applications · Retinal and Optic Conditions

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

Jiangsu Key Laboratory of Medical Optics, Suzhou Institute of Biomedical Engineering and Technology, Chinese Academy of Sciences, Suzhou 215163, China

Department of Biomedical Engineering, University of Science and Technology of China, Hefei 230041, China

School of Optoelectronic Science and Engineering, University of Electronic Science and Technology of China, Chengdu 611731, China

Center for Excellence in Brain Science and Intelligence Technology, Chinese Academy of Sciences, Shanghai 200031, China

Funding

Gusu Innovation and Entrepreneurship Leading Talents in Suzhou City · Jiangsu Province Key R&D Program · Natural Science Foundation of Jiangsu Province · National Key R&D Program of China · National Natural Science Foundation of China · Youth Innovation Promotion Association of Chinese Academy of Sciences · Frontier Science Research Project of the Chinese Academy of Sciences · Strategic Priority Research Program of the Chinese Academy of Sciences

References (24)

Show source-provided references
1. Ocular vascular occlusive disorders: Natural history of visual outcome10.1016/j.preteyeres.2014.04.001 ↗
2. Retinal and ophthalmic artery occlusions preferred practice pattern®10.1016/j.ophtha.2019.09.028 ↗
3. Retinal vascular layers imaged by fluorescein angiography and optical coherence tomography angiography10.1001/jamaophthalmol.2014.3616 ↗
4. Peripheral areas of nonperfusion in treated central retinal vein occlusion as imaged by wide-field fluorescein angiography10.1097/IAE.0b013e31820c841e ↗
5. Ultra-widefield fundus fluorescein angiography in pediatric retinal vascular diseases10.4103/ijo.IJO_1688_18 ↗
6. Peripheral retinal ischemia, as evaluated by ultra-widefield fluorescein angiography, is associated with macular edema in patients with diabetic retinopathy
7. Ultra-wide-field fluorescein angiography of the ocular fundus10.1016/j.ajo.2005.02.055 ↗
8. Indocyanine green angiography10.1136/bjo.80.3.263 ↗
9. Eye-motion-corrected optical coherence tomography angiography using Lissajous scanning10.1364/BOE.9.001111 ↗
10. Kamran, S.A., Hossain, K.F., Tavakkoli, A., Zuckerbrod, S., Baker, S.A., and Sanders, K.M. (2020, January 5–7). Fundus2Angio: A conditional GAN architecture for generating fluorescein angiography images from retinal fundus photography. Proceedings of the International Symposium on Visual Computing, San Diego, CA, USA.10.1007/978-3-030-64559-5_10 ↗
11. Kamran, S.A., Hossain, K.F., Tavakkoli, A., and Zuckerbrod, S.L. (2021, January 10–15). Attention2angiogan: Synthesizing fluorescein angiography from retinal fundus images using generative adversarial networks. Proceedings of the 2020 25th International Conference on Pattern Recognition (ICPR), Milan, Italy.10.1109/ICPR48806.2021.9412428 ↗
12. Hervella, Á.S., Rouco, J., Novo, J., and Ortega, M. (2018, January 16–20). Retinal image understanding emerges from self-supervised multimodal reconstruction. Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention, Granada, Spain.10.1007/978-3-030-00928-1_37 ↗
13. Li, W., Kong, W., Chen, Y., Wang, J., He, Y., Shi, G., and Deng, G. (2020). Generating fundus fluorescence angiography images from structure fundus images using generative adversarial networks. arXiv.
14. A novel deep learning conditional generative adversarial network for producing angiography images from retinal fundus photographs10.1038/s41598-020-78696-2 ↗
15. Schiffers, F., Yu, Z., Arguin, S., Maier, A., and Ren, Q. (2018). Synthetic fundus fluorescein angiography using deep neural networks. Bildverarbeitung für die Medizin 2018, Springer Vieweg.10.1007/978-3-662-56537-7_64 ↗
16. Li, K., Yu, L., Wang, S., and Heng, P.A. (2019, January 13). Unsupervised retina image synthesis via disentangled representation learning. Proceedings of the International Workshop on Simulation and Synthesis in Medical Imaging, Shenzhen, China.10.1007/978-3-030-32778-1_4 ↗
17. Cai, Z., Xin, J., Wu, J., Liu, S., Zuo, W., and Zheng, N. (2020, January 20–24). Triple multi-scale adversarial learning with self-attention and quality loss for unpaired fundus fluorescein angiography synthesis. Proceedings of the 2020 42nd Annual International Conference of the IEEE Engineering in Medicine Biology Society (EMBC), Montreal, QC, Canada.10.1109/EMBC44109.2020.9176302 ↗
18. Hervella, Á.S., Rouco, J., Novo, J., and Ortega, M. (2019, January 14–19). Deep multimodal reconstruction of retinal images using paired or unpaired data. Proceedings of the 2019 International Joint Conference on Neural Networks (IJCNN), Budapest, Hungary.10.1109/IJCNN.2019.8852082 ↗
19. Li, W., He, Y., Kong, W., Wang, J., Deng, G., Chen, Y., and Shi, G. (2021, January 27). SequenceGAN: Generating fundus fluorescence angiography sequences from structure fundus image. Proceedings of the International Workshop on Simulation and Synthesis in Medical Imaging, Strasbourg, France.10.1007/978-3-030-87592-3_11 ↗
20. A review of central retinal artery occlusion: Clinical presentation and management10.1038/eye.2013.25 ↗
21. Retinal-vein occlusion10.1056/NEJMcp1003934 ↗
22. Isola, P., Zhu, J.Y., Zhou, T., and Efros, A.A. (2010, January 13–18). Image-to-image translation with conditional adversarial networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, San Francisco, CA, USA.
23. Zhu, J.Y., Park, T., Isola, P., and Efros, A.A. (2017, January 22–29). Unpaired image-to-image translation using cycle-consistent adversarial networks. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.10.1109/ICCV.2017.244 ↗
24. Image quality assessment: From error visibility to structural similarity10.1109/TIP.2003.819861 ↗