journal article·2024-11-18Open access

Transformer-Enhanced Retinal Vessel Segmentation for Diabetic Retinopathy Detection Using Attention Mechanisms and Multi-Scale Fusion

Hyung-Joo Kim, Hassan Eesaar, Kil To Chong

Applied Sciences· Volume 14· Issue 22

Abstract

Eye health has become a significant concern in recent years, given the rising prevalence of visual impairment resulting from various eye disorders and related factors. Global surveys suggest that approximately 2.2 billion individuals are visually impaired, with at least 1 billion affected by treatable diseases or ailments. Early detection, treatment, and screening for fundus diseases are crucial in addressing these challenges. In this study, we propose a novel segmentation model for retinal vascular delineation aimed at diagnosing diabetic retinopathy. The model integrates CBAM (Channel-Attention and Spatial-Attention) for enhanced feature representation, JPU (Joint Pyramid Upsampling) for multi-scale feature fusion, and transformer blocks for contextual understanding. Leveraging deep-learning techniques, our proposed model outperforms existing approaches in retinal vascular segmentation, like achieving a Mean IOU of 0.8047, Recall of 0.7254, Precision of 0.8492, F1 Score of 0.7824, and Specificity of 0.9892 for CHASEDB1 dataset. Extensive evaluations on benchmark datasets demonstrate its efficacy, highlighting its potential for automated diabetic retinopathy screening.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: Retinal Imaging and Analysis · Brain Tumor Detection and Classification · Gaze Tracking and Assistive Technology

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

Electronics and Telecommunications Research Institute, Daejeon 34129, Republic of Korea

Department of Electronics and Information Engineering, Jeonbuk National University, Jeonju 54896, Republic of Korea

Advances Electronics and Information Research Center, Jeonbuk National University, Jeonju 54896, Republic of Korea

Funding

National Research Foundation of Korea

References (52)

Show source-provided references
1. Residual connection-based encoder decoder network (RCED-Net) for retinal vessel segmentation10.1109/ACCESS.2020.3008899 ↗
2. Tungsattayathitthan, U., Rattanalert, N., and Sittivarakul, W. (2023). Long-term visual acuity outcome of pediatric uveitis patients presenting with severe visual impairment. Sci. Rep., 13.10.1038/s41598-023-29159-x ↗
3. Computers in imaging and health care: Now and in the future10.1007/BF03168389 ↗
4. Automated measurement of the arteriolar-to-venular width ratio in digital color fundus photographs10.1109/TMI.2011.2159619 ↗
5. Orlando, J.I., Barbosa Breda, J., Van Keer, K., Blaschko, M.B., Blanco, P.J., and Bulant, C.A. (2018, January 16–20). Towards a glaucoma risk index based on simulated hemodynamics from fundus images. Proceedings of the Medical Image Computing and Computer Assisted Intervention–MICCAI 2018: 21st International Conference, Granada, Spain. Proceedings, Part II 11.10.1007/978-3-030-00934-2_8 ↗
6. Automated retinal image quality assessment on the UK Biobank dataset for epidemiological studies10.1016/j.compbiomed.2016.01.027 ↗
7. Retinal image registration using topological vascular tree segmentation and bifurcation structures10.1016/j.bspc.2014.10.009 ↗
8. End-to-end adversarial retinal image synthesis10.1109/TMI.2017.2759102 ↗
9. Segmentation of vessel-like patterns using mathematical morphology and curvature evaluation10.1109/83.931095 ↗
10. Segmentation of retinal blood vessels by combining the detection of centerlines and morphological reconstruction10.1109/TMI.2006.879955 ↗
11. Robust retinal vessel segmentation via locally adaptive derivative frames in orientation scores10.1109/TMI.2016.2587062 ↗
12. Ryu, J., Rehman, M.U., Nizami, I.F., and Chong, K.T. (2023). SegR-Net: A deep learning framework with multi-scale feature fusion for robust retinal vessel segmentation. Comput. Biol. Med., 163.10.1016/j.compbiomed.2023.107132 ↗
13. A review on recent developments for detection of diabetic retinopathy10.1155/2016/6838976 ↗
14. Diameter changes of retinal vessels in diabetic retinopathy10.1007/s11892-017-0909-9 ↗
15. Mayya, V., Kamath, S., and Kulkarni, U. (2021). Automated microaneurysms detection for early diagnosis of diabetic retinopathy: A Comprehensive review. Comput. Methods Programs Biomed. Update, 1.10.1016/j.cmpbup.2021.100013 ↗
16. Rehman, M.U., Ryu, J., Nizami, I.F., and Chong, K.T. (2023). RAAGR2-Net: A brain tumor segmentation network using parallel processing of multiple spatial frames. Comput. Biol. Med., 152.10.1016/j.compbiomed.2022.106426 ↗
17. Rehman, M.U., Akhtar, S., Zakwan, M., and Mahmood, M.H. (2022). Novel architecture with selected feature vector for effective classification of mitotic and non-mitotic cells in breast cancer histology images. Biomed. Signal Process. Control, 71.10.1016/j.bspc.2021.103212 ↗
18. Segmenting retinal blood vessels with deep neural networks10.1109/TMI.2016.2546227 ↗
19. Ronneberger, O., Fischer, P., and Brox, T. (2015, January 5–9). U-net: Convolutional networks for biomedical image segmentation. Proceedings of the Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany. Proceedings, Part III 18.
20. Joint segment-level and pixel-wise losses for deep learning based retinal vessel segmentation10.1109/TBME.2018.2828137 ↗
21. Segment 2D and 3D filaments by learning structured and contextual features10.1109/TMI.2016.2623357 ↗
22. Dong, S., Zhao, J., Zhang, M., Shi, Z., Deng, J., Shi, Y., Tian, M., and Zhuo, C. (2020, January 4–8). Deu-net: Deformable u-net for 3d cardiac mri video segmentation. Proceedings of the Medical Image Computing and Computer Assisted Intervention–MICCAI 2020: 23rd International Conference, Lima, Peru. Proceedings, Part IV 23.10.1007/978-3-030-59719-1_10 ↗
23. Fu, H., Xu, Y., Lin, S., Kee Wong, D.W., and Liu, J. (2016, January 17–21). Deepvessel: Retinal vessel segmentation via deep learning and conditional random field. Proceedings of the Medical Image Computing and Computer-Assisted Intervention–MICCAI 2016: 19th International Conference, Athens, Greece. Proceedings, Part II 19.10.1007/978-3-319-46723-8_16 ↗
24. Segnet: A deep convolutional encoder-decoder architecture for image segmentation10.1109/TPAMI.2016.2644615 ↗
25. Deep learning attention mechanism in medical image analysis: Basics and beyonds
26. Oktay, O., Schlemper, J., Folgoc, L.L., Lee, M., Heinrich, M., Misawa, K., Mori, K., McDonagh, S., Hammerla, N.Y., and Kainz, B. (2018). Attention u-net: Learning where to look for the pancreas. arXiv.
27. Qin, Y., Song, D., Chen, H., Cheng, W., Jiang, G., and Cottrell, G. (2017). A dual-stage attention-based recurrent neural network for time series prediction. arXiv.10.24963/ijcai.2017/366 ↗
28. Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., and Gelly, S. (2020). An image is worth 16x16 words: Transformers for image recognition at scale. arXiv.
29. Crossformer++: A versatile vision transformer hinging on cross-scale attention10.1109/TPAMI.2023.3341806 ↗
30. Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs10.1109/TPAMI.2017.2699184 ↗
31. Zhao, H., Shi, J., Qi, X., Wang, X., and Jia, J. (2017, January 21–26). Pyramid scene parsing network. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.10.1109/CVPR.2017.660 ↗
32. Islam, M.M., Poly, T.N., Walther, B.A., Yang, H.C., and Li, Y.C. (2020). Artificial intelligence in ophthalmology: A meta-analysis of deep learning models for retinal vessels segmentation. J. Clin. Med., 9.10.3390/jcm9041018 ↗
33. Xiao, X., Lian, S., Luo, Z., and Li, S. (2018, January 19–21). Weighted res-unet for high-quality retina vessel segmentation. Proceedings of the 2018 9th International Conference on Information Technology in Medicine and Education (ITME), Hangzhou, China.10.1109/ITME.2018.00080 ↗
34. Zhang, J., Zhang, Y., and Xu, X. (2021, January 6–12). Pyramid u-net for retinal vessel segmentation. Proceedings of the ICASSP 2021–2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Toronto, ON, Canada.10.1109/ICASSP39728.2021.9414164 ↗
35. Liu, X., Zhang, D., Yao, J., and Tang, J. (2023). Transformer and convolutional based dual branch network for retinal vessel segmentation in OCTA images. Biomed. Signal Process. Control, 83.10.1016/j.bspc.2023.104604 ↗
36. Chen, D., Yang, W., Wang, L., Tan, S., Lin, J., and Bu, W. (2022). PCAT-UNet: UNet-like network fused convolution and transformer for retinal vessel segmentation. PLoS ONE, 17.10.1371/journal.pone.0262689 ↗
37. MBHFuse: A multi-branch heterogeneous global and local infrared and visible image fusion with differential convolutional amplification features10.1016/j.optlastec.2024.111666 ↗
38. Global attention network with multiscale feature fusion for infrared small target detection10.1016/j.optlastec.2023.110012 ↗
39. AM-SegNet for additive manufacturing in situ X-ray image segmentation and feature quantification10.1080/17452759.2024.2325572 ↗
40. Zhou, W., Bai, W., Ji, J., Yi, Y., Zhang, N., and Cui, W. (2023). Dual-path multi-scale context dense aggregation network for retinal vessel segmentation. Comput. Biol. Med., 164.10.1016/j.compbiomed.2023.107269 ↗
41. Liu, Y., Shen, J., Yang, L., Bian, G., and Yu, H. (2023). ResDO-UNet: A deep residual network for accurate retinal vessel segmentation from fundus images. Biomed. Signal Process. Control, 79.10.1016/j.bspc.2022.104087 ↗
42. Jiang, Y., Liang, J., Cheng, T., Lin, X., Zhang, Y., and Dong, J. (2022). MTPA_Unet: Multi-scale transformer-position attention retinal vessel segmentation network joint transformer and CNN. Sensors, 22.10.3390/s22124592 ↗
43. Xie, Y., Shang, J., Yang, Q., Qian, X., Zhang, H., and Tang, X. (2024). ARSA-UNet: Atrous residual network based on Structure-Adaptive model for retinal vessel segmentation. Biomed. Signal Process. Control, 96.10.1016/j.bspc.2024.106595 ↗
44. A combination of multi-scale and attention based on the U-shaped network for retinal vessel segmentation10.1002/ima.23045 ↗
45. Gabor-net with multi-scale hierarchical fusion of features for fundus retinal blood vessel segmentation10.1016/j.bbe.2024.05.004 ↗
46. Ridge-based vessel segmentation in color images of the retina10.1109/TMI.2004.825627 ↗
47. An ensemble classification-based approach applied to retinal blood vessel segmentation10.1109/TBME.2012.2205687 ↗
48. Locating blood vessels in retinal images by piecewise threshold probing of a matched filter response10.1109/42.845178 ↗
49. Budai, A., Bock, R., Maier, A., Hornegger, J., and Michelson, G. (2013). Robust vessel segmentation in fundus images. Int. J. Biomed. Imaging, 2013.10.1155/2013/154860 ↗
50. Advanced drone-based weed detection using feature-enriched deep learning approach10.1016/j.knosys.2024.112655 ↗
51. Stimulus-guided adaptive transformer network for retinal blood vessel segmentation in fundus images10.1016/j.media.2023.102929 ↗
52. Zhang, H., Ni, W., Luo, Y., Feng, Y., Song, R., and Wang, X. (2023). TUnet-LBF: Retinal fundus image fine segmentation model based on transformer Unet network and LBF. Comput. Biol. Med., 159.10.1016/j.compbiomed.2023.106937 ↗