Polarformer: Optic Disc and Cup Segmentation Using a Hybrid CNN-Transformer and Polar Transformation
Source abstract
Abstract
The segmentation of optic disc (OD) and optic cup (OC) are used in the automatic diagnosis of glaucoma. However, the spatially ambiguous boundary and semantically uncertain region-of-interest area in pictures may lead to the degradation of the performance of precise segmentation of the OC and OD. Unlike most existing methods, including the variants of CNNs (Convolutional Neural Networks) and U-Net, which limit the contributions of rich global features, we instead propose a hybrid CNN-transformer and polar transformation network, dubbed as Polarformer, which aims to extract discriminative and semantic features for robust OD and OC segmentation. Our Polarformer typically exploits contextualized features among all input units and models the correlation of structural relationships under the paradigm of the transformer backbone. More specifically, our learnable polar transformer module optimizes the polar transformations by sampling images in the Cartesian space and then mapping them back to the polar coordinate system for masked-image reconstruction. Extensive experimental results present that our Polarformer achieves superior performance in comparison to most state-of-the-art methods on three publicly available datasets.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Retinal Imaging and Analysis · Glaucoma and retinal disorders · Medical Imaging and Analysis
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
School of Information Engineering, Ningxia University, Yinchuan 750021, China
Collaborative Innovation Center for Ningxia Big Data and Artificial Intelligence Co-Founded by Ningxia Municipality and Ministry of Education, Yinchuan 750021, China
Funding
National Science Foundation of China · Key Research and Development Program of Ningxia Hui Autonomous Region · National Science Foundation of Ningxia
References (49)
Show source-provided references
1. Optic disc and optic cup segmentation methodologies for glaucoma image detection: A survey10.1155/2015/180972 ↗
2. Lin, L., Wang, Z., Wu, J., Huang, Y., Lyu, J., Cheng, P., Wu, J., and Tang, X. (October, January 27). Bsda-net: A boundary shape and distance aware joint learning framework for segmenting and classifying octa images. Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention, Strasbourg, France.
3. Detecting the optic disc boundary in digital fundus images using morphological, edge detection, and feature extraction techniques10.1109/TMI.2010.2053042 ↗
4. Template matching via densities on the roto-translation group10.1109/TPAMI.2017.2652452 ↗
5. Joint optic disc and cup boundary extraction from monocular fundus images10.1016/j.cmpb.2017.06.004 ↗
6. Joint optic disc and cup segmentation based on multi-label deep network and polar transformation10.1109/TMI.2018.2791488 ↗
7. Li, S., Sui, X., Luo, X., Xu, X., Liu, Y., and Goh, R. (2021). Medical image segmentation using squeeze-and-expansion transformers. arXiv.10.24963/ijcai.2021/112 ↗
8. Optic disc and cup segmentation methods for glaucoma detection with modification of U-Net convolutional neural network10.1134/S1054661817030269 ↗
9. Segmentation of optic disc, fovea and retinal vasculature using a single convolutional neural network10.1016/j.jocs.2017.02.006 ↗
10. Cheng, P., Lin, L., Huang, Y., Lyu, J., and Tang, X. (2021, January 13–16). Prior guided fundus image quality enhancement via contrastive learning. Proceedings of the 2021 IEEE 18th International Symposium on Biomedical Imaging (ISBI), Nice, France.10.1109/ISBI48211.2021.9434005 ↗
11. Huang, Y., Zhong, Z., Yuan, J., and Tang, X. (2020). Efficient and robust optic disc detection and fovea localization using region proposal network and cascaded network. Biomed. Signal Process. Control, 60.10.1016/j.bspc.2020.101939 ↗
12. Liu, S., Qi, L., Qin, H., Shi, J., and Jia, J. (2018, January 18–23). Path aggregation network for instance segmentation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00913 ↗
13. Optic disc segmentation in fundus images using adversarial training10.1049/iet-ipr.2018.5922 ↗
14. Mohan, D., Kumar, J.H., and Seelamantula, C.S. (2018, January 7–10). High-performance optic disc segmentation using convolutional neural networks. Proceedings of the 2018 25th IEEE International Conference on Image Processing (ICIP), Athens, Greece.10.1109/ICIP.2018.8451543 ↗
15. Long, J., Shelhamer, E., and Darrell, T. (2015, January 7–12). Fully convolutional networks for semantic segmentation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA.10.1109/CVPR.2015.7298965 ↗
16. Zhang, Z., Fu, H., Dai, H., Shen, J., Pang, Y., and Shao, L. (2019, January 13–17). Et-net: A generic edge-attention guidance network for medical image segmentation. Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention, Shenzhen, China.10.1007/978-3-030-32239-7_49 ↗
17. Yin, P., Wu, Q., Xu, Y., Min, H., Yang, M., Zhang, Y., and Tan, M. (2019, January 13–17). PM-Net: Pyramid multi-label network for joint optic disc and cup segmentation. Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention, Shenzhen, China.10.1007/978-3-030-32239-7_15 ↗
18. Salehinejad, H., Valaee, S., Dowdell, T., and Barfett, J. (2018, January 15–20). Image augmentation using radial transform for training deep neural networks. Proceedings of the ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing(ICASSP), Calgary, AB, Canada.10.1109/ICASSP.2018.8462241 ↗
19. Liu, Q., Hong, X., Ke, W., Chen, Z., and Zou, B. (2019). DDNet: Cartesian-polar dual-domain network for the joint optic disc and cup segmentation. arXiv.10.1016/j.neucom.2019.05.039 ↗
20. Fast optic disc segmentation in retina using polar transform10.1109/ACCESS.2017.2723320 ↗
21. Jiang, R., and Mei, S. (2019, January 22–25). Polar coordinate convolutional neural network: From rotation-invariance to translation-invariance. Proceedings of the 2019 IEEE International Conference on Image Processing (ICIP), Taipei, Taiwan.10.1109/ICIP.2019.8802940 ↗
22. Kim, J., Jung, W., Kim, H., and Lee, J. (2020). Cycnn: A rotation invariant cnn using polar mapping and cylindrical convolution layers. arXiv.
23. Attention is all you need
24. Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., and Gelly, S. (2020). An image is worth 16x16 words: Transformers for image recognition at scale. arXiv.
25. Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., and Guo, B. (2021, January 10–17). Swin transformer: Hierarchical vision transformer using shifted windows. Proceedings of the IEEE/CVF International Conference on Computer Vision, Montreal, BC, Canada.10.1109/ICCV48922.2021.00986 ↗
26. Carion, N., Massa, F., Synnaeve, G., Usunier, N., Kirillov, A., and Zagoruyko, S. (2020, January 23–28). End-to-end object detection with transformers. Proceedings of the European Conference on Computer Vision, Tel Aviv, Israel.10.1007/978-3-030-58452-8_13 ↗
27. Toshev, A., and Szegedy, C. (2014, January 23–28). Deeppose: Human pose estimation via deep neural networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA.10.1109/CVPR.2014.214 ↗
28. Wei, S.E., Ramakrishna, V., Kanade, T., and Sheikh, Y. (2016, January 27–30). Convolutional pose machines. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.10.1109/CVPR.2016.511 ↗
29. Zhang, F., Zhu, X., Dai, H., Ye, M., and Zhu, C. (2020, January 13–19). Distribution-aware coordinate representation for human pose estimation. Proceedings of the IEEE/CVF conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.10.1109/CVPR42600.2020.00712 ↗
30. Optic disk and cup segmentation through fuzzy broad learning system for glaucoma screening10.1109/TII.2020.3000204 ↗
31. The canonical coordinates method for pattern deformation: Theoretical and computational considerations10.1109/34.177382 ↗
32. Spatial transformer networks
33. Zheng, S., Lu, J., Zhao, H., Zhu, X., Luo, Z., Wang, Y., Fu, Y., Feng, J., Xiang, T., and Torr, P.H. (2021, January 20–25). Rethinking semantic segmentation from a sequence-to-sequence perspective with transformers. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA.10.1109/CVPR46437.2021.00681 ↗
34. Chen, J., Lu, Y., Yu, Q., Luo, X., Adeli, E., Wang, Y., Lu, L., Yuille, A.L., and Zhou, Y. (2021). Transunet: Transformers make strong encoders for medical image segmentation. arXiv.
35. Refuge challenge: A unified framework for evaluating automated methods for glaucoma assessment from fundus photographs10.1016/j.media.2019.101570 ↗
36. Sivaswamy, J., Krishnadas, S.R., Datt Joshi, G., Jain, M., and Syed Tabish, A.U. (May, January 28). Drishti-GS: Retinal image dataset for optic nerve head(ONH) segmentation. Proceedings of the 2014 IEEE 11th International Symposium on Biomedical Imaging (ISBI), Beijing, China.10.1109/ISBI.2014.6867807 ↗
37. Fumero, F., Alayon, S., Sanchez, J.L., Sigut, J., and Gonzalez-Hernandez, M. (2011, January 27–30). RIM-ONE: An Open Retinal Image Database for Optic Nerve Evaluation. Proceedings of the 2011 24th International Symposium on Computer-Based Medical Systems, Bristol, UK. CBMS ’11.10.1109/CBMS.2011.5999143 ↗
38. Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., and Fei-Fei, L. (2009, January 20–25). Imagenet: A large-scale hierarchical image database. Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition, Miami, FL, USA.10.1109/CVPR.2009.5206848 ↗
39. Ronneberger, O., Fischer, P., and Brox, T. (2015). U-Net: Convolutional networks for biomedical image segmentation. arXiv.10.1007/978-3-319-24574-4_28 ↗
40. Zhou, Z., Rahman Siddiquee, M.M., Tajbakhsh, N., and Liang, J. (2018). Unet++: A nested u-net architecture for medical image segmentation. Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support, Springer.10.1007/978-3-030-00889-5_1 ↗
41. Huang, H., Lin, L., Tong, R., Hu, H., Zhang, Q., Iwamoto, Y., Han, X., Chen, Y.W., and Wu, J. (2020, January 4–8). Unet 3+: A full-scale connected unet for medical image segmentation. Proceedings of the ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing(ICASSP), Barcelona, Spain.10.1109/ICASSP40776.2020.9053405 ↗
42. Oktay, O., Schlemper, J., Folgoc, L.L., Lee, M., Heinrich, M., Misawa, K., Mori, K., McDonagh, S., Hammerla, N.Y., and Kainz, B. (2018). Attention u-net: Learning where to look for the pancreas. arXiv.
43. Joint optic disc and optic cup segmentation based on boundary prior and adversarial learning10.1007/s11548-021-02373-6 ↗
44. NENet: Nested EfficientNet and adversarial learning for joint optic disc and cup segmentation10.1016/j.media.2021.102253 ↗
45. Fan, D.P., Ji, G.P., Zhou, T., Chen, G., Fu, H., Shen, J., and Shao, L. (2020, January 4–8). Pranet: Parallel reverse attention network for polyp segmentation. Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention, Lima, Peru.10.1007/978-3-030-59725-2_26 ↗
46. nnU-Net: A self-configuring method for deep learning-based biomedical image segmentation10.1038/s41592-020-01008-z ↗
47. DeepLabv3+ _encoder-decoder with Atrous separable convolution for semantic image segmentation
48. He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 26–30). Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.10.1109/CVPR.2016.90 ↗
49. Tan, M., and Le, Q. (2019, January 10–15). Efficientnet: Rethinking model scaling for convolutional neural networks. Proceedings of the International Conference on Machine Learning, Long Beach, CA, USA.