Dealing with Unreliable Annotations: A Noise-Robust Network for Semantic Segmentation through A Transformer-Improved Encoder and Convolution Decoder
Source abstract
Abstract
Conventional deep learning methods have shown promising results in the medical domain when trained on accurate ground truth data. Pragmatically, due to constraints like lack of time or annotator inexperience, the ground truth data obtained from clinical environments may not always be impeccably accurate. In this paper, we investigate whether the presence of noise in ground truth data can be mitigated. We propose an innovative and efficient approach that addresses the challenge posed by noise in segmentation labels. Our method consists of four key components within a deep learning framework. First, we introduce a Vision Transformer-based modified encoder combined with a convolution-based decoder for the segmentation network, capitalizing on the recent success of self-attention mechanisms. Second, we consider a public CT spine segmentation dataset and devise a preprocessing step to generate (and even exaggerate) noisy labels, simulating real-world clinical situations. Third, to counteract the influence of noisy labels, we incorporate an adaptive denoising learning strategy (ADL) into the network training. Finally, we demonstrate through experimental results that the proposed method achieves noise-robust performance, outperforming existing baseline segmentation methods across multiple evaluation metrics.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: COVID-19 diagnosis using AI · Medical Imaging and Analysis · AI in cancer detection
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Department of Computer Science, University of Oxford, Oxford OX1 3QG, UK
References (41)
Show source-provided references
1. Ronneberger, O., Fischer, P., and Brox, T. (2015, January 5–9). U-net: Convolutional networks for biomedical image segmentation. Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention, Munich, Germany.10.1007/978-3-319-24574-4_28 ↗
2. Zhou, Z., Siddiquee, M.M.R., Tajbakhsh, N., and Liang, J. (2018). Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support, Springer.
3. Wang, Z., and Voiculescu, I. (2021, January 1–5). Quadruple augmented pyramid network for multi-class COVID-19 segmentation via CT. Proceedings of the 2021 43rd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC), Guadalajara, Mexico.10.1109/EMBC46164.2021.9629904 ↗
4. Gao, Y., Guo, J., Fu, C., Wang, Y., and Cai, S. (2023). VLSM-Net: A Fusion Architecture for CT Image Segmentation. Appl. Sci., 13.10.3390/app13074384 ↗
5. Ultrasound image segmentation: A survey10.1109/TMI.2006.877092 ↗
6. Wang, Z., and Voiculescu, I. (2022, January 22). Triple-view feature learning for medical image segmentation. Proceedings of the Resource-Efficient Medical Image Analysis: First MICCAI Workshop, REMIA 2022, Singapore.10.1007/978-3-031-16876-5_5 ↗
7. Benjdira, B., Ouni, K., Al Rahhal, M.M., Albakr, A., Al-Habib, A., and Mahrous, E. (2020). Spinal cord segmentation in ultrasound medical imagery. Appl. Sci., 10.10.3390/app10041370 ↗
8. Wang, Z. (2020). Deep learning in medical ultrasound image segmentation: A review. arXiv.
9. Chen, J., Lu, Y., Yu, Q., Luo, X., Adeli, E., Wang, Y., Lu, L., Yuille, A.L., and Zhou, Y. (2021). Transunet: Transformers make strong encoders for medical image segmentation. arXiv.
10. Milletari, F., Navab, N., and Ahmadi, S.A. (2016, January 25–28). V-net: Fully convolutional neural networks for volumetric medical image segmentation. Proceedings of the 2016 fourth international conference on 3D vision (3DV), Stanford, CA, USA.10.1109/3DV.2016.79 ↗
11. Oktay, O., Schlemper, J., Folgoc, L.L., Lee, M., Heinrich, M., Misawa, K., Mori, K., McDonagh, S., Hammerla, N.Y., and Kainz, B. (2018). Attention u-net: Learning where to look for the pancreas. arXiv.
12. Çiçek, Ö., Abdulkadir, A., Lienkamp, S.S., Brox, T., and Ronneberger, O. (2016, January 17–21). 3D U-Net: Learning dense volumetric segmentation from sparse annotation. Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention, Athens, Greece.10.1007/978-3-319-46723-8_49 ↗
13. Kolařík, M., Burget, R., Uher, V., Říha, K., and Dutta, M.K. (2019). Optimized high resolution 3d dense-u-net network for brain and spine segmentation. Appl. Sci., 9.10.3390/app9030404 ↗
14. Fully Dense UNet for 2-D Sparse Photoacoustic Tomography Artifact Removal10.1109/JBHI.2019.2912935 ↗
15. Resunet-a: A deep learning framework for semantic segmentation of remotely sensed data10.1016/j.isprsjprs.2020.01.013 ↗
16. Atrous residual interconnected encoder to attention decoder framework for vertebrae segmentation via 3D volumetric CT images10.1016/j.engappai.2022.105102 ↗
17. Wang, Z., Zhang, Z., and Voiculescu, I. (2021, January 19–22). RAR-U-Net: A residual encoder to attention decoder by residual connections framework for spine segmentation under noisy labels. Proceedings of the 2021 IEEE International Conference on Image Processing (ICIP), Anchorage, AK, USA.10.1109/ICIP42928.2021.9506085 ↗
18. Huang, H., Lin, L., Tong, R., Hu, H., Zhang, Q., Iwamoto, Y., Han, X., Chen, Y.W., and Wu, J. (2020, January 4–8). Unet 3+: A full-scale connected unet for medical image segmentation. Proceedings of the ICASSP 2020–2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Barcelona, Spain.10.1109/ICASSP40776.2020.9053405 ↗
19. MultiResUNet: Rethinking the U-Net architecture for multimodal biomedical image segmentation10.1016/j.neunet.2019.08.025 ↗
20. Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., and Gelly, S. (2020). An image is worth 16 × 16 words: Transformers for image recognition at scale. arXiv.
21. Attention is all you need
22. Cao, H., Wang, Y., Chen, J., Jiang, D., Zhang, X., Tian, Q., and Wang, M. (2022, January 23–27). Swin-unet: Unet-like pure transformer for medical image segmentation. Proceedings of the Computer Vision–ECCV 2022 Workshops, Tel Aviv, Israel. Part III.10.1007/978-3-031-25066-8_9 ↗
23. Hatamizadeh, A., Tang, Y., Nath, V., Yang, D., Myronenko, A., Landman, B., Roth, H.R., and Xu, D. (2022, January 3–8). Unetr: Transformers for 3d medical image segmentation. Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, Waikoloa, HI, USA.10.1109/WACV51458.2022.00181 ↗
24. Wang, Z., Zheng, J.Q., and Voiculescu, I. (2022, January 27–29). An uncertainty-aware transformer for MRI cardiac semantic segmentation via mean teachers. Proceedings of the Medical Image Understanding and Analysis: 26th Annual Conference, MIUA 2022, Cambridge, UK.10.1007/978-3-031-12053-4_37 ↗
25. Wang, J., Zhang, H., and Yi, Z. (2023). CCTrans: Improving Medical Image Segmentation with Contoured Convolutional Transformer Network. Mathematics, 11.10.3390/math11092082 ↗
26. Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., and Guo, B. (2021, January 20–25). Swin transformer: Hierarchical vision transformer using shifted windows. Proceedings of the IEEE/CVF International Conference on Computer Vision, Nashville, TN, USA.10.1109/ICCV48922.2021.00986 ↗
27. Bortsova, G., Dubost, F., Hogeweg, L., Katramados, I., and De Bruijne, M. (2019, January 13–17). Semi-supervised medical image segmentation via learning consistency under transformations. Proceedings of the Medical Image Computing and Computer Assisted Intervention–MICCAI 2019: 22nd International Conference, Shenzhen, China. Proceedings, Part VI 22.10.1007/978-3-030-32226-7_90 ↗
28. Li, S., Zhang, C., and He, X. (2020, January 4–8). Shape-aware semi-supervised 3D semantic segmentation for medical images. Proceedings of the Medical Image Computing and Computer Assisted Intervention–MICCAI 2020: 23rd International Conference, Lima, Peru. Proceedings, Part I 23.10.1007/978-3-030-59710-8_54 ↗
29. Weakly supervised segmentation of COVID19 infection with scribble annotation on CT images10.1016/j.patcog.2021.108341 ↗
30. Lee, H., and Jeong, W.K. (2020, January 4–8). Scribble2label: Scribble-supervised cell segmentation via self-generating pseudo-labels with consistency. Proceedings of the Medical Image Computing and Computer Assisted Intervention–MICCAI 2020: 23rd International Conference, Lima, Peru. Proceedings, Part I 23.10.1007/978-3-030-59710-8_2 ↗
31. Kervadec, H., Dolz, J., Wang, S., Granger, E., and Ayed, I.B. (2020, January 6–8). Bounding boxes for weakly supervised segmentation: Global constraints get close to full supervision. Proceedings of the Medical Imaging with Deep Learning, PMLR, Montreal, QC, Canada.
32. Learning from weak and noisy labels for semantic segmentation10.1109/TPAMI.2016.2552172 ↗
33. A noise-robust framework for automatic segmentation of COVID-19 pneumonia lesions from CT images10.1109/TMI.2020.3000314 ↗
34. Ba, J.L., Kiros, J.R., and Hinton, G.E. (2016). Layer normalization. arXiv.
35. Hendrycks, D., and Gimpel, K. (2016). Gaussian error linear units (gelus). arXiv.
36. Huang, J., Qu, L., Jia, R., and Zhao, B. (November, January 27). O2u-net: A simple noisy label detection approach for deep neural networks. Proceedings of the IEEE International Conference on Computer Vision, Seoul, Republic of Korea.
37. Yao, J., Burns, J.E., Munoz, H., and Summers, R.M. (2012, January 1–5). Detection of vertebral body fractures based on cortical shell unwrapping. Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention, Nice, France.10.1007/978-3-642-33454-2_63 ↗
38. Chaurasia, A., and Culurciello, E. (2017, January 10–13). Linknet: Exploiting encoder representations for efficient semantic segmentation. Proceedings of the 2017 IEEE Visual Communications and Image Processing (VCIP), St. Petersburg, FL, USA.10.1109/VCIP.2017.8305148 ↗
39. Kim, S.W., Kook, H.K., Sun, J.Y., Kang, M.C., and Ko, S.J. (2018, January 8–14). Parallel feature pyramid network for object detection. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.10.1007/978-3-030-01228-1_15 ↗
40. Ioffe, S., and Szegedy, C. (2015, January 6–11). Batch normalization: Accelerating deep network training by reducing internal covariate shift. Proceedings of the International Conference on Machine Learning, Pmlr, Lille, France.
41. Dropout: A simple way to prevent neural networks from overfitting