Unsupervised Domain Adaptation for Automatic Polyp Segmentation Using Synthetic Data
Source abstract
Abstract
Colorectal cancer is a significant health concern that can often be prevented through early detection of precancerous polyps during routine screenings. Although artificial intelligence (AI) methods have shown potential in reducing polyp miss rates, clinical adoption remains limited due to concerns over patient privacy, limited access to annotated data, and the high cost of expert labeling. To address these challenges, we propose an unsupervised domain adaptation (UDA) approach that leverages a fully synthetic colonoscopy dataset, SynthColon, and adapts it to real-world, unlabeled data. Our method builds on the DAFormer framework and integrates a Transformer-based hierarchical encoder, a context-aware feature fusion decoder, and a self-training strategy. We evaluate our approach on the Kvasir-SEG and CVC-ClinicDB datasets. Results show that our method achieves improved segmentation performance of 69% mIoU compared to the baseline approach from the original SynthColon study and remains competitive with models trained on enhanced versions of the dataset.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Colorectal Cancer Screening and Detection · COVID-19 diagnosis using AI · AI in cancer detection
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Biomedical Engineering Laboratory, School of Electrical and Computer Engineering, National Technical University of Athens, Zografou Polytechnic Campus, 15772 Athens, Greece
Department of Biomedical Engineering, University of West Attica, 12243 Athens, Greece
References (75)
Show source-provided references
1. American Cancer Society (2025, April 21). Key Statistics for Colorectal Cancer. 2024. Available online: https://www.cancer.org/cancer/types/colon-rectal-cancer/about/key-statistics.html.
2. Analysis of Survival Among Adults with Early-Onset Colorectal Cancer in the National Cancer Database10.1001/jamanetworkopen.2021.12539 ↗
3. Sikora, N., Manschke, R.L., Tang, A.M., Dunstan, P., Harris, D.A., and Yang, S. (2025). ColonScopeX: Leveraging Explainable Expert Systems with Multimodal Data for Improved Early Diagnosis of Colorectal Cancer. arXiv.
4. Colorectal cancer screening: Recommendations for physicians and patients from the US Multi-Society Task Force on colorectal cancer10.1053/j.gastro.2017.05.013 ↗
5. Diagnostic miss rate for colorectal cancer: An audit
6. Accuracy of diagnosing invasive colorectal cancer using computer-aided endocytoscopy10.1055/s-0043-105486 ↗
7. A guide to deep learning in healthcare10.1038/s41591-018-0316-z ↗
8. A survey on bias and fairness in machine learning10.1145/3457607 ↗
9. Overcoming Data Scarcity in Biomedical Imaging with a Foundational Multi-Task Model10.1038/s43588-024-00662-z ↗
10. The future of digital health with federated learning10.1038/s41746-020-00323-1 ↗
11. European Union (2016). Regulation
12. (EU) 2016/679 of the European Parliament and of the Council of 27 April 2016 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data (General Data Protection Regulation). Off. J. Eur. Union, L119, 1-88. Available online: https://eur-lex.europa.eu/eli/reg/2016/679/oj.
13. Yao, L., Prosky, J., Covington, B., and Lyman, K. (2019, January 8–10). A Strong Baseline for Domain Adaptation and Generalization in Medical Imaging. Extended Abstract Track. Proceedings of the Medical Imaging with Deep Learning (MIDL 2019), London, UK.
14. Hu, S., Liao, Z., and Xia, Y. (2023, January 8–12). Devil is in Channels: Contrastive Single Domain Generalization for Medical Image Segmentation. Proceedings of the Medical Image Computing and Computer Assisted Intervention—MICCAI 2023: 26th International Conference, Vancouver, BC, Canada.10.1007/978-3-031-43901-8_2 ↗
15. Ren, G., Lazarou, M., Yuan, J., and Stathaki, T. (2023, January 17–24). Towards Automated Polyp Segmentation Using Weakly- and Semi-Supervised Learning and Deformable Transformers. Proceedings of the 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Vancouver, BC, Canada.10.1109/CVPRW59228.2023.00458 ↗
16. Basaran, B.D., Zhang, W., Qiao, M., Kainz, B., Matthews, P.M., and Bai, W. (2023, January 12). LesionMix: A Lesion-Level Data Augmentation Method for Medical Image Segmentation. Proceedings of the Data Augmentation, Labelling, and Imperfections: Third MICCAI Workshop, DALI 2023, Held in Conjunction with MICCAI 2023, Vancouver, BC, Canada.10.1007/978-3-031-58171-7_8 ↗
17. Moreu, E., McGuinness, K., and O’Connor, N.E. (2021, January 7–8). Synthetic data for unsupervised polyp segmentation. Proceedings of the 29th Irish Conference on Artificial Intelligence and Cognitive Science (AICS 2021), Dublin, Ireland.
18. Zhu, J.Y., Park, T., Isola, P., and Efros, A.A. (2017, January 22–29). Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks. Proceedings of the IEEE International Conference on Computer Vision (ICCV 2017), Venice, Italy.10.1109/ICCV.2017.244 ↗
19. Jha, D., Smedsrud, P.H., Riegler, M.A., Halvorsen, P., de Lange, T., Johansen, D., and Johansen, H.D. (2020, January 5–8). Kvasir-SEG: A Segmented Polyp Dataset. Proceedings of the 26th International Conference on MultiMedia Modeling (MMM 2020), Daejeon, Korea.10.1007/978-3-030-37734-2_37 ↗
20. Hoyer, L., Dai, D., and Gool, L.V. (2022, January 18–24). DAFormer: Improving Network Architectures and Training Strategies for Domain-Adaptive Semantic Segmentation. Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA.10.1109/CVPR52688.2022.00969 ↗
21. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, L., and Polosukhin, I. (2017, January 4–9). Attention Is All You Need. Proceedings of the 31st International Conference on Neural Information Processing Systems, Long Beach, CA, USA.
22. Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., and Gelly, S. (2021, January 23–26). An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New York City, NY, USA.
23. Wang, W., Xie, E., Li, X., Fan, D., Song, K., Liang, D., Lu, T., Luo, P., and Shao, L. (2021, January 10–17). Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions. Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision (ICCV), Montreal, QC, Canada.10.1109/ICCV48922.2021.00061 ↗
24. Xie, E., Wang, W., Yu, Z., Anandkumar, A., Alvarez, J.M., and Luo, P. (2021, January 6–14). SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers. Proceedings of the Neural Information Processing Systems (NeurIPS), Virtual Conference.
25. PVT v2: Improved baselines with pyramid vision transformer10.1007/s41095-022-0274-8 ↗
26. Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., and Guo, B. (2021, January 10–17). Swin Transformer: Hierarchical Vision Transformer using Shifted Windows. Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision (ICCV), Montreal, QC, Canada.10.1109/ICCV48922.2021.00986 ↗
27. WM-DOVA maps for accurate polyp highlighting in colonoscopy: Validation vs. saliency maps from physicians10.1016/j.compmedimag.2015.02.007 ↗
28. Ronneberger, O., Fischer, P., and Brox, T. (2015, January 5–9). U-Net: Convolutional Networks for Biomedical Image Segmentation. Proceedings of the 18th International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI 2015), Munich, Germany.10.1007/978-3-319-24574-4_28 ↗
29. Zhou, Z., Siddiquee, M.M.R., Tajbakhsh, N., and Liang, J. (2018, January 20). UNet++: A Nested U-Net Architecture for Medical Image Segmentation. Proceedings of the 4th International Workshop, DLMIA 2018, and 8th International Workshop, ML-CDS 2018, Held in Conjunction with MICCAI 2018, Granada, Spain.
30. Jha, D., Smedsrud, P.H., Riegler, M.A., Johansen, D., de Lange, T., Halvorsen, P., and Johansen, H.D. (2019, January 9–11). ResUNet++: An Advanced Architecture for Medical Image Segmentation. Proceedings of the 2019 IEEE International Symposium on Multimedia (ISM), San Diego, CA, USA.10.1109/ISM46123.2019.00049 ↗
31. Jha, D., Riegler, M.A., Johansen, D., Halvorsen, P., and Johansen, H.D. (2020, January 28–30). DoubleU-Net: A Deep Convolutional Neural Network for Medical Image Segmentation. Proceedings of the 2020 IEEE 33rd International Symposium on Computer-Based Medical Systems (CBMS), Rochester, MN, USA.10.1109/CBMS49503.2020.00111 ↗
32. He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27–30). Deep Residual Learning for Image Recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.10.1109/CVPR.2016.90 ↗
33. Kim, T., Lee, H., and Kim, D. (2021, January 17). UACANet: Uncertainty Augmented Context Attention for Polyp Segmentation. Proceedings of the 29th ACM International Conference on Multimedia, ACM, Virtual Event.10.1145/3474085.3475375 ↗
34. Fan, D.P., Ji, G.P., Zhou, T., Chen, G., Fu, H., Shen, J., and Shao, L. (2020, January 4–8). PraNet: Parallel Reverse Attention Network for Polyp Segmentation. Proceedings of the 23rd International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI 2020), Lima, Peru.10.1007/978-3-030-59725-2_26 ↗
35. Zhang, Y., Liu, H., and Hu, Q. (October, January 27). TransFuse: Fusing Transformers and CNNs for Medical Image Segmentation. Proceedings of the 24th International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI 2021), Strasbourg, France.10.1007/978-3-030-87193-2_2 ↗
36. Polyp-PVT: Polyp Segmentation with Pyramid Vision Transformers10.26599/AIR.2023.9150015 ↗
37. Rahman, M.M., and Marculescu, R. (2023, January 2–7). Medical Image Segmentation via Cascaded Attention Decoding. Proceedings of the 2023 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), Waikoloa, HI, USA.10.1109/WACV56688.2023.00616 ↗
38. Shi, W., Xu, J., and Gao, P. (2022, January 26–28). SSformer: A Lightweight Transformer for Semantic Segmentation. Proceedings of the 2022 IEEE 24th International Workshop on Multimedia Signal Processing (MMSP), Shanghai, China.10.1109/MMSP55362.2022.9949177 ↗
39. Fitzgerald, K., and Matuszewski, B. (2023). FCB-SwinV2 Transformer for Polyp Segmentation. arXiv.
40. Choudhuri, A., Gao, Z., Zheng, M., Planche, B., Chen, T., and Wu, Z. (2025). PolypSegTrack: Unified Foundation Model for Colonoscopy Video Analysis. arXiv.
41. Kirillov, A., Mintun, E., Ravi, N., Mao, H., Rolland, C., Gustafson, L., Xiao, T., Whitehead, S., Berg, A.C., and Lo, W.Y. (2023, January 2–6). Segment Anything. Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), Paris, France.10.1109/ICCV51070.2023.00371 ↗
42. Li, H., Zhang, D., Yao, J., Han, L., Li, Z., and Han, J. (2024, January 6–10). ASPS: Augmented Segment Anything Model for Polyp Segmentation. Proceedings of the Medical Image Computing and Computer Assisted Intervention—MICCAI, Marrakesh, Morocco.10.1007/978-3-031-72114-4_12 ↗
43. Li, Y., Hu, M., and Yang, X. (2024, January 3). Polyp-SAM: Transfer SAM for Polyp Segmentation. Proceedings of the Medical Imaging 2024: Computer-Aided Diagnosis, San Diego, CA, USA.10.1117/12.3006809 ↗
44. Rahman, M.M., Munir, M., Jha, D., Bagci, U., and Marculescu, R. (2024, January 17–21). PP-SAM: Perturbed Prompts for Robust Adaptation of Segment Anything Model for Polyp Segmentation. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, Seattle, WA, USA.10.1109/CVPRW63382.2024.00504 ↗
45. Mao, X., Xing, X., Meng, F., Liu, J., Bai, F., Nie, Q., and Meng, M. (2025). One Polyp Identifies All: One-Shot Polyp Segmentation with SAM via Cascaded Priors and Iterative Prompt Evolution. arXiv.
46. Mansoori, M., Shahabodini, S., Abouei, J., Plataniotis, K.N., and Mohammadi, A. (2024). Polyp SAM 2: Advancing Zero shot Polyp Segmentation in Colorectal Cancer Detection. arXiv.
47. WeakPolyp-SAM: Segment Anything Model-driven weakly-supervised polyp segmentation10.1016/j.knosys.2025.113701 ↗
48. Goodfellow, I.J., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., and Bengio, Y. (2014, January 8–13). Generative Adversarial Networks. Proceedings of the 27th International Conference on Neural Information Processing Systems (NeurIPS 2014), Montreal, QC, Canada.
49. Gong, R., Li, W., Chen, Y., and Gool, L.V. (2019, January 15–20). DLOW: Domain Flow for Adaptation and Generalization. Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA.10.1109/CVPR.2019.00258 ↗
50. Tzeng, E., Hoffman, J., Saenko, K., and Darrell, T. (2017, January 21–26). Adversarial Discriminative Domain Adaptation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.10.1109/CVPR.2017.316 ↗
51. Tsai, Y.H., Hung, W.C., Schulter, S., Sohn, K., Yang, M.H., and Chandraker, M. (2018, January 18–23). Learning to Adapt Structured Output Space for Semantic Segmentation. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2018), Salt Lake City, UT, USA.10.1109/CVPR.2018.00780 ↗
52. Tsai, Y.H., Sohn, K., Schulter, S., and Chandraker, M. (November, January 27). Domain Adaptation for Structured Output via Discriminative Patch Representations. Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision (ICCV), Seoul, Republic of Korea.10.1109/ICCV.2019.00154 ↗
53. Lee, D.H. (2013, January 16–21). Pseudo-Label: The Simple and Efficient Semi-Supervised Learning Method for Deep Neural Networks. Proceedings of the ICML 2013 Workshop: Challenges in Representation Learning (WREPL), Atlanta, GA, USA.
54. Sohn, K., Berthelot, D., Li, C.L., Zhang, Z., Carlini, N., Cubuk, E.D., Kurakin, A., Zhang, H., and Raffel, C. (2020, January 6–12). FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence. Proceedings of the 34th International Conference on Neural Information Processing Systems (NeurIPS 2020), Virtual Event.
55. Zou, Y., Yu, Z., Kumar, B.V.K.V., and Wang, J. (2018, January 8–14). Domain Adaptation for Semantic Segmentation via Class-Balanced Self-Training. Proceedings of the 15th European Conference on Computer Vision (ECCV 2018), Munich, Germany.10.1007/978-3-030-01219-9_18 ↗
56. Zou, Y., Yu, Z., Liu, X., Kumar, B.V.K.V., and Wang, J. (November, January 27). Confidence Regularized Self-Training. Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV 2019), Seoul, Republic of Korea.10.1109/ICCV.2019.00608 ↗
57. Wang, Y., Wang, H., Shen, Y., Fei, J., Li, W., Jin, G., Wu, L., Zhao, R., and Le, X. (2022, January 18–24). Semi-Supervised Semantic Segmentation Using Unreliable Pseudo-Labels. Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA.10.1109/CVPR52688.2022.00421 ↗
58. Sakaridis, C., Dai, D., Hecker, S., and Gool, L.V. (2018, January 8–14). Model Adaptation with Synthetic and Real Data for Semantic Dense Foggy Scene Understanding. Proceedings of the 15th European Conference on Computer Vision (ECCV 2018), Munich, Germany.10.1007/978-3-030-01261-8_42 ↗
59. Yang, Y., and Soatto, S. (2020, January 13–19). FDA: Fourier Domain Adaptation for Semantic Segmentation. Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.10.1109/CVPR42600.2020.00414 ↗
60. Hinton, G., Vinyals, O., and Dean, J. (2014, January 8–13). Distilling the Knowledge in a Neural Network. Proceedings of the NIPS 2014 Deep Learning Workshop, Montreal, QC, Canada.
61. Tarvainen, A., and Valpola, H. (2017, January 4–9). Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results. Proceedings of the 31st International Conference on Neural Information Processing Systems (NeurIPS 2017), Long Beach, CA, USA.
62. Olsson, V., Tranheden, W., Pinto, J., and Svensson, L. (2021, January 5–9). ClassMix: Segmentation-Based Data Augmentation for Semi-Supervised Learning. Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV 2021), Waikoloa, HI, USA.10.1109/WACV48630.2021.00141 ↗
63. Tranheden, W., Olsson, V., Pinto, J., and Svensson, L. (2021, January 5–9). DACS: Domain Adaptation via Cross-domain Mixed Sampling. Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV 2021), Waikoloa, HI, USA.10.1109/WACV48630.2021.00142 ↗
64. Uncertainty-aware consistency regularization for cross-domain semantic segmentation10.1016/j.cviu.2022.103448 ↗
65. This Intestine Does Not Exist: Multiscale Residual Variational Autoencoder for Realistic Wireless Capsule Endoscopy Image Generation10.1109/ACCESS.2024.3366801 ↗
66. Barua, H.B., Stefanov, K., Wong, K., Dhall, A., and Krishnasamy, G. (2024). GTA-HDR: A Large-Scale Synthetic Dataset for HDR Image Reconstruction. arXiv.10.1109/WACV61041.2025.00765 ↗
67. Ros, G., Sellart, L., Materzynska, J., Vazquez, D., and Lopez, A.M. (2016, January 27–30). The SYNTHIA Dataset: A Large Collection of Synthetic Images for Semantic Segmentation of Urban Scenes. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.10.1109/CVPR.2016.352 ↗
68. Joint one-sided synthetic unpaired image translation and segmentation for colorectal cancer prevention10.1111/exsy.13137 ↗
69. Vezakis, I.A., Georgas, K., Fotiadis, D., and Matsopoulos, G.K. (2024, January 15–19). EffiSegNet: Gastrointestinal Polyp Segmentation through a Pre-Trained EfficientNet-based Network with a Simplified Decoder. Proceedings of the 46th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC 2024), Orlando, FL, USA.10.1109/EMBC53108.2024.10782015 ↗
70. Moreu, E., Arazo, E., McGuinness, K., and O’Connor, N.E. (2023, January 18–23). Self-Supervised and Semi-Supervised Polyp Segmentation using Synthetic Data. Proceedings of the 2023 International Joint Conference on Neural Networks (IJCNN), Gold Coast, Australia.10.1109/IJCNN54540.2023.10191896 ↗
71. Huang, C.H., Wu, H.Y., and Lin, Y.L. (2021, January 6–11). HarDNet-MSEG: A Simple Encoder-Decoder Polyp Segmentation Neural Network that Achieves over 0.9 Mean Dice and 86 FPS. Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP 2021), Toronto, ON, Canada.
72. McDuff, D., Curran, T., and Kadambi, A. (2023). Synthetic Data in Healthcare. arXiv.
73. Chen, J., Chun, D., Patel, M., Chiang, E., James, J., Capobianco, J., Lipson, J., Hong, C., Natarajan, K., and Cole, C.L. (2019). The validity of synthetic clinical data: A validation study of a leading synthetic data generator (Synthea) using clinical quality measures. BMC Med. Inform. Decis. Mak., 19.10.1186/s12911-019-0793-0 ↗
74. Mathew, S., Nadeem, S., Kumari, S., and Kaufman, A. (2020, January 13–19). Augmenting Colonoscopy Using Extended and Directional CycleGAN for Lossy Image Translation. Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.10.1109/CVPR42600.2020.00475 ↗
75. Iacono, P., and Khan, N. (2023). Structure Preserving Cycle-GAN for Unsupervised Medical Image Domain Adaptation. arXiv.10.32920/22734377.v1 ↗