An Investigation of Transfer Learning Approaches to Overcome Limited Labeled Data in Medical Image Analysis
Source abstract
Abstract
A significant amount of research has investigated automating medical diagnosis using deep learning. However, because medical data are collected through diagnostic tests, deep learning methods used in existing studies have had a disadvantage in that the number of training samples is insufficient and the labeling cost is high. Training approaches considering the common characteristics of medical images are needed. Therefore, in this study, we investigated approaches to overcome the lack of data for representative medical imaging tasks using transfer learning technologies. The tasks were divided into image classification, object detection, and segmentation, commonly needed functions in medical image analyses. We proposed transfer learning approaches suitable for each task that can be applied when there are little medical image data available. These approaches were experimentally validated in the following applications that share similar issues of lacking data: cervical cancer classification (image classification), skin lesion detection and classification (object detection and classification), and pressure ulcer segmentation (segmentation). We also proposed multi-task learning and ensemble learning that can be applied to these applications. Finally, the approaches were compared with state-of-the-art results. In cervical cancer analysis, the performance was improved by 5.4% in sensitivity. Skin lesion classification showed improvement in accuracy of 8.7%, precision of 28.3%, and sensitivity of 39.7%. Finally, pressure ulcer segmentation improved in accuracy by 1.2%, intersection over union by 16.9%, and Dice similarity coefficient by 3.5%.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: AI in cancer detection · Cervical Cancer and HPV Research · Radiomics and Machine Learning in Medical Imaging
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Department of Artificial Intelligence, Dongguk University, Seoul 04620, Republic of Korea
Funding
MSIT(Ministry of Science and ICT), Korea, under the ITRC(Information Technology Research Center) support program · Artificial Intelligence Convergence Innovation Human Resources Development
References (97)
Show source-provided references
1. Object recognition with gradient-based learning
2. He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27–30). Deep Residual Learning for Image Recognition. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.10.1109/CVPR.2016.90 ↗
3. Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., and Rabinovich, A. (2015, January 7–12). Going deeper with convolutions. Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA.10.1109/CVPR.2015.7298594 ↗
4. Huang, G., Liu, Z., Van Der Maaten, L., and Weinberger, K.Q. (2017, January 21–26). Densely Connected Convolutional Networks. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.10.1109/CVPR.2017.243 ↗
5. Liu, Z., Mao, H., Wu, C.Y., Feichtenhofer, C., Darrell, T., and Xie, S. (2022, January 18–24). A ConvNet for the 2020s. Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA.10.1109/CVPR52688.2022.01167 ↗
6. Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., and Fei-Fei, L. (2009, January 20–25). ImageNet: A large-scale hierarchical image database. Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition, Miami, FL, USA.10.1109/CVPR.2009.5206848 ↗
7. Attentive Transfer Learning via Self-Supervised Learning for Cervical Dysplasia Diagnosis
8. Hwang, S., and Kim, H. (2016). Self-transfer learning for fully weakly supervised object localization. arXiv.10.1007/978-3-319-46723-8_28 ↗
9. A Dual Attention Network for Skin Lesion Classification
10. Deep learning techniques for medical image segmentation: Achievements and challenges10.1007/s10278-019-00227-x ↗
11. Multi-task learning for segmentation and classification of tumors in 3D automated breast ultrasound images10.1016/j.media.2020.101918 ↗
12. Multi-feature based benchmark for cervical dysplasia classification evaluation10.1016/j.patcog.2016.09.027 ↗
13. Cervix cancer classification using colposcopy images by deep learning method
14. An Observational Study of Deep Learning and Automated Evaluation of Cervical Images for Cancer Screening10.1093/jnci/djy225 ↗
15. Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks10.1109/TPAMI.2016.2577031 ↗
16. A fully automated deep learning pipeline for cervical cancer classification10.1016/j.eswa.2019.112951 ↗
17. A Spatial Regulated Patch-Wise Approach for Cervical Dysplasia Diagnosis
18. An Application of Transfer Learning and Ensemble Learning Techniques for Cervical Histopathology Image Classification10.1109/ACCESS.2020.2999816 ↗
19. Cervix Image Classification for Prognosis of Cervical Cancer using Deep Neural Network with Transfer Learning10.4108/eai.12-4-2021.169183 ↗
20. Improving cervical cancer classification with imbalanced datasets combining taming transformers with t2t-vit10.1007/s11042-022-12670-0 ↗
21. Esser, P., Rombach, R., and Ommer, B. (2020, January 13–19). Taming Transformers for High-Resolution Image Synthesis. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.10.1109/CVPR46437.2021.01268 ↗
22. Yuan, L., Chen, Y., Wang, T., Yu, W., Shi, Y., Jiang, Z., Tay, F.H., Feng, J., and Yan, S. (2021, January 11–17). Tokens-to-Token ViT: Training Vision Transformers From Scratch on ImageNet. Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision (ICCV), Montreal, BC, Canada.10.1109/ICCV48922.2021.00060 ↗
23. Ghantasala, G.S.P., Hung, B.T., and Chakrabarti, P. (2023, January 27–29). An Approach For Cervical and Breast Cancer Classification Using Deep Learning: A Comprehensive Survey. Proceedings of the 2023 International Conference on Computer Communication and Informatics (ICCCI), Budapest, Hungary.10.1109/ICCCI56745.2023.10128454 ↗
24. Kalbhor, M.M., and Shinde, S.V. (2023, January 5–7). Cervical cancer diagnosis using convolution neural network: Feature learning and transfer learning approaches. Proceedings of the Soft Comput, Chongqing, China.10.1007/s00500-023-08969-1 ↗
25. Deo, B.S., Pal, M., Panigarhi, P.K., and Pradhan, A. (2023). CerviFormer: A Pap-smear based cervical cancer classification method using cross attention and latent transformer. arXiv.
26. Zhou, Z., Shin, J., Zhang, L., Gurudu, S., Gotway, M., and Liang, J. (2017, January 21–26). Fine-tuning convolutional neural networks for biomedical image analysis: Actively and incrementally. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.10.1109/CVPR.2017.506 ↗
27. Convolutional neural networks for medical image analysis: Full training or fine tuning?10.1109/TMI.2016.2535302 ↗
28. Identifying medical diagnoses and treatable diseases by image-based deep learning10.1016/j.cell.2018.02.010 ↗
29. Multi-task transfer learning deep convolutional neural network: Application to computer-aided diagnosis of breast cancer on mammograms10.1088/1361-6560/aa93d4 ↗
30. Predicting Breast Cancer Response to Neoadjuvant Chemotherapy Using Ensemble Deep Transfer Learning Based on CT Images
31. de Bruijne, M., Cattin, P.C., Cotin, S., Padoy, N., Speidel, S., Zheng, Y., and Essert, C. (2021). Medical Image Computing and Computer Assisted Intervention—MICCAI 2021, Springer.10.1007/978-3-030-87237-3 ↗
32. Gheflati, B., and Rivaz, H. (2021). Vision transformer for classification of breast ultrasound images. arXiv.10.1109/EMBC48229.2022.9871809 ↗
33. Yang, H., Chen, J., and Xu, M. (2021, January 11–14). Fundus disease image classification based on improved transformer. Proceedings of the International Conference on Neuromorphic Computing (ICNC), Wuhan, China.10.1109/ICNC52316.2021.9608181 ↗
34. Ikromjanov, K., Bhattacharjee, S., Hwang, Y.B., Sumon, R.I., Kim, H.C., and Choi, H.K. (2022, January 21–24). Whole Slide Image Analysis and Detection of Prostate Cancer using Vision Transformers. Proceedings of the 2022 International Conference on Artificial Intelligence in Information and Communication (ICAIIC), Korea, Republic of Korea.10.1109/ICAIIC54071.2022.9722635 ↗
35. de Bruijne, M., Cattin, P.C., Cotin, S., Padoy, N., Speidel, S., Zheng, Y., and Essert, C. (2021). Medical Image Computing and Computer Assisted Intervention—MICCAI 2021, Springer.10.1007/978-3-030-87240-3 ↗
36. Scale-aware transformers for diagnosing melanocytic lesions10.1109/ACCESS.2021.3132958 ↗
37. Detecting COVID-19 in chest images based on deep transfer learning and machine learning algorithms10.1186/s43055-021-00524-y ↗
38. Costa, G., Paiva, A., Júnior, G.B., and Ferreira, M. (2021, January 15–18). COVID-19 automatic diagnosis with CT images using the novel Transformer architecture. Proceedings of the Anais do XXI Simpósio Brasileiro de Computação Aplicada à Saúde, SBC, Online.10.5753/sbcas.2021.16073 ↗
39. Liang, S. (2021). A hybrid deep learning framework for covid-19 detection via 3d chest ct images. arXiv.10.1109/ICCVW54120.2021.00062 ↗
40. Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., and Gelly, S. (2021, January 3–7). An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale. Proceedings of the International Conference on Learning Representations(ICLR), Virtual.
41. Sufian, M.M., Moung, E.G., Hijazi, M.H.A., Yahya, F., Dargham, J.A., Farzamnia, A., Sia, F., and Mohd Naim, N.F. (2023). COVID-19 Classification through Deep Learning Models with Three-Channel Grayscale CT Images. Big Data Cogn. Comput., 7.10.3390/bdcc7010036 ↗
42. Constantinou, M., Exarchos, T., Vrahatis, A.G., and Vlamos, P. (2023). COVID-19 Classification on Chest X-ray Images Using Deep Learning Methods. Int. J. Environ. Res. Public Health, 20.10.3390/ijerph20032035 ↗
43. Skin lesion classification using cnn with novel regularizer10.1109/ACCESS.2019.2906241 ↗
44. Deep Learning Solutions for Skin Cancer Detection and Diagnosis10.1007/978-3-030-40850-3_8 ↗
45. Sirotkin, K., Escudero-Vinolo, M., Carballeira, P., and SanMiguelv, J. (2021). Improved skin lesion recognition by a Self-Supervised Curricular Deep Learning approach. arXiv.
46. Mahbod, A., Schaefer, G., Wang, C., Ecker, R., and Ellinge, I. (2019, January 12–17). Skin lesion classification using hybrid deep neural networks. Proceedings of the ICASSP 2019—2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Brighton, UK.10.1109/ICASSP.2019.8683352 ↗
47. Masood, A., Al-Jumaily, A., and Anam, K. (2015, January 22–24). Self-supervised learning model for skin cancer diagnosis. Proceedings of the International IEEE/EMBS Conference on Neural Engineering (NER), Montpellier, France.10.1109/NER.2015.7146798 ↗
48. Pal, A., Ray, S., and Garain, U. (2018). Skin disease identification from dermoscopy images using deep convolutional neural network. arXiv.
49. Carcagnì, P., Leo, M., Cuna, A., Mazzeo, P.L., Spagnolo, P., Celeste, G., and Distante, C. (2019). Image Analysis and Processing—ICIAP 2019, Springer.
50. Liao, H., and Luo, J. (2017, January 4–9). A deep multi-task learning approach to skin lesion classification. Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence Workshops, San Francisco, CA, USA.
51. A novel framework of multiclass skin lesion recognition from dermoscopic images using deep learning and explainable AI10.3389/fonc.2023.1151257 ↗
52. Skin-Net: A novel deep residual network for skin lesions classification using multilevel feature extraction and cross-channel correlation with detection of outlier10.1186/s40537-023-00769-6 ↗
53. Girshick, R. (2015, January 7–13). Fast R-CNN. Proceedings of the 2015 IEEE International Conference on Computer Vision (ICCV), Washington, DC, USA.10.1109/ICCV.2015.169 ↗
54. He, K., Gkioxari, G., Dollár, P., and Girshick, R. (2017, January 22–29). Mask R-CNN. Proceedings of the 2017 IEEE International Conference on Computer Vision (ICCV), Venice, Italy.10.1109/ICCV.2017.322 ↗
55. Redmon, J., and Farhadi, A. (2018). YOLOv3: An Incremental Improvement. arXiv.
56. Bochkovskiy, A., Wang, C., and Liao, H. (2020). YOLOv4: Optimal Speed and Accuracy of Object Detection. arXiv.
57. Zahia, S., Garcia-Zapirain, B., and Elmaghraby, A. (2020). Integrating 3D Model Representation for an Accurate Non-Invasive Assessment of Pressure Injuries with Deep Learning. Sensors, 20.10.3390/s20102933 ↗
58. COVID-19 detection in chest X-ray images using a deep learning approach
59. Artificial intelligence applied to chest X-ray images for the automatic detection of COVID-19: A thoughtful evaluation approach10.1109/ACCESS.2020.3044858 ↗
60. Explainable deep learning for pulmonary disease and coronavirus COVID-19 detection from X-rays10.1016/j.cmpb.2020.105608 ↗
61. Navab, N., Hornegger, J., Wells, W.M., and Frangi, A.F. (2015). Medical Image Computing and Computer-Assisted Intervention—MICCAI 2015, Springer.10.1007/978-3-319-24571-3 ↗
62. Wang, C., Yan, X., Smith, M., Kochhar, K., Rubin, M., Warren, S.M., Wrobel, J., and Lee, H. (2015, January 25–29). A unified framework for automatic wound segmentation and analysis with deep convolutional neural networks. Proceedings of the 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Milan, Italy.
63. Pholberdee, N., Pathompatai, C., and Taeprasartsit, P. (2018, January 11–13). Study of chronic wound image segmentation: Impact of tissue type and color data augmentation. Proceedings of the 15th International Joint Conference on Computer Science and Software Engineering (JCSSE), Nakhonpathom, Thailand.10.1109/JCSSE.2018.8457392 ↗
64. Goyal, M., Yap, M.H., Reeves, N.D., Rajbhandari, S., and Spragg, J. (2017, January 5–8). Fully convolutional networks for diabetic foot ulcer segmentation. Proceedings of the 2017 IEEE International Conference on Systems, Man, and Cybernetics (SMC), Banff, AB, Canada.10.1109/SMC.2017.8122675 ↗
65. Classification of pressure ulcer tissues with 3D convolutional neural network10.1007/s11517-018-1835-y ↗
66. Chronic wound healing assessment system based on different features modalities and non-negative matrix factorization (NMF) feature reduction10.1109/ACCESS.2019.2923962 ↗
67. Convolutional neural networks for wound detection: The role of artificial intelligence in wound care10.12968/jowc.2019.28.Sup10.S13 ↗
68. Oota, S.R., Rowtula, V., Mohammed, S., Liu, M., and Gupta, M. (2023, January 2–7). WSNet: Towards an Effective Method for Wound Image Segmentation. Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), Waikoloa, HI, USA.10.1109/WACV56688.2023.00325 ↗
69. Interactive Skin Wound Segmentation Based on Feature Augment Networks10.1109/JBHI.2023.3270711 ↗
70. Simultaneous Segmentation and Classification of Pressure Injury Image Data Using Mask-R-CNN10.1155/2023/3858997 ↗
71. Aldughayfiq, B., Ashfaq, F., Jhanjhi, N.Z., and Humayun, M. (2023). YOLO-Based Deep Learning Model for Pressure Ulcer Detection and Classification. Healthcare, 11.10.3390/healthcare11091222 ↗
72. Ourselin, S., Joskowicz, L., Sabuncu, M.R., Unal, G., and Wells, W. (2016). Medical Image Computing and Computer-Assisted Intervention—MICCAI 2016, Springer.10.1007/978-3-319-46726-9 ↗
73. Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-López, C., and Fichtinger, G. (2018). Medical Image Computing and Computer Assisted Intervention–MICCAI 2018, Springer.10.1007/978-3-030-00928-1 ↗
74. Javanmardi, M., and Tasdizen, T. (2018, January 4–7). Domain adaptation for biomedical image segmentation using adversarial training. Proceedings of the IEEE 15th International Symposium on Biomedical Imaging (ISBI), Washington, DC, USA.10.1109/ISBI.2018.8363637 ↗
75. Gao, Y., Zhou, M., Liu, D., and Metaxas, D. (2022). A multi-scale transformer for medical image segmentation: Architectures, model efficiency, and benchmarks. arXiv.
76. You, C., Zhao, R., Liu, F., Chinchali, S.P., Topcu, U., Staib, L.H., and Duncan, J.S. (2022). Class-Aware Generative Adversarial Transformers for Medical Image Segmentation. arXiv.
77. H-DenseUNet: Hybrid Densely Connected UNet for Liver and Tumor Segmentation From CT Volumes10.1109/TMI.2018.2845918 ↗
78. Huang, H., Lin, L., Tong, R., Hu, H., Qiaowei, Z., Iwamoto, Y., Han, X.H., Chen, Y.W., and Wu, J. (2020, January 4–8). UNet 3+: A Full-Scale Connected UNet for Medical Image Segmentation. Proceedings of the International Conference on Acoustics, Speech and Signal Processing (ICASSP), Barcelona, Spain.10.1109/ICASSP40776.2020.9053405 ↗
79. nnU-Net: A self-configuring method for deep learning-based biomedical image segmentation10.1038/s41592-020-01008-z ↗
80. Crimi, A., and Bakas, S. (2022). Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries, Springer.10.1007/978-3-031-08999-2 ↗
81. Andrearczyk, V., Oreiller, V., Hatt, M., and Depeursinge, A. (2022). Head and Neck Tumor Segmentation and Outcome Prediction, Springer.10.1007/978-3-030-98253-9 ↗
82. Özcan, F., Uçan, O.N., Karaçam, S., and Tunçman, D. (2023). Fully Automatic Liver and Tumor Segmentation from CT Image Using an AIM-Unet. Bioengineering, 10.10.3390/bioengineering10020215 ↗
83. 2.5D cascaded context-based network for liver and tumor segmentation from CT images10.3934/era.2023221 ↗
84. Aggarwal, M., Tiwari, A.K., Sarathi, M.P., and Bijalwan, A. (2023). An early detection and segmentation of Brain Tumor using Deep Neural Network. BMC Med. Inform. Decis. Mak., 23.10.1186/s12911-023-02174-8 ↗
85. Brain Tumor Segmentation from 3D MRI Scans Using U-Net10.1007/s42979-023-01854-6 ↗
86. National Library of Medicine (U.S.)—The Cleveland Clinic (n.d.) (2023, January 05). An Innovative Treatment for Cervical Precancer (UH3). 2017. ClinicalTrials.gov Identifier: NCT03084081, Available online: https://classic.clinicaltrials.gov/ct2/show/study/NCT03084081.
87. The HAM10000 dataset, a large collection of multi-source dermatoscopic images of common pigmented skin lesions10.1038/sdata.2018.161 ↗
88. Yang, X., Zeng, Z., Yeo, S., Tan, C., Tey, H.L., and Su, Y. (2017). A novel multi-task deep learning model for skin lesion segmentation and classification. arXiv.
89. Chae, J., Zhang, Y., Zimmermann, R., Kim, D., and Kim, J. (2022). Intelligent Systems and Applications, Springer.
90. Kendall, A., Gal, Y., and Cipolla, R. (2018, January 18–23). Multi-task learning using uncertainty to weigh losses for scene geometry and semantics. Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR), Salt Lake City, UT, USA.
91. Design and methods of a population-based natural history study of cervical neoplasia in a rural province of Costa Rica: The Guanacaste Project10.1590/S1020-49891997000500005 ↗
92. (2023, April 05). Intel & MobileODT Cervical Cancer Screening Competition. Available online: https://www.kaggle.com/c/intel-mobileodt-cervical-cancer-screening.
93. Codella, N.C.F., Gutman, D., Celebi, M.E., Helba, B., Marchetti, M.A., Dusza, S.W., Kalloo, A., Liopyris, K., Mishra, N., and Kittler, H. (2018, January 4–7). Skin lesion analysis toward melanoma detection: A challenge at the 2017 International symposium on biomedical imaging (ISBI), hosted by the international skin imaging collaboration (ISIC). Proceedings of the 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018), Washington, DC, USA.10.1109/ISBI.2018.8363547 ↗
94. Crawshaw, M. (2020). Multi-task learning with deep neural networks: A Survey. arXiv.
95. Chae, J., Hong, K., and Kim, J. (2021). A pressure ulcer care system for remote medical assistance: Residual U-Net with an attention model based for wound area segmentation. arXiv.
96. Homas, S. (2023, April 05). Medetec Wound Database. Available online: http://www.medetec.co.uk/files/medetec-image-databases.html.
97. Fully automatic wound segmentation with deep convolutional neural networks10.1038/s41598-020-78799-w ↗