Deep Learning Approaches to Colorectal Cancer Diagnosis: A Review
Source abstract
Abstract
Unprecedented breakthroughs in the development of graphical processing systems have led to great potential for deep learning (DL) algorithms in analyzing visual anatomy from high-resolution medical images. Recently, in digital pathology, the use of DL technologies has drawn a substantial amount of attention for use in the effective diagnosis of various cancer types, especially colorectal cancer (CRC), which is regarded as one of the dominant causes of cancer-related deaths worldwide. This review provides an in-depth perspective on recently published research articles on DL-based CRC diagnosis and prognosis. Overall, we provide a retrospective synopsis of simple image-processing-based and machine learning (ML)-based computer-aided diagnosis (CAD) systems, followed by a comprehensive appraisal of use cases with different types of state-of-the-art DL algorithms for detecting malignancies. We first list multiple standardized and publicly available CRC datasets from two imaging types: colonoscopy and histopathology. Secondly, we categorize the studies based on the different types of CRC detected (tumor tissue, microsatellite instability, and polyps), and we assess the data preprocessing steps and the adopted DL architectures before presenting the optimum diagnostic results. CRC diagnosis with DL algorithms is still in the preclinical phase, and therefore, we point out some open issues and provide some insights into the practicability and development of robust diagnostic systems in future health care and oncology.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: AI in cancer detection · Colorectal Cancer Screening and Detection · Radiomics and Machine Learning in Medical Imaging
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Department of Information and Communication Engineering, Changwon National University, Changwon 51140, Korea
Funding
National Research Foundation of Korea · Industry-Academic Cooperation Based Platform R&D funded Korea Ministry of SMEs and Startups
References (93)
Show source-provided references
1. Global cancer statistics 2018: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries10.3322/caac.21492 ↗
2. Colorectal Cancer10.1136/bmj.39321.527384.BE ↗
3. Global Patterns and Trends in Colorectal Cancer Incidence and Mortality10.1136/gutjnl-2015-310912 ↗
4. Geboes, K., and Mourin, A.J. (2021, July 08). Endoscopy and Histopathology. Available online: https://www.intechopen.com/chapters/44215.
5. Interobserver Variability in Breast Carcinoma Grading Results in Prognostic Stage Differences10.1016/j.humpath.2019.09.006 ↗
6. Computer-based Classification of Small Colorectal Polyps by Using Narrow-band Imaging with Optical Magnification10.1016/j.gie.2011.08.001 ↗
7. Novel Computer-aided Diagnostic System for Colorectal Lesions by Using Endocytoscopy (with videos)10.1016/j.gie.2014.09.008 ↗
8. Effectiveness of computer-aided diagnosis of colorectal lesions using novel software for magnifying narrow-band imaging: A pilot study10.1055/s-0043-105490 ↗
9. Computer-aided colorectal tumor classification in NBI endoscopy using local features10.1016/j.media.2012.08.003 ↗
10. Computer-aided diagnosis of colorectal polyp histology by using a real-time image recognition system and narrow-band imaging magnifying colonoscopy10.1016/j.gie.2015.08.004 ↗
11. Computer-aided detection of early Barrett’s neoplasia using volumetric laser endomicroscopy10.1016/j.gie.2017.03.011 ↗
12. Computer-aided diagnosis of colorectal polyps using linked color imaging colonoscopy to predict histology10.1038/s41598-019-39416-7 ↗
13. Identification of tumor epithelium and stroma in tissue microarrays using texture analysis10.1186/1746-1596-7-22 ↗
14. Deep convolutional neural networks [Lecture Notes]10.1109/MSP.2018.2842646 ↗
15. Deep D2C-Net: Deep learning-based display-to-camera communications10.1364/OE.422591 ↗
16. Deep learning-based point scanning super resolution imaging10.1038/s41592-021-01080-z ↗
17. Medical imaging data and datasets in the era of machine learning10.1007/s10278-017-9976-3 ↗
18. Histopathological image analysis: A review10.1109/RBME.2009.2034865 ↗
19. Pogorelov, K., Randel, K.R., Griwodz, C., Eskeland, S.L., Lange, T., Johansen, D., Spampinato, C., Nguyen, D.T.D., Lux, M., and Schmidt, P.T. (2017, January 20–23). Kvasir: A multi-class image dagaset for computer aided gastrointestinal disease detection. Proceedings of the 8th ACM on Multimedia Systems Conference (MMSYS), Taipei, Taiwan.10.1145/3083187.3083212 ↗
20. Pogorelov, K., Randel, K.R., Griwodz, C., Eskeland, S.L., Lange, T., Johansen, D., Spampinato, C., Nguyen, D.T.D., Lux, M., and Schmidt, P.T. (2017, January 20–23). Nerthus: A bowel preparation quality video dataset. Proceedings of the 8th ACM on Multimedia Systems Conference, Taipei, Taiwan.
21. HyperKvasir, a comprehensive multi-class image and video dataset for gastrointestinal endoscopy10.1038/s41597-020-00622-y ↗
22. Bright spot regions segmentation and classification for specular highlights detection in colonoscopy videos10.1007/s00138-017-0864-0 ↗
23. WM-DOVA maps for accurate polyp highlighting in colonoscopy: Validation vs. saliency maps from physicians10.1016/j.compmedimag.2015.02.007 ↗
24. Ali, S., Ghatwary, N., Braden, B., Lamarque, D., Bailey, A., Realdon, S., Cannizzaro, R., Rittscher, J., Daul, C.D., and East, J. (2003). Endoscopy disease detection challenge. arXiv.
25. Multi-class texture analysis in colorectal cancer histology10.1038/srep27988 ↗
26. MILD-net: Minimal information loss dilated network for gland instance segmentation in colon histology images10.1016/j.media.2018.12.001 ↗
27. Graham, S., Vu, Q.D., Raza, S.E.A., Azam, A., Tsang, Y.W., Kwak, J.T., and Rajpoot, N. (2019). Hover-Net: Simultaneous segmentation and classification of nuclei in multi-tissue histology images. arXiv.10.1016/j.media.2019.101563 ↗
28. Context aware convolutional neural network for grading of colorectal cancer histology images10.1109/TMI.2020.2971006 ↗
29. PAIP 2019: Liver cancer segmentation challenge10.1016/j.media.2020.101854 ↗
30. A Stochastic Polygons Model for Glandular Structures in Colon Histology Images10.1109/TMI.2015.2433900 ↗
31. Barbano, C.A., Perlo, D., Tartaglione, E., Fiandrotti, A., Bertero, L., Cassoni, P., and Grangetto, M. (2021). UniToPatho, a labeled histopathological dataset for colorectal polyps classification and adenoma dysplasia grading. arXiv.10.1109/ICIP42928.2021.9506198 ↗
32. Ponzio, F., Macii, E., Ficarram, E., and Cataldo, S.D. (2018, January 19–21). Colorectal cancer classification using deep convolutional networks. Proceedings of the 11th International Joint Conference on Biomedical Engineering Systems and Technologies-BIOIMAGING, Madeira, Portugal.10.5220/0006643100580066 ↗
33. Deep learning-based tissue analysis predicts outcome in colorectal cancer10.1038/s41598-018-21758-3 ↗
34. Simonyan, K., and Zisserman, A. (2014). Very deep convolutional networks for large-scale image recognition. arXiv.
35. Pulver, A., and Lyu, S. (2017, January 14–19). LSTM with working memory. Proceedings of the International Joint Conference on Neural Networks (IJCNN), Anchorage, AK, USA.10.1109/IJCNN.2017.7965940 ↗
36. Yuw, X., Dimitriou, N., and Arandjelovic, O. (2019). Colorectal cancer outcome prediction from H&E whole slide images using machine learning and automatically inferred phenotype profiles. arXiv.
37. Jogin, M., Madhulika, M.S., Divya, G.D., Meghana, R.K., and Apoorva, S. (2018, January 18–19). Feature Extraction using Convolution Neural Networks (CNN) and Deep Learning. Proceedings of the 3rd IEEE International Conference on Recent Trends in Electronics, Information & Communication Technology (RTEICT), Bangalore, India.10.1109/RTEICT42901.2018.9012507 ↗
38. Kather, J.N., Krisam, J., Charoentong, P., Luedde, T., Herpel, E., Weis, C.A., Gaiser, T., Marx, A., Valous, N.A., and Ferber, D. (2019). Predicting survival from colorectal cancer histology slides using deep learning: A retrospective multicenter study. PLoS Med., 16.10.1371/journal.pmed.1002730 ↗
39. Macenko, M., Niethammer, M., Marron, J.S., Borland, D., Woosely, J.T., Guan, X., Schmitt, C., and Thomas, N.E. (July, January 28). A method for normalizing histology slides for quantitative analysis. Proceedings of the 2009 IEEE International Symposium on Biomedical Imaging: From Nano to Macro, Boston, MA, USA.10.1109/ISBI.2009.5193250 ↗
40. Krizhevsky, A., Sutskever, I., and Hinton, G.E. (2012, January 3–6). ImageNet classification with deep convolutional neural networks. Proceedings of the 25th International Conference on Neural Information Processing Systems, Lake Tahoe, NV, USA.
41. Iandola, F.N., Han, S., Moskewicz, M.W., Ashraf, K., Dallay, W.J., and Keutzer, K. (2021). SqueezeNet: AlexNet-level accuracy with 50× fewer parameters and <0.5 MB model size. arXiv.
42. Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., and Rabinovich, A. (2015, January 7–12). Going deeper with convolutions. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA.10.1109/CVPR.2015.7298594 ↗
43. He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27–30). Deep Residual Learning for Image Recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.10.1109/CVPR.2016.90 ↗
44. Automatic deep learning-based colorectal adenoma detection system and its similarities with pathologists10.1136/bmjopen-2019-036423 ↗
45. Liu, D., and Yu, J. (2009, January 12–14). Otsu Method and K-means. Proceedings of the Ninth International Conference on Hybrid Intelligent Systems, Shenyang, China.10.1109/HIS.2009.74 ↗
46. DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs10.1109/TPAMI.2017.2699184 ↗
47. Choi, K., Choi, S.J., and Kim, E.S. (2020, January 20–24). Computer-Aided Diagnosis for Colorectal Cancer using Deep Learning with Visual Explanations. Proceedings of the 42nd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC), Montreal, Canada.10.1109/EMBC44109.2020.9176653 ↗
48. Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., and Wojna, Z. (2016, January 27–30). Rethinking the Inception Architecture for Computer Vision. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.10.1109/CVPR.2016.308 ↗
49. Huang, G., Liu, Z., Maaten, L.V.D., and Weinberger, K.Q. (2017, January 21–26). Densely Connected Convolutional Networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.10.1109/CVPR.2017.243 ↗
50. Deep learning for prediction of colorectal cancer outcome: A discovery and validation study10.1016/S0140-6736(19)32998-8 ↗
51. Interpretable survival prediction for colorectal cancer using deep learning10.1038/s41746-021-00427-2 ↗
52. Prediction of the histology of colorectal neoplasm in white light colonoscopic images using deep learning algorithms10.1038/s41598-021-84299-2 ↗
53. Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., and Chen, L. (2018, January 18–23). MobileNetV2: Inverted Residuals and Linear Bottlenecks. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00474 ↗
54. Howard, A.G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., Andreetto, M., and Adam, H. (2017). MobileNets: Efficient convolutional neural networks for mobile vision applications. arXiv.
55. Microsatellite instability in colorectal cancer
56. Microsatellite instability: A review of what the oncologist should know10.1186/s12935-019-1091-8 ↗
57. Development and validation of MRI-based deep learning models for prediction of microsatellite instability in rectal cancer10.1002/cam4.3957 ↗
58. Deng, J., Dong, W., Socher, R., Li, L., Li, K., and Li, F.F. (2009, January 20–25). ImageNet: A large-scale hierarchical image database. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Miami, FL, USA.10.1109/CVPR.2009.5206848 ↗
59. Deep learning can predict microsatellite instability directly from histology in gastrointestinal cancer10.1038/s41591-019-0462-y ↗
60. Kingma, D.P., and Ba, J. (2014). Adam: A method for stochastic optimization. arXiv.
61. Lee, H., Seo, J., Lee, G., Park, J., Yeo, D., and Hong, A. (2020). Two-Stage Classification Method for MSI Status Prediction Based on Deep Learning Approach. Appl. Sci., 11.10.3390/app11010254 ↗
62. Lin, T., Dollár, P., Girshick, R., He, K., Hariharan, B., and Belongie, S. (2017, January 21–26). Feature Pyramid Networks for Object Detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.10.1109/CVPR.2017.106 ↗
63. Clinical-Grade Detection of Microsatellite Instability in Colorectal Tumors by Deep Learning10.1053/j.gastro.2020.06.021 ↗
64. Zhang, X., Zhou, X., Lin, M., and Sun, J. (2018, January 18–23). ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00716 ↗
65. A deep learning model to predict RNA-Seq expression of tumors from whole slide images10.1038/s41467-020-17678-4 ↗
66. Deep learning model for the prediction of microsatellite instability in colorectal cancer: A diagnostic study10.1016/S1470-2045(20)30535-0 ↗
67. Development and interpretation of a pathomics-based model for the prediction of microsatellite instability in colorectal cancer10.7150/thno.49864 ↗
68. A comprehensive review of deep learning in colon cancer10.1016/j.compbiomed.2020.104003 ↗
69. Deep learning to find colorectal polyps in colonoscopy: A systematic literature review10.1016/j.artmed.2020.101923 ↗
70. Godkhindi, A.M., and Gowda, R.M. (2017, January 1–2). Automated detection of polyps in CT colonography images using deep learning algorithms in colon cancer diagnosis. Proceedings of the International Conference on Energy, Communication, Data Analytics and Soft Computing (ICECDS), Chennai, India.10.1109/ICECDS.2017.8389744 ↗
71. Duran-Lopez, L., Luna-Perejon, F., Amaya-Rodriguez, I., Civit-Masot, J., Civit-Balcells, A., Vincente-Diaz, S., and Linares-Barranco, A. (2019, January 25–27). Polyp detection in gastrointestinal images using faster regional convolutional neural network. Proceedings of the 14th International Joint Conference on Computer Vision, Imaging, and Computer Graphics Theory and Applications, Prague, Czech Republic.10.5220/0007698400002108 ↗
72. Ren, S., He, K., Girshick, R., and Sun, J. (2015). Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks. arXiv.
73. Real-time detection of colon polyps during colonoscopy using deep learning: Systematic validation with four independent datasets10.1038/s41598-020-65387-1 ↗
74. Redmon, J., and Farhadi, A. (2017, January 21–26). YOLO9000: Better, Faster, Stronger. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.10.1109/CVPR.2017.690 ↗
75. Colorectal Disease Classification Using Efficiently Scaled Dilation in Convolutional Neural Network10.1109/ACCESS.2020.2996770 ↗
76. Li, Y., Zhang, X., and Chen, D. (2018, January 18–23). CSRNet: Dilated Convolutional Neural Networks for Understanding the Highly Congested Scenes. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00120 ↗
77. Ghiasi, G., Lin, T.Y., and Le, Q.V. (2018). DropBlock: A regularization method for convolutional networks. arXiv.
78. Li, K., Fathan, I.F., Patel, K., Zhang, T., Zhong, C., Bansal, A., Rastogi, A., Wang, J.S., and Wang, G. (2021). Colonoscopy polyp detection and classification: Dataset creation and comparative evaluations. PLoS ONE, 16.10.1371/journal.pone.0255809 ↗
79. Redmon, J., and Farhadi, A. (2018). YOLOv3: AN incremental Improvement. arXiv.
80. Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C., and Berg, A.C. (2015). SSD: Single Shot MultiBox Detector. arXiv.10.1007/978-3-319-46448-0_2 ↗
81. Lin, T.Y., Goyal, P., Girshick, R., He, K., and Dollar, P. (2017). Focal loss for object detection. arXiv.10.1109/ICCV.2017.324 ↗
82. Li, Z., Peng, C., Yu, G., Zhang, X., Deng, Y., and Sun, J. (2018). DetNet: A backbone network for object detection. arXiv.10.1007/978-3-030-01240-3_21 ↗
83. Zhang, S., Wen, L., Bian, X., Lei, Z., and Li, S.Z. (2017). Single shot refinement neural network for object detection. arXiv.10.1109/CVPR.2018.00442 ↗
84. Bochkovskiy, A., Wang, C., and Liao, H.M. (2020). YOLOv4: Optimal speed and accuracy of object detection. arXiv.
85. Zhang, S., Chi, C., Yao, Y., Lei, Z., and Li, S.Z. (2020). Bridging the gap between anchor-based and anchor-free detection via adaptive training sample selection. arXiv.10.1109/CVPR42600.2020.00978 ↗
86. Jha, D., Ali, S., Tomar, N.K., Johansen, H.D., Johansen, D.D., Rittscher, J., Riegler, M.A., and Halvorsen, P. (2020). Real time polyp detection, localization, and segmentation in colonoscopy using deep learning. arXiv.10.1109/ACCESS.2021.3063716 ↗
87. Tan, M., Pang, R., and Le, Q.V. (2020, January 13–19). EfficientDet: Scalable and Efficient Object Detection. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.10.1109/CVPR42600.2020.01079 ↗
88. Tan, M., and Le, Q.V. (2019). EfficientNet: Rethinking model scaling for convolutional neural networks. arXiv.
89. Automated classification and segmentation in colorectal images based on self-placed transfer network
90. Ronnerberger, O., Fischer, P., and Brox, T. (2015). U-Net: Convolutional networks for biomedical image segmentation. arXiv.10.1007/978-3-319-24574-4_28 ↗
91. Goodfellow, I.J., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., and Bengio, Y. (2014). Generative Adversarial Networks. arXiv.
92. Gatys, L.A., Ecker, A.S., and Bethge, M. (2015). A neural algorithm of artistic style. arXiv.10.1167/16.12.326 ↗
93. Zhou, S.K., Greenspan, H., Davatzikos, C., Duncan, J.S., Ginneken, B., Madabhushi, A., Prince, J.L., Rueckert, D., and Summers, R.M. (2021). A review of deep learning in medical imaging: Imaging traits, technology trends, case studies with progress highlights, and future promises. arXiv.10.1109/JPROC.2021.3054390 ↗