A Reinforcement Learning Algorithm for Automated Detection of Skin Lesions
Source abstract
Abstract
Skin cancers are increasing at an alarming rate, and detection in the early stages is essential for advanced treatment. The current segmentation methods have limited labeling ability to the ground truth images due to the numerous noisy expert annotations present in the datasets. The precise boundary segmentation is essential to correctly locate and diagnose the various skin lesions. In this work, the lesion segmentation method is proposed as a Markov decision process. It is solved by training an agent to segment the region using a deep reinforcement-learning algorithm. Our method is similar to the delineation of a region of interest by the physicians. The agent follows a set of serial actions for the region delineation, and the action space is defined as a set of continuous action parameters. The segmentation model learns in continuous action space using the deep deterministic policy gradient algorithm. The proposed method enables continuous improvement in performance as we proceed from coarse segmentation results to finer results. Finally, our proposed model is evaluated on the International Skin Imaging Collaboration (ISIC) 2017 image dataset, Human against Machine (HAM10000), and PH2 dataset. On the ISIC 2017 dataset, the algorithm achieves an accuracy of 96.33% for the naevus cases, 95.39% for the melanoma cases, and 94.27% for the seborrheic keratosis cases. The other metrics are evaluated on these datasets and rank higher when compared with the current state-of-the-art lesion segmentation algorithms.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Cutaneous Melanoma Detection and Management · Nonmelanoma Skin Cancer Studies · AI in cancer detection
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Department of Computer and Information Science, Faculty of Science and IT, Universiti Teknologi Petronas, UTP, Seri Iskandar 32610, Malaysia
Production and Systems, Graduate School of Information, Waseda University, Wakamatsu, Kitakyushu 8080135, Japan
Department of Computer Science, School of Information Technology, Monash University, Bandar Sunway, Subang Jaya 47500, Malaysia
Funding
Yayasan UTP
References (79)
Show source-provided references
1. Cancer statistics, 201610.3322/caac.21332 ↗
2. Dermatologist-level classification of skin cancer with deep neural networks10.1038/nature21056 ↗
3. Mobile teledermatology for skin tumour screening: Diagnostic accuracy of clinical and dermoscopic image tele-evaluation using cellular phones10.1111/j.1365-2133.2011.10208.x ↗
4. Alves, J., Moreira, D., Alves, P., Rosado, L., and Vasconcelos, M.J.M. (2019). Automatic focus assessment on dermoscopic images acquired with smartphones. Sensors, 19.10.3390/s19224957 ↗
5. Fighting melanoma with smartphones: A snapshot of where we are a decade after app stores opened their doors10.1016/j.ijmedinf.2018.08.004 ↗
6. ABCD rule of dermatoscopy: A new practical method for early recognition of malignant melanoma
7. The clinical diagnosis of early malignant melanoma: Expansion of the ABCD criteria to improve diagnostic sensitivity10.5070/D366X188MB ↗
8. Epiluminescence microscopy for the diagnosis of doubtful melanocytic skin lesions: Comparison of the ABCD rule of dermatoscopy and a new 7-point checklist based on pattern analysis10.1001/archderm.134.12.1563 ↗
9. In vivo epiluminescence microscopy of pigmented skin lesions. I. Pattern analysis of pigmented skin lesions10.1016/S0190-9622(87)70239-4 ↗
10. Automated melanoma recognition in dermoscopy images via very deep residual networks10.1109/TMI.2016.2642839 ↗
11. Automatic skin lesion classification based on mid-level feature learning10.1016/j.compmedimag.2020.101765 ↗
12. Codella, N.C., Gutman, D., Celebi, M.E., Helba, B., Marchetti, M.A., Dusza, S.W., Kalloo, A., Liopyris, K., Mishra, N., and Kittler, H. (2018, January 4–7). Skin lesion analysis toward melanoma detection: A challenge at the 2017 international symposium on biomedical imaging (ISBI), hosted by the international skin imaging collaboration (ISIC). Proceedings of the 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018), Washington, DC, USA.10.1109/ISBI.2018.8363547 ↗
13. Li, Y., and Shen, L. (2018). Skin lesion analysis towards melanoma detection using deep learning network. Sensors, 18.10.3390/s18020556 ↗
14. FCA-net: Adversarial learning for skin lesion segmentation based on multi-scale features and factorized channel attention10.1109/ACCESS.2019.2940418 ↗
15. Yang, X., Zeng, Z., Yeo, S.Y., Tan, C., Tey, H.L., and Su, Y. (2017). A novel multi-task deep learning model for skin lesion segmentation and classification. arXiv.
16. A mutual bootstrapping model for automated skin lesion segmentation and classification10.1109/TMI.2020.2972964 ↗
17. Humayun, J., Malik, A.S., and Kamel, N. (2011, January 17–18). Multilevel thresholding for segmentation of pigmented skin lesions. Proceedings of the IEEE International Conference on Imaging Systems and Techniques, Penang, Malaysia.10.1109/IST.2011.5962214 ↗
18. Mirikharaji, Z., and Hamarneh, G. (2018, January 16–20). Star shape prior in fully convolutional networks for skin lesion segmentation. Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention, Granada, Spain.10.1007/978-3-030-00937-3_84 ↗
19. Skin lesion segmentation using fully convolutional networks: A comparative experimental study10.1016/j.eswa.2020.113742 ↗
20. “GrabCut” interactive foreground extraction using iterated graph cuts10.1145/1015706.1015720 ↗
21. Interactive medical image segmentation using deep learning with image-specific fine tuning10.1109/TMI.2018.2791721 ↗
22. Boykov, Y.Y., and Jolly, M.P. (2001, January 7–14). Interactive graph cuts for optimal boundary & region segmentation of objects in ND images. Proceedings of the Eighth IEEE International Conference on Computer Vision, Vancouver, BC, Canada.
23. Xie, N., Zhao, T., Tian, F., Zhang, X.H., and Sugiyama, M. (2015, January 25–31). Stroke-based stylization learning and rendering with inverse reinforcement learning. Proceedings of the Twenty-Fourth International Joint Conference on Artificial Intelligence, Buenos Aires, Argentina.
24. Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., and Riedmiller, M. (2013). Playing atari with deep reinforcement learning. arXiv.
25. Lillicrap, T.P., Hunt, J.J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., and Wierstra, D. (2015). Continuous control with deep reinforcement learning. arXiv.
26. Automatic skin lesion segmentation via iterative stochastic region merging10.1109/TITB.2011.2157829 ↗
27. Active contours-based segmentation and lesion periphery analysis for characterization of skin lesions in dermoscopy images10.1109/JBHI.2018.2832455 ↗
28. Abbas, Q., Fondón, I., Sarmiento, A., and Celebi, M.E. (2014, January 22–24). An improved segmentation method for non-melanoma skin lesions using active contour model. Proceedings of the International Conference Image Analysis and Recognition, Vilamoura, Portugal.10.1007/978-3-319-11755-3_22 ↗
29. A multi-direction GVF snake for the segmentation of skin cancer images10.1016/j.patcog.2008.09.007 ↗
30. Jafari, M.H., Samavi, S., Soroushmehr, S.M.R., Mohaghegh, H., Karimi, N., and Najarian, K. (2016, January 25–28). Set of descriptors for skin cancer diagnosis using non-dermoscopic color images. Proceedings of the 2016 IEEE International Conference on Image Processing (ICIP), Phoenix, AZ, USA.10.1109/ICIP.2016.7532837 ↗
31. Ali, A.R., Couceiro, M.S., and Hassenian, A.E. (2014, January 14–16). Melanoma detection using fuzzy C-means clustering coupled with mathematical morphology. Proceedings of the 14th International Conference on Hybrid Intelligent Systems, Mubarak Al-Abdullah, Kuwait.10.1109/HIS.2014.7086175 ↗
32. He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27–30). Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.10.1109/CVPR.2016.90 ↗
33. Huang, G., Liu, Z., Maaten, L.V.D., and Weinberger, K.Q. (2017, January 21–26). Densely connected convolutional networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.10.1109/CVPR.2017.243 ↗
34. Maninis, K.K., Caelles, S., Pont-Tuset, J., and Gool, L.V. (2018, January 18–23). Deep extreme cut: From extreme points to object segmentation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00071 ↗
35. Jafari, M.H., Karimi, N., Nasr-Esfahani, E., Samavi, S., Soroushmehr, S.M.R., Ward, K., and Najarian, K. (2016, January 4–8). Skin lesion segmentation in clinical images using deep learning. Proceedings of the 23rd International Conference on Pattern Recognition (ICPR), Cancun, Mexico.10.1109/ICPR.2016.7899656 ↗
36. Ronneberger, O., Fischer, P., and Brox, T. (2015, January 5–9). U-net: Convolutional networks for biomedical image segmentation. Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention, Munich, Germany.10.1007/978-3-319-24574-4_28 ↗
37. Berseth, M. (2017). ISIC 2017-skin lesion analysis towards melanoma detection. arXiv.
38. Chang, H. (2017). Skin cancer reorganization and classification with deep neural network. arXiv.
39. Liu, L., Mou, L., Zhu, X.X., and Mandal, M. (2019, January 5–8). Skin Lesion Segmentation based on improved U-net. Proceedings of the IEEE Canadian Conference of Electrical and Computer Engineering (CCECE), Edmonton, AB, Canada.10.1109/CCECE.2019.8861848 ↗
40. Abhishek, K., Hamarneh, G., and Drew, M.S. (2020, January 14–19). Illumination-based transformations improve skin lesion segmentation in dermoscopic images. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, Seattle, WA, USA.
41. Yuan, Y. (2017). Automatic skin lesion segmentation with fully convolutional-deconvolutional networks. arXiv.10.1109/TMI.2017.2695227 ↗
42. Skin lesion segmentation in dermoscopy images via deep full resolution convolutional networks10.1016/j.cmpb.2018.05.027 ↗
43. Step-wise integration of deep class-specific learning for dermoscopic image segmentation10.1016/j.patcog.2018.08.001 ↗
44. Sarker, M.M.K., Rashwan, H.A., Akram, F., Banu, S.F., Saleh, A., Singh, V.K., Chowdhury, F.U., Abdulwahab, S., Romani, S., and Radeva, P. (2018, January 16–20). SLSDeep: Skin lesion segmentation based on dilated residual and pyramid pooling networks. Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention, Granada, Spain.10.1007/978-3-030-00934-2_3 ↗
45. Long, J., Shelhamer, E., and Darrell, T. (2015, January 7–12). Fully convolutional networks for semantic segmentation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA.10.1109/CVPR.2015.7298965 ↗
46. A deep supervised approach for ischemic lesion segmentation from multimodal MRI using Fully Convolutional Network10.1016/j.asoc.2019.105685 ↗
47. Milletari, F., Navab, N., and Ahmadi, S.A. (2016, January 25–28). V-net: Fully convolutional neural networks for volumetric medical image segmentation. Proceedings of the Fourth International Conference on 3D Vision (3DV), Stanford, CA, USA.10.1109/3DV.2016.79 ↗
48. Cheng, T., Wang, X., Huang, L., and Liu, W. (2020, January 23–28). Boundary-preserving mask r-cnn. Proceedings of the European Conference on Computer Vision, Glasgow, UK.10.1007/978-3-030-58568-6_39 ↗
49. Kim, M., Woo, S., Kim, D., and Kweon, I.S. (2021, January 5–9). The devil is in the boundary: Exploiting boundary representation for basis-based instance segmentation. Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, Waikola, HI, USA.10.1109/WACV48630.2021.00097 ↗
50. Castrejon, L., Kundu, K., Urtasun, R., and Fidler, S. (2017, January 21–26). Annotating object instances with a polygon-rnn. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.10.1109/CVPR.2017.477 ↗
51. Chen, L.C., Zhu, Y., Papandreou, G., Schroff, F., and Adam, H. (2018, January 8–14). Encoder-decoder with atrous separable convolution for semantic image segmentation. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.10.1007/978-3-030-01234-2_49 ↗
52. Dai, J., He, K., and Sun, J. (2016, January 27–30). Instance-aware semantic segmentation via multi-task network cascades. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.10.1109/CVPR.2016.343 ↗
53. Supervoxel graph cuts: An effective method for ggo candidate regions extraction on CT images10.1109/MCE.2019.2941468 ↗
54. Wound intensity correction and segmentation with convolutional neural networks10.1002/cpe.3927 ↗
55. Automatic classification of lung nodules on MDCT images with the temporal subtraction technique10.1007/s11548-017-1598-1 ↗
56. Schulman, J., Wolski, F., Dhariwal, P., Radford, A., and Klimov, O. (2017). Proximal policy optimization algorithms. arXiv.
57. Real-time personalization and recommendation in Adaptive Learning Management System10.1007/s12652-020-01729-1 ↗
58. Adaptive packet scheduling in IoT environment based on Q-learning10.1016/j.procs.2018.10.178 ↗
59. Evaluating reinforcement learning agents for anatomical landmark detection10.1016/j.media.2019.02.007 ↗
60. Caicedo, J.C., and Lazebnik, S. (2015, January 7–13). Active object localization with deep reinforcement learning. Proceedings of the IEEE International Conference on Computer Vision, Santiago, Chile.10.1109/ICCV.2015.286 ↗
61. Zhu, Y., Mottaghi, R., Kolve, E., Lim, J.J., Gupta, A., Fei-Fei, L., and Farhadi, A. (June, January 29). Target-driven visual navigation in indoor scenes using deep reinforcement learning. Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), Marina Bay Sands, Singapore.
62. Liu, F., Li, S., Zhang, L., Zhou, C., Ye, R., Wang, Y., and Lu, J. (2017, January 22–29). 3DCNN-DQN-RNN: A deep reinforcement learning framework for semantic parsing of large-scale 3D point clouds. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.10.1109/ICCV.2017.605 ↗
63. Rao, Y., Lu, J., and Zhou, J. (2017, January 22–29). Attention-aware deep reinforcement learning for video face recognition. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.10.1109/ICCV.2017.424 ↗
64. Sahba, F., Tizhoosh, H.R., and Salama, M.M. (2008). Application of reinforcement learning for segmentation of transrectal ultrasound images. BMC Med. Imaging, 8.10.1186/1471-2342-8-8 ↗
65. Q-learning10.1007/BF00992698 ↗
66. Wang, Z., Sarcar, S., Liu, J., Zheng, Y., and Ren, X. (2018). Outline objects using deep reinforcement learning. arXiv.
67. Song, G., Myeong, H., and Lee, K.M. (2018, January 18–23). Seednet: Automatic seed generation with deep reinforcement learning for robust interactive segmentation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.
68. The HAM10000 dataset, a large collection of multi-source dermatoscopic images of common pigmented skin lesions10.1038/sdata.2018.161 ↗
69. Finlayson, G.D., and Trezzi, E. (2004, January 9–12). Shades of gray and colour constancy. Proceedings of the Color and Imaging Conference, Scottsdale, AZ, USA.10.2352/CIC.2004.12.1.art00008 ↗
70. Silver, D., Lever, G., Heess, N., Degris, T., Wierstra, D., and Riedmiller, M. (2014, January 21–26). Deterministic policy gradient algorithms. Proceedings of the International conference on machine learning, Beijing, China.
71. Sutton, R.S., McAllester, D.A., Singh, S.P., and Mansour, Y. (2000, January 6–12). Policy gradient methods for reinforcement learning with function approximation. Proceedings of the Advances in Neural Information Processing Systems, Virtual.
72. Ternary adversarial networks with self-supervision for zero-shot cross-modal retrieval10.1109/TCYB.2019.2928180 ↗
73. Ioffe, S., and Szegedy, C. (2015, January 6–11). Batch normalization: Accelerating deep network training by reducing internal covariate shift. Proceedings of the International Conference on Machine Learning, Lille, France.
74. Shi, W., Caballero, J., Huszár, F., Totz, J., Aitken, A.P., Bishop, R., Rueckert, D., and Wang, Z. (2016, January 27–30). Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.10.1109/CVPR.2016.207 ↗
75. Automated skin lesion segmentation using attention-based deep convolutional neural network10.1016/j.bspc.2020.102358 ↗
76. SLSNet: Skin lesion segmentation using a lightweight generative adversarial network10.1016/j.eswa.2021.115433 ↗
77. Liu, L., Tsui, Y.Y., and Mandal, M. (2021). Skin lesion segmentation using deep learning with auxiliary task. J. Imaging, 7.10.3390/jimaging7040067 ↗
78. Knowledge-aware Deep Framework for Collaborative Skin Lesion Segmentation and Melanoma Recognition10.1016/j.patcog.2021.108075 ↗
79. Lightweight encoder-decoder model for automatic skin lesion segmentation10.1016/j.imu.2021.100640 ↗