Instance Segmentation Based on Deep Convolutional Neural Networks and Transfer Learning for Unconstrained Psoriasis Skin Images
Source abstract
Abstract
In this paper, an efficient instance segmentation scheme based on deep convolutional neural networks is proposed to deal with unconstrained psoriasis images for computer-aided diagnosis. To achieve instance segmentation, the You Only Look At CoefficienTs (YOLACT) network composed of backbone, feature pyramid network (FPN), Protonet, and prediction head is used to deal with psoriasis images. The backbone network is used to extract feature maps from an image, and FPN is designed to generate multiscale feature maps for effectively classifying and localizing objects with multiple sizes. The prediction head is used to predict the classification information, bounding box information, and mask coefficients of objects. Some prototypes generated by Protonet are combined with mask coefficients to estimate the pixel-level shapes for objects. To achieve instance segmentation for unconstrained psoriasis images, YOLACT++ with a pretrained model is retrained via transfer learning. To evaluate the performance of the proposed scheme, unconstrained psoriasis images with different severity levels are collected for testing. As for subjective testing, the psoriasis regions and normal skin areas can be located and classified well. The four performance indices of the proposed scheme were higher than 93% after cross validation. About object localization, the Mean Average Precision (mAP) rates of the proposed scheme were at least 85.9% after cross validation. As for efficiency, the frames per second (FPS) rate of the proposed scheme reached up to 15. In addition, the F1_score and the execution speed of the proposed scheme were higher than those of the Mask Region-Based Convolutional Neural Networks (R-CNN)-based method. These results show that the proposed scheme based on YOLACT++ can not only detect psoriasis regions but also distinguish psoriasis pixels from background and normal skin pixels well. Furthermore, the proposed instance segmentation scheme outperforms the Mask R-CNN-based method for unconstrained psoriasis images.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Digital Imaging for Blood Diseases · Psoriasis: Treatment and Pathogenesis · Systemic Lupus Erythematosus Research
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Department of Computer Science and Information Engineering, National Chin-Yi University of Technology, Taichung 411, Taiwan
Department of Computer Science and Information Engineering, Da-Yeh University, Changhua 515, Taiwan
Department of Health Services Administration, China Medical University, Taichung 411, Taiwan
References (38)
Show source-provided references
1. First review on psoriasis severity risk stratification: An engineering perspective10.1016/j.compbiomed.2015.05.005 ↗
2. Fuzzy based visual texture feature for psoriasis image analysis
3. Automatic segmentation of scaling in 2-D psoriasis skin images10.1109/TMI.2012.2236349 ↗
4. Segmentation of psoriasis vulgaris images using multiresolution-based orthogonal subspace techniques10.1109/TSMCB.2005.859935 ↗
5. Psoriasis image identification using k-means clustering with morphological processing10.1016/j.measurement.2011.02.006 ↗
6. Lesion area assessment in psoriasis patients10.3109/03091902.2011.554598 ↗
7. Sonka, M., Hlavac, V., and Boyle, R. (2008). Image Processing, Analysis, and Machine Vision.
8. Computer-aided diagnosis of psoriasis skin images with HOS, texture and color features: A first comparative study of its kind10.1016/j.cmpb.2015.11.013 ↗
9. Computer-aided diagnosis (CAD) system based on multi-layer feature fusion network for skin lesion recognition in dermoscopy images10.1007/s11042-019-07988-1 ↗
10. Krizhevsky, A., Sutskever, I., and Hinton, G.E. (2012, January 13–16). Imagenet classification with deep convolutional neural networks. Proceedings of the 25th International Conference on Neural Information Processing Systems, Siem Reap, Cambodia.
11. Minaee, S., Boykov, Y., Porikli, F., Plaza, A., Kehtarnavaz, N., and Terzopoulos, D. (2020). Image segmentation using deep learning: A survey. arXiv.10.1109/TPAMI.2021.3059968 ↗
12. Syu, J.M., Lai, C.H., Lin, G.S., and Chai, S.K. (2018, January 12–15). Psoriasis detection based on deep neural network. Proceedings of the 2018 IEEE International Conference on Consumer Electronics—TW (ICCE-TW), Las Vegas, NV, USA.10.1109/ICCE-China.2018.8448609 ↗
13. Ronneberger, O., Fischer, P., and Brox, T. (2015). U-Net: Convolutional networks for biomedical image segmentation. arXiv.10.1007/978-3-319-24574-4_28 ↗
14. Deep learning approaches to biomedical image segmentation
15. Deep learning for cardiac image segmentation: A review10.3389/fcvm.2020.00025 ↗
16. Real-time segmentation and tracking of excised corneal contour by deep neural networks for DALK surgical navigation10.1016/j.cmpb.2020.105679 ↗
17. Lei, T., Wang, R., Wan, Y., Zhang, B., Meng, H., and Nandi, A.K. (2020). Medical Image Segmentation Using Deep Learning: A Survey. arXiv.
18. MIScnn: A framework for medical image segmentation with convolutional neural networks and deep learning
19. BB-UNet: U-Net with bounding box prior10.1109/JSTSP.2020.3001502 ↗
20. Deep learning techniques for medical image segmentation: Achievements and challenges10.1007/s10278-019-00227-x ↗
21. Mustafina, V.S., and Sukhov, M.V. (2020, January 6–9). Identification of Psoriasis by Images Using Convolutional Neural Networks. Proceedings of the 2020 International Multi-Conference on Industrial Engineering and Modern Technologies, Vladivostok, Russia.10.1109/FarEastCon50210.2020.9271632 ↗
22. Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C.Y., and Berg, A. (2016, January 11–14). SSD: Single shot multibox detector. Proceedings of the European Conference on Computer Vision, Amsterdam, The Netherlands.10.1007/978-3-319-46448-0_2 ↗
23. Ren, S., He, K., Girshick, R., and Sun, J. (2015, January 8–13). Faster R-CNN: Towards realtime object detection with region proposal networks. Proceedings of the 28th International Conference on Neural Information Processing Systems, Montreal, QC, Canada.
24. He, K., Gkioxari, G., Dollar, P., and Girshick, R. (2017). Mask R-CNN. arXiv.10.1109/ICCV.2017.322 ↗
25. Li, Y., Qi, H., Dai, J., Ji, X., and Wei, Y. (2017, January 16–20). Fully convolutional instance aware semantic segmentation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA.10.1109/CVPR.2017.472 ↗
26. Bolya, D., Zhou, C., Xiao, F., and Lee, Y.J. (2019, January 27–28). YOLACT: Real-time Instance Segmentation. Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), Seoul, Korea.10.1109/ICCV.2019.00925 ↗
27. Xie, E., Sun, P., Song, X., Wang, W., Liu, X., Liang, D., Sehn, C., and Luo, P. (2019). PolarMask: Single shot instance segmentation with polar representation. arXiv.10.1109/CVPR42600.2020.01221 ↗
28. Yu, J., Yao, J., Zhang, J., Yu, Z., and Tao, D. (2019). SPRNet: Single pixel reconstruction for one-stage instance segmentation. arXiv.
29. A Survey on Transfer Learning10.1109/TKDE.2009.191 ↗
30. Lin, T.Y., Maire, M., Belongie, S., Bourdev, L., Girshick, R., Hays, J., Perona, P., Ramanan, D., Zitnick, C.L., and Dollár, P. (2015). Microsoft COCO: Common objects in context. arXiv.10.1007/978-3-319-10602-1_48 ↗
31. A survey on Image Data Augmentation for Deep Learning10.1186/s40537-019-0197-0 ↗
32. Zhu, X., Hu, H., Lin, S., and Dai, J. (2018). Deformable ConvNets v2: More Deformable, Better Results. arXiv.10.1109/CVPR.2019.00953 ↗
33. Bolya, D., Zhou, C., Xiao, F., and Lee, Y.J. (2020). YOLACT++: Better Real-time Instance Segmentation. arXiv.10.1109/ICCV.2019.00925 ↗
34. A gender classification scheme Based on multi-region feature extraction and information fusion for unconstrained Images10.1007/s11042-015-2797-9 ↗
35. Detection of Frame Duplication Forgery in Videos Based on Spatial and Temporal Analysis
36. A feature-based Scheme for detecting and classifying video-shot transitions based on spatio-temporal analysis and fuzzy classification10.1142/S0218001409007521 ↗
37. Vision-Based Patient Identification Recognition Based on Image Content Analysis and Support Vector Machine for Medical Information System10.1186/s13634-020-00686-3 ↗
38. Duda, R.O., Hart, P.E., and Stork, D.G. (2001). Pattern Classification, John Wiley & Sons, Inc.