High Performing Facial Skin Problem Diagnosis with Enhanced Mask R-CNN and Super Resolution GAN
Source abstract
Abstract
Facial skin condition is perceived as a vital indicator of the person’s apparent age, perceived beauty, and degree of health. Machine-learning-based software analytics on facial skin conditions can be a time- and cost-efficient alternative to the conventional approach of visiting facial skin care shops or dermatologist’s offices. However, the conventional CNN-based approach is shown to be limited in the diagnosis performance due to the intrinsic characteristics of facial skin problems. In this paper, the technical challenges in facial skin problem diagnosis are first addressed, and a set of 5 effective tactics are proposed to overcome the technical challenges. A total of 31 segmentation models are trained and applied to the experiments of validating the proposed tactics. Through the experiments, the proposed approach provides 83.38% of the diagnosis performance, which is 32.58% higher than the performance of conventional CNN approach.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Cutaneous Melanoma Detection and Management · Face recognition and analysis · Industrial Vision Systems and Defect Detection
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
IBM Corporation, Costa Mesa, CA 92626, USA
Department of Software, Soongsil University, Seoul 06978, Republic of Korea
References (59)
Show source-provided references
1. An Automatic Diagnosis Method of Facial Acne Vulgaris Based on Convolutional Neural Network10.1038/s41598-018-24204-6 ↗
2. A Deep Learning-Based Facial Acne Classification System10.2147/CCID.S360450 ↗
3. A Novel Convolutional Neural Network for the Diagnosis and Classification of Rosacea: Usability Study
4. DiaMole: Mole Detection and Segmentation Software for Mobile Phone Skin Images
5. A deep learning, image based approach for automated diagnosis for inflammatory skin diseases10.21037/atm.2020.04.39 ↗
6. Gerges, F., Shih, F., and Azar, D. (2021, January 24–26). Automated Diagnosis of Acne and Rosacea using Convolution Neural Networks. Proceedings of the 2021 4th International Conference on Artificial Intelligence and Pattern Recognition (AIPR 2021), Xiamen, China.10.1145/3488933.3488993 ↗
7. HSV model-based segmentation driven facial acne detection using deep learning10.1111/exsy.12760 ↗
8. ScarNet: Development and Validation of a Novel Deep CNN Model for Acne Scar Classification With a New Dataset10.1109/ACCESS.2021.3138021 ↗
9. Bekmirzaev, S., Oh, S., and Yo, S. (2019, January 27–28). RethNet: Object-by-Object Learning for Detecting Facial Skin Problems. Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW), Seoul, Republic of Korea.10.1109/ICCVW.2019.00054 ↗
10. Skin Lesion Classification Using CNNs With Patch-Based Attention and Diagnosis-Guided Loss Weighting10.1109/TBME.2019.2915839 ↗
11. Gulzar, Y., and Khan, S.A. (2022). Skin Lesion Segmentation Based on Vision Transformers and Convolutional Neural Networks—A Comparative Study. Appl. Sci., 12.10.3390/app12125990 ↗
12. Skin disease diagnosis with deep learning: A review10.1016/j.neucom.2021.08.096 ↗
13. MDSSD: Multi-scale deconvolutional single shot detector for small objects10.1007/s11432-019-2723-1 ↗
14. Automatic pixel-level detection of vertical cracks in asphalt pavement based on GPR investigation and improved mask R-CNN10.1016/j.autcon.2022.104689 ↗
15. Attention Mask R-CNN for Ship Detection and Segmentation From Remote Sensing Images10.1109/ACCESS.2020.2964540 ↗
16. Liu, M., Wang, X., Zhou, A., Fu, X., Ma, Y., and Piao, C. (2020). UAV-YOLO: Small Object Detection on Unmanned Aerial Vehicle Perspective. Sensors, 20.10.3390/s20082238 ↗
17. Modified U-Net (mU-Net) with Incorporation of Object-Dependent High Level Features for Improved Liver and Liver-Tumor Segmentation in CT Images10.1109/TMI.2019.2948320 ↗
18. IMIIN: An Inter-modality Information Interaction Network for 3D Multi-modal Breast Tumor Segmentation10.1016/j.compmedimag.2021.102021 ↗
19. Small object detection via dual inspection mechanism for UAV visual images10.1007/s10489-021-02512-1 ↗
20. Chen, C., Zhong, J., and Tan, Y. (2019). Multiple-Oriented and Small Object Detection with Convolutional Neural Networks for Aerial Image. Remote Sens., 11.10.3390/rs11182176 ↗
21. Lightweight and computationally faster Hypermetropic Convolutional Neural Network for small size object detection10.1016/j.imavis.2022.104396 ↗
22. Collaborative Network for Super-Resolution and Semantic Segmentation of Remote Sensing Images
23. Deep learning in pore scale imaging and modeling10.1016/j.earscirev.2021.103555 ↗
24. Super-Resolution Integrated Building Semantic Segmentation for Multi-Source Remote Sensing Imagery10.1109/ACCESS.2019.2928646 ↗
25. Aboobacker, S., Verma, A., Vijayasenan, D., Suresh, P.K., and Sreeram, S. (2022, January 24–27). Semantic Segmentation on Low Resolution Cytology Images of Pleural and Peritoneal Effusion. Proceedings of the 2022 National Conference on Communications (NCC 2022), Mumbai, India.10.1109/NCC55593.2022.9806747 ↗
26. Fromm, M., Berrendorf, M., Faerman, E., Chen, Y., Schüss, B., and Schubert, M. (2019, January 8–11). XD-STOD: Cross-Domain Super resolution for Tiny Object Detection. Proceedings of the 2019 International Conference on Data Mining Workshops (ICDMW 2019), Beijing, China.10.1109/ICDMW.2019.00031 ↗
27. LFG-Net: Low-Level Feature Guided Network for Precise Ship Instance Segmentation in SAR Images10.1109/TGRS.2022.3188677 ↗
28. M3Net: A multi-model, multi-size, and multi-view deep neural network for brain magnetic resonance image segmentation10.1016/j.patcog.2019.03.004 ↗
29. Deep learning of the sectional appearances of 3D CT images for anatomical structure segmentation based on an FCN voting method10.1002/mp.12480 ↗
30. Liu, Y., Kwak, H.-S., and Oh, I.-S. (2022). Cerebrovascular Segmentation Model Based on Spatial Attention-Guided 3D Inception U-Net with Multi-Directional MIPs. Appl. Sci., 12.10.3390/app12052288 ↗
31. Automatic segmentation with detection of local segmentation failures in cardiac MRI10.1038/s41598-020-77733-4 ↗
32. Scherr, T., Löffler, K., Böhland, M., and Mikut, R. (2020). Cell segmentation and tracking using CNN-based distance predictions and a graph-based matching strategy. PLoS ONE, 15.10.1371/journal.pone.0243219 ↗
33. Larrazabal, A.J., Martinez, C., and Ferrante, E. (2019, January 13–17). Anatomical Priors for Image Segmentation via Post-processing with Denoising Autoencoders. Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI 2019), Shenzhen, China.10.52591/lxai2019120825 ↗
34. Chan, R., Rottmann, M., and Gottschalk, H. (2021, January 1–17). Entropy Maximization and Meta Classification for Out-of-Distribution Detection in Semantic Segmentation. Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), Montreal, QC, Canada.10.1109/ICCV48922.2021.00508 ↗
35. CNL-UNet: A novel lightweight deep learning architecture for multimodal biomedical image segmentation with false output suppression10.1016/j.bspc.2021.102959 ↗
36. Khan, S.A., Gulzar, Y., Turaev, S., and Peng, Y.S. (2021). A Modified HSIFT Descriptor for Medical Image Classification of Anatomy Objects. Symmetry, 13.10.3390/sym13111987 ↗
37. Hamid, Y., Elyassami, S., Gulzar, Y., Balasaraswathi, V.R., Habuza, T., and Wani, S. (2022). An improvised CNN model for fake image detection. Int. J. Inf. Technol., 2022.10.1007/s41870-022-01130-5 ↗
38. Parallel distributed processing model with local space-invariant interconnections and its optical architecture10.1364/AO.29.004790 ↗
39. An Evaluation of Deep Learning Methods for Small Object Detection
40. A survey and performance evaluation of deep learning methods for small object detection10.1016/j.eswa.2021.114602 ↗
41. Mask-guided SSD for small-object detection10.1007/s10489-020-01949-0 ↗
42. Facial skin pores: A multiethnic study10.2147/CCID.S74401 ↗
43. National Cancer Institute (2023, January 04). Common Moles, Dysplastic Nevi, and Risk of Melanoma, Available online: https://www.cancer.gov/types/skin/moles-fact-sheet.
44. He, K., Gkioxari, G., Dollár, P., and Girshick, R. (2017, January 22–29). Mask R-CNN. Proceedings of the 2017 IEEE International Conference on Computer Vision (ICCV), Venice, Italy.10.1109/ICCV.2017.322 ↗
45. Ledig, C., Theis, L., Huszár, F., Caballero, J., Cunningham, A., Acosta, A., Aitken, A.P., Tejani, A., Totz, J., and Wang, Z. (2017, January 21–26). Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.10.1109/CVPR.2017.19 ↗
46. Review of deep learning: Concepts, CNN architectures, challenges, applications, future directions10.1186/s40537-021-00444-8 ↗
47. Conceptual Understanding of Convolutional Neural Network- A Deep Learning Approach10.1016/j.procs.2018.05.069 ↗
48. Luo, C., Li, X., Wang, L., He, J., Li, D., and Zhou, J. How Does the Data set Affect CNN-based Image Classification Performance? In Proceedings of the 2018 5th International Conference on Systems and Informatics (ICSAI 2018), Nanjing, China, 10–12 November 2018.10.1109/ICSAI.2018.8599448 ↗
49. Fine-Grained Visual Categorization by Localizing Object Parts With Single Image10.1109/TMM.2020.2993960 ↗
50. Avianto, D., and Harjoko, A. (2022). Afiahayati CNN-Based Classification for Highly Similar Vehicle Model Using Multi-Task Learning. J. Imaging, 8.10.3390/jimaging8110293 ↗
51. Ju, M., Moon, S., and Yoo, C.D. (2019, January 1–3). Object Detection for Similar Appearance Objects Based on Entropy. Proceedings of the 2019 7th International Conference on Robot Intelligence Technology and Applications (RiTA 2019), Daejeon, Republic of Korea.10.1109/RITAPP.2019.8932791 ↗
52. Jang, W., and Lee, E.C. (2021). Multi-Class Parrot Image Classification Including Subspecies with Similar Appearance. Biology, 10.10.3390/biology10111140 ↗
53. (2023, January 04). Facial Landmarks Shape Predictor. Available online: https://github.com/codeniko/shape_predictor_81_face_landmarks.
54. Wu, X., Wen, N., Liang, J., Lai, Y.K., She, D., Cheng, M., and Yang, J. (November, January 27). Joint Acne Image Grading and Counting via Label Distribution Learning. Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision (ICCV 2019), Seoul, Republic of Korea.10.1109/ICCV.2019.01074 ↗
55. A Style-Based Generator Architecture for Generative Adversarial Networks10.1109/TPAMI.2020.2970919 ↗
56. Wu, X. (2023, January 04). Pytorch Implementation of Joint Acne Image Grading and Counting via Label Distribution Learning. Available online: https://github.com/xpwu95/LDL.
57. NVDIA Research Lab (2023, January 04). FFHQ Datase. Available online: https://github.com/NVlabs/ffhq-dataset.
58. Thomaz, C.E. (2023, January 04). FEI Face Database. Available online: https://fei.edu.br/~cet/facedatabase.html.
59. (2023, January 04). COCO Annotator. Available online: https://github.com/jsbroks/coco-annotator.