Multi-Scale Attention Fusion with Lesion-Area Focus for Knowledge-Enhanced Dermoscopic Skin Lesion Classification
Source abstract
Abstract
Skin diseases are common conditions that pose a significant threat to human health, and automated classification plays an important role in assisting clinical diagnosis. However, existing image classification approaches based on convolutional neural networks (CNNs) and Transformers have inherent limitations. CNNs are constrained in capturing global features, whereas Transformers are less effective in modeling local details. Given the characteristics of dermoscopic images, both local and global features are equally crucial for classification tasks. To address this issue, we propose an improved Swin Transformer-based model, termed MaLafFormer, which incorporates a Modulated Fusion of Multi-scale Attention (MFMA) module and a Lesion-Area Focus (LAF) module to enhance global modeling, emphasize critical local regions, and improve lesion boundary perception. Experimental results on the ISIC2018 dataset show that MaLafFormer achieves 84.35% ± 0.56% accuracy (mean of three runs), outperforming the baseline 77.98% ± 0.34% by 6.37%, and surpasses other compared methods across multiple metrics, thereby validating its effectiveness for skin lesion classification tasks.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Cutaneous Melanoma Detection and Management · Face recognition and analysis · AI in cancer detection
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
College of Mathematics and Computer Science, Zhejiang A & F University, Hangzhou 311300, China
Institute of Informatics, Georg-August-Universität Göttingen, 37073 Göttingen, Germany
Funding
Humanity and Social Science Foundation of Ministry of Education of China · Zhejiang Province Soft Science Research Program Project · Open Access Publication Funds/transformative agreements of the Göttingen University · China Scholarship Council
References (60)
Show source-provided references
1. Dialogue between Skin Microbiota and Immunity10.1126/science.1260144 ↗
2. Artificial Intelligence and Machine Learning Algorithms for Early Detection of Skin Cancer in Community and Primary Care Settings: A Systematic Review10.1016/S2589-7500(22)00023-1 ↗
3. 33256 Skin Cancer Screening Practices among Dermatologists: A Survey Study10.1016/j.jaad.2022.06.844 ↗
4. Feature-Based Image Patch Approximation for Lung Tissue Classification10.1109/TMI.2013.2241448 ↗
5. Thanh, D.N.H., Erkan, U., Prasath, V.B.S., Kumar, V., and Hien, N.N. (2019, January 16–17). A Skin Lesion Segmentation Method for Dermoscopic Images Based on Adaptive Thresholding with Normalization of Color Models. Proceedings of the 2019 6th International Conference on Electrical and Electronics Engineering (ICEEE), Istanbul, Turkey.10.1109/ICEEE2019.2019.00030 ↗
6. ImageNet Classification with Deep Convolutional Neural Networks10.1145/3065386 ↗
7. Artificial Intelligence Entering the Pathology Arena in Oncology: Current Applications and Future Perspectives10.1016/j.annonc.2025.03.006 ↗
8. Simonyan, K., and Zisserman, A. (2015). Very Deep Convolutional Networks for Large-Scale Image Recognition. arXiv.
9. Skin Lesion Classification Using Modified Deep and Multi-Directional Invariant Handcrafted Features10.1016/j.jnca.2024.103949 ↗
10. Howard, A.G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., Andreetto, M., and Adam, H. (2017). MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications. arXiv.
11. Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., and Chen, L.-C. (2018, January 18–23). MobileNetV2: Inverted Residuals and Linear Bottlenecks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00474 ↗
12. Howard, A., Sandler, M., Chu, G., Chen, L.-C., Chen, B., Tan, M., Wang, W., Zhu, Y., Pang, R., and Vasudevan, V. (November, January 27). Searching for MobileNetV3. Proceedings of the IEEE/CVF International Conference on Computer Vision, Seoul, Republic of Korea.
13. Elaziz, M.A., Dahou, A., Mabrouk, A., El-Sappagh, S., and Aseeri, A.O. (2023). An Efficient Artificial Rabbits Optimization Based on Mutation Strategy For Skin Cancer Prediction. Comput. Biol. Med., 163.10.1016/j.compbiomed.2023.107154 ↗
14. Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., and Gelly, S. (2021). An Image Is Worth 16x16 Words: Transformers for Image Recognition at Scale. arXiv.
15. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, L., and Polosukhin, I. (2023). Attention Is All You Need. arXiv.
16. Wang, R., Chen, X., Wang, X., Wang, H., Qian, C., Yao, L., and Zhang, K. (2024). A Novel Approach for Melanoma Detection Utilizing GAN Synthesis and Vision Transformer. Comput. Biol. Med., 176.10.1016/j.compbiomed.2024.108572 ↗
17. Hou, Z., Yu, B., and Tao, D. (2022, January 18–24). BatchFormer: Learning to Explore Sample Relationships for Robust Representation Learning. Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA.10.1109/CVPR52688.2022.00711 ↗
18. Manzari, O.N., Asgariandehkordi, H., Koleilat, T., Xiao, Y., and Rivaz, H. (2025). Medical Image Classification with KAN-Integrated Transformers and Dilated Neighborhood Attention. arXiv.
19. Fusion of Transformer Attention and CNN Features for Skin Cancer Detection10.1016/j.asoc.2024.112013 ↗
20. Lilhore, U.K., Sharma, Y.K., Simaiya, S., Alroobaea, R., Baqasah, A.M., Alsafyani, M., and Alhazmi, A. (2025). SkinEHDLF a Hybrid Deep Learning Approach for Accurate Skin Cancer Classification in Complex Systems. Sci. Rep., 15.10.1038/s41598-025-98205-7 ↗
21. Akilandasowmya, G., Nirmaladevi, G., Suganthi, S., and Aishwariya, A. (2024). Skin Cancer Diagnosis: Leveraging Deep Hidden Features and Ensemble Classifiers for Early Detection and Classification. Biomed. Signal Process. Control, 88.10.1016/j.bspc.2023.105306 ↗
22. Breast Cancer Diagnosis in DCE-MRI Using Mixture Ensemble of Convolutional Neural Networks10.1016/j.patcog.2017.08.004 ↗
23. Two-Stage Selective Ensemble of CNN via Deep Tree Training for Medical Image Classification10.1109/TCYB.2021.3061147 ↗
24. A Novel Approach for Liver Image Classification: PH-C-ELM10.1016/j.measurement.2019.01.060 ↗
25. Combining CNN-Based Histologic Whole Slide Image Analysis and Patient Data to Improve Skin Cancer Classification10.1016/j.ejca.2021.02.032 ↗
26. Huo, X., Sun, G., Tian, S., Wang, Y., Yu, L., Long, J., Zhang, W., and Li, A. (2024). HiFuse: Hierarchical Multi-Scale Feature Fusion Network for Medical Image Classification. Biomed. Signal Process. Control, 87.10.1016/j.bspc.2023.105534 ↗
27. Automated Abnormality Classification of Chest Radiographs Using Deep Convolutional Neural Networks10.1038/s41746-020-0273-z ↗
28. TRk-CNN: Transferable Ranking-CNN for Image Classification of Glaucoma, Glaucoma Suspect, and Normal Eyes10.1016/j.eswa.2021.115211 ↗
29. Deep Echocardiography: Data-Efficient Supervised and Semi-Supervised Deep Learning towards Automated Diagnosis of Cardiac Disease10.1038/s41746-018-0065-x ↗
30. PENet—a Scalable Deep-Learning Model for Automated Diagnosis of Pulmonary Embolism Using Volumetric CT Imaging10.1038/s41746-020-0266-y ↗
31. Forecast Stock Price Based on GRA-LoGo Model of Information Filtering Networks
32. Nonlinear Parsimonious Modeling Based on Copula–LoGo10.1016/j.eswa.2024.124774 ↗
33. A Novel Combining Method of Dynamic and Static Web Crawler with Parallel Computing10.1007/s11042-023-17925-y ↗
34. Multi-Feature Stock Price Prediction by LSTM Networks Based on VMD and TMFG10.1186/s40537-025-01127-4 ↗
35. ViTAEv2: Vision Transformer Advanced by Exploring Inductive Bias for Image Recognition and Beyond10.1007/s11263-022-01739-w ↗
36. Multiscale Sparse Cross-Attention Network for Remote Sensing Scene Classification
37. AgriTL-ViT: A Vision Transformer Model with Attention Techniques for Classification of Plant Leaf Disease10.1016/j.eswa.2025.128793 ↗
38. Image Super-Resolution Using Dilated Window Transformer10.1109/ACCESS.2023.3284539 ↗
39. Transformer and GAN-Based Super-Resolution Reconstruction Network for Medical Images10.26599/TST.2022.9010071 ↗
40. Zhang, Y., Zhou, S., and Li, H. (2024, January 16). Depth Information Assisted Collaborative Mutual Promotion Network for Single Image Dehazing. Proceedings of the 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.10.1109/CVPR52733.2024.00275 ↗
41. TransGAN: Two Pure Transformers Can Make One Strong GAN, and That Can Scale Up
42. Crowson, K., Baumann, S.A., Birch, A., Abraham, T.M., Kaplan, D.Z., and Shippole, E. (2024, January 21–27). Scalable High-Resolution Pixel-Space Image Synthesis with Hourglass Diffusion Transformers. Proceedings of the Forty-First International Conference on Machine Learning, Vienna, Austria.
43. Chen, S., Xu, M., Ren, J., Cong, Y., He, S., Xie, Y., Sinha, A., Luo, P., Xiang, T., and Perez-Rua, J.-M. (2024, January 16). GenTron: Diffusion Transformers for Image and Video Generation. Proceedings of the 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.10.1109/CVPR52733.2024.00616 ↗
44. Zhu, L., Huang, Z., Liao, B., Liew, J.H., Yan, H., Feng, J., and Wang, X. (2025, January 10–17). DiG: Scalable and Efficient Diffusion Models with Gated Linear Attention. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA.10.1109/CVPR52734.2025.00718 ↗
45. Xin, C., Liu, Z., Zhao, K., Miao, L., Ma, Y., Zhu, X., Zhou, Q., Wang, S., Li, L., and Yang, F. (2022). An Improved Transformer Network for Skin Cancer Classification. Comput. Biol. Med., 149.10.1016/j.compbiomed.2022.105939 ↗
46. Boosted Nutcracker Optimizer and Chaos Game Optimization with Cross Vision Transformer for Medical Image Classification10.1016/j.eij.2024.100457 ↗
47. RanMerFormer: Randomized Vision Transformer with Token Merging for Brain Tumor Classification10.1016/j.neucom.2023.127216 ↗
48. Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., and Guo, B. (2021, January 11–17). Swin Transformer: Hierarchical Vision Transformer Using Shifted Windows. Proceedings of the IEEE/CVF International Conference on Computer Vision, Virtual.10.1109/ICCV48922.2021.00986 ↗
49. UniFormer: Unifying Convolution and Self-Attention for Visual Recognition10.1109/TPAMI.2023.3282631 ↗
50. Training Data-Efficient Image Transformers & Distillation through Attention
51. Yuan, L., Chen, Y., Wang, T., Yu, W., Shi, Y., Jiang, Z., Tay, F.E.H., Feng, J., and Yan, S. (2021, January 11–17). Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNet. Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision (ICCV), Montreal, QC, Canada.10.1109/ICCV48922.2021.00060 ↗
52. Mehta, S., and Rastegari, M. (2022). Mobilevit: Light-Weight, General-Purpose, and Mobile-Friendly Vision Transformer. arXiv.
53. GasHis-Transformer: A Multi-Scale Visual Transformer Approach for Gastric Histopathological Image Detection10.1016/j.patcog.2022.108827 ↗
54. CellViT: Vision Transformers for Precise Cell Segmentation and Classification10.1016/j.media.2024.103143 ↗
55. Mammography Classification with Multi-View Deep Learning Techniques: Investigating Graph and Transformer-Based Architectures10.1016/j.media.2024.103320 ↗
56. Ferrari, V., Hebert, M., Sminchisescu, C., and Weiss, Y. (2018). Module. Computer Vision–ECCV 2018, Springer International Publishing.10.1007/978-3-030-01240-3 ↗
57. Codella, N., Rotemberg, V., Tschandl, P., Celebi, M.E., Dusza, S., Gutman, D., Helba, B., Kalloo, A., Liopyris, K., and Marchetti, M. (2019). Skin Lesion Analysis Toward Melanoma Detection 2018: A Challenge Hosted by the International Skin Imaging Collaboration (ISIC). arXiv.
58. Liu, Z., Mao, H., Wu, C.-Y., Feichtenhofer, C., Darrell, T., and Xie, S. (2022, January 19–24). A ConvNet for the 2020s. Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA.10.1109/CVPR52688.2022.01167 ↗
59. Yue, Y., and Li, Z. (2024). MedMamba: Vision Mamba for Medical Image Classification. arXiv.
60. Liu, Y., Tian, Y., Zhao, Y., Yu, H., Xie, L., Wang, Y., Ye, Q., Jiao, J., and Liu, Y. (2024). VMamba: Visual State Space Model. arXiv.