A Deep Learning Method for Early Detection of Diabetic Foot Using Decision Fusion and Thermal Images
Source abstract
Abstract
Diabetes mellitus (DM) is one of the major diseases that cause death worldwide and lead to complications of diabetic foot ulcers (DFU). Improper and late handling of a diabetic foot patient can result in an amputation of the patient’s foot. Early detection of DFU symptoms can be observed using thermal imaging with a computer-assisted classifier. Previous study of DFU detection using thermal image only achieved 97% of accuracy, and it has to be improved. This article proposes a novel framework for DFU classification based on thermal imaging using deep neural networks and decision fusion. Here, decision fusion combines the classification result from a parallel classifier. We used the convolutional neural network (CNN) model of ShuffleNet and MobileNetV2 as the baseline classifier. In developing the classifier model, firstly, the MobileNetV2 and ShuffleNet were trained using plantar thermogram datasets. Then, the classification results of those two models were fused using a novel decision fusion method to increase the accuracy rate. The proposed framework achieved 100% accuracy in classifying the DFU thermal images in binary classes of positive and negative cases. The accuracy of the proposed Decision Fusion (DF) was increased by about 3.4% from baseline ShuffleNet and MobileNetV2. Overall, the proposed framework outperformed in classifying the images compared with the state-of-the-art deep learning and the traditional machine-learning-based classifier.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Diabetic Foot Ulcer Assessment and Management · Infrared Thermography in Medicine · Spectroscopy Techniques in Biomedical and Chemical Research
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Department of Electrical and Computer Engineering, Universitas Syiah Kuala, Banda Aceh 23111, Indonesia
Telematics Research Center, Universitas Syiah Kuala, Banda Aceh 23111, Indonesia
Medical Faculty, Universitas Syiah Kuala, Banda Aceh 23111, Indonesia
Department of Informatics, Universitas Syiah Kuala, Banda Aceh 23111, Indonesia
Funding
Ministry of Education, Culture, Research, and Technology, the Republic of Indonesia
References (50)
Show source-provided references
1. American Diabetes Association (2009). Diagnosis and classification of diabetes mellitus. Diabetes Care, 32, S62–S67.10.2337/dc09-S062 ↗
2. Computer aided detection of diabetic foot ulcer using asymmetry analysis of texture and temperature features10.1016/j.infrared.2020.103219 ↗
3. Neuropathic complications: Type II diabetes mellitus and other risky parameters using machine learning algorithms
4. Study protocol for a randomized controlled trial to test for preventive effects of diabetic foot ulceration by telemedicine that includes sensor-equipped insoles combined with photo documentation10.1186/s13063-019-3623-x ↗
5. Wijesinghe, I., Gamage, C., Perera, I., and Chitraranjan, C. (2019, January 3–5). A smart telemedicine system with deep learning to manage diabetic retinopathy and foot ulcers. Proceedings of the 2019 Moratuwa Engineering Research Conference (MERCon), Moratuwa, Sri Lanka.10.1109/MERCon.2019.8818682 ↗
6. Infrared thermography: A non-invasive window into thermal physiology10.1016/j.cbpa.2016.02.022 ↗
7. Cruz-Vega, I., Hernandez-Contreras, D., Peregrina-Barreto, H., Rangel-Magdaleno, J.d.J., and Ramirez-Cortes, J.M. (2020). Deep Learning Classification for Diabetic Foot Thermograms. Sensors, 20.10.3390/s20061762 ↗
8. Dfunet: Convolutional neural networks for diabetic foot ulcer classification10.1109/TETCI.2018.2866254 ↗
9. Vardasca, R., Magalhaes, C., Seixas, A., Carvalho, R., and Mendes, J. (2019, January 1–5). Diabetic foot monitoring using dynamic thermography and AI classifiers. Proceedings of the 3rd Quantitative InfraRed Thermography Asia Conference (QIRT Asia 2019), Tokyo, Japan.10.21611/qirt.2019.027 ↗
10. DFU_QUTNet: Diabetic foot ulcer classification using novel deep convolutional neural network10.1007/s11042-019-07820-w ↗
11. Gamage, C., Wijesinghe, I., and Perera, I. (2019, January 28–30). Automatic Scoring of Diabetic Foot Ulcers through Deep CNN Based Feature Extraction with Low Rank Matrix Factorization. Proceedings of the 2019 IEEE 19th International Conference on Bioinformatics and Bioengineering (BIBE), Athens, Greece.10.1109/BIBE.2019.00069 ↗
12. Rania, N., Douzi, H., Yves, L., and Sylvie, T. (2020, January 4–6). Semantic Segmentation of Diabetic Foot Ulcer Images: Dealing with Small Dataset in DL Approaches. Proceedings of the International Conference on Image and Signal Processing, Marrakesh, Morocco.10.1007/978-3-030-51935-3_17 ↗
13. Goyal, M., Yap, M.H., Reeves, N.D., Rajbhandari, S., and Spragg, J. Fully convolutional networks for diabetic foot ulcer segmentation. Proceedings of the 2017 IEEE international conference on systems, man, and cybernetics (SMC), Banff, AB, Canada, 5–8 October 2017.10.1109/SMC.2017.8122675 ↗
14. A machine learning model for early detection of diabetic foot using thermogram images10.1016/j.compbiomed.2021.104838 ↗
15. A survey of decision fusion and feature fusion strategies for pattern classification10.4103/0256-4602.64604 ↗
16. Application of infrared thermography in computer aided diagnosis10.1016/j.infrared.2014.06.001 ↗
17. Optical illusions from visual data analysis: Example of the New Zealand asthma mortality epidemic10.1016/S0895-4356(97)00158-3 ↗
18. Computer aided diagnosis of diabetic foot using infrared thermography: A review10.1016/j.compbiomed.2017.10.030 ↗
19. A review on recent progress in thermal imaging and deep learning approaches for breast cancer detection10.1109/ACCESS.2020.3004056 ↗
20. BreaCNet: A high-accuracy breast thermogram classifier based on mobile convolutional neural network10.3934/mbe.2022060 ↗
21. Plantar thermogram database for the study of diabetic foot complications10.1109/ACCESS.2019.2951356 ↗
22. He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27–30). Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.10.1109/CVPR.2016.90 ↗
23. Huang, G., Liu, Z., Van Der Maaten, L., and Weinberger, K.Q. (2017, January 21–26). Densely connected convolutional networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.10.1109/CVPR.2017.243 ↗
24. Chollet, F. (2017, January 21–26). Xception: Deep learning with depthwise separable convolutions. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.10.1109/CVPR.2017.195 ↗
25. Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., and Rabinovich, A. (2015, January 7–12). Going deeper with convolutions. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA.10.1109/CVPR.2015.7298594 ↗
26. Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., and Chen, L.C. (2018, January 18–22). Mobilenetv2: Inverted residuals and linear bottlenecks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00474 ↗
27. Zhang, X., Zhou, X., Lin, M., and Sun, J. (2018, January 18–22). Shufflenet: An extremely efficient convolutional neural network for mobile devices. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00716 ↗
28. Krizhevsky, A., Sutskever, I., and Hinton, G.E. (2012, January 3–6). Imagenet classification with deep convolutional neural networks. Proceedings of the Advances in Neural Information Processing Systems, Lake Tahoe, NV, USA.
29. Ioannou, Y., Robertson, D., Cipolla, R., and Criminisi, A. (2017, January 21–26). Deep roots: Improving cnn efficiency with hierarchical filter groups. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.10.1109/CVPR.2017.633 ↗
30. Xie, S., Girshick, R., Dollár, P., Tu, Z., and He, K. (2017, January 21–26). Aggregated residual transformations for deep neural networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.10.1109/CVPR.2017.634 ↗
31. Howard, A.G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., Andreetto, M., and Adam, H. (2017). Mobilenets: Efficient convolutional neural networks for mobile vision applications. arXiv.
32. Tan, M., and Le, Q. (2019, January 17–23). Efficientnet: Rethinking model scaling for convolutional neural networks. Proceedings of the International Conference on Machine Learning, Baltimore, MD, USA.
33. Roggen, D., Tröster, G., and Bulling, A. (2013). Signal processing technologies for activity-aware smart textiles. Multidisciplinary Know-How for Smart-Textiles Developers, Elsevier.10.1533/9780857093530.2.329 ↗
34. Decision fusion for image quality assessment using an optimization approach10.1109/LSP.2015.2500819 ↗
35. VPRS-based regional decision fusion of CNN and MRF classifications for very fine resolution remotely sensed images10.1109/TGRS.2018.2822783 ↗
36. Convolutional and Deep Neural Networks based techniques for extracting the age-relevant features of the speaker
37. Medical image fusion: A survey of the state of the art10.1016/j.inffus.2013.12.002 ↗
38. Deep learning decision fusion for the classification of urban remote sensing data10.1117/1.JRS.12.016038 ↗
39. Medical image classification based on deep features extracted by deep model and statistic feature fusion with multilayer perceptron10.1155/2018/2061516 ↗
40. A hybrid approach of neural networks for age and gender classification through decision fusion10.1016/j.bspc.2021.102459 ↗
41. A review: Deep learning for medical image segmentation using multi-modality fusion10.1016/j.array.2019.100004 ↗
42. Roslidar, R., Saddami, K., Arnia, F., Syukri, M., and Munadi, K. (2019, January 22–24). A study of fine-tuning CNN models based on thermal imaging for breast cancer classification. Proceedings of the 2019 IEEE International Conference on Cybernetics and Computational Intelligence (CyberneticsCom), Banda Aceh, Indonesia.10.1109/CYBERNETICSCOM.2019.8875661 ↗
43. Deep learning for infrared thermal image based machine health monitoring10.1109/TMECH.2017.2722479 ↗
44. Rizka, R., Saddami, K., Roslidar, R., Munadi, K., and Fitri, A. (2021). On Reducing the ShuffleNet Block for Mobile-based Breast Cancer Detection Using Thermogram: Performance Evaluation. Appl. Comput. Intell. Soft Comput., under review.
45. Infrared thermal imaging for automated detection of diabetic foot complications10.1177/193229681300700504 ↗
46. Brain tumor segmentation using convolutional neural networks in MRI images10.1109/TMI.2016.2538465 ↗
47. Dai, Y., Gao, Y., and Liu, F. (2021). TransMed: Transformers Advance Multi-Modal Medical Image Classification. Diagnostics, 11.10.3390/diagnostics11081384 ↗
48. A novel deep learning based framework for the detection and classification of breast cancer using transfer learning10.1016/j.patrec.2019.03.022 ↗
49. Working set selection using second order information for training support vector machines
50. Balestriero, R., Bottou, L., and LeCun, Y. (2022). The effects of regularization and data augmentation are class dependent. arXiv.