Federated Learning: Centralized and P2P for a Siamese Deep Learning Model for Diabetes Foot Ulcer Classification
Source abstract
Abstract
It is a known fact that AI models need massive amounts of data for training. In the medical field, the data are not necessarily available at a single site but are distributed over several sites. In the field of medical data sharing, particularly among healthcare institutions, the need to maintain the confidentiality of sensitive information often restricts the comprehensive utilization of real-world data in machine learning. To address this challenge, our study experiments with an innovative approach using federated learning to enable collaborative model training without compromising data confidentiality and privacy. We present an adaptation of the federated averaging algorithm, a predominant centralized learning algorithm, to a peer-to-peer federated learning environment. This adaptation led to the development of two extended algorithms: Federated Averaging Peer-to-Peer and Federated Stochastic Gradient Descent Peer-to-Peer. These algorithms were applied to train deep neural network models for the detection and monitoring of diabetic foot ulcers, a critical health condition among diabetic patients. This study compares the performance of Federated Averaging Peer-to-Peer and Federated Stochastic Gradient Descent Peer-to-Peer with their centralized counterparts in terms of model convergence and communication costs. Additionally, we explore enhancements to these algorithms using targeted heuristics based on client identities and f1-scores for each class. The results indicate that models utilizing peer-to-peer federated averaging achieve a level of convergence that is comparable to that of models trained via conventional centralized federated learning approaches. This represents a notable progression in the field of ensuring the confidentiality and privacy of medical data for training machine learning models.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Privacy-Preserving Technologies in Data · Artificial Intelligence in Healthcare and Education · COVID-19 Digital Contact Tracing
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Department of Computer Science, XLIM, UMR CNRS 7252, University of Limoges, Avenue Albert Thomas, 87060 Limoges, France
Department of Applied Computer Science, Université des Mascareignes, Avenue de la Concorde, Roches Brunesl-Rose Hill 71259, Mauritius
3iL Ingénieurs, 43 Rue de Sainte Anne, 87015 Limoges, France
Faculty of Science and Technology, University of Limoges, 23 Avenue Albert Thomas, 87060 Limoges, France
Funding
XLIM, UMR CNRS
References (37)
Show source-provided references
1. Nasser, A.R., Hasan, A.M., Humaidi, A.J., Alkhayyat, A., Alzubaidi, L., Fadhel, M.A., Santamaría, J., and Duan, Y. (2021). Iot and cloud computing in health-care: A new wearable device and cloud-based deep learning algorithm for monitoring of diabetes. Electronics, 10.10.3390/electronics10212719 ↗
2. IDF (2021). IDF Diabetes Atlas, International Diabetes Federation. [10th ed.]. Technical Report.
3. Dermatologist-level classification of skin cancer with deep neural networks10.1038/nature21056 ↗
4. Skin Cancer Detection Based on Deep Learning
5. Goodfellow, I., Bengio, Y., and Courville, A. (2016). Deep Learning, MIT Press.
6. Sun, C., Shrivastava, A., Singh, S., and Gupta, A. (2017, January 22–29). Revisiting unreasonable effectiveness of data in deep learning era. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.10.1109/ICCV.2017.97 ↗
7. Will artificial intelligence solve the human resource crisis in healthcare?
8. Artificial intelligence in healthcare10.1038/s41551-018-0305-z ↗
9. McMahan, B., Moore, E., Ramage, D., Hampson, S., and Agüera y Arcas, B. (2017, January 20–22). Communication-efficient learning of deep networks from decentralized data. Proceedings of the Artificial Intelligence and Statistics, Fort Lauderdale, FL, USA.
10. Advances and open problems in federated learning10.1561/2200000083 ↗
11. Federated learning: Challenges, methods, and future directions
12. Cola: Decentralized linear learning
13. Can decentralized algorithms outperform centralized algorithms? A case study for decentralized parallel stochastic gradient descent
14. Decentralized learning works: An empirical comparison of gossip learning and federated learning10.1016/j.jpdc.2020.10.006 ↗
15. Yap, M.H., Cassidy, B., and Kendrick, C. (2022). Diabetic Foot Ulcers Grand Challenge, Springer International Publishing.10.1007/978-3-030-94907-5 ↗
16. Yap, M.H., Cassidy, B., Pappachan, J.M., O’Shea, C., Gillespie, D., and Reeves, N.D. (2021, January 27–30). Analysis towards classification of infection and ischaemia of diabetic foot ulcers. Proceedings of the 2021 IEEE EMBS International Conference on Biomedical and Health Informatics (BHI), Athens, Greece.10.1109/BHI50953.2021.9508563 ↗
17. DFU-SIAM a Novel Diabetic Foot Ulcer Classification with Deep Learning10.1109/ACCESS.2023.3312531 ↗
18. Signature verification using a “siamese” time delay neural network
19. Chicco, D. (2021). Artificial Neural Networks, Springer.
20. One-shot learning of object categories10.1109/TPAMI.2006.79 ↗
21. Konevcny, J., McMahan, H.B., Yu, F.X., Richtarik, P., Suresh, A.T., and Bacon, D. (2016). Federated learning: Strategies for improving communication efficiency. arXiv.
22. Federated reinforcement learning: Techniques, applications, and open challenges
23. Sirohi, D., Kumar, N., Rana, P.S., Tanwar, S., Iqbal, R., and Hijjii, M. (2023). Artificial Intelligence Review, Springer.
24. Vanhaesebrouck, P., Bellet, A., and Tommasi, M. (2017, January 20–22). Decentralized collaborative learning of personalized models over networks. Proceedings of the Artificial Intelligence and Statistics, Fort Lauderdale, FL, USA.
25. Moshawrab, M., Adda, M., Bouzouane, A., Ibrahim, H., and Raad, A. (2023). Reviewing Federated Machine Learning and Its Use in Diseases Prediction. Sensors, 23.10.3390/s23042112 ↗
26. The internet of federated things10.1109/ACCESS.2021.3127448 ↗
27. Shokri, R., and Shmatikov, V. (2015, January 12–16). Privacy-preserving deep learning. Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security, Denver, CO, USA.10.1145/2810103.2813687 ↗
28. Decentralized federated averaging10.1109/TPAMI.2022.3196503 ↗
29. Asad, M., Moustafa, A., Ito, T., and Aslam, M. (2021, January 5–7). Evaluating the communication efficiency in federated learning algorithms. Proceedings of the 2021 IEEE 24th International Conference on Computer Supported Cooperative Work in Design (CSCWD), Dalian, China.10.1109/CSCWD49262.2021.9437738 ↗
30. Lin, T., Stich, S.U., Patel, K.K., and Jaggi, M. (2018). Don’t use large mini-batches, use local sgd. arXiv.
31. Roy, A.G., Siddiqui, S., Pölsterl, S., Navab, N., and Wachinger, C. (2019). Braintorrent: A peer-to-peer environment for decentralized federated learning. arXiv.
32. Distributed load forecasting using smart meter data: Federated learning with Recurrent Neural Networks10.1016/j.ijepes.2021.107669 ↗
33. Hadsell, R., Chopra, S., and LeCun, Y. (2006, January 17–22). Dimensionality reduction by learning an invariant mapping. Proceedings of the 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’06), New York, NY, USA.
34. Chaudhuri, K., and Salakhutdinov, R. (2019, January 10–15). EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks. Proceedings of the 36th International Conference on Machine Learning, Long Beach, CA, USA.
35. Meila, M., and Zhang, T. (2021, January 18–24). EfficientNetV2: Smaller Models and Faster Training. Proceedings of the 38th International Conference on Machine Learning, Online. Proceedings of Machine Learning Research.
36. Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., and Gelly, S. (2020). An image is worth 16x16 words: Transformers for image recognition at scale. arXiv.
37. Sculley, D., Holt, G., Golovin, D., Davydov, E., Phillips, T., Ebner, D., Chaudhary, V., Young, M., Crespo, J.F., and Dennison, D. (2015, January 7–12). Hidden Technical Debt in Machine Learning Systems. Proceedings of the 28th International Conference on Neural Information Processing Systems (NIPS’15), Montreal, QC, Canada.