A Hierarchical Federated Learning Algorithm Based on Time Aggregation in Edge Computing Environment
Source abstract
Abstract
Federated learning is currently a popular distributed machine learning solution that often experiences cumbersome communication processes and challenging model convergence in practical edge deployments due to the training nature of its model information interactions. The paper proposes a hierarchical federated learning algorithm called FedDyn to address these challenges. FedDyn uses dynamic weighting to limit the negative effects of local model parameters with high dispersion and speed-up convergence. Additionally, an efficient aggregation-based hierarchical federated learning algorithm is proposed to improve training efficiency. The waiting time is set at the edge layer, enabling edge aggregation within a specified time, while the central server waits for the arrival of all edge aggregation models before integrating them. Dynamic grouping weighted aggregation is implemented during aggregation based on the average obsolescence of local models in various batches. The proposed algorithm is tested on the MNIST and CIFAR-10 datasets and compared with the FedAVG algorithm. The results show that FedDyn can reduce the negative effects of non-independent and identically distributed (IID) data on the model and shorten the total training time by 30% under the same accuracy rate compared to FedAVG.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Privacy-Preserving Technologies in Data · Mobile Crowdsensing and Crowdsourcing · Privacy, Security, and Data Protection
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
College of Information Science and Engineering, Shenyang Ligong University, Shenyang 110159, China
References (25)
Show source-provided references
1. Konečný, J., McMahan, H.B., Ramage, D., and Richtárik, P. (2016). Federated Optimization: Distributed Machine Learning for On-Device Intelligence. arXiv.
2. Data privacy issue in Federated Learning Resolution using Block Chain10.21015/vtse.v9i4.726 ↗
3. Heterogeneity-aware fair federated learning10.1016/j.ins.2022.11.031 ↗
4. Communication-Efficient Federated Deep Learning with Layerwise Asynchronous Model Update and Temporally Weighted Aggregation10.1109/TNNLS.2019.2953131 ↗
5. Communication-Efficient Federated Learning for Wireless Edge Intelligence in IoT10.1109/JIOT.2019.2956615 ↗
6. Ternary Compression for Communication-Efficient Federated Learning10.1109/TNNLS.2020.3041185 ↗
7. Mobile Cloud Computing Models Security Issues: A Systematic Review10.1016/j.jnca.2021.103152 ↗
8. Federated learning on non-IID data: A survey10.1016/j.neucom.2021.07.098 ↗
9. Challenges and future directions of secure federated learning: A survey10.1007/s11704-021-0598-z ↗
10. A Survey on Federated Learning: Challenges and Applications10.1007/s13042-022-01647-y ↗
11. Towards Energy Efficient Federated Learning over 5G+ Mobile Devices10.1109/MWC.003.2100028 ↗
12. Nishio, T., and Yonetani, R. (2019, January 20–24). Client Selection for Federated Learning with Heterogeneous Resources in Mobile Edge. Proceedings of the 2019 IEEE International Conference on Communications, Shanghai, China.10.1109/ICC.2019.8761315 ↗
13. MHAT: An Efficient Model-heterogenous Aggregation Training Scheme for Federated Learning10.1016/j.ins.2021.01.046 ↗
14. Federated Optimization in Heterogeneous Networks
15. A Survey on Security and Privacy of Federated Learning10.1016/j.future.2020.10.007 ↗
16. Hanzely, F., and Richtárik, P. (2020). Federated Learning of a Mixture of Global and Local Models. arXiv.
17. Arivazhagan, M.G., Aggarwal, V., Singh, A.K., and Choudhary, S. (2019). Federated Learning with Personalization Layers. arXiv.
18. Silva, A., Metcalf, K., Apostoloff, N., and Theobald, B.J. (2022). FedEmbed: Personalized Private Federated Learning. arXiv.
19. Personalized Federated Learning for Intelligent IoT Applications: A Cloud-Edge Based Framework10.1109/OJCS.2020.2993259 ↗
20. Liu, L., Zhang, J., Song, S.H., and Letaief, K.B. (2020, January 7–11). Client-Edge-Cloud Hierarchical Federated Learning. Proceedings of the ICC 2020—2020 IEEE International Conference on Communications, Dublin, Ireland.10.1109/ICC40277.2020.9148862 ↗
21. Multi-user Shared Access (MUSA) Procedure for Device Discovery in D2D Communication10.1007/s11235-020-00716-3 ↗
22. Zhang, Z., Ma, S., Nie, J., Wu, Y., Yan, Q., Xu, X., and Niyato, D. (2021, January 20–22). Semi-Supervised Federated Learning with non-IID Data: Algorithm and System Design. Proceedings of the 2021 IEEE 23rd Int Conf on High Performance Computing & Communications, Haikou, China.10.1109/HPCC-DSS-SmartCity-DependSys53884.2021.00047 ↗
23. Low Precision Decentralized Distributed Training over IID and non-IID Data10.1016/j.neunet.2022.08.032 ↗
24. Hsu, T.M.H., Qi, H., and Brown, M. (2020, January 23–28). Federated Visual Classification with Real-World Data Distribution. Proceedings of the Computer Vision—ECCV 2020: 16th European Conference, Glasgow, UK. Proceedings, Part X 16.10.1007/978-3-030-58607-2_5 ↗
25. Xin, B., Yang, W., Geng, Y., Chen, S., Wang, S., and Huang, L. (2020, January 4–8). Private FL-GAN: Differential Privacy Synthetic Data Generation Based on Federated Learning. Proceedings of the 2020 IEEE International Conference on Acoustics, Speech and Signal Processing, Barcelona, Spain.10.1109/ICASSP40776.2020.9054559 ↗