Federated Deep Reinforcement Learning for Energy-Efficient Edge Computing Offloading and Resource Allocation in Industrial Internet
Source abstract
Abstract
Industrial Internet mobile edge computing (MEC) deploys edge servers near base stations to bring computing resources to the edge of industrial networks to meet the energy-saving requirements of Industrial Internet terminal devices. This paper considers a wireless MEC system in an intelligent factory that has multiple edge servers and mobile smart industrial terminal devices. In this paper, the terminal device has the choice of either offloading the task in whole or in part to the edge server, or performing it locally. Through combined optimization of the task offload ratio, number of subcarriers, transmission power, and computing frequency, the system can achieve minimum total energy consumption. A computing offloading and resource allocation approach that combines federated learning (FL) and deep reinforcement learning (DRL) is suggested to address the optimization problem. According to the simulation results, the proposed algorithm displays fast convergence. Compared with baseline algorithms, this algorithm has significant advantages in optimizing the performance of energy consumption.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: IoT and Edge/Fog Computing · Age of Information Optimization · Privacy-Preserving Technologies in Data
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Key Laboratory of Modern Measurement and Control Technology, Ministry of Education, Beijing Information Science and Technology University, Beijing 100101, China
Funding
Beijing Natural Science Foundation · R&D Program of Beijing Municipal Education Commission
References (52)
Show source-provided references
1. The Industrial Internet of Things (IIoT) An Analysis Framework10.1016/j.compind.2018.04.015 ↗
2. On the Latency, Rate, and Reliability Tradeoff in Wireless Networked Control Systems for IIoT10.1109/JIOT.2020.3007070 ↗
3. Multi-Objective Computation Sharing in Energy and Delay Constrained Mobile Edge Computing Environments10.1109/TMC.2020.2994232 ↗
4. Edge Computing in Industrial Internet of Things: Architecture, Advances and Challenges10.1109/COMST.2020.3009103 ↗
5. A Survey on Mobile Edge Computing: The Communication Perspective10.1109/COMST.2017.2745201 ↗
6. Edge Intelligence: Paving the Last Mile of Artificial Intelligence with Edge Computing10.1109/JPROC.2019.2918951 ↗
7. Yang, L., Dai, Z., and Li, K. (2019, January 10–12). An Offloading Strategy Based on Cloud and Edge Computing for Industrial Internet. Proceedings of the 2019 IEEE 21st International Conference on High Performance Computing and Communications; IEEE 17th International Conference on Smart City; IEEE 5th International Conference on Data Science and Systems (HPCC/SmartCity/DSS), Zhangjiajie, China.10.1109/HPCC/SmartCity/DSS.2019.00228 ↗
8. Energy Aware Edge Computing: A Survey10.1016/j.comcom.2020.01.004 ↗
9. Partial Offloading Scheduling and Power Allocation for Mobile Edge Computing Systems10.1109/JIOT.2019.2911455 ↗
10. Blockchain at the Edge: Performance of Resource-Constrained IoT Network10.1109/TPDS.2020.3013892 ↗
11. Task Allocation Strategy for MEC-Enabled IIoTs via Bayesian Network Based Evolutionary Computation10.1109/TII.2020.3019572 ↗
12. Residual Energy Maximization-Based Resource Allocation in Wireless-Powered Edge Computing Industrial IoT10.1109/JIOT.2021.3082161 ↗
13. Edge Computing Resource Allocation Algorithm for NB-IoT Based on Deep Reinforcement Learning
14. Data Heterogeneity-Robust Federated Learning via Group Client Selection in Industrial IoT10.1109/JIOT.2022.3161943 ↗
15. Silver, D., Lever, G., Heess, N., Degris, T., Wierstra, D., and Riedmiller, M. (2014, January 21–26). Deterministic Policy Gradient Algorithms. Proceedings of the 31st International Conference on Machine Learning, Beijing, China.
16. Konečný, J., McMahan, H.B., Ramage, D., and Richtárik, P. (2016). Federated Optimization: Distributed Machine Learning for on-Device Intelligence. arXiv.
17. Shu, C., Zhao, Z., Han, Y., and Min, G. (2019, January 10–13). Dependency-Aware and Latency-Optimal Computation Offloading for Multi-User Edge Computing Networks. Proceedings of the 16th Annual IEEE International Conference on Sensing, Communication, and Networking (SECON), Boston, MA, USA.10.1109/SAHCN.2019.8824941 ↗
18. Joint Task Offloading and Payment Determination for Mobile Edge Computing: A Stable Matching Based Approach10.1109/TVT.2020.3013622 ↗
19. Joint Task Offloading and Resource Allocation for Multi-Server Mobile-Edge Computing Networks10.1109/TVT.2018.2881191 ↗
20. Computation Offloading and Resource Allocation For Cloud Assisted Mobile Edge Computing in Vehicular Networks10.1109/TVT.2019.2917890 ↗
21. Collaborative Cloud-Edge-End Task Offloading in Mobile-Edge Computing Networks with Limited Communication Capability10.1109/TCCN.2020.3018159 ↗
22. EIHDP: Edge-Intelligent Hierarchical Dynamic Pricing Based on Cloud-Edge-Client Collaboration for IoT Systems10.1109/TC.2021.3060484 ↗
23. Adaptive Multi-Cost Routing Protocol to Enhance Lifetime for Wireless Body Area Network
24. Energy-Efficient Resource Allocation for Mobile-Edge Computation Offloading10.1109/TWC.2016.2633522 ↗
25. Energy-Efficient Joint Task Offloading and Resource Allocation in OFDMA-Based Collaborative Edge Computing10.1109/TWC.2021.3108641 ↗
26. Deep Reinforcement Learning-Based Dynamic Resource Management for Mobile Edge Computing in Industrial Internet of Things10.1109/TII.2020.3028963 ↗
27. Reinforcement Learning-Based Mobile Offloading for Edge Computing Against Jamming and Interference10.1109/TCOMM.2020.3007742 ↗
28. Decentralized Computation Offloading for Multi-User Mobile Edge Computing: A Deep Reinforcement Learning Approach10.1186/s13638-020-01801-6 ↗
29. Offloading and Resource Allocation with General Task Graph in Mobile Edge Computing: A Deep Reinforcement Learning Approach10.1109/TWC.2020.2993071 ↗
30. Adaptive Computing Scheduling for Edge-Assisted Autonomous Driving10.1109/TVT.2021.3062653 ↗
31. A Task Offloading Algorithm with Cloud Edge Jointly Load Balance Optimization Based on Deep Reinforcement Learning for Unmanned Surface Vehicles10.1109/ACCESS.2022.3150406 ↗
32. Deep Reinforcement Learning for Dynamic Computation Offloading and Resource Allocation in Cache-Assisted Mobile Edge Computing Systems10.23919/ICN.2020.0014 ↗
33. Enhanced Online Q-Learning Scheme for Resource Allocation with Maximum Utility and Fairness in Edge-IoT Networks10.1109/TNSE.2020.3015689 ↗
34. Human-Level Control through Deep Reinforcement Learning10.1038/nature14236 ↗
35. Lillicrap, T.P., Hunt, J.J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., and Wierstra, D. (2015). Continuous Control with Deep Reinforcement Learning. arXiv.
36. McMahan, B., Moore, E., Ramage, D., Hampson, S., and Aguera y Arcas, B. (2017, January 20–22). Communication-Efficient Learning of Deep Networks from Decentralized Data. Proceedings of the 20th International Conference on Artificial Intelligence and Statistics, Fort Lauderdale, FL, USA.
37. Konečný, J., McMahan, H.B., Yu, F.X., Richtarik, P., Suresh, A.T., and Bacon, D. (2016). Federated Learning: Strategies for Improving Communication Efficiency. arXiv.
38. Yaqoob, M.M., Nazir, M., Yousafzai, A., Khan, M.A., Shaikh, A.A., Algarni, A.D., and Elmannai, H. (2022). Modified Artificial Bee Colony Based Feature Optimized Federated Learning for Heart Disease Diagnosis in Healthcare. Appl. Sci., 12.10.3390/app122312080 ↗
39. Yaqoob, M.M., Nazir, M., Khan, M.A., Qureshi, S., and Al-Rasheed, A. (2023). Hybrid Classifier-Based Federated Learning in Health Service Providers for Cardiovascular Disease Prediction. Appl. Sci., 13.10.3390/app13031911 ↗
40. When Deep Reinforcement Learning Meets Federated Learning: Intelligent Multitimescale Resource Management for Multiaccess Edge Computing in 5G Ultradense Network10.1109/JIOT.2020.3026589 ↗
41. Wang, H., Kaplan, Z., Niu, D., and Li, B. (2020, January 6–9). Optimizing Federated Learning on Non-IID Data with Reinforcement Learning. Proceedings of the IEEE INFOCOM 2020—IEEE Conference on Computer Communications, Toronto, ON, Canada.10.1109/INFOCOM41043.2020.9155494 ↗
42. Khodadadian, S., Sharma, P., Joshi, G., and Maguluri, S.T. (2022, January 17–23). Federated Reinforcement Learning: Linear Speedup Under Markovian Sampling. Proceedings of the 39th International Conference on Machine Learning, Baltimore, MD, USA. Available online: https://proceedings.mlr.press/v162/khodadadian22a/khodadadian22a.pdf.
43. Resource Allocation in IoT Edge Computing via Concurrent Federated Reinforcement Learning10.1109/JIOT.2021.3086910 ↗
44. HFEL: Joint Edge Association and Resource Allocation for Cost-Efficient Hierarchical Federated Edge Learning10.1109/TWC.2020.3003744 ↗
45. Federated Multiagent Actor–Critic Learning for Age Sensitive Mobile-Edge Computing10.1109/JIOT.2021.3078514 ↗
46. Liu, L., Zhang, J., Song, S.H., and Letaief, K.B. (2020, January 7–11). Client-Edge-Cloud Hierarchical Federated Learning. Proceedings of the ICC 2020—2020 IEEE International Conference on Communications (ICC), Dublin, Ireland.10.1109/ICC40277.2020.9148862 ↗
47. Wireless Edge Computing with Latency and Reliability Guarantees10.1109/JPROC.2019.2917084 ↗
48. Applications of Deep Reinforcement Learning in Communications and Networking: A Survey10.1109/COMST.2019.2916583 ↗
49. A Bilevel Optimization Approach for Joint Offloading Decision and Resource Allocation in Cooperative Mobile Edge Computing10.1109/TCYB.2019.2916728 ↗
50. (2023, February 24). White Paper on Edge Computing Network of Industrial Internet in the 5G Era. Available online: http://www.ecconsortium.org/Uploads/file/20201209/1607521755435690.pdf.
51. Smart Resource Allocation for Mobile Edge Computing: A Deep Reinforcement Learning Approach10.1109/TETC.2019.2902661 ↗
52. Liu, K.-H., Hsu, Y.-H., Lin, W.-N., and Liao, W. (April, January 29). Fine-Grained Offloading for Multi-Access Edge Computing with Actor-Critic Federated Learning. Proceedings of the IEEE Wireless Communications and Networking Conference (WCNC), Nanjing, China.