A Novel Deep Reinforcement Learning Approach for Task Offloading in MEC Systems
Source abstract
Abstract
With the internet developing rapidly, mobile edge computing (MEC) has been proposed to offer computational capabilities to tackle the high latency caused by innumerable data and applications. Due to limited computing resources, the innovation of computation offloading technology for an MEC system remains challenging, and can lead to transmission delays and energy consumption. This paper focuses on a task-offloading scheme for an MEC-based system where each mobile device is an independent agent and responsible for making a schedule based on delay-sensitive tasks. Nevertheless, the time-varying network dynamics and the heterogeneous features of real-time data tasks make it difficult to find an optimal solution for task offloading. Existing centralized-based or distributed-based algorithms require huge computational resources for complex problems. To address the above problem, we design a novel deep reinforcement learning (DRL)-based approach by using a parameterized indexed value function for value estimation. Additionally, the task-offloading problem is simulated as a Markov decision process (MDP) and our aim is to reduce the total delay of data processing. Experimental results have shown that our algorithm significantly promotes the users’ offloading performance over traditional methods.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: IoT and Edge/Fog Computing · Age of Information Optimization · Molecular Communication and Nanonetworks
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
College of Photonic and Electronic Engineering, Fujian Normal University, Fuzhou 350007, China
School of Information and Communication Engineering, Beijing University of Posts and Telecommunications, Beijing 100876, China
Funding
National Natural Science Foundation of China · Special Funds of the Central Government Guiding Local Science and Technology Development · Key provincial scientific and technological innovation projects · Natural Science Foundation of Fujian Province, China
References (34)
Show source-provided references
1. Toward a unified elastic computing platform for smartphones with cloud support10.1109/MNET.2013.6616113 ↗
2. Satyanarayanan, M. (2010, January 15). Mobile computing: The next decade. Proceedings of the 1st ACM Workshop on Mobile Cloud Computing and Services: Social Networks and Beyond, San Francisco, CA, USA.10.1145/1810931.1810936 ↗
3. Task offloading and resource allocation for mobile edge computing by deep reinforcement learning based on SARSA10.1109/ACCESS.2020.2981434 ↗
4. Kim, N., Lee, Y., Lee, C., Nguyen, T.V., and Cho, S. (2020, January 21–23). GPU-specific Task Offloading in the Mobile Edge Computing Network. Proceedings of the 2020 International Conference on Information and Communication Technology Convergence (ICTC), Jeju, Korea.10.1109/ICTC49870.2020.9289354 ↗
5. Liu, L., Ge, H., Li, S., Chen, X., Gong, H., and Cui, Y. (2021, January 24–26). Resource Allocation Strategy Based on Improved Auction Algorithm in Mobile Edge Computing Environment. Proceedings of the 2021 IEEE 6th International Conference on Cloud Computing and Big Data Analytics (ICCCBDA), Chengdu, China.10.1109/ICCCBDA51879.2021.9442523 ↗
6. Chen, S., Sun, S., Chen, H., Ruan, J., and Wang, Z.A. (October, January 30). Game Theoretic Approach to Task Offloading for Multi-Data-Source Tasks in Mobile Edge Computing. Proceedings of the 2021 IEEE International Conference on Parallel & Distributed Processing with Applications, Big Data & Cloud Computing, Sustainable Computing & Communications, Social Computing & Networking (ISPA/BDCloud/SocialCom/SustainCom), New York, NY, USA.10.1109/ISPA-BDCloud-SocialCom-SustainCom52081.2021.00111 ↗
7. Kim, K., Lynskey, J., Kang, S., and Hong, C.S. (2019, January 9–11). Prediction based sub-task offloading in mobile edge computing. Proceedings of the 2019 International Conference on Information Networking (ICOIN), Kuala Lumpur, Malaysia.10.1109/ICOIN.2019.8718183 ↗
8. Energy-optimal and delay-bounded computation offloading in mobile edge computing with heterogeneous clouds10.23919/JCC.2020.05.015 ↗
9. Deep reinforcement learning for dynamic computation offloading and resource allocation in cache-assisted mobile edge computing systems10.23919/ICN.2020.0014 ↗
10. Cheng, K., Teng, Y., Sun, W., Liu, A., and Wang, X. (2018, January 20–24). Energy-efficient joint offloading and wireless resource allocation strategy in multi-MEC server systems. Proceedings of the 2018 IEEE International Conference on Communications (ICC), Kansas City, MO, USA.10.1109/ICC.2018.8422877 ↗
11. QoS-aware robotic streaming workflow allocation in cloud robotics systems10.1109/TSC.2018.2803826 ↗
12. Joint optimization of radio and computational resources allocation in blockchain-enabled mobile edge computing systems10.1109/TWC.2020.2982627 ↗
13. Zhang, K., Cao, J., Leng, S., Shao, C., and Zhang, Y. (2019, January 9–11). Mining task offloading in mobile edge computing empowered blockchain. Proceedings of the 2019 IEEE International Conference on Smart Internet of Things (SmartIoT), Tianjin, China.10.1109/SmartIoT.2019.00043 ↗
14. Distributed Task Offloading Optimization with Queueing Dynamics in Multiagent Mobile-Edge Computing Networks10.1109/JIOT.2021.3063509 ↗
15. Computation offloading with instantaneous load billing for mobile edge computing10.1109/TSC.2020.2996764 ↗
16. QoS driven task offloading with statistical guarantee in mobile edge computing
17. Online Offloading Scheduling for Noma-aided Mec under Partial Device Knowledge10.1109/JIOT.2021.3091849 ↗
18. Mubeen, A., Ibrahim, M., Bibi, N., Baz, M., Hamam, H., and Cheikhrouhou, O. (2021). Alts: An Adaptive Load Balanced Task Scheduling Approach for Cloud Computing. Processes, 9.10.3390/pr9091514 ↗
19. Nabi, S., Ahmad, M., Ibrahim, M., and Hamam, H. (2022). AdPSO: Adaptive PSO-based task scheduling approach for cloud computing. Sensors, 22.10.3390/s22030920 ↗
20. Space/aerial-assisted computing offloading for IoT applications: A learning-based approach10.1109/JSAC.2019.2906789 ↗
21. Messaoudi, F., Ksentini, A., and Bertin, P. (2017, January 4–8). On using edge computing for computation offloading in mobile network. Proceedings of the GLOBECOM 2017—2017 IEEE Global Communications Conference, Singapore.10.1109/GLOCOM.2017.8254635 ↗
22. Online task scheduling and resource allocation for intelligent NOMA-based industrial Internet of Things10.1109/JSAC.2020.2980908 ↗
23. Efficient task scheduling for servers with dynamic states in vehicular edge computing10.1016/j.comcom.2019.11.019 ↗
24. Resource allocation with edge computing in IoT networks via machine learning10.1109/JIOT.2020.2970110 ↗
25. A DRL agent for jointly optimizing computation offloading and resource allocation in MEC10.1109/JIOT.2021.3081694 ↗
26. Gao, H., Wang, X., Ma, X., Wei, W., and Mumtaz, S. (2020). Com-DDPG: A multiagent reinforcement learning-based offloading strategy for mobile edge computing. arXiv.
27. Multiagent deep reinforcement learning for joint multichannel access and task offloading of mobile-edge computing in industry 4.010.1109/JIOT.2020.2968951 ↗
28. Wang, X., Ye, J., and Lui, J.C.S. (2022, January 2–5). Decentralized task offloading in edge computing: A multi-user multi-armed bandit approach. Proceedings of the IEEE INFOCOM 2022—IEEE Conference on Computer Communications, London, UK.10.1109/INFOCOM48880.2022.9796961 ↗
29. Tang, H., Wu, H., Qu, G., and Li, R. (2022). Double Deep Q-Network based Dynamic Framing Offloading in Vehicular Edge Computing. IEEE Trans. Netw. Sci. Eng.10.1109/TNSE.2022.3172794 ↗
30. Effective capacity-based resource allocation in mobile edge computing with two-stage tandem queues10.1109/TCOMM.2019.2920835 ↗
31. A distributed model-free ride-sharing approach for joint matching, pricing, and dispatching using deep reinforcement learning10.1109/TITS.2021.3096537 ↗
32. Deep reinforcement learning based computation offloading and resource allocation for low-latency fog radio access networks10.23919/ICN.2020.0020 ↗
33. A hybrid multiple access scheme via deep learning-based detection10.1109/JSYST.2020.2975666 ↗
34. UWB receiver via deep learning in MUI and ISI scenarios10.1109/TVT.2020.2972510 ↗