A Modified Quad Q Network Algorithm for Predicting Resource Management
Source abstract
Abstract
As the resource management systems continues to grow, the resource distribution system is expected to expand steadily. The demand response system enables producers to reduce the consumption costs of an enterprise during fluctuating periods in order balance the supply grid and resell the remaining resources of the product to generate revenue. Q-learning, a reinforcement learning algorithm based on a resource distribution compensation mechanism, is used to make optimal decisions to schedule the operation of smart factory appliances. In this paper, we proposed an effective resource management system for enterprise demand response using a Quad Q Network algorithm. The proposed algorithm is based on a Deep Q Network algorithm that directly integrates supply-demand inputs into control logic and employs fuzzy inference as a reward mechanism. In addition to using uses the Compare Optimizer method to reduce the loss value of the proposed Q Network Algorithm, Quad Q Network also maintains a high accuracy with fewer epochs. The proposed algorithm was applied to market capitalization data obtained from Google and Apple. Also, we verified that the Compare Optimizer used in Quad Q Network derives the minimum loss value through the double operation of Double Q value.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: IoT and Edge/Fog Computing · Smart Grid Energy Management · Data Stream Mining Techniques
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Department of ICT Convergence System Engineering, Chonnam National University, 77, Yongbong-ro, Buk-gu, Gwangju 500-757, Korea
Team of Energy Platform, Digital Transformation Department, Korea Electric Power Corporation (KEPCO), 55, Jeollyeok-ro, Naju, Jeollanam-do 58322, Korea
Interdisciplinary Program of Digital Future Convergence Service, Chonnam National University, 77, Yongbong-ro, Buk-gu, Gwangju 500-757, Korea
Funding
IITP · Ministry of Education(MOE, Korea) and National Research Foundation of Korea(NRF)
References (37)
Show source-provided references
1. Demand Response Strategy Based on Reinforcement Learning and Fuzzy Reasoning for Home Energy Management10.1109/ACCESS.2020.2974286 ↗
2. Convergence of Blockchain and Edge Computing for Secure and Scalable IIoT Critical Infrastructures in Industry 4.010.1109/JIOT.2020.3025916 ↗
3. Vaio, A.D., Boccia, F., Landriani, L., and Palladino, R. (2020). Artificial Intelligence in the Agri-Food System: Rethinking Sustainable Business Models in the COVID-19 Scenario. Sustainability, 12.10.3390/su12124851 ↗
4. Agbehadji, I.E., Awuzie, B.O., Ngowi, A.B., and Millham, R. (2020). Review of Big Data Analytics, Artificial Intelligence and Nature-Inspired Computing Models towards Accurate Detection of COVID-19 Pandemic Cases and Contact Tracing. Int. J. Environ. Res. Public Health, 17.10.3390/ijerph17155330 ↗
5. Lee, S., and Choi, D.-H. (2020). Energy management of smart home with appliances, energy storage system and electric vehicle: A hierarchical deep reinforcement learning approach. Sensors, 20.10.3390/s20072157 ↗
6. Athanassopoulos, E., and Voskoglou, M.G. (2020). A philosophical treatise on the connection of scientific reasoning with fuzzy logic. Mathematics, 8.10.3390/math8060875 ↗
7. Chen, C.-H., Jeng, S.-Y., and Lin, C.-J. (2020). Mobile Robot Wall-Following Control Using Fuzzy Logic Controller with Improved Differential Search and Reinforcement Learning. Mathematics, 8.10.3390/math8081254 ↗
8. Rustum, R., Kurichiyanil, A.M.J., Forrest, S., Sommariva, C., Adeloye, A.J., Zounemat-Kermani, M., and Scholz, M. (2020). Sustainability Ranking of Desalination Plants Using Mamdani Fuzzy Logic Inference Systems. Sustainability, 12.10.3390/su12020631 ↗
9. Gowida, A., Elkatatny, S., Al-Afnan, S., and Abdulraheem, A. (2020). New computational artificial intelligence models for generating synthetic formation bulk density logs while drilling. Sustainability, 12.10.3390/su12020686 ↗
10. Q-Learning-Based Fuzzy Logic for Multi-objective Routing Algorithm in Flying Ad Hoc Networks10.1007/s11277-020-07181-w ↗
11. Chen, S., Lin, T., Jheng, K., and Wu, C. (2020). Application of Fuzzy Theory and Optimum Computing to the Obstacle Avoidance Control of Unmanned Underwater Vehicles. Appl. Sci., 10.10.3390/app10176105 ↗
12. Gao, H., Ran, L.G., Wei, G.W., Wei, C., and Wu, J. (2020). VIKOR Method for MAGDM Based on Q-Rung Interval-Valued Orthopair Fuzzy Information and Its Application to Supplier Selection of Medical Consumption Products. Int. J. Environ. Res. Public Health, 17.10.3390/ijerph17020525 ↗
13. Bylykbashi, K., Qafzezi, E., Ampririt, P., Ikeda, M., Matsuo, K., and Barolli, L. (2020). Performance Evaluation of an Integrated Fuzzy-Based Driving-Support System for Real-Time Risk Management in VANETs. Sensors, 20.10.3390/s20226537 ↗
14. Alshorman, Omar Alshorman. Fuzzy-Based Fault-Tolerant Control for Omnidirectional Mobile Robot10.3390/machines8030055 ↗
15. Chen, L., Hu, X., Tang, B., and Cheng, Y. (2020). Conditional DQN-based motion planning with fuzzy logic for autonomous driving. IEEE Trans. Intell. Transp. Syst., 1–12.
16. Deep reinforcement learning for dynamic spectrum sensing and aggregation in multi-channel wireless networks10.1109/TCCN.2020.2982895 ↗
17. The Application of deep reinforcement learning to distributed spectrum access in dynamic heterogeneous environments with partial observations10.1109/TWC.2020.2984227 ↗
18. Spectrum access in cognitive radio using a two-stage reinforcement learning approach10.1109/JSTSP.2018.2798920 ↗
19. Applications of deep reinforcement learning in communications and networking: A Survey10.1109/COMST.2019.2916583 ↗
20. Wang, S., Liu, H., Gomes, P.H., and Krishnamachari, B. (2017, January 26–29). Deep reinforcement learning for dynamic multichannel access. Proceedings of the International Conference on Computing, Networking and Communications (ICNC), Silicon Valley, CA, USA.
21. Schaul, T., Quan, J., Antonoglou, I., and Silver, D. (2016). Prioritized experienc replay. arXiv.
22. Ye, H., and Li, G.Y. (2018, January 20–24). Deep reinforcement learning for resource allocation in V2V communications. Proceedings of the 2018 IEEE International Conference on Communications (ICC), Kansas City, MO, USA.10.1109/ICC.2018.8422586 ↗
23. Deep reinforcement learning based dynamic channel allocation algorithm in multibeam satellite systems10.1109/ACCESS.2018.2809581 ↗
24. Deep reinforcement learning based spectrum resource management for industrial internet of things10.1109/JIOT.2020.3022861 ↗
25. A new deep Q-learningbased transmission scheduling mechanism for the cognitive Internet of Things10.1109/JIOT.2017.2759728 ↗
26. Deep reinforcement learning for dynamic multichannel access in wireless networks10.1109/TCCN.2018.2809722 ↗
27. Zhong, C., Lu, Z., Gursoy, M.C., and Velipasalar, S. (2018, January 26–29). Actor-Critic deep reinforcement learning for dynamic multichannel access. Proceedings of the 2018 IEEE Global Conference on Signal and Information Processing (GlobalSIP), Anaheim, CA, USA.10.1109/GlobalSIP.2018.8646405 ↗
28. Distributive dynamic spectrum access through deep reinforcement learning: A reservoir computing-based approach10.1109/JIOT.2018.2872441 ↗
29. Naparstek, O., and Kobi, C. (2017, January 4–8). Deep multi-user reinforcement learning for dynamic spectrum access in multichannel wireless networks. Proceedings of the Globecom 2017—2017 IEEE Global Communications Conference, Singapore.10.1109/GLOCOM.2017.8254101 ↗
30. Q-learning based spectrum access for multimedia transmission over cognitive radio networks10.1109/TCCN.2020.3027297 ↗
31. Aref, M.A., Jayaweera, S.K., and Machuzak, S. (2017, January 19–22). Multi-agent reinforcement learning based cognitive anti-jamming. Proceedings of the 2017 IEEEWireless Communications and Networking Conference (WCNC), San Francisco, CA, USA.10.1109/WCNC.2017.7925694 ↗
32. Multi-agent deep reinforcement learning-based cooperative spectrum sensing with upper confidence bound exploration10.1109/ACCESS.2019.2937108 ↗
33. Nabipour, M., Nayyeri, P., Jabani, H., Mosavi, A., Salwana, E., and Shahab, S. (2020). Deep Learning for Stock Market Prediction. Entropy, 22.10.20944/preprints202003.0256.v1 ↗
34. Gu, J., Fang, Y., Sheng, Z., and Wen, P. (2020). Double Deep Q-Network with a Dual-Agent for Traffic Signal Control. Appl. Sci., 10.10.3390/app10051622 ↗
35. Polvara, R., Patacchiola, M., Hanheide, M., and Neumann, G. (2020). Sim-to-Real quadrotor landing via sequential deep Q-Networks and domain randomization. Robotics, 9.10.3390/robotics9010008 ↗
36. Sun, Y., Ran, X., Zhang, G., Xu, H., and Wang, X. (2020). AUV 3D Path Planning Based on the Improved Hierarchical Deep Q Network. J. Mar. Sci. Eng., 8.10.3390/jmse8020145 ↗
37. Li, D., Xu, S., and Li, P. (2021). Deep reinforcement learning-empowered resource allocation for mobile edge computing in cellular v2x networks. Sensors, 21.10.3390/s21020372 ↗