Sensor-Generated In Situ Data Management for Smart Grids: Dynamic Optimization Driven by Double Deep Q-Network with Prioritized Experience Replay
Source abstract
Abstract
According to forecast data from the State Grid Corporation of China, the number of terminal devices connected to the power grid is expected to reach the scale of 2 billion within the next five years. With the continuous growth in the number and variety of terminal devices in the smart grid, traditional cloud-edge-end architecture will face the increasing issue of response latency. In this context, in situ computing, as a paradigm for local or near-source data processing within cloud-edge-end architecture, is gradually becoming a key technological pathway in industrial systems. The in situ server system, by deploying servers near terminals, enables the near-data-source processing of terminal-generated in situ data, representing an important implementation of in situ computing. To enhance the processing efficiency and response capability of in situ data in smart grid scenarios, this study designs an in situ data processing mechanism and an access demand management framework. Due to the heterogeneity of in situ server performance, there are variations in response capabilities for access demands across rounds. This study introduces a double deep Q-network with prioritized experience replay to assist in response decision-making. Simulation experiments show that the proposed method reduces waiting latency and response latency by an average of 67.69% and 68.77%, respectively, compared to traditional algorithms and other reinforcement learning algorithms, verifying its effectiveness in in situ data management. This scheme can also be widely applied to in situ computing scenarios with low-latency data management, such as smart cities and industrial IoT.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: IoT and Edge/Fog Computing · Smart Grid Security and Resilience · Data Stream Mining Techniques
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Qingdao Institute of Software, College of Computer Science and Technology, China University of Petroleum (East China), Qingdao 266580, China
Shandong Key Laboratory of Intelligent Oil & Gas Industrial Software, Qingdao 266580, China
School of Management, Beijing Union University, Beijing 100101, China
State Key Laboratory of Media Convergence and Communication, Communication University of China, Beijing 100024, China
Funding
Natural Science Foundation of Shandong Province · National Natural Science Foundation of China · Beijing Social Science Foundation Program · open research subject of the State Key Laboratory of Intelligent Game · Key Laboratory of Ethnic Language Intelligent Analysis and Security Governance of MOE · Teaching Reform Project of the Beijing Union University “Research on the Construction of an Innovation and Entrepreneurship Education Ecosystem for E-commerce Majors Empowered by Digital Intelligence”
References (31)
Show source-provided references
1. Meta-RHDC: Meta Reinforcement Learning Driven Hybrid Lyrebird Falcon Optimization for Dynamic Load Balancing in Cloud Computing10.1109/ACCESS.2025.3544775 ↗
2. Cheng, Z., Zhang, L., Hao, Z., Qu, S., Liu, J., Ding, X., Liu, C., Li, N., and Li, R. (2024, January 4–7). Assisting Offshore Production Optimization Technology in Intelligent Completion Operation Based on Edge-Cloud Collaborative Technologies. Proceedings of the Abu Dhabi International Petroleum Exhibition and Conference, Abu Dhabi, United Arab Emirates.10.2118/222157-MS ↗
3. Li, J., Xia, Y., Sun, X., Chen, P., Li, X., and Feng, J. (2024, January 7–13). Delay-Aware Service Caching in Edge Cloud: An Adversarial Semi-Bandits Learning-Based Approach. Proceedings of the 2024 IEEE 17th International Conference on Cloud Computing (CLOUD), Shenzhen, China.10.1109/CLOUD62652.2024.00053 ↗
4. A thermostatically controlled loads regulation method based on hybrid communication and cloud–edge–end collaboration10.1016/j.egyr.2024.12.010 ↗
5. Multi-Dimensional Optimization for Collaborative Task Scheduling in Cloud-Edge-End System10.1016/j.simpat.2025.103099 ↗
6. Xiao, L., Jia, G., and Lin, X. (2024, January 26–28). Construction of Manufacturing Data Acquisition System Based on Edge Computing. Proceedings of the 2024 International Conference on Computers, Information Processing and Advanced Education (CIPAE), Ottawa, ON, Canada.10.1109/CIPAE64326.2024.00110 ↗
7. Wan, Z., Zhao, S., Dong, X., Huang, Z., and Tan, Y. (2024, January 10–14). Joint DRL and ASL-Based “Cloud-Edge-End” Collaborative Caching Optimization for Metaverse Scenarios. Proceedings of the 2024 IEEE 30th International Conference on Parallel and Distributed Systems (ICPADS), Belgrade, Serbia.10.1109/ICPADS63350.2024.00052 ↗
8. Role of machine learning approach for industrial internet of things (IIoT) in cloud environment-a systematic review
9. Tang, Y. (2022, January 16–18). Tiansuan Constellation: Intelligent Software-Defined Microsatellite with Orbital Attention for Sustainable Development Goals. Proceedings of the International Conference on Big Data Intelligence and Computing, Shanghai, China.10.2139/ssrn.4267675 ↗
10. Edge Graph Intelligence: Reciprocally Empowering Edge Networks with Graph Intelligence10.1109/COMST.2025.3527561 ↗
11. Cloud-edge orchestration-based bi-level autonomous process control for mass individualization of rapid printed circuit boards prototyping services10.1016/j.jmsy.2022.03.008 ↗
12. Multi source data security protection of smart grid based on edge computing10.1016/j.measen.2024.101288 ↗
13. Hong, H., Suo, Z., Wu, H., Li, D., Wang, J., Lu, H., Zhang, Y., and Lu, H. (2022, January 14–16). Large-scale heterogeneous terminal management technology for power Internet of Things platform. Proceedings of the 2022 2nd International Conference on Consumer Electronics and Computer Engineering (ICCECE), Guangzhou, China.10.1109/ICCECE54139.2022.9712797 ↗
14. He, X., Dong, H., Yang, W., and Li, W. (2023). Multi-source information fusion technology and its application in smart distribution power system. Sustainability, 15.10.3390/su15076170 ↗
15. Zeng, F., Pang, C., and Tang, H. (2024). Sensors on internet of things systems for the sustainable development of smart cities: A systematic literature review. Sensors, 24.10.3390/s24072074 ↗
16. Towards sustainable in-situ server systems in the big data era10.1145/2872887.2750381 ↗
17. Cisco (2025, May 15). Cisco Technology Radar Trends, 2014. White Paper. Available online: https://davidhoglund.typepad.com/files/tech-radar-trends-infographics.pdf.
18. Zhang, F., Lasluisa, S., Jin, T., Rodero, I., Bui, H., and Parashar, M. (2012, January 10–16). In situ feature-based objects tracking for large-scale scientific simulations. Proceedings of the 2012 SC Companion: High Performance Computing, Networking Storage and Analysis, Salt Lake City, UT, USA.10.1109/SC.Companion.2012.100 ↗
19. Lakshminarasimhan, S., Boyuka, D.A., Pendse, S.V., Zou, X., Jenkins, J., Vishwanath, V., Papka, M.E., and Samatova, N.F. (2013, January 17–21). Scalable in situ scientific data encoding for analytical query processing. Proceedings of the 22nd International Symposium on High-Performance Parallel and Distributed Computing, New York, NY, USA.10.1145/2493123.2465527 ↗
20. Park, J., Choi, S., Kim, J., Koo, G., Yoon, M.K., and Oh, Y. (ACM Trans. Storage, 2025). TM-Training: An Energy-Efficient Tiered Memory System for Deep Learning Training in NPUs, ACM Trans. Storage, in press.10.1145/3721484 ↗
21. Anjum, M., Kraiem, N., Min, H., Dutta, A.K., Daradkeh, Y.I., and Shahab, S. (2025). Opportunistic access control scheme for enhancing IoT-enabled healthcare security using blockchain and machine learning. Sci. Rep., 15.10.1038/s41598-025-90908-1 ↗
22. Guan, H., Yu, L., Zhou, L., Xiong, L., Chowdhury, K., Xie, L., Xiao, X., and Zou, J. (2025). Privacy and Accuracy-Aware AI/ML Model Deduplication. arXiv.
23. Yuan, Q., Pi, Y., Kou, L., Zhang, F., Li, Y., and Zhang, Z. (2022). Multi-source data processing and fusion method for power distribution internet of things based on edge intelligence. Front. Energy Res., 10.10.3389/fenrg.2022.891867 ↗
24. Zielosko, B., Jabloński, K., and Dmytrenko, A. (2025). Exploiting Data Distribution: A Multi-Ranking Approach. Entropy, 27.10.3390/e27030278 ↗
25. A survey on edge computing systems and tools10.1109/JPROC.2019.2920341 ↗
26. Towards edge computing in intelligent manufacturing: Past, present and future10.1016/j.jmsy.2022.01.010 ↗
27. Van Hasselt, H., Guez, A., and Silver, D. (2016, January 12–17). Deep reinforcement learning with double q-learning. Proceedings of the AAAI Conference on Artificial Intelligence, Phoenix, AZ, USA.10.1609/aaai.v30i1.10295 ↗
28. Schaul, T., Quan, J., Antonoglou, I., and Silver, D. (2015). Prioritized experience replay. arXiv.
29. Resource Allocation Based on Deep Reinforcement Learning in IoT Edge Computing10.1109/JSAC.2020.2986615 ↗
30. Melo, F.S. (2001). Convergence of Q-Learning: A Simple Proof, Institute of Systems and Robotics, University of Coimbra. Technical Report.
31. The measurement of observer agreement for categorical data10.2307/2529310 ↗