A Two-Layer User Energy Management Strategy for Virtual Power Plants Based on HG-Multi-Agent Reinforcement Learning

Household loads are becoming dominant in virtual power plants (VPP). However, their dispatch potential has not yet been explored due to the lack of detailed user power management. To solve this issue, a novel two-layer user energy management strategy based on HG-multi-agent reinforcement learning has been proposed in this paper. Firstly, a novel two-layer optimization framework is established, where the upper layer is applied to coordinate the scheduling and benefit allocation among various stakeholders and the lower layer is applied to execute intelligent decision-making for users. Secondly, the mathematical model for the framework is established, where a detailed household power management model is proposed in the lower layer, and the generated predicted power demands are used to replace the conventional aggregate model in the upper layer. As a result, the energy consumption behaviors of household users can be precisely described in the scheduling scheme. Furthermore, an HG-multi-agent reinforcement-based method is applied to accelerate the game-solving process. Case study results indicate that the proposed method leads to a reduction in user costs and an increase in VPP profit.