Research on Energy-Saving and Efficiency-Improving Optimization of a Four-Way Shuttle-Based Dense Three-Dimensional Warehouse System Based on Two-Stage Deep Reinforcement Learning

In the context of rapid development within the logistics sector and widespread advocacy for sustainable development, this paper proposes enhancements to the task scheduling and path planning components of four-way shuttle systems. The focus lies on refining and innovating modeling approaches and algorithms to address issues in complex environments such as uneven task distribution, poor adaptability to dynamic conditions, and high rates of idle vehicle operation. These improvements aim to enhance system performance, reduce energy consumption, and achieve sustainable development. Therefore, this paper presents an energy-saving and efficiency-enhancing optimization study for a four-way shuttle-based high-density automated warehouse system, utilizing deep reinforcement learning. In terms of task scheduling, a collaborative scheduling algorithm based on an Improved Genetic Algorithm (IGA) and Multi-Agent Deep Deterministic Policy Gradient (MADDPG) has been designed. In terms of path planning, this paper provides the A*-DQN method, which integrates the A* algorithm(A*) with Deep Q-Networks (DQN). Through combining multiple layout scenarios and adjusting various parameters, simulation experiments verified that the system error is within 5% or less. Compared to existing methods, the total task duration, path planning length, and energy consumption per order decreased by approximately 12.84%, 9.05%, and 16.68%, respectively. The four-way shuttle vehicle can complete order tasks with virtually no conflicts. The conclusions of this paper have been validated through simulation experiments.