journal article·2025-04-25Open access

Graph-Driven Deep Reinforcement Learning for Vehicle Routing Problems with Pickup and Delivery

Dapeng Yan, Qingshu Guan, Bei Ou, Bowen Yan, Hui Cao

Applied Sciences· Volume 15· Issue 9

Abstract

Recently, the vehicle routing problem with pickup and delivery (VRP-PD) has attracted increasing interest due to its widespread applications in real-life logistics and transportation. However, existing learning-based methods often fail to fully exploit hierarchical graph structures, leading to suboptimal performance. In this study, we propose a graph-driven deep reinforcement learning (GDRL) approach that employs an encoder–decoder framework to address this shortcoming. The encoder incorporates stacked graph convolution modules (GCMs) to aggregate neighborhood information via updated edge features, producing enriched node representations for subsequent decision-making. The single-head attention decoder then applies a computationally efficient compatibility layer to sequentially determine the next node to visit. Extensive experiments demonstrate that the proposed GDRL achieves superior performance over both heuristic and learning-based baselines, reducing route length by up to 5.81% across synthetic and real-world datasets. Furthermore, GDRL also exhibits strong generalization capability across diverse problem scales and node distributions, highlighting its potential for real-world deployment.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: Vehicle Routing Optimization Methods · Assembly Line Balancing Optimization · Advanced Manufacturing and Logistics Optimization

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

School of Electrical Engineering, Xi’an Jiaotong University, Xi’an 710049, China

State Key Laboratory of Electrical Insulation and Power Equipment, Xi’an Jiaotong University, Xi’an 710049, China

Funding

National Natural Science Foundation of China · Natural Science Basic Research Program of Shaanxi Province · State Key Laboratory of Electrical Insulation and Power Equipment

References (36)

Show source-provided references
1. A multi-visit flexible-docking vehicle routing problem with drones for simultaneous pickup and delivery services10.1016/j.ejor.2023.06.021 ↗
2. Ferroelectric compute-in-memory annealer for combinatorial optimization problems10.1038/s41467-024-46640-x ↗
3. Guan, Q., Cao, H., Zhong, X., Yan, D., and Xue, S. (2025). Hisom: Hierarchical Self-Organizing Map for Solving Multiple Traveling Salesman Problems. Networks.10.1002/net.22268 ↗
4. Pickup and delivery problem with electric vehicles and time windows considering queues10.1016/j.trc.2024.104829 ↗
5. Synergetic attention-driven transformer: A Deep reinforcement learning approach for vehicle routing problems10.1016/j.eswa.2025.126961 ↗
6. Double-deck multi-agent pickup and delivery: Multi-robot rearrangement in large-scale warehouses10.1109/LRA.2023.3272272 ↗
7. Centralized deep reinforcement learning method for dynamic multi-vehicle pickup and delivery problem with crowdshippers10.1109/TITS.2024.3352143 ↗
8. Guan, Q., Hong, X., Ke, W., Zhang, L., Sun, G., and Gong, Y. (October, January 27). Kohonen self-organizing map based route planning: A revisit. Proceedings of the 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Prague, Czech Republic.10.1109/IROS51168.2021.9636025 ↗
9. Planning of truck platooning for road-network capacitated vehicle routing problem10.1016/j.tre.2024.103898 ↗
10. A branch-and-cut algorithm for the time-dependent vehicle routing problem with time windows and combinatorial auctions10.1016/j.cor.2024.106807 ↗
11. A lightweight genetic algorithm with variable neighborhood search for multi-depot vehicle routing problem with time windows10.1016/j.asoc.2024.111789 ↗
12. Ma, X., and Liu, C. (2024). Improved Ant Colony Algorithm for the Split Delivery Vehicle Routing Problem. Appl. Sci., 14.10.3390/app14125090 ↗
13. A new hyper-heuristic based on adaptive simulated annealing and reinforcement learning for the capacitated electric vehicle routing problem10.1016/j.eswa.2024.124197 ↗
14. Deep reinforcement learning based energy management strategies for electrified vehicles: Recent advances and perspectives10.1016/j.rser.2023.114248 ↗
15. High-efficiency reinforcement learning with hybrid architecture photonic integrated circuit10.1038/s41467-024-45305-z ↗
16. Token-based deep reinforcement learning for Heterogeneous VRP with Service Time Constraints10.1016/j.knosys.2024.112173 ↗
17. Machine learning to solve vehicle routing problems: A survey10.1109/TITS.2023.3334976 ↗
18. An exact algorithm for the pickup and delivery problem with time windows10.1287/opre.1100.0881 ↗
19. A branch-and-price-and-cut algorithm for a pickup and delivery problem in retailing10.1016/j.omega.2018.09.014 ↗
20. A branch-and-cut algorithm for the split-demand one-commodity pickup-and-delivery travelling salesman problem10.1016/j.ejor.2021.05.040 ↗
21. The location-routing problem with simultaneous pickup and delivery: Formulations and a heuristic approach10.1016/j.omega.2011.09.002 ↗
22. A hyper heuristic for the green vehicle routing problem with simultaneous pickup and delivery10.1016/j.cie.2020.107010 ↗
23. An extension of the Lin-Kernighan-Helsgaun TSP solver for constrained traveling salesman and vehicle routing problems
24. Kool, W., van Hoof, H., and Welling, M. (2019, January 6–9). Attention, Learn to Solve Routing Problems!. Proceedings of the International Conference on Learning Representations, New Orleans, LA, USA.
25. Pomo: Policy optimization with multiple optima for reinforcement learning
26. Learning feature embedding refiner for solving vehicle routing problems10.1109/TNNLS.2023.3285077 ↗
27. Meta-learning-based deep reinforcement learning for multiobjective optimization problems10.1109/TNNLS.2022.3148435 ↗
28. Self-organizing feature maps for the vehicle routing problem with backhauls10.1007/s10951-006-6774-z ↗
29. Deep reinforcement learning for solving vehicle routing problems with backhauls10.1109/TNNLS.2024.3371781 ↗
30. On the exact solution of vehicle routing problems with backhauls10.1016/j.ejor.2020.04.047 ↗
31. Nickel, S., Steinhardt, C., Schlenker, H., Burkart, W., and Reuter-Oppermann, M. (2021). Ibm ilog cplex optimization studio. Angewandte Optimierung mit IBM ILOG CPLEX Optimization Studio: Modellierung von Planungs-und Entscheidungsproblemen des Operations Research mit OPL, Springer.10.1007/978-3-662-62185-1 ↗
32. A genetic algorithm based approach to vehicle routing problem with simultaneous pick-up and deliveries10.1016/j.cie.2011.11.025 ↗
33. A parallel simulated annealing method for the vehicle routing problem with simultaneous pickup–delivery and time windows10.1016/j.cie.2015.02.005 ↗
34. Solving the vehicle routing problem with simultaneous pickup and delivery by an effective ant colony optimization
35. An adaptive large neighborhood search heuristic for solving the reliable multiple allocation hub location problem under hub disruptions
36. Achamrah, F.E. (2022). Modelling and Solving Complex Vehicle Routing Problems with Integrated Management of Shared Inventories in Supply Chains. [Ph.D. Thesis, Université Paris-Saclay].