D*-KDDPG: An Improved DDPG Path-Planning Algorithm Integrating Kinematic Analysis and the D* Algorithm
Source abstract
Abstract
To address the limitations of the Deep Deterministic Policy Gradient (DDPG) in robot path planning, we propose an improved DDPG method that integrates kinematic analysis and D* algorithm, termed D*-KDDPG. Firstly, the current work promotes the reward function of DDPG to account for the robot’s kinematic characteristics and environment perception ability. Secondly, informed by the global path information provided by the D* algorithm, DDPG successfully avoids getting trapped in local optima within complex environments. Finally, a comprehensive set of simulation experiments is carried out to investigate the effectiveness of D*-KDDPG within various environments. Simulation results indicate that D*-KDDPG completes strategy learning within only 26.7% of the training steps required by the original DDPG, retrieving enhanced navigation performance and promoting safety. D*-KDDPG outperforms D*-DWA with better obstacle avoidance performance in dynamic environments. Despite a 1.8% longer path, D*-KDDPG reduces navigation time by 16.2%, increases safety distance by 72.1%, and produces smoother paths.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Robotic Path Planning Algorithms · Autonomous Vehicle Technology and Safety · Robotic Locomotion and Control
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
School of Mechatronics Engineering, Henan University of Science and Technology, Luoyang 471003, China
Longmen Laboratory, Luoyang 471000, China
Key Laboratory of Mechanical Design and Transmission System of Henan Province, Luoyang 471000, China
Collaborative Innovation Center of Machinery Equipment Advanced Manufacturing of Henan Province, Luoyang 471000, China
Funding
National Science Foundation of China · Technology Projects of Longmen Laboratory
References (18)
Show source-provided references
1. Survey of Deep Reinforcement Learning for Motion Planning of Autonomous Vehicles10.1109/TITS.2020.3024655 ↗
2. Immune Deep Reinforcement Learning-Based Path Planning for Mobile Robot in Unknown Environment10.1016/j.asoc.2023.110601 ↗
3. Lillicrap, T.P., Hunt, J.J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., and Wierstra, D. (2015). Continuous Control with Deep Reinforcement Learning. arXiv.
4. Zhang, H., Lin, W., and Chen, A. (2018). Path Planning for the Mobile Robot: A Review. Symmetry, 10.10.3390/sym10100450 ↗
5. Wang, X., Zhang, H., Liu, S., Wang, J., Wang, Y., and Shangguan, D. (2022). Path Planning of Scenic Spots Based on Improved A* Algorithm. Sci. Rep., 12.10.1038/s41598-022-05386-6 ↗
6. Jiang, C., Zhu, H., and Xie, Y. (2023, January 15–17). Dynamic Obstacle Avoidance Research for Mobile Robots Incorporating Improved A-Star Algorithm and DWA Algorithm. Proceedings of the 2023 3rd International Conference on Computer Science, Electronic Information Engineering and Intelligent Control Technology (CEI), Wuhan, China.10.1109/CEI60616.2023.10528067 ↗
7. Guan, C., and Wang, S. (2022, January 2). Robot Dynamic Path Planning Based on Improved A* and DWA Algorithms. Proceedings of the 2022 4th International Conference on Control and Robotics (ICCR), Guangzhou, China.10.1109/ICCR55715.2022.10053929 ↗
8. Research on Obstacle Avoidance of Mobile Robot Based on Improved DWA Algorithm
9. Watkins, C.J.C.H. (2024, July 25). Learning from Delayed Rewards. Available online: https://www.academia.edu/download/50360235/Learning_from_delayed_rewards_20161116-28282-v2pwvq.pdf.
10. Human-Level Control through Deep Reinforcement Learning10.1038/nature14236 ↗
11. Van Hasselt, H., Guez, A., and Silver, D. (2016, January 12–17). Deep Reinforcement Learning with Double Q-Learning. Proceedings of the AAAI Conference on Artificial Intelligence, Phoenix, AZ, USA.10.1609/aaai.v30i1.10295 ↗
12. Wang, Z., Schaul, T., Hessel, M., Hasselt, H., Lanctot, M., and Freitas, N. (2016, January 20–22). Dueling Network Architectures for Deep Reinforcement Learning. Proceedings of the International Conference on Machine Learning, New York, NY, USA.
13. Silver, D., Lever, G., Heess, N., Degris, T., Wierstra, D., and Riedmiller, M. (2014, January 21–26). Deterministic Policy Gradient Algorithms. Proceedings of the International Conference on Machine Learning, Beijing, China.
14. Multi-Objective Vehicle Following Decision Algorithm Based on Reinforcement Learning
15. Safe, Efficient, and Comfortable Velocity Control Based on Reinforcement Learning for Autonomous Driving10.1016/j.trc.2020.102662 ↗
16. Predictive Hierarchical Reinforcement Learning for Path-Efficient Mapless Navigation with Moving Target10.1016/j.neunet.2023.06.007 ↗
17. Chen, Y., and Liang, L. (2023). SLP-Improved DDPG Path-Planning Algorithm for Mobile Robot in Large-Scale Dynamic Environment. Sensors, 23.10.3390/s23073521 ↗
18. The Curve of Least Energy10.1145/356056.356061 ↗