LSTM-DQN-APF Path Planning Algorithm Empowered by Twins in Complex Scenarios
Source abstract
Abstract
In response to the issues of unreachable targets, local minima, and insufficient real-time performance in drone path planning in urban low-altitude complex scenarios, this paper proposes a fusion algorithm based on digital twin, integrating LSTM (long short-term memory), DQN (Deep Q-Network), and APF (artificial potential field). The algorithm relies on a twin system, integrating multi-sensor fusion technology and Kalman filtering to input obstacle information and UAV trajectory predictions into the DQN, which outputs action decisions for intelligent obstacle avoidance. Additionally, to address the blind search problem in trajectory planning, the algorithm introduces exploration rewards and heuristic reward components, as well as adding velocity and acceleration compensation terms to the attraction and repulsion functions, reducing the path deviation of UAVs during dynamic obstacle avoidance. Finally, to tackle the issues of insufficient training sample size and simulation accuracy, this paper leverages a digital twin platform, utilizing a dual feedback mechanism from virtual and physical environments to generate a large number of complex urban scenario samples. This approach effectively enhances the diversity and accuracy of training samples while significantly reducing the experimental costs of the algorithm. The results demonstrate that the LSTM-DQN-APF algorithm, combined with the digital twin platform, can significantly improve the issues of unreachable goals, local optimality, and real-time performance in UAV operations in complex environments. Compared to traditional algorithms, it notably enhances path planning speed and obstacle avoidance success rates. After thorough training, the proposed improved algorithm can be applied to real-world UAV systems, providing reliable technical support for applications such as smart city inspections and emergency rescue operations.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Robotic Path Planning Algorithms · Mobile Agent-Based Network Management · Network Packet Processing and Optimization
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
School of Computer Science and Engineering, Xi’an Technological University, Xi’an 710021, China
References (31)
Show source-provided references
1. Motion control of obstacle avoidance for the robot arm via improved path planning algorithm10.1007/s40430-024-05301-2 ↗
2. Dong, Z., Chen, Z., Zhou, R., and Zhang, R. (2011, January 21–23). A hybrid approach of virtual force and A∗ search algorithm for UAV path re-planning. Proceedings of the 2011 6th IEEE Conference on Industrial Electronics and Applications, Beijing, China.
3. An Improved Q-RRT* Algorithm Based on Virtual Light10.32604/csse.2021.016273 ↗
4. Ye, L., Li, J., and Li, P. (2024). Improving path planning for mobile robots in complex orchard environments: The continuous bidirectional Quick-RRT* algorithm. Front. Plant Sci., 15.10.3389/fpls.2024.1337638 ↗
5. Wang, C., Cheng, C., Yang, D., Pan, G., and Zhang, F. (2022). Path planning in localization uncertaining environment based on Dijkstra method. Front. Neurorobotics, 16.10.3389/fnbot.2022.821991 ↗
6. Time-dependent Dijkstra’s algorithm under bipolar neutrosophic fuzzy environment
7. Real-time path planning and following of a gliding robotic dolphin within a hierarchical framework10.1109/TVT.2021.3066482 ↗
8. Zhang, H., Tao, Y., and Zhu, W. (2023). Global Path Planning of Unmanned Surface Vehicle Based on Improved A-Star Algorithm. Sensors, 23.10.3390/s23146647 ↗
9. Wang, J. (2021). A Simulated Annealing Algorithm and Grid Map-Based UAV Coverage Path Planning Method for 3D Reconstruction. Electronics, 10.10.3390/electronics10070853 ↗
10. Research on task allocation of UAV cluster based on particle swarm quantization algorithm10.3934/mbe.2023002 ↗
11. Optimization of Dynamic Obstacle Avoidance Path of Multirotor UAV Based on Ant Colony Algorithm
12. Cooperative localisation of UAV swarm based on adaptive SA-PSO algorithm10.1017/aer.2022.54 ↗
13. UAV navigation in high dynamic environments: A deep reinforcement learning approach10.1016/j.cja.2020.05.011 ↗
14. A review On reinforcement learning: Introduction and applications in industrial process control10.1016/j.compchemeng.2020.106886 ↗
15. Multiagent Deep Reinforcement Learning Algorithms in StarCraft II: A Review10.1109/ACCESS.2024.3497322 ↗
16. Digital-Twin-Assisted Task Assignment in Multi-UAV Systems: A Deep Reinforcement Learning Approach10.1109/JIOT.2023.3263574 ↗
17. Watkins, C.J.C.H. (1989). Learning from Delayed Rewards. [Doctoral Dissertation, University of Cambridge].
18. An effective initialization method for genetic algorithm-based robot path planning using a directed acyclic graph10.1016/j.ins.2015.11.004 ↗
19. Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., and Riedmiller, M. (2013). Playing Atari with Deep Reinforcement Learning. arXiv.
20. Long Short-Term Memory10.1162/neco.1997.9.8.1735 ↗
21. Sun, Y., Fesenko, H., Kharchenko, V., Zhong, L., Kliushnikov, I., Illiashenko, O., Morozova, O., and Sachenko, A. (2022). UAV and IoT-based systems for the monitoring of industrial facilities using digital twins: Methodology, reliability models, and application. Sensors, 22.10.3390/s22176444 ↗
22. Seismic fragility analysis of deteriorated bridge structures employing a UAV inspection-based updated digital twin10.1007/s00158-022-03445-0 ↗
23. Deep Reinforcement Learning for Flocking Motion of Multi-UAV Systems: Learn From a Digital Twin10.1109/JIOT.2021.3127873 ↗
24. Digital Twin-Enabled Computation Offloading in UAV-Assisted MEC Emergency Networks10.1109/LWC.2023.3283912 ↗
25. Grigoropoulos, N., and Lalis, S. (2020, January 14–16). Simulation and Digital Twin Support for Managed UAV Applications. Proceedings of the 2020 IEEE/ACM 24th International Symposium on Distributed Simulation and Real Time Applications (DS-RT), Prague, Czech Republic.10.1109/DS-RT50469.2020.9213676 ↗
26. The digital twin revolution10.1038/s43588-021-00077-0 ↗
27. Micro Air Vehicle Link (MAVlink) in a Nutshell: A Survey10.1109/ACCESS.2019.2924410 ↗
28. Hutter, M., and Siegwart, R. (2018). AirSim: High-Fidelity Visual and Physical Simulation for Autonomous Vehicles. Field and Service Robotics, Springer International Publishing.10.1007/978-3-319-67361-5 ↗
29. Schulman, J., Wolski, F., Dhariwal, P., Radford, A., and Klimov, O. (2017). Proximal Policy Optimization Algorithms. arXiv.
30. Lillicrap, T.P., Hunt, J.J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., and Wierstra, D. (2015). Continuous control with deep reinforcement learning. arXiv.
31. Data-Driven Learning for H∞ Control of Adaptive Cruise Control Systems