An Improved Deep Reinforcement Learning-Based UAV Area Coverage Algorithm for an Unknown Dynamic Environment
Source abstract
Abstract
With the widespread application of unmanned aerial vehicle technology in search and detection, express delivery and other fields, the requirements for unmanned aerial vehicle dynamic area coverage algorithms has become higher. For an unknown dynamic environment, an improved Dual-Attention Mechanism Double Deep Q-network area coverage algorithm is proposed in this paper. Firstly, a dual-channel attention mechanism is designed to deal with flight environment information. It can extract and fuse the features of the local obstacle information and full-area coverage information. Then, based on the traditional Double Deep Q-network algorithm, an adaptive exploration decay strategy and a coverage reward function are designed based on the real-time area coverage rate to meet the requirement of a low repeated coverage rate. The proposed algorithm can avoid dynamic obstacles and achieve global coverage under low repeated coverage rate conditions. Finally, with Python 3.12 and PyTorch 2.2.1 environment as the training platform, the simulation results show that, compared with the Soft Actor–Critic algorithm, the Double Deep Q-network algorithm, and the Attention Mechanism Double Deep Q-network algorithm, the proposed algorithm in this paper can complete the area coverage task in a dynamic and complex environment with a lower repeated coverage rate and higher coverage efficiency.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Distributed Control Multi-Agent Systems · UAV Applications and Optimization · Robotic Path Planning Algorithms
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Shaanxi Autonomous Systems and Intelligent Control International Joint Research Center, Xi’an Technological University, Xi’an 710021, China
Beijing System Design Institute of Electro-Mechanic Engineering, Beijing 100854, China
Funding
Key Research and Development Program of Shaanxi · The Youth Innovation Team of Shaanxi Universities · Young Talent Nurturing Program of Shaanxi Provincial Science and Technology Association
References (32)
Show source-provided references
1. Li, W., and Zhen, Z. (2021, January 22–24). Intelligent surveillance and reconnaissance mode of police UAV based on grid. In Proceeding of the 2021 7th International Symposium on Mechatronics and Industrial Informatics (ISMII), Zhuhai, China.
2. Wang, S., Han, Y., Chen, J., Zhang, Z., Wang, G., and Du, N. (2018, January 10–12). A deep-learning-based sea search and rescue algorithm by UAV remote sensing. Proceedings of the 2018 IEEE CSAA Guidance, Navigation and Control Conference (CGNCC), Xiamen, China.10.1109/GNCC42960.2018.9019134 ↗
3. Sutton, R.S., and Barto, A.G. (1998). Reinforcement Learning: An Introduction, MIT Press.10.1109/TNN.1998.712192 ↗
4. Ohi, A.Q., Mridha, M.F., Monowar, M.M., and Hamid, M.A. (2020). Exploring optimal control of epidemic spread using reinforcement learning. Sci. Rep., 10.10.1038/s41598-020-79147-8 ↗
5. Van Hasselt, H., Guez, A., and Silver, D. (2016, January 12–17). Deep reinforcement learning with double q-learning. Proceedings of the AAAI Conference on Artificial Intelligence, Phoenix, AZ, USA.10.1609/aaai.v30i1.10295 ↗
6. Tan, X., Han, L., Gong, H., and Wu, Q. (2023). Biologically inspired complete coverage path planning algorithm based on Q-learning. Sensors, 23.10.3390/s23104647 ↗
7. Saha, O., Ren, G., Heydari, J., Ganapathy, V., and Shah, M. (2021, January 4–6). Deep reinforcement learning based online area covering autonomous robot. Proceedings of the 2021 7th International Conference on Automation, Robotics and Applications (ICARA), Prague, Czech Republic.10.1109/ICARA51699.2021.9376477 ↗
8. A Spiral Coverage Path Planning Algorithm for Nonomnidirectional Robots10.1002/rob.22516 ↗
9. Multi-agent dynamic area coverage based on reinforcement learning with connected agents10.32604/csse.2023.031116 ↗
10. Wang, Y., Wang, Z., Xing, N., and Zhao, S. (2023, January 4–6). UAV Coverage Path Planning Based on Deep Reinforcement Learning. Proceedings of the 2023 IEEE 6th International Conference on Computer and Communication Engineering Technology (CCET), Beijing, China.10.1109/CCET59170.2023.10335145 ↗
11. Zhang, N., Yue, L., Zhang, Q., Gao, C., Zhang, B., and Wang, Y. (2025). A UAV Coverage Path Planning Method Based on a Diameter–Height Model for Mountainous Terrain. Appl. Sci., 15.10.3390/app15041988 ↗
12. Zhang, N., Zhang, B., Zhang, Q., Gao, C., Feng, J., and Yue, L. (2025). Large-Area Coverage Path Planning Method Based on Vehicle–UAV Collaboration. Appl. Sci., 15.10.3390/app15031247 ↗
13. Regional coverage maximization: Alternative geographical space abstraction and modeling10.1111/gean.12121 ↗
14. Le, A.V., Parween, R., Elara Mohan, R., Khanh Nhan, N.H., and Enjikalayil, R. (2020). Optimization complete area coverage by reconfigurable hTrihex tiling robot. Sensors, 20.10.3390/s20113170 ↗
15. Real-time path planning of unmanned aerial vehicle for target tracking and obstacle avoidance in complex dynamic environment10.1016/j.ast.2015.09.037 ↗
16. UAV dynamic path planning based on obstacle position prediction in an unknown environment10.1109/ACCESS.2021.3128295 ↗
17. Velocity obstacle guided motion planning method in dynamic environments10.1016/j.jksuci.2023.101889 ↗
18. Nonlinear MPC for collision avoidance and control of UAVs with dynamic obstacles10.1109/LRA.2020.3010730 ↗
19. Liu, J., Luo, W., Zhang, G., and Li, R. (2025). Unmanned Aerial Vehicle Path Planning in Complex Dynamic Environments Based on Deep Reinforcement Learning. Machines, 13.10.3390/machines13020162 ↗
20. A hierarchical reinforcement learning algorithm based on attention mechanism for UAV autonomous navigation10.1109/TITS.2022.3225721 ↗
21. Attention mechanisms in computer vision: A survey10.1007/s41095-022-0271-y ↗
22. A review on the attention mechanism of deep learning10.1016/j.neucom.2021.03.091 ↗
23. Rizzolatti, G., and Craighero, L. (1998). Advances in Psychological Science, Psychology Press.
24. YOLOv3-A: A traffic sign detection network based on attention mechanism
25. Qiu, S., Wu, Y., Anwar, S., and Li, C. (2021, January 1–3). Investigating attention mechanism in 3D point cloud object detection. Proceedings of the 2021 International Conference on 3D Vision (3DV), London, UK.10.1109/3DV53792.2021.00050 ↗
26. Cabreira, T.M., Brisolara, L.B., and Paulo, R.F.J. (2019). Survey on coverage path planning with unmanned aerial vehicles. Drones, 3.10.3390/drones3010004 ↗
27. Yuan, J., Liu, Z., Lian, Y., Chen, L., An, Q., Wang, L., and Ma, B. (2022). Global optimization of UAV area coverage path planning based on good point set and genetic algorithm. Aerospace, 9.10.3390/aerospace9020086 ↗
28. Concentrated coverage path planning algorithm of UAV formation for aerial photography10.1109/JSEN.2022.3168840 ↗
29. Wu, X., Wang, G., and Shen, N. (2023). Research on obstacle avoidance optimization and path planning of autonomous vehicles based on attention mechanism combined with multimodal information decision-making thoughts of robots. Front. Neurorobotics, 17.10.3389/fnbot.2023.1269447 ↗
30. Chame, H.F., and Chevallereau, C. (2016, January 8–12). A top-down and bottom-up visual attention model for humanoid object approaching and obstacle avoidance. Proceedings of the 2016 XIII Latin American Robotics Symposium and IV Brazilian Robotics Symposium (LARS/SBR), Recife, Brazil.10.1109/LARS-SBR.2016.12 ↗
31. Westheider, J., Rückin, J., and Popović, M. (2023, January 1–5). Multi-UAV adaptive path planning using deep reinforcement learning. Proceedings of the 2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Detroit, MI, USA.10.1109/IROS55552.2023.10342516 ↗
32. A new method of mobile ad hoc network routing based on greed forwarding improvement strategy10.1109/ACCESS.2019.2950266 ↗