Fast and Efficient Data Collection Management Approach with Two-Layer UAV Network with Massive Sensor Nodes
Source abstract
Abstract
Large-scale UAV data collection creates a tension among wide-area coverage, operational efficiency, and delivery continuity. Data must be continuously delivered to a base-station coordinator, but real-time replanning becomes increasingly difficult as the number of sensors and UAVs grows. Standard vehicle-routing methods slow down once routes have to be regenerated often, while reinforcement learning struggles with fixed-wing UAVs that cannot hover or turn sharply. We address this with a two-layer framework. In the lower layer, multirotor UAVs visit sensor nodes and buffer the collected payload until it is retrieved by a fixed-wing UAV. Their routes come from clustering the nodes and solving a capacitated vehicle routing problem within each cluster, with the cost biased toward older data and a short cooldown against immediate revisits. In the upper layer, fixed-wing UAVs deliver the buffered payload to the base-station coordinator, guided by a Multi-Agent Proximal Policy Optimization (MAPPO) policy that receives a local buffer-summary map and selected high-priority cells from a compact global summary. A spacing reward encourages separation before agents enter close-proximity states, instead of only penalizing collisions afterward. Component-level experiments show that the lower-layer planner handles up to 600 active routing targets within 1.3 s on average and that the age/cooldown objective improves freshness and revisit behavior. In integrated simulations with 1000 nodes, 32 multirotor UAVs, and 2 fixed-wing UAVs, the learned fixed-wing policy maintains collection performance comparable to a strong exclusive greedy baseline while recording no collision or persistent-proximity termination events over the reported data-generation-rate sweep. These results support the proposed framework as a scalable coordination-layer design for dynamic sensor workloads, where adaptive multirotor routing and motion-constrained fixed-wing retrieval are evaluated together under a shared data-generation workload.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: UAV Applications and Optimization · Distributed Control Multi-Agent Systems · Reinforcement Learning in Robotics
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Department of Data Engineering, Pukyong National University, Busan 48513, Republic of Korea
Department of Computer and Artificial Intelligence Engineering, Pukyong National University, Busan 48513, Republic of Korea
School of Electrical Engineering, Korea University, Seoul 02841, Republic of Korea
Funding
Korea Research Institute for defense Technology planning and advancement · Defense Acquisition Program Administration
References (51)
Show source-provided references
1. Unmanned Aerial Vehicles: A Survey on Civil Applications and Key Research Challenges10.1109/ACCESS.2019.2909530 ↗
2. Yu, Y., and Lee, S. (2023). Efficient Multi-UAV Path Planning for Collaborative Area Search Operations. Appl. Sci., 13.10.3390/app13158728 ↗
3. Data Collection in Studies on Internet of Things (IoT), Wireless Sensor Networks (WSNs), and Sensor Cloud (SC): Similarities and Differences10.1109/ACCESS.2022.3161929 ↗
4. Yang, Y., Yang, T., Wu, X., and Hu, B. (2025). UAV-Assisted Multi-Task Federated Learning with Task Knowledge Sharing. Proceedings of the ICC 2025—IEEE International Conference on Communications, Montreal, QC, Canada, 8–12 June 2025, IEEE.10.1109/ICC52391.2025.11162109 ↗
5. Data-Driven Flight Control of Internet-of-Drones for Sensor Data Aggregation Using Multi-Agent Deep Reinforcement Learning10.1109/MWC.002.2100681 ↗
6. Exploration in deep reinforcement learning: A survey10.1016/j.inffus.2022.03.003 ↗
7. Ekechi, C.C., Elfouly, T., Alouani, A., and Khattab, T. (2025). A Survey on UAV Control with Multi-Agent Reinforcement Learning. Drones, 9.10.3390/drones9070484 ↗
8. A Conflict-Based Search Framework for Multiobjective Multiagent Path Finding10.1109/TASE.2022.3183183 ↗
9. Greedy Priority-Based Search for Suboptimal Multi-Agent Path Finding
10. Liang, P., Li, Z., You, D., and Li, Z. (2024). A Conflict-free multi-agent path planning method based on Improve A-star algorithm. Proceedings of the 2024 4th International Conference on Electrical Engineering and Control Science (IC2ECS), Nanjing, China, 27–29 December 2024, IEEE.10.1109/IC2ECS64405.2024.10928312 ↗
11. Wang, S., Xu, H., Zhang, Y., Lin, J., Lu, C., Wang, X., and Li, W. (2025). Where Paths Collide: A Comprehensive Survey of Classic and Learning-Based Multi-Agent Pathfinding. arXiv.
12. Machine Learning to Solve Vehicle Routing Problems: A Survey10.1109/TITS.2023.3334976 ↗
13. A comprehensive survey on the Multiple Traveling Salesman Problem: Applications, approaches and taxonomy10.1016/j.cosrev.2021.100369 ↗
14. The flying sidekick traveling salesman problem: Optimization of drone-assisted parcel delivery10.1016/j.trc.2015.03.005 ↗
15. Vehicle routing problem with drones10.1016/j.trb.2019.03.005 ↗
16. Fast Replanning for Navigation in Unknown Terrain10.1109/TRO.2004.838026 ↗
17. Lifelong Planning A*10.1016/j.artint.2003.12.001 ↗
18. Rolling-Horizon Genetic Algorithm for Adaptive Path Planning in Hazardous Environments10.1016/j.compeleceng.2025.110820 ↗
19. Schulman, J., Wolski, F., Dhariwal, P., Radford, A., and Klimov, O. (2017). Proximal Policy Optimization Algorithms. arXiv.
20. Lowe, R., Wu, Y., Tamar, A., Harb, J., Abbeel, P., and Mordatch, I. (2017). Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments. arXiv.
21. Yu, C., Velu, A., Vinitsky, E., Gao, J., Wang, Y., Bayen, A., and Wu, Y. (2022). The Surprising Effectiveness of PPO in Cooperative, Multi-Agent Games. arXiv.10.52202/068431-1787 ↗
22. Su, K., and Qian, F. (2023). Multi-UAV Cooperative Searching and Tracking for Moving Targets Based on Multi-Agent Reinforcement Learning. Appl. Sci., 13.10.3390/app132111905 ↗
23. Liu, Y., Li, X., Wang, J., Wei, F., and Yang, J. (2024). Reinforcement-Learning-Based Multi-UAV Cooperative Search for Moving Targets in 3D Scenarios. Drones, 8.10.3390/drones8080378 ↗
24. Liao, W.C., Wu, T.R., and Wu, I.C. (2025). Dynamic Sight Range Selection in Multi-Agent Reinforcement Learning. arXiv.10.65109/NEWL1408 ↗
25. Sahingoz, O.K. (2013). Mobile networking with UAVs: Opportunities and challenges. Proceedings of the 2013 International Conference on Unmanned Aircraft Systems (ICUAS), Atlanta, GA, USA, 28–31 May 2013, IEEE.10.1109/ICUAS.2013.6564779 ↗
26. Hierarchical routing for multi-layer ad-hoc wireless networks with UAVs
27. Reinforcement Learning in the Sky: A Survey on Enabling Intelligence in NTN-Based Communications10.1109/ACCESS.2023.3236801 ↗
28. Intelligent Spectrum Sharing in Integrated TN-NTNs: A Hierarchical Deep Reinforcement Learning Approach10.1109/MWC.001.2400421 ↗
29. Xin, Y., Li, L., Ning, Y., Yang, Y., and Shi, P. (2025). UAV-Assisted Cooperative Charging and Data Collection Strategy for Heterogeneous Wireless Sensor Networks. Drones, 9.10.3390/drones9120859 ↗
30. Dechicha, M.I., Eudes, A., De Nazelle, P., and Marzat, J. (2025). Exploration Strategies for a Heterogeneous UAV-UGV Tandem. Proceedings of the 2025 European Conference on Mobile Robots (ECMR), Padova, Italy, 2–5 September 2025, IEEE.10.1109/ECMR65884.2025.11163236 ↗
31. A regulatory-no-fly-zone-aware drone-rider takeouts routing optimization10.1016/j.cie.2026.111992 ↗
32. Energy-Efficient Data Collection and Task Offloading Optimization in Heterogeneous Multi-Tier AAV Systems via Deep Reinforcement Learning10.1109/TVT.2025.3622980 ↗
33. Le, C., Vu, T.X., and Chatzinotas, S. (2024). Cooperative UAVs with Asynchronous Multi-agent Learning for Remote Data Collection. Proceedings of the 2024 IEEE Globecom Workshops (GC Wkshps), Cape Town, South Africa, 8–12 December 2024, IEEE.10.1109/GCWkshp64532.2024.11101581 ↗
34. Lee, C., Lee, S., Kim, T., Bang, I., Lee, J.H., and Chae, S.H. (2024). Multi-Agent Deep Reinforcement Learning-Based Multi-UAV Path Planning for Wireless Data Collection and Energy Transfer. Proceedings of the 2024 Fifteenth International Conference on Ubiquitous and Future Networks (ICUFN), Budapest, Hungary, 2–5 July 2024, IEEE.10.1109/ICUFN61752.2024.10625275 ↗
35. Zhang, Y., Zhao, Q., Mao, P., Bai, Q., Li, F., and Pavlova, S. (2024). Design and Control of an Ultra-Low-Cost Logistic Delivery Fixed-Wing UAV. Appl. Sci., 14.10.3390/app14114358 ↗
36. Halder, S., and Afsari, K. (2023). Robots in Inspection and Monitoring of Buildings and Infrastructure: A Systematic Review. Appl. Sci., 13.10.3390/app13042304 ↗
37. Khanzada, H.R., Maqsood, A., and Basit, A. (2025). Reinforcement learning for UAV flight controls: Evaluating continuous space reinforcement learning algorithms for fixed-wing UAVs. PLoS ONE, 20.10.1371/journal.pone.0334219 ↗
38. A Review of Reinforcement Learning for Fixed-Wing Aircraft Control Tasks10.1109/ACCESS.2024.3433540 ↗
39. Hu, T., Zhu, W., and Yan, Y. (2025). Reward Hacking in Reinforcement Learning and RLHF: A Multidisciplinary Examination of Vulnerabilities, Mitigation Strategies, and Alignment Challenges. Proceedings of the 2025 5th Intelligent Cybersecurity Conference (ICSC), Tampa, FL, USA, 19–22 May 2025, IEEE.10.1109/ICSC65596.2025.11140447 ↗
40. Fu, J., Zhao, X., Yao, C., Wang, H., Han, Q., and Xiao, Y. (2025). Reward Shaping to Mitigate Reward Hacking in RLHF. arXiv.
41. A Survey and Framework of Cooperative Perception: From Heterogeneous Singleton to Hierarchical Cooperation10.1109/TITS.2024.3436012 ↗
42. Furnon, V., and Perron, L. (OR-Tools Routing Library, 2025). OR-Tools Routing Library, Version 9.14.
43. Human-level control through deep reinforcement learning10.1038/nature14236 ↗
44. A Brief Survey of Deep Reinforcement Learning10.1109/MSP.2017.2743240 ↗
45. Konda, V., and Tsitsiklis, J. (1999). Actor-Critic Algorithms. Proceedings of the Advances in Neural Information Processing Systems, MIT Press.
46. Schulman, J., Moritz, P., Levine, S., Jordan, M., and Abbeel, P. (2018). High-Dimensional Continuous Control Using Generalized Advantage Estimation. arXiv.
47. Sutton, R.S., and Barto, A.G. (2018). Reinforcement Learning: An Introduction, MIT Press.
48. Lai, J., Wang, D., and Zhao, Y. (2025). IMSBA: A Novel Integrated Sensing and Communication Beam Allocation Based on Multi-Agent Reinforcement Learning for mmWave Internet of Vehicles. Appl. Sci., 15.10.3390/app15116069 ↗
49. Kang, C., Xu, J., Bian, Y., and Tian, W. (2025). Path Navigation for Multi-Agent Flight System Based on Distributed Model Predictive Control and Reinforcement Learning. Appl. Sci., 15.10.3390/app15042234 ↗
50. MARL-Based Multi-Satellite Intelligent Task Planning Method10.1109/ACCESS.2023.3337358 ↗
51. VMAS: A Vectorized Multi-agent Simulator for Collective Robot Learning10.1007/978-3-031-51497-5_4 ↗