journal article·2025-04-14Open access

Design of Swarm Intelligence Control Based on Double-Layer Deep Reinforcement Learning

Xiangpei Yan, Guorui Yu, Guoke Huang, Ruchuan Zhou, Liu Hao

Applied Sciences· Volume 15· Issue 8

Abstract

Traditional methods have limitations regarding efficient collaboration and dynamic response in complex dynamic environments. Although existing swarm intelligence control methods possess certain adaptive optimization capabilities, they still face challenges in individual and global collaborative optimization and adaptability. To address this challenge, a swarm intelligence control design method based on double-layer deep reinforcement learning (D-DRL) is proposed. This method uses a double architecture where the inner layer is responsible for dynamic decision-making and behavior optimization, and the outer layer manages resource allocation and strategy optimization. The dynamic interaction between the inner and outer layers, coupled with multi-level collaborative optimization, enhances the system’s adaptability and operating performance. The results of the unmanned aerial vehicle (UAV) swarm case study show that our method achieves effective convergence and outperforms existing swarm intelligence control approaches. Specifically, it simultaneously optimizes energy efficiency and task completion amount with a superior performance. This improvement significantly enhances the comprehensive task effectiveness of the swarm.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: Advanced Sensor and Control Systems

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

AVIC China Helicopter Design and Research Institute, Jingdezhen 333001, China

References (41)

Show source-provided references
1. Artificial Intelligence for Safety-Critical Systems in Industrial and Transportation Domains: A Survey
2. Swarm Intelligence Research: From Bio-Inspired Single-Population Swarm Intelligence to Human-Machine Hybrid Swarm Intelligence10.1007/s11633-022-1367-7 ↗
3. Advancing Electric Vehicle Charging Ecosystems with Intelligent Control of DC Microgrid Stability10.1109/TIA.2024.3413052 ↗
4. Swarm Control for Distributed Construction: A Computational Complexity Perspective10.1145/3555078 ↗
5. Energy Efficiency in Cloud Computing Data Centers: A Survey on Software Technologies10.1007/s10586-022-03713-0 ↗
6. Joint Optimization of Resource Allocation, Phase Shift, and UAV Trajectory for Energy-efficient RIS-assisted UAV-enabled MEC Systems10.1109/TGCN.2023.3287604 ↗
7. Swarm Autonomy: From Agent Functionalization to Machine Intelligence10.1002/adma.202312956 ↗
8. From PID to Swarms: A Decade of Advancements in Drone Control and Path Planning—A Systematic Review (2013–2023)10.1016/j.swevo.2024.101626 ↗
9. A Survey of Multi-Agent Systems on Distributed Formation Control10.1142/S2301385024500274 ↗
10. Saha, D., Bazmohammadi, N., Vasquez, J.C., and Guerrero, J.M. (2023). Multiple Microgrids: A Review of Architectures and Operation and Control Strategies. Energies, 16.10.3390/en16020600 ↗
11. Applications. Fuzzy Rule-based Neural Network for High-speed Train Manufacturing System Scheduling Problem10.1007/s00521-022-07190-9 ↗
12. Centralized Control Method for Voltage Coordination Challenges with OLTC and D-STATCOM in Smart Distribution Networks based IoT Communication Protocol10.1109/ACCESS.2023.3242236 ↗
13. An Online Reinforcement Learning-based Energy Management Strategy for Microgrids with Centralized Control10.1109/TIA.2024.3430264 ↗
14. Task-effective Compression of Observations for the Centralized Control of a Multiagent System over Bit-budgeted Channels10.1109/JIOT.2023.3312553 ↗
15. Multi-objective Rule System Based Control Model with Tunable Parameters for Swarm Robotic Control in Confined Environment10.23919/CSMS.2023.0022 ↗
16. Distributed Event-Triggered Voltage Restoration and Optimal Power Sharing Control for an Islanded DC Microgrid10.1016/j.ijepes.2023.109308 ↗
17. Distributed Adaptive Fault-Tolerant Control for High-Speed Trains Using Multi-Agent System Model10.1109/TVT.2023.3328640 ↗
18. Data-Driven Consensus Control of Fully Distributed Event-Triggered Multi-Agent Systems10.1007/s11432-022-3629-1 ↗
19. Distributed Stochastic Algorithm Based on Enhanced Genetic Algorithm for Path Planning of Multi-UAV Cooperative Area Search10.1109/TITS.2023.3258482 ↗
20. Improved Artificial Bee Colony Algorithm-Based Path Planning of Unmanned Autonomous Helicopter Using Multi-Strategy Evolutionary Learning10.1016/j.ast.2022.107374 ↗
21. Distributed Swarm System with Hybrid-Flocking Control for Small Fixed-Wing UAVs: Algorithms and Flight Experiments10.1016/j.eswa.2023.120457 ↗
22. Fast DC-Link Voltage Control Based on Power Flow Management Using Linear ADRC Combined with Hybrid Salp Particle Swarm Algorithm for PV/Wind Energy Conversion System10.1016/j.ijhydene.2024.02.325 ↗
23. Improved Reinforcement Learning Strategy of Energy Storage Units for Frequency Control of Hybrid Power Systems10.1016/j.est.2023.108248 ↗
24. Multi-Objective Deep Reinforcement Learning for Computation Offloading in UAV-Assisted Multi-Access Edge Computing10.1016/j.ins.2023.119154 ↗
25. UAV Swarm Cooperative Target Search: A Multi-Agent Reinforcement Learning Approach10.1109/TIV.2023.3316196 ↗
26. Chen, J., Zhao, Y., Wang, M., Yang, K., Ge, Y., Wang, K., Lin, H., Pan, P., Hu, H., and He, Z. (2025). Multi-Timescale Reward-Based DRL Energy Management for Regenerative Braking Energy Storage System. IEEE Trans. Transp. Electrif.10.1109/TTE.2025.3528255 ↗
27. Modeling Energy Consumption of Small Drones for Swarm Missions10.1109/TITS.2024.3350042 ↗
28. Unmanned aerial vehicles: A review10.1016/j.cogr.2022.12.004 ↗
29. Decentralized Swarms of Unmanned Aerial Vehicles for Search and Rescue Operations without Explicit Communication10.1007/s10514-022-10066-5 ↗
30. State-of-the-Art and Future Research Challenges in UAV Swarms10.1109/JIOT.2024.3364230 ↗
31. Larkin, E.V., Akimenko, T.A., and Bogomolov, A.V. (2023). The Swarm Hierarchical Control System. International Conference on Swarm Intelligence, Springer Nature.10.1007/978-3-031-36622-2_3 ↗
32. Proximal Policy Optimization with Policy Feedback10.1109/TSMC.2021.3098451 ↗
33. Lowe, R., Wu, Y., Tamar, A., Harb, J., Abbeel, P., and Mordatch, I. (2017). Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments. arXiv.
34. Distributed Nash Equilibrium Seeking in Games with Partial Decision Information: A Survey. Proc10.1109/JPROC.2023.3234687 ↗
35. Barman, S., Khan, A., Maiti, A., and Sawarni, A. (2023, January 7–14). Fairness and Welfare Quantification for Regret in Multi-Armed Bandits. Proceedings of the AAAI Conference on Artificial Intelligence, Washington, DC, USA.10.1609/aaai.v37i6.25829 ↗
36. Stein, A., Salvioli, M., Garjani, H., Dubbeldam, J., Viossat, Y., Brown, J.S., and Staňková, K. (2023). Stackelberg Evolutionary Game Theory: How to Manage Evolving Systems. Philos. Trans. R. Soc. Lond. B Biol. Sci., 378.10.1098/rstb.2021.0495 ↗
37. Liu, F., Zhang, T., Zhang, C., Liu, L., Wang, L., and Liu, B. (2023). A Review of the Evaluation System for Curriculum Learning. Electronics, 12.10.3390/electronics12071676 ↗
38. Curriculum Based Reinforcement Learning for Traffic Simulations10.1016/j.cag.2023.04.009 ↗
39. Bu, Y., Yan, Y., and Yang, Y. (2024). Advancement Challenges in UAV Swarm Formation Control: A Comprehensive Review. Drones, 8.10.3390/drones8070320 ↗
40. Swarm Intelligence Algorithms for Multiple Unmanned Aerial Vehicles Collaboration: A Comprehensive Review10.1007/s10462-022-10281-7 ↗
41. Toward Autonomous Multi-UAV Wireless Network: A Survey of Reinforcement Learning-Based Approaches10.1109/COMST.2023.3323344 ↗