journal article·2025-03-18Open access

Voronoi-GRU-Based Multi-Robot Collaborative Exploration in Unknown Environments

Yang Lei, Jian Hou, Peixin Ma, Mingze Ma

Applied Sciences· Volume 15· Issue 6

Abstract

In modern society, the autonomous exploration of unknown environments has attracted extensive attention due to its broad applications, such as in search and rescue operations, planetary exploration, and environmental monitoring. This paper proposes a novel collaborative exploration strategy for multiple mobile robots, aiming to quickly realize the exploration of entire unknown environments. Specifically, we investigate a hierarchical control architecture, comprising an upper decision-making layer and a lower planning and mapping layer. In the upper layer, the next frontier point for each robot is determined using Voronoi partitioning and the Multi-Agent Twin Delayed Deep Deterministic policy gradient (MATD3) deep reinforcement learning algorithm in a centralized training and decentralized execution framework. In the lower layer, navigation planning is achieved using A* and Timed Elastic Band (TEB) algorithms, while an improved Cartographer algorithm is used to construct a joint map for the multi-robot system. In addition, the improved Robot Operating System (ROS) and Gazebo simulation environments speed up simulation times, further alleviating the slow training of high-precision simulation engines. Finally, the simulation results demonstrate the superiority of the proposed strategy, which achieves over 90% exploration coverage in unknown environments with a significantly reduced exploration time. Compared to MATD3, Multi-Agent Proximal Policy Optimization (MAPPO), Rapidly-Exploring Random Tree (RRT), and Cost-based methods, our strategy reduces time consumption by 41.1%, 47.0%, 63.9%, and 74.9%, respectively.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: Robotic Path Planning Algorithms · Robotics and Sensor-Based Localization · Reinforcement Learning in Robotics

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

School of Computer Science and Technology, Zhejiang Sci-Tech University, Hangzhou 310018, China

Beijing Zhongke Huiling Robot Technology Co., Ltd., Beijing 100192, China

School of Information Engineering, Wenzhou Business College, Wenzhou 325035, China

Funding

Zhejiang Provincial Natural Science Foundation of China · Department of Education of Zhejiang Province of China

References (51)

Show source-provided references
1. Voronoi-based multi-robot autonomous exploration in unknown environments via deep reinforcement learning10.1109/TVT.2020.3034800 ↗
2. H2GNN: Hierarchical-hops graph neural networks for multi-robot exploration in unknown environments10.1109/LRA.2022.3146912 ↗
3. Deep Reinforcement Learning-Based Mapless Navigation for Mobile Robot in Unknown Environment with Local Optima10.1109/LRA.2024.3511437 ↗
4. Decentralized and centralized planning for multi-robot additive manufacturing10.1115/1.4055735 ↗
5. Modeling and simulation of cooperative exploration strategies in unknown environments10.23919/CSMS.2023.0014 ↗
6. Yan, X., Zeng, Z., He, K., and Hong, H. (2023). Multi-robot cooperative autonomous exploration via task allocation in terrestrial environments. Front. Neurorobot., 17.10.3389/fnbot.2023.1179033 ↗
7. Robotic urban search and rescue: A survey from the control perspective10.1007/s10846-013-9822-x ↗
8. Robust Online Multi-robot Simultaneous Exploration and Coverage Path Planning10.1109/ACCESS.2024.3403658 ↗
9. Breitenmoser, A., Schwager, M., Metzger, J.C., Siegwart, R., and Rus, D. (2010, January 3–7). Voronoi coverage of non-convex environments with a group of networked robots. Proceedings of the 2010 IEEE International Conference on Robotics and Automation, Anchorage, AK, USA.10.1109/ROBOT.2010.5509696 ↗
10. Yu, J., Tong, J., Xu, Y., Xu, Z., Dong, H., Yang, T., and Wang, Y. (June, January 30). Smmr-explore: Submap-based multi-robot exploration system with multi-robot multi-target potential field exploration method. Proceedings of the 2021 IEEE International Conference on Robotics and Automation (ICRA), Xi’an, China.10.1109/ICRA48506.2021.9561328 ↗
11. Unknown environment exploration of multi-robot system with the FORDPSO10.1016/j.swevo.2015.09.004 ↗
12. Ant colony optimization10.1109/MCI.2006.329691 ↗
13. A critical review of communications in multi-robot systems10.1007/s43154-022-00090-9 ↗
14. Multi-robot distributed communication in heterogeneous robotic systems on 5G networking10.1109/MWC.001.2200315 ↗
15. A decentralized dynamic self-triggered control approach to consensus of multiagent systems10.1109/TSMC.2023.3274551 ↗
16. Racer: Rapid collaborative exploration with a decentralized multi-uav system10.1109/TRO.2023.3236945 ↗
17. Reinforcement learning in robotics: A survey10.1177/0278364913495721 ↗
18. Multi-agent actor-critic for mixed cooperative-competitive environments
19. Ackermann, J., Gabler, V., Osa, T., and Sugiyama, M. (2019). Reducing overestimation bias in multi-agent domains using double centralized critics. arXiv.
20. Guyon, I., Luxburg, U.V., Bengio, S., Wallach, H., Fergus, R., Vishwanathan, S., and Garnett, R. (2017). EX2: Exploration with Exemplar Models for Deep Reinforcement Learning. Proceedings of the Advances in Neural Information Processing Systems, Curran Associates, Inc.
21. Gu, S., Holly, E., Lillicrap, T., and Levine, S. (June, January 29). Deep reinforcement learning for robotic manipulation with asynchronous off-policy updates. Proceedings of the 2017 IEEE International Conference on Robotics and Automation (ICRA), Singapore.10.1109/ICRA.2017.7989385 ↗
22. Coordinated multi-robot exploration under communication constraints using decentralized markov decision processes
23. Papoudakis, G., Christianos, F., Rahman, A., and Albrecht, S.V. (2019). Dealing with non-stationarity in multi-agent deep reinforcement learning. arXiv.
24. Zhang, K., Yang, Z., and Başar, T. (2021). Multi-agent reinforcement learning: A selective overview of theories and algorithms. Handbook of Reinforcement Learning and Control, Springer.10.1007/978-3-030-60990-0_12 ↗
25. Gupta, J.K., Egorov, M., and Kochenderfer, M. (2017, January 8–12). Cooperative multi-agent control using deep reinforcement learning. Proceedings of the Autonomous Agents and Multiagent Systems: AAMAS 2017 Workshops, São Paulo, Brazil. Best Papers, Revised Selected Papers 16.
26. HMS-RRT: A novel hybrid multi-strategy rapidly-exploring random tree algorithm for multi-robot collaborative exploration in unknown environments10.1016/j.eswa.2024.123238 ↗
27. Zhou, C., Li, J., Shi, Y., and Lin, Z. (2023). Research on multi-robot formation control based on matd3 algorithm. Appl. Sci., 13.10.3390/app13031874 ↗
28. Deep multiagent reinforcement learning: Challenges and directions10.1007/s10462-022-10299-x ↗
29. Dy, J., and Krause, A. (2018, January 10–15). Addressing Function Approximation Error in Actor-Critic Methods. Proceedings of the 35th International Conference on Machine Learning, Stockholm, Sweden.
30. A TD3-based multi-agent deep reinforcement learning method in mixed cooperation-competition environment10.1016/j.neucom.2020.05.097 ↗
31. Padrao, P., Fuentes, J., Bobadilla, L., and Smith, R.N. (2022). Estimating spatio-temporal fields through reinforcement learning. Front. Robot. AI, 9.10.3389/frobt.2022.878246 ↗
32. Ding, C., Zhou, Y., Pu, G., and Zhang, H. (2022). Low carbon economic dispatch of power system at multiple time scales considering GRU wind power forecasting and integrated carbon capture. Front. Energy Res., 10.10.3389/fenrg.2022.953883 ↗
33. Multi-robot collaborative hunting in cluttered environments with obstacle-avoiding Voronoi cells10.1109/JAS.2023.124041 ↗
34. Towards heterogeneous robot team path planning: Acquisition of multiple routes with a modified spline-based algorithm10.1051/matecconf/201711302015 ↗
35. Reinforcement learning algorithms: A brief survey10.1016/j.eswa.2023.120495 ↗
36. Human-level control through deep reinforcement learning10.1038/nature14236 ↗
37. A formal basis for the heuristic determination of minimum cost paths10.1109/TSSC.1968.300136 ↗
38. Cao, X., Xu, Y., Yao, Y., and Zhi, C. (2023). An Improved Hybrid A* Algorithm of Path Planning for Hotel Service Robot. Int. J. Adv. Comput. Sci. Appl., 14.10.14569/IJACSA.2023.0141091 ↗
39. Wu, J., Ma, X., Peng, T., and Wang, H. (2021). An improved timed elastic band (TEB) algorithm of autonomous ground vehicle (AGV) in complex environment. Sensors, 21.10.3390/s21248312 ↗
40. Hess, W., Kohler, D., Rapp, H., and Andor, D. (2016, January 16–21). Real-time loop closure in 2D LIDAR SLAM. Proceedings of the 2016 IEEE international Conference on Robotics and Automation (ICRA), Stockholm, Sweden.10.1109/ICRA.2016.7487258 ↗
41. Quigley, M. (2009, January 12–17). ROS: An open-source Robot Operating System. Proceedings of the IEEE International Conference on Robotics and Automation, Kobe, Japan.
42. Koenig, N., and Howard, A. (October, January 28). Design and use paradigms for gazebo, an open-source multi-robot simulator. Proceedings of the 2004 IEEE/RSJ international Conference on Intelligent Robots and Systems (IROS) (IEEE Cat. No. 04CH37566), Sendai, Japan.
43. Multi-agent actor-critic with hierarchical graph attention network
44. The surprising effectiveness of ppo in cooperative multi-agent games
45. Umari, H., and Mukhopadhyay, S. (2017, January 24–28). Autonomous robotic exploration based on multiple rapidly-exploring randomized trees. Proceedings of the 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Vancouver, BC, Canada.10.1109/IROS.2017.8202319 ↗
46. Exploration strategies based on multi-criteria decision making for searching environments in rescue operations10.1007/s10514-011-9249-9 ↗
47. Chen, L., Zhao, Y., Zhao, H., and Zheng, B. (2021). Non-communication decentralized multi-robot collision avoidance in grid map workspace with double deep Q-network. Sensors, 21.10.3390/s21030841 ↗
48. Bashabsheh, M. (2024). Comprehensive and Simulated Modeling of a Centralized Transport Robot Control System. Int. J. Adv. Comput. Sci. Appl., 15.10.14569/IJACSA.2024.0150552 ↗
49. Tobin, J., Fong, R., Ray, A., Schneider, J., Zaremba, W., and Abbeel, P. (2017, January 24–28). Domain randomization for transferring deep neural networks from simulation to the real world. Proceedings of the 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Vancouver, BC, Canada.10.1109/IROS.2017.8202133 ↗
50. A survey on multi-robot coverage path planning for model reconstruction and mapping10.1007/s42452-019-0872-y ↗
51. Zhou, M., Li, J., Wang, C., Wang, J., and Wang, L. (2024). Applications of Voronoi Diagrams in Multi-Robot Coverage: A Review. J. Mar. Sci. Eng., 12.10.3390/jmse12061022 ↗