journal article·2019-07-29Open access

Multi-Robot Path Planning Method Using Reinforcement Learning

Hyansu Bae, Gidong Kim, Jonguk Kim, Dianwei Qian, Sukgyu Lee

Applied Sciences· Volume 9· Issue 15

Abstract

This paper proposes a noble multi-robot path planning algorithm using Deep q learning combined with CNN (Convolution Neural Network) algorithm. In conventional path planning algorithms, robots need to search a comparatively wide area for navigation and move in a predesigned formation under a given environment. Each robot in the multi-robot system is inherently required to navigate independently with collaborating with other robots for efficient performance. In addition, the robot collaboration scheme is highly depends on the conditions of each robot, such as its position and velocity. However, the conventional method does not actively cope with variable situations since each robot has difficulty to recognize the moving robot around it as an obstacle or a cooperative robot. To compensate for these shortcomings, we apply Deep q learning to strengthen the learning algorithm combined with CNN algorithm, which is needed to analyze the situation efficiently. CNN analyzes the exact situation using image information on its environment and the robot navigates based on the situation analyzed through Deep q learning. The simulation results using the proposed algorithm shows the flexible and efficient movement of the robots comparing with conventional methods under various environments.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: Robotic Path Planning Algorithms · Reinforcement Learning in Robotics · Robot Manipulation and Learning

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

Department of Electrical Engineering Yeungnam University, Gyeongsan 38541, Korea

TycoAMP, Gyeongsan 38541, Korea

Korea Polytechnic VII, Changwon 51519, Korea

School of Control and Computer Engineering, North China Electric Power University, Beijing 102206, China

References (46)

Show source-provided references
1. Pattern Recognition and Machine Learning
2. Deep learning and its applications to signal and information processing [exploratory dsp]10.1109/MSP.2010.939038 ↗
3. Deep neural Networks for acoustic modeling in speech recognition10.1109/MSP.2012.2205597 ↗
4. Graves, A., Abdel-rahman, M., and Geoffrey, H. (2013, January 26–31). Speech recognition with deep recurrent neural networks. Proceedings of the 2013 IEEE International Conference on Acoustics, Speech and Signal Processing, Vancouver, BC, Canada.10.1109/ICASSP.2013.6638947 ↗
5. Kumar, A. (2016, January 19–24). Ask me anything: Dynamic memory networks for natural language processing. Proceedings of the International Conference on Machine Learning, New York, NY, USA.
6. (2010, July 11). Machine Learning and Natural Language Processing. Available online: http://l2r.cs.uiuc.edu/~danr/Teaching/CS546-13/Papers/marquez-LNLP00.pdf.
7. Manning, C. (2014, January 22–27). The Stanford CoreNLP natural language processing toolkit. Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics: System Demonstrations, Baltimore, MD, USA.10.3115/v1/P14-5010 ↗
8. Collobert, R., and Jason, W. (2008, January 5–9). A unified architecture for natural language processing: Deep neural networks with multitask learning. Proceedings of the 25th International Conference on Machine Learning, Helsinki, Finland.10.1145/1390156.1390177 ↗
9. Machine learning for medical diagnosis: History, state of the art and perspective10.1016/S0933-3657(01)00077-X ↗
10. Shvets, A.A., Rakhlin, A., Kalinin, A.A., and Iglovikov, V.I. (2018, January 17–20). Automatic instrument segmentation in robot-assisted surgery using deep learning. Proceedings of the 2018 17th IEEE International Conference on Machine Learning and Applications (ICMLA), Orlando, FL, USA.10.1109/ICMLA.2018.00100 ↗
11. Bottou, L. (2010, January 22–27). Large-scale machine learning with stochastic gradient descent. Proceedings of the COMPSTAT’2010, Paris, France.10.1007/978-3-7908-2604-3_16 ↗
12. Natural actor-critic10.1016/j.neucom.2007.11.026 ↗
13. A novel actor-identifier architecture for approximate optimal control of uncertain nonlinear systems10.1016/j.automatica.2012.09.019 ↗
14. Online actor-critic algorithm to solve the continuous-time infinite horizon optimal control problem10.1016/j.automatica.2010.02.018 ↗
15. Florensa, C., Degrave, J., Heess, N., Springenberg, J.T., and Riedmiller, M. (2019). Self-supervised learning of image embedding for continuous control. arXiv.
16. Lillicrap Timothy, P. (2015). Continuous control with deep reinforcement learning. arXiv.
18. Littman, M.L. (1994, January 10–13). Markov games as a framework for multi-agent-reinforcement learning. Proceedings of the Eleventh International Conference on Machine Learning, New Brunswick, NJ, USA.10.1016/B978-1-55860-335-6.50027-1 ↗
19. Structure in the space of value functions10.1023/A:1017944732463 ↗
20. Fuzzy Q-Learning for multi-agent decentralized energy management in microgrids10.1016/j.apenergy.2018.03.017 ↗
21. Keselman, A., Ten, S., Ghazali, A., and Jubeh, M. (2018). Reinforcement Learning with A* and a Deep Heuristic. arXiv.
22. Stentz, A. (1997). Optimal and efficient path planning for partially known environments. Intelligent Unmanned Ground Vehicles, Springer.10.1007/978-1-4615-6325-9_11 ↗
23. New potential functions for mobile robot path planning10.1109/70.880813 ↗
24. Robot path planning in uncertain environment using multi-objective particle swarm optimization10.1016/j.neucom.2012.09.019 ↗
25. Tharwat, A., Elhoseny, M., Hassanien, A.E., Gabel, T., and Kumar, A. (2018). Intelligent Bézier curve-based path planning model using Chaotic Particle Swarm Optimization algorithm. Clust. Comput., 1–22.10.1007/s10586-018-2360-3 ↗
26. Bezier curve based path planning in a dynamic field using modified genetic algorithm10.1016/j.jocs.2017.08.004 ↗
27. Dynamic path planning for autonomous driving on various roads with avoidance of static and moving obstacles10.1016/j.ymssp.2017.07.019 ↗
28. New path planning model for mobile anchor-assisted localization in wireless sensor networks10.1007/s11276-017-1493-2 ↗
29. Path planning for mobile robot using self-adaptive learning particle swarm optimization10.1007/s11432-016-9115-2 ↗
30. Thrun, S., Wolfram, B., and Dieter, F. (2000, January 24–28). A real-time algorithm for mobile robot mapping with applications to multi-robot and 3D mapping. Proceedings of the ICRA, San Francisco, CA, USA.
31. Bruce, J., and Manuela, V. (November, January 31). Real-time randomized path planning for robot navigation. Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems, Takamatsu, Japan.
32. Cooperative multi-robot belief space planning for autonomous navigation in unknown environments10.1007/s10514-017-9620-6 ↗
33. Fan, T., Long, P., Liu, W., and Pan, J. (2018). Fully distributed multi-robot collision avoidance via deep reinforcement learning for safe and efficient navigation in complex scenarios. arXiv.
34. Van Den Berg, J., Dave, F., and James, K. (2006, January 15–19). Anytime path planning and replanning in dynamic environments. Proceedings of the 2006 IEEE International Conference on Robotics and Automation, Orlando, FL, USA.
35. Optimal path planning of mobile robots: A review10.5897/IJPS11.1745 ↗
36. Van, H., Hado, A.G., and David, S. (2016, January 12–17). Deep reinforcement learning with double q-learning. Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence, Phoenix, AZ, USA.
37. Mnih, V. (2013). Playing atari with deep reinforcement learning. arXiv.
38. Mnih, V. (2016, January 19–24). Asynchronous methods for deep reinforcement learning. Proceedings of the International Conference on Machine Learning, New York, NY, USA.
39. Ren, S. (2015, January 7–12). Faster r-cnn: Towards real-time object detection with region proposal networks. Proceedings of the Twenty-ninth Conference on Neural Information Processing Systems, Montréal, QC, Canada.
40. Kalchbrenner, N., Edward, G., and Phil, B. (2014). A convolutional neural network for modelling sentences. arXiv.10.3115/v1/P14-1062 ↗
41. Convolutional neural networks for speech recognition10.1109/TASLP.2014.2339736 ↗
42. Simonyan, K., and Zisserman, A. (2014). Very deep convolutional networks for large-scale image recognition. arXiv.
43. A formal basis for the heuristic determination of minimum cost paths10.1109/TSSC.1968.300136 ↗
44. Commanded motion optimization to reduce residual vibration10.1115/1.4029575 ↗
45. Distributed multi-robot formation control in dynamic environments10.1007/s10514-018-9783-9 ↗
46. Neural network-based region reaching formation control for multi-robot systems in obstacle environment10.1016/j.neucom.2018.12.051 ↗