Dynamic Obstacle Avoidance and Path Planning through Reinforcement Learning
Source abstract
Abstract
The use of reinforcement learning (RL) for dynamic obstacle avoidance (DOA) algorithms and path planning (PP) has become increasingly popular in recent years. Despite the importance of RL in this growing technological era, few studies have systematically reviewed this research concept. Therefore, this study provides a comprehensive review of the literature on dynamic reinforcement learning-based path planning and obstacle avoidance. Furthermore, this research reviews publications from the last 5 years (2018–2022) to include 34 studies to evaluate the latest trends in autonomous mobile robot development with RL. In the end, this review shed light on dynamic obstacle avoidance in reinforcement learning. Likewise, the propagation model and performance evaluation metrics and approaches that have been employed in previous research were synthesized by this study. Ultimately, this article’s major objective is to aid scholars in their understanding of the present and future applications of deep reinforcement learning for dynamic obstacle avoidance.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Robotic Path Planning Algorithms · Reinforcement Learning in Robotics · Robotic Locomotion and Control
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
College of Computing and Informatics, University of Sharjah, Sharjah P.O. Box 27272, United Arab Emirates
Autonomous Robotics Research Center, Technology Innovation Institute, Abu Dhabi P.O. Box 9639, United Arab Emirates
References (62)
Show source-provided references
1. Deep reinforcement learning for adaptive path planning and control of an autonomous underwater vehicle10.1016/j.apor.2022.103326 ↗
2. Kozjek, D., Malus, A., and Vrabič, R. (2021). Reinforcement-Learning-Based Route Generation for Heavy-Traffic Autonomous Mobile Robot Systems. Sensors, 21.10.3390/s21144809 ↗
3. Long, P., Fan, T., Liao, X., Liu, W., Zhang, H., and Pan, J. (2018, January 21–25). Towards optimally decentralized multi-robot collision avoidance via deep reinforcement learning. Proceedings of the 2018 IEEE International Conference on Robotics And Automation (ICRA), Brisbane, QLD, Australia.10.1109/ICRA.2018.8461113 ↗
4. Khatib, O. (1985, January 25–28). Real-time obstacle avoidance for manipulators and mobile robots. Proceedings of the 1985 IEEE International Conference on Robotics and Automation, St. Louis, MO, USA.
5. Bin Issa, R., Das, M., Rahman, S., Barua, M., Rhaman, K., Ripon, K.S.N., and Alam, G.R. (2021). Double Deep Q-Learning and Faster R-CNN-Based Autonomous Vehicle Navigation and Obstacle Avoidance in Dynamic Environment. Sensors, 21.10.3390/s21041468 ↗
6. Reactive navigation under a fuzzy rules-based scheme and reinforcement learning for mobile robots10.7717/peerj-cs.556 ↗
7. Reinforcement learning in robotics: A survey10.1177/0278364913495721 ↗
8. Van Hasselt, H., Guez, A., and Silver, D. (2016, January 12–17). Deep Reinforcement Learning with Double Q-Learning. Proceedings of the AAAI Conference on Artificial Intelligence, Phoenix, AZ, USA.10.1609/aaai.v30i1.10295 ↗
9. Song, H., Li, A., Wang, T., and Wang, M. (2021). Multimodal Deep Reinforcement Learning with Auxiliary Task for Obstacle Avoidance of Indoor Mobile Robot. Sensors, 21.10.3390/s21041363 ↗
10. Convergence-Based Exploration Algorithm for Reinforcement Learning Recommended Citation. Electr
11. Xie, L., Wang, S., Markham, A., and Trigoni, N. (2017). Towards Monocular Vision based Obstacle Avoidance through Deep Reinforcement Learning. arXiv.
12. Akanksha, E., Sharma, N., and Gulati, K. (2021, January 8–10). Review on Reinforcement Learning, Research Evolution and Scope of Application. Proceedings of the 2021 5th International Conference on Computing Methodologies and Communication (ICCMC), Erode, India.10.1109/ICCMC51019.2021.9418283 ↗
13. Deep Learning for Consumer Devices and Services: Pushing the limits for machine learning, artificial intelligence, and computer vision10.1109/MCE.2016.2640698 ↗
14. Systematic literature reviews in software engineering—A systematic literature review10.1016/j.infsof.2008.09.009 ↗
15. Zhu, Y., Mottaghi, R., Kolve, E., Lim, J.J., Gupta, A., Fei-Fei, L., and Farhadi, A. (June, January 29). Target-driven visual navigation in indoor scenes using deep reinforcement learning. Proceedings of the 2017 IEEE International Conference on Robotics and Automation (ICRA), Singapore.10.1109/ICRA.2017.7989381 ↗
16. Efficient Exploration for Multi-Agent Reinforcement Learning via Transferable Successor Features10.1109/JAS.2022.105809 ↗
17. Secured Convolutional Layer IP Core in Convolutional Neural Network Using Facial Biometric10.1109/TCE.2022.3190069 ↗
18. Motion planning and control for mobile robot navigation using machine learning: A survey10.1007/s10514-022-10039-8 ↗
19. Motion Planning in Dynamic Environments Using Velocity Obstacles10.1177/027836499801700706 ↗
20. Deep Reinforcement Learning Controller for 3D Path Following and Collision Avoidance by Autonomous Underwater Vehicles
21. Characterization of Semantic Segmentation Models on Mobile Platforms for Self-Navigation in Disaster-Struck Zones10.1109/ACCESS.2022.3190014 ↗
22. Joseph, L.M.I.L., Goel, P., Jain, A., Rajyalakshmi, K., Gulati, K., and Singh, P. (2021, January 7–9). A Novel Hybrid Deep Learning Algorithm for Smart City Traffic Congestion Predictions. Proceedings of the 2021 6th International Conference on Signal Processing, Computing and Control (ISPCC), Solan, India.10.1109/ISPCC53510.2021.9609467 ↗
23. Cochrane update. ‘Scoping the scope’ of a cochrane review10.1093/pubmed/fdr015 ↗
24. Guidelines for conducting systematic mapping studies in software engineering: An update10.1016/j.infsof.2015.03.007 ↗
25. Moher, D., Liberati, A., Tetzlaff, J., and Altman, D.G. (2009). Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement. PLoS Med., 6.10.1371/journal.pmed.1000097 ↗
26. Higgins, J.P., and Green, S. (2008). Cochrane Handbook for Systematic Reviews of Interventions, Wiley-Blackwell.10.1002/9780470712184 ↗
27. Aromataris, E., and Munn, Z. (2020). JBI Manual for Evidence Synthesis, JBI.
28. Wang, N., Zhang, D., and Wang, Y. (2020, January 27–29). Learning to Navigate for Mobile Robot with Continual Reinforcement Learning. Proceedings of the 39th Chinese Control Conference, Shenyang, China.10.23919/CCC50068.2020.9188558 ↗
29. Chen, C., Jeng, S., and Lin, C. (2020). Mobile Robot Wall-Following Control Using Fuzzy Logic Controller with Improved Differential Search and Reinforcement Learning. Mathematics, 8.10.3390/math8081254 ↗
30. Path planning of mobile robot in unknown dynamic continuous environment using reward-modified deep Q-network10.1002/oca.2781 ↗
31. Gao, R., Gao, X., Liang, P., Han, F., Lan, B., Li, J., Li, J., and Li, S. (2019, January 3–5). Motion Control of Non-Holonomic Constrained Mobile Robot Using Deep Reinforcement Learning. Proceedings of the 2019 IEEE 4th International Conference on Advanced Robotics and Mechatronics (ICARM), Toyonaka, Japan.10.1109/ICARM.2019.8834284 ↗
32. Lee, H., and Jeong, J. (2021). Mobile Robot Path Optimization Technique Based on Reinforcement Learning Algorithm in Warehouse Environment. Appl. Sci., 11.10.3390/app11031209 ↗
33. Xiang, J., Li, G., Dong, X., and Ren, Z. (2019, January 22–24). Continuous Control with Deep Reinforcement Learning for Mobile Robot Navigation. Proceedings of the 2019 Chinese Automation Congress (CAC), Hangzhou, China.10.1109/CAC48633.2019.8996652 ↗
34. Mobile Robot Obstacle Avoidance Base on Deep Reinforcement Learning
35. Reinforcement Learning for Mobile Robot Obstacle Avoidance under Dynamic Environments
36. Reinforcement learning-based dynamic obstacle avoidance and integration of path planning10.1007/s11370-021-00387-2 ↗
37. Xiaoxian, S., Chenpeng, Y., Haoran, Z., Chengju, L., and Qijun, C. (2020, January 9–11). Obstacle Avoidance Algorithm for Mobile Robot Based on Deep Reinforcement Learning in Dynamic Environments. Proceedings of the 2020 IEEE 16th International Conference on Control & Automation (ICCA), Singapore.10.1109/ICCA51439.2020.9264363 ↗
38. Deep reinforcement learning for modeling human locomotion control in neuromechanical simulation10.1186/s12984-021-00919-y ↗
39. Optimal path-planning of nonholonomic terrain robots for dynamic obstacle avoidance using single-time velocity estimator and reinforcement learning approach10.1109/ACCESS.2019.2950166 ↗
40. Autonomous navigation of a mobile robot in dynamic indoor environments using SLAM and reinforcement learning10.1088/1757-899X/402/1/012022 ↗
41. Ruan, X., Ren, D., Zhu, X., and Huang, J. (2019, January 3–5). Mobile Robot Navigation based on Deep Reinforcement Learning. Proceedings of the 2019 Chinese Control and Decision Conference (CCDC), Nanchang, China.10.1109/CCDC.2019.8832393 ↗
42. Path planning method with improved artificial potential field—A reinforcement learning perspective10.1109/ACCESS.2020.3011211 ↗
43. Han, S., Choi, H., Benz, P., and Loaiciga, J. (2018, January 15–17). Sensor-Based Mobile Robot Navigation via Deep Reinforcement Learning. Proceedings of the IEEE International Conference on Big Data and Smart Computing, Shanghai, China.10.1109/BigComp.2018.00030 ↗
44. Gao, J., Ye, W., Guo, J., and Li, Z. (2020). Deep Reinforcement Learning for Indoor Mobile Robot Path Planning. Sensors, 20.10.3390/s20195493 ↗
45. Neural networks based reinforcement learning for mobile robots obstacle avoidance10.1016/j.eswa.2016.06.021 ↗
46. Reinforcement Learning-Based Continuous Action Space Path Planning Method for Mobile Robots
47. Deep Reinforcement Learning based Path Planning for Mobile Robot in Unknown Environment10.1088/1742-6596/1576/1/012009 ↗
48. Wall-following and Navigation Control of Mobile Robot Using Reinforcement Learning Based on Dynamic Group Artificial Bee Colony10.1007/s10846-017-0743-y ↗
49. Mobile Robot Path Planning Using a QAPF Learning Algorithm for Known and Unknown Environments10.1109/ACCESS.2022.3197628 ↗
50. A novel mobile robot navigation method based on deep reinforcement learning10.1177/1729881420921672 ↗
51. Everett, M., Chen, Y.F., and How, J.P. (2018, January 1–5). Motion planning among dynamic, decision-making agents with deep reinforcement learning. Proceedings of the 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) Madrid, Spain.10.1109/IROS.2018.8593871 ↗
52. Deep reinforcement learning robot for search and rescue applications: Exploration in unknown cluttered environments10.1109/LRA.2019.2891991 ↗
53. Application of Deep Reinforcement Learning Tracking Control of 3WD Omnidirectional Mobile Robot10.5755/j01.itc.50.3.25979 ↗
54. A deep reinforcement learning based method for real-time path planning and dynamic obstacle avoidance10.1016/j.neucom.2022.05.006 ↗
55. Bernat, J., Czopek, P., and Bartosik, S. (2022). Analysis of Mobile Robot Control by Reinforcement Learning Algorithm. Electronics, 11.10.3390/electronics11111754 ↗
56. A multi-robot path-planning algorithm for autonomous navigation using meta-reinforcement learning based on transfer learning10.1016/j.asoc.2021.107605 ↗
57. Baek, D., Hwang, M., Kim, H., and Kwon, D.S. (2018, January 26–30). Path planning for automation of surgery robot based on probabilistic roadmap and reinforcement learning. Proceedings of the 2018 15th International Conference on Ubiquitous Robots (UR), Honolulu, HI, USA.10.1109/URAI.2018.8441801 ↗
58. The Path Planning of Mobile Robot by Neural Networks and Hierarchical Reinforcement Learning10.3389/fnbot.2020.00063 ↗
59. A Path-Integral-Based Reinforcement Learning Algorithm for Path Following of an Autoassembly Mobile Robot10.1109/TNNLS.2019.2955699 ↗
60. NN Reinforcement Learning Adaptive Control for a Class of Nonstrict-Feedback Discrete-Time Systems10.1109/TCYB.2020.2963849 ↗
61. Sutskever, I., Vinyals, O., and Le, Q.V. (2014, January 8–13). Sequence to sequence learning with neural networks. Proceedings of the Advances in Neural Information Processing Systems, Montreal, QC, Canada.
62. Reinforcement learning for robot research: A comprehensive review and open issues