Path-Following and Obstacle Avoidance Control of Nonholonomic Wheeled Mobile Robot Based on Deep Reinforcement Learning
Source abstract
Abstract
In this paper, a novel path-following and obstacle avoidance control method is given for nonholonomic wheeled mobile robots (NWMRs), based on deep reinforcement learning. The model for path-following is investigated first, and then applied to the proposed reinforcement learning control strategy. The proposed control method can achieve path-following control through interacting with the environment of the set path. The path-following control method is mainly based on the design of the state and reward function in the training of the reinforcement learning. For extra obstacle avoidance problems in following, the state and reward function is redesigned by utilizing both distance and directional perspective aspects, and a minimum representative value is proposed to deal with the occurrence of multiple obstacles in the path-following environment. Through the reinforcement learning algorithm deep deterministic policy gradient (DDPG), the NWMR can gradually achieve the path it is required to follow and avoid the obstacles in simulation experiments, and the effectiveness of the proposed algorithm is verified.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Robotic Path Planning Algorithms · Reinforcement Learning in Robotics · Control and Dynamics of Mobile Robots
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Guangzhou Civil Aviation College, Guangzhou 510403, China
School of Mechanical & Automotive Engineering, South China University of Technology, Guangzhou 510641, China
Funding
Key-Area Research and Development Program of Guangdong Province
References (35)
Show source-provided references
1. Faulwasser, T., Kern, B., and Findeisen, R. (2009, January 15–18). Model predictive path-following for constrained nonlinear systems. Proceedings of the 48h IEEE Conference on Decision and Control (CDC) Held Jointly with 2009 28th Chinese Control Conference, Shanghai, China.10.1109/CDC.2009.5399744 ↗
2. Path-following control of Mecanum-wheels omnidirectional mobile robots using nonsingular terminal sliding mode10.1016/j.ymssp.2020.107128 ↗
3. Path Following of Wheeled Mobile Robots Using Online-Optimization-Based Guidance Vector Field10.1109/TMECH.2021.3077911 ↗
4. Neural adaptive command filtered control for cooperative path following of multiple underactuated autonomous underwater vehicles along one path10.1109/TSMC.2021.3062077 ↗
5. Finite-time unknown observer based coordinated path-following control of unmanned underwater vehicles10.1016/j.jfranklin.2021.01.028 ↗
6. Deep reinforcement learning for quadrotor path following with adaptive velocity10.1007/s10514-020-09951-8 ↗
7. A constrained error-based MPC for path following of quadrotor with stability analysis10.1007/s11071-019-04859-0 ↗
8. A guiding vector-field algorithm for path-following control of nonholonomic mobile robots10.1109/TCST.2017.2705059 ↗
9. Information-Aware Lyapunov-Based MPC in a Feedback-Feedforward Control Strategy for Autonomous Robots10.1109/LRA.2022.3149299 ↗
10. Subari, M.A., Hudha, K., Kadir, Z.A., Dardin, S.M.F.S.M., and Amer, N.H. (2022). Path following control of tracked vehicle using modified sup controller optimized with particle swarm optimization (PSO). Int. J. Dyn. Control., 1–10.10.1007/s40435-021-00900-6 ↗
11. Rukmana, M.A.F., Widyotriatmo, A., and Siregar, P.I. (2021, January 25–27). Anti-Jackknife Autonomous Truck Trailer for Path Following Control Using Genetic Algorithm. Proceedings of the 2021 International Conference on Instrumentation, Control, and Automation (ICA), Bandung, Indonesia.
12. Fuzzy static output feedback control for path following of autonomous vehicles with transient performance improvements10.1109/TITS.2019.2924705 ↗
13. Martinsen, A.B., and Lekkas, A.M. (2018, January 22–25). Curved path following with deep reinforcement learning: Results from three vessel models. Proceedings of the OCEANS 2018 MTS/IEEE Charleston, Charleston, SC, USA.10.1109/OCEANS.2018.8604829 ↗
14. Algorithms for reinforcement learning
15. Reinforcement learning based model-free optimized trajectory tracking strategy design for an AUV10.1016/j.neucom.2021.10.056 ↗
16. Cao, S., Sun, L., Jiang, J., and Zuo, Z. (2021). Reinforcement Learning-Based Fixed-Time Trajectory Tracking Control for Uncertain Robotic Manipulators with Input Saturation. IEEE Trans. Neural Netw. Learn. Syst.
17. Heuristic and deep reinforcement learning-based PID control of trajectory tracking in a ball-and-plate system
18. Trajectory tracking control for mobile robots using reinforcement learning and PID10.1007/s40998-019-00286-4 ↗
19. Deep reinforcement learning-based controller for path following of an unmanned surface vehicle10.1016/j.oceaneng.2019.04.099 ↗
20. Three-dimensional path-following control of a robotic airship with reinforcement learning10.1155/2019/7854173 ↗
21. Liu, M., Zhao, F., Yin, J., Niu, J., and Liu, Y. (2021). Reinforcement-Tracking: An Effective Trajectory Tracking and Navigation Method for Autonomous Urban Driving. IEEE Trans. Intell. Transp. Syst.10.1109/TITS.2021.3066366 ↗
22. Path following optimization for an underactuated USV using smoothly-convergent deep reinforcement learning10.1109/TITS.2020.2989352 ↗
23. Deterministic policy gradient with integral compensator for robust quadrotor control10.1109/TSMC.2018.2884725 ↗
24. A path-integral-based reinforcement learning algorithm for path following of an autoassembly mobile robot10.1109/TNNLS.2019.2955699 ↗
25. Chen, L., Chen, Y., Yao, X., Shan, Y., and Chen, L. (2019, January 9–12). An adaptive path tracking controller based on reinforcement learning with urban driving application. Proceedings of the 2019 IEEE Intelligent Vehicles Symposium (IV), Paris, France.10.1109/IVS.2019.8814130 ↗
26. Combined path-following and obstacle avoidance control of a wheeled robot10.1177/0278364907076790 ↗
27. Taming an autonomous surface vehicle for path following and collision avoidance using deep reinforcement learning10.1109/ACCESS.2020.2976586 ↗
28. Quadrotor Path Following and Reactive Obstacle Avoidance with Deep Reinforcement Learning
29. Kanayama, Y., Kimura, Y., Miyazaki, F., and Noguchi, T. (1990, January 13–18). A stable tracking control method for an autonomous mobile robot. Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), Cincinnati, OH, USA.
30. Nonlinear model predictive control for constrained output path following10.1109/TAC.2015.2466911 ↗
31. Sutton, R.S., and Barto, A.G. (2018). Reinforcement Learning: An Introduction, MIT Press.
32. Human-level control through deep reinforcement learning10.1038/nature14236 ↗
33. Silver, D., Lever, G., Heess, N., Degris, T., Wierstra, D., and Riedmiller, M. (2014, January 22–24). Deterministic policy gradient algorithms. Proceedings of the International Conference on Machine Learning, PMLR, Bejing, China.
34. Lillicrap, T.P., Hunt, J.J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., and Wierstra, D. (2015). Continuous control with deep reinforcement learning. arXiv.
35. Trajectory Tracking Control of Nonholonomic Wheeled Mobile Robots Using Model Predictive Control Subjected to Lyapunov-based Input Constraints10.1007/s12555-019-0814-x ↗