Vehicle Lateral Control Based on Augmented Lagrangian DDPG Algorithm
Source abstract
Abstract
This paper studies the safe trajectory tracking control of intelligent vehicles, which is still an open and challenging problem. A deep reinforcement learning algorithm based on augmented Lagrangian safety constraints is proposed to the lateral control of vehicle trajectory tracking. First, the tracking control of intelligent vehicles is described as a reinforcement learning process based on the Constrained Markov Decision Process (CMDP). The actor-critic neural network based reinforcement learning framework is established and the environment of reinforcement learning is designed to include the vehicle model, tracking model, road model and reward function. Secondly, the augmented Lagrangian Deep Deterministic Policy Gradient (DDPG) method is proposed for updating, in which a replay separation buffer method is used to solve the problem of sample correlation, and a neural network with the same structure is copied to solve the update divergence problem. Finally, a vehicle lateral control approach is obtained, whose effectiveness and advantages over existing results are verified through simulation results.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Simulation and Modeling Applications · Advanced Algorithms and Applications
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Key Laboratory of Smart Manufacturing in Energy Chemical Process, Ministry of Education, East China University of Science and Technology, Shanghai 200237, China
References (26)
Show source-provided references
1. Wang, M., Niu, C., Wang, Z., Jiang, Y., Jian, J., and Tang, X. (2024). Model and Parameter Adaptive MPC Path Tracking Control Study of Rear-Wheel-Steering Agricultural Machinery. Agriculture, 14.10.3390/agriculture14060823 ↗
2. Tan, Q., Dai, P., Zhang, Z., and Katupitiya, J. (2018). MPC and PSO Based Control Methodology for Path Tracking of 4WS4WD Vehicles. Appl. Sci., 8.10.3390/app8061000 ↗
3. Zou, T., You, Y., Meng, H., and Chang, Y. (2022). Research on Six-Wheel Distributed Unmanned Vehicle Path Tracking Strategy Based on Hierarchical Control. Biomimetics, 7.10.3390/biomimetics7040238 ↗
4. Hajjami, L.E., Mellouli, E., Žuraulis, V., Berrada, M., and Boumhidi, I. (2023). A Robust Intelligent Controller for Autonomous Ground Vehicle Longitudinal Dynamics. Appl. Sci., 13.10.3390/app13010501 ↗
5. Wang, Y., Wang, Z., Shi, D., Chu, F., Guo, J., and Wang, J. (2024). Optimized Longitudinal and Lateral Control Strategy of Intelligent Vehicles Based on Adaptive Sliding Mode Control. World Electr. Veh. J., 15.10.3390/wevj15090387 ↗
6. Oh, K., and Seo, J. (2023). Development of a Sliding-Mode-Control-Based Path-Tracking Algorithm with Model-Free Adaptive Feedback Action for Autonomous Vehicles. Sensors, 23.10.3390/s23010405 ↗
7. A Reinforcement Learning-Based Adaptive Path Tracking Approach for Autonomous Driving10.1109/TVT.2020.3014628 ↗
8. Improved DDPG Algorithm Based on Offline Model Pre-Training Learning
9. Luo, Z., Zhou, J., and Wen, G. (2022, January 4–7). Deep Reinforcement Learning Based Tracking Control of Unmanned Vehicle with Safety Guarantee. Proceedings of the 2022 13th Asian Control Conference (ASCC), Jeju Island, Republic of Korea.10.23919/ASCC56756.2022.9828057 ↗
10. Special Vehicle Route Optimization Strategy Based on Route Search DQN
11. Deep Reinforcement Learning Based Path Tracking Controller for Autonomous Vehicle10.1177/0954407020954591 ↗
12. Application of SAC-Based Autonomous Vehicle Control Method
13. Ye, B.L., Wang, X., Li, L.X., and Wu, W.M. (2024). Vehicle Intelligent Control Method Based on Deep Reinforcement Learning PPO. Comput. Eng., 1–14.
14. Mnih, V. (2013). Playing Atari with Deep Reinforcement Learning. arXiv.
15. Haarnoja, T., Zhou, A., Abbeel, P., and Levine, S. (2018, January 10–15). Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor. Proceedings of the International Conference on Machine Learning, PMLR, Stockholm, Sweden.
16. Fujimoto, S., Hoof, H., and Meger, D. (2018, January 10–15). Addressing Function Approximation Error in Actor-Critic Methods. Proceedings of the International Conference on Machine Learning, PMLR, Stockholm, Sweden.
17. Lillicrap, T.P. (2015). Continuous Control with Deep Reinforcement Learning. arXiv.
18. Hassan, I.A., Ragheb, H., Sharaf, A.M., and Attia, T. (2024, January 21–23). Reinforcement Learning for Precision Navigation: DDQN-Based Trajectory Tracking in Unmanned Ground Vehicles. Proceedings of the 2024 14th International Conference on Electrical Engineering (ICEENG), Cairo, Egypt.10.1109/ICEENG58856.2024.10566281 ↗
19. Li, M., Liu, H., Wang, H., and Xia, M. (2023, January 26–30). Trustworthy Dynamic Object Tracking Using Deep Reinforcement Learning with the Self-Attention Mechanism. Proceedings of the 2023 IEEE 19th International Conference on Automation Science and Engineering (CASE), Auckland, New Zealand.10.1109/CASE56687.2023.10260504 ↗
20. Srikonda, S., Norris, W.R., Nottage, D., and Soylemezoglu, A. (2022). Deep Reinforcement Learning for Autonomous Dynamic Skid Steer Vehicle Trajectory Tracking. Robotics, 11.10.3390/robotics11050095 ↗
21. Gu, S., Yang, L., Du, Y., Chen, G., Walter, F., Wang, J., and Knoll, A. (2022). A Review of Safe Reinforcement Learning: Methods, Theory and Applications. arXiv.
22. Carr, S., Jansen, N., Junges, S., and Topcu, U. (2023, January 7–14). Safe Reinforcement Learning via Shielding under Partial Observability. Proceedings of the AAAI Conference on Artificial Intelligence, Washington, DC, USA.10.1609/aaai.v37i12.26723 ↗
23. Simão, T.D., Suilen, M., and Jansen, N. (2023, January 7–14). Safe Policy Improvement for POMDPs via Finite-State Controllers. Proceedings of the AAAI Conference on Artificial Intelligence, Washington, DC, USA.10.1609/aaai.v37i12.26763 ↗
24. Park, J., Seo, Y., Shin, J., Lee, H., Abbeel, P., and Lee, K. (2022). SURF: Semi-Supervised Reward Learning with Data Augmentation for Feedback-Efficient Preference-Based Reinforcement Learning. arXiv.
25. Safe Exploration in Model-Based Reinforcement Learning Using Control Barrier Functions10.1016/j.automatica.2022.110684 ↗
26. (2018). Passenger Cars: Test Tracks for a Severe Lane-Change Manoeuvre: Part 1: Double Lane-Change (Standard No. ISO 3888-1:1999).