Reinforcement-Learning-Based Trajectory Learning in Frenet Frame for Autonomous Driving
Source abstract
Abstract
Autonomous driving is a complex problem that requires intelligent decision making, and it has recently garnered significant interest due to its potential advantages in convenience and safety. In autonomous driving, conventional path planning to reach a destination is a time-consuming challenge. Therefore, learning-based approaches have been successfully applied to the controller or decision-making aspects of autonomous driving. However, these methods often lack explainability, as passengers cannot discern where the vehicle is headed. Additionally, most experiments primarily focus on highway scenarios, which do not effectively represent road curvature. To address these issues, we propose a reinforcement-learning-based trajectory learning in the Frenet frame (RLTF), which involves learning trajectories in the Frenet frame. Learning trajectories enable the consideration of future states and enhance explainability. We demonstrate that RLTF achieves a 100% success rate in the simulation environment, considering future states on curvy roads with continuous obstacles while overcoming issues associated with the Frenet frame.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Autonomous Vehicle Technology and Safety · Robotic Path Planning Algorithms · Reinforcement Learning in Robotics
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Department of Artificial Intelligence, Chung-Ang University, Seoul 06974, Republic of Korea
Autonomous Driving Center, R&D Division, Hyundai Motor Company, Gyeonggi 13449, Republic of Korea
Funding
Autonomous Driving Center, R&D Division, Hyundai Motor Company · the Chung-Ang University Research Grants in 2023
References (40)
Show source-provided references
1. Motion planning for autonomous driving: The state of the art and future perspectives10.1109/TIV.2023.3274536 ↗
2. Maurer, M., Gerdes, J.C., Lenz, B., and Winner, H. (2016). Autonomous Driving: Technical, Legal and Social Aspects, Springer Nature.10.1007/978-3-662-48847-8 ↗
3. A survey of motion planning and control techniques for self-driving urban vehicles10.1109/TIV.2016.2578706 ↗
4. Self-driving cars: A survey10.1016/j.eswa.2020.113816 ↗
5. Botezatu, A.P., Burlacu, A., and Orhei, C. (2024). A Review of Deep Learning Advancements in Road Analysis for Autonomous Driving. Appl. Sci., 14.10.3390/app14114705 ↗
6. A Survey of Deep Learning Applications to Autonomous Vehicle Control10.1109/TITS.2019.2962338 ↗
7. Rausch, V., Hansen, A., Solowjow, E., Liu, C., Kreuzer, E., and Hedrick, J.K. (2017, January 24–26). Learning a Deep Neural Net Policy for End-to-End Control of Autonomous Vehicles. Proceedings of the 2017 American Control Conference (ACC), Seattle, WA, USA.10.23919/ACC.2017.7963716 ↗
8. Codevilla, F., Müller, M., López, A., Koltun, V., and Dosovitskiy, A. (2018, January 21–25). End-to-end Driving via Conditional Imitation Learning. Proceedings of the 2018 IEEE International Conference on Robotics and Automation (ICRA), Brisbane, Australia.10.1109/ICRA.2018.8460487 ↗
9. Li, Y., Zhu, Z., and Li, X. (2024). Reinforcement Learning Based Speed Control with Creep Rate Constraints for Autonomous Driving of Mining Electric Locomotives. Appl. Sci., 14.10.20944/preprints202404.1512.v1 ↗
10. Highway Decision-Making and Motion Planning for Autonomous Driving via Soft Actor-Critic10.1109/TVT.2022.3151651 ↗
11. Efficient Deep Reinforcement Learning With Imitative Expert Priors for Autonomous Driving10.1109/TNNLS.2022.3142822 ↗
12. Chen, Y., Dong, C., Palanisamy, P., Mudalige, P., Muelling, K., and Dolan, J.M. (2019, January 16–17). Attention-Based Hierarchical Deep Reinforcement Learning for Lane Change Behaviors in Autonomous Driving. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, Long Beach, CA, USA.10.1109/CVPRW.2019.00172 ↗
13. Li, D., Zhao, D., Zhang, Q., and Chen, Y. (2018). Reinforcement Learning and Deep Learning based Lateral Control for Autonomous Driving. arXiv.
14. Leurent, E. (2024, August 06). A Survey of State-Action Representations for Autonomous Driving. Available online: https://hal.science/hal-01908175.
15. Autonomous driving on curvy roads without reliance on frenet frame: A cartesian-based trajectory planning method10.1109/TITS.2022.3145389 ↗
16. A Review of Motion Planning Techniques for Automated Vehicles10.1109/TITS.2015.2498841 ↗
17. Randomized kinodynamic planning10.1177/02783640122067453 ↗
18. Probabilistic roadmaps for path planning in high-dimensional configuration spaces10.1109/70.508439 ↗
19. Werling, M., Ziegler, J., Kammel, S., and Thrun, S. (2010, January 3–7). Optimal Trajectory Generation for Dynamic Street Scenarios in a Frenet Frame. Proceedings of the 2010 IEEE International Conference on Robotics and Automation, Anchorage, AK, USA.10.1109/ROBOT.2010.5509799 ↗
20. Local path planning for off-road autonomous driving with avoidance of static obstacles10.1109/TITS.2012.2198214 ↗
21. Farag, W., and Saleh, Z. (2018, January 10–20). Behavior Cloning for Autonomous Driving using Convolutional Neural Networks. Proceedings of the 2018 International Conference on Innovation and Intelligence for Informatics, Computing, and Technologies (3ICT), Sakhier, Bahrain.10.1109/3ICT.2018.8855753 ↗
22. Kicki, P., Gawron, T., and Skrzypczyński, P. (2020). A Self-Supervised Learning Approach to Rapid Path Planning for Car-Like Vehicles Maneuvering in Urban Environment. arXiv.
23. Zhang, J., Kong, A., Tang, Y., Lv, Z., Guo, L., and Hang, P. (2024). Application of Data-Driven Model Predictive Control for Autonomous Vehicle Steering. arXiv.
24. Wang, J., Zheng, Y., Xu, Q., and Li, K. (2022, January 8–10). Data-Driven Predictive Control for Connected and Autonomous Vehicles in Mixed Traffic. Proceedings of the American Control Conference, ACC 2022, Atlanta, GA, USA.10.23919/ACC53348.2022.9867378 ↗
25. Vu, T.M., Moezzi, R., Cyrus, J., and Hlava, J. (2021). Model Predictive Control for Autonomous Driving Vehicles. Electronics, 10.10.3390/electronics10212593 ↗
26. Chen, Z., Lai, J., Li, P., Awad, O.I., and Zhu, Y. (2024). Prediction Horizon-Varying Model Predictive Control (MPC) for Autonomous Vehicle Control. Electronics, 13.10.21203/rs.3.rs-3850749/v1 ↗
27. Yu, S., Sheng, E., Zhang, Y., Li, Y., Chen, H., and Hao, Y. (2022). Efficient Nonlinear Model Predictive Control of Automated Vehicles. Mathematics, 10.10.3390/math10214163 ↗
28. Williams, G., Drews, P., Goldfain, B., Rehg, J.M., and Theodorou, E.A. (2016, January 16–21). Aggressive Driving with Model Predictive Path Integral Control. Proceedings of the 2016 IEEE International Conference on Robotics and Automation (ICRA), Stockholm, Sweden.10.1109/ICRA.2016.7487277 ↗
29. Information-theoretic model predictive control: Theory and applications to autonomous driving10.1109/TRO.2018.2865891 ↗
30. Deep Reinforcement Learning for Autonomous Driving: A Survey10.1109/TITS.2021.3054625 ↗
31. Chen, J., Yuan, B., and Tomizuka, M. (2019, January 27–30). Model-Free Deep Reinforcement Learning for Urban Autonomous Driving. Proceedings of the 2019 IEEE Intelligent Transportation Systems Conference (ITSC), Auckland, New Zealand.10.1109/ITSC.2019.8917306 ↗
32. Wang, P., Li, H., and Chan, C.Y. (2019, January 9–12). Continuous Control for Automated Lane Change Behavior Based on Deep Deterministic Policy Gradient Algorithm. Proceedings of the 2019 IEEE Intelligent Vehicles Symposium (IV), Paris, France.10.1109/IVS.2019.8813903 ↗
33. Naveed, K.B., Qiao, Z., and Dolan, J.M. (2021, January 19–22). Trajectory Planning for Autonomous Vehicles Using Hierarchical Reinforcement Learning. Proceedings of the 2021 IEEE International Intelligent Transportation Systems Conference (ITSC), Indianapolis, IN, USA.10.1109/ITSC48978.2021.9564634 ↗
34. Ye, L., Zhou, Z., and Wang, J. (2023). Improving the Generalizability of Trajectory Prediction Models with Frenet-Based Domain Normalization. arXiv.10.1109/ICRA48891.2023.10160788 ↗
35. Meyer, E., Brenner, M., Zhang, B., Schickert, M., Musani, B., and Althoff, M. (2023). Geometric deep learning for autonomous driving: Unlocking the power of graph neural networks with CommonRoad-Geometric. arXiv.10.1109/IV55152.2023.10186741 ↗
36. Mirchevska, B., Werling, M., and Boedecker, J. (2023). Optimizing trajectories for highway driving with offline reinforcement learning. Front. Future Transp., 4.10.3389/ffutr.2023.1076439 ↗
37. Moghadam, M., Alizadeh, A., Tekin, E., and Elkaim, G.H. (2021, January 23–27). A Deep Reinforcement Learning Approach for Long-Term Short-Term Planning on Frenet Frame. Proceedings of the 2021 IEEE 17th International Conference on Automation Science and Engineering (CASE), Lyon, France.10.1109/CASE49439.2021.9551598 ↗
38. Choi, S., Lee, K., and Oh, S. (2016, January 9–14). Gaussian Random Paths for Real-Time Motion Planning. Proceedings of the 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Daejeon, Republic of Korea.10.1109/IROS.2016.7759237 ↗
39. Andersen, H., Chong, Z.J., Eng, Y.H., Pendleton, S., and Ang, M.H. (2016, January 12–15). Geometric Path Tracking Algorithm for Autonomous Driving in Pedestrian Environment. Proceedings of the 2016 IEEE International Conference on Advanced Intelligent Mechatronics (AIM), Banff, AB, Canada.10.1109/AIM.2016.7577010 ↗
40. Pinneri, C., Sawant, S., Blaes, S., Achterhold, J., Stueckler, J., Rolinek, M., and Martius, G. (2021, January 8–11). Sample-Efficient Cross-Entropy Method for Real-Time Planning. Proceedings of the Conference on Robot Learning, London, UK. PMLR.