A Multi-Objective Optimal Control Method for Navigating Connected and Automated Vehicles at Signalized Intersections Based on Reinforcement Learning
Source abstract
Abstract
The emergence and application of connected and automated vehicles (CAVs) have played a positive role in improving the efficiency of urban transportation and achieving sustainable development. To improve the traffic efficiency at signalized intersections in a connected environment while simultaneously reducing energy consumption and ensuring a more comfortable driving experience, this study investigates a flexible and real-time control method to navigate the CAVs at signalized intersections utilizing reinforcement learning (RL). Initially, control of CAVs at intersections is formulated as a Markov Decision Process (MDP) based on the vehicles’ motion state and the intersection environment. Subsequently, a comprehensive reward function is formulated considering energy consumption, efficiency, comfort, and safety. Then, based on the established environment and the twin delayed deep deterministic policy gradient (TD3) algorithm, a control algorithm for CAVs is designed. Finally, a simulation study is conducted using SUMO, with Lankershim Boulevard as the research scenario. Results indicate that the proposed methods yield a 13.77% reduction in energy consumption and a notable 18.26% decrease in travel time. Vehicles controlled by the proposed method also exhibit smoother driving trajectories.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Traffic control and management · Traffic Prediction and Management Techniques · Transportation Planning and Optimization
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Jiangsu Key Laboratory of Urban ITS, Southeast University, Nanjing 211189, China
Department of Intelligent Transportation and Spatial Informatics, School of Transportation, Southeast University, Nanjing 211189, China
Funding
National Key R&D Program of China · Natural Science Foundation of Xizang Autonomous Region · Trans-portation Science and Technology Project of Sichuan Province · Science and technology project of Jiangsu transport
References (43)
Show source-provided references
1. Traffic coordination at road intersections: Autonomous decision-making algorithms using model-based heuristics10.1109/MITS.2016.2630585 ↗
2. A car-following model for connected and automated vehicles with heterogeneous time delays under fixed and switching communication topologies10.1109/TITS.2021.3134419 ↗
3. Deng, Z., Shi, Y., Han, Q., Lv, L., and Shen, W.M. (2020). A conflict duration graph-based coordination method for connected and automated vehicles at signal-free intersections. Appl. Sci., 10.10.3390/app10186223 ↗
4. Improving method of real-time offset tuning for arterial signal coordination using probe trajectory data10.1177/1687814016683355 ↗
5. Model for developing an eco-driving strategy of a passenger vehicle based on the least fuel consumption10.1016/j.apenergy.2008.12.017 ↗
6. Eco-approach with traffic prediction and experimental validation for connected and autonomous vehicle10.1109/TITS.2020.2972198 ↗
7. Integrated Optimization of Traffic Signals and Vehicle Trajectories at Isolated Urban Intersections10.1016/j.trb.2018.04.007 ↗
8. Eco Approaching at an Isolated Signalized Intersection under Partially Connected and Automated Vehicles Environment10.1016/j.trc.2017.04.001 ↗
9. A less-disturbed ecological driving strategy for connected and automated vehicles10.1109/TIV.2021.3112499 ↗
10. Integrated optimization of power management and vehicle motion control for autonomous hybrid electric vehicles10.1109/TVT.2023.3270127 ↗
11. Energy-optimal speed control for electric vehicles on signalized arterials10.1109/TITS.2015.2422778 ↗
12. An eco-driving system for electric vehicles with signal control under v2x environment10.1016/j.trc.2018.06.002 ↗
13. Energy-efficient adaptive cruise control for electric connected and autonomous vehicles10.1109/MITS.2019.2919556 ↗
14. Dynamic eco-driving for signalized arterial corridors and its indirect network-wide energy/emissions benefits10.1080/15472450.2012.712494 ↗
15. Differentiated speed planning for connected and automated electric vehicles at signalized intersections considering dynamic wireless power transfer
16. Eco-driving method for signalized intersection based on departure time prediction
17. An extended car-following model incorporating the effects of lateral gap and gradient10.1016/j.physa.2018.02.162 ↗
18. Traffic flow prediction with big data: A deep learning approach
19. Jiang, X., Zhang, J., and Wang, B. (2022). Energy-efficient driving for adaptive traffic signal control environment via explainable reinforcement learning. Appl. Sci., 12.10.3390/app12115380 ↗
20. Longitudinal control of connected and automated vehicles among signalized intersections in mixed traffic flow with deep reinforcement learning approach10.1016/j.physa.2023.129189 ↗
21. Developing an eco-driving application for semi-actuated signalized intersections and modeling the market penetration rates of eco-driving10.1177/0361198119839960 ↗
22. Eco-driving at signalized intersections: A parameterized reinforcement learning approach
23. Bin Al Islam, S.M.A., Abdul Aziz, H.M., Wang, H., and Young, S.E. (2018, January 7). Minimizing energy consumption from connected signalized intersections by reinforcement learning. Proceedings of the 21st International Conference on Intelligent Transportation Systems (ITSC), Maui, HI, USA.10.1109/ITSC.2018.8569891 ↗
24. Hybrid Deep Reinforcement Learning based Eco-Driving for Low-Level Connected and Automated Vehicles along Signalized Corridors10.1016/j.trc.2021.102980 ↗
25. Jointly dampening traffic oscillations and improving energy consumption with electric, connected and automated vehicles: A reinforcement learning based approach10.1016/j.apenergy.2019.114030 ↗
26. Modeling car following behavior of autonomous driving vehicles based on deep reinforcement learning
27. A study on autonomous intersection management: Planning-based strategy improved by convolutional neural network10.1007/s12205-021-2093-3 ↗
28. Tran, Q.-D., and Bae, S.-H. (2021). An Efficiency Enhancing Methodology for Multiple Autonomous Vehicles in an Urban Network Adopting Deep Reinforcement Learning. Appl. Sci., 11.10.3390/app11041514 ↗
29. Li, J., Wu, X., and Fan, J. (2022, January 1–4). Speed planning for connected and automated vehicles in urban scenarios using deep reinforcement learning. Proceedings of the 2022 IEEE Vehicle Power and Propulsion Conference (VPPC), Merced, CA, USA.10.1109/VPPC55846.2022.10003458 ↗
30. Multi-agent deep reinforcement learning for urban traffic light control in vehicular networks10.1109/TVT.2020.2997896 ↗
31. A survey of application of deep reinforcement learning in urban traffic signal control methods
32. Development of an Efficient driving strategy for connected and automated vehicles at signalized intersections: A reinforcement learning approach10.1109/TITS.2019.2942014 ↗
33. Zhuang, H., Lei, C., Chen, Y., and Tan, X. (2023). Cooperative Decision-Making for Mixed Traffic at an Unsignalized Intersection Based on Multi-Agent Reinforcement Learning. Appl. Sci., 13.10.3390/app13085018 ↗
34. Online longitudinal trajectory planning for connected and autonomous vehicles in mixed traffic flow with deep reinforcement learning approach10.1080/15472450.2022.2046472 ↗
35. Kurczveil, T., López, P.Á., and Schnieder, E. (2013, January 15–17). Implementation of an energy model and a charging infrastructure in SUMO. Proceedings of the 1st International Conference on Simulation of Urban Mobility, Berlin, Germany.
36. Safety evaluation for connected and autonomous vehicles’ exclusive lanes considering penetrate ratios and impact of trucks using surrogate safety measures
37. A platoon based cooperative eco-driving model for mixed automated and human-driven vehicles at a signalized intersection10.1016/j.trc.2018.05.025 ↗
38. Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., and Riedmiller, M. (2013). Playing atari with deep reinforcement learning. arXiv.
39. Hu, H., Wang, Y., Tong, W., Zhao, J., and Gu, Y. (2023). Path planning for autonomous vehicles in unknown dynamic environment based on deep reinforcement learning. Appl. Sci., 13.10.3390/app131810056 ↗
40. Lillicrap, T.P., Hunt, J.J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., and Wierstra, D. (2016, January 2–4). Continuous control with deep reinforcement learning. Proceedings of the International Conference on Learning Representations (ICLR), San Juan, Puerto Rico.
41. Fujimoto, S., Hoof, H., and Meger, D. (2018, January 10–15). Addressing function approximation error in actor-critic methods. Proceedings of the International Conference on Machine Learning (ICML), Stockholm, Sweden.
42. Erdmann, J. (2014, January 15). Lane-changing model in SUMO. Proceedings of the SUMO 2014, Berlin, Germany.
43. Garcia, A.G., Tria, L.A.R., and Talampas, M.C.R. (2019, January 19–21). Development of an energy-efficient routing algorithm for electric vehicles. Proceedings of the IEEE Transportation Electrification Conference and Expo (ITEC), Detroit, MI, USA.10.1109/ITEC.2019.8790491 ↗