Routing Control Optimization for Autonomous Vehicles in Mixed Traffic Flow Based on Deep Reinforcement Learning
Source abstract
Abstract
With recent technological advancements, the commercialization of autonomous vehicles (AVs) is expected to be realized soon. However, it is anticipated that a mixed traffic of AVs and human-driven vehicles (HVs) will persist for a considerable period until the Market Penetration Rate reaches 100%. During this phase, AVs and HVs will interact and coexist on the roads. Such an environment can cause unpredictable and dynamic traffic conditions due to HVs, which results in traffic problems including traffic congestion. Therefore, the routes of AVs must be controlled in a mixed traffic environment. This study proposes a multi-objective vehicle routing control method using a deep Q-network to control the driving direction at intersections in a mixed traffic environment. The objective is to distribute the traffic flow and control the routes safely and efficiently to their destination. Simulation results showed that the proposed method outperformed existing methods in terms of the driving distance, time, and waiting time of AVs, particularly in more dynamic traffic environments. Consequently, the traffic became smooth as it moved along optimal routes.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Traffic control and management · Traffic Prediction and Management Techniques · Autonomous Vehicle Technology and Safety
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Department of IT Engineering, Sookmyung Women’s University, Seoul 04310, Republic of Korea
Division of Artificial Intelligence Engineering, Sookmyung Women’s University, Seoul 04310, Republic of Korea
Department of Highway & Transportation Research, Korea Institute of Civil Engineering and Building Technology, Goyang 10223, Republic of Korea
Funding
Korea Institute of Civil Engineering and Building Technology (KICT) · National Research Foundation of Korea (NRF)
References (26)
Show source-provided references
1. National Highway Traffic Safety Administration (2016). Federal Automated Vehicles Policy: Accelerating the Next Revolution in Roadway Safety.
2. Connected and Autonomous Vehicles and Infrastructures: A Literature Review10.1007/s42947-021-00130-1 ↗
3. Alexander, D., and Gartner, J. Self-Driving Vehicles, Autonomous Parking, and Other Advanced Driver Assistance Systems, Global Market Analysis and Forecasts, 2013.
4. Alonso, R.M., Ciuffo, B., Makridis, M., and Thiel, C. (2017). The Revolution of Driving: From Connected Vehicles to Coordinated Automated Road Transport (C-ART), Publications Office of the European Union.
5. Park, S., and Ritchie, S.G. (2004, January 11–15). Exploring the Relationship Between Freeway Speed Variance, Lane Changing, and Vehicle Heterogeneity. Proceedings of the 83rd Annual Meeting of Transportation Research Board, Washington, DC, USA.
6. A Survey on the Electric Vehicle Routing Problem: Variants and Solution Approaches
7. Vehicle Routing Problems over Time: A Survey10.1007/s10479-021-04488-0 ↗
8. Chen, D., Jiang, L., Wang, Y., and Li, Z. (2020, January 1–3). Autonomous Driving using Safe Reinforcement Learning by Incorporating a Regret-based Human Lane-Changing Decision Model. Proceedings of the 2020 American Control Conference (ACC), Denver, CO, USA.10.23919/ACC45564.2020.9147626 ↗
9. Human-Level Control through Deep Reinforcement Learning10.1038/nature14236 ↗
10. Robust Dynamic Multi-Objective Vehicle Routing Optimization Method
11. Solving the Open Vehicle Routing Problem with Capacity and Distance Constraints with A Biased Random Key Genetic Algorithm
12. Ant Colony Optimized Routing Strategy for Electric Vehicles
13. Shi, T., Wang, P., Cheng, X., Chan, C.Y., and Huang, D. (2019, January 27–30). Driving Decision and Control for Automated Lane Change Behavior based on Deep Reinforcement Learning. Proceedings of the 2019 IEEE Intelligent Transportation Systems Conference (ITSC), Auckland, New Zealand.10.1109/ITSC.2019.8917392 ↗
14. Ye, F., Cheng, X., Wang, P., Chan, C.Y., and Zhang, J. (November, January 19). Automated Lane Change Strategy using Proximal Policy Optimization-based Deep Reinforcement Learning. Proceedings of the IEEE Intelligent Vehicles Symposium (IV), Las Vegas, NV, USA.
15. Space-weighted Information Fusion Using Deep Reinforcement Learning: The Context of Tactical Control of Lane-changing Autonomous Vehicles and Connectivity Range Assessment10.1016/j.trc.2021.103192 ↗
16. Gu, Y., Yuan, K., Yang, S., Ning, M., and Huang, Y. (2022, January 28–30). Mandatory Lane-Changing Decision-Making in Dense Traffic for Autonomous Vehicles based on Deep Reinforcement Learning. Proceedings of the 2022 6th CAA International Conference on Vehicular Control and Intelligence (CVCI), Nanjing, China.10.1109/CVCI56766.2022.9964906 ↗
17. Zhao, W., Guo, H., Zhao, X., and Dai, Q. (2020, January 6–8). Intelligent Vehicle Path Planning Based on Q-Learning Algorithm with Consideration of Smoothness. Proceedings of the 2020 Chinese Automation Congress (CAC), Shanghai, China.10.1109/CAC51589.2020.9326831 ↗
18. Koh, S.S., Zhou, B., Yang, P., Yang, Z., Fang, H., and Feng, J. (2018, January 28–30). Reinforcement Learning for Vehicle Route Optimization in SUMO. Proceedings of the 2018 IEEE 20th International Conference on High Performance Computing and Communications, Exeter, UK.10.1109/HPCC/SmartCity/DSS.2018.00242 ↗
19. Real-Time Deep Reinforcement Learning based Vehicle Navigation10.1016/j.asoc.2020.106694 ↗
20. Trajectory Planning and Control of Autonomous Vehicles for Static Vehicle Avoidance in Dynamic Traffic Environments10.1109/ACCESS.2023.3236816 ↗
21. A Hybrid Motion Planning Framework for Autonomous Driving in Mixed Traffic Flow
22. Learning-Based Adaptive Optimal Control for Connected Vehicles in Mixed Traffic: Robustness to Driver Reaction Time10.1109/TCYB.2020.3029077 ↗
23. Lopez, P.A., Behrisch, M., Walz, L.B., Erdmann, J., Flötteröd, Y.P., Hilbrich, R., Lücken, L., Rummel, J., Wagner, P., and Wiessner, E. (2018, January 4–7). Microscopic Traffic Simulation using SUMO. Proceedings of the 21st International Conference on Intelligent Transportation Systems (ITSC), Maui, HI, USA.10.1109/ITSC.2018.8569938 ↗
24. Congested Traffic States in Empirical Observations and Microscopic Simulations10.1103/PhysRevE.62.1805 ↗
25. A Survey on Autonomous Vehicle Control in the Era of Mixed-Autonomy: From Physics-Based to AI-Guided Driving Policy Learning10.1016/j.trc.2021.103008 ↗
26. Sungwon, M., Seolwon, K., and Yujin, L. (2024, January 5–8). Real-Time Trajectory Control for Vehicle based on Deep Reinforcement Learning. Proceedings of the IEEE 42nd International Conference on Consumer Electronics, Las Vegas, NV, USA.