Deep Reinforcement Learning for Vehicle Platooning at a Signalized Intersection in Mixed Traffic with Partial Detection
Source abstract
Abstract
The intersection management system can increase traffic capacity, vehicle safety, and the smoothness of all vehicle movement. Platoons of connected vehicles (CVs) use communication technologies to share information with each other and with infrastructures. In this paper, we proposed a deep reinforcement learning (DRL) model that applies to vehicle platooning at an isolated signalized intersection with partial detection. Moreover, we identified hyperparameters and tested the system with different numbers of vehicles (1, 2, and 3) in the platoon. To compare the effectiveness of the proposed model, we implemented two benchmark options, actuated traffic signal control (ATSC) and max pressure (MP). The experimental results demonstrated that the DRL model has many outstanding advantages compared to other models. Through the learning process, the average waiting time of vehicles in the DRL method was improved by 20% and 28% compared with the ATSC and MP options. The results also suggested that the DRL model is effective when the CV penetration rate is over 20%.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Traffic control and management · Traffic Prediction and Management Techniques · Traffic and Road Safety
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Smart Transportation Lab, Department of Spatial Information Engineering, Pukyong National University, Busan 48513, Korea
Faculty of Civil Engineering, Nha Trang University, Nha Trang 57000, Vietnam
References (55)
Show source-provided references
1. Urban traffic congestion: A search for new solutions
2. Socio economic valuation of traffic delays10.1016/j.trpro.2016.11.104 ↗
3. Large scale simulation of CO2 emissions caused by urban car traffic: An agent-based network approach10.1016/j.jclepro.2018.02.113 ↗
4. Vehicle-to-vehicle (V2V) and vehicle-to-infrastructure (V2I) communication in a heterogeneous wireless network—Performance evaluation10.1016/j.trc.2016.03.008 ↗
5. Cooperative intersection management: A survey10.1109/TITS.2015.2471812 ↗
6. Intelligent connected vehicles: The industrial practices and impacts on automotive value-chains in China10.1080/13602381.2017.1340178 ↗
7. Future automotive architecture and the impact of IT trends10.1109/MS.2017.69 ↗
8. Connected & autonomous vehicles—Environmental impacts—A review10.1016/j.scitotenv.2019.135237 ↗
9. Traffic capacity implications of automated vehicles mixed with regular vehicles10.1080/15472450.2017.1404680 ↗
10. Car-following model of connected and autonomous vehicles considering both average headway and electronic throttle angle10.1142/S0217984921502572 ↗
11. Human-like car-following model for autonomous vehicles considering the cut-in behavior of other vehicles in mixed traffic10.1016/j.aap.2019.105260 ↗
12. Evaluating the safety impact of connected and autonomous vehicles on motorways10.1016/j.aap.2018.12.019 ↗
13. Impact of cooperative adaptive cruise control on multilane freeway merge capacity10.1080/15472450.2018.1438275 ↗
14. Intersection management via vehicle connectivity: The intersection cooperative adaptive cruise control system concept10.1080/15472450.2014.889918 ↗
15. The impact of cooperative adaptive cruise control on traffic-flow characteristics10.1109/TITS.2006.884615 ↗
16. Segata, M. (2017, January 8–10). Platooning in SUMO: An open-source Implementation. Proceedings of the SUMO User Conference 2017, Berlin, Germany.
17. Segata, M., Joerer, S., Bloessl, B., Sommer, C., Dressler, F., and Cigno, R.L. (2014, January 3–5). Plexe: A Platooning Extension for Veins. Proceedings of the 2014 IEEE Vehicular Networking Conference, Paderborn, Germany.10.1109/VNC.2014.7013309 ↗
18. A Taxonomy of Optimization Factors for Platooning10.1109/TITS.2020.2994537 ↗
19. An autonomous connected platoon-based system for city-logistics: Development and examination of travel time aspects
20. Liang, K.Y., Martensson, J., and Johansson, K.H. (2016, January 1–4). Experiments on Platoon Formation of Heavy Trucks in Traffic. Proceedings of the 2016 IEEE 19th International Conference on Intelligent Transportation Systems (ITSC), Macau, China.10.1109/ITSC.2016.7795804 ↗
21. The impact of flexible platoon formation operations10.1109/TIV.2019.2955898 ↗
22. Heinovski, J., and Dressler, F. (2018, January 5–7). Platoon Formation: Optimized Car to Platoon Assignment Strategies and Protocols. Proceedings of the 2018 IEEE Vehicular Networking Conference (VNC), Taipei, Taiwan.10.1109/VNC.2018.8628396 ↗
23. A Platoon Formation Framework in a Mixed Traffic Environment10.1109/LCSYS.2021.3092188 ↗
24. Flow-aware platoon formation of Connected Automated Vehicles in a mixed traffic with human-driven vehicles10.1016/j.trc.2021.103442 ↗
25. Platoon management with cooperative adaptive cruise control enabled by VANET
26. Simulation Framework for Vehicle Platooning and Car-following behaviors under Connected-Vehicle Environment10.1016/j.sbspro.2013.08.105 ↗
27. Analytical analysis of the effect of maximum platoon size of connected and automated vehicles10.1016/j.trc.2020.102882 ↗
28. Branke, J., Goldate, P., and Prothmann, H. (2007, January 18–20). Actuated Traffic Signal Optimisation Using Evolutionary Algorithms. Proceedings of the 6th European Congress and Exhibition on Intelligent Transport Systems and Services (ITS07), Aalborg, Denmark.
29. Taale, H. (2000, January 6–9). Optimising Traffic Signal Control with Evolutionary Algorithms. Proceedings of the 7th World Congress on Intelligent Transport Systems, Turin, Italy.
30. Max pressure control of a network of signalized intersections10.1016/j.trc.2013.08.014 ↗
31. Adaptive max pressure control of network of signalized intersections10.1016/j.ifacol.2016.10.366 ↗
32. Ferreira, M., Fernandes, R., Conceição, H., Viriyasitavat, W., and Tonguz, O.K. (2010, January 20–24). Self-Organized Traffic Control. Proceedings of the seventh ACM international workshop on Vehicular Internetworking, Chicago, IL, USA.10.1145/1860058.1860077 ↗
33. Lowrie, P.R. (1990). SCATS: Sydney Co-Ordinated Adaptive Traffic System: A Traffic Responsive Method of Controlling Urban Traffic, Roads and Traffic Authority NSW.
34. Hunt, P., Robertson, D.I., Bretherton, R.D., and Winton, R.I. (1981). SCOOT-a Traffic Responsive Method of Coordinating Signals, Transport and Road Research Laboratory (TRRL).
35. ACS-lite algorithmic architecture: Applying adaptive control system technology to closed-loop traffic signal control systems10.3141/1856-19 ↗
36. Traffic flow management of autonomous vehicles using deep reinforcement learning and smart rerouting10.1109/ACCESS.2021.3063463 ↗
37. A deep q learning network for traffic lights’ cycle control in vehicular networks10.1109/TVT.2018.2890726 ↗
38. Vidali, A., Crociani, L., Vizzari, G., and Bandini, S. (2019, January 26–28). A deep reinforcement learning approach to adaptive traffic lights management. Proceedings of the WOA, Parma, Italy.
39. Tran, Q.D., and Bae, S.H. (2020). Proximal policy optimization through a deep reinforcement learning framework for multiple autonomous vehicles at a non-signalized intersection. Appl. Sci., 10.10.3390/app10165722 ↗
40. A reinforcement learning-based vehicle platoon control strategy for reducing energy consumption in traffic oscillations10.1109/TNNLS.2021.3071959 ↗
41. Hybrid reinforcement learning-based eco-driving strategy for connected and automated vehicles at signalized intersections10.1109/TITS.2022.3145798 ↗
42. Development of an efficient driving strategy for connected and automated vehicles at signalized intersections: A reinforcement learning approach10.1109/TITS.2019.2942014 ↗
43. Reinforcement learning-based control of signalized intersections having platoons10.1109/ACCESS.2022.3149161 ↗
44. Deep reinforcement learning aided platoon control relying on V2X information10.1109/TVT.2022.3161585 ↗
45. Liu, T., Lei, L., Zheng, K., and Zhang, K. (2022). Autonomous platoon control with integrated deep reinforcement learning and dynamic programming. arXiv.
46. Using reinforcement learning with partial vehicle detection for intelligent traffic signal control10.1109/TITS.2019.2958859 ↗
47. Ducrocq, R., and Farhi, N. (2021). Deep reinforcement Q-learning for intelligent traffic signal control with partial detection. arXiv.
48. Lopez, P.A., Behrisch, M., Bieker-Walz, L., Erdmann, J., Flotterod, Y.P., Hilbrich, R., Lucken, L., Rummel, J., Wagner, P., and Wießner, E. (2018, January 4–7). Microscopic Traffic Simulation using SUMO. Proceedings of the 2018 21st IEEE International Conference on Intelligent Transportation Systems, Maui, HI, USA.10.1109/ITSC.2018.8569938 ↗
49. Institute of Transportation Systems of the German Aerospace Center (2021, August 01). Traffic Control Interface-SUMO Documentation. Available online: https://sumo.dlr.de/docs/Simpla.html.
50. Wegener, A., Piorkowski, M., Raya, M., Hellbruck, H., Fischer, S., and Hubaux, J.P. (2008, January 14–17). TraCI: An Interface for Coupling Road Traffic and Network Simulators. Proceedings of the 11th Communications and Networking Simulation Symposium (CNS), Ottawa, ON, Canada.10.1145/1400713.1400740 ↗
51. Krauss, S. (1998). Microscopic Modeling of Traffic Flow: Investigation of Collision Free Vehicle Dynamics. [Ph.D. Thesis, University of Cologne].
52. Genders, W., and Razavi, S. (2016). Using a deep reinforcement learning agent for traffic signal control. arXiv.
53. Kingma Diederik, P., and Adam, J.B. (2015, January 7–9). A Method for Stochastic Optimization. Proceedings of the 3rd International Conference for Learning Representations, San Diego, CA, USA.
54. Poole, D., and Mackworth, A. (2017). Artificial Intelligence: Foundations of Computational Agents, Cambridge University Press. Available online: https://artint.info/2e/html/ArtInt2e.Ch12.S6.html.10.1017/9781108164085 ↗
55. Gettman, D., Folk, E., Curtis, E., Ormand, K.K.D., Mayer, M., and Flanigan, E. (2013). Measure of Effectiveness and Validation Guidance for Adaptive Signal Control Technologies.