Adaptive Traffic Signal Control Using Deep Reinforcement Learning with Noise Injection
Source abstract
Abstract
Adaptive traffic signal control (ATSC) remains a critical challenge for urban mobility. In this direction, deep reinforcement learning (DRL) has been widely investigated for ATSC, showing promising improvements in simulated environments. However, a noticeable gap remains between simulation-based results and practical implementations, due to reward formulations that do not address phase instability. Stochastic variations may trigger premature phase changes (“flickers”), affecting signal behavior and potentially limiting deployment in real scenarios. Although several works have examined delay, queues, and decentralized coordination, stability-focused variables remain comparatively less explored, particularly in single yet complex intersections. This study proposes a decentralized DRL model for ATSC with noise injection (ATSC-DRLNI) applied to a single intersection, introducing a stability-oriented reward function that integrates flickers, queue length, and advantage actor-critic (A2C) learning feedback. The model is evaluated in the Simulation of Urban MObility (SUMO) platform and compared against seven baseline methods, using real traffic data from a Mexican city for calibration and validation. Results suggest that penalizing flickers may contribute to more stable phase transitions, while reductions of up to 40% in queue length were observed in heavy-traffic scenarios. These findings indicate that incorporating stability-related variables into reward functions may help in implementing DRL-based ATSC studies.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Traffic control and management · Traffic Prediction and Management Techniques · Traffic and Road Safety
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Instituto de Ingeniería, Universidad Nacional Autónoma de México, Ciudad de México 04510, Mexico
Escuela Nacional de Estudios Superiores Unidad Morelia, Universidad Nacional Autónoma de México, Morelia 58190, Mexico
Funding
Programa de Apoyo a Proyectos de Investigación e Innovación Tecnológica (PAPIIT), DGAPA-UNAM
References (79)
Show source-provided references
1. Urban traffic congestion: Its economic and social causes and consequences10.18356/f43aec5d-en ↗
2. Bull, A., United Nations, Economic Commission for Latin America and the Caribbean, and Deutsche Gesellschaft für Technische Zusammenarbeit (2003). Traffic Congestion: The Problem and How to Deal with It, Cuadernos de la CEPAL, United Nations, Economic Commission for Latin America and the Caribbean.
3. González-Aliste, P., Derpich, I., and López, M. (2023). Reducing Urban Traffic Congestion via Charging Price. Sustainability, 15.10.3390/su15032086 ↗
4. Herath Bandara, S.J., and Thilakarathne, N. (2025). Economic and Public Health Impacts of Transportation-Driven Air Pollution in South Asia. Sustainability, 17.10.3390/su17052306 ↗
5. Krzyzanowski, M., Kuna-Dibbert, B., and Schneider, J. (2005). Health Effects of Transport-Related Air Pollution, WHO Regional Office Europe.
6. Air pollution and health risks due to vehicle traffic10.1016/j.scitotenv.2013.01.074 ↗
7. INRIX (2025). 2025 INRIX Global Traffic Scorecard, INRIX. Available online: https://inrix.com/scorecard/.
8. Road transport externalities in Mexico: Estimates and international comparisons10.1016/j.tranpol.2013.08.004 ↗
9. Salazar-Carrillo, J., Torres-Ruiz, M., Davis, C.A., Quintero, R., Moreno-Ibarra, M., and Guzmán, G. (2021). Traffic Congestion Analysis Based on a Web-GIS and Data Mining of Traffic Events from Twitter. Sensors, 21.10.3390/s21092964 ↗
10. Thorne, E., Chong Ling, E., and Phillips, W. (2024). Assessment of the Economic Costs of Vehicle Traffic Congestion in the Caribbean: A Case Study of Trinidad and Tobago, Naciones Unidas Comisión Económica para América Latina y el Caribe (CEPAL). Technical Report.
11. Fleming, S. (2026, March 01). Traffic Congestion Cost the US Economy Nearly $87 Billion in 2018. World Economic Forum, 2019. Available online: https://www.weforum.org/stories/2019/03/traffic-congestion-cost-the-us-economy-nearly-87-billion-in-2018/.
12. Pop, M.D., Pandey, J., and Ramasamy, V. (2020, January 4–5). Future Networks 2030: Challenges in Intelligent Transportation Systems. Proceedings of the 2020 8th International Conference on Reliability, Infocom Technologies and Optimization (Trends and Future Directions) (ICRITO), Noida, India.10.1109/ICRITO48877.2020.9197951 ↗
13. Transport challenges in rapidly growing cities: Is there a magic bullet?10.1080/01441647.2019.1654201 ↗
14. Congestion, M.U.T. (2007). Managing Urban Traffic Congestion, OECD Publishing.
15. The traffic signal control problem for intersections: A review10.1186/s12544-020-00440-8 ↗
16. Optimization and simulation of fixed-time traffic signal control in real-world applications10.1016/j.procs.2019.04.113 ↗
17. Koonce, P., and Rodegerdts, L. (2008). Traffic Signal Timing Manual, Federal Highway Administration. Technical Report.
18. State-of-art review of traffic signal control methods: Challenges and opportunities10.1186/s12544-020-00439-1 ↗
19. Serban, A.A., and Frunzete, M. (2022, January 26–29). Analysis and comparison between urban traffic control systems. Proceedings of the 2022 IEEE 28th International Symposium for Design and Technology in Electronic Packaging (SIITME), Bucharest, Romania.10.1109/SIITME56728.2022.9988444 ↗
20. Cools, S.B., Gershenson, C., and D’Hooghe, B. (2013). Self-Organizing Traffic Lights: A Realistic Simulation. Advances in Applied Self-Organizing Systems, Springer.10.1007/978-1-4471-5113-5_3 ↗
21. Cools, S.B., Gershenson, C., and D’Hooghe, B. (2008). Self-Organizing Traffic Lights: A Realistic Simulation. Advanced Information and Knowledge Processing, Springer.10.1007/978-1-84628-982-8_3 ↗
22. Ferreira, M., Fernandes, R., Conceição, H., Viriyasitavat, W., and Tonguz, O.K. (2010, January 24). Self-organized traffic control. Proceedings of the Seventh ACM International Workshop on VehiculAr InterNETworking, VANET ’10, New York, NY, USA.10.1145/1860058.1860077 ↗
23. RHODES to intelligent transportation systems10.1109/MIS.2005.15 ↗
24. Lowrie, P. (1990). SCATS, Sydney Co-Ordinated Adaptive Traffic System: A Traffic Responsive Method of Controlling Urban Traffic, Roads and Traffic Authority of New South Wales, Traffic Control Section.
25. Adaptive Traffic Signal Control for large-scale scenario with Cooperative Group-based Multi-agent reinforcement learning10.1016/j.trc.2021.103046 ↗
26. Different Fuzzy Logic Control Strategies for Traffic Signal Timing Control with State Inputs10.1016/j.ifacol.2021.06.032 ↗
27. A Review on Swarm Intelligence and Evolutionary Algorithms for Solving the Traffic Signal Control Problem10.1109/TITS.2020.3014296 ↗
28. Deep learning for visual understanding: A review10.1016/j.neucom.2015.09.116 ↗
29. Adaptive traffic light control using deep reinforcement learning technique10.1007/s11042-023-16112-3 ↗
30. Deep Reinforcement Learning based approach for Traffic Signal Control10.1016/j.trpro.2022.02.035 ↗
31. Tan, J., Yuan, Q., Guo, W., Xie, N., Liu, F., Wei, J., and Zhang, X. (2022). Deep Reinforcement Learning for Traffic Signal Control Model and Adaptation Study. Sensors, 22.10.3390/s22228732 ↗
32. An Intelligent IoT Based Traffic Light Management System: Deep Reinforcement Learning10.3390/smartcities5040066 ↗
33. Ibrokhimov, B., Kim, Y.J., and Kang, S. (2022). Biased Pressure: Cyclic Reinforcement Learning Model for Intelligent Traffic Signal Control. Sensors, 22.10.3390/s22072818 ↗
34. CenLight: Centralized traffic grid signal optimization via action and state decomposition10.1049/itr2.12354 ↗
35. Abdelghaffar, H.M., and Rakha, H.A. (2019). A Novel Decentralized Game-Theoretic Adaptive Traffic Signal Controller: Large-Scale Testing. Sensors, 19.10.3390/s19102282 ↗
36. CVLight: Decentralized learning for adaptive traffic signal control with connected vehicles10.1016/j.trc.2022.103728 ↗
37. Centralised and decentralised signal timing optimisation approaches for network traffic control10.1016/j.trc.2019.05.007 ↗
38. Urban traffic signal control with connected and automated vehicles: A survey10.1016/j.trc.2019.01.026 ↗
39. Multi-agent deep reinforcement learning for large-scale traffic signal control10.1109/TITS.2019.2901791 ↗
40. A survey on deep reinforcement learning approaches for traffic signal control10.1016/j.engappai.2024.108100 ↗
41. Cyber Traffic Light: Safe Cooperation for Autonomous Vehicles at Dynamic Intersections10.1109/TITS.2022.3146457 ↗
42. Wang, B., He, Z., Sheng, J., and Chen, Y. (2022). Deep Reinforcement Learning for Traffic Light Timing Optimization. Processes, 10.10.3390/pr10112458 ↗
43. El-Qoraychy, F.Z., Dridi, M., and Creput, J.C. (2024, January 2–4). Deep Reinforcement Learning for Vehicle Intersection Management in High-Dimensional Action Spaces. Proceedings of the 2024 7th International Conference on Machine Learning and Machine Intelligence (MLMI), MLMI ’24, New York, NY, USA.10.1145/3696271.3696278 ↗
44. Shi, Y., Wang, Z., LaClair, T.J., Wang, C.R., Shao, Y., and Yuan, J. (2023). A Novel Deep Reinforcement Learning Approach to Traffic Signal Control with Connected Vehicles. Appl. Sci., 13.10.3390/app13042750 ↗
45. A review of reinforcement learning applications in adaptive traffic signal control10.1049/itr2.12208 ↗
46. Majstorović, Ž., Tišljarić, L., Ivanjko, E., and Carić, T. (2023). Urban Traffic Signal Control under Mixed Traffic Flows: Literature Review. Appl. Sci., 13.10.3390/app13074484 ↗
47. Zhong, D., and Boukerche, A. (2019, January 25–29). Traffic Signal Control Using Deep Reinforcement Learning with Multiple Resources of Rewards. Proceedings of the 16th ACM International Symposium on Performance Evaluation of Wireless Ad Hoc, Sensor, & Ubiquitous Networks, PE-WASUN ’19, New York, NY, USA.10.1145/3345860.3361522 ↗
48. Adaptive Traffic Signal Control Model on Intersections Based on Deep Reinforcement Learning
49. Pan, T. (2023). Traffic Light Control with Reinforcement Learning. arXiv.
50. Intelligent traffic light design and control in smart cities: A survey on techniques and methodologies
51. Li, Z., Xu, C., and Zhang, G. (2021). A Deep Reinforcement Learning Approach for Traffic Signal Control Optimization. arXiv.
52. Nawar, M., Fares, A., and Al-Sammak, A. (2019, January 15–16). Rainbow Deep Reinforcement Learning Agent for Improved Solution of the Traffic Congestion. Proceedings of the 2019 7th International Japan-Africa Conference on Electronics, Communications, and Computations, (JAC-ECC), Alexandria, Egypt.10.1109/JAC-ECC48896.2019.9051262 ↗
53. Mirbakhsh, S., and Azizi, M. (2024). Adaptive traffic signal safety and efficiency improvement by multi objective deep reinforcement learning approach. arXiv.
54. Shao, J., Zheng, C., Chen, Y., Huang, Y., and Zhang, R. (2024). MoveLight: Enhancing Traffic Signal Control through Movement-Centric Deep Reinforcement Learning. arXiv.
55. Sattarzadeh, A.R., and Pathirana, P.N. (2024, January 20–23). Enhancing Adaptive Traffic Control Systems with Deep Reinforcement Learning and Graphical Models. Proceedings of the 2024 IEEE International Conference on Future Machine Learning and Data Science (FMLDS), Sydney, NSW, Australia.10.1109/FMLDS63805.2024.00017 ↗
56. Dhashyanth, N., Hemchand, R., Priyanga, R., Soorya, S., and Sudheesh, P. (2024, January 13–15). Adaptive Traffic Control Using Deep Reinforcement Learning. Proceedings of the 2024 IEEE Pune Section International Conference (PuneCon), Pune, India.
57. A Deep Reinforcement Learning Approach to Traffic Signal Control With Temporal Traffic Pattern Mining10.1109/TITS.2021.3107258 ↗
58. Gradient-based learning applied to document recognition10.1109/5.726791 ↗
59. Zhang, A., Lipton, Z.C., Li, M., and Smola, A.J. (2023). Dive into Deep Learning, Cambridge University Press. Available online: https://D2L.ai.
60. State-of-the-art in artificial neural network applications: A survey10.1016/j.heliyon.2018.e00938 ↗
61. Goodfellow, I., Bengio, Y., and Courville, A. (2016). Deep Learning, MIT Press. Available online: http://www.deeplearningbook.org.
62. An introduction to deep reinforcement learning10.1561/2200000071 ↗
63. A review On reinforcement learning: Introduction and applications in industrial process control10.1016/j.compchemeng.2020.106886 ↗
64. Average reward reinforcement learning: Foundations, algorithms, and empirical results10.1023/A:1018064306595 ↗
65. A Gentle Introduction to Reinforcement Learning and its Application in Different Fields10.1109/ACCESS.2020.3038605 ↗
66. Aubret, A., Matignon, L., and Hassas, S. (2023). An Information-Theoretic Perspective on Intrinsic Motivation in Reinforcement Learning: A Survey. Entropy, 25.10.3390/e25020327 ↗
67. A Brief Study of Deep Reinforcement Learning with Epsilon-Greedy Exploration10.12785/ijcds/110144 ↗
68. Dabney, W., Ostrovski, G., and Barreto, A. (2021, January 3–7). Temporally-Extended ε-Greedy Exploration. Proceedings of the International Conference on Learning Representations, Online.
69. Solla, S., Leen, T., and Müller, K. (1999). Actor-Critic Algorithms. Proceedings of the Advances in Neural Information Processing Systems, MIT Press.
70. A survey of actor-critic reinforcement learning: Standard and natural policy gradients10.1109/TSMCC.2012.2218595 ↗
71. Single-Loop Deep Actor-Critic for Constrained Reinforcement Learning With Provable Convergence10.1109/TSP.2024.3461963 ↗
72. Noise injection for training artificial neural networks: A comparison with weight decay and early stopping10.1118/1.3213517 ↗
73. Training with Noise is Equivalent to Tikhonov Regularization10.1162/neco.1995.7.1.108 ↗
74. Fujimoto, S., Hoof, H., and Meger, D. (2018, January 10–15). Addressing function approximation error in actor-critic methods. Proceedings of the International Conference on Machine Learning, PMLR, Stockholm, Sweden.
75. Explicit Regularisation in Gaussian Noise Injections
76. Evans, J.D. (1996). Straightforward Statistics for the Behavioral Sciences, Thomson Brooks/Cole Publishing Co.
77. Little, J.D., and Graves, S.C. (2008). Little’s law. Building Intuition: Insights from Basic Operations Management Models and Principles, Springer.10.1007/978-0-387-73699-0_5 ↗
78. Enhancing Robustness of Deep Reinforcement Learning Based Adaptive Traffic Signal Controllers in Mixed Traffic Environments Through Data Fusion and Multi-Discrete Actions10.1109/TITS.2024.3399066 ↗
79. Friesen, M., Tan, T., Jasperneite, J., and Wang, J. (2021). Application of Multi-Agent Deep Reinforcement Learning to Optimize Real-World Traffic Signal Controls. Authorea Prepr.10.36227/techrxiv.16974493 ↗