journal article·2025-05-28Open access

Data-Aware Path Planning for Autonomous Vehicles Using Reinforcement Learning

Yousef AlSaqabi, Bhaskar Krishnamachari

Applied Sciences· Volume 15· Issue 11

Abstract

This paper addresses the challenge of optimizing path planning for autonomous vehicles in urban environments by considering both traffic and bandwidth variability on the road. Traditional path planning methods are inadequate for the needs of interconnected vehicles that require significant real-time data transfer. We propose a reinforcement learning approach for path planning, formulated to use road traffic conditions and bandwidth availability. This approach optimizes routes by minimizing travel time while maximizing data transfer capability. We create a realistic simulation environment using GraphML, incorporating real-world map data and vehicle mobility patterns to evaluate the effectiveness of our approach. Through comprehensive testing against various baselines, our reinforcement learning model demonstrates the ability to adapt and find optimal paths that significantly outperform conventional strategies. These results emphasize the feasibility of using reinforcement learning for dynamic path optimization and highlight its potential to improve both the efficiency of travel and the reliability of data-driven decisions in autonomous vehicular networks.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: Robotic Path Planning Algorithms · Autonomous Vehicle Technology and Safety · Reinforcement Learning in Robotics

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

Department of Electrical and Computer Engineering, University of Southern California, Los Angeles, CA 90007, USA

Department of Electrical Engineering, Kuwait University, Kuwait City 72304, Kuwait

References (83)

Show source-provided references
1. Intel Corporation (2024, March 10). Data Is the New Oil in the Future of Automated Driving. Available online: https://www.intel.com/content/www/us/en/newsroom/news/self-driving-cars-data.html.
2. Advanced planning for autonomous vehicles using reinforcement learning and deep inverse reinforcement learning10.1016/j.robot.2019.01.003 ↗
3. Singh, S. (2024, March 10). Critical Reasons for Crashes Investigated in the National Motor Vehicle Crash Causation Survey, Available online: https://crashstats.nhtsa.dot.gov/Api/Public/ViewPublication/812506.
4. Integration challenges of intelligent transportation systems with connected vehicle, cloud computing, and internet of things technologies10.1109/MWC.2015.7368833 ↗
5. Route planning considerations for autonomous vehicles10.1109/MCOM.2018.1800135 ↗
6. Machine learning for vehicular networks: Recent advances and application examples10.1109/MVT.2018.2811185 ↗
7. Brandes, U., Eiglsperger, M., Lerner, J., and Pich, C. (2013). Graph markup language (GraphML). Handbook of Graph Drawing and Visualization, Chapman and Hall/CRC.
8. A survey of path planning algorithms for autonomous vehicles10.4271/02-14-01-0007 ↗
9. Global and local path planning study in a ROS-based research platform for autonomous vehicles
10. A note on two problems in connection with graphs10.1007/BF01386390 ↗
11. A formal basis for the heuristic determination of minimum cost paths10.1109/TSSC.1968.300136 ↗
12. Generalized best-first search strategies and the optimality of A*10.1145/3828.3830 ↗
13. A survey of motion planning and control techniques for self-driving urban vehicles10.1109/TIV.2016.2578706 ↗
14. Bast, H., Delling, D., Goldberg, A.V., Müller-Hannemann, M., Pajor, T., Sanders, P., Wagner, D., and Werneck, R.F. (2016). Route planning in transportation networks. Algorithm Engineering, Springer.10.1007/978-3-319-49487-6_2 ↗
15. Stentz, A. (1994, January 8–13). Optimal and efficient path planning for partially-known environments. Proceedings of the 1994 IEEE International Conference on Robotics and Automation, San Diego, CA, USA.
16. Nguyen, H.H., Kim, D.H., Kim, C.K., Yim, H., Jeong, S.K., and Kim, S.B. (July, January 28). A simple path planning for automatic guided vehicle in unknown environment. Proceedings of the 2017 14th International Conference on Ubiquitous Robots and Ambient Intelligence (URAI), Jeju, Republic of Korea.10.1109/URAI.2017.7992745 ↗
17. Khatib, O. (1990). Real-time obstacle avoidance for manipulators and mobile robots. Autonomous Robot Vehicles, Springer.
18. Ganeshmurthy, M.S., and Suresh, G.R. (2015, January 26–28). Path planning algorithm for autonomous mobile robot in dynamic environment. Proceedings of the 2015 3rd International Conference on Signal Processing, Communication and Networking, Chennai, India.10.1109/ICSCN.2015.7219901 ↗
19. Wang, L., and Luo, C. (2018). A hybrid genetic tabu search algorithm for mobile robot to solve AS/RS path planning. Int. J. Robot. Autom., 33.10.2316/Journal.206.2018.2.206-5102 ↗
20. LaValle, S. (1998). Rapidly-exploring random trees: A new tool for path planning. Annu. Res. Rep.
21. Deep reinforcement learning for autonomous driving: A survey10.1109/TITS.2021.3054625 ↗
22. Nešmark, J.I., and Fufaev, N.A. (2004). Dynamics of Nonholonomic Systems, American Mathematical Society.10.1090/mmono/033 ↗
23. LaValle, S.M., and Kuffner, J.J. (1999, January 10–15). Randomized kinodynamic planning. Proceedings of the 1999 IEEE International Conference on Robotics and Automation (Cat. No.99CH36288C), Detroit, MI, USA.
24. Loaiza Quintana, C., Climent, L., and Arbelaez, A. (2022, January 10–14). Iterated Local Search for the eBuses Charging Location Problem. Proceedings of the Parallel Problem Solving from Nature—PPSN XVII: 17th International Conference, Dortmund, Germany.10.1007/978-3-031-14721-0_24 ↗
25. Pattern recognition and machine learning10.1117/1.2819119 ↗
26. Deep learning and its applications to signal and information processing10.1109/MSP.2010.939038 ↗
27. Deep neural networks for acoustic modeling in speech recognition: The shared views of four research groups10.1109/MSP.2012.2205597 ↗
28. Graves, A., Mohamed, A., and Hinton, G. (2013). Speech recognition with deep recurrent neural networks. arXiv.10.1109/ICASSP.2013.6638947 ↗
29. Kumar, A., Irsoy, O., Ondruska, P., Iyyer, M., Bradbury, J., Gulrajani, I., Zhong, V., Paulus, R., and Socher, R. (2015). Ask me anything: Dynamic memory networks for natural language processing. arXiv.
30. A survey of the usages of deep learning for natural language processing10.1109/TNNLS.2020.2979670 ↗
31. Manning, C., Surdeanu, M., Bauer, J., Finkel, J., Bethard, S., and McClosky, D. (2014, January 23–24). The Stanford CoreNLP natural language processing toolkit. Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics: System Demonstrations, Baltimore, MD, USA.10.3115/v1/P14-5010 ↗
32. Collobert, R., and Weston, J. (2008, January 5–9). A unified architecture for natural language processing: Deep neural networks with multitask learning. Proceedings of the 25th International Conference on Machine Learning, Helsinki, Finland.10.1145/1390156.1390177 ↗
33. Machine learning for medical diagnosis: History, state of the art and perspective10.1016/S0933-3657(01)00077-X ↗
34. Shvets, A.A., Rakhlin, A., Kalinin, A.A., and Iglovikov, V.I. (2018, January 17–20). Automatic instrument segmentation in robot-assisted surgery using deep learning. Proceedings of the 17th IEEE International Conference on Machine Learning and Applications, Orlando, FL, USA.10.1101/275867 ↗
35. Reinforcement learning: An introduction10.1109/TNN.1998.712192 ↗
36. Bae, H., Kim, G.D., Kim, J., Qian, D., and Lee, S.G. (2019). Multi-robot path planning method using reinforcement learning. Appl. Sci., 9.10.3390/app9153057 ↗
37. New potential functions for mobile robot path planning10.1109/70.880813 ↗
38. Robot path planning in uncertain environment using multi-objective particle swarm optimization10.1016/j.neucom.2012.09.019 ↗
39. Intelligent Bézier curve-based path planning model using chaotic particle swarm optimization algorithm10.1007/s10586-018-2360-3 ↗
40. Bezier curve based path planning in a dynamic field using modified genetic algorithm10.1016/j.jocs.2017.08.004 ↗
41. New path planning model for mobile anchor-assisted localization in wireless sensor networks10.1007/s11276-017-1493-2 ↗
42. Natural actor-critic10.1016/j.neucom.2007.11.026 ↗
43. A novel actor–critic–identifier architecture for approximate optimal control of uncertain nonlinear systems10.1016/j.automatica.2012.09.019 ↗
44. Online actor–critic algorithm to solve the continuous-time infinite horizon optimal control problem10.1016/j.automatica.2010.02.018 ↗
45. Florensa, C., Degrave, J., Heess, N., Springenberg, J.T., and Riedmiller, M. (2019). Self-supervised learning of image embedding for continuous control. arXiv.
47. Lillicrap, T.P., Hunt, J.J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., and Wierstra, D. (2015). Continuous control with deep reinforcement learning. arXiv.
48. Fuzzy Q-learning for multi-agent decentralized energy management in microgrids10.1016/j.apenergy.2018.03.017 ↗
49. Using feedback in collaborative reinforcement learning to adaptively optimize MANET routing10.1109/TSMCA.2005.846390 ↗
50. Doddalinganavar, S.S., Tergundi, P.V., and Patil, R.S. (2019, January 25–27). Survey on deep reinforcement learning protocol in VANET. Proceedings of the 1st International Conference on Advances in Information Technology, Chikmagalur, India.10.1109/ICAIT47043.2019.8987282 ↗
51. QMR: Q-learning based multi-objective optimization routing protocol for flying ad hoc networks10.1016/j.comcom.2019.11.011 ↗
52. The unmanned aerial vehicle routing and trajectory optimisation problem, a taxonomic review10.1016/j.cie.2018.04.037 ↗
53. Reinforcement learning-based routing protocols for vehicular ad hoc networks: A comparative survey10.1109/ACCESS.2021.3058388 ↗
54. Hierarchical routing for vehicular ad hoc networks via reinforcement learning10.1109/TVT.2018.2887282 ↗
55. Misra, S., Woungang, I., and Misra, S.C. (2009). Guide to Wireless Ad Hoc Networks, Springer Science & Business Media.10.1007/978-1-84800-328-6 ↗
56. Hartenstein, H., and Laberteaux, K. (2009). VANET: Vehicular Applications and Inter-Networking Technologies, John Wiley & Sons.10.1002/9780470740637 ↗
57. Vigo, D., and Toth, P. (2014). Vehicle Routing, Society for Industrial and Applied Mathematics.
58. Rasheed, A., Gillani, S., Ajmal, S., and Qayyum, A. Vehicular ad hoc network (VANET): A survey, challenges, and applications. Proceedings of the Vehicular Ad-Hoc Networks for Smart Cities, Advances in Intelligent Systems and Computing.
59. Routing in vehicular ad hoc networks: A survey10.1109/MVT.2007.912927 ↗
60. Reinforcement learning: A survey10.1613/jair.301 ↗
61. Chettibi, S., and Chikhi, S. (2011, January 26–28). A survey of reinforcement learning based routing protocols for mobile ad-hoc networks. Proceedings of the Recent Trends in Wireless and Mobile Networks, Ankara, Turkey. Communications in Computer and Information Science.10.1007/978-3-642-21937-5_1 ↗
62. Video-based lane estimation and tracking for driver assistance: Survey, system, and evaluation10.1109/TITS.2006.869595 ↗
63. Application of time-variant predictive control to modelling driver steering skill10.1080/00423110903551626 ↗
64. You, C., and Tsiotras, P. (2016, January 6–8). Optimal two-point visual driver model and controller development for driver-assist systems for semi-autonomous vehicles. Proceedings of the 2016 American Control Conference (ACC), Boston, MA, USA.10.1109/ACC.2016.7526607 ↗
65. Zafeiropoulos, S., and Tsiotras, P. (2014, January 4–6). Design of a lane-tracking driver steering assist system and its interaction with a two-point visual driver model. Proceedings of the 2014 American Control Conference, Portland, OR, USA.10.1109/ACC.2014.6858867 ↗
66. Lange, S., Riedmiller, M., and Voigtländer, A. (2012, January 10–15). Autonomous reinforcement learning on raw visual input data in a real world application. Proceedings of the 2012 International Joint Conference on Neural Networks (IJCNN), Brisbane, QLD, Australia.10.1109/IJCNN.2012.6252823 ↗
67. Human-like autonomous car-following model with deep reinforcement learning10.1016/j.trc.2018.10.024 ↗
68. Sallab, A., Abdou, M., Perot, E., and Yogamani, S. (2016). End-to-End Deep Reinforcement Learning for Lane Keeping Assist. arXiv.
69. Deep distributional reinforcement learning based high-level driving policy determination10.1109/TIV.2019.2919467 ↗
70. Bouton, M., Nakhaei, A., Fujimura, K., and Kochenderfer, M.J. (2019, January 27–30). Cooperation-aware reinforcement learning for merging in dense traffic. Proceedings of the IEEE Intelligent Transportation Systems Conference, Auckland, New Zealand.10.1109/ITSC.2019.8916924 ↗
71. Deep multi-agent reinforcement learning for highway on-ramp merging in mixed traffic10.1109/TITS.2023.3285442 ↗
72. A decision-making strategy for vehicle autonomous braking in emergency via deep reinforcement learning10.1109/TVT.2020.2986005 ↗
73. Coordination control strategy for human-machine cooperative steering of intelligent vehicles: A reinforcement learning approach10.1109/TITS.2022.3187016 ↗
74. Deep reinforcement learning for intelligent transportation systems: A survey10.1109/TITS.2020.3008612 ↗
75. Fei, Y., Xing, L., Yao, L., Yang, Z., and Zhang, Y. (2025). Deep reinforcement learning for decision making of autonomous vehicles in non-lane-based traffic environments. PLoS ONE, 20.10.1371/journal.pone.0320578 ↗
76. Hussain, Q., Noor, A., Qureshi, M., Li, J., Rahman, A., Bakry, A., Mahmood, T., and Rehman, A. (2025). Reinforcement learning based route optimization model to enhance energy efficiency in internet of vehicles. Sci. Rep., 15.10.1038/s41598-025-86608-5 ↗
77. Sun, P., He, J., Wan, J., Guan, Y., Liu, D., Su, X., and Li, L. (2025). Deep reinforcement learning based low energy consumption scheduling approach design for urban electric logistics vehicle networks. Sci. Rep., 15.10.1038/s41598-025-92916-7 ↗
78. Wang, X., Qian, B., Zhuo, J., and Liu, W. (2025). An Autonomous Vehicle Behavior Decision Method Based on Deep Reinforcement Learning with Hybrid State Space and Driving Risk. Sensors, 25.10.3390/s25030774 ↗
79. Audinys, R., Slikas, Z., Radkevicius, J., Sutas, M., and Ostreika, A. (2025). Deep Reinforcement Learning for a Self-Driving Vehicle Operating Solely on Visual Information. Electronics, 14.10.3390/electronics14050825 ↗
80. Enhancing Autonomous Driving Decision: A Hybrid Deep Reinforcement Learning-Kinematic-Based Autopilot Framework for Complex Motorway Scenes10.1109/TITS.2025.3529838 ↗
81. AlSaqabi, Y., and Chattopadhyay, S. (2023). Driving with Guidance: Exploring the Trade-Off Between GPS Utility and Privacy Concerns Among Drivers. arXiv.
82. AlSaqabi, Y., and Krishnamachari, B. (2022, January 19–23). Trip planning for autonomous vehicles with wireless data transfer needs using reinforcement learning. Proceedings of the IEEE International Conference on Mobile Ad-hoc and Sensor Systems, Denver, CO, USA.10.1109/MASS56207.2022.00010 ↗
83. Ng, A., Harada, D., and Russell, S.J. (1999, January 27–30). Policy Invariance Under Reward Transformations: Theory and Application to Reward Shaping. Proceedings of the International Conference on Machine Learning, Bled, Slovenia.