From Virtual to Reality: A Deep Reinforcement Learning Solution to Implement Autonomous Driving with 3D-LiDAR
Source abstract
Abstract
Autonomous driving technology faces significant challenges in processing complex environmental data and making real-time decisions. Traditional supervised learning approaches heavily rely on extensive data labeling, which incurs substantial costs. This study presents a complete implementation framework combining Deep Deterministic Policy Gradient (DDPG) reinforcement learning with 3D-LiDAR perception techniques for practical application in autonomous driving. DDPG meets the continuous action space requirements of driving, and the point cloud processing module uses a traditional algorithm combined with attention mechanisms to provide high awareness of the environment. The solution is first validated in a simulation environment and then successfully migrated to a real environment based on a 1/10-scale F1tenth experimental vehicle. The experimental results show that the method proposed in this study is able to complete the autonomous driving task in the real environment, providing a feasible technical path for the engineering application of advanced sensor technology combined with complex learning algorithms in the field of autonomous driving.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Autonomous Vehicle Technology and Safety · Robotic Path Planning Algorithms · Reinforcement Learning in Robotics
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Faculty of Applied Sciences, Macao Polytechnic University, Macao SAR 999078, China
Department of Computer Science and Engineering—DISI, University of Bologna, 47521 Cesena, Italy
Autonomous Robotics Research Center, Technology Innovation Institute (TII), Abu Dhabi P.O. Box 9639, United Arab Emirates
Department of Computer Science, University of California, Los Angeles, CA 90095, USA
Funding
Macao Polytechnic University
References (37)
Show source-provided references
1. Analysis of the potential of autonomous vehicles in reducing the emissions of greenhouse gases in road transport10.1016/j.proeng.2017.06.061 ↗
2. A Comprehensive Review of Environmental and Economic Impacts of Autonomous Vehicles10.59247/csol.v2i3.131 ↗
3. A Survey of Autonomous Driving: Common Practices and Emerging Technologies10.1109/ACCESS.2020.2983149 ↗
4. Chen, R.C., Saravanarajan, V.S., Chen, L.S., and Yu, H. (2022). Road Segmentation and Environment Labeling for Autonomous Vehicles. Appl. Sci., 12.10.3390/app12147191 ↗
5. Li, Y. (2017). Deep Reinforcement Learning: An Overview. arXiv.
6. Human-Level Control through Deep Reinforcement Learning10.1038/nature14236 ↗
7. Lillicrap, T.P., Hunt, J.J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., and Wierstra, D. (2015). Continuous Control with Deep Reinforcement Learning. arXiv.
8. A Review of LiDAR Sensor Technologies for Perception in Automated Driving10.54097/ajst.v3i3.2993 ↗
9. Liu, Y., and Diao, S. (2024). An Automatic Driving Trajectory Planning Approach in Complex Traffic Scenarios Based on Integrated Driver Style Inference and Deep Reinforcement Learning. PLoS ONE, 19.10.1371/journal.pone.0297192 ↗
10. From Naturalistic Traffic Data to Learning-Based Driving Policy: A Sim-to-Real Study10.1109/TVT.2023.3307409 ↗
11. A Deep Reinforcement Learning-Based Active Suspension Control Algorithm Considering Deterministic Experience Tracing for Autonomous Vehicles10.1016/j.asoc.2024.111259 ↗
12. Bosello, M., Tse, R., and Pau, G. (2022, January 8–11). Train in Austria, Race in Montecarlo: Generalized RL for Cross-Track F1 Tenth LiDAR-Based Races. Proceedings of the 2022 IEEE 19th Annual Consumer Communications and Networking Conference (CCNC), Las Vegas, NV, USA.10.1109/CCNC49033.2022.9700730 ↗
13. Li, Z., Yuan, S., Yin, X., Li, X., and Tang, S. (2023). Research into Autonomous Vehicles Following and Obstacle Avoidance Based on Deep Reinforcement Learning Method under Map Constraints. Sensors, 23.10.3390/s23020844 ↗
14. Li, W., Zhang, Y., Shi, X., and Qiu, F. (2022). A Decision-Making Strategy for Car Following Based on Naturalist Driving Data via Deep Reinforcement Learning. Sensors, 22.10.3390/s22208055 ↗
15. Towards Robust Decision-Making for Autonomous Driving on Highway10.1109/TVT.2023.3268500 ↗
16. Explaining a Deep Reinforcement Learning (DRL)-Based Automated Driving Agent in Highway Simulations10.1109/ACCESS.2023.3259544 ↗
17. Self-Learned Autonomous Driving at Unsignalized Intersections: A Hierarchical Reinforced Learning Approach for Feasible Decision-Making10.1109/TITS.2023.3285440 ↗
18. Vision-Based Autonomous Driving: A Hierarchical Reinforcement Learning Approach10.1109/TVT.2023.3266940 ↗
19. Integrating Big Data Analytics in Autonomous Driving: An Unsupervised Hierarchical Reinforcement Learning Approach10.1016/j.trc.2024.104606 ↗
20. Deep reinforcement learning based control for Autonomous Vehicles in CARLA10.1007/s11042-021-11437-3 ↗
21. Petryshyn, B., Postupaiev, S., Ben Bari, S., and Ostreika, A. (2024). Deep Reinforcement Learning for Autonomous Driving in Amazon Web Services DeepRacer. Information, 15.10.3390/info15020113 ↗
22. Wang, W., Luo, H., Zheng, Q., Wang, C., and Guo, W. (2020, January 17–20). A Deep Reinforcement Learning Framework for Vehicle Detection and Pose Estimation in 3D Point Clouds. Proceedings of the 6th International Conference on Artificial Intelligence and Security (ICAIS 2020), Hohhot, China.10.1007/978-3-030-57881-7_36 ↗
23. Lee, Y., and Park, S. (2020). A Deep Learning-Based Perception Algorithm Using 3D LiDAR for Autonomous Driving: Simultaneous Segmentation and Detection Network (SSADNet). Appl. Sci., 10.10.3390/app10134486 ↗
24. Chen, Y., Tse, R., Bosello, M., Aguiari, D., Tang, S.K., and Pau, G. (2022, January 20–23). Enabling Deep Reinforcement Learning Autonomous Driving by 3D-LiDAR Point Clouds. Proceedings of the Fourteenth International Conference on Digital Image Processing (ICDIP 2022), Wuhan, China.10.1117/12.2644369 ↗
25. Zhai, Y., Liu, Z., Miao, Y., and Wang, H. (2022, January 25–27). Efficient Reinforcement Learning for 3D LiDAR Navigation of Mobile Robot. Proceedings of the 41st Chinese Control Conference (CCC 2022), Hefei, China.10.23919/CCC55666.2022.9902734 ↗
26. Sánchez, M., Morales, J., and Martínez, J.L. (2023). Reinforcement and Curriculum Learning for Off-Road Navigation of a UGV with a 3D LiDAR. Sensors, 23.10.3390/s23063239 ↗
27. Bellman, R., and Kalaba, R.E. (1965). Dynamic Programming and Modern Control Theory, Academic Press.
28. Sutton, R.S., and Barto, A.G. (1998). The reinforcement learning problem. Reinforcement learning: An introduction, MIT Press.10.1109/TNN.1998.712192 ↗
29. Silver, D., Lever, G., Heess, N., Degris, T., Wierstra, D., and Riedmiller, M. (2014, January 21–26). Deterministic Policy Gradient Algorithms. Proceedings of the International Conference on Machine Learning (ICML 2014), Beijing, China.
30. Mnih, V. (2013). Playing atari with deep reinforcement learning. arXiv.
31. Qi, C.R., Su, H., Mo, K., and Guibas, L.J. (2017, January 21–26). PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2017), Honolulu, HI, USA.
32. Vaswani, A. (2017). Attention Is All You Need. Advances in Neural Information Processing Systems (NeurIPS 2017), Curran Associates, Inc.
33. Dynamic Graph CNN for Learning on Point Clouds
34. PCT: Point Cloud Transformer10.1007/s41095-021-0229-5 ↗
35. Zhao, H., Jiang, L., Jia, J., Torr, P.H., and Koltun, V. (2021, January 10–17). Point Transformer. Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV 2021), Montreal, QC, Canada.10.1109/ICCV48922.2021.01595 ↗
36. Dosovitskiy, A., Ros, G., Codevilla, F., Lopez, A., and Koltun, V. (2017, January 13–15). CARLA: An Open Urban Driving Simulator. Proceedings of the Conference on Robot Learning (CoRL 2017), Mountain View, CA, USA.
37. O’Kelly, M., Sukhil, V., Abbas, H., Harkins, J., Kao, C., Pant, Y.V., and Bertogna, M. (2019). F1/10: An Open-Source Autonomous Cyber-Physical Platform. arXiv.