Imitative Reinforcement Learning Fusing Mask R-CNN Perception Algorithms
Source abstract
Abstract
Autonomous urban driving navigation is still an open problem and has ample room for improvement in unknown complex environments. This paper proposes an end-to-end autonomous driving approach that combines Conditional Imitation Learning (CIL), Mask R-CNN with DDPG. In the first stage, data acquisition is first performed by using CARLA, a high-fidelity simulation software. Data collected by CARLA is used to train the Mask R-CNN network, which is used for object detection and segmentation. The segmented images are transformed into the backbone of CIL to perform supervised Imitation Learning (IL). DDPG means using Reinforcement Learning for further training in the second stage, which shares the learned weights from the pre-trained CIL model. The combination of the two methods is an innovative way of considering. The benefit is that it is possible to speed up training considerably and obtain super-high levels of performance beyond humans. We conduct experiments on the CARLA driving benchmark of urban driving. In the final experiments, our algorithm outperforms the original MP by 30%, CIL by 33%, and CIRL by 10% in the most difficult tasks, dynamic navigation tasks, and in new environments and new weather, demonstrating that the two-stage framework proposed in this paper shows remarkable generalization capability in unknown environments on navigation tasks.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Advanced Neural Network Applications · Autonomous Vehicle Technology and Safety · Video Surveillance and Tracking Methods
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Key Laboratory of Advanced Manufacturing Technology for Auto Parts, Ministry of Education, Chongqing University of Technology, Chongqing 401320, China
Chongqing Chang’an Automobile Co., Chongqing 400023, China
College of Mechanical Engineering, Chongqing University of Technology, Chongqing 401320, China
Funding
Special key project of technological innovation and application development of Chongqing · Science and Technology Research Project of Chongqing Municipal Education Commission
References (28)
Show source-provided references
1. Chen, C., Seff, A., Kornhauser, A., and Xiao, J. (2015, January 7–13). Deepdriving: Learning affordance for direct perception in autonomous driving. Proceedings of the IEEE International Conference on Computer Vision, Santiago, Chile.10.1109/ICCV.2015.312 ↗
2. Heess, N., TB, D., Sriram, S., Lemmon, J., Merel, J., Wayne, G., Tassa, Y., Erez, T., Wang, Z., and Ali Eslami, S.M. (2017). Emergence of locomotion behaviours in rich environments. arXiv.
3. Tian, Y., Pei, K., Jana, S., and Ray, B. (June, January 27). Deeptest: Automated testing of deep-neural-network-driven autonomous cars. Proceedings of the 40th International Conference on Software Engineering, Gothenburg, Sweden.
4. Tariq, A., Osama, S.M., and Gillani, A. (2016, January 16–19). Development of a low cost and light weight uav for photogrammetry and precision land mapping using aerial imagery. Proceedings of the 2016 International Conference on Frontiers of Information Technology (FIT), Como, Italy.10.1109/FIT.2016.072 ↗
5. Von Bernuth, A., Volk, G., and Bringmann, O. (2019, January 27–30). Simulating photo-realistic snow and fog on existing images for enhanced CNN training and evaluation. Proceedings of the 2019 IEEE Intelligent Transportation Systems Conference (ITSC), Auckland, New Zealand.10.1109/ITSC.2019.8917367 ↗
6. Small object detection in unmanned aerial vehicle images using feature fusion and scaling-based single shot detector with spatial context analysis10.1109/TCSVT.2019.2905881 ↗
7. Faisal, M.M., Mohammed, M.S., Abduljabar, A.M., Abdulhussain, S.H., Mahmmod, B.M., Khan, W., and Hussain, A. (2021, January 7–10). Object Detection and Distance Measurement Using AI. Proceedings of the 2021 14th International Conference on Developments in eSystems Engineering (DeSE), Sharjah, United Arab Emirates.10.1109/DeSE54285.2021.9719469 ↗
8. Dosovitskiy, A., Ros, G., Codevilla, F., Lopez, A., and Koltun, V. (2017, January 13–15). CARLA: An open urban driving simulator. Proceedings of the Conference on Robot Learning, Mountain View, CA, USA. PMLR.
9. Codevilla, F., Santana, E., López, A.M., and Gaidon, A. (November, January 27). Exploring the Limitations of Behavior Cloning for Autonomous Driving. Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision (ICCV), Seoul, Republic of Korea.10.1109/ICCV.2019.00942 ↗
10. He, K., Gkioxari, G., Dollár, P., and Girshick, R. (2017, January 22–29). Mask r-cnn. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.10.1109/ICCV.2017.322 ↗
11. Lillicrap, T.P., Hunt, J.J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., and Wierstra, D. (2015). Continuous control with deep reinforcement learning. arXiv.
12. Michaelis, C., Mitzkus, B., Geirhos, R., Rusak, E., Bringmann, O., Ecker, A.S., Bethge, M., and Brendel, W. (2019). Benchmarking robustness in object detection: Autonomous driving when winter is coming. arXiv.
13. Tomy, A., Paigwar, A., Mann, K.S., Renzaglia, A., and Laugier, C. (2022, January 23–27). Fusing Event-based and RGB camera for Robust Object Detection in Adverse Conditions. Proceedings of the 2022 International Conference on Robotics and Automation (ICRA), Philadelphia, PA, USA.10.1109/ICRA46639.2022.9812059 ↗
14. Attention transfer from human to neural networks for road object detection in winter10.1049/ipr2.12562 ↗
15. Simonyan, K., and Zisserman, A. (2014). Very deep convolutional networks for large-scale image recognition. arXiv.
16. He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27–30). Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.10.1109/CVPR.2016.90 ↗
17. Long, J., Shelhamer, E., and Darrell, T. (2015, January 7–12). Fully convolutional networks for semantic segmentation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA.10.1109/CVPR.2015.7298965 ↗
18. A traffic surveillance system for obtaining comprehensive information of the passing vehicles based on instance segmentation10.1109/TITS.2020.3001154 ↗
19. Bojarski, M., del Testa, D., Dworakowski, D., Firner, B., Flepp, B., Goyal, P., Jackel, L.D., Monfort, M., Muller, U., and Zhang, J. (2016). End to end learning for self-driving cars. arXiv.
20. Bojarski, M., Yeres, P., Choromanska, A., Choromanski, K., Firner, B., Jackel, L., and Muller, U. (2017). Explaining how a deep neural network trained with end-to-end learning steers a car. arXiv.
21. Eraqi, H.M., Moustafa, M.N., and Honer, J. (2017). End-to-end deep learning for steering autonomous vehicles considering temporal dependencies. arXiv.
22. Codevilla, F., Müller, M., López, A., Koltun, V., and Dosovitskiy, A. (2018, January 21–25). End-to-end driving via conditional imitation learning. Proceedings of the 2018 IEEE International Conference on Robotics and Automation (ICRA), Brisbane, Australia.10.1109/ICRA.2018.8460487 ↗
23. Wang, Q., Chen, L., and Tian, W. (2018). End-to-end driving simulation via angle branched network. arXiv.
24. Bouton, M., Karlsson, J., Nakhaei, A., Fujimura, K., Kochenderfer, M.J., and Tumova, J. (2019). Reinforcement learning with probabilistic guarantees for autonomous driving. arXiv.
25. Chen, J., Yuan, B., and Tomizuka, M. (, January 27–30). Model-free deep reinforcement learning for urban autonomous driving. Proceedings of the 2019 IEEE Intelligent Transportation Systems Conference (ITSC), Auckland, New Zealand.10.1109/ITSC.2019.8917306 ↗
26. Kendall, A., Hawke, J., Janz, D., Mazur, P., Reda, D., Allen, J.-M., Lam, V.-D., Bewley, A., and Shah, A. (2019, January 20–24). Learning to drive in a day. Proceedings of the 2019 International Conference on Robotics and Automation (ICRA), Montreal, QC, Canada.10.1109/ICRA.2019.8793742 ↗
27. Peng, M., Gong, Z., Sun, C., Chen, L., and Cao, D. (August, January 31). Imitative Reinforcement Learning Fusing Vision and Pure Pursuit for Self-driving. Proceedings of the 2020 IEEE International Conference on Robotics and Automation (ICRA), Paris, France.10.1109/ICRA40945.2020.9197027 ↗
28. Liang, X., Wang, T., Yang, L., and Xing, E. (2018, January 8–14). Cirl: Controllable imitative reinforcement learning for vision-based self-driving. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.10.1007/978-3-030-01234-2_36 ↗