Optimizing Pretrained Autonomous Driving Models Using Deep Reinforcement Learning
Source abstract
Abstract
Vision-based end-to-end navigation systems have shown impressive capabilities, especially when combined with Imitation Learning (IL) and advanced Deep Learning architectures, such as Transformers. One such example is CIL++, a Transformer-based architecture that learns to map navigation states to vehicle controls based on expert demonstrations only. Nevertheless, reliance on experts’ datasets limits generalization and can lead to failures in unknown circumstances. Deep Reinforcement Learning (DRL) can address this issue by fine-tuning the pretrained policy, using a reward function that aims to improve its weaknesses through interaction with the environment. However, fine-tuning with DRL can lead to the Catastrophic Forgetting (CF) problem, where a policy forgets the expert behaviors learned from the demonstrations as it learns to optimize the new reward function. In this paper, we present CILRLv3, a DRL-based training method that is immune to CF, enabling pretrained navigation agents to improve their driving skills across new scenarios.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Advanced Neural Network Applications · Domain Adaptation and Few-Shot Learning · Multimodal Machine Learning Applications
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
School of Informatics, Aristotle University of Thessasloniki, 54124 Thessaloniki, Greece
References (19)
Show source-provided references
1. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., and Polosukhin, I. (2017, January 4–9). Attention is all you need. Proceedings of the 31st Conference on Neural Information Processing Systems (NIPS 2017), Long Beach, CA, USA.
2. Carion, N., Massa, F., Synnaeve, G., Usunier, N., Kirillov, A., and Zagoruyko, S. (2020, January 23–28). End-to-end object detection with transformers. Proceedings of the European Conference on Computer Vision, Glasgow, UK.10.1007/978-3-030-58452-8_13 ↗
3. Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., and Gelly, S. (2020). An image is worth 16x16 words: Transformers for image recognition at scale. arXiv.
4. A survey of end-to-end driving: Architectures and training methods10.1109/TNNLS.2020.3043505 ↗
5. Video pretraining (vpt): Learning to act by watching unlabeled online videos
6. Liang, X., Wang, T., Yang, L., and Xing, E. (2018, January 8–14). Cirl: Controllable imitative reinforcement learning for vision-based self-driving. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.10.1007/978-3-030-01234-2_36 ↗
7. Kochliaridis, V., Kostinoudis, E., and Vlahavas, I. (2024, January 11–13). Optimizing Pretrained Transformers for Autonomous Driving. Proceedings of the 13th Hellenic Conference on Artificial Intelligence (SETN ’24), Piraeus, Greece.10.1145/3688671.3688778 ↗
8. Xiao, Y., Codevilla, F., Porres, D., and López, A.M. (2023, January 1–5). Scaling Vision-Based End-to-End Autonomous Driving with Multi-View Attention Learning. Proceedings of the 2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Detroit, MI, USA.10.1109/IROS55552.2023.10341506 ↗
9. Kochliaridis, V., Moumtzidellis, F., and Vlahavas, I. (2025, January 23–25). Scaling Multi-Frame Transformers for End-to-End Driving. Proceedings of the International Conference on Agents and Artificial Intelligence, Porto, Portugal.10.5220/0013156700003890 ↗
10. Park, Y., Jun, W., and Lee, S. (2025). A Comparative Study of Deep Reinforcement Learning Algorithms for Urban Autonomous Driving: Addressing the Geographic and Regulatory Challenges in CARLA. Appl. Sci., 15.10.3390/app15126838 ↗
11. Lu, Y., Fu, J., Tucker, G., Pan, X., Bronstein, E., Roelofs, R., Sapp, B., White, B., Faust, A., and Whiteson, S. (2023, January 1–5). Imitation is not enough: Robustifying imitation with reinforcement learning for challenging driving scenarios. Proceedings of the 2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Detroit, MI, USA.10.1109/IROS55552.2023.10342038 ↗
12. Haarnoja, T., Zhou, A., Abbeel, P., and Levine, S. (2018, January 10–15). Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor. Proceedings of the International Conference on Machine Learning, Stockholm, Sweden.
13. Coelho, D., Oliveira, M., and Santos, V. (2024, January 20–27). Rlfold: Reinforcement learning from online demonstrations in urban autonomous driving. Proceedings of the AAAI Conference on Artificial Intelligence, Vancouver, BC, Canada.10.1609/aaai.v38i10.29049 ↗
14. Wolczyk, M., Cupiał, B., Ostaszewski, M., Bortkiewicz, M., Zaja˛c, M., Pascanu, R., Kucin´ ski, Ł., and Miłos´, P. (2024, January 21–27). Fine-tuning Reinforcement Learning Models is Secretly a Forgetting Mitigation Problem. Proceedings of the International Conference on Machine Learning, Vienna, Austria.
15. Zhang, J.O., Sax, A., Zamir, A., Guibas, L., and Malik, J. (2020). Side-tuning: A baseline for network adaptation via additive side networks. Proceedings of the Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, 23–28 August 2020, Springer. Proceedings, Part III 16.
16. Cobbe, K.W., Hilton, J., Klimov, O., and Schulman, J. (2021, January 18–24). Phasic policy gradient. Proceedings of the International Conference on Machine Learning, Virtual Event.
17. Sutton, R.S., and Barto, A.G. (2018). Reinforcement Learning: An Introduction, MIT Press.
18. Dosovitskiy, A., Ros, G., Codevilla, F., Lopez, A., and Koltun, V. (2017, January 13–15). CARLA: An open urban driving simulator. Proceedings of the Conference on Robot Learning, Mountain View, CA, USA.
19. CGAOA-STRA-BiConvLSTM: An automated deep learning framework for global TEC map prediction10.1007/s10291-025-01814-y ↗