Project and Development of a Reinforcement Learning Based Control Algorithm for Hybrid Electric Vehicles
Source abstract
Abstract
Hybrid electric vehicles are, nowadays, considered as one of the most promising technologies for reducing on-road greenhouse gases and pollutant emissions. Such a goal can be accomplished by developing an intelligent energy management system which could lead the powertrain to exploit its maximum energetic performances under real-world driving conditions. According to the latest research in the field of control algorithms for hybrid electric vehicles, Reinforcement Learning has emerged between several Artificial Intelligence approaches as it has proved to retain the capability of producing near-optimal solutions to the control problem even in real-time conditions. Nevertheless, an accurate design of both agent and environment is needed for this class of algorithms. Within this paper, a detailed plan for the complete project and development of an energy management system based on Q-learning for hybrid powertrains is discussed. An integrated modular software framework for co-simulation has been developed and it is thoroughly described. Finally, results have been presented about a massive testing of the agent aimed at assessing for the change in its performance when different training parameters are considered.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Electric and Hybrid Vehicle Technologies · Electric Vehicles and Infrastructure · Transportation and Mobility Innovations
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Politecnico di Torino, Department of Energetics, Interdepartmental Center for Automotive Research and Sustainable Mobility (CARS@PoliTO), Corso Duca Degli Abruzzi 24, 10129 Turin, Italy
Addfor Industriale Srl, P.zza Solferino 7, 10121 Turin, Italy
Politecnico di Torino, Department of Mathematical Sciences, Corso Duca Degli Abruzzi 24, 10129 Turin, Italy
References (37)
Show source-provided references
1. Onori, S., Serrao, L., and Rizzoni, G. (2016). Hybrid Electric Vehicles, Springer.10.1007/978-1-4471-6781-5 ↗
2. A review of micro and mild hybrid systems10.1016/j.egyr.2019.08.077 ↗
3. Modelling and control of hybrid electric vehicles (A comprehensive review)10.1016/j.rser.2017.01.075 ↗
4. A survey on through-the-road hybrid electric vehicles10.3390/electronics9050879 ↗
5. A comprehensive review on hybrid electric vehicles: Architectures and components10.1007/s40534-019-0184-3 ↗
6. Analyzing the improvements of energy management systems for hybrid electric vehicles using a systematic literature review: How far are these controls from rule-based controls used in commercial vehicles?10.3390/app10238744 ↗
7. Energy Management in Plug-in Hybrid Electric Vehicles: Recent Progress and a Connected Vehicles Perspective10.1109/TVT.2016.2582721 ↗
8. Energy management systems for electrified powertrains: State-of-the-art review and future trends10.1109/TVT.2019.2914457 ↗
9. Hofman, T., van Druten, R.M., Steinbuch, M., and Serrarens, A.F.A. (2008). Rule-Based Equivalent Fuel Consumption Minimization Strategies for Hybrid Vehicles, IFAC. IFAC Proceedings Volumes 2008.10.3182/20080706-5-KR-1001.00953 ↗
10. Robust equivalent consumption-based controllers for a dual-mode diesel parallel HEV10.1016/j.enconman.2016.08.021 ↗
11. Power management strategy for a parallel hybrid electric truck10.1109/TCST.2003.815606 ↗
12. Model predictive control power management strategies for HEVs: A review10.1016/j.jpowsour.2016.11.106 ↗
13. Powertrain Control for Hybrid-Electric Vehicles Using Supervised Machine Learning10.3390/vehicles2020015 ↗
14. An unsupervised machine-learning technique for the definition of a rule-based control strategy in a complex HEV10.4271/2016-01-1243 ↗
15. A review of reinforcement learning based energy management systems for electrified powertrains: Progress, challenge, and potential solution10.1016/j.rser.2021.111833 ↗
16. Sutton, R.S., and Barto, A.G. (2018). Reinforcement Learning: An Introduction, MIT Press.
17. Zhu, Z., Liu, Y., and Canova, M. (2020, January 1–3). Energy Management of Hybrid Electric Vehicles via Deep Q-Networks. Proceedings of the 2020 American Control Conference (ACC), Denver, CO, USA.10.23919/ACC45564.2020.9147479 ↗
18. Energy management based on reinforcement learning with double deep Q-learning for a hybrid electric tracked vehicle10.1016/j.apenergy.2019.113708 ↗
19. Energy management for a power-split hybrid electric bus via deep reinforcement learning with terrain information10.1016/j.apenergy.2019.113762 ↗
20. Reinforcement Learning of Adaptive Energy Management With Transition Probability for a Hybrid Electric Tracked Vehicle10.1109/TIE.2015.2475419 ↗
21. Reinforcement Learning Optimized Look-Ahead Energy Management of a Parallel Hybrid Electric Vehicle10.1109/TMECH.2017.2707338 ↗
22. Optimization of Energy Management Strategy for Range-Extended Electric Vehicle Using Reinforcement Learning and Neural Network
23. Biswas, A., Anselma, P.G., and Emadi, A. (2019, January 19–21). Real-Time Optimal Energy Management of Electrified Powertrains with Reinforcement Learning. Proceedings of the 2019 IEEE Transportation Electrification Conference and Expo (ITEC), Detroit, MI, USA.10.1109/ITEC.2019.8790482 ↗
24. Parametric study on reinforcement learning optimized energy management strategy for a hybrid electric vehicle10.1016/j.apenergy.2019.114200 ↗
25. Ehsani, M., Gao, Y., Longo, S., and Ebrahimi, K.M. (2018). Modern Electric Hybrid Electric and Fuel Cell Vehicles, CRC Press.10.1201/9781420054002 ↗
26. Sundström, O., and Guzzella, L. (2009, January 8–10). A generic dynamic programming Matlab function. Proceedings of the 2009 IEEE Control Applications, (CCA) & Intelligent Control, (ISIC), St. Petersburg, Russia.10.1109/CCA.2009.5281131 ↗
27. Cost-optimized design of a dual-mode diesel parallel hybrid electric vehicle for several driving missions and market scenarios10.1016/j.apenergy.2016.05.080 ↗
28. Finesso, R., Misul, D., Spessa, E., and Venditti, M. (2018). Optimal design of power-split HEVs based on total cost of ownership and CO2 emission minimization. Energies, 11.10.3390/en11071705 ↗
29. Optimal mesh discretization of the dynamic programming for hybrid electric vehicles10.1016/j.apenergy.2021.116920 ↗
30. Spaan, M.T.J. (2012). Partially Observable Markon Decision Processes. Reinforcement Learning. Adaptation, Learning, and Optimization, Springer.10.1007/978-3-642-27645-3_12 ↗
31. Watkins, C.J.C.H. (1989). Learning from Delayed Rewards. [PhD Thesis, King’s College].
32. Brockman, G., Cheung, V., Pettersson, L., Schneider, J., Schulman, J., Tang, J., and Zaremba, W. (2016). OpenAI Gym. arXiv.
33. Van Rossum, G. (2020). The Python Lybrary Reference, Release 3.8.5, Python Software Foundation.
34. Finesso, R., Spessa, E., and Venditti, M. (2014). Optimization of the layout and control strategy for parallel through-the-road hybrid electric vehicles. SAE Tech. Pap., 1.10.4271/2014-01-1798 ↗
35. Anselma, P.G., Belingardi, G., Falai, A., Maino, C., Miretti, F., Misul, D., and Spessa, E. (2019, January 2–4). Comparing parallel hybrid electric vehicle powertrains for real-world driving. Proceedings of the 2019 AEIT International Conference of Electrical and Electronic Technologies for Automotive (AEIT AUTOMOTIVE), Turin, Italy.10.23919/EETA.2019.8804609 ↗
36. A deep neural network based model for the prediction of hybrid electric vehicles carbon dioxide emissions10.1016/j.egyai.2021.100073 ↗
37. Experimental analyses and clustering of travel choice behaviours by floating car big data in a large urban area10.1049/iet-its.2018.0015 ↗