journal article·2026-03Open access

Pack Transformer: Transformer-based hybrid model for lithium-ion battery packs

Habtamu Hailemichael, Beshah Ayalew, Miriam Figueroa-Santos, Morgan Barron

Abstract

Accurate modeling of lithium-ion battery (LiB) packs is crucial for enhancing their performance, longevity, and safety. This requires capturing the complex electrochemical and thermal dynamics within individual pack segments (e.g., cells) and the dynamic interactions across the pack. In this work, we propose a novel hybrid modeling approach that integrates a structurally identifiable distributed equivalent circuit model (ECM) coupled with a lumped thermal (LT) model for each pack segment with a Transformer-based residual that corrects ECM< model predictions. By leveraging the Transformer’s self-attention mechanism, the residual enables high-resolution modeling of both temporal dependencies and spatial interactions between pack segments. To reduce the computational burden and enable LiB dynamics predictions over long sequences, we evaluate efficient self-attention mechanisms for our Transformer-based hybrid model (Pack Transformer), including Longformer, Factorized Encoder, and Factorized Attention architectures. In numerical experiments using a high-fidelity LiB pack model as a reference, we find that the Pack Transformer successfully predicts state trajectories for each pack segment over extended periods, achieving very low RMSE values of 0.008V for voltage and 0.1 °C for temperature. Compared to simpler hybrid model approaches that use varying levels of lumping, the transformer model is able to capture the spatio-temporal dynamics of the pack at a much higher resolution. Our approach paves the way for more accurate and safer predictive control of LiB packs, including applications in battery balancing and thermal management.
Source: openalex_inverted_index · Quality: baseline

Research context

OpenAlex topics: Advanced Battery Technologies Research · Electromagnetic Compatibility and Noise Suppression · Advanced DC-DC Converters

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

References (27)

Show source-provided references
1. Attention is all you need
2. Plett · 2015
3. Cell state-of-charge inconsistency estimation for LiFePO4 battery pack in hybrid electric vehicles using mean-difference model10.1016/j.apenergy.2013.05.048 ↗
4. A model-based state-of-charge estimation method for series-connected lithium-ion battery pack considering fast-varying cell temperature10.1016/j.energy.2019.07.063 ↗
5. Modeling of galvanostatic charge and discharge of the lithium/polymer/insertion cell10.1149/1.2221597 ↗
6. Review of models for predicting the cycling performance of lithium ion batteries10.1016/j.jpowsour.2005.05.070 ↗
7. State-of-charge estimation of li-ion batteries using deep neural networks: A machine learning approach10.1016/j.jpowsour.2018.06.104 ↗
8. State-of-charge estimation of lithium-ion batteries based on gated recurrent neural network10.1016/j.energy.2019.03.059 ↗
9. Continual hybrid model learning for lithium-ion batteries10.1016/j.est.2024.115105 ↗
10. Deep learning approach towards accurate state of charge estimation for lithium-ion batteries using self-supervised transformer model10.1038/s41598-021-98915-8 ↗
11. State of charge estimation for lithium-ion battery using transformer with immersion and invariance adaptive observer10.1016/j.est.2021.103768 ↗
12. Beltagy · 2020
13. Is space-time attention all you need for video understanding?
14. A. Arnab, M. Dehghani, G. Heigold, C. Sun, M. Lučić, C. Schmid, Vivit: A video vision transformer, in: Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021, pp. 6836–6846.10.1109/ICCV48922.2021.00676 ↗
15. Optimal fast charging of lithium-ion batteries through continual hybrid model learning10.1016/j.ifacol.2025.01.081 ↗
16. Adaptive and safe fast charging of lithium-ion batteries via hybrid model learning and control barrier functions10.1016/j.est.2025.117056 ↗
17. Dosovitskiy · 2020
18. Fundamentals of recurrent neural network (RNN) and long short-term memory (LSTM) network10.1016/j.physd.2019.132306 ↗
19. Roformer: Enhanced transformer with rotary position embedding10.1016/j.neucom.2023.127063 ↗
20. Rotary position embedding for vision transformer
21. Thermal-electrochemical parameters of a high energy lithium-ion cylindrical battery
22. Python battery mathematical modelling (PyBaMM)10.5334/jors.309 ↗
23. Steinstraeter · 2020
24. Learning phrase representations using RNN encoder-decoder for statistical machine translation
25. Loshchilov · 2017
26. Yu · 2017
27. A comprehensive equivalent circuit model for lithium-ion batteries, incorporating the effects of state of health, state of charge, and temperature on model parameters10.1016/j.est.2021.103252 ↗