journal article·2025-02-03Open access

Dual-Modality Transformer with Time Series Imaging for Robust Epileptic Seizure Prediction

Jiahao Qin, Zijia Liu, Jihong Zhuang, Feng Liu

Applied Sciences· Volume 15· Issue 3

Abstract

Automated EEG classification algorithms for seizures can facilitate the clinical diagnosis of epilepsy, enabling more expedient and precise classification. However, existing EEG signal preprocessing methods oriented towards artifact removal and signal enhancement have demonstrated suboptimal accuracy and robustness. In response to this challenge, we propose an Adaptive Dual-Modality Learning Model (ADML) for epileptic seizure prediction by combining time series imaging with Transformer-based architecture. Our approach effectively captures both temporal dependencies and spatial relationships in EEG signals through a specialized attention mechanism. Evaluated on the CHB-MIT and Bonn datasets, our method achieves 98.7% and 99.2% accuracy, respectively, significantly outperforming existing approaches. The model demonstrates strong generalization capability across datasets while maintaining computational efficiency. Cross-dataset validation confirms the robustness of our approach, with consistent performance above 96% accuracy. These results suggest that our dual-modality approach provides a reliable and practical solution for clinical epileptic seizure prediction.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: EEG and Brain-Computer Interfaces · Epilepsy research and treatment · Neonatal and fetal brain pathology

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

Faculty of Science and Engineering, University of Liverpool, Liverpool L69 7ZX, UK

School of Mathematics and Physics, Xi’an Jiaotong-Liverpool University, Suzhou 215123, China

School of Management, University of Liverpool, Liverpool L69 7ZX, UK

School of Computer Science and Technology, East China Normal University, Shanghai 200062, China

References (56)

Show source-provided references
2. Initial Management of Seizure in Adults10.1056/NEJMcp2024526 ↗
4. Epilepsy in China: Major progress in the past two decades10.1016/S1474-4422(21)00023-5 ↗
5. Improving epilepsy diagnosis across the lifespan: Approaches and innovations10.1016/S1474-4422(24)00079-6 ↗
6. Epilepsy detection in 121 patient populations using hypercube pattern from EEG signals10.1016/j.inffus.2023.03.022 ↗
7. An Attention-Based Wavelet Convolution Neural Network for Epilepsy EEG Classification10.1109/TNSRE.2022.3166181 ↗
8. Ahmad, I., Wang, X., Javeed, D., Kumar, P., Samuel, O.W., and Chen, S. (2023). A Hybrid Deep Learning Approach for Epileptic Seizure Detection in EEG signals. IEEE J. Biomed. Health Inform., 1–12.10.1109/JBHI.2023.3265983 ↗
9. Assessment of Multivariate Information Transmission in Space-Time-Frequency Domain: A Case Study for EEG Signals10.1109/TNSRE.2023.3260143 ↗
10. Boonyakitanont, P., Lek-uthai, A., Chomtho, K., and Songsiri, J. (2020). A review of feature extraction and performance evaluation in epileptic seizure detection using EEG. Biomed. Signal Process. Control, 57.10.1016/j.bspc.2019.101702 ↗
11. Cura, O.K., Ozdemir, M.A., and Akan, A. (2021, January 18–21). Epileptic EEG Classification Using Synchrosqueezing Transform with Machine and Deep Learning Techniques. Proceedings of the 2020 28th European Signal Processing Conference (EUSIPCO), Amsterdam, The Netherlands.10.23919/Eusipco47968.2020.9287347 ↗
12. Qiu, X., Yan, F., and Liu, H. (2023). A difference attention ResNet-LSTM network for epileptic seizure detection using EEG signal. Biomed. Signal Process. Control, 83.10.1016/j.bspc.2023.104652 ↗
13. Mumtaz, W., Rasheed, S., and Irfan, A. (2021). Review of challenges associated with the EEG artifact removal methods. Biomed. Signal Process. Control, 68.10.1016/j.bspc.2021.102741 ↗
14. Diagnostics and prognostics of planetary gearbox using CWT, auto regression (AR) and K-means algorithm10.1016/j.apacoust.2021.108314 ↗
15. Limitations in wavelet analysis of non-stationary atmospheric gravity wave signatures in temperature profiles10.5194/amt-17-4659-2024 ↗
16. Konar, P., Saha, M., Sil, J., and Chattopadhyay, P. (2013, January 16–19). Fault diagnosis of induction motor using CWT and rough-set theory. Proceedings of the 2013 IEEE Symposium on Computational Intelligence in Control and Automation (CICA). IEEE, Singapore.10.1109/CICA.2013.6611658 ↗
17. An enhanced empirical wavelet transform for noisy and non-stationary signal processing10.1016/j.dsp.2016.09.012 ↗
18. IC-U-Net: A U-Net-based Denoising Autoencoder Using Mixtures of Independent Components for Automatic EEG Artifact Removal10.1016/j.neuroimage.2022.119586 ↗
19. Motion Artifacts Suppression From EEG Signals Using an Adaptive Signal Denoising Method10.1109/TIM.2022.3142037 ↗
20. EEG channel selection using Gramian Angular Fields and spectrograms for energy data visualization10.1016/j.engappai.2024.108305 ↗
21. A Long Short-Term Memory Network for Sparse Spatiotemporal EEG Source Imaging10.1109/TMI.2021.3097758 ↗
22. Multivariate Time Series Forecasting Based on Elastic Net and High-Order Fuzzy Cognitive Maps: A Case Study on Human Action Prediction Through EEG Signals10.1109/TFUZZ.2020.2998513 ↗
23. Forecasting with time series imaging10.1016/j.eswa.2020.113680 ↗
24. Lu, L., and Zhiguang, W. (2018, January 2–7). Encoding temporal markov dynamics in graph for visualizing and mining time series. Proceedings of the Workshops at the Thirty-Second AAAI Conference on Artificial Intelligence, New Orleans, LA, USA.
25. Jiang, J.R., and Yen, C.T. (2020, January 28–30). Markov Transition Field and Convolutional Long Short-Term Memory Neural Network for Manufacturing Quality Prediction. Proceedings of the 2020 IEEE International Conference on Consumer Electronics—Taiwan (ICCE-Taiwan), Taoyuan, Taiwan.10.1109/ICCE-Taiwan49838.2020.9258285 ↗
26. Wang, Z., and Oates, T. (2015). Imaging time-series to improve classification and imputation. arXiv.
27. Discrimination of Types of Seizure Using Brain Rhythms Based on Markov Transition Field and Deep Learning10.1109/OJIM.2022.3202555 ↗
28. Emotion stimuli-based surface electromyography signal classification employing Markov transition field and deep neural networks10.1016/j.measurement.2021.110470 ↗
29. Identification and monitoring of brain activity based on stochastic relevance analysis of short–time EEG rhythms10.1186/1475-925X-13-S1-I1 ↗
30. Chen, D., Wan, S., Xiang, J., and Bao, F.S. (2017). A high-performance seizure detection algorithm based on Discrete Wavelet Transform (DWT) and EEG. PLoS ONE, 12.10.1371/journal.pone.0173138 ↗
31. Seizure classification in EEG signals utilizing Hilbert-Huang transform10.1186/1475-925X-10-38 ↗
32. EMD-based temporal and spectral features for the classification of EEG signals using supervised learning10.1109/TNSRE.2015.2441835 ↗
33. Application of tunable-Q wavelet transform based nonlinear features in epileptic seizure detection10.1007/s10470-019-01424-y ↗
34. Mean amplitude spectrum based epileptic state classification for seizure prediction using convolutional neural networks10.1007/s12652-019-01220-6 ↗
35. A generalized synchrosqueezing transform for enhancing signal time–frequency representation10.1016/j.sigpro.2012.02.019 ↗
36. Time–Frequency Representation and Convolutional Neural Network-Based Emotion Recognition10.1109/TNNLS.2020.3008938 ↗
37. Zhang, R., Jia, J., and Zhang, R. (2022). EEG analysis of Parkinson’s disease using time–frequency analysis and deep learning. Biomed. Signal Process. Control, 78.10.1016/j.bspc.2022.103883 ↗
38. An efficient seizure prediction method using KNN-based undersampling and linear frequency measures10.1016/j.jneumeth.2014.05.019 ↗
39. EEG signal processing for epilepsy seizure detection using 5-level Db4 discrete wavelet transform, GA-based feature selection and ANN/SVM classifiers10.1007/s12652-020-02837-8 ↗
40. Automated diagnosis of epileptic EEG using entropies10.1016/j.bspc.2011.07.007 ↗
41. A multi-channel feature fusion CNN-BI-LSTM epilepsy EEG classification and prediction model based on attention mechanism10.1109/ACCESS.2023.3287927 ↗
42. Random neural networks with negative and positive signals and product form solution10.1162/neco.1989.1.4.502 ↗
43. A review about deep learning methods and applications
44. Wang, X., Wang, Y., Liu, D., Wang, Y., and Wang, Z. (2023). Automated recognition of epilepsy from EEG signals using a combining space–time algorithm of CNN-LSTM. Sci. Rep., 13.10.1038/s41598-023-41537-z ↗
45. Najafi, T., Jaafar, R., Remli, R., and Wan Zaidi, W.A. (2022). A classification model of EEG signals based on rnn-lstm for diagnosing focal and generalized epilepsy. Sensors, 22.10.3390/s22197269 ↗
46. Velasco-Gallego, C., and Lazakis, I. (September, January 28). Analysis of Time Series Imaging Approaches for the Application of Fault Classification of Marine Systems. Proceedings of the 32nd European Safety and Reliability Conference, Dublin, Ireland.
47. Wang, Z., and Oates, T. (2015, January 25–30). Encoding time series as images for visual inspection and classification using tiled convolutional neural networks. Proceedings of the Workshops at the Twenty-Ninth AAAI Conference on Artificial Intelligence, Austin, TX, USA.
48. Ko, D.W., and Yang, J.J. (2022). EEG-Based schizophrenia diagnosis through time series image conversion and deep learning. Electronics, 11.10.3390/electronics11142265 ↗
49. Zhao, D.D., and Zhao, Q. (2023, January 17–19). EEG-Based Cross-Subject Emotion Recognition Using GADF and CADAN. Proceedings of the 2023 China Automation Congress (CAC), Chongqing, China.10.1109/CAC59555.2023.10450444 ↗
50. Shankar, A., Khaing, H.K., Dandapat, S., and Barma, S. (2020, January 7–9). Epileptic seizure classification based on Gramian angular field transformation and deep learning. Proceedings of the 2020 IEEE Applied Signal Processing Conference (ASPCON). IEEE, Kolkata, India.10.1109/ASPCON49795.2020.9276717 ↗
51. Xu, G., Ren, T., Chen, Y., and Che, W. (2020). A One-Dimensional CNN-LSTM Model for Epileptic Seizure Recognition Using EEG Signal Analysis. Front. Neurosci., 14.10.3389/fnins.2020.578126 ↗
52. Epileptic seizure prediction via multidimensional transformer and recurrent neural network fusion10.1186/s12967-024-05678-7 ↗
53. Exploring the Applicability of Transfer Learning and Feature Engineering in Epilepsy Prediction Using Hybrid Transformer Model10.1109/TNSRE.2023.3244045 ↗
54. EEG-based seizure prediction via hybrid vision transformer and data uncertainty learning10.1016/j.engappai.2023.106401 ↗
55. EEGWaveNet: Multiscale CNN-Based Spatiotemporal Feature Extraction for EEG Seizure Detection10.1109/TII.2021.3133307 ↗
56. Detection of epileptic seizure using EEG signals analysis based on deep learning techniques10.1016/j.chaos.2024.114700 ↗