Wavelet-Enhanced Transformer for Adaptive Multi-Period Time Series Forecasting
Source abstract
Abstract
Time series analysis is of critical importance in a wide range of applications, including weather forecasting, anomaly detection, and action recognition. Accurate time series forecasting requires modeling complex temporal dependencies, particularly multi-scale periodic patterns. To address this challenge, we propose a novel Wavelet-Enhanced Transformer (Wave-Net). Wave-Net transforms 1D time series data into 2D matrices based on periodicity, enhancing the capture of temporal patterns through convolutional filters. This paper introduces Wave-Net, a model that incorporates wavelet and Fourier transforms for feature extraction, along with an enhanced cycle offset and optimized dynamic K for improved robustness. The Transformer layer is further refined to bolster long-term modeling capabilities. Evaluations on real-world benchmarks demonstrate that Wave-Net consistently achieves state-of-the-art performance across mainstream time series analysis tasks.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Time Series Analysis and Forecasting · Stock Market Forecasting Methods · Traffic Prediction and Management Techniques
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Faculty of Data Science, City University of Macau, Macau 999078, China
Funding
Guangdong Peizheng College
References (54)
Show source-provided references
1. Autoformer: Decomposition transformers with auto-correlation for long-term series forecasting
2. Xu, J., Wu, H., Wang, J., and Long, M. (2021). Anomaly transformer: Time series anomaly detection with association discrepancy. arXiv.
3. The interpolation of time series by related series10.1080/01621459.1962.10500812 ↗
4. Unsupervised scalable representation learning for multivariate time series
5. Time-series forecasting with deep learning: A survey10.1098/rsta.2020.0209 ↗
6. Time-series data mining10.1145/2379776.2379788 ↗
7. Deep learning for time series analysis: A survey10.1145/3379504 ↗
8. Box, G.E., Jenkins, G.M., Reinsel, G.C., and Ljung, G.M. (2015). Time Series Analysis: Forecasting and Control, John Wiley & Sons.
9. Hamilton, J.D. (2020). Time Series Analysis, Princeton University Press.10.2307/j.ctv14jx6sm ↗
10. Markovian RNN: An adaptive time series prediction network with HMM-based switching for nonstationary environments10.1109/TNNLS.2021.3100528 ↗
11. HyPhAI v1.0: Hybrid Physics-AI architecture for cloud cover nowcasting
12. STA-TCN: Spatial-temporal attention over temporal convolutional network for next point-of-interest recommendation10.1145/3596497 ↗
13. A novel electronic nose classification prediction method based on TETCN10.1016/j.snb.2024.135272 ↗
14. Scatterbrain: Unifying sparse and low-rank attention
15. Efficient content-based sparse attention with routing transformers10.1162/tacl_a_00353 ↗
16. A Novel Spatiotemporal Correlation Anomaly Detection Method Based on Time-Frequency-Domain Feature Fusion and a Dynamic Graph Neural Network in Wireless Sensor Network10.1109/JSEN.2025.3549220 ↗
17. A frequency-spatial hybrid attention mechanism improved tool wear state recognition method guided by structure and process parameters10.1016/j.measurement.2023.112833 ↗
18. Cheng, M., Liu, Z., Tao, X., Liu, Q., Zhang, J., Pan, T., Zhang, S., He, P., Zhang, X., and Wang, D. (2025). A comprehensive survey of time series forecasting: Concepts, challenges, and future directions. TechRxiv.10.36227/techrxiv.174430535.53879341/v1 ↗
19. Wu, H., Hu, T., Liu, Y., Zhou, H., Wang, J., and Long, M. (2022). Timesnet: Temporal 2d-variation modeling for general time series analysis. arXiv.
20. Attention based spatial-temporal graph convolutional networks for traffic flow forecasting
21. Rethinking long-term time series forecasting: From the perspective of frequency domain analysis10.1145/3623404 ↗
22. Wang, Z., Chen, Y., Liu, W., and Sun, J. (2024). Mamba: Multi-scale analysis with memory-based attention for time series forecasting. Adv. Neural Inf. Process. Syst., 36.
23. Zhou, H., Zhang, S., Peng, J., Zhang, S., Li, J., Xiong, H., and Zhang, W. (2021, January 2–9). Informer: Beyond efficient transformer for long sequence time-series forecasting. Proceedings of the AAAI Conference on Artificial Intelligence, Virtually. Version 3.
24. STL: A seasonal-trend decomposition
25. Autoregressive conditional heteroscedasticity with estimates of the variance of United Kingdom inflation
26. Learning representations by back-propagating errors10.1038/323533a0 ↗
27. Long short-term memory10.1162/neco.1997.9.8.1735 ↗
28. Bai, S., Kolter, J.Z., and Koltun, V. (2018). An empirical evaluation of generic convolutional and recurrent networks for sequence modeling. arXiv.
29. Neural network models for time series forecasts10.1287/mnsc.42.7.1082 ↗
30. Oreshkin, B.N., Carpov, D., Chapados, N., and Bengio, Y. (2019, January 6–9). N-BEATS: Neural basis expansion analysis for interpretable time series forecasting. Proceedings of the International Conference on Learning Representations, New Orleans, LA, USA.
31. Attention is all you need
32. Li, S., Jin, X., Xuan, Y., Zhou, X., Chen, W., Wang, Y.X., and Yan, X. (2019, January 8–14). Enhancing the locality and breaking the memory bottleneck of transformer on time series forecasting. Proceedings of the Advances in Neural Information Processing Systems, Vancouver, BC, Canada.
33. Kitaev, N., Kaiser, L., and Levskaya, A. (2020, January 26–30). Reformer: The efficient transformer. Proceedings of the International Conference on Learning Representations, Addis Ababa, Ethiopia.
34. Liu, S., Yu, H., Liao, J., Li, J., Lin, W., Liu, A.X., and Gao, S. (2021, January 3–7). Pyraformer: Low-complexity pyramidal attention for long-range time series modeling and forecasting. Proceedings of the International Conference on Learning Representations, Online.
35. Zhou, T., Ma, Z., Wen, Q., Wang, X., Sun, L., and Jin, R. (2022, January 17–23). Fedformer: Frequency enhanced decomposed transformer for long-term series forecasting. Proceedings of the International Conference on Machine Learning, PMLR, Baltimore, MD, USA.
36. Zhang, Y., and Yan, J. (2023, January 1–5). Crossformer: Transformer utilizing cross-dimension dependency for multivariate time series forecasting. Proceedings of the Eleventh International Conference on Learning Representations, Kigali, Rwanda.
37. Liu, Y., Hu, T., Zhang, H., Wu, H., Wang, S., and Long, M. (2022, January 10–16). Non-stationary transformers: Exploring the stationarity in time series forecasting. Proceedings of the Advances in Neural Information Processing Systems, New Orleans, LA, USA.
38. Nie, Y., Nguyen, N.H., Sinthong, P., and Kalagnanam, J. (2022, January 25–29). A time series is worth 64 words: Long-term forecasting with transformers. Proceedings of the International Conference on Learning Representations, Virtual Event.
39. Goodfellow, I., Bengio, Y., and Courville, A. (2016). Deep Learning, MIT Press.
40. Determination of temporal and spatial characteristics of atmospheric gravity waves combining cross-spectral analysis and wavelet transformation10.1029/2004JD004906 ↗
41. Dai, T., Wu, B., Liu, P., Li, N., Bao, J., Jiang, Y., and Xia, S.T. (2024, January 7–11). Periodicity decoupling framework for long-term series forecasting. Proceedings of the Twelfth International Conference on Learning Representations, Vienna, Austria.
42. Nussbaumer, H.J. (1981). The fast Fourier transform. Fast Fourier Transform and Convolution Algorithms, Springer.10.1007/978-3-662-00551-4 ↗
43. Leveraging hybrid deep learning models for enhanced multivariate time series forecasting10.1007/s11063-024-11656-3 ↗
44. Video salient object detection via fully convolutional networks10.1109/TIP.2017.2754941 ↗
45. Hyndman, R.J., and Athanasopoulos, G. (2021). Forecasting: Principles and practice, A Widely Recognized Textbook That Explains and Uses MSE and MAE as Fundamental Accuracy Measures; OTexts. [3rd ed.].
46. Root mean square error (RMSE) or mean absolute error (MAE)?—Arguments against avoiding RMSE in the literature10.5194/gmd-7-1247-2014 ↗
47. Liu, Y., Hu, T., Zhang, H., Wu, H., Wang, S., Ma, L., and Long, M. (2023). itransformer: Inverted transformers are effective for time series forecasting. arXiv.
48. Woo, G., Liu, C., Sahoo, D., Kumar, A., and Hoi, S.C.H. (2022). ETSformer: Exponential Smoothing Transformers for Time-series Forecasting. arXiv.
49. Liu, Y., Li, G., Payne, T.R., Yue, Y., and Man, K.L. (2024). Non-Stationary Transformer Architecture: A Versatile Framework for Recommendation Systems. Electronics, 13.10.20944/preprints202405.0378.v1 ↗
50. Zhou, H., Zhang, S., Peng, J., Zhang, S., Li, J., Xiong, H., and Zhang, W. (2020). Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting. arXiv.10.1609/aaai.v35i12.17325 ↗
51. Su, C. (2022). DLinear Makes Efficient Long-Term Predictions, Association for Computing Machinery.
52. Long time series of ocean wave prediction based on PatchTST model10.1016/j.oceaneng.2024.117572 ↗
53. Zhang, T., Zhang, Y., Cao, W., Bian, J., Yi, X., Zheng, S., and Li, J. (2022). Less Is More: Fast Multivariate Time Series Forecasting with Light Sampling-oriented MLP Structures. arXiv.
54. Generalized autoregressive conditional heteroskedasticity10.1016/0304-4076(86)90063-1 ↗