Mitigating Long-Term Forecasting Bias in Time-Series Neural Networks via Ensemble of Short-Term Dependencies
Source abstract
Abstract
Time-series forecasting is essential for predicting future trends based on historical data, with significant applications in meteorology, transportation, and finance. However, existing models often exhibit unsatisfactory performance in long-term forecasting scenarios. To address this limitation, we propose the Time-Series Neural Networks via Ensemble of Short-Term Dependencies (TSNN-ESTD). This model leverages iTransformer as the base predictor to simultaneously train short-term and long-term forecasting models. The vanilla iTransformer’s linear decoding layer is optimized by replacing it with an LSTM layer, and an additional long-term model is introduced to enhance stability. The ensemble strategy employs short-term predictions to correct the bias in long-term forecasts. Our extensive experiments demonstrate that TSNN-ESTD reduces the MSE and MAE by 9.17% and 2.3% on five benchmark datasets.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Stock Market Forecasting Methods · Time Series Analysis and Forecasting · Energy Load and Power Forecasting
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
School of Computer and Information Engineering, Shanghai Polytechnic University, Shanghai 201209, China
Funding
Natural Science Foundation of Shanghai · National Natural Science Foundation of China · Shanghai Sailing Program
References (36)
Show source-provided references
1. Deep learning for precipitation nowcasting: A survey from the perspective of time series forecasting10.1016/j.eswa.2024.126301 ↗
2. JDeep learning-based time series forecasting10.1007/s10462-024-10989-8 ↗
3. Box, G.E.P., Jenkins, G.M., Reinsel, G.C., and Ljung, G.M. (2015). Time Series Analysis: Forecasting and Control, John Wiley & Sons. [5th ed.].
4. Mehrmolaei, S., and Keyvanpour, M.R. (2016, January 9). Time series forecasting using improved ARIMA. Proceedings of the 2016 Artificial Intelligence and Robotics (IRANOPEN), Qazvin, Iran.10.1109/RIOS.2016.7529496 ↗
5. Forecasting seasonals and trends by exponentially weighted moving averages10.1016/j.ijforecast.2003.09.015 ↗
6. Ralaivola, L., and D’Alche-Buc, F. (2008, January 1–8). Time Series Filtering, Smoothing and Learning using the Kernel Kalman Filter. Proceedings of the IEEE International Joint Conference on Neural Networks, Hong Kong, China.
7. A new approach to the economic analysis of nonstationary time series and the business cycle
8. A state space framework for automatic forecasting using exponential smoothing methods10.1016/S0169-2070(01)00110-8 ↗
9. Machine learning advances for time series forecasting10.1111/joes.12429 ↗
10. 25 years of time series forecasting10.1016/j.ijforecast.2006.01.001 ↗
11. An empirical comparison of machine learning models for time series forecasting10.1080/07474938.2010.481556 ↗
12. Time series forecasting using a hybrid ARIMA and neural network model10.1016/S0925-2312(01)00702-0 ↗
13. Support vector machine with adaptive parameters in financial time series forecasting10.1109/TNN.2003.820556 ↗
14. Meta-learning for time series forecasting and forecast combination10.1016/j.neucom.2009.09.020 ↗
15. A gradient boosting approach to the Kaggle load forecasting competition10.1016/j.ijforecast.2013.07.005 ↗
16. Bontempi, G., Ben Taieb, S., and Le Borgne, Y.A. (2012). Machine learning strategies for time series forecasting. European Big Data Management and Analytics Summer School, Springer.10.1007/978-3-642-36318-4_3 ↗
17. Zeng, A., Chen, M., Zhang, L., and Xu, Q. (2023, January 7–14). Are transformers effective for time series forecasting?. Proceedings of the AAAI Conference on Artificial Intelligence, Washington, DC, USA.
18. Li, Z., Qi, S., Li, Y., and Xu, Z. (2023). Revisiting Long-term Time Series Forecasting: An Investigation on Linear Mapping. arXiv.
19. Zhou, N., Zeng, H., and Zhou, J. (2024, January 19–21). DLinear-Based Prediction of the RUL of PEMFC. Proceedings of the 2024 4th International Conference on Consumer Electronics and Computer Engineering (ICCECE), Guangzhou, China.10.1109/ICCECE61317.2024.10504160 ↗
20. Festag, S., Denzler, J., and Sprechelsen, C. (2022). Generative adversarial networks for biomedical time series forecasting and imputation. J. Biomed. Inform., 129.10.1016/j.jbi.2022.104058 ↗
21. Staffini, A. (2022). Stock Price Forecasting by a Deep Convolutional Generative Adversarial Network. Front. Artif. Intell., 5.10.3389/frai.2022.837596 ↗
22. Oreshkin, B., Carpov, D., and Chapados, N. (2020, January 26–30). Neural basis expansion analysis for interpretable time series forecasting. Proceedings of the International Conference on Learning Representations, Addis Ababa, Ethiopia.
23. Bai, S.J., Kolter, J.Z., and Koltun, V. (2018). An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling. arXiv.
24. Ultra-short term wind power prediction applying a novel model named SATCN-LSTM10.1016/j.enconman.2021.115036 ↗
25. Attention-based temporal–spatial convolutional network for ultra-short-term load forecasting10.1016/j.epsr.2023.109329 ↗
26. Deep learning for time series forecasting: Tutorial and literature survey10.1145/3533382 ↗
27. Woo, G., Liu, C., Kumar, A., Xiong, C., Savarese, S., and Sahoo, D. (2024, January 21–27). Unified training of universal time series forecasting transformers. Proceedings of the International Conference on Machine Learning, Vienna, Austria.
28. Liu, Y., Hu, T., Zhang, H., Wu, H., Wang, S., Ma, L., and Long, M. (2023, January 1–5). itransformer: Inverted transformers are effective for time series forecasting. Proceedings of the International Conference on Learning Representations, Kigali, Rwanda.
29. Nie, Y., Nguyen, N.H., Sinthong, P., and Kalagnanam, J. (2023, January 1–5). A time series is worth 64 words: Long-term forecasting with transformers. Proceedings of the International Conference on Learning Representations, Kigali, Rwanda.
30. Zhang, Y., and Yan, J. (2023, January 1–5). Crossformer: Transformer utilizing cross-dimension dependency for multivariate time series forecasting. Proceedings of the International Conference on Learning Representations, Kigali, Rwanda.
31. Das, A., Kong, W., Leach, A., Mathur, S., Sen, R., and Yu, R. (2023). Long-term forecasting with tide: Time-series dense encoder. arXiv.
32. Wu, H., Hu, T., Liu, Y., Zhou, H., Wang, J., and Long, M. (2023, January 1–5). Timesnet: Temporal 2d-variation modeling for general time series analysis. Proceedings of the International Conference on Learning Representations, Kigali, Rwanda.
33. Scinet: Time series modeling and forecasting with sample convolution and interaction
34. Zhou, T., Ma, Z., Wen, Q., Wang, X., Sun, L., and Jin, R. (2022, January 17–23). Fedformer: Frequency enhanced decomposed transformer for long-term series forecasting. Proceedings of the International Conference on Machine Learning, PMLR, Baltimore, MD, USA.
35. Non-stationary transformers: Exploring the stationarity in time series forecasting
36. Autoformer: Decomposition transformers with auto-correlation for long-term series forecasting