A Survey of Machine Learning Methods for Time Series Prediction
Source abstract
Abstract
This study provides a comprehensive survey of the top-performing research papers in the field of time series prediction, offering insights into the most effective machine learning techniques, including tree-based, deep learning, and hybrid methods. It explores key factors influencing the model performance, such as the type of time series task, dataset size, and the time interval of historical data. Additionally, this study investigates potential biases in model development and weighs the trade-offs between the computational costs and performance. A detailed analysis of the most used error metrics and hyperparameter tuning methods in the reviewed papers is included. Furthermore, this study evaluates the results from prominent forecasting competitions, such as M5 and M6, to enrich the analysis. The findings of this paper highlight that tree-based methods like LightGBM 4.6.0 and deep learning methods like recurrent neural networks deliver the best performance in time series forecasting, with tree-based methods offering a significant advantage in terms of their computational efficiency. This paper concludes with practical recommendations for approaching time series forecasting tasks, offering valuable insights and actionable strategies that can enhance the accuracy and reliability of predictions derived from time series data.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Time Series Analysis and Forecasting · Energy Load and Power Forecasting · Traffic Prediction and Management Techniques
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Institute for Artificial Intelligence, University of Georgia, Athens, GA 30602, USA
References (95)
Show source-provided references
1. Time-series forecasting with deep learning: A survey10.1098/rsta.2020.0209 ↗
2. Forecast Methods for Time Series Data: A Survey10.1109/ACCESS.2021.3091162 ↗
3. Deep learning based multi-temporal crop classification10.1016/j.rse.2018.11.032 ↗
4. Short-term forecasting of passenger demand under on-demand ride services: A spatio-temporal deep learning approach10.1016/j.trc.2017.10.016 ↗
5. A Model Combining Convolutional Neural Network and LightGBM Algorithm for Ultra-Short-Term Wind Power Forecasting10.1109/ACCESS.2019.2901920 ↗
6. A Short-Term Load Forecasting Method Using Integrated CNN and LSTM Network10.1109/ACCESS.2021.3060654 ↗
7. Multi-step forecasting for big data time series based on ensemble learning10.1016/j.knosys.2018.10.009 ↗
8. Comparative study on total nitrogen prediction in wastewater treatment plant and effect of various feature selection methods on machine learning algorithms performance10.1016/j.jwpe.2021.102033 ↗
9. Comparative assessment of environmental variables and machine learning algorithms for maize yield prediction in the US Midwest10.1088/1748-9326/ab7df9 ↗
10. Advanced machine learning techniques for building performance simulation: A comparative analysis10.1080/19401493.2018.1498538 ↗
11. Time series prediction for output of multi-region solar power plants10.1016/j.apenergy.2019.114001 ↗
12. Accurate prediction of water quality in urban drainage network with integrated EMD-LSTM model10.1016/j.jclepro.2022.131724 ↗
13. Mazzia, V., Khaliq, A., and Chiaberge, M. (2020). Improvement in Land Cover and Crop Classification based on Temporal Features Learning from Sentinel-2 Data Using Recurrent-Convolutional Neural Network (R-CNN). Appl. Sci., 10.10.3390/app10010238 ↗
14. Rainfall prediction: A comparative analysis of modern machine learning algorithms for time-series forecasting
15. Shin, Y., Kim, T., Hong, S., Lee, S., Lee, E., Hong, S., Lee, C., Kim, T., Park, M.S., and Park, J. (2020). Prediction of Chlorophyll-a Concentrations in the Nakdong River Using Machine Learning Methods. Water, 12.10.3390/w12061822 ↗
16. Time series prediction of COVID-19 transmission in America using LSTM and XGBoost algorithms10.1016/j.rinp.2021.104462 ↗
17. Gradient boosting machine for predicting return temperature of district heating system: A case study for residential buildings in Tianjin10.1016/j.jobe.2019.100950 ↗
18. Predicting city-scale daily electricity consumption using data-driven models10.1016/j.adapen.2021.100025 ↗
19. Using graph structural information about flows to enhance short-term demand prediction in bike-sharing systems10.1016/j.compenvurbsys.2020.101521 ↗
20. Prediction of residential district heating load based on machine learning: A case study10.1016/j.energy.2021.120950 ↗
21. Zahoor Ali, K., Muhammad, A., Javaid, N., Malik, N.S., Shafiq, M., and Choi, J.-G. (2020). Electricity Theft Detection Using Supervised Learning Techniques on Smart Meter Data. Sustainability, 12.10.3390/su12198023 ↗
22. Hussein, E.A., Thron, C., Ghaziasgar, M., Bagula, A., and Vaccari, M. (2020). Groundwater Prediction Using Machine-Learning Tools. Algorithms, 13.10.3390/a13110300 ↗
23. Super-resolution of subsurface temperature field from remote sensing observations based on machine learning
24. Kumar, V., Kedam, N., Sharma, K.V., Mehta, D.J., and Caloiero, T. (2023). Advanced Machine Learning Techniques to Improve Hydrological Prediction: A Comparative Analysis of Streamflow Prediction Models. Water, 15.10.3390/w15142572 ↗
25. Short-Term Prediction of Urban Rail Transit Passenger Flow in External Passenger Transport Hub Based on LSTM-LGB-DRS10.1109/TITS.2020.3017109 ↗
26. A new dam structural response estimation paradigm powered by deep learning and transfer learning techniques10.1177/14759217211009780 ↗
27. Prediction of Highway Tunnel Pavement Performance Based on Digital Twin and Multiple Time Series Stacking10.1155/2020/8824135 ↗
28. A deep LSTM network for the Spanish electricity consumption forecasting10.1007/s00521-021-06773-2 ↗
29. Predicting post-stroke pneumonia using deep neural network approaches10.1016/j.ijmedinf.2019.103986 ↗
30. Pearson Correlation Coefficient-Based Performance Enhancement of Broad Learning System for Stock Price Prediction
31. A Contemporary Review on Drought Modeling Using Machine Learning Approaches
32. A Cross-Sectional Machine Learning Approach for Hedge Fund Return Prediction and Selection10.1287/mnsc.2020.3696 ↗
33. López Santos, M., García-Santiago, X., Echevarría Camarero, F., Blázquez Gil, G., and Carrasco Ortega, P. (2022). Application of Temporal Fusion Transformer for Day-Ahead PV Power Forecasting. Energies, 15.10.3390/en15145232 ↗
34. A proactive lane-changing risk prediction framework considering driving intention recognition and different lane-changing patterns10.1016/j.aap.2021.106500 ↗
35. Prodhan, F.A., Zhang, J., Yao, F., Shi, L., Pangali Sharma, T.P., Zhang, D., Cao, D., Zheng, M., Ahmed, N., and Mohana, H.P. (2021). Deep Learning for Monitoring Agricultural Drought in South Asia Using Remote Sensing Data. Remote Sens., 13.10.3390/rs13091715 ↗
36. Khan, P.W., Byun, Y.C., Park, N., Waqas Khan, P., Byun, Y.-C., Lee, S.-J., and Park, N. (2020). Machine Learning Based Hybrid System for Imputation and Efficient Energy Demand Forecasting. Energies, 13.10.3390/en13112681 ↗
37. Combining deep neural network and bibliometric indicator for emerging research topic prediction10.1016/j.ipm.2021.102611 ↗
38. Real-time rainfall-runoff prediction using light gradient boosting machine coupled with singular spectrum analysis10.1016/j.jhydrol.2021.127124 ↗
39. Random forest: An optimal chlorophyll-a algorithm for optically complex inland water suffering atmospheric correction uncertainties10.1016/j.jhydrol.2022.128685 ↗
40. Temporal stability in predictive process monitoring10.1007/s10618-018-0575-9 ↗
41. Time Series Classification of Cryptocurrency Price Trend Based on a Recurrent LSTM Neural Network
42. Geng, L., Che, T., Ma, M., Tan, J., and Wang, H. (2021). Corn Biomass Estimation by Integrating Remote Sensing and Long-Term Observation Data Based on Machine Learning Techniques. Remote Sens., 13.10.3390/rs13122352 ↗
43. Ngarambe, J., Irakoze, A., Yun, G.Y., and Kim, G. (2020). Comparative Performance of Machine Learning Algorithms in the Prediction of Indoor Daylight Illuminances. Sustainability, 12.10.3390/su12114471 ↗
44. Seydi, S.T., Amani, M., and Ghorbanian, A. (2022). A Dual Attention Convolutional Neural Network for Crop Classification Using Time-Series Sentinel-2 Imagery. Remote Sens., 14.10.3390/rs14030498 ↗
45. Oil well production prediction based on CNN-LSTM model with self-attention mechanism10.1016/j.energy.2023.128701 ↗
46. Interpretability of deep learning models for crop yield forecasting10.1016/j.compag.2023.107663 ↗
47. A novel approach based on integration of convolutional neural networks and echo state network for daily electricity demand prediction10.1016/j.energy.2023.127430 ↗
48. A hybrid decomposition and Machine learning model for forecasting Chlorophyll-a and total nitrogen concentration in coastal waters10.1016/j.jhydrol.2023.129207 ↗
49. Ensemble-based machine learning approach for improved leak detection in water mains10.2166/hydro.2021.093 ↗
50. Using improved gradient-boosted decision tree algorithm based on Kalman filter (GBDT-KF) in time series prediction10.1007/s11227-019-03130-y ↗
51. Global models for time series forecasting: A Simulation study10.1016/j.patcog.2021.108441 ↗
52. Dynamic Time Features Expanding and Extracting Method for Prediction Model of Sintering Process Quality Index10.1109/TII.2021.3086763 ↗
53. Deep Neural Network and Time Series Approach for Finance Systems: Predicting the Movement of the Indian Stock Market10.4018/JOEUC.20210901.oa10 ↗
54. Freeway Travel Time Prediction Using Deep Hybrid Model—Taking Sun Yat-Sen Freeway as an Example10.1109/TVT.2020.2999358 ↗
55. Shi, J., He, G., and Liu, X. (2018, January 22–24). Anomaly Detection for Key Performance Indicators Through Machine Learning. Proceedings of the 2018 International Conference on Network Infrastructure and Digital Content (IC-NIDC), Guiyang, China.10.1109/ICNIDC.2018.8525714 ↗
56. A hybrid deep learning framework with CNN and Bi-directional LSTM for store item demand forecasting10.1016/j.compeleceng.2022.108358 ↗
57. Empirical Analysis for Crime Prediction and Forecasting Using Machine Learning and Deep Learning Techniques10.1109/ACCESS.2021.3078117 ↗
58. Qiu, H., Luo, L., Su, Z., Zhou, L., Wang, L., and Chen, Y. (2020). Machine learning approaches to predict peak demand days of cardiovascular admissions considering environmental exposure. BMC Med. Inform. Decis. Mak., 20.10.1186/s12911-020-1101-8 ↗
59. Ribeiro, A.M.N.C., do Carmo, P.R.X., Endo, P.T., Rosati, P., and Lynn, T. (2022). Short- and Very Short-Term Firm-Level Load Forecasting for Warehouses: A Comparison of Machine Learning and Deep Learning Models. Energies, 15.10.20944/preprints202201.0107.v1 ↗
60. Demand forecasting at retail stage for selected vegetables: A performance analysis10.1108/JM2-11-2018-0192 ↗
61. Air Quality Prediction Based on Integrated Dual LSTM Model10.1109/ACCESS.2021.3093430 ↗
62. Ibañez, S.C., Dajac, C.V.G., Liponhay, M.P., Legara, E.F.T., Esteban, J.M.H., and Monterola, C.P. (2022). Forecasting Reservoir Water Levels Using Deep Neural Networks: A Case Study of Angat Dam in the Philippines. Water, 14.10.3390/w14010034 ↗
63. Stock-Price Forecasting Based on XGBoost and LSTM10.32604/csse.2022.017685 ↗
64. Application of ensemble RNN deep neural network to the fall detection through IoT environment10.1016/j.aej.2020.06.056 ↗
65. From driving behavior to energy consumption: A novel method to predict the energy consumption of electric bus10.1016/j.energy.2022.125188 ↗
66. Grey models for short-term queue length predictions for adaptive traffic signal control10.1016/j.eswa.2021.115618 ↗
67. Predicting long-term monthly electricity demand under future climatic and socioeconomic changes using data-driven methods: A case study of Hong Kong10.1016/j.scs.2021.102936 ↗
68. A comparative analysis of SLR, MLR, ANN, XGBoost and CNN for crop height estimation of sunflower using Sentinel-1 and Sentinel-210.1016/j.asr.2022.11.046 ↗
69. A Novel Forecasting Model for Solar Power Generation by a Deep Learning Framework with Data Preprocessing and Postprocessing10.1109/TIA.2022.3212999 ↗
70. Ultra-short-term wind power prediction method based on FTI-VACA-XGB model10.1016/j.eswa.2023.121185 ↗
71. Wang, X., Li, Y., Qiao, Q., Tavares, A., and Liang, Y. (2023). Water Quality Prediction Based on Machine Learning and Comprehensive Weighting Methods. Entropy, 25.10.3390/e25081186 ↗
72. Tunnel deformation prediction during construction: An explainable hybrid model considering temporal and static factors10.1016/j.compstruc.2024.107276 ↗
73. Optimizing long-short-term memory models via metaheuristics for decomposition aided wind energy generation forecasting10.1007/s10462-023-10678-y ↗
74. Time series prediction of sea surface temperature based on BiLSTM model with attention mechanism10.1016/j.seares.2024.102472 ↗
75. Alternate pathway for regional flood frequency analysis in data-sparse region10.1016/j.jhydrol.2024.130635 ↗
76. Dynamic real-time forecasting technique for reclaimed water volumes in urban river environmental management10.1016/j.envres.2024.118267 ↗
77. A state detection method of offshore wind turbines’ gearbox bearing based on the transformer and GRU10.1088/1361-6501/ad0956 ↗
78. HDTO-DeepAR: A novel hybrid approach to forecast surface water quality indicators10.1016/j.jenvman.2024.120091 ↗
79. Enhancing wastewater treatment through artificial intelligence: A comprehensive study on nutrient removal and effluent quality prediction10.1016/j.jwpe.2024.105212 ↗
80. Oyucu, S., Dümen, S., Duru, İ., Aksöz, A., and Biçer, E. (2024). Discharge Capacity Estimation for Li-Ion Batteries: A Comparative Study. Symmetry, 16.10.3390/sym16040436 ↗
81. Prediction of crop yield in India using machine learning and hybrid deep learning models10.1007/s11600-024-01312-8 ↗
82. Chen, T., and Guestrin, C. (2016, January 13–17). XGBoost: A Scalable Tree Boosting System. Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Francisco, CA, USA.10.1145/2939672.2939785 ↗
83. Ke, G., Meng, Q., Finley, T., Wang, T., Chen, W., Ma, W., Ye, Q., and Liu, T.-Y. (2017, January 4–9). LightGBM: A highly efficient gradient boosting decision tree. Proceedings of the 31st International Conference on Neural Information Processing Systems, Long Beach, CA, USA.
84. Prokhorenkova, L., Gusev, G., Vorobev, A., Dorogush, A.V., and Gulin, A. (2018, January 3–8). CatBoost: Unbiased boosting with categorical features. Proceedings of the 32nd International Conference on Neural Information Processing Systems, Montréal, QC, Canada.
85. Introduction to multi-layer feed-forward neural networks10.1016/S0169-7439(97)00061-0 ↗
86. Recent advances in convolutional neural networks10.1016/j.patcog.2017.10.013 ↗
87. Fundamentals of Recurrent Neural Network (RNN) and Long Short-Term Memory (LSTM) network10.1016/j.physd.2019.132306 ↗
88. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., and Polosukhin, I. (2017, January 4–9). Attention is all you need. Proceedings of the 31st International Conference on Neural Information Processing Systems, Long Beach, CA, USA.
89. An improved ICEEMDAN-MPA-GRU model for GNSS height time series prediction with weighted quality evaluation index10.1007/s10291-025-01867-z ↗
90. M5 accuracy competition: Results, findings, and conclusions10.1016/j.ijforecast.2021.11.013 ↗
91. The M5 uncertainty competition: Results, findings and conclusions10.1016/j.ijforecast.2021.10.009 ↗
92. Makridakis, S., Spiliotis, E., Hollyman, R., Petropoulos, F., Swanson, N., and Gaba, A. (2024). The M6 Forecasting Competition: Bridging the Gap Between Forecasting and Investment Decisions. Int. J. Forecast., in press.10.1016/j.ijforecast.2024.11.002 ↗
93. Woo, G., Liu, C., Kumar, A., Xiong, C., Savarese, S., and Sahoo, D. (2024, January 21–27). Unified training of universal time series forecasting transformers. Proceedings of the 41st International Conference on Machine Learning, Vienna, Austria.
94. Diffusion models for time-series applications: A survey10.1631/FITEE.2300310 ↗
95. Oh, Y., Kam, S., Lee, J., Lim, D.-Y., Kim, S., and Bui, A. (2025). Comprehensive Review of Neural Differential Equations for Time Series Analysis. arXiv.