Short-Term Energy Consumption Forecasting Analysis Using Different Optimization and Activation Functions with Deep Learning Models
Source abstract
Abstract
Modelling events that change over time is one of the most difficult problems in data analysis. Forecasting of time-varying electric power values is also an important problem in data analysis. Regression methods, machine learning, and deep learning methods are used to learn different patterns from data and develop a consumption prediction model. The aim of this study is to determine the most successful models for short-term power consumption prediction with deep learning and to achieve the highest prediction accuracy. In this study, firstly, the data was evaluated and organized with exploratory data analysis (EDA) on a ready dataset and the features of the data were extracted. Studies were carried out on long short-term memory (LSTM), gated recurrent unit (GRU), simple recurrent neural networks (SimpleRNN) and bidirectional long short-term memory (BiLSTM) architectures. First, four architectures were used with 11 different optimization methods. In this study, it was seen that a high success rate of 0.9972 was achieved according to the R2 score index. In the following, the first study was tried with different epoch numbers. Afterwards, this study was carried out with 264 separate models produced using four architectures, 11 optimization methods, and six activation functions in order. The results of all these studies were obtained according to the root mean square error (RMSE), mean absolute error (MAE), and R2_score indexes. The R2_score indexes graphs are presented. Finally, the 10 most successful applications are listed.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Energy Load and Power Forecasting · Stock Market Forecasting Methods · Solar Radiation and Photovoltaics
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Ergani Vocational School, Dicle University, 21280 Diyarbakır, Türkiye
Department of Electrical and Electronics Engineering, Engineering Faculty, Inonu University, 44280 Malatya, Türkiye
References (56)
Show source-provided references
1. Ali, M., Khan, D.M., Alshanbari, H.M., and El-Bagoury, A.A.-A.H. (2023). Prediction of Complex Stock Market Data Using an Improved Hybrid EMD-LSTM Model. Appl. Sci., 13.10.3390/app13031429 ↗
2. Mndawe, S.T., Paul, B.S., and Doorsamy, W. (2022). Development of a Stock Price Prediction Framework for Intelligent Media and Technical Analysis. Appl. Sci., 12.10.3390/app12020719 ↗
3. Jarrah, M., and Derbali, M. (2023). Predicting Saudi Stock Market Index by Using Multivariate Time Series Based on Deep Learning. Appl. Sci., 13.10.20944/preprints202306.1537.v1 ↗
4. Huang, D., Zhang, Q., Wen, Z., Hu, M., and Xu, W. (2023). Research on a Time Series Data Prediction Model Based on Causal Feature Weight Adjustment. Appl. Sci., 13.10.3390/app131910782 ↗
5. Cheng, C.-H., and Chen, Y.-S. (2007, January 19–22). Fundamental Analysis of Stock Trading Systems using Classification Techniques. Proceedings of the 2007 International Conference on Machine Learning and Cybernetics, Hong Kong, China.10.1109/ICMLC.2007.4370359 ↗
6. Vargas, M.R., de Lima, B.S.L.P., and Evsukoff, A.G. (2017, January 26–28). Deep Learning for Stock Market Prediction from Financial News Articles. Proceedings of the 2017 IEEE International Conference on Computational Intelligence and Virtual Environments for Measurement Systems and Applications (CIVEMSA), Annecy, France.10.1109/CIVEMSA.2017.7995302 ↗
7. Li, C., Zhao, M., Liu, Y., and Xu, F. (2020, January 18–20). Air Temperature Forecasting using Traditional and Deep Learning Algorithms. Proceedings of the 7th International Conference on Information Science and Control Engineering (ICISCE), Changsha, China.10.1109/ICISCE50968.2020.00049 ↗
8. Escalona-Llaguno, M.I., Solís-Sánchez, L.O., Castañeda-Miranda, C.L., Olvera-Olvera, C.A., Martinez-Blanco, M.d.R., Guerrero-Osuna, H.A., Castañeda-Miranda, R., Díaz-Flórez, G., and Ornelas-Vargas, G. (2024). Comparative Analysis of Solar Radiation Forecasting Techniques in Zacatecas, Mexico. Appl. Sci., 14.10.3390/app14177449 ↗
9. Probabilistic Solar Generation Forecasting for Rapidly Changing Weather Conditions10.1109/ACCESS.2024.3407778 ↗
10. Wu, D., Jia, Z., Zhang, Y., and Wang, J. (2023). Predicting Temperature and Humidity in Roadway with Water Trickling Using Principal Component Analysis-Long Short-Term Memory-Genetic Algorithm Method. Appl. Sci., 13.10.3390/app132413343 ↗
11. Rosca, C.-M., Carbureanu, M., and Stancu, A. (2025). Data-Driven Approaches for Predicting and Forecasting Air Quality in Urban Areas. Appl. Sci., 15.10.3390/app15084390 ↗
12. Remaining Useful Life Predictor for EV Batteries Using Machine Learning10.1109/ACCESS.2024.3461802 ↗
13. Wang, H., Wang, H., Jiang, G., Li, J., and Wang, Y. (2019). Early fault detection of wind turbines based on operational condition clustering and optimized deep belief network modeling. Energies, 12.10.3390/en12060984 ↗
14. Zou, Y., Sun, W., Wang, H., Xu, T., and Wang, B. (2025). Research on Bearing Remaining Useful Life Prediction Method Based on Double Bidirectional Long Short-Term Memory. Appl. Sci., 15.10.3390/app15084441 ↗
15. Wu, M., Yue, C., Zhang, F., Sun, R., Tang, J., Hu, S., Zhao, N., and Wang, J. (2024). State of Health Estimation and Remaining Useful Life Prediction of Lithium-Ion Batteries by Charging Feature Extraction and Ridge Regression. Appl. Sci., 14.10.3390/app14083153 ↗
16. Alsmadi, L., Lei, G., and Li, L. (2025). Forecasting Day-Ahead Electricity Demand in Australia Using a CNN-LSTM Model with an Attention Mechanism. Appl. Sci., 15.10.3390/app15073829 ↗
17. Mukhtar, M., Oluwasanmi, A., Yimen, N., Qinxiu, Z., Ukwuoma, C.C., Ezurike, B., and Bamisile, O. (2022). Development and Comparison of Two Novel Hybrid Neural Network Models for Hourly Solar Radiation Prediction. Appl. Sci., 12.10.3390/app12031435 ↗
18. Ayaz Atalan, Y., and Atalan, A. (2025). Testing the Wind Energy Data Based on Environmental Factors Predicted by Machine Learning with Analysis of Variance. Appl. Sci., 15.10.3390/app15010241 ↗
19. (2025, May 29). Energy Forecasting. A Blog by Dr. Tao Hong. Available online: http://blog.drhongtao.com/2014/10/very-short-short-medium-long-term-load-forecasting.html.
20. Electrical load forecasting models: A critical systematic review10.1016/j.scs.2017.08.009 ↗
21. Day-ahead electricity price forecasting using wavelet transform combined with ARIMA and GARCH models10.1016/j.apenergy.2010.05.012 ↗
22. Esener, I.I., Yüksel, T., and Kurban, M. (2012, January 12–15). Artificial Intelligence Based Hybrid Structures for Short-Term Load Forecasting Without Temperature Data. Proceedings of the 11th International Conference on Machine Learning and Applications, Boca Raton, FL, USA.10.1109/ICMLA.2012.169 ↗
23. Zhang, X.M., Grolinger, K., and Capretz, M.A.M. (2018, January 17–20). Forecasting Residential Energy Consumption Using Support Vector Regressions. Proceedings of the IEEE International Conference on Machine Learning and Applications, Orlando, FL, USA.10.1109/ICMLA.2018.00024 ↗
24. Yang, M., Li, W., Zhang, H., and Wang, H. (2016, January 27–28). Parameters Optimization Improvement of SVM on Load Forecasting. Proceedings of the 8th International Conference on Intelligent Human-Machine Systems and Cybernetics (IHMSC), Hangzhou, China.10.1109/IHMSC.2016.269 ↗
25. Evolutionary double attention-based long short-term memory model for building energy prediction: Case study of a green building10.1016/j.apenergy.2021.116660 ↗
26. Deep learning-based feature engineering methods for improved building energy prediction10.1016/j.apenergy.2019.02.052 ↗
27. Zhu, T., Ran, Y., Zhou, X., and Wen, Y. (2024). A Survey of Predictive Maintenance: Systems, Purposes and Approaches. arXiv.
28. Assessment of deep recurrent neural network-based strategies for short-term building energy predictions10.1016/j.apenergy.2018.12.004 ↗
29. Short-Term Residential Load Forecasting Based on LSTM Recurrent Neural Network10.1109/TSG.2017.2753802 ↗
30. Short-Term Load Forecasts Using LSTM Networks10.1016/j.egypro.2019.01.952 ↗
31. Wang, C., Yan, Z., Li, Q., Zhu, Z., and Zhang, C. (2024). Energy Consumption Prediction for Drilling Pumps Based on a Long Short-Term Memory Attention Method. Appl. Sci., 14.10.3390/app142210750 ↗
32. Siami-Namini, S., Tavakoli, N., and Namin, A.S. (2018, January 17–20). A Comparison of ARIMA and LSTM in Forecasting Time Series. Proceedings of the 17th IEEE International Conference on Machine Learning and Applications (ICMLA), Orlando, FL, USA.10.1109/ICMLA.2018.00227 ↗
33. IDahlan, A., Ariateja, D., Hamami, F. (2021, January 28–30). The Implementation of Building Intelligent Smart Energy using LSTM Neural Network. Proceedings of the 2021 International Conference on Artificial Intelligence and Mechatronics Systems (AIMS), Bandung, Indonesia.10.1109/AIMS52415.2021.9466046 ↗
34. Short-term load forecasting based on LSTM networks considering attention mechanism10.1016/j.ijepes.2021.107818 ↗
35. LSTM-based long-term energy consumption prediction with periodicity10.1016/j.energy.2020.117197 ↗
36. Ungureanu, S., Topa, V., and Cziker, A.C. (2021). Deep Learning for Short-Term Load Forecasting—Industrial Consumer Case Study. Appl. Sci., 11.10.3390/app112110126 ↗
37. Almalki, A.J., and Wocjan, P. (2020, January 16–18). Forecasting Method based upon GRU-based Deep Learning Model. Proceedings of the 2020 International Conference on Computational Science and Computational Intelligence (CSCI), Las Vegas, NV, USA.10.1109/CSCI51800.2020.00096 ↗
38. Insights Into LSTM Fully Convolutional Networks for Time Series Classification10.1109/ACCESS.2019.2916828 ↗
39. Yang, S., Yu, X., and Zhou, Y. (2020, January 1–4). LSTM and GRU Neural Network Performance Comparison Study: Taking Yelp Review Dataset as an Example. Proceedings of the 2020 International Workshop on Electronic Communication and Artificial Intelligence (IWECAI), Shanghai, China.10.1109/IWECAI50956.2020.00027 ↗
40. Siami-Namini, S., Tavakoli, N., and Namin, A.S. (2019, January 9–12). The Performance of LSTM and BiLSTM in Forecasting Time Series. Proceedings of the IEEE International Conference on Big Data (Big Data), Los Angeles, CA, USA.10.1109/BigData47090.2019.9005997 ↗
41. Xu, J., and Zeng, P. (2024, January 19–21). Short-term Load Forecasting by BiLSTM Model Based on Multidimensional Time-domain Feature. Proceedings of the 4th International Conference on Neural Networks, Information and Communication Engineering (NNICE), Guangzhou, China.10.1109/NNICE61279.2024.10498827 ↗
42. Predictions for COVID-19 with deep learning models of LSTM, GRU, and Bi-LSTM10.1016/j.chaos.2020.110212 ↗
43. Price Forecasting in Stock Exchange with Deep Learning Methods
44. Irfan, M., Shaf, A., Ali, T., Zafar, M., Rahman, S., Mursal, S.N.F., AlThobiani, F., Almas, M.A., Attar, H.M., and Abdussamiee, N. (2023). Multi-region electricity demand prediction with ensemble deep neural networks. PLoS ONE, 18.10.1371/journal.pone.0285456 ↗
45. IoT-Based Energy Consumption Prediction Using Transformers10.54287/gujsa.1438011 ↗
46. Efficient Short-Term Electricity Load Forecasting for Effective Energy Management
47. (2025, May 29). Energy Consumption Dataset, Kaggle. Available online: https://www.kaggle.com/datasets/raminhuseyn/energy-consumption-dataset/data.
48. (2025, May 29). Time Series Forecasting: Exploratory Data Analysis, Kaggle. Available online: https://www.kaggle.com/code/raminhuseyn/time-series-forecasting-exploratory-data-analysis.
49. (2025, May 29). Time Series Forecasting: A Practical Guide to Exploratory Data Analysis, Towards Data Science. Available online: https://towardsdatascience.com/time-series-forecasting-a-practical-guide-to-exploratory-data-analysis-a101dc5f85b1/.
50. Deep learning neural networks for short-term photovoltaic power forecasting10.1016/j.renene.2021.02.166 ↗
51. Comparative study of long short-term memory (LSTM), bidirectional LSTM, and traditional machine learning approaches for energy consumption prediction10.1177/01445987241269496 ↗
52. A deep LSTM network for the Spanish electricity consumption forecasting10.1007/s00521-021-06773-2 ↗
53. Alsharekh, M.F., Habib, S., Dewi, D.A., Albattah, W., Islam, M., and Albahli, S. (2022). Improving the Efficiency of Multistep Short-Term Electricity Load Forecasting via R-CNN with ML-LSTM. Sensors, 22.10.3390/s22186913 ↗
54. Hourly energy consumption forecasting by LSTM and ARIMA methods
55. Machine Learning, Deep Learning and Statistical Analysis for forecasting building energy consumption—A systematic review10.1016/j.engappai.2022.105287 ↗
56. Performance Evaluation of Activation Functions in Deep Residual Networks for Short-Term Load Forecasting10.1109/ACCESS.2025.3565798 ↗