Solar Irradiance Forecasting with Transformer Model
Source abstract
Abstract
Solar energy is one of the most popular sources of renewable energy today. It is therefore essential to be able to predict solar power generation and adapt energy needs to these predictions. This paper uses the Transformer deep neural network model, in which the attention mechanism is typically applied in NLP or vision problems. Here, it is extended by combining features based on their spatiotemporal properties in solar irradiance prediction. The results were predicted for arbitrary long-time horizons since the prediction is always 1 day ahead, which can be included at the end along the timestep axis of the input data and the first timestep representing the oldest timestep removed. A maximum worst-case mean absolute percentage error of 3.45% for the one-day-ahead prediction was obtained, which gave better results than the directly competing methods.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Solar Radiation and Photovoltaics · Energy Load and Power Forecasting · Photovoltaic System Optimization Techniques
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Department of Applied Informatics, Faculty of Natural Sciences, University of Ss. Cyril and Methodius, J. Herdu 2, 917 01 Trnava, Slovakia
Funding
the Cultural and Educational Grant Agency MŠVVaŠ SR · the Slovak Research and Development Agency
References (35)
Show source-provided references
1. Ariga, K., and Zheng, K. (2022, August 24). Forecasting Solar Irradiance from Time Series with Exogenous Information. Available online: https://kaiyuzheng.me/projects/projects/ForecastingSolar546.pdf.
2. (2022, August 01). NASA Power Project, Available online: https://power.larc.nasa.gov.
3. Zelikman, E., Zhou, S., Irvin, J., Raterink, C., Sheng, H., Avati, A., Kelly, J., Rajagopal, R., Ng, A.Y., and Gagne, D. (2020). Short–term solar irradiance forecasting using calibrated probabilistic models. arXiv.
4. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., and Polosukhin, I. (2017). Attention is all you need. Adv. Neural Inf. Process. Syst., 30, Available online: https://proceedings.neurips.cc/paper/2017/file/3f5ee243547dee91fbd053c1c4a845aa-Paper.pdf.
5. (2022, August 01). TensorFlow Framework. Available online: https://www.tensorflow.org.
6. Brahma, B., and Wadhvani, R. (2020). Solar irradiance forecasting based on deep learning methodologies and multi–site data. Symmetry, 12.10.3390/sym12111830 ↗
7. Solar irradiance forecasting using deep neural networks10.1016/j.procs.2017.09.045 ↗
8. Alharbi, F.R., and Csala, D. (2021). Wind Speed and Solar Irradiance Prediction Using a Bidirectional Long Short-Term Memory Model Based on Neural Networks. Energies, 14, Available online: https://www.mdpi.com/1996-1073/14/20/6501.10.3390/en14206501 ↗
9. Prediction of solar radiation for solar systems by using ANN models with different back propagation algorithms10.1016/j.jart.2016.05.001 ↗
10. Zafar, R., Vu, B.H., Husein, M., and Chung, I.Y. (2021). Day–Ahead Solar Irradiance Forecasting Using Hybrid Recurrent Neural Network with Weather Classification for Power System Scheduling. Appl. Sci., 11, Available online: https://www.mdpi.com/2076-3417/11/15/6738.10.3390/app11156738 ↗
11. Wang, F., Yu, Y., Zhang, Z., Li, J., Zhen, Z., and Li, K. (2018). Wavelet decomposition and convolutional LSTM networks based improved deep learning model for solar irradiance forecasting. Appl. Sci., 8, Available online: https://www.mdpi.com/2076-3417/8/8/1286.10.3390/app8081286 ↗
12. Short-term solar irradiance forecasting model based on artificial neural network using statistical feature parameters10.3390/en5051355 ↗
13. Husein, M., and Chung, I.Y. (2019). Day–ahead solar irradiance forecasting for microgrids using a long short–term memory recurrent neural network: A deep learning approach. Energies, 12, Available online: https://www.mdpi.com/1996-1073/12/10/1856.10.3390/en12101856 ↗
14. Mendonça de Paiva, G., Pires Pimentel, S., Pinheiro Alvarenga, B., Gonçalves Marra, E., Mussetta, M., and Leva, S. (2020). Multiple site intraday solar irradiance forecasting by machine learning algorithms: MGGP and MLP neural networks. Energies, 13, Available online: https://www.mdpi.com/1996-1073/13/11/3005.10.3390/en13113005 ↗
15. Park, S., Kim, Y., Ferrier, N.J., Collis, S.M., Sankaran, R., and Beckman, P.H. (2021). Prediction of solar irradiance and photovoltaic solar energy product based on cloud coverage estimation using machine learning methods. Atmosphere, 12, Available online: https://www.mdpi.com/2073-4433/12/3/395/htm.10.3390/atmos12030395 ↗
16. Yang, L., Gao, X., Hua, J., Wu, P., Li, Z., and Jia, D. (2020). Very short-term surface solar irradiance forecasting based on FengYun–4 geostationary satellite. Sensors, 20, Available online: https://www.mdpi.com/1424-8220/20/9/2606/htm.10.3390/s20092606 ↗
17. (2022, August 18). Solar Power by Country. Available online: https://worldpopulationreview.com/country-rankings/solar-power-by-country.
18. Srivastava, R., Tiwari, A.N., and Giri, V.K. (2018, January 2–4). Forecasting of solar radiation in India using various ANN models. Proceedings of the 2018 5th IEEE Uttar Pradesh Section International Conference on Electrical, Electronics and Computer Engineering (UPCON), Gorakhpur, India. Available online: https://www.researchgate.net/publication/331421649_Forecasting_of_Solar_Radiation_in_India_Using_Various_ANN_Models.10.1109/UPCON.2018.8597170 ↗
19. Short-term PV power forecasting in India: Recent developments and policy analysis10.1007/s40095-021-00468-z ↗
20. Mukherjee, A., Ain, A., and Dasgupta, P. (2018, January 12–15). Solar irradiance prediction from historical trends using deep neural networks. Proceedings of the 2018 IEEE International Conference on Smart Energy Grid Engineering (SEGE), Oshawa, ON, Canada. Available online: https://ieeexplore.ieee.org/document/8499394.10.1109/SEGE.2018.8499394 ↗
21. Rezatofighi, H., Tsoi, N., Gwak, J., Sadeghian, A., Reid, I., and Savarese, S. (2019, January 15–20). Generalized intersection over union: A metric and a loss for bounding box regression. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA. Available online: https://www.computer.org/csdl/proceedings-article/cvpr/2019/329300a658/1gyrWyRZ0k0.10.1109/CVPR.2019.00075 ↗
22. Chollet, F. (2021). Deep Learning with Python, Simon and Schuster.
23. Keisler, R. (2022). Forecasting Global Weather with Graph Neural Networks. arXiv.
24. (2022, August 01). Meteorological Wind Direction. Available online: http://tornado.sfsu.edu/geosciences/classes/m430/Wind/WindDirection.html.
25. Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., and Gelly, S. (2020). An image is worth 16x16 words: Transformers for image recognition at scale. arXiv.
26. Bao, H., Dong, L., and Wei, F. (2021). Beit: Bert pre–training of image transformers. arXiv.
27. Loshchilov, I., and Hutter, F. (2016). SGDR: Stochastic Gradient Descent with Warm Restarts. arXiv.
28. Chen, P., Chen, G., and Zhang, S. (2022, August 01). Log Hyperbolic Cosine Loss Improves Variational Auto-Encoder. Available online: https://openreview.net/forum?id=rkglvsC9Ym.
29. Artificial neural network estimation of global solar radiation using air temperature and relative humidity10.1016/j.enpol.2007.09.033 ↗
30. Forecasting based on neural network approach of solar potential in Turkey10.1016/j.renene.2004.09.020 ↗
31. Use of artificial neural networks for mapping of solar potential in Turkey10.1016/S0306-2619(03)00137-5 ↗
32. Day-ahead photovoltaic power production forecasting methodology based on machine learning and statistical post-processing10.1016/j.apenergy.2020.115023 ↗
33. Dropout: A simple way to prevent neural networks from overfitting
34. He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27–30). Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA. Available online: https://arxiv.org/abs/1512.03385?context=cs.10.1109/CVPR.2016.90 ↗
35. Loshchilov, I., and Hutter, F. (2017). Decoupled weight decay regularization. arXiv.