Satellite-Based High-Precision Clear-Sky Irradiance Estimation Using Machine Learning and Physical Model Harmonization
Source abstract
Abstract
Accurate short-term estimation of clear-sky Global Horizontal Irradiance (GHI) is vital for solar resource assessment and grid operations, yet existing methods rely on sparse radiometers and coarse global weather reanalysis (e.g., MERRA-2 at 50–70 km spatial resolution with 1 month latency). To achieve scalability in high-precision estimation, we propose a framework that removes dependence on ground measurements by combining multi-satellite observations with reanalysis variables in a physics-supervised machine-learning paradigm. We developed a multi-source-fused high-resolution environmental dataset with 5 min granularity and 1 km spatial precision, incorporating Geostationary Operational Environmental Satellite (GOES-16) observations, polar-orbiting satellite (AURA) data, and MERRA-2 reanalysis. As supervisory physics, we harmonized two complementary parameterized radiative transfer models (MAC2 and REST2V5). The harmonized GHI estimates are used as training labels for a Multilayer Perceptron (MLP) and a Residual Long Short-Term Memory (LSTM) network model. The trained MLP model achieved a root mean square error (RMSE) of 66.67 W/m2, representing a 7.50% reduction over the conventional MERRA-2-driven baseline. For 30-min-ahead forecasting, the LSTM model reduced RMSE by 29.37% over the persistence baseline. Evaluated at four climatically diverse U.S. sites, the system achieves ground-sensor-like accuracy and is deployable anywhere within GOES-16 coverage.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Solar Radiation and Photovoltaics · Atmospheric aerosols and clouds · Solar and Space Plasma Dynamics
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Graduate School of Science and Engineering, Saitama University, 255-Shimo-Okubo, Sakura Ward, Saitama 338-8570, Saitama, Japan
Space Data Frontiers Research Center, Fujitsu Research, Fujitsu Limited, 4-1-1 Kamikodanaka, Nakahara Ward, Kawasaki 211-8588, Kanagawa, Japan
Funding
Ministry of Education, Culture, Sports, Science and Technology
References (89)
Show source-provided references
1. Stein, J., Hansen, C., and Reno, M. (2012). Global Horizontal Irradiance Clear Sky Models: Implementation and Analysis, Sandia National Laboratories. Technical Report SAND2012-2389.
2. Diagne, H.M., Lauret, P., and David, M. (2025, August 18). Solar Irradiation Forecasting: State-of-the-Art and Proposition for Future Developments for Small-Scale Insular Grids. Available online: https://hal.science/hal-00918150v1.
3. On the importance of clearsky model in short-term solar radiation forecasting10.1016/j.solener.2025.113490 ↗
4. Solar forecasts based on the clear sky index or the clearness index: Which is better?10.3390/solar2040026 ↗
5. Mendyl, A., Mabasa, B., Bouzghiba, H., and Weidinger, T. (2022). Calibration and validation of global horizontal irradiance clear sky models against McClear clear sky model in Morocco. Appl. Sci., 13.10.3390/app13010320 ↗
6. Solar Irradiance Variability and Climate10.1146/annurev-astro-082812-141007 ↗
7. Short-term forecasting of solar irradiance without local telemetry: A generalized model using satellite data10.1016/j.solener.2018.07.050 ↗
8. Pandžić, F., and Capuder, T. (2023). Advances in short-term solar forecasting: A review and benchmark of machine learning methods and relevant data sources. Energies, 17.10.3390/en17010097 ↗
9. Solar irradiance variability around Asia Pacific: Spatial and temporal perspective for active use of solar energy10.1016/j.solener.2024.112678 ↗
10. Long-term spatial and temporal solar resource variability over America using the NSRDB version 3 (1998–2017)10.1016/j.rser.2020.110285 ↗
11. Jaroslav, H. (2012). Topographic solar radiation modeling for environmental applications. Encyclopedia of Sustainability Science and Technology, Springer.10.1007/978-1-4419-0851-3_452 ↗
12. Short term fluctuations of wind and solar power systems10.1088/1367-2630/18/6/063027 ↗
13. A review of key power system stability challenges for large-scale PV integration10.1016/j.rser.2014.09.027 ↗
14. A review on data-driven methods for solar energy forecasting10.1016/j.apenergy.2025.126631 ↗
15. SURFRAD—A National Surface Radiation Budget Network for Atmospheric Research10.1175/1520-0477(2000)081<2341:SANSRB>2.3.CO;2 ↗
16. The Modern-Era Retrospective Analysis for Research and Applications, Version 2 (MERRA-2)10.1175/JCLI-D-16-0758.1 ↗
17. The CAMS reanalysis of atmospheric composition10.5194/acp-19-3515-2019 ↗
18. ECMWF (2025, April 23). CAMS: Reanalysis Data Documentation—CopernicusKnowledge Base—ECMWF Confluence Wiki. Available online: https://confluence.ecmwf.int/display/CKB/CAMS3A+Reanalysis+data+documentation.
19. Influences of atmospheric reanalysis on the accuracy of clear-sky irradiance estimates: Comparing MERRA-2 and CAMS10.1016/j.atmosenv.2022.119080 ↗
20. NASA Global Modeling and Assimilation Office (2025, April 23). MERRA-2 Data Access, Available online: https://gmao.gsfc.nasa.gov/reanalysis/merra-2/data_access.
21. Copernicus Atmosphere Monitoring Service (2025, April 23). CAMS Global Reanalysis (EAC4). Available online: https://ads.atmosphere.copernicus.eu/datasets/cams-global-reanalysis-eac4.
22. NASA Global Modeling and Assimilation Office (2025, August 05). MERRA-2 AMIP, Available online: https://gmao.gsfc.nasa.gov/gmao-products/merra-2/merra-2-amip_merra-2/.
23. Worldwide performance assessment of 75 global clear-sky irradiance models using Principal Component Analysis10.1016/j.rser.2019.04.006 ↗
24. Solar radiation statistics for Iran10.1016/0038-092X(78)90013-0 ↗
25. Monthly mean solar radiation statistics for Australia10.1016/0038-092X(76)90022-0 ↗
26. Meinel, A.B., and Meinel, M.P. (1975). Applied Solar Energy. An Introduction, Addison-Wesley Publishing Co.
27. The measurement of solar spectral irradiance at different terrestrial elevations10.1016/0038-092X(70)90006-X ↗
28. Direct solar transmittance and irradiance predictions with broadband models. Part I: Detailed theoretical performance assessment10.1016/S0038-092X(03)00195-6 ↗
29. A Fast All-sky Radiation Model for Solar applications (FARMS): Algorithm and performance evaluation10.1016/j.solener.2016.06.003 ↗
30. A new airmass independent formulation for the Linke turbidity coefficient10.1016/S0038-092X(02)00045-2 ↗
31. REST2: High-performance solar radiation model for cloudless-sky irradiance, illuminance, and photosynthetically active radiation–Validation with a benchmark dataset10.1016/j.solener.2007.04.008 ↗
32. Estimating solar irradiance and components10.1016/0038-092X(82)90280-8 ↗
33. Palani, K., Kota, R., Azad, A.P., and Arya, V. (2025, August 20). Blue Skies: A Methodology for Data-Driven Clear Sky Modelling. Available online: https://www.ijcai.org/proceedings/2017/528.10.24963/ijcai.2017/528 ↗
34. Generative adversarial networks and convolutional neural networks based weather classification model for day ahead short-term photovoltaic power forecasting10.1016/j.enconman.2018.11.074 ↗
35. Forecasting hourly global horizontal solar irradiance in South Africa using machine learning models10.1109/ACCESS.2020.3034690 ↗
36. Hybrid auto-regressive neural network model for estimating global solar radiation in Bandar Abbas, Iran10.1007/s12665-015-4970-x ↗
37. Solar radiation prediction using recurrent neural network and artificial neural network: A case study with comparisons10.1016/j.renene.2020.04.042 ↗
38. Jia, D., Hua, J., Wang, L., Guo, Y., Guo, H., Wu, P., Liu, M., and Yang, L. (2021). Estimations of global horizontal irradiance and direct normal irradiance by using Fengyun-4A satellite data in northern China. Remote Sens., 13.10.3390/rs13040790 ↗
39. Deep learning for global horizontal irradiation estimation from MSG SEVIRI data10.1080/01431161.2025.2474169 ↗
40. Sky image-based solar forecasting using deep learning with heterogeneous multi-location data: Dataset fusion versus transfer learning10.1016/j.apenergy.2024.123467 ↗
41. Radovan, A., Šunde, V., Kučak, D., and Ban, Ž. (2021). Solar irradiance forecast based on cloud movement prediction. Energies, 14.10.3390/en14133775 ↗
42. Improved satellite-based intra-day solar forecasting with a chain of deep learning models10.1016/j.enconman.2024.118598 ↗
43. The method Heliosat-2 for deriving shortwave solar radiation from satellite images10.1016/j.solener.2004.04.017 ↗
44. The National Solar Radiation Data Base (NSRDB)10.1016/j.rser.2018.03.003 ↗
45. Solargis (2026, February 09). Solargis Satellite-Based Model. Available online: https://kb.solargis.com/docs/satellite-based-irradiance.
46. Site-adaptation of satellite-based DNI and GHI time series: Overview and SolarGIS approach10.1063/1.4949234 ↗
47. Joint Research Centre, European Commission (2026, February 09). PVGIS Data Sources & Calculation Methods. Available online: https://joint-research-centre.ec.europa.eu/photovoltaic-geographical-information-system-pvgis/getting-started-pvgis/pvgis-data-sources-calculation-methods_en.
48. A new database of global and direct solar radiation using the eastern meteosat satellite, models and validation10.3390/rs6098165 ↗
49. Kim, C.K., Kim, H.G., and Kang, Y.H. (2022). Improved clear sky model from in situ observations and spatial distribution of aerosol optical depth for satellite-derived solar irradiance over the Korean peninsula. Remote Sens., 14.10.3390/rs14092167 ↗
50. Sultana, N., and Tsutsumida, N. (2024, January 7–12). Combining Machine Learning with Physics-Based Mathematical Models for Near-Real-Time Clear-Sky Irradiance Prediction from Geostationary Satellite Imagery. Proceedings of the IGARSS 2024—2024 IEEE International Geoscience and Remote Sensing Symposium, Athens, Greece.10.1109/IGARSS53475.2024.10642113 ↗
51. (2025, April 23). OMI/Aura TOMS-Like Ozone and Radiative Cloud Fraction L3 1 Day 0.25 Degree x 0.25 Degree V004 (OMTO3e) at GES DISC—Dataset—NASA Open Data Portal, Available online: https://data.nasa.gov/dataset/omi-aura-toms-like-ozone-and-radiative-cloud-fraction-l3-1-day-0-25-degree-x-0-25-degree-v-f4a14.
52. (2025, April 22). OMI/Aura NO2 Cloud-Screened Total and Tropospheric Column L3 Global Gridded 0.25 Degree x 0.25 Degree V3 (OMNO2d) at GES DISC, Available online: https://cmr.earthdata.nasa.gov/search/concepts/C1266136111-GES_DISC.html.
53. National Centers for Environmental Information (NCEI) (2025, April 23). NOAA GOES-R Series Advanced Baseline Imager (ABI) Level 2 Aerosol Optical Depth (AOD), Available online: https://www.ncei.noaa.gov/access/metadata/landing-page/bin/iso?id=gov.noaa.ncdc:C01511.
54. National Centers for Environmental Information (NCEI) (2025, April 23). NOAA GOES-R Series Advanced Baseline Imager (ABI) Level 2 Total Precipitable Water (TPW), Available online: https://www.ncei.noaa.gov/access/metadata/landing-page/bin/iso?id=gov.noaa.ncdc:C01515.
55. National Centers for Environmental Information (NCEI) (2025, April 23). NOAA GOES-R Series Advanced Baseline Imager (ABI) Level 2 Land Surface Albedo (LSA), Available online: https://www.ncei.noaa.gov/access/metadata/landing-page/bin/iso?id=gov.noaa.ncdc:C01535.
56. Effects of increasing spatial resolution on the spatial information content and accuracy of downward surface shortwave radiation
57. A GeoNEX-based high-spatiotemporal-resolution product of land surface downward shortwave radiation and photosynthetically active radiation10.5194/essd-15-1419-2023 ↗
58. Global hourly, 5 km, all-sky land surface temperature data from 2011 to 2021 based on integrating geostationary and polar-orbiting satellite data10.5194/essd-15-869-2023 ↗
59. Generating Global LAnd Surface Satellite incident shortwave radiation and photosynthetically active radiation products from multiple satellite data10.1016/j.rse.2014.07.003 ↗
60. Team, G.W. (2025, April 23). Global Radiation and Aerosols (Colorado)—NOAA Global Monitoring Laboratory, Available online: https://gml.noaa.gov/grad/surfrad/tablemt.html.
61. Team, G.W. (2025, April 23). Global Radiation and Aerosols (Illinois)—NOAA Global Monitoring Laboratory, Available online: https://gml.noaa.gov/grad/surfrad/bondvill.html.
62. Team, G.W. (2025, April 23). Global Radiation and Aerosols (Pennsylvania)—NOAA Global Monitoring Laboratory, Available online: https://gml.noaa.gov/grad/surfrad/pennstat.html.
63. The ARM Southern Great Plains (SGP) Site10.1175/AMSMONOGRAPHS-D-16-0004.1 ↗
64. (2025, April 19). ABI GOES-R Series, Available online: https://www.goes-r.gov/spacesegment/abi.html.
65. (2025, April 09). MERRA-2 tavg1_2d_slv_Nx: 2d, 1-Hourly, Time-Averaged, Single-Level, Assimilation, Single-Level Diagnostics 0.625 x 0.5 Degree V5.12.4 (M2T1NXSLV) at GES DISC—Dataset—NASA Open Data Portal, Available online: https://data.nasa.gov/dataset/merra-2-tavg1-2d-slv-nx-2d1-hourlytime-averagedsingle-levelassimilationsingle-level-diagno-f23a5.
66. (2025, April 01). MERRA-2 tavg1_2d_aer_Nx: 2d, 1-Hourly, Time-Averaged, Single-Level, Assimilation, Aerosol Diagnostics 0.625 x 0.5 Degree V5.12.4 (M2T1NXAER) at GES DISC—Dataset—NASA Open Data Portal, Available online: https://data.nasa.gov/dataset/merra-2-tavg1-2d-aer-nx-2d1-hourlytime-averagedsingle-levelassimilationaerosol-diagnostics-b1b07.
67. (2025, April 02). MERRA-2 tavg1_2d_rad_Nx: 2d, 1-Hourly, Time-Averaged, Single-Level, Assimilation, Radiation Diagnostics 0.625 x 0.5 Degree V5.12.4 (M2T1NXRAD) at GES DISC, Available online: https://cmr.earthdata.nasa.gov/search/concepts/C1276812851-GES_DISC.html.
68. Temporal and spatial variability of aerosol optical depth over South India as inferred from MODIS10.1016/j.jastp.2012.12.010 ↗
69. irradpy: Python package for MERRA-2 download, extraction and usage for clear-sky irradiance modelling10.1016/j.solener.2020.02.061 ↗
70. High-resolution (1 km) Köppen–Geiger maps for 1901–2099 based on constrained CMIP6 projections10.1038/s41597-023-02549-6 ↗
71. Advances in Understanding Clouds from ISCCP10.1175/1520-0477(1999)080<2261:AIUCFI>2.0.CO;2 ↗
72. Trends in ISCCP, MISR, and MODIS cloud-top-height and optical-depth histograms10.1002/jgrd.50207 ↗
73. Li, D., Saito, M., and Yang, P. (2023). Time-Dependent Systematic Biases in Inferring Ice Cloud Properties from Geostationary Satellite Observations. Remote Sens., 15.10.3390/rs15030855 ↗
74. Lin, J., Bao, Y., Petropoulos, G.P., Mehraban, A., Pang, F., and Liu, W. (2023). Cloud-Type Classification for Southeast China Based on Geostationary Orbit EO Datasets and the LighGBM Model. Remote Sens., 15.10.3390/rs15245660 ↗
75. Physics-guided machine learning for improved accuracy of the National Solar Radiation Database10.1016/j.solener.2022.01.004 ↗
76. McClear: A new model estimating downwelling solar radiation at ground level in clear-sky conditions10.5194/amt-6-2403-2013 ↗
77. Technical note: The libRadtran software package for radiative transfer calculations–description and examples of use10.5194/acp-5-1855-2005 ↗
78. The Moderate Resolution Imaging Spectroradiometer (MODIS): Land remote sensing for global change research10.1109/36.701075 ↗
79. All sky imaging-based short-term solar irradiance forecasting with Long Short-Term Memory networks10.1016/j.solener.2024.112463 ↗
80. Long short term memory–convolutional neural network based deep hybrid approach for solar irradiance forecasting10.1016/j.apenergy.2021.117061 ↗
81. Validation of the improved GOES-16 aerosol optical depth product over North America
82. The incidence and magnitude of the hot-spot bidirectional reflectance distribution function (BRDF) signature in GOES-16 Advanced Baseline Imager (ABI) 10 and 15 minute reflectance over North America10.1016/j.rse.2021.112638 ↗
83. An enhanced VIIRS aerosol optical depth product over land: Improved retrieval of aerosol optical depth over GOES-16 ABI using an empirical bias correction algorithm10.5194/amt-13-5955-2020 ↗
84. Copernicus Atmosphere Monitoring Service (CAMS) (2026, March 15). CAMS Global Atmospheric Composition Forecasts. ECMWF, Copernicus Atmosphere Monitoring Service, 2025. Global Analysis and Forecast Daily Data at Approximately 40 km Spatial Resolution (T511 Spectral Grid, 137 Levels). Available online: https://www.ecmwf.int/en/forecasts/dataset/cams-global-atmospheric-composition-forecasts.
85. Thaker, J., Höller, R., and Kapasi, M. (2024). Short-Term Solar Irradiance Prediction with a Hybrid Ensemble Model Using EUMETSAT Satellite Images. Energies, 17.10.3390/en17020329 ↗
86. Prasad, A.A., and Kay, M. (2021). Prediction of Solar Power Using Near-Real Time Satellite Data. Energies, 14.10.3390/en14185865 ↗
87. Noura, H.N., Allal, Z., Salman, O., and Chahine, K. (2025). Efficient Machine Learning-Based Prediction of Solar Irradiance Using Multi-Site Data. Future Internet, 17.10.3390/fi17080336 ↗
88. Analyzing GOES-R ABI BRDF-adjusted EVI2 time series by comparing with VIIRS observations over the CONUS10.1016/j.rse.2023.113972 ↗
89. Sultana, N. (2026, March 12). Clear Sky Environmental Data. GitHub Repository. Available online: https://github.com/Nifat-Sultana/clear_sky_env_data/.