journal article·2025-07-26Open access

HyMePre: A Spatial–Temporal Pretraining Framework with Hypergraph Neural Networks for Short-Term Weather Forecasting

Fei Wang, Dawei Lin, Baojun Chen, Guodong Jing, Yi Geng, Xudong Ge, Daoming Wei, Ning Zhang

Applied Sciences· Volume 15· Issue 15

Abstract

Accurate short-term weather forecasting plays a vital role in disaster response, agriculture, and energy management, where timely and reliable predictions are essential for decision-making. Graph neural networks (GNNs), known for their ability to model complex spatial structures and relational data, have achieved remarkable success in meteorological forecasting by effectively capturing spatial dependencies among distributed weather stations. However, most existing GNN-based approaches rely on pairwise station connections, limiting their capacity to represent higher-order spatial interactions. Moreover, their dependence on supervised learning makes them vulnerable to spatial heterogeneity and temporal non-stationarity. This paper introduces a novel spatial–temporal pretraining framework, Hypergraph-enhanced Meteorological Pretraining (HyMePre), which combines hypergraph neural networks with self-supervised learning to model high-order spatial dependencies and improve generalization across diverse climate regimes. HyMePre employs a two-stage masking strategy, applying spatial and temporal masking separately, to learn disentangled representations from unlabeled meteorological time series. During forecasting, dynamic hypergraphs group stations based on meteorological similarity, explicitly capturing high-order dependencies. Extensive experiments on large-scale reanalysis datasets show that HyMePre outperforms conventional GNN models in predicting temperature, humidity, and wind speed. The integration of pretraining and hypergraph modeling enhances robustness to noisy data and improves generalization to unseen climate patterns, offering a scalable and effective solution for operational weather forecasting.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: Hydrological Forecasting Using AI · Climate variability and models · Energy Load and Power Forecasting

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

Key Laboratory of Smart Earth, Beijing 100029, China

CMA Key Laboratory of Cloud-Precipitation Physics and Weather Modification, Beijing 100081, China

CMA Weather Modification Centre, Beijing 100081, China

National Key Laboratory of Intelligent Spatial Information, Beijing 100029, China

Army 32152, People’s Liberation Army (PLA), Shijiazhuang 050000, China

Funding

Key Laboratory of Smart Earth · Key R&D Projects in Ningxia · National Key R&D Program of China · CMA Innovative and Development Program

References (43)

Show source-provided references
1. Intergovernmental Panel on Climate Change (IPCC) (2023). Climate Change 2021: The Physical Science Basis, Cambridge University Press. Contribution of Working Group I to the Sixth Assessment Report of the Intergovernmental Panel on Climate Change.10.1017/9781009157896 ↗
2. Global observed changes in daily climate extremes of temperature and precipitation10.1029/2005JD006290 ↗
3. Comparison of machine learning methods for photovoltaic power forecasting based on numerical weather prediction10.1016/j.rser.2022.112364 ↗
4. One-day-ahead probabilistic wind speed forecast based on optimized numerical weather prediction data10.1016/j.enconman.2018.03.030 ↗
5. Devlin, J., Chang, M.W., Lee, K., and Toutanova, K. (2019, January 2–7). BERT: Pre-training of deep bidirectional transformers for language understanding. Proceedings of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Minneapolis, MN, USA.
6. He, K., Chen, X., Xie, S., Li, Y., Dollár, P., and Girshick, R. (2022, January 19–24). Masked autoencoders are scalable vision learners. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.10.1109/CVPR52688.2022.01553 ↗
7. Li, Z., Rao, Z., Pan, L., Wang, P., and Xu, Z. (2023). Ti-MAE: Self-Supervised Masked Time Series Autoencoders. arXiv.
8. Bao, H., Dong, L., Piao, S., and Wei, F. (2022). BEiT: BERT Pre-Training of Image Transformers. arXiv.
9. Nie, Y., Nguyen, N.H., Sinthong, P., and Kalagnanam, J. (2023). A Time Series is Worth 64 Words: Long-term Forecasting with Transformers. arXiv.
10. HiSTGNN: Hierarchical spatio-temporal graph neural network for weather forecasting10.1016/j.ins.2023.119580 ↗
11. Lira, H., Martí, L., and Sánchez-Pi, N. (2022). A graph neural network with spatio-temporal attention for multi-sources time series data: An application to frost forecast. Sensors, 22.10.3390/s22041486 ↗
12. GraphCast: Learning skillful medium-range global weather forecasting10.1126/science.adi2336 ↗
13. NeuralGCM: Neural general circulation models for weather and climate10.1038/s41586-024-07744-y ↗
14. Bi, K., Xie, L., Zhang, H., Chen, X., Gu, X., and Tian, Q. (2022). Pangu-Weather: A 3D High-Resolution Model for Fast and Accurate Global Weather Forecast. arXiv.
15. Feng, Y., You, H., Zhang, Z., Ji, R., and Gao, Y. (February, January 27). Hypergraph neural networks. Proceedings of the Thirty-Third AAAI Conference on Artificial Intelligence (AAAI-19), Honolulu, HI, USA.
16. Hypergraph convolution and hypergraph attention10.1016/j.patcog.2020.107637 ↗
17. Yadati, N., Nimishakavi, M., Yadav, P., Nitin, V., Louis, A., and Talukdar, P. (2019, January 8–14). HyperGCN: A new method of training graph convolutional networks on hypergraphs. Proceedings of the Advances in Neural Information Processing Systems (NeurIPS), Vancouver, BC, Canada.
18. Chen, H., Rossi, R.A., Kim, S., Mahadik, K., and Eldardiry, H. (2025, January 3–7). Probabilistic Hypergraph Recurrent Neural Networks for Time-series Forecasting. Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining, Toronto, ON, Canada.10.1145/3690624.3709202 ↗
19. Regional heatwave prediction using graph neural network and weather station data10.1029/2023GL103405 ↗
20. Davidson, M., and Moodley, D. (2022, January 5–9). ST-GNNs for weather prediction in South Africa. Proceedings of the Southern African Conference for Artificial Intelligence Research, Stellenbosch, South Africa.10.1007/978-3-031-22321-1_7 ↗
21. Liu, H., Han, Q., Sun, H., Sheng, J., and Yang, Z. (2023). Spatiotemporal adaptive attention graph convolution network for city-level air quality prediction. Sci. Rep., 13.10.1038/s41598-023-39286-0 ↗
22. Promoting wind energy for sustainable development by precise wind speed prediction based on graph neural networks10.1016/j.renene.2022.09.036 ↗
23. Spatio-temporal graph deep neural network for short-term wind speed forecasting10.1109/TSTE.2018.2844102 ↗
24. Wilson, T., Tan, P.N., and Luo, L. (2018, January 17–20). A low rank weighted graph convolutional approach to weather prediction. Proceedings of the 2018 IEEE International Conference on Data Mining (ICDM), Singapore.10.1109/ICDM.2018.00078 ↗
25. A novel physics-aware graph network using high-order numerical methods in weather forecasting model10.1016/j.knosys.2024.112158 ↗
26. Group-aware graph neural network for nationwide city air quality forecasting
27. Joint air quality and weather prediction based on multi-adversarial spatiotemporal networks
28. Utility of Graph Neural Networks in Short-to Medium-Range Weather Forecasting10.32604/cmc.2025.063373 ↗
29. Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., and Stoyanov, V. (2019). RoBERTa: A robustly optimized BERT pretraining approach. arXiv.
30. Lan, Z., Chen, M., Goodman, S., Gimpel, K., Sharma, P., and Soricut, R. (2019). ALBERT: A lite BERT for self-supervised learning of language representations. arXiv.
31. Shao, Z., Zhang, Z., Wang, F., and Xu, Y. (2022, January 14–18). Pre-training enhanced spatial-temporal graph neural network for multivariate time series forecasting. Proceedings of the 28th ACM SIGKDD Conference Knowledge Discovery and Data Mining, Washington, DC, USA.10.1145/3534678.3539396 ↗
32. Agarwal, S., Branson, K., and Belongie, S. (2006, January 25–29). Higher order learning with graphs. Proceedings of the 23rd International Conference on Machine Learning, Pittsburgh, PA, USA.10.1145/1143844.1143847 ↗
33. Bretto, A. (2013). Hypergraph theory. An Introduction. Mathematical Engineering, Springer.10.1007/978-3-319-00080-0 ↗
34. Zhang, R., Zou, Y., and Ma, J. (2019). Hyper-SAGNN: A self-attention based graph neural network for hypergraphs. arXiv.
35. Yi, J., and Park, J. (2020, January 6–10). Hypergraph convolutional recurrent neural network. Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, Virtual Event.10.1145/3394486.3403389 ↗
36. Sawhney, R., Agarwal, S., Wadhwa, A., and Shah, R.R. (2020, January 17–20). Spatiotemporal hypergraph convolution network for stock movement forecasting. Proceedings of the 2020 IEEE International Conference on Data Mining (ICDM), Sorrento, Italy.10.1109/ICDM50108.2020.00057 ↗
37. Metro passenger flow prediction via dynamic hypergraph convolution networks10.1109/TITS.2021.3072743 ↗
38. Multi-Information Aggregation and Estrangement HyperGraph Convolutional Networks for Spatiotemporal Weather Forecasting10.1109/TGRS.2024.3486684 ↗
39. Han, H., Zhang, M., Hou, M., Zhang, F., Wang, Z., Chen, E., Wang, H., Ma, J., and Liu, Q. (2020, January 17–20). STGCN: A spatial-temporal aware graph learning method for POI recommendation. Proceedings of the 2020 IEEE International Conference on Data Mining (ICDM), Sorrento, Italy.10.1109/ICDM50108.2020.00124 ↗
40. Guo, S., Lin, Y., Feng, N., Song, C., and Wan, H. (February, January 27). Attention based spatial-temporal graph convolutional networks for traffic flow forecasting. Proceedings of the AAAI Conference on Artificial Intelligence, Honolulu, HI, USA.
41. Adaptive graph convolutional recurrent network for traffic forecasting
42. Xu, Z., Yao, Q., Li, Y., and Yang, Q. (2023). Understanding and simplifying architecture search in spatio-temporal graph neural networks. Trans. Mach. Learn. Res., Available online: https://openreview.net/forum?id=4jEuiMPKSF.
43. Wang, Y., Xu, Y., Yang, J., Wu, M., Li, X., Xie, L., and Chen, Z. (2024, January 20–27). Fully-connected spatial-temporal graph for multivariate time-series data. Proceedings of the AAAI Conference on Artificial Intelligence, Vancouver, BC, Canada.10.1609/aaai.v38i14.29500 ↗