journal article·2024-11-15Open access

Radar Echo Extrapolation Based on Translator Coding and Decoding Conditional Generation Adversarial Network

Xingang Mou, Yuan He, Wenfeng Li, Xiao Zhou

Applied Sciences· Volume 14· Issue 22

Abstract

In response to the shortcomings of current spatiotemporal prediction models, which frequently encounter difficulties in temporal feature extraction and the forecasting of medium to high echo intensity regions over extended sequences, this study presents a novel model for radar echo extrapolation that combines a translator encoder-decoder architecture with a spatiotemporal dual-discriminator conditional generative adversarial network (STD-TranslatorNet). Initially, an image reconstruction network is established as the generator, employing a combination of a temporal attention unit (TAU) and an encoder–decoder framework. Within this architecture, both intra-frame static attention and inter-frame dynamic attention mechanisms are utilized to derive attention weights across image channels, thereby effectively capturing the temporal evolution of time series images. This approach enhances the network’s capacity to comprehend local spatial features alongside global temporal dynamics. The encoder–decoder configuration further bolsters the network’s proficiency in feature extraction through image reconstruction. Subsequently, the spatiotemporal dual discriminator is crafted to encapsulate both temporal correlations and spatial attributes within the generated image sequences. This design serves to effectively steer the generator’s output, thereby augmenting the realism of the produced images. Lastly, a composite multi-loss function is proposed to enhance the network’s capability to model intricate spatiotemporal evolving radar echo data, facilitating a more comprehensive assessment of the quality of the generated images, which in turn fortifies the network’s robustness. Experimental findings derived from the standard radar echo dataset (SRAD) reveal that the proposed radar echo extrapolation technique exhibits superior performance, with average critical success index (CSI) and probability of detection (POD) metrics per frame increasing by 6.9% and 7.6%, respectively, in comparison to prior methodologies.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: Image and Signal Denoising Methods · Underwater Acoustics Research · Speech and Audio Processing

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

School of Mechanical and Electronic Engineering, Wuhan University of Technology, Wuhan 430070, China

References (42)

Show source-provided references
1. Nowcasting of convective rainfall using volumetric radar observations10.1109/TGRS.2020.2984594 ↗
2. Nowcasting thunderstorms: A status report10.1175/1520-0477(1998)079<2079:NTASR>2.0.CO;2 ↗
3. Use of NWP for nowcasting convective precipitation: Recent progress and challenges10.1175/BAMS-D-11-00263.1 ↗
4. Cheung, P., and Yeung, H.Y. (2012, January 6–10). Application of optical-flow technique to significant convection nowcast for terminal areas in Hong Kong. Proceedings of the 3rd WMO International Symposium on Nowcasting and Very Short-Range Forecasting (WSN12), Rio de Janeiro, Brazil.
5. Convolutional LSTM network: A machine learning approach for precipitation nowcasting
6. Predrnn: Recurrent neural networks for predictive learning using spatiotemporal lstms
7. Wang, Y., Gao, Z., Long, M., Wang, J., and Philip, S.Y. (2018, January 10–15). Predrnn++: Towards a resolution of the deep-in-time dilemma in spatio-temporal predictive learning. Proceedings of the International Conference on Machine Learning, Stockholm, Sweden.
8. Wang, Y., Jiang, L., Yang, M.-H., Li, L.-J., Long, M., and Fei-Fei, L. (May, January 30). Eidetic 3D LSTM: A model for video prediction and beyond. Proceedings of the International Conference on Learning Representations, Vancouver, BC, Canada.
9. Wang, Y., Zhang, J., Zhu, H., Long, M., Wang, J., and Yu, P.S. (2019, January 15–20). Memory in memory: A predictive neural network for learning higher-order non-stationarity from spatiotemporal dynamics. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.10.1109/CVPR.2019.00937 ↗
10. Patraucean, V., Handa, A., and Cipolla, R. (2015). Spatio-temporal video autoencoder with differentiable memory. arXiv.
11. Castrejon, L., Ballas, N., and Courville, A. (November, January 27). Improved conditional vrnns for video prediction. Proceedings of the IEEE/CVF International Conference on Computer Vision, Seoul, Republic of Korea.
12. Yu, W., Lu, Y., Easterbrook, S., and Fidler, S. (May, January 26). Efficient and information-preserving future frame prediction and beyond. Proceedings of the International Conference on Learning Representations, Virtual.
13. Learning to decompose and disentangle representations for video prediction
14. Guen, V.L., and Thome, N. (2020, January 13–19). Disentangling physical dynamics from unknown factors for unsupervised video prediction. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.
15. Generative adversarial nets
16. Mou, X., Liu, C., and Zhou, X. (2021, January 25–27). Radar Echo Image Prediction Algorithm Based on Multi-scale Encoding-Decoding Network. Proceedings of the Intelligent Data Engineering and Automated Learning–IDEAL 2021: 22nd International Conference, IDEAL 2021, Manchester, UK.10.1007/978-3-030-91608-4_61 ↗
17. Kwon, Y.H., and Park, M.G. (2019, January 15–20). Predicting future frames using retrospective cycle gan. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.10.1109/CVPR.2019.00191 ↗
18. Zhu, J.-Y., Park, T., Isola, P., and Efros, A.A. (2017, January 22–29). Unpaired image-to-image translation using cycle-consistent adversarial networks. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.10.1109/ICCV.2017.244 ↗
19. Yi, Z., Zhang, H., Tan, P., and Gong, M. (2017, January 22–29). Dualgan: Unsupervised dual learning for image-to-image translation. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.10.1109/ICCV.2017.310 ↗
20. Farooq, J., Bazaz, M.A., and Rafiq, D. (2023, January 21–23). Multiscale Autoencoder-RNN Architecture for Mitigating Error Accumulation in Long-Term Forecasting. Proceedings of the 2023 International Conference on Emerging Techniques in Computational Intelligence (ICETCI), Hyderabad, India.10.1109/ICETCI58599.2023.10331086 ↗
21. Tan, C., Gao, Z., Wu, L., Xu, Y., Xia, J., Li, S., and Li, S.Z. (2023, January 17–24). Temporal attention unit: Towards efficient spatiotemporal predictive learning. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, BC, Canada.10.1109/CVPR52729.2023.01800 ↗
22. Badrinarayanan, V., Kendall, A., and SegNet, R.C. (2015). A deep convolutional encoder-decoder architecture for image segmentation. arXiv.
23. Tran, D., Bourdev, L., Fergus, R., Torresani, L., and Paluri, M. (2015, January 7–15). Learning spatiotemporal features with 3d convolutional networks. Proceedings of the IEEE International Conference on Computer Vision, Santiago, Chile.10.1109/ICCV.2015.510 ↗
24. Sajjadi, M.S.M., Vemulapalli, R., and Brown, M. (2018, January 18–23). Frame-recurrent video super-resolution. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00693 ↗
25. Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., and Rabinovich, A. (2015, January 7–12). Going deeper with convolutions. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA.10.1109/CVPR.2015.7298594 ↗
26. Mirza, M. (2014). Conditional generative adversarial nets. arXiv.
27. He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27–30). Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.10.1109/CVPR.2016.90 ↗
28. Ding, X., Zhang, X., Han, J., and Ding, G. (2022, January 18–24). Scaling up your kernels to 31 × 31: Revisiting large kernel design in cnns. Proceedings of the the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.10.1109/CVPR52688.2022.01166 ↗
29. Lin, M. (2013). Network in Network. arXiv.
30. Miyato, T., Kataoka, T., Koyama, M., and Yoshida, Y. (2018). Spectral normalization for generative adversarial networks. arXiv.
31. Image quality assessment: From error visibility to structural similarity10.1109/TIP.2003.819861 ↗
32. An overview of nowcasting development, applications, and services in the Hong Kong Observatory10.1007/s13351-014-4048-9 ↗
33. Identification of Hail, Strong Winds, and Heavy Rainfall Using CINRAD-SA Radar Products
34. A method for detecting rainfall from X-band marine radar images10.1109/ACCESS.2020.2968601 ↗
35. Korhonen, J., and You, J. (2012, January 5–7). Peak signal-to-noise ratio revisited: Is simple beautiful?. Proceedings of the 2012 Fourth International Workshop on Quality of Multimedia Experience, Melbourne, Australia.10.1109/QoMEX.2012.6263880 ↗
36. The ecological approach to the visual perception of pictures10.2307/1574154 ↗
37. Liu, Z., Mao, H., Wu, C.-Y., Feichtenhofer, C., Darrell, T., and Xie, S. (2022, January 18–24). A convnet for the 2020s. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.10.1109/CVPR52688.2022.01167 ↗
38. Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., and Guo, B. (2021, January 11–17). Swin transformer: Hierarchical vision transformer using shifted windows. Proceedings of the IEEE/CVF International Conference on Computer Vision, Montreal, BC, Canada.10.1109/ICCV48922.2021.00986 ↗
39. Visual attention network10.1007/s41095-023-0364-2 ↗
40. Dosovitskiy, A. (2020). An image is worth 16x16 words: Transformers for image recognition at scale. arXiv.
41. Tan, C., Gao, Z., Li, S., and Li, S.Z. (2022). Simvp: Towards simple yet powerful spatiotemporal predictive learning. arXiv.
42. Sevir: A storm event imagery dataset for deep learning applications in radar and satellite meteorology