Hierarchical symmetric GAN for Thangka image generation
Source abstract
Abstract
Text-to-Thangka generation requires preserving both semantic accuracy and textural details. Current methods struggle with fine-grained feature extraction, multi-level feature integration, and discriminator overfitting due to limited Thangka data. We present HST-GAN, a novel framework combining parallel hybrid attention with differentiable symmetric augmentation. The architecture features a Parallel Spatial-Channel Attention module (PSCA) for precise localization of deity facial features and ritual object textures, along with a Hierarchical Feature Fusion Network (HLFN) for multi-scale alignment. The framework’s Differentiable Symmetric Augmentation (DiffAugment) dynamically adjusts discriminator inputs to prevent overfitting while improving generalization. On the T2IThangka dataset, HST-GAN achieves an Inception Score of 2.08 and reduces Fréchet Inception Distance to 87.91, demonstrating superior performance over baselines on the Oxford-102 benchmark.
Source: openalex_inverted_index · Quality: baseline
Research context
OpenAlex topics: Generative Adversarial Networks and Image Synthesis · Face recognition and analysis · Advanced Neural Network Applications
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
References (28)
Show source-provided references
1. Li, Y. & Liu, X. Sketch-based Thangka image retrieval. In Proc. 5th Advanced Information Technology, Electronic and Automation Control Conference (IAEAC). 2066–2070 (IEEE, 2021).10.1109/IAEAC50856.2021.9390657 ↗
2. Goodfellow, I. J. et al. Generative adversarial nets. In Proc. 28th International Conference on Neural Information Processing Systems 2672–2680 (ACM, 2014).
3. Sheynin, S. et al. kNN-diffusion: image generation via large-scale retrieval. Preprint at https://doi.org/10.48550/arXiv.2204.02849 (2022).10.48550/arXiv.2204.02849 ↗
4. Zhang, H. et al. StackGAN: Text to photo-realistic image synthesis with stacked generative adversarial networks. In Proc. IEEE International Conference on Computer Vision. 5908–5916. https://doi.org/10.1109/ICCV.2017.629 (IEEE, 2017).10.1109/ICCV.2017.629 ↗
5. Xu, T. et al. AttnGAN: Fine-grained text to image generation with attentional generative adversarial networks. In Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. 1316–1324. https://doi.org/10.1109/CVPR.2018.00143 (IEEE, 2018).10.1109/CVPR.2018.00143 ↗
6. Tao, M. et al. DF-GAN: A simple and effective baseline for text-to-image synthesis. In Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVRP) 16515–16525 (IEEE, 2022).10.1109/CVPR52688.2022.01602 ↗
7. Zhu, M. et al. DM-GAN: dynamic memory generative adversarial networks for text-to-image synthesis. In Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition. 5795–5803 (IEEE, 2019).10.1109/CVPR.2019.00595 ↗
8. Kang M. et al. Scaling up GANs for text-to-image synthesis. In Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition. 10124–10134 (IEEE, 2023).10.1109/CVPR52729.2023.00976 ↗
9. Saharia, C. et al. Photorealistic text-to-image diffusion models with deep language understanding. Adv. Neural Inf. Process. Syst. 36, 36479–36494 (2022).
10. Zheng, W. et al. (2024). CogView3: finer and faster text-to-image generation via relay diffusion. In: Leonardis, A., Ricci, E., Roth, S., Russakovsky, O., Sattler, T., Varol, G. (eds) Computer Vision. ECCV 2024. Lecture Notes in Computer Science, vol 15135. 1–22 (Springer, 2024).
11. Lin, C., Lu, X. & Chen, G. RATLIP: generative adversarial CLIP text-to-image synthesis based on recurrent affine transformations. https://doi.org/10.48550/arXiv.2405.08114 (2024).10.48550/arXiv.2405.08114 ↗
12. Tao M. et al. GALIP: generative adversarial CLIPs for text-to-image synthesis. In Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition. 14214–14223 (IEEE, 2023).10.1109/CVPR52729.2023.01366 ↗
13. Yang, B. et al. SF-GAN: semantic fusion generative adversarial networks for text-to-image synthesis. Expert Syst. Appl. 262, 125583 (2025).10.1016/j.eswa.2024.125583 ↗
14. Woo, S., Park, J., Lee, J.-Y. & Kweon, I. S. CBAM: convolutional block attention module. In Computer Vision - ECCV (eds. Ferrari, V. et al.) 3–19 (Springer, 2018).10.1007/978-3-030-01234-2_1 ↗
15. Zhang, H. et al. StackGAN + +: Realistic image synthesis with stacked generative adversarial networks. IEEE Trans. Pattern Anal. Mach. Intell. 41, 1947–1962 (2019).10.1109/TPAMI.2018.2856256 ↗
16. Brock, A. et al. Large Scale GAN Training for High Fidelity Natural Image Synthesis. International Conference on Learning Representations (ICLR), 2019.
17. Gulrajani, I. et al. Improved training of Wasserstein GANs. In Proc. 31st International Conference on Neural Information Processing Systems. Long Beach, California, USA: Curran Associates Inc., 5769–5779 (2017).
18. Zhang, H. et al. Consistency regularization for generative adversarial networks. In Proc. 6th International Conference on Learning Representations. (2020).
19. Zhao, S. et al. Differentiable augmentation for data-efficient GAN Training. Adv. Neural Inf. Process. Syst. 33, 7559–7570 (2020).
20. Hou, L. et al. Augmentation-aware self-supervision for data-efficient GAN training[J]. Adv. Neural Inf. Process. Syst. 36, 31601–31620 (2023).
21. Ye, S., Wang, H., Tan, M. & Liu, F. Recurrent affine transformation for text-to-image synthesis. IEEE Trans. Multimed. 26, 462–473 (2024).10.1109/TMM.2023.3266607 ↗
22. Hu, W., Zhang, F. & Zhao, Y. Thangka image captioning model with salient attention and local interaction aggregator. Herit. Sci. 12, 407 (2024).10.1186/s40494-024-01518-5 ↗
23. Nilsback, M.-E. & Zisserman, A. Automated flower classification over a large number of classes. In Proc.Indian Conference on Computer Vision, Graphics and Image Processing. 722–729. https://doi.org/10.1109/ICVGIP.2008.47 (2008).10.1109/ICVGIP.2008.47 ↗
24. Salimans, T. et al. Improved techniques for training GANs. Adv. Neural Inf. Process. Syst. 29, 2234–2242 (2016).
25. Heusel, M. et al. GANs trained by a two time-scale update rule converge to a local Nash equilibrium. Adv. Neural Inf. Process. Syst. 30, 6629–6640 (2017).
26. Liao, W. et al. Text-to-image generation with semantic-spatial aware GAN. Proc. IEEE Conf. Comput. Vis. Pattern Recognit. 2022, 18166–18175, https://doi.org/10.1109/CVPR52688.2022.01765 (2022).10.1109/CVPR52688.2022.01765 ↗
27. Lee, M. & Seok, J. Controllable generative adversarial network. IEEE Access 7, 28158–28169 (2019).10.1109/ACCESS.2019.2899108 ↗
28. Gu, S. et al. Vector quantized diffusion model for text-to-image synthesis. Proc. IEEE Conf. Comput. Vis. Pattern Recognit. 2022, 10686–10696 (2022).