Enhanced Chinese mural face generation via FreqSplitAttention and dual mask discriminator
Source abstract
Abstract
Chinese mural faces embody significant cultural, religious, and historical values, but their preservation is challenged by natural degradation and data scarcity. Existing generation methods struggle with poor detail retention, style inconsistency, and overfitting due to limited data. To address these limitations, we propose an enhanced StyleGAN2 architecture integrating FreqSplitAttention and Dual Mask Discriminator (DMD). FreqSplitAttention employs Fourier transforms to decouple and process high- and low-frequency components independently enhancing both global structural coherence and fine-grained texture reconstruction. The DMD operates in spatial and spectral domains to mitigate overfitting and shortcut learning. Furthermore, We construct the Dunhuang Mural Face (DMF) dataset, comprising 9552 high-quality images at 256 × 256 resolution. Experimental results demonstrate that our method outperforms state-of-the-art models on DMF, achieving superior performance with an FID of 27.74, KID of 10.22, and IS of 2.19. Our work offers a robust solution for mural face generation and contributes to digital cultural heritage preservation.
Source: openalex_inverted_index · Quality: baseline
Research context
OpenAlex topics: Face recognition and analysis · Generative Adversarial Networks and Image Synthesis · Advanced Image Processing Techniques
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
References (45)
Show source-provided references
1. Sun, T. et al. Restoring Dunhuang murals: crafting cultural heritage preservation knowledge into immersive virtual reality experience design. Int. J. Hum.-Comput. Interact. 40, 2019–2040 (2024).10.1080/10447318.2023.2232976 ↗
2. Guan, Y. Aesthetic conceptions and cultural symbols in traditional Chinese painting. Trans/Form/Ação 47, e0240066 (2024).10.1590/0101-3173.2024.v47.n4.e0240066 ↗
3. Mol, V. R. & Maheswari, P. U. The digital reconstruction of degraded ancient temple murals using dynamic mask generation and an extended exemplar-based region-filling algorithm. Herit. Sci. 9, 137 (2021).10.1186/s40494-021-00604-2 ↗
4. Gong, Z., Wang, R. & Xia, G. Augmented reality (AR) as a tool for engaging museum experience: a case study on Chinese art pieces. Digit. 2, 33–45 (2022).10.3390/digital2010002 ↗
5. Li, F. & Xia, S. International cooperation in the digital preservation of the cultural heritage of the Dunhuang Mogao Grottoes. Libr. Trends 71, 345–363 (2023).10.1353/lib.2023.a925014 ↗
6. Karras, T., Laine, S. & Aila, T. A style-based generator architecture for generative adversarial networks. In: Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. 4401–4410 (2019).10.1109/CVPR.2019.00453 ↗
7. Karras, T. et al. Analyzing and improving the image quality of StyleGAN. In: Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. 8107–8116 (2020).10.1109/CVPR42600.2020.00813 ↗
8. Karras, T. et al. Training generative adversarial networks with limited data. Adv. Neural Inf. Process. Syst. 33, 12104–12114 (2020).
9. Yazici, Y. et al. Empirical analysis of overfitting and mode drop in GAN training. In: Proc. IEEE Int. Conf. Image Process. 1651–1655 (2020).10.1109/ICIP40778.2020.9191083 ↗
10. Ojha, U. et al. Few-shot image generation via cross-domain correspondence. In: Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. 10743–10752 (2021).10.1109/CVPR46437.2021.01060 ↗
11. Gal, R. et al. StyleGAN-NADA: CLIP-guided domain adaptation of image generators. ACM Transactions on Graphics (TOG). 41, 1–13 (2022).10.1145/3528223.3530164 ↗
12. Goodfellow, I. et al. Generative adversarial nets. Adv. Neural Inf. Process. Syst. 27, 2672–2680 (2014).
13. Karras, T. et al. Alias-free generative adversarial networks. Adv. Neural Inf. Process. Syst. 34, 852–863 (2021).
14. Jiang, L., Dai, B., Wu, W. & Loy, C. C. Deceive D: Adaptive pseudo augmentation for GAN training with limited data. Adv. Neural Inf. Process. Syst. 34, 21655–21667 (2021).
15. Li, Z. et al. FakeCLR: Exploring contrastive learning for solving latent discontinuity in data-efficient GANs. In: Eur. Conf. Comput. Vis. 598–615 (2022).10.1007/978-3-031-19784-0_35 ↗
16. Chung, J. et al. Diversity-aware channel pruning for StyleGAN compression. In: Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. 7902–7911 (2024).10.1109/CVPR52733.2024.00755 ↗
17. Schwarz, K., Liao, Y. & Geiger, A. On the frequency bias of generative models. Adv. Neural Inf. Process. Syst. 34, 18126–18136 (2021).
18. Wang, Z., Chi, Z. & Zhang, Y. FreGAN: Exploiting frequency components for training GANs under limited data. Adv. Neural Inf. Process. Syst. 35, 33387–33399 (2022).
19. Gal, R. et al. Swagan: A style-based wavelet-driven generative model. ACM Trans. Graph. 40, 1–11 (2021).10.1145/3450626.3459836 ↗
20. Yang, M. et al. WaveGAN: Frequency-aware GAN for high-fidelity few-shot image generation. In: Eur. Conf. Comput. Vis. 1–17 (2022).10.1007/978-3-031-19784-0_1 ↗
21. Jiang, L. et al. Focal frequency loss for image reconstruction and synthesis. In: Proc. IEEE/CVF Int. Conf. Comput. Vis. 13919–13929 (2021).
22. Zhu, J., Ma, H. & Chen, J. High-quality and diverse few-shot image generation via masked discrimination. IEEE Trans. Image Process. 2950–2965 (2024).10.1109/TIP.2024.3385295 ↗
23. Yang, C. et al. Improving GANs with a dynamic discriminator. Adv. Neural Inf. Process. Syst. 35, 15093–15104 (2022).
24. Huang, J. et al. Masked generative adversarial networks are data-efficient generation learners. Adv. Neural Inf. Process. Syst. 35, 2154–2167 (2022).
25. Huang, M. et al. Towards accurate image coding: improved autoregressive image generation with dynamic vector quantization. In: Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. 22596–22605 (2023).10.1109/CVPR52729.2023.02164 ↗
26. Huang, M. et al. Not all image regions matter: masked vector quantization for autoregressive image generation. In: Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. 2002–2011 (2023).10.1109/CVPR52729.2023.00199 ↗
27. Ho, J., Jain, A. & Abbeel, P. Denoising diffusion probabilistic models. Adv. Neural Inf. Process. Syst. 33, 6840–6851 (2020).
28. Song, J., Meng, C. & Ermon, S. Denoising diffusion implicit models. In: 9th Int. Conf. Learn. Represent. (ICLR) (2021).
29. Wang, Z., Zheng, H., He, P., Chen, W. & Zhou, M. Diffusion-GAN: Training GANs with diffusion. In: 11th Int. Conf. Learn. Represent. (ICLR) (2023).
30. Zhang, B. et al. StyleSwin: Transformer-based GAN for high-resolution image generation. In: Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. 11304–11314 (2022).10.1109/CVPR52688.2022.01102 ↗
31. Dosovitskiy, A. et al. An image is worth 16×16 words: transformers for image recognition at scale. In: 9th Int. Conf. Learn. Represent. (ICLR) (2021).
32. Wang, X. et al. Real-ESRGAN: Training real-world blind super-resolution with pure synthetic data. In: Proc. IEEE/CVF Int. Conf. Comput. Vis. 1905–1914 (2021).10.1109/ICCVW54120.2021.00217 ↗
33. Heusel, M. et al. GANs trained by a two time-scale update rule converge to a local Nash equilibrium. In: Adv. Neural Inf. Process. Syst. 30, 6626–6637 (2017).
34. Bińkowski, M. et al. Demystifying MMD GANs. In: Int. Conf. Learn. Represent. (ICLR) (2018).
35. Salimans, T. et al. Improved techniques for training GANs. Adv. Neural Inf. Process. Syst. 29, 2226–2234 (2016).
36. Horé, A. & Ziou, D. Image quality metrics: PSNR vs. SSIM. In: 20th Int. Conf. Pattern Recognit. 2366–2369 (2010).10.1109/ICPR.2010.579 ↗
37. Wang, Z., Simoncelli, E. P. & Bovik, A. C. Multiscale structural similarity for image quality assessment. In: 37th Asilomar Conf. Signals, Syst. Comput. 1398–1402 (2003).
38. Kingma, D. P. & Ba, J. Adam: A method for stochastic optimization. In: 3rd Int. Conf. Learn. Represent. (ICLR) (2015).
39. Zhao, Z. et al. Improved consistency regularization for GANs. In: Proc. AAAI Conf. Artif. Intell. 35, 11033–11041 (2021).10.1609/aaai.v35i12.17317 ↗
40. Liu, Z. et al. Swin transformer: Hierarchical vision transformer using shifted windows. In: Proc. IEEE/CVF Int. Conf. Comput. Vis. 10012–10022 (2021).10.1109/ICCV48922.2021.00986 ↗
41. Woo, S., Park, J., Lee, J. Y. & Kweon, I. S. CBAM: Convolutional block attention module. In: Eur. Conf. Comput. Vis. 3–19 (2018).10.1007/978-3-030-01234-2_1 ↗
42. Han, D. et al. Agent attention: On the integration of softmax and linear attention. In: Eur. Conf. Comput. Vis. 124–140 (2024).10.1007/978-3-031-72973-7_8 ↗
43. Florian, M. K. et al. The Gini coefficient as a tool for image family identification in strong lensing systems with multiple images. Astrophys. J. Lett. 816, L23 (2016).10.3847/2041-8205/816/2/L23 ↗
44. Rao, Y. et al. Global filter networks for image classification. Adv. Neural Inf. Process. Syst. 34, 980–993 (2021).
45. Liu, H. et al. Adaptivemix: Improving GAN training via feature space shrinkage. In: Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. 16219–16229 (2023).10.1109/CVPR52729.2023.01556 ↗