OS-CTPIG: Augmenting and constructing a Chinese textile pattern image dataset with center radial layout using one-shot image generation methods
Source abstract
Abstract
Traditional Chinese textile patterns are often used as decorative elements on garments, fabrics, and accessories, adding unique cultural charm and aesthetic value to textile products. Recently, the big advances in data-driven artificial intelligence (AI) technology offer new ways to study textile patterns, but collecting enough data for training is challenging. One-shot image synthesis technology can help in constructing datasets under data-scarce conditions. Nevertheless, existing one-shot image generation methods are inadequate in fulfilling the structural layout and efficiency requirements for the production of traditional Chinese textile patterns. This paper proposes a one-shot Chinese textile pattern image generation (OS-CTPIG) method, which facilitates the generation of Chinese textile patterns with central radial structures, faithful colors, and exquisite details. A multiscale framework is employed for the generation of details in a coarse-to-fine manner. The simultaneous control over color and structure, along with the respective optimization at the color space and spatial feature space, complemented by a color leakage constraint module, is designed to preserve the traditional aesthetic standards in structures. Moreover, the integration of an energy function-enhanced sliced Wasserstein distance optimization objective and a dynamic task-weighting mechanism is proposed to improve computational efficiency. The experimental results demonstrate that the OS-CTPIG method is both effective and efficient in generating traditional Chinese textile pattern images under one-shot conditions. Subsequently, a dataset comprising over 3000 images of traditional Chinese textile pattern images has been constructed successfully.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Generative Adversarial Networks and Image Synthesis · Computer Graphics and Visualization Techniques · Face recognition and analysis
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Communication University of China
School of Cybersecurity, Xi'an Polytechnic University, Xi'an, Shaanxi, China
Shaanxi Key Laboratory of Clothing Intelligence, School of Computer Science, Xi'an Polytechnic University, Xi'an, China
Funding
the Natural Science Basic Research Program of Shaanxi Province · the Science and Technology Achievements Transformation Project of Shaanxi · the Key Research and Development Program of Shaanxi Province · the National Social Science Foundation of China Art Project
References (61)
Show source-provided references
1. Li M, Wang Y, Xu Y-Q. Computing for Chinese cultural heritage. Vis Informat 2022; 6: 1–13.10.1016/j.visinf.2021.12.006 ↗
2. 10.1016/j.ipm.2021.10275310.1016/j.ipm.2021.102753 ↗
3. 10.1016/j.patrec.2020.02.01710.1016/j.patrec.2020.02.017 ↗
4. 10.1080/09540091.2020.182278010.1080/09540091.2020.1822780 ↗
5. 10.1108/AA-07-2022-018310.1108/AA-07-2022-0183 ↗
6. 10.1016/j.compind.2022.10377910.1016/j.compind.2022.103779 ↗
7. 10.1177/0165551522107433610.1177/01655515221074336 ↗
8. 10.1145/353286710.1145/3532867 ↗
9. 10.1007/s11063-023-11323-z10.1007/s11063-023-11323-z ↗
10. 10.1016/j.ipm.2023.10354010.1016/j.ipm.2023.103540 ↗
11. 10.1007/s00521-022-07088-610.1007/s00521-022-07088-6 ↗
12. 10.1007/s00766-024-00415-410.1007/s00766-024-00415-4 ↗
13. Abdollahzadeh M Malekzadeh T Teo CTH et al. A survey on generative modeling with limited data few shots and zero shot http://arxiv.org/abs/2307.14397 (2023 accessed 7 June 2024).
14. Lv C, Zhang D, Geng S, et al. Color transfer for images: A survey. ACM Trans Multimedia Comput Commun Appl 2023; 3635152.
15. Wang Y Gonzalez-Garcia A Berga D et al. MineGAN: Effective knowledge transfer from GANs to target domains with few images. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition 2020 pp. 9332–9341.10.1109/CVPR42600.2020.00935 ↗
16. 10.1109/TMM.2021.307772910.1109/TMM.2021.3077729 ↗
17. Yang M Wang Z. Image synthesis under limited data: A survey and taxonomy https://arxiv.org/abs/2307.16879 [cs] (2023 accessed 27 March 2024).
18. Sushko V Gall J Khoreva A. One-shot GAN: Learning to generate samples from single images and videos. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition 2021 pp. 2596–2600.10.1109/CVPRW53098.2021.00293 ↗
19. Kirillov A Mintun E Ravi N et al. Segment anything. In: Proceedings of the IEEE/CVF International Conference on Computer Vision 2023 pp. 4015–4026.10.1109/ICCV51070.2023.00371 ↗
20. 10.1007/s10851-014-0506-310.1007/s10851-014-0506-3 ↗
21. Karras T, Aittala M, Hellsten J, et al. Training generative adversarial networks with limited data. Adv Neur Informat Process Syst 2020; 33: 12104–12114.
22. Jiang L, Dai B, Wu W, et al. Deceive d: Adaptive pseudo augmentation for gan training with limited data. Advances in Neural Information Processing Systems 2021; 34: 21655–21667.
23. Li Z, Usman M, Tao R, et al. A systematic survey of regularization and normalization in GANs. ACM Comput Surv 2023; 55: 1–37.10.1145/3569928 ↗
24. Liu B Zhu Y Song K et al. Towards faster and stabilized GAN training for high-fidelity few-shot image synthesis. In: International Conference on Learning Representations 2020.
25. Zhao M Cong Y Carin L. On leveraging pretrained GANs for generation with limited data. In: International conference on Machine Learning. PMLR 2020 pp. 11340–11351.
26. Moon T Choi M Lee G et al. Fine-tuning diffusion models with limited data. In: NeurIPS 2022 Workshop on Score-Based Methods 2022.
27. Hospedales T, Antoniou A, Micaelli P, et al. Meta-learning in neural networks: A survey. IEEE Trans Pattern Anal Machine Intell 2021; 44: 5149–5169.
28. Hong Y Niu L Zhang J et al. Few-shot image generation using discrete content representation. In: Proceedings of the 30th ACM International Conference on Multimedia 2022 pp. 2796–2804.10.1145/3503161.3548158 ↗
29. Gu Z Li W Huo J et al. LofGAN: Fusing local representations for few-shot image generation. In: Proceedings of the IEEE/CVF International Conference on Computer Vision 2021 pp. 8463–8471.10.1109/ICCV48922.2021.00835 ↗
30. Shaham TR Dekel T Michaeli T. SinGAN: Learning a generative model from a single natural image. In: Proceedings of the IEEE/CVF International Conference on Computer Vision 2019 pp. 4570–4580.10.1109/ICCV.2019.00467 ↗
31. Zhang Z Han C Guo T. ExsinGAN: Learning an explainable generative model from a single image. arXiv preprint arXiv:210507350.
32. Wang W Bao J Zhou W et al. Sindiffusion: Learning a diffusion model from a single natural image. arXiv preprint arXiv:221112445.
33. Nikankin Y Haim N Irani M. Sinfusion: Training diffusion models on a single image or video. arXiv preprint arXiv:221111743.
34. Granot N Feinstein B Shocher A et al. Drop the GAN: In defense of patches nearest neighbors as single image generative models. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition 2022 pp. 13460–13469.10.1109/CVPR52688.2022.01310 ↗
35. Elnekave A Weiss Y. Generating natural images with direct patch distributions matching. In: European Conference on Computer Vision. Berlin: Springer 2022 pp. 544–560.10.1007/978-3-031-19790-1_33 ↗
36. Nietert S, Goldfeld Z, Sadhu R, et al. Statistical, robustness, and computational guarantees for sliced Wasserstein distances. Adv Neur Informat Process Syst 2022; 35: 28179–28193.
37. Le D Nguyen H Nguyen K et al. Fast approximation of the generalized sliced-Wasserstein distance. In: ICASSP 2024 - 2024 IEEE International Conference on Acoustics Speech and Signal Processing (ICASSP). Piscataway NJ: IEEE 2024 pp. 6920–6924.10.1109/ICASSP48485.2024.10447733 ↗
38. Kolouri S, Nadjahi K, Simsekli U, et al. Generalized sliced Wasserstein distances. Adv Neur Informat Process Syst 2019. https://proceedings.neurips.cc/paper_files/paper/2019/hash/f0935e4cd5920aa6c7c996a5ee53a70f-Abstract.html
39. Deshpande I Hu Y-T Sun R et al. Max-sliced Wasserstein distance and its use for GANs. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition 2019 pp. 10648–10656.10.1109/CVPR.2019.01090 ↗
40. Xi J, Niles-Weed J. Distributional convergence of the sliced Wasserstein process. Adv Neur Informat Process Syst 2022; 35: 13961–13973.
41. Bonet C Berg P Courty N et al. Spherical sliced-Wasserstein. arXiv preprint arXiv:220608780.
42. Crawshaw M. Multi-task learning with deep neural networks: A survey. arXiv preprint arXiv:200909796.
43. Chen Z Badrinarayanan V Lee C-Y et al. Gradnorm: Gradient normalization for adaptive loss balancing in deep multitask networks. In: International Conference on Machine Learning. PMLR 2018 pp. 794–803.
44. Yu T, Kumar S, Gupta A, et al. Gradient surgery for multi-task learning. Adv Neur Informat Process Syst 2020; 33: 5824–5836.
45. Liu B, Feng Y, Stone P, et al. FAMO: Fast adaptive multitask optimization. Adv Neur Informat Process Syst 2023; 36: 57226–57243.
46. Zhao X Li H Shen X et al. A modulation module for multi-task learning with applications in image retrieval. In: Proceedings of the European Conference on Computer Vision (ECCV) 2018 pp. 401–416.10.1007/978-3-030-01246-5_25 ↗
47. Chen Z, Ngiam J, Huang Y, et al. Just pick a sign: Optimizing deep multitask models with gradient sign dropout. Adv Neur Informat Process Syst 2020; 33: 2039–2050.
48. Sanh V Wolf T Ruder S. A hierarchical multi-task approach for learning embeddings from semantic tasks. In: Proceedings of the AAAI Conference on Artificial Intelligence 2019 pp. 6949–6956.10.1609/aaai.v33i01.33016949 ↗
49. Nguyen K, Ho N. Energy-based sliced Wasserstein distance. Adv Neur Informat Process Syst 2023; 36: 18046–18075.
50. Fu S Hamilton M Brandt L et al. Featup: A model-agnostic framework for features at any resolution. arXiv preprint arXiv:240310516.
51. Oquab M Darcet T Moutakanni T et al. Dinov2: Learning robust visual features without supervision. arXiv preprint arXiv:230407193.
52. 10.1109/TPAMI.2022.315224710.1109/TPAMI.2022.3152247 ↗
53. 10.1109/TIP.2012.221405010.1109/TIP.2012.2214050 ↗
54. Wang J Chan KC Loy CC. Exploring clip for assessing the look and feel of images. In: Proceedings of the AAAI Conference on Artificial Intelligence 2023 pp. 2555–2563.10.1609/aaai.v37i2.25353 ↗
55. 10.1016/j.cviu.2016.12.00910.1016/j.cviu.2016.12.009 ↗
56. Talebi H, Milanfar P. NIMA: Neural image assessment. IEEE Trans Image Process 2017; 27(8): 3998–4011.10.1109/TIP.2018.2831899 ↗
57. Zhang Z Liu Y Han C et al. PetsGAN: Rethinking priors for single image generation. In: Proceedings of the AAAI Conference on Artificial Intelligence 2022 pp. 3408–3416.10.1609/aaai.v36i3.20251 ↗
58. Zhang Y Tang F Dong W et al. Domain enhanced arbitrary image style transfer via contrastive learning. In: Special Interest Group on Computer Graphics and Interactive Techniques Conference Proceedings pp. 1–8.10.1145/3528233.3530736 ↗
59. Chen H Zhao L Li J et al. TSSAT: Two-stage statistics-aware transformation for artistic style transfer. In: Proceedings of the 31st ACM International Conference on Multimedia 2023.10.1145/3581783.3611819 ↗
60. Bhattacharyya A. On a measure of divergence between two statistical populations defined by their probability distribution. Bull Calcutta Math Soc 1943; 35: 99–110.
61. Zhao F. Ornamental Patterns from Ancient Chinese Textiles: Han to Tang Dynasties. Zhejiang University Press, 2018, pp. 10–108.