Garment pattern generation method based on Diffusion Model and 3D reconstruction technology
Source abstract
Abstract
The swift advancement of image generation technology has rendered the generation of batches of automatic fashion design images feasible. Nevertheless, the manner of extracting corresponding patterns promptly and precisely from the generated personalized fashion images persists as a challenge. This study has formulated a 2D–3D–2D pattern generation methodology. This approach initially employs a fine-tuned diffusion model to generate high-quality garment images, subsequently utilizes a large-scale image 3D reconstruction model to automatically effect 3D reconstruction of the garment images, and ultimately uses 3D model reverse engineering technology to convert the 3D garment model into the corresponding garment pattern. We generated a dress pattern as an exemplar to validate the method. The experimental results indicated that the method is capable of converting fashion images generated by the diffusion model into their corresponding 2D patterns. The average error between the 2D pattern and the 3D model was less than 5 mm, and the 3D virtual try-on results of the generated pattern conformed to the style of the generated image, thereby verifying the efficacy of the method. Compared with manual pattern drafting, the time consumption was significantly reduced, providing robust support for the automated coupling of fashion design and pattern production.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: 3D Shape Modeling and Analysis · Generative Adversarial Networks and Image Synthesis · Human Motion and Animation
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Dalian Polytechnic University
Funding
"Textile Light" China Textile Industry Federation Higher Education Teaching Reform Research Project · Liaoning Revitalization Talents Program
References (56)
Show source-provided references
1. Sohl-Dickstein J Weiss EA Maheswaranathan N et al. Deep unsupervised learning using nonequilibrium thermodynamics. In: Proceedings of the 32nd international conference on machine learning (eds F Bach and D Blei) Lille France 6–11 July 2015 pp. 2256–2265. New York USA: ACM.
2. Goodfellow I Pouget-Abadie J Mirza M et al. Generative adversarial nets. In: Advances in neural information processing systems 27 (eds Z Ghahramani M Welling C Cortes et al.) Montreal Canada 8–13 December 2013 pp. 2672–2680. Cambridge USA: MIT Press.
3. 10.1080/09540091.2020.182278010.1080/09540091.2020.1822780 ↗
4. Ren Yujia, Chen L, Chen Y. Garment style interactive design based on DCGAN algorithm. Wool Text J 2022; 50: 93–97.
5. Tirtawan T, Susanto EK, Zaman PL, et al. Batik clothes auto-fashion using conditional generative adversarial network and U-Net. (EIConCIT) 2021: 145–150.
6. Wang L Liu J Cheng G et al. Constructing a personalized AI assistant for shear wall layout using Stable Diffusion. ArXiv https://arxiv.org/abs/2305.10830 (2023 accessed 21 October 2024).
7. Wang C, Chung J. Research on AI painting generation technology based on the [Stable Diffusion]. Int J Adv Smart Converg 2023; 12: 90–95.
8. 10.1109/TMM.2023.331829710.1109/TMM.2023.3318297 ↗
9. 10.1108/0955622071081952810.1108/09556220710819528 ↗
10. 10.1016/j.compind.2010.03.00210.1016/j.compind.2010.03.002 ↗
11. Deitke M Schwenk D Salvador D et al. Objaverse: a universe of annotated 3D objects. In: 2023 IEEE/CVF conference on computer vision and pattern recognition (eds xxxx) Vancouver Canada 17–24 June 2023 pp. 13142–13153. Piscataway NJ: IEEE.10.1109/CVPR52729.2023.01263 ↗
12. Deitke M Liu R Wallingford M et al. Objaverse-xl: A universe of 10m+ 3d objects. Advances in Neural Information Processing Systems 2023 36: 35799–35813.
13. Wu J Zhang C Xue T et al. Learning a probabilistic latent space of object shapes via 3D generative-adversarial modeling. In: NIPS’16: proceedings of the 30th international conference on neural information processing systems (eds DD Lee and U von Luxburg) Barcelona Spain 5–10 December 2016 pp. 82–90. Red Hook USA: Curran Associates.
14. Wang Z Wang Y Chen Y et al. CRM: single image to 3D textured mesh with convolutional reconstruction model. In: European conference on computer vision (eds A Leonardis E Ricci S Roth et al.) Nicosia Cyprus 2–3 April 2025 pp. 57–74. Switzerland: Springer Cham.10.1007/978-3-031-72751-1_4 ↗
15. Boss M, Huang Z, Vasishta A, et al. SF3D: stable fast 3D mesh reconstruction with UV-unwrapping and illumination disentanglement. ArXiv 2024: arXiv:2408.00653.
16. 10.1108/0955622111109673210.1108/09556221111096732 ↗
17. 10.1002/hfm.2018810.1002/hfm.20188 ↗
18. Song Y Ermon S. Generative modeling by estimating gradients of the data distribution. ArXiv https://arxiv.org/abs/1907.05600 (2019 accessed 18 September 2024).
19. Ho J Jain A Abbeel P. Denoising diffusion probabilistic models. In: Advances in neural information processing systems 34 (eds M Ranzato A Beygelzimer Y Dauphin et al.) pp. 6840–6851. Cambridge USA: MIT Press 2020.
20. Dhariwal P Nichol A. Diffusion models beat GANs on image synthesis. In Advances in neural information processing systems 35 (eds M Ranzato A Beygelzimer Y Dauphin et al.) 6–14 2021 pp. 8780–8794. Cambridge USA: MIT Press.
21. Rombach R Blattmann A Lorenz D et al. High-resolution image synthesis with latent diffusion models. In: 2022 IEEE/CVF conference on computer vision and pattern recognition (eds Lisa O’Conner) New Orleans USA 18–24 June 2022 pp. 10684–10695. Piscataway NJ: IEEE.10.1109/CVPR52688.2022.01042 ↗
22. Jaruga-Rozdolska A. Artificial intelligence as part of future practices in the architect’s work: MidJourney generative tool as part of a process of creating an architectural form. Architectus 2022; 3: 95–104.
23. Mirza M Osindero S. Conditional generative adversarial nets. In: Advances in neural information processing systems 27 (eds Z Ghahramani M Welling C Cortes et al). Montreal Canada 8–13 2014 pp. 5767–5777. Cambridge USA: MIT Press.
24. Cao Z Simon T WEI S E et al. Realtime multi-person 2D pose estimation using part affinity fields. In: 2017 IEEE conference on computer vision and pattern recognition (eds Lisa O’Conner) Honolulu USA 21–26 July 2017 pp. 7291–7299. Piscataway NJ: IEEE.10.1109/CVPR.2017.143 ↗
25. 10.1109/TPAMI.1986.476785110.1109/TPAMI.1986.4767851 ↗
26. HU E Shen Y Wallis P et al. LoRA: low-rank adaptation of large language models. ArXiv https://arxiv.org/abs/2106.09685v2 (2021 accessed 20 October 2024).
27. Ruiz N Li Y Jampani V et al. DreamBooth: fine tuning text-to-image diffusion models for subject-driven generation. In: 2023 IEEE/CVF conference on computer vision and pattern recognition (eds Lisa O’Conner) Vancouver Canada 17–24 June 2023 pp. 22500–22510. Piscataway NJ: IEEE.10.1109/CVPR52729.2023.02155 ↗
28. Gal R, Alaluf Y, Atzmon Y, et al. An image is worth one word: personalizing text-to-image generation using textual inversion. ArXiv 2022: arXiv:2208.01618.
29. Bennis C, Vezien JM, Iglesias G. Piecewise surface flattening for non-distorted texture mapping. ACM 1991; 25: 237–246
30. 10.1016/S0167-8396(00)00017-010.1016/S0167-8396(00)00017-0 ↗
31. 10.1016/0010-4485(91)90060-A10.1016/0010-4485(91)90060-A ↗
32. 10.1007/s12221-010-0142-510.1007/s12221-010-0142-5 ↗
33. 10.1108/0955622071081952810.1108/09556220710819528 ↗
34. 10.1108/0955622131132630110.1108/09556221311326301 ↗
35. 10.1177/0040517522111416410.1177/00405175221114164 ↗
36. 10.1016/0010-4485(91)90006-I10.1016/0010-4485(91)90006-I ↗
37. 10.1177/004051750606696610.1177/0040517506066966 ↗
38. 10.1109/38.53627610.1109/38.536276 ↗
39. 10.1016/0010-4485(94)90074-410.1016/0010-4485(94)90074-4 ↗
40. Hong Y, Zhang K, Gu J, et al. LRM: large reconstruction model for single image to 3D. ArXiv 2023: arXiv:2311.04400.
41. Tochilkin D, Pankratz D, Liu Z, et al. TripoSR: fast 3D object reconstruction from a single image. ArXiv 2024: arXiv:2403.02151.
42. Li P, Liu Y, Long X, et al. Era3D: high-resolution multiview diffusion using efficient row-wise attention. ArXiv 2024: arXiv:2405.11616.
43. Xu J, Cheng W, Gao Y, et al. InstantMesh: efficient 3D mesh generation from a single image with sparse-view large reconstruction models. ArXiv 2024: arXiv:2404.07191.
44. 10.1007/978-3-031-73235-5_110.1007/978-3-031-73235-5_1 ↗
45. Zou ZX Yu Z Guo YC et al. Triplane meets Gaussian splatting: fast and generalizable single-view 3D reconstruction with transformers. In: 2024 IEEE/CVF conference on computer vision and pattern recognition (eds Cristina Ceballos) Seattle USA 16–22 June 2024 pp. 10324–10335. Piscataway NJ: IEEE.10.1109/CVPR52733.2024.00983 ↗
46. Long X Guo YC Lin C et al. Wonder3D: single image to 3D using cross-domain diffusion. In: 2024 IEEE/CVF conference on computer vision and pattern recognition (eds Cristina Ceballos) Seattle USA 16–22 June 2024 pp. 9970–9980. Piscataway NJ: IEEE.10.1109/CVPR52733.2024.00951 ↗
47. Wu K Liu F Cai Z et al. Unique3D: high-quality and efficient 3D mesh generation from a single image. ArXiv https://arxiv.org/abs/2405.20343 (2024 accessed 23 August 2024).
48. Radford A Kim JW Hallacy C et al. Learning transferable visual models from natural language supervision. Proceedings of the 38th international conference on machine learning. ICML 2021: 8748–8763.
49. Pascual R Maiza A Sesma-Sara M et al. Enhancing DreamBooth with LoRA for generating unlimited characters with Stable Diffusion. 2024 international joint conference on neural networks Yokohama Japan 30 June–5 July 2024 pp. 1–8. Location: IEEE.10.1109/IJCNN60899.2024.10651300 ↗
50. Gu Y, Fang X, Deng X. A new Chinese landscape paintings generation model based on Stable Diffusion using DreamBooth. ArXiv 2024: arXiv:2408.08561.
51. Jang W Agapito L. CodeNeRF: disentangled neural radiance fields for object categories. In: 2021 IEEE/CVF international conference on computer vision (eds Lisa O’Conner) Montreal Canada 10–17 October 2021 pp. 12949–12958. Piscataway NJ: IEEE.10.1109/ICCV48922.2021.01271 ↗
52. Müller N Simonelli A Porzi L et al. AutoRF: learning 3D object radiance fields from single view observations. In: 2022 IEEE/CVF conference on computer vision and pattern recognition (eds Lisa O’Conner) New Orleans USA 18–24 June 2022 pp. 3971–3980. Piscataway NJ: IEEE.10.1109/CVPR52688.2022.00394 ↗
53. Caron M Touvron H Misra I et al. Emerging properties in self-supervised vision transformers. In: 2021 IEEE/CVF international conference on computer vision (eds Lisa O’Conner) Montreal Canada 10–17 October 2021 pp. 9650–9660. Piscataway NJ: IEEE.10.1109/ICCV48922.2021.00951 ↗
54. Li J Li D Xiong C et al. BLIP: bootstrapping language-image pre-training for unified vision-language understanding and generation. ArXiv https://arxiv.org/abs/2201.12086 (2022 accessed 16 September 2024).
55. Park J Ko B Jang H. StyleBoost: a study of personalizing text-to-image generation in any style using DreamBooth. 2023 14th international conference on information and communication technology convergence (ICTC). Red Hook USA: Curran Associates 2023 pp. 93–98.10.1109/ICTC58733.2023.10392676 ↗
56. Nathanael O T, Lumentut J S, Muliawan N H, et al. HYPNOS: highly precise foreground-focused diffusion finetuning for inanimate objects. ArXiv 2024: arXiv:2410.14265.