journal article·2025-02-01Open access

From Words to Worlds: Exploring Generative 3D Models in Design and Fabrication

Valdemar Danry, Cenk Guzelis, Lingdong Huang, Neil Gershenfeld, Pattie Maes

Abstract

The integration of artificial intelligence (AI) into the design and fabrication process has opened up novel pathways for producing custom objects and altered the traditional creative workflow. In this article, we present Depthfusion, a novel text-to-3D model generation system that empowers users to rapidly create detailed 3D models from textual or 2D image inputs, and explore the application of text-to-3D models within different fabrication techniques. Depthfusion leverages current text-to-image AI technologies such as Midjourney, Stable Diffusion, and DALL-E and integrates them with advanced mesh inflation and depth mapping techniques. This approach yields a high degree of artistic control and facilitates the production of high-resolution models that are compatible with various 3D printing methods. Our results include a biomimetic tableware set that merges intricate design with functionality, a large-scale ceramic vase illustrating the potential for additive manufacturing in ceramics, and even a sneaker-shaped bread product achieved by converting AI design into a baked form. These projects showcase the diverse possibilities for AI in the design and crafting of objects across mediums, pushing the boundaries of what is traditionally considered feasible in bespoke manufacturing.
Source: openalex_inverted_index · Quality: baseline

Research context

OpenAlex topics: Design Education and Practice · Architecture and Computational Design · 3D Shape Modeling and Analysis

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

MIT Media Lab, Massachusetts Institute of Technology, Cambridge, Massachusetts, USA.

Department of Experimental Architecture, Innsbruck University, Innsbruck, Austria.

Center for Bits and Atoms, Massachusetts Institute of Technology, Cambridge, Massachusetts, USA.

References (31)

Show source-provided references
1. 1. Ramesh A, Pavlov M, Goh G, et al. Zero-Shot Text-to-Image Generation. In: International conference on machine learning PMLR; 2021;8821–8831.
2. Ramesh A · arXiv Preprint · 2022
3. 10.1109/CVPR52688.2022.0104210.1109/CVPR52688.2022.01042 ↗
4. 10.1007/s43503-023-00018-y10.1007/s43503-023-00018-y ↗
5. 10.1109/CVPR52688.2022.0009410.1109/CVPR52688.2022.00094 ↗
6. Poole B · arXiv Preprint · 2022
7. Wang Z · arXiv Preprint · 2023
8. 8. Lin C-H, Gao J, Tang L, et al. Magic3d: High-Resolution Text-to-3D Content Creation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition 2023; pp. 300–309.10.1109/CVPR52729.2023.00037 ↗
9. Ho J · J Mach Learn Res · 2022 · 23 · 2249
10. Nichol A · arXiv Preprint · 2021
11. 11. Kim G, Kwon T, Ye JC. Diffusionclip: Text-Guided Diffusion Models for Robust Image Manipulation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition 2022; pp. 2426–2435.10.1109/CVPR52688.2022.00246 ↗
12. Saharia C · Adv Neural Inform Proces Syst · 2022 · 35 · 36479
13. Hakimshafaei M · 2023
14. Nichol A · arXiv Preprint · 2022
15. 15. Liu R, Wu R, Van Hoorick B, et al. Zero-1-to-3: Zero-Shot One Image to 3D Object. In: Proceedings of the IEEE/CVF International Conference on Computer Vision 2023; pp. 9298–9309.10.1109/ICCV51070.2023.00853 ↗
16. Yi T · arXiv Preprint · 2023
17. Gatis D · 2024
18. 10.1145/3414685.341780510.1145/3414685.3417805 ↗
19. 10.1177/0954406222107950610.1177/09544062221079506 ↗
20. 10.1111/jace.1370010.1111/jace.13700 ↗
21. Loos A · 2014
22. Semper G · 2004
23. Riegl A · 2018
24. Sullivan L · 2010
25. Stap D · 163
26. Palmer A · Colo Nat Resources Energy Env’t L Rev · 2023 · 34 · 77
27. Lipson H · 2013
28. Epstein Z · arXiv Preprint · 2022
29. 10.1016/j.jclepro.2013.01.01410.1016/j.jclepro.2013.01.014 ↗
30. 10.1016/j.spc.2022.11.00910.1016/j.spc.2022.11.009 ↗
31. 10.1007/s11356-020-10657-810.1007/s11356-020-10657-8 ↗