A unified framework of a multi-modal, property-aware diffusion model for 3D reconstruction and on-demand design

3D structures provide critical insights into material behavior, essential for deciphering structure-property-response relationships. In practice, limited 2D data are available, making 3D reconstruction and on-demand design a highly ill-posed, under-constrained problem. Existing methods are limited by model accuracy, computational cost, and memory overhead, hindering applicability in complex design tasks. This study develops an end-to-end multi-modal, property-aware diffusion (MPaDiffusion) model, unifying 3D structure generation and on-demand design. This model incorporates 2D image slices, segmentation masks, and stress-strain data to learn structure-property relationships. The interactive segmentation algorithm extracts mask information as a multi-modal input. Combining mean-squared error loss, Kullback-Leibler divergence, and physics-consistency loss ensures geometrical fidelity and accurate mechanical response. The anisotropy index evaluates the anisotropic degree of the generated structures, and the statistical evaluation system assesses the generation fidelity. The results show that MPaDiffusion generates high-fidelity 3D structures and reproduces material mechanical properties. Efficiency and generalizability support material performance prediction tasks, shedding valuable light on 3D reconstruction and on-demand design.