Image Style Transfer for Exhibition Hall Design Based on Multimodal Semantic-Enhanced Algorithm
Source abstract
Abstract
Although existing style transfer techniques have made significant progress in the field of image generation, there are still some challenges in the field of exhibition hall design. The existing style transfer methods mainly focus on the transformation of single dimensional features, but ignore the deep integration of content and style features in exhibition hall design. In addition, existing methods are deficient in detail retention, especially in accurately capturing and reproducing local textures and details while preserving the content image structure. In addition, point-based attention mechanisms tend to ignore the complexity and diversity of image features in multi-dimensional space, resulting in alignment problems between features in different semantic areas, resulting in inconsistent stylistic features in content areas. In this context, this paper proposes a semantic-enhanced multimodal style transfer algorithm tailored for exhibition hall design. The proposed approach leverages a multimodal encoder architecture to integrate information from text, source images, and style images, using separate encoder modules for each modality to capture shallow, deep, and semantic features. A novel Style Transfer Convolution (STConv) convolutional kernel, based on the Visual Geometry Group (VGG) 19 network, is introduced to improve feature extraction in style transfer. Additionally, an enhanced Transformer encoder is incorporated to capture contextual semantic information within images, while the CLIP model is employed for text data processing. A hybrid attention module is designed to precisely capture style features, achieving multimodal feature fusion via a diffusion model that generates exhibition hall design images aligned with stylistic requirements. Quantitative experiments show that compared with the most advanced algorithms, the proposed method has achieved significant performance improvement on both Fréchet Inception Distance (FID) and Kernel Inception Distance (KID) indexes. For example, on the ExpoArchive dataset, the proposed method has a FID value of 87.9 and a KID value of 1.98, which is significantly superior to other methods.
Source: openalex_inverted_index · Quality: baseline
Research context
OpenAlex topics: Digital Media and Visual Art · 3D Surveying and Cultural Heritage
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
References (39)
Show source-provided references
1. Image style transfer using convolutional neural networks
2. Yeh MC, Tang S, Bhattad A, Forsyth DA, Yeh MC, Tang S, et al. Quantitative evaluation of style transfer. arXiv: 1804.00118. 2018.
3. A novel fuzzy neural network architecture search framework for defect recognition with uncertainties10.1109/TFUZZ.2024.3373792 ↗
4. Liu S, Lin T, He D, Li F, Wang M, Li X, et al. AdaAttN: revisit attention mechanism in arbitrary neural style transfer. arXiv:2108.03647. 2021.10.1109/ICCV48922.2021.00658 ↗
5. Luo X, Han Z, Yang L, Zhang L. Consistent style transfer. arXiv:2201.02233. 2022.
6. Texture synthesis by non-parametric sampling
7. Gooch · Non-photorealistic rendering · 200110.1201/9781439864173 ↗
8. Style transfer via texture synthesis10.1109/TIP.2017.2678168 ↗
9. Mordvintsev A, Olah C, Tyka M. Inceptionism: going deeper into neural networks. [cited 2025 Jan 1]. Available from: https://research.google/blog/inceptionism-going-deeper-into-neural-networks/.
10. Combining Markov random fields and convolutional neural networks for image synthesis
11. Radford A, Metz L, Chintala S, Dinakaran R, Easom P, Zhang L, et al. Unsupervised representation learning with deep convolutional generative adversarial networks. arXiv:1511.06434. 2015.
12. Wasserstein generative adversarial networks
13. A style-based generator architecture for generative adversarial networks
14. Attention-aware multi-stroke style transfer
15. StyTr2: image style transfer with transformers
16. Delbracio M, Milanfar P. Inversion by direct iteration: an alternative to denoising diffusion for image restoration. arXiv:2303.11435. 2023.
17. Denoising diffusion probabilistic models
18. Rombach R, Blattmann A, Lorenz D, Esser P, Ommer B, Mantri KSI, et al. High-resolution image synthesis with latent diffusion models. arXiv:2112.10752. 2021.10.1109/CVPR52688.2022.01042 ↗
19. Zhang X, Song Y, Song T, Yang D, Ye Y, Zhou J, et al. AKConv: convolutional kernel with arbitrary sampled shapes and arbitrary number of parameters. arXiv:2311.11587. 2023.
20. Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, et al. Attention is all you need. arXiv:1706.03762. 2017.
21. SeaFormer: squeeze-enhanced axial transformer for mobile semantic segmentation
22. Swin transformer: hierarchical vision transformer using shifted windows
23. CCNet: crisscross attention for semantic segmentation
24. Ho J, Kalchbrenner N, Weissenborn D, Salimans T. Axial attention in multidimensional transformers. arXiv:1912.12180. 2019.
25. Summer2Winter Yosemite: CycleGAN's Summer Winter Images Dataset at Yosemite.[cited 2025 Jan 1].Available from: https://www.kaggle.com/datasets/balraj98/summer2winter-yosemite.
26. The Cityscapes Dataset: 5 000 images with high quality annotations • 20 000 images with coarse annotations • 50 different cities.[cited 2025 Jan 1].Available from: https://www.cityscapes-dataset.com/.
27. Larger Google Sat2Map dataset.[cited 2025 Jan 1].Available from: https://github.com/taesungp/larger-google-sat2maps-dataset.
28. Heusel M, Ramsauer H, Unterthiner T, Nessler B, Hochreiter S. GANs trained by a two time-scale update rule converge to a local Nash equilibrium. arXiv:1706.08500. 2017.
29. Reusing discriminators for encoding: towards unsupervised image-to-image translation
30. Neural optimal transport
31. Unpaired image-to-image translation using cycle-consistent adversarial networks
32. Multimodal unsupervised image-to-image translation
33. Benaim S, Wolf L. One-sided unsupervised domain mapping. arXiv:1706.00826. 2017.
34. Geometry-consistent generative adversarial networks for one-sided unsupervised domain mapping
35. Contrastive learning for unpaired image-to-image translation
36. SDEdit: guided image synthesis and editing with stochastic differential equations
37. Hertz A, Mokady R, Tenenbaum J, Aberman K, Pritch Y, Cohen-Or D. Prompt-to-prompt image editing with cross attention control. arXiv:2208.01626. 2022.
38. Unpaired image-to-image translation via neural schrödinger bridge
39. Katherine C. Clip-guided diffusion. [cited 2025 Jan 1]. Available from: https://github.com/afiaka87/clip-guided-diffusion.