FG-Text-SD: Training-Free Controllable Scene Text–Image Generation for Low-Resource Southeast Asian Languages
Source abstract
Abstract
Scene text–image generation aims to synthesize natural images containing readable and visually coherent text. Although recent diffusion-based methods have shown promising results, they often struggle with low-resource Southeast Asian languages because of complex glyph structures, limited language resources, and weak alignment between generated text and background carriers. To address this issue, we propose FG-Text-SD, a training-free controllable scene text–image generation framework built on Stable Diffusion. The proposed framework organizes multiple text instances in an instance-level manner, injects rendered glyph structure priors into the denoising process to stabilize complex character shapes, modulates cross-attention with carrier-aware masks to improve text-to-surface alignment, and employs OCR-guided local repainting to correct residual local errors. Experiments are conducted on AnyText-benchmark, CVTG-2K, and a newly constructed evaluation set covering Thai, Lao, Khmer, and Burmese. The proposed method achieves strong performance on both public benchmarks and low-resource language subsets, improving text accuracy, readability, and spatial consistency without additional model retraining. These results demonstrate that FG-Text-SD provides an effective solution for controllable scene text–image generation in low-resource multilingual settings.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Generative Adversarial Networks and Image Synthesis · Multimodal Machine Learning Applications · Computer Graphics and Visualization Techniques
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Qingdao Institute of Software, College of Computer Science and Technology, China University of Petroleum (East China), Qingdao 266580, China
Shandong Key Laboratory of Intelligent Oil & Gas Industrial Software, Qingdao 266580, China
Funding
Natural Science Foundation of Shandong Province · National Natural Science Foundation of China · Qingdao Natural Science Foundation
References (38)
Show source-provided references
1. Gupta, A., Vedaldi, A., and Zisserman, A. (2016, January 27–30). Synthetic Data for Text Localisation in Natural Images. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.10.1109/CVPR.2016.254 ↗
2. Wu, L., Zhang, C., Liu, J., Han, J., Liu, J., Ding, E., and Bai, X. (2019, January 21–25). Editing text in the wild. Proceedings of the 27th ACM International Conference on Multimedia (MM ’19), Nice, France.10.1145/3343031.3350929 ↗
3. Long, S., He, X., and Yao, C. (2020, January 13–19). UnrealText: Synthesizing realistic scene text images from the unreal world. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.
4. Rombach, R., Blattmann, A., Lorenz, D., Esser, P., and Ommer, B. (2022, January 18–24). High-resolution image synthesis with latent diffusion models. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA.10.1109/CVPR52688.2022.01042 ↗
5. Denoising diffusion probabilistic models
6. Tuo, Y., Wang, W., and Gao, W. (2024, January 7–11). AnyText: Multilingual visual text generation and editing. Proceedings of the International Conference on Learning Representations (ICLR), Vienna, Austria.
7. Ma, J., Zhao, L., Chen, H., Wang, J., Qiao, Y., and Dong, C. (2023). GlyphDraw: Learning to draw Chinese characters in image synthesis models coherently. arXiv.
8. TextDiffuser: Diffusion models as text painters
9. Nayef, N., Patel, Y., Busta, M., Chowdhury, P.N., Karatzas, D., Khot, W., Lim, J.P., Umapada, P., Rigaud, C., and Bouillon, M. (2019, January 20–25). ICDAR2019 robust reading challenge on multi-lingual scene text detection and recognition—ICDAR-MLT-2019. Proceedings of the International Conference on Document Analysis and Recognition (ICDAR), Sydney, Australia.10.1109/ICDAR.2019.00254 ↗
10. Ramesh, A., Dhariwal, P., Nichol, A., Chu, C., and Chen, M. (2022). Hierarchical text-conditional image generation with CLIP latents. arXiv.
11. Photorealistic text-to-image diffusion models with deep language understanding
12. Podell, D., English, Z., Lacey, K., Blattmann, A., Dockhorn, T., Müller, J., Penna, J., and Rombach, R. (2024, January 7–11). SDXL: Improving latent diffusion models for high-resolution image synthesis. Proceedings of the International Conference on Learning Representations (ICLR), Vienna, Austria.
13. Esser, P., Kulal, S., Blattmann, A., Entezari, R., Müller, J., Saini, H., Levi, Y., Fasoli, D., Nizan, O., and Deckers, J. (2024, January 21–27). Scaling rectified flow transformers for high-resolution image synthesis. Proceedings of the International Conference on Machine Learning (ICML), Vienna, Austria.
14. (2025, July 10). Black Forest Labs. FLUX. Available online: https://github.com/black-forest-labs/flux.
15. Zhang, L., Rao, A., and Agrawala, M. (2023, January 2–6). Adding conditional control to text-to-image diffusion models. Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), Paris, France.10.1109/ICCV51070.2023.00355 ↗
16. Li, Y., Liu, H., Wu, Q., Mu, F., Yang, J., Gao, J., and Lee, Y.J. (2023, January 18–22). GLIGEN: Open-set grounded text-to-image generation. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Vancouver, BC, Canada.10.1109/CVPR52729.2023.02156 ↗
17. Hertz, A., Mokady, R., Tenenbaum, J., Aberman, K., Pritch, Y., and Cohen-Or, D. (2023, January 1–5). Prompt-to-prompt image editing with cross attention control. Proceedings of the International Conference on Learning Representations (ICLR), Kigali, Rwanda.
18. Chen, J., Huang, Y., Lv, T., Cui, L., Chen, Q., and Wei, F. (2024, January 4–8). Training-free layout control with cross-attention guidance. Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), Waikoloa, HI, USA.10.1109/WACV57701.2024.00526 ↗
19. SynthText3D: Synthesizing scene text images from 3D virtual worlds10.1007/s11432-019-2737-0 ↗
20. ABCNet v2: Adaptive bezier-curve network for real-time end-to-end text spotting10.1109/TITS.2021.3075225 ↗
21. Few-shot font generation by learning content and style representations
22. GlyphControl: Glyph conditional control for visual text generation
23. Chen, J., Huang, Y., Lv, T., Cui, L., Chen, Q., and Wei, F. (October, January 29). TextDiffuser-2: Unleashing the power of language models for text rendering. Proceedings of the European Conference on Computer Vision (ECCV), Milan, Italy.
24. Zhao, M., and Lian, J. (2023). UDiffText: A unified framework for high-quality text synthesis in arbitrary images via character-aware diffusion models. arXiv.10.1007/978-3-031-72751-1_13 ↗
25. Zhang, L., Chen, X., Wang, Y., Lu, Y., and Qiao, Y. (2023). Brush your text: Synthesize any scene text on images via diffusion model. arXiv.10.1609/aaai.v38i7.28550 ↗
26. Tuo, Y., Geng, Y., and Bo, L. (2024). AnyText2: A unified vision-language model for text generation and understanding. arXiv.
27. Liu, Z., Li, Y., Luo, Y., Wang, H., and Jin, L. (2024). Glyph-ByT5-v2: A strong language model for multilingual visual text generation. arXiv.
28. Li, C., Jiang, C., Liu, X., Zhao, J., and Wang, G. (2024). JoyType: A robust design for multilingual visual text creation. arXiv.
29. Kingma, D.P., and Welling, M. (2014, January 14–16). Auto-encoding variational Bayes. Proceedings of the International Conference on Learning Representations (ICLR), Banff, AB, Canada.
30. Huang, X., and Belongie, S. (2017, January 22–29). Arbitrary style transfer in real-time with adaptive instance normalization. Proceedings of the IEEE International Conference on Computer Vision (ICCV), Venice, Italy.10.1109/ICCV.2017.167 ↗
31. Ronneberger, O., Fischer, P., and Brox, T. (2015, January 5–9). U-Net: Convolutional networks for biomedical image segmentation. Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI), Munich, Germany.10.1007/978-3-319-24574-4_28 ↗
32. Du, Y., Li, C., Guo, R., Yin, X., Wei, W., Li, L., Zhang, K., Zhou, X., Huan, J., and Wen, L. (2020). PP-OCR: A practical ultra lightweight OCR system. arXiv.
33. Wei, H., Sun, Y., and Li, Y. (2025). DeepSeek-OCR: Contexts optical compression. arXiv.
34. Du, N., Chen, Z., Chen, Z., Jiang, Z., and Tai, Y. (2025). TextCrafter: Accurately rendering multiple texts in complex visual scenes. arXiv.
35. Goldhahn, D., Eckart, T., and Quasthoff, U. (2012, January 21–27). Building large monolingual dictionaries at the Leipzig Corpora Collection: From 100 to 200 languages. Proceedings of the International Conference on Language Resources and Evaluation (LREC), Istanbul, Turkey.
36. Ortiz Suárez, P.J., Romary, L., and Sagot, B. (2020, January 5–10). A monolingual approach to contextualized word embeddings for mid-resource languages. Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL), Online.10.18653/v1/2020.acl-main.156 ↗
37. Hessel, J., Holtzman, A., Forbes, M., Bras, R.L., and Choi, Y. (2021, January 7–11). CLIPScore: A reference-free evaluation metric for image captioning. Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP), Punta Cana, Dominican Republic.10.18653/v1/2021.emnlp-main.595 ↗
38. Lu, R., Zhang, Y., Liu, J.-M., Wang, H., and Song, Y. (2025). EasyText: Controllable diffusion transformer for multilingual text rendering. arXiv.10.1609/aaai.v40i9.37697 ↗