Patch-Font: Enhancing Few-Shot Font Generation with Patch-Based Attention and Multitask Encoding
Source abstract
Abstract
Few-shot font generation seeks to create high-quality fonts using minimal reference style images, addressing traditional font design’s labor-intensive and time-consuming nature, particularly for languages with large character sets like Chinese and Korean. Existing methods often require multi-stage training or predefined components, which can be time-consuming and limit generalizability. This paper introduces Patch-Font, a novel single-stage method that overcomes the limitations of prior approaches, such as multi-stage training or reliance on predefined components, by integrating a patch-based attention mechanism and a multitask encoder. Patch-Font jointly captures global style elements (e.g., overall font family characteristics) and local style details (e.g., serifs, stroke shapes), ensuring high fidelity to the target style while maintaining computational efficiency. Our approach incorporates triplet margin loss with hard positive/negative mining to disentangle style from content and a style fidelity loss to enhance local style consistency. Experiments on Korean (printed and handwritten) and Chinese fonts demonstrate that Patch-Font outperforms state-of-the-art methods in style accuracy, perceptual quality, and generation speed while generalizing robustly to unseen characters and font styles. By simplifying the font creation process and delivering high-quality results, Patch-Font represents a significant step forward in making font design more accessible and scalable for diverse languages.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Generative Adversarial Networks and Image Synthesis · Computer Graphics and Visualization Techniques · Video Analysis and Summarization
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
School of Computer Science and Engineering, Soongsil University, 369 Sangdo-Ro, Seoul 06978, Republic of Korea
References (33)
Show source-provided references
1. Park, S., Chun, S., Cha, J., Lee, B., and Shim, H. (2009). Few-shot font generation with localized style representations and factorization. arXiv.
2. Park, S., Chun, S., Cha, J., Lee, B., and Shim, H. (2021, January 11–17). Multiple heads are better than one: Few-shot font generation with multiple localized experts. Proceedings of the IEEE/CVF International Conference on Computer Vision, Montreal, BC, Canada.10.1109/ICCV48922.2021.01364 ↗
3. Pan, W., Zhu, A., Zhou, X., Iwana, B.K., and Li, S. (2023, January 1–6). Few Shot Font Generation Via Transferring Similarity Guided Global Style and Quantization Local Style. Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), Paris, France.10.1109/ICCV51070.2023.01787 ↗
4. Cha, J., Chun, S., Lee, G., Lee, B., Kim, S., and Lee, H. (2020, January 23–28). Few-shot compositional font generation with dual memory. Proceedings of the Computer Vision—ECCV 2020: 16th European Conference, Glasgow, UK. Proceedings, Part XIX 16.10.1007/978-3-030-58529-7_43 ↗
5. Real-time high quality font generation with Conditional Font GAN10.1016/j.eswa.2022.118907 ↗
6. Exploiting mixing regularization for truly unsupervised font synthesis10.1016/j.patrec.2023.03.019 ↗
7. Learning font-style space using style-guided discriminator for few-shot font generation10.1016/j.eswa.2023.122817 ↗
8. Li, C., Taniguchi, Y., Lu, M., and Konomi, S. (2021, January 11–17). Few-shot font style transfer between different languages. Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, Virtual.10.1109/WACV48630.2021.00048 ↗
9. ImageNet Classification with Deep Convolutional Neural Networks
10. Simonyan, K., and Zisserman, A. (2014). Very deep convolutional networks for large-scale image recognition. arXiv.
11. He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27–30). Deep residual learning for image recognition. Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.10.1109/CVPR.2016.90 ↗
12. Generative adversarial networks10.1145/3422622 ↗
13. Radford, A., Metz, L., and Chintala, S. (2015). Unsupervised representation learning with deep convolutional generative adversarial networks. arXiv.
14. Arjovsky, M., Chintala, S., and Bottou, L. (2017, January 6–11). Wasserstein Generative Adversarial Networks. Proceedings of the 34th International Conference on Machine Learning, Sydney, Australia.
15. Zhu, J.Y., Park, T., Isola, P., and Efros, A.A. (2017, January 22–29). Unpaired image-to-image translation using cycle-consistent adversarial networks. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.10.1109/ICCV.2017.244 ↗
16. Isola, P., Zhu, J.Y., Zhou, T., and Efros, A.A. (2017, January 21–26). Image-to-image translation with conditional adversarial networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.10.1109/CVPR.2017.632 ↗
17. Baek, K., Choi, Y., Uh, Y., Yoo, J., and Shim, H. (2021, January 11–17). Rethinking the truly unsupervised image-to-image translation. Proceedings of the IEEE/CVF International Conference on Computer Vision, Montreal, BC, Canada.10.1109/ICCV48922.2021.01389 ↗
18. Saito, K., Saenko, K., and Liu, M.Y. (2020, January 23–28). Coco-funit: Few-shot unsupervised image translation with a content conditioned style encoder. Proceedings of the European Conference on Computer Vision, Glasgow, UK.10.1007/978-3-030-58580-8_23 ↗
19. Zhu, J.Y., Zhang, R., Pathak, D., Darrell, T., Efros, A.A., Wang, O., and Shechtman, E. (2017, January 4–9). Toward multimodal image-to-image translation. Proceedings of the Advances in Neural Information Processing Systems, Long Beach, CA, USA.
20. Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., and Gelly, S. (2020). An image is worth 16 × 16 words: Transformers for image recognition at scale. arXiv.
21. Lee, K., Chang, H., Jiang, L., Zhang, H., Tu, Z., and Liu, C. (2021). Vitgan: Training gans with vision transformers. arXiv.
22. Liu, W., Liu, F., Ding, F., He, Q., and Yi, Z. (2022, January 18–24). Xmp-font: Self-supervised cross-modality pre-training for few-shot font generation. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.10.1109/CVPR52688.2022.00775 ↗
23. Tang, L., Cai, Y., Liu, J., Hong, Z., Gong, M., Fan, M., Han, J., Liu, J., Ding, E., and Wang, J. (2022, January 18–24). Few-Shot Font Generation by Learning Fine-Grained Local Styles. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA.10.1109/CVPR52688.2022.00774 ↗
24. Huang, X., and Belongie, S. (2017, January 22–29). Arbitrary style transfer in real-time with adaptive instance normalization. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.10.1109/ICCV.2017.167 ↗
25. A Style-Based Generator Architecture for Generative Adversarial Networks10.1109/TPAMI.2020.2970919 ↗
26. Wang, C., Zhou, M., Ge, T., Jiang, Y., Bao, H., and Xu, W. (2023, January 17–24). CF-Font: Content Fusion for Few-Shot Font Generation. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Vancouver, BC, Canada.10.1109/CVPR52729.2023.00185 ↗
27. Schroff, F., Kalenichenko, D., and Philbin, J. (2015, January 7–12). FaceNet: A Unified Embedding for Face Recognition and Clustering. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA.10.1109/CVPR.2015.7298682 ↗
28. Hermans, A., Beyer, L., and Leibe, B. (2017). In Defense of the Triplet Loss for Person Re-Identification. arXiv.
29. Huang, X., Liu, M.Y., Belongie, S., and Kautz, J. (2018, January 8–14). Multimodal unsupervised image-to-image translation. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.10.1007/978-3-030-01219-9_11 ↗
30. Zhang, R., Isola, P., Efros, A.A., Shechtman, E., and Wang, O. (2018, January 18–22). The unreasonable effectiveness of deep features as a perceptual metric. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00068 ↗
31. Image quality assessment: From error visibility to structural similarity10.1109/TIP.2003.819861 ↗
32. Gans trained by a two time-scale update rule converge to a local nash equilibrium
33. Liu, M.Y., Huang, X., Mallya, A., Karras, T., Aila, T., Lehtinen, J., and Kautz, J. (November, January 27). Few-shot unsupervised image-to-image translation. Proceedings of the IEEE/CVF International Conference on Computer Vision, Seoul, Republic of Korea.