Unsupervised Scene Image Text Segmentation Based on Improved CycleGAN
Source abstract
Abstract
Scene image text segmentation is an important task in computer vision, but the complexity and diversity of backgrounds make it challenging. All supervised image segmentation tasks require paired semantic label data to ensure the accuracy of segmentation, but semantic labels are often difficult to obtain. To solve this problem, we propose an unsupervised scene image text segmentation model based on the image style transfer model cyclic uniform Generation Adversarial network (CycleGAN), which is trained by partial unpaired label data. Text segmentation is achieved by converting a complex background to a simple background. Since the images generated by CycleGAN cannot retain the details of the text content, we also introduced the Atrous spatial Pyramid pool module (ASPP) to obtain the features of the text from multiple scales. The resulting image quality is improved. The proposed method is verified by experiments on a synthetic data set, the IIIT 5k word data set and the MACT data set, which effectively segments the text and preserves the details of the text content.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Handwritten Text Recognition Techniques · Image Processing and 3D Reconstruction · Image Retrieval and Classification Techniques
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
School of Cyberspace Security and Computer, Hebei University, Baoding 071000, China
Institute of Intelligence Image and Document Information Processing, Hebei University, Baoding 071000, China
References (29)
Show source-provided references
1. Jiao, S., Li, X., and Lu, X. (2006, January 16–20). An improved Ostu method for image segmentation. Proceedings of the 2006 8th international Conference on Signal Processing, Guilin, China.10.1109/ICOSP.2006.345705 ↗
2. On the rationale of maximum-entropy methods10.1109/PROC.1982.12425 ↗
3. Hageman, L.A., and Young, D.M. (2012). Applied Iterative Methods, Courier Corporation.
4. Super-resolution methods in MRI: Can they improve the trade-off between resolution, signal-to-noise ratio, and acquisition time?10.1002/mrm.24187 ↗
5. Comparison of Niblack inspired binarization methods for ancient documents
6. Kia, O.E., and Sanvola, J. (1998, January 7–9). Active multimedia documents for mobile services. Proceedings of the 1998 IEEE Second Workshop on Multimedia Signal Processing (Cat. No. 98EX175), Redondo Beach, CA, USA.
7. Ye, Q., Gao, W., and Huang, Q. (2004, January 24–27). Automatic text segmentation from complex background. Proceedings of the 2004 International Conference on Image Processing, ICIP’04, Singapore.
8. Mishra, A., Alahari, K., and Jawahar, C. (2011, January 18–21). An MRF model for binarization of natural scene text. Proceedings of the 2011 International Conference on Document Analysis and Recognition, Beijing, China.10.1109/ICDAR.2011.12 ↗
9. Integrating multiple character proposals for robust scene text extraction10.1016/j.imavis.2013.08.007 ↗
10. Mishra, A., Alahari, K., and Jawahar, C. (2012, January 16–21). Top-down and bottom-up cues for scene text recognition. Proceedings of the 2012 IEEE Conference on Computer Vision and Pattern Recognition, Providence, RI, USA.10.1109/CVPR.2012.6247990 ↗
11. Color text extraction with selective metric-based clustering10.1016/j.cviu.2006.11.010 ↗
12. Kita, K., and Wakahara, T. (2010, January 23–26). Binarization of color characters in scene images using k-means clustering and support vector machines. Proceedings of the 2010 20th International Conference on Pattern Recognition, Istanbul, Turkey.10.1109/ICPR.2010.779 ↗
13. Zhu, Y., Sun, J., and Naoi, S. (2011, January 22). Recognizing natural scene characters by convolutional neural network and bimodal image enhancement. Proceedings of the Camera-Based Document Analysis and Recognition: 4th International Workshop, CBDAR 2011, Beijing, China.10.1007/978-3-642-29364-1_6 ↗
14. Zhang, Y., Qiu, Z., Yao, T., Liu, D., and Mei, T. (1997, January 17–19). Fully convolutional adaptation networks for semantic segmentation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, San Juan, PR, USA.
15. Hoffman, J., Tzeng, E., Park, T., Zhu, J.Y., Isola, P., Saenko, K., Efros, A., and Darrell, T. (2018, January 10–15). Cycada: Cycle-consistent adversarial domain adaptation. Proceedings of the International Conference on Machine Learning, Stockholm, Sweden.
16. Chen, Y.C., Lin, Y.Y., Yang, M.H., and Huang, J.B. (2019, January 15–20). Crdoco: Pixel-level domain transfer with cross-domain consistency. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.10.1109/CVPR.2019.00189 ↗
17. Zhu, J.Y., Park, T., Isola, P., and Efros, A.A. (2017, January 22–29). Unpaired image-to-image translation using cycle-consistent adversarial networks. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.10.1109/ICCV.2017.244 ↗
18. Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs10.1109/TPAMI.2017.2699184 ↗
19. Weak supervision for generating pixel–level annotations in scene text segmentation10.1016/j.patrec.2020.06.023 ↗
20. Veit, A., Matera, T., Neumann, L., Matas, J., and Belongie, S. (2016). Coco-text: Dataset and benchmark for text detection and recognition in natural images. arXiv.
21. Nayef, N., Yin, F., Bizid, I., Choi, H., Feng, Y., Karatzas, D., Luo, Z., Pal, U., Rigaud, C., and Chazalon, J. (2017, January 9–15). Icdar2017 robust reading challenge on multi-lingual scene text detection and script identification-RRC-MLT. Proceedings of the 2017 14th IAPR International Conference on Document Analysis and Recognition (ICDAR), Kyoto, Japan.10.1109/ICDAR.2017.237 ↗
22. Semi-supervised pixel-level scene text segmentation by mutually guided network10.1109/TIP.2021.3113157 ↗
23. Hertzmann, A., Jacobs, C.E., Oliver, N., Curless, B., and Salesin, D.H. (2001, January 12–17). Image analogies. Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques, Los Angeles, CA, USA.10.1145/383259.383295 ↗
24. Isola, P., Zhu, J.Y., Zhou, T., and Efros, A.A. (2017, January 21–26). Image-to-image translation with conditional adversarial networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.10.1109/CVPR.2017.632 ↗
25. Zhu, J.Y., Zhang, R., Pathak, D., Darrell, T., Efros, A.A., Wang, O., and Shechtman, E. (2017). Toward multimodal image-to-image translation. Adv. Neural Inf. Process. Syst., 30.
26. Wang, T.C., Liu, M.Y., Zhu, J.Y., Tao, A., Kautz, J., and Catanzaro, B. (2018, January 18–22). High-resolution image synthesis and semantic manipulation with conditional gans. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00917 ↗
27. Shaohui, L. (2024, May 19). A Synthetic Data Generator for Text Recognition. Available online: https://gitcode.net/mirrors/Belval/TextRecognitionDataGenerator.git.
28. Mishra, A., Alahari, K., and Jawahar, C. (2012, January 3–7). Scene text recognition using higher order language priors. Proceedings of the BMVC-British Machine Vision Conference, BMVA, Surrey, UK.10.5244/C.26.127 ↗
29. Multi-scene ancient Chinese text recognition with deep coupled alignments10.1016/j.asoc.2021.107475 ↗