Text Removal for Trademark Images Based on Self-Prompting Mechanisms and Multi-Scale Texture Aggregation
Source abstract
Abstract
With the rapid development of electronic business, there has been a surge in incidents of trademark infringement, making it imperative to improve the accuracy of trademark retrieval systems as a key measure to combat such illegal behaviors. Evidently, the textual information encompassed within trademarks substantially influences the precision of search results. Considering the diversity of trademark text and the complexity of its design elements, accurately locating and analyzing this text poses a considerable challenge. Against this background, this research has developed an original self-prompting text removal model, denoted as “Self-prompting Trademark Text Removal Based on Multi-scale Texture Aggregation” (abbreviated as MTF-STTR). This model astutely applies a text detection network to automatically generate the required input cues for the Segment Anything Model (SAM) while incorporating the technological benefits of diffusion models to attain a finer level of trademark text removal. To further elevate the performance of the model, we introduce two innovative architectures to the text detection network: the Integrated Differentiating Feature Pyramid (IDFP) and the Texture Fusion Module (TFM). These mechanisms are capable of efficiently extracting multilevel features and multiscale textual information, which enhances the model’s stability and adaptability in complex scenarios. The experimental validation has demonstrated that the trademark text erasure model designed in this paper achieves a peak signal-to-noise ratio as high as 40.1 dB on the SCUT-Syn dataset, which is an average improvement of 11.3 dB compared with other text erasure models. Furthermore, the text detection network component of the designed model attains an accuracy of up to 89.9% on the CTW1500 dataset, representing an average enhancement of 10 percentage points over other text detection networks.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Handwritten Text Recognition Techniques · Image Processing and 3D Reconstruction · Advanced Steganography and Watermarking Techniques
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Computer Science and Technology Department, China Jiliang University, Hangzhou 310018, China
Funding
National Key Research and Development Program of China
References (33)
Show source-provided references
1. Feng, Y., Shi, C., Qi, C., Xu, J., Xiao, B., and Wang, C. (2018, January 14). Aggregation of reversal invariant features from edge images for large-scale trademark retrieval. Proceedings of the 2018 4th International Conference on Control, Automation and Robotics (ICCAR), Auckland, New Zealand.10.1109/ICCAR.2018.8384705 ↗
2. Component-based attention for large-scale trademark retrieval10.1109/TIFS.2019.2959921 ↗
3. Near-duplicate image detection in a visually salient riemannian space10.1109/TIFS.2012.2206386 ↗
4. Curved scene text detection via transverse and longitudinal sequence connection10.1016/j.patcog.2019.02.002 ↗
5. Tursun, O., Aker, C., and Kalkan, S. (2017). A large-scale dataset and benchmark for similar trademark retrieval. arXiv.
6. Kirillov, A., Mintun, E., Ravi, N., Mao, H., Rolland, C., Gustafson, L., Xiao, T., Whitehead, S., Berg, A.C., and Lo, W.Y. (2023, January 15). Segment anything. Proceedings of the IEEE/CVF International Conference on Computer Vision, Paris, France.10.1109/ICCV51070.2023.00371 ↗
7. Carreira-Perpinán, M.A. (2008, January 23–28). Generalised blurring mean-shift algorithms for nonparametric clustering. Proceedings of the 2008 IEEE Conference on Computer Vision and Pattern Recognition, Anchorage, AK, USA.10.1109/CVPR.2008.4587435 ↗
8. Shi, J., and Qi, C. (2013, January 15–18). Sparse modeling based image inpainting with local similarity constraint. Proceedings of the 2013 IEEE International Conference on Image Processing, Melbourne, Australia.10.1109/ICIP.2013.6738282 ↗
9. Nakamura, T., Zhu, A., Yanai, K., and Uchida, S. (2017, January 9–15). Scene text eraser. Proceedings of the 2017 14th IAPR International Conference on Document Analysis and Recognition (ICDAR), Kyoto, Japan.10.1109/ICDAR.2017.141 ↗
10. Erasenet: End-to-end text removal in the wild10.1109/TIP.2020.3018859 ↗
11. Tursun, O., Zeng, R., Denman, S., Sivapalan, S., Sridharan, S., and Fookes, C. (2019, January 20–25). Mtrnet: A generic scene text eraser. Proceedings of the 2019 International Conference on Document Analysis and Recognition (ICDAR), Sydney, Australia.10.1109/ICDAR.2019.00016 ↗
12. MTRNet++: One-stage mask-based scene text eraser10.1016/j.cviu.2020.103066 ↗
13. Zdenek, J., and Nakayama, H. (2020, January 13–19). Erasing scene text with weak supervision. Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, Seattle, WA, USA.10.1109/WACV45572.2020.9093544 ↗
14. Stroke-based scene text erasing using synthetic data for training10.1109/TIP.2021.3125260 ↗
15. Conrad, B., and Chen, P.I. (2021, January 19–22). Two-stage seamless text erasing on real-world scene images. Proceedings of the 2021 IEEE International Conference on Image Processing (ICIP), Anchorage, AK, USA.10.1109/ICIP42928.2021.9506394 ↗
16. Toward backdoor attacks for image captioning model in deep neural networks10.1155/2022/1525052 ↗
17. Rombach, R., Blattmann, A., Lorenz, D., Esser, P., and Ommer, B. (2022, January 18–24). High-resolution image synthesis with latent diffusion models. Proceedings of the IEEE/CVF Conference on Computer vision and Pattern Recognition, New Orleans, LA, USA.10.1109/CVPR52688.2022.01042 ↗
18. Real-time scene text detection with differentiable binarization and adaptive scale fusion10.1109/TPAMI.2022.3155612 ↗
19. Mlp-mixer: An all-mlp architecture for vision
20. Yu, W., Luo, M., Zhou, P., Si, C., Zhou, Y., Wang, X., Feng, J., and Yan, S. (2022, January 18–24). Metaformer is actually what you need for vision. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.10.1109/CVPR52688.2022.01055 ↗
21. Centralized feature pyramid for object detection10.1109/TIP.2023.3297408 ↗
22. Peng, Z., Huang, W., Gu, S., Xie, L., Wang, Y., Jiao, J., and Ye, Q. (2021, January 11–17). Conformer: Local Features Coupling Global Representations for Visual Recognition. Proceedings of the IEEE/CVF International Conference on Computer Vision, Montreal, BC, Canada.10.1109/ICCV48922.2021.00042 ↗
23. Liu, Z., Hu, H., Lin, Y., Yao, Z., Xie, Z., Wei, Y., Ning, J., Cao, Y., Zhang, Z., and Dong, L. (2022, January 18–24). Swin transformer v2: Scaling up capacity and resolution. Proceedings of the IEEE/CVF Conference on Computer vision and Pattern Recognition, New Orleans, LA, USA.10.1109/CVPR52688.2022.01170 ↗
24. Wang, Z., Li, Z., Sun, J., and Xu, Y. (2018, January 9–12). Selective convolutional features based generalized-mean pooling for fine-grained image retrieval. Proceedings of the 2018 IEEE Visual Communications and Image Processing (VCIP), Taichung, Taiwan.10.1109/VCIP.2018.8698729 ↗
25. Jaderberg, M., Simonyan, K., and Zisserman, A. (2015). Spatial transformer networks. arXiv.
26. Sohl-Dickstein, J., Weiss, E., Maheswaranathan, N., and Ganguli, S. (2015, January 6–11). Deep unsupervised learning using nonequilibrium thermodynamics. Proceedings of the International Conference on Machine Learning, PMLR, Lille, France.
27. Zhang, S., Liu, Y., Jin, L., Huang, Y., and Lai, S. (February, January 27). Ensnet: Ensconce text in the wild. Proceedings of the AAAI Conference on Artificial Intelligence, Honolulu, HI, USA.
28. Karatzas, D., Shafait, F., Uchida, S., Iwamura, M., i Bigorda, L.G., Mestre, S.R., Mas, J., Mota, D.F., Almazan, J.A., and De Las Heras, L.P. (September, January 30). ICDAR 2013 robust reading competition. Proceedings of the 2013 12th International Conference on Document Analysis and Recognition, Athens, Greece.10.1109/ICDAR.2013.221 ↗
29. Nayef, N., Yin, F., Bizid, I., Choi, H., Feng, Y., Karatzas, D., Luo, Z., Pal, U., Rigaud, C., and Chazalon, J. (2017, January 9–15). Icdar2017 robust reading challenge on multi-lingual scene text detection and script identification-rrc-mlt. Proceedings of the 2017 14th IAPR International Conference on Document Analysis and Recognition (ICDAR), Kyoto, Japan.10.1109/ICDAR.2017.237 ↗
30. Long, S., Ruan, J., Zhang, W., He, X., Wu, W., and Yao, C. (2018, January 8–14). Textsnake: A flexible representation for detecting text of arbitrary shapes. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.10.1007/978-3-030-01216-8_2 ↗
31. Wang, W., Xie, E., Song, X., Zang, Y., Wang, W., Lu, T., Yu, G., and Shen, C. (November, January 27). Efficient and accurate arbitrary-shaped text detection with pixel aggregation network. Proceedings of the IEEE/CVF International Conference on Computer Vision, Seoul, Republic of Korea.
32. Isola, P., Zhu, J.Y., Zhou, T., and Efros, A.A. (2017, January 21–26). Image-to-image translation with conditional adversarial networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.10.1109/CVPR.2017.632 ↗
33. Scene text removal via cascaded text stroke detection and erasing10.1007/s41095-021-0242-8 ↗