Stroke-Based Autoencoders: Self-Supervised Learners for Efficient Zero-Shot Chinese Character Recognition
Source abstract
Abstract
Chinese characters carry a wealth of morphological and semantic information; therefore, zero-shot Chinese character recognition with the morphology of Chinese characters has drawn significant attention. The previous methods are mainly based on radical-level decomposition or stroke-level decomposition, which usually cannot capture adequately the structural and spatial information of Chinese characters. In this paper, we develop a stroke-based autoencoder (SAE), to model the sophisticated morphology of Chinese characters with a self-supervised method. Following its canonical writing order, we first represent a Chinese character as a series of stroke images with a fixed writing order, and then our SAE model is trained to reconstruct this stroke image sequence. This pre-trained SAE model can predict the stroke image series for unseen characters, as long as their strokes or radicals are in the training set. We have designed two contrasting SAE architectures on different forms of stroke images. One is fine-tuned on existing stroke-based method for zero-shot recognition of handwritten Chinese characters, and the other is applied to enrich the Chinese word embeddings from their morphological features. The experimental results validate that after pre-training, our SAE architecture outperforms other existing methods in zero-shot recognition and enhances the representation of Chinese characters with their abundant morphological and semantic information.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Handwritten Text Recognition Techniques · Domain Adaptation and Few-Shot Learning · Advanced Neural Network Applications
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Department of Mathematics, Wuhan University of Technology, Wuhan 430070, China
Lane Department of Computer Science and Electrical Engineering, West Virginia University, Morgantown, VA 26506, USA
Funding
National Natural Science Foundation of China · China Scholarship Council · National Innovation and Entrepreneurship Training Program for College Students
References (50)
Show source-provided references
1. Chen, J., Li, B., and Xue, X. (2021, January 19–26). Zero-Shot Chinese Character Recognition with Stroke-Level Decomposition. Proceedings of the International Joint Conference on Artificial Intelligence IJCAI, Virtual.10.24963/ijcai.2021/85 ↗
2. Zeng, J., Xu, R., Wu, Y., Li, H., and Lu, J. (2022). STAR: Zero-Shot Chinese Character Recognition with Stroke-and Radical-Level Decompositions. arXiv.
3. StrokeGAN: Reducing Mode Collapse in Chinese Font Generation via Stroke Encoding
4. Wen, Q., Li, S., Han, B., and Yuan, Y. (2021, January 20–24). Zigan: Fine-grained chinese calligraphy font generation via a few-shot style transfer approach. Proceedings of the 29th ACM International Conference on Multimedia, Virtual.10.1145/3474085.3475225 ↗
5. Wu, S., Song, X., and Feng, Z. (2021). MECT: Multi-Metadata Embedding based Cross-Transformer for Chinese Named Entity Recognition. arXiv.10.18653/v1/2021.acl-long.121 ↗
6. Li, J., and Meng, K. (2021). MFE-NER: Multi-feature Fusion Embedding for Chinese Named Entity Recognition. arXiv.
7. Sun, Z., Li, X., Sun, X., Meng, Y., Ao, X., He, Q., Wu, F., and Li, J. (2021, January 1–6). ChineseBERT: Chinese Pretraining Enhanced by Glyph and Pinyin Information. Proceedings of the Joint Conference of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (ACL-IJCNLP 2021), Bangkok, Thailand.10.18653/v1/2021.acl-long.161 ↗
8. Offline signature recognition and forgery detection using deep learning10.1016/j.procs.2020.03.133 ↗
9. Ren, H., Pan, M., Li, Y., Zhou, X., and Luo, J. (2020, January 6–10). ST-SiameseNet: Spatio-Temporal Siamese Networks for Human Mobility Signature Identification. Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, Virtual.10.1145/3394486.3403183 ↗
10. Maekawa, K., Tomiura, Y., Fukuda, S., Ishita, E., and Uchiyama, H. (2019, January 4–7). Improving OCR for historical documents by modeling image distortion. Proceedings of the International Conference on Asian Digital Libraries, Kuala Lumpur, Malaysia.10.1007/978-3-030-34058-2_31 ↗
11. Huang, S., Wang, H., Liu, Y., Shi, X., and Jin, L. (2019, January 20–25). Obc306: A large-scale oracle bone character recognition dataset. Proceedings of the 2019 International Conference on Document Analysis and Recognition (ICDAR), Sydney, NSW, Australia.10.1109/ICDAR.2019.00114 ↗
12. Wang, W., Zhang, J., Du, J., Wang, Z.R., and Zhu, Y. (2018, January 5–8). Denseran for offline handwritten chinese character recognition. Proceedings of the 2018 16th International Conference on Frontiers in Handwriting Recognition (ICFHR), Niagara Falls, NY, USA.10.1109/ICFHR-2018.2018.00027 ↗
13. Zero-shot Handwritten Chinese Character Recognition with hierarchical decomposition embedding10.1016/j.patcog.2020.107488 ↗
14. Zhang, J., Zhu, Y., Du, J., and Dai, L. (2018, January 23–27). Radical analysis network for zero-shot learning in printed Chinese character recognition. Proceedings of the 2018 IEEE International Conference on Multimedia and Expo (ICME), San Diego, CA, USA.10.1109/ICME.2018.8486456 ↗
15. Wang, S., Huang, G., and Luo, X. (2021). Hippocampus-heuristic Character Recognition Network for Zero-shot Learning. arXiv.10.1016/j.patcog.2022.108818 ↗
16. Lladós, J., Lopresti, D., and Uchida, S. (2021, January 5–10). Zero-Shot Chinese Text Recognition via Matching Class Embedding. Proceedings of the Document Analysis and Recognition—ICDAR 2021, Lausanne, Switzerland.10.1007/978-3-030-86334-0 ↗
17. Diao, X., Shi, D., Tang, H., Li, Y., Wu, L., and Xu, H. (2022). REZCR: Zero-shot Character Recognition via Radical Extraction. arXiv.
18. Haralambous, Y. (2011). Seeking Meaning in a Space Made out of Strokes, Radicals, Characters and Compounds. arXiv.
19. Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., and Fei-Fei, L. (2009, January 20–25). ImageNet: A Large-Scale Hierarchical Image Database. Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition CVPR09, Miami, FL, USA.10.1109/CVPR.2009.5206848 ↗
20. Devlin, J., Chang, M.W., Lee, K., and Toutanova, K. (2018). Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv.
21. He, K., Chen, X., Xie, S., Li, Y., Doll’ar, P., and Girshick, R.B. (2021). Masked Autoencoders Are Scalable Vision Learners. arXiv.10.1109/CVPR52688.2022.01553 ↗
22. Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., and Gelly, S. (2021). An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale. arXiv.
23. Bao, H., Dong, L., and Wei, F. (2021). BEiT: BERT Pre-Training of Image Transformers. arXiv.
24. Stacked Denoising Autoencoders: Learning Useful Representations in a Deep Network with a Local Denoising Criterion
25. Chen, M., Radford, A., Wu, J., Jun, H., Dhariwal, P., Luan, D., and Sutskever, I. (2020, January 13–18). Generative Pretraining From Pixels. Proceedings of the Thirty-Seventh International Conference on Machine Learning ICML, Virtual Event.
26. Ramesh, A., Pavlov, M., Goh, G., Gray, S., Voss, C., Radford, A., Chen, M., and Sutskever, I. (2021). Zero-Shot Text-to-Image Generation. arXiv.
27. Kodirov, E., Xiang, T., and Gong, S. (2017, January 21–26). Semantic Autoencoder for Zero-Shot Learning. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.10.1109/CVPR.2017.473 ↗
28. Attribute-Based Classification for Zero-Shot Visual Object Categorization10.1109/TPAMI.2013.140 ↗
29. Cao, S., Lu, W., Zhou, J., and Li, X. (2018, January 2–7). cw2vec: Learning Chinese Word Embeddings with Stroke n-gram Information. Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence AAAI, New Orleans, LA, USA.10.1609/aaai.v32i1.12029 ↗
30. Li, Y., Zhao, Y., Hu, B., Chen, Q., Xiang, Y., Wang, X., Ding, Y., and Ma, L. (2021). GlyphCRM: Bidirectional Encoder Representation for Chinese Character with its Glyph. arXiv.
31. Camacho-Collados, J., Pilehvar, M.T., Collier, N., and Navigli, R. (2017). Semeval-2017 Task 2: Multilingual and Cross-Lingual Semantic Word Similarity, Association for Computational Linguistics.10.18653/v1/S17-2002 ↗
32. Li, S., Zhao, Z., Hu, R., Li, W., Liu, T., and Du, X. (2018). Analogical reasoning on chinese morphological and semantic relations. arXiv.10.18653/v1/P18-2023 ↗
33. Qiu, Y., Li, H., Li, S., Jiang, Y., Hu, R., and Yang, L. (2018). Chinese Computational Linguistics and Natural Language Processing Based on Naturally Annotated Big Data, Springer.
34. Enriching Word Vectors with Subword Information10.1162/tacl_a_00051 ↗
35. Holistic versus analytic processing: Evidence for a different approach to processing of Chinese at the word and character levels in Chinese children10.1016/j.jecp.2010.06.006 ↗
36. Vaswani, A., Shazeer, N.M., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, L., and Polosukhin, I. (2017). Attention is All you Need. arXiv.
37. He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27–30). Deep Residual Learning for Image Recognition. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.10.1109/CVPR.2016.90 ↗
38. Ba, J.L., Kiros, J.R., and Hinton, G.E. (2016). Layer Normalization. arXiv.
39. (2022, March 21). Hanzi Writer. Available online: https://hanziwriter.org/.
40. Online and offline handwritten Chinese character recognition: Benchmarking on new databases10.1016/j.patcog.2012.06.021 ↗
41. Yin, F., Wang, Q.F., Zhang, X.Y., and Liu, C.L. (2013, January 25–28). ICDAR 2013 Chinese Handwriting Recognition Competition. Proceedings of the 2013 12th International Conference on Document Analysis and Recognition, Washington, DC, USA.10.1109/ICDAR.2013.218 ↗
42. Loshchilov, I., and Hutter, F. (2017). Decoupled weight decay regularization. arXiv.
43. Loshchilov, I., and Hutter, F. (2016). Sgdr: Stochastic gradient descent with warm restarts. arXiv.
44. Zhong, Z., Jin, L., and Xie, Z. (2015, January 23–26). High performance offline handwritten chinese character recognition using googlenet and directional feature maps. Proceedings of the 2015 13th International Conference on Document Analysis and Recognition (ICDAR), Tunis, Tunisia.10.1109/ICDAR.2015.7333881 ↗
45. Online and offline handwritten chinese character recognition: A comprehensive study and new benchmark10.1016/j.patcog.2016.08.005 ↗
46. Improving offline handwritten Chinese character recognition by iterative refinement10.1109/ICDAR.2017.11 ↗
47. Xiao, Y., Meng, D., Lu, C., and Tang, C.K. (2019, January 20–25). Template-instance loss for offline handwritten Chinese character recognition. Proceedings of the 2019 International Conference on Document Analysis and Recognition (ICDAR), Sydney, NSW, Australia.10.1109/ICDAR.2019.00058 ↗
48. Radical aggregation network for few-shot offline handwritten Chinese character recognition10.1016/j.patrec.2019.08.005 ↗
49. Visualizing data using t-SNE
50. Algorithm AS 136: A k-means clustering algorithm