journal article·2024-02-29Open access

Intelligent Evaluation of Chinese Hard-Pen Calligraphy Using a Siamese Transformer Network

Fei Yan, Xueping Lan, Hua Zhang, Linjing Li

Applied Sciences· Volume 14· Issue 5

Abstract

The essence of Chinese calligraphy inheritance resides in calligraphy education. However, it encounters challenges such as a scarcity of calligraphy instructors, time-consuming and inefficient manual assessment methods, and inconsistent evaluation criteria. In response to these challenges, this paper introduces a deep learning-based automatic calligraphy evaluation model. Initially, hard-pen handwriting samples from 100 volunteers were collected and preprocessed to create a dataset consisting of 4800 samples, along with the corresponding label files for hard-pen calligraphy evaluation. Subsequently, YOLOv5 was utilized for region detection and character recognition on the evaluation samples to obtain the corresponding standard samples. Lastly, a Siamese metric model, with VGG16 as the primary feature extraction submodule, was developed for hard-pen calligraphy evaluation. To improve feature extraction and propagation, a transformer structure was introduced to extract global information from both the evaluated and standard samples, thereby optimizing the evaluation results. Experimental results demonstrate that the proposed model achieves a precision of 0.75, recall of 0.833, and mAP of 0.990 on the hard-pen calligraphy evaluation dataset, effectively realizing automatic calligraphy evaluation. This model presents a novel approach for intelligently assessing hard-pen calligraphy.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: Color Science and Applications · Image Enhancement Techniques · Industrial Vision Systems and Defect Detection

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

School of Information Engineering, Southwest University of Science and Technology, Mianyang 621010, China

Mianyang Polytechnic, Mianyang 621000, China

Funding

Mianyang Polytechnic Foundation for Science and Technology

References (39)

Show source-provided references
1. Style-adaptive photo aesthetic rating via convolutional neural networks and multi-task learning10.1016/j.neucom.2018.06.099 ↗
2. Inkthetics: A comprehensive computational model for aesthetic evaluation of Chinese ink paintings10.1109/ACCESS.2020.3044573 ↗
3. Image/video aesthetic quality management based on fusing 3D CNN and shallow features10.1016/j.future.2020.12.027 ↗
4. A Novel Webpage Layout Aesthetic Evaluation Model for Quantifying Webpage Layout Design10.1016/j.ins.2021.06.071 ↗
5. Distinctive image features from scale-invariant keypoints10.1023/B:VISI.0000029664.99615.94 ↗
6. Dalal, N., and Triggs, B. (2005, January 20–26). Histograms of oriented gradients for human detection. Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05), San Diego, CA, USA.
7. Uncertainty relation for resolution in space, spatial frequency, and orientation optimized by two-dimensional visual cortical filters10.1364/JOSAA.2.001160 ↗
8. Support-vector networks10.1007/BF00994018 ↗
10. Nearest neighbor pattern classification10.1109/TIT.1967.1053964 ↗
11. Chen, T., and Guestrin, C. (2016, January 13–17). Xgboost: A scalable tree boosting system. Proceedings of the 22nd ACM Sigkdd International Conference on Knowledge Discovery and Data Mining, San Francisco, CA, USA.10.1145/2939672.2939785 ↗
12. Ke, G., Meng, Q., Finley, T., Wang, T., Chen, W., Ma, W., Ye, Q., and Liu, T.-Y. (2017, January 4–9). Lightgbm: A highly efficient gradient boosting decision tree. Proceedings of the Neural Information Processing Systems 2017, Long Beach, CA, USA.
13. An interactive grading and learning system for chinese calligraphy10.1007/s00138-007-0076-0 ↗
14. Gao, Y., Jin, L., and Li, N. (2011, January 6–8). Chinese handwriting quality evaluation based on analysis of recognition confidence. Proceedings of the 2011 IEEE International Conference on Information and Automation, Shenzhen, China.
15. Computationally evaluating and synthesizing Chinese calligraphy10.1016/j.neucom.2013.12.013 ↗
16. Sun, R., Lian, Z., Tang, Y., and Xiao, J. (August, January 28). Aesthetic Visual Quality Evaluation of Chinese Handwritings. Proceedings of the IJCAI 2015, Buenos Aires, Argentina.
17. A hierarchical evaluation approach of learning Chinese calligraphy
18. Evaluation of Chinese Calligraphy by Using DBSC Vectorization and ICP Algorithm
19. Zhou, D., Ge, J., Wu, R., Chao, F., Yang, L., and Zhou, C. (2017, January 29–31). A computational evaluation system of Chinese calligraphy via extended possibility-probability distribution method. Proceedings of the 2017 13th International Conference on Natural Computation, Fuzzy Systems and Knowledge Discovery (ICNC-FSKD), Guilin, China.10.1109/FSKD.2017.8393393 ↗
20. Design and Application of a Fuzzy Evaluation Algorithm for Stroke Force in Calligraphy Teaching10.3991/ijet.v13i07.8792 ↗
21. MSCAN: Multimodal self-and-collaborative attention network for image aesthetic prediction tasks10.1016/j.neucom.2020.10.046 ↗
22. Personality-assisted multi-task learning for generic and personalized image aesthetics assessment10.1109/TIP.2020.2968285 ↗
23. Qiang, X., Wu, M., and Luo, L. (2019, January 2–4). Research on Hard-tipped Calligraphy Classification Based on Deep Learning Method. Proceedings of the 2019 8th International Conference on Educational and Information Technology, Cambridge, UK.10.1145/3318396.3318417 ↗
24. Srafe: Siamese regression aesthetic fusion evaluation for Chinese calligraphic copy10.1049/cit2.12095 ↗
25. Xu, P., Wang, L., Guan, Z., Zheng, X., Chen, X., Tang, Z., Fang, D., Gong, X., and Wang, Z. (2018, January 13–19). Evaluating brush movements for Chinese calligraphy: A computer vision based approach. Proceedings of the 27th International Joint Conference on Artificial Intelligence IJCAI 2018, Stockholm, Sweden.10.24963/ijcai.2018/146 ↗
26. Wang, Z., and Lv, R. (2021, January 16–17). Design of calligraphy aesthetic evaluation model based on deep learning and writing action. Proceedings of the International Conference on Computing, Control and Industrial Engineering, Hangzhou, China.
27. Zhang, H., Guo, J., Chen, G., and Li, C. (2009, January 27–28). HCL2000-A large-scale handwritten Chinese character database for handwritten character recognition. Proceedings of the 2009 10th International Conference on Document Analysis and Recognition, Wuhan, China.10.1109/ICDAR.2009.15 ↗
28. Fu, Q., Ding, X., Li, T., and Liu, C. (2007, January 23–26). An effective and practical classifier fusion strategy for improving handwritten character recognition. Proceedings of the Ninth International Conference on Document Analysis and Recognition (ICDAR 2007), Curitiba, Brazil.10.1109/ICDAR.2007.4377073 ↗
29. Liu, C.-L., Yin, F., Wang, D.-H., and Wang, Q.-F. (2011, January 18–21). CASIA online and offline Chinese handwriting databases. Proceedings of the 2011 International Conference on Document Analysis and Recognition, Beijing, China.10.1109/ICDAR.2011.17 ↗
30. A threshold selection method from gray-level histograms10.1109/TSMC.1979.4310076 ↗
31. Bochkovskiy, A., Wang, C.-Y., and Liao, H.-Y.M. (2020). Yolov4: Optimal speed and accuracy of object detection. arXiv.
32. Bromley, J., Guyon, I., Lecun, Y., Sckinger, E., and Shah, R. (, 1993). Signature Verification Using a Siamese Time Delay Neural Network. Proceedings of the Advances in Neural Information Processing Systems 6, 7th NIPS Conference, Denver, CO, USA.10.1142/9789812797926_0003 ↗
33. Simonyan, K., and Zisserman, A. (2014). Very Deep Convolutional Networks for Large-Scale Image Recognition. arXiv.
34. Sawarkar, A.D., Shrimankar, D.D., Ali, S., Agrahari, A., and Singh, L. (2024). Bamboo Plant Classification Using Deep Transfer Learning with a Majority Multiclass Voting Algorithm. Appl. Sci., 14.10.3390/app14031023 ↗
35. Cubic convolution interpolation for digital image processing10.1109/TASSP.1981.1163711 ↗
36. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., and Polosukhin, I. (2017, January 4–9). Attention is all you need. Proceedings of the Advances in Neural Information Processing Systems, Long Beach, CA, USA.
37. Ulyanov, D., Vedaldi, A., and Lempitsky, V. (2016). Instance normalization: The missing ingredient for fast stylization. arXiv.
38. Ba, J.L., Kiros, J.R., and Hinton, G.E. (2016). Layer normalization. arXiv.
39. He, K., Zhang, X., Ren, S., and Sun, J. (July, January 26). Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition 2016, Las Vegas, NV, USA.10.1109/CVPR.2016.90 ↗