Material classification method of traditional Chinese painting image based on prototypical network
Source abstract
Abstract
Intelligent material recognition in traditional Chinese painting images is vital for digital cultural heritage preservation. However, existing methods struggle with limited samples and weak feature representation. This paper proposes a new classification framework based on the prototypical network with a ResNet18 backbone. To enhance sensitivity to fine-grained details, a cropping-based augmentation strategy is introduced during inference. Additionally, a multitask learning scheme is employed to improve generalization and enrich global representations, where the auxiliary task of dynasty classification is trained jointly with the main task of material classification. Furthermore, an ensemble voting strategy is applied to refine the final predictions. Experiments on a self-constructed dataset demonstrate the robustness of the entire strategy under few-shot scenarios, achieving 80% accuracy with merely 30 samples and outperforming comparative methods. Moreover, this work provides practical value for the development of intelligent tools to support the digital preservation of traditional Chinese art.
Source: openalex_inverted_index · Quality: baseline
Research context
OpenAlex topics: Digital Media and Visual Art · AI and Big Data Applications · Advanced Technology in Applications
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
References (53)
Show source-provided references
1. Tan, W. R. et al. Ceci n’est pas une pipe: a deep convolutional network for fine-art paintings classification. In Proc. 2016 IEEE International Conference on Image Processing (ICIP) 3703–3707 (IEEE, 2016).10.1109/ICIP.2016.7533051 ↗
2. Jing, Y. et al. Neural style transfer: a review. IEEE Trans. Vis. Comput. Graph 26, 3365–3385 (2019).10.1109/TVCG.2019.2921336 ↗
3. Cetinic, E. & She, J. Understanding and creating art with AI: review and outlook. ACM Trans. Multimed. Comput. Commun. Appl. TOMM 18, 1–22 (2022).10.1145/3475799 ↗
4. Dobbs, T. et al. Contemporary art authentication with large-scale classification. Big Data Cogn. Comput. 7, 162 (2023).10.3390/bdcc7040162 ↗
5. Zhang, X., Berghe, I. V. & Wyeth, P. Heat and moisture promoted deterioration of raw silk estimated by amino acid analysis. J. Cult. Herit. 12, 408–411 (2011).10.1016/j.culher.2011.03.002 ↗
6. Jiang, S. et al. An effective method to detect and categorize digitized traditional Chinese paintings. Pattern Recogn. Lett. 27, 734–746 (2006).10.1016/j.patrec.2005.10.017 ↗
7. Liong, S. T. et al. Automatic traditional Chinese painting classification: a benchmarking analysis. Comput. Intell. 36, 1183–1199 (2020).10.1111/coin.12328 ↗
8. Sheng, J. & Li, Y. Classification of traditional Chinese paintings using a modified embedding algorithm. J. Electron Imaging 28, 023013–023013 (2019).10.1117/1.JEI.28.2.023013 ↗
9. Meng, Q. et al. The classification of traditional Chinese painting based on CNN. In Cloud Computing and Security: 4th International Conference (ICCCS 2018) 232–241 (Springer, 2018).10.1007/978-3-030-00009-7_22 ↗
10. Li, D. & Zhang, Y. Multi-instance learning algorithm based on LSTM for Chinese painting image classification. IEEE Access 8, 179336–179345 (2020).10.1109/ACCESS.2020.3027982 ↗
11. Yang, D., Ye, X. & Guo, B. Application of multitask joint sparse representation algorithm in Chinese painting image classification. Complexity 2021, 5546338 (2021).10.1155/2021/5546338 ↗
12. Sun, M. et al. Monte Carlo convex hull model for classification of traditional Chinese paintings. Neurocomputing 171, 788–797 (2016).10.1016/j.neucom.2015.08.013 ↗
13. Jiang, W. et al. MTFFNet: a multi-task feature fusion framework for Chinese painting classification. Cogn. Comput. 13, 1287–1296 (2021).10.1007/s12559-021-09896-9 ↗
14. Geng, J. et al. MCCFNet: multi-channel color fusion network for cognitive classification of traditional chinese paintings. Cogn. Comput. 15, 2050–2061 (2023).10.1007/s12559-023-10172-1 ↗
15. Li, Z. et al. A survey of convolutional neural networks: analysis, applications, and prospects. IEEE Trans. Neural Netw. Learn. Syst. 33, 6999–7019 (2021).10.1109/TNNLS.2021.3084827 ↗
16. Weiss, K., Khoshgoftaar, T. M. & Wang, D. A survey of transfer learning.J. Big Data 3, 1–40 (2016).10.1186/s40537-016-0043-6 ↗
17. Wang, Y. et al. Generalizing from a few examples: a survey on few-shot learning. ACM Comput. Surv. 53, 1–34 (2020).
18. Haralick, R. M., Shanmugam, K. & Dinstein, I. Textural features for image classification. IEEE Trans. Syst. Man Cybern. SMC-3, 610–621 (1973).10.1109/TSMC.1973.4309314 ↗
19. Losson, O. et al. Color texture analysis using CFA chromatic co-occurrence matrices. Comput. Vis. Image Underst. 117, 747–763 (2013).10.1016/j.cviu.2013.03.001 ↗
20. Lowe, D. G. Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60, 91–110 (2004).10.1023/B:VISI.0000029664.99615.94 ↗
21. Wang, Z. et al. Supervised heterogeneous sparse feature selection for Chinese paintings classification. J. Comput. Aided Des. Comput. Graph 25, 1848–1855 (2013). (in Chinese with an English abstract).
22. Sheng, J. C. Automatic categorization of traditional Chinese paintings based on wavelet transform. Comput. Sci. 41, 317–319 (2014). (in Chinese with an English abstract).
23. Sheng, J. C. & Jiang, J. M. Recognition of Chinese artists via windowed and entropy balanced fusion in classification of their authored ink and wash paintings (IWPs). Pattern Recogn. 47, 612–622 (2014).10.1016/j.patcog.2013.08.017 ↗
24. LeCun, Y., Bengio, Y. & Hinton, G. Deep learning. Nature 521, 436–444 (2015).10.1038/nature14539 ↗
25. Sun, M. et al. Brushstroke based sparse hybrid convolutional neural networks for author classification of Chinese ink-wash paintings. In Proc. 2015 IEEE International Conference on Image (ICIP) 626–630 (IEEE, 2015).10.1109/ICIP.2015.7350874 ↗
26. Sheng, J. C. & Li, Y. Z. Learning artistic objects for improved classification of Chinese paintings. Int. J. Image Graph 23, 1193–1206 (2018). (in Chinese with an English abstract).
27. Zhou, Y. Research on Chinese painting image classification method based on improved convolutional neural network model. J. Jiamusi Univ. Nat. Sci. Ed. 39, 112–115 (2021). (in Chinese with an English abstract).
28. Yu, Q. & Shi, C. An image classification approach for painting using improved convolutional neural algorithm. Soft Comput. 28, 847–873 (2024).10.1007/s00500-023-09420-1 ↗
29. Wang, J. et al. Recent advances of few-shot learning methods and applications. Sci. China Technol. Sci. 66, 920–944 (2023).10.1007/s11431-022-2133-1 ↗
30. Xiao, Z. Research on Painting Image Classification based on Convolutional Neural Network (China Jiliang University, 2020) (in Chinese with an English abstract).
31. Chen, H. Y. Chinese Painting Image Classification Based on Deep Learning (Hangzhou Dianzi University, 2021) (in Chinese with an English abstract).
32. Li, K. et al. Dongba painting few-shot classification based on graph neural network. J. Comput. Aided Des. Comput. Graph 33, 1073–1083 (2021). (in Chinese with an English abstract).10.1016/S0010-4485(00)00144-5 ↗
33. Xu, Q. H. & Qian, W. H. Dongba painting few-shot classification combining spatial information and distribution relationship. J. Comput. Aided Des. Comput. Graph. 35, 1417–1428 (2023). (in Chinese with an English abstract).
34. Ding, Y. et al. TCP-RBA: semi-supervised learning for traditional chinese painting classification with random brushwork augment. J. Intell. Fuzzy Syst. 46, 10653–10663 (2024).
35. Zhang, Y. & Yang, Q. A survey on multi-task learning. IEEE Trans. Knowl. Data Eng. 34, 5586–5609 (2021).10.1109/TKDE.2021.3070203 ↗
36. Dorozynski, M. Addressing class imbalance for training a multi-task classifier in the context of silk heritage. ISPRS Ann. Photogramm. Remote Sens. Spatial Inf. Sci. X-1/W1-2023, 175–184 (2023).10.5194/isprs-annals-X-1-W1-2023-175-2023 ↗
37. Bianco, S. et al. Multitask painting categorization by deep multibranch neural network. Expert Syst. Appl. 135, 90–101 (2019).10.1016/j.eswa.2019.05.036 ↗
38. Snell, J., Swersky, K. & Zemel, R. Prototypical networks for few-shot learning. In Proc. Neural Information Processing Systems 30 (NIPS, 2017).
39. Chen, J. et al. Research on few-shot paleontological fossil identification based on improved prototype network. Geol. Rev. 69, 1967–1979 (2023) (in Chinese with an English abstract).
40. Khan, M. A. H. et al. Few-shot classification and anatomical localization of tissues in SPECT imaging. In Proc. 2024 IEEE Nuclear Science Symposium (NSS), Medical Imaging Conference (MIC) and Room Temperature Semiconductor Detector Conference (RTSD) (IEEE, 2024).10.1109/NSS/MIC/RTSD57108.2024.10656015 ↗
41. He, K. et al. Deep residual learning for image recognition. In Proc. 29th IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 770–778 (IEEE, 2016).10.1109/CVPR.2016.90 ↗
42. Caruana, R. Multitask learning. Mach. Learn. 28, 41–75 (1997).10.1023/A:1007379606734 ↗
43. Deng, Y. et al. Feature-guided multitask change detection network. IEEE J. Sel. Top. Appl. Earth Observ. Remote Sens. 15, 9667–9679 (2022).10.1109/JSTARS.2022.3215773 ↗
44. Breiman, L. Random forests. Mach. Learn. 45, 5–32 (2001).10.1023/A:1010933404324 ↗
45. Cortes, C. & Vapnik, V. Support-vector networks. Mach. Learn. 20, 273 (1995).10.1023/A:1022627411411 ↗
46. Rumelhart, D. E., Hinton, G. E. & Williams, R. J. Learning representations by back-propagating errors. Nature 323, 533–536 (1986).10.1038/323533a0 ↗
47. Ke, G. et al. Lightgbm: a highly efficient gradient boosting decision tree. In Proc. Neural Information Processing Systems 30 (NIPS, 2017).
48. Chen, T. & Guestrin, C. Xgboost: a scalable tree boosting system. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining 785–794 (ACM, 2016).10.1145/2939672.2939785 ↗
49. Seiffert, C. et al. RUSBoost: a hybrid approach to alleviating class imbalance. IEEE Trans. Syst. Man Cyber. A 40, 185–197 (2009).10.1109/TSMCA.2009.2029559 ↗
50. Radford, A. et al. Learning transferable visual models from natural language supervision. In Proc. International Conference on Machine Learning 8748–8763 (PMLR, 2021).
51. Wang, Z. et al. Chinese painting emotion classification based on convolution neural network and SVM. J. Nanjing Norm. Univ. Nat. Sci. Ed. 40, 74–79 (2017). (in Chinese with an English abstract).
52. Yang, B. et al. Chinese painting image classification based on convolution neural network. Softw. Guide 18, 5–8 (2019). (in Chinese with an English abstract).
53. Guo, D. Research and Application of Chinese Painting Image Classification Method Based on Semi-supervised Learning (Communication University of China, 2023) (in Chinese with an English abstract).