OBI-CMF: Self-supervised learning with contrastive masked frequency modeling for oracle bone inscription recognition
Source abstract
Abstract
Oracle bone inscriptions (OBI) are precious historical records of the Shang Dynasty around 3000 years ago and have profound cultural significance. OBI recognition reveals the origin of Chinese characters and enriches linguistic theory. However, existing OBI images generally have problems with noise interference and intra-class variability, which pose significant challenges to the accuracy and efficiency of traditional manual recognition methods and supervised learning networks. To address this issue, we propose a novel self-supervised learning approach with c ontrastive m asked f requency modeling for OBI recognition, named OBI-CMF. Firstly, the contrastive learning and masking model are integrated within the self-supervised learning framework so that our OBI-CMF can efficiently extract and learn both global and local features of OBI images. Secondly, inter-domain supervision is achieved by learning OBI features in the spatial and frequency domains, which significantly improves the robustness and generalization ability of our OBI-CMF. Finally, a linear classifier is trained using the representations obtained from the pre-trained model. Furthermore, based on the OBC306 dataset, we construct three subsets, OBIR-10, OBIR-100, and OBIR-ID, to validate the advantages of our OBI-CMF from the perspective of balanced and imbalanced data. The performance of our model is evaluated on three subsets of the OBC306 public dataset. Experimental results show that our OBI-CMF achieves 99.57% on OBIR-10 and 95.08% on OBIR-100, which outperforms some supervised and self-supervised learning methods in the OBI recognition tasks. Moreover, it is the first attempt to apply self-supervised learning based on the frequency domain for OBI recognition. We hope this success can provide a new avenue for the virtual protection of cultural relics.
Source: openalex_inverted_index · Quality: baseline
Research context
OpenAlex topics: Natural Language Processing Techniques · Speech Recognition and Synthesis · Handwritten Text Recognition Techniques
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
References (49)
Show source-provided references
1. Christina, C. The Chinese history that is written in bone. Am. Sci. 106, 133 (2018).10.1511/2018.106.3.133 ↗
2. Flad, R. K. Divination and power: a multiregional view of the development of oracle bone divination in early china. Curr. Anthropol. 49, 403–437 (2008).10.1086/588495 ↗
3. Guo, Z. et al. An improved neural network model based on inception-v3 for oracle bone inscription character recognition. Sci. Program 2022, 1–8 (2022).
4. Fu, X., Yang, Z., Zeng, Z., Zhang, Y. & Zhou, Q. Improvement of oracle bone inscription recognition accuracy: a deep learning perspective. ISPRS Int J. Geo-Inf. 11, 45 (2022).10.3390/ijgi11010045 ↗
5. Xuzheng, L., Hengjin, C. & Li, L. Recognition of oracle radical based on the capsule network. CAAI Trans. Intell. Syst. 15, 243–254 (2020).
6. Wang, M., Deng, W. & Su, S. Oracle character recognition using unsupervised discriminative consistency network. Pattern Recognit. 148, 110180 (2024).10.1016/j.patcog.2023.110180 ↗
7. Huang, S., Wang, H., Liu, Y., Shi, X. & Jin, L. Obc306: a large-scale oracle bone character recognition dataset. In 2019 International Conference on Document Analysis and Recognition, 681–688 (IEEE, 2019).10.1109/ICDAR.2019.00114 ↗
8. Li, J. et al. Towards better long-tailed oracle character recognition with adversarial data augmentation. Pattern Recognit. 140, 109534 (2023).10.1016/j.patcog.2023.109534 ↗
9. Zhang, H. mixup: beyond empirical risk minimization. arXiv:1710.09412 (2017).10.1007/978-1-4899-7687-1_79 ↗
10. Goodfellow, I. et al. Generative adversarial networks[J]. Commun. ACM 63, 139–144 (2020).10.1145/3422622 ↗
11. Dalal, N. & Triggs, B. Histograms of oriented gradients for human detection. In 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Vol. 1, 886–893 (IEEE, 2005).10.1109/CVPR.2005.177 ↗
12. Liu, M., Liu, G., Liu, Y. & Jiao, Q. Oracle bone inscriptions recognition based on deep convolutional neural network. J. Image Graph. 8, 114–119 (2020).10.18178/joig.8.4.114-119 ↗
13. Simonyan, K. & Zisserman, A. Very deep convolutional networks for large-scale image recognition. arXiv:1409.1556 (2014).
14. He, K., Zhang, X., Ren, S. & Sun, J. Deep residual learning for image recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 770–778 (2016).10.1109/CVPR.2016.90 ↗
15. Iandola, F. N. Squeezenet: Alexnet-level accuracy with 50× fewer parameters and <0.5 mb model size. arXiv:1602.07360 (2016).
16. Krizhevsky, A., Sutskever, I. & Hinton, G. E. Imagenet classification with deep convolutional neural networks. Adv. Neural Inf. Process. Syst. 25, (2012).
17. Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J. & Wojna, Z. Rethinking the inception architecture for computer vision. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2818–2826 (2016).10.1109/CVPR.2016.308 ↗
18. Zhang, Y., Zhang, H., Liu, Y., Yang, Q. & Liu, C. Oracle character recognition by nearest neighbor classification with deep metric learning. In 2019 International Conference on Document Analysis and Recognition, 309–314 (2019).10.1109/ICDAR.2019.00057 ↗
19. Huang, G., Liu, Z., Van Der Maaten, L. & Weinberger, K. Q. Densely connected convolutional networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 4700–4708 (2017).10.1109/CVPR.2017.243 ↗
20. Li, J., Wang, Q., Zhang, R. & Huang, K. Mix-up augmentation for oracle character recognition with imbalanced data distribution. In Document Analysis and Recognition—ICDAR 2021: 16th International Conference, Lausanne, Switzerland, Proceedings, Part I 16, 237–251 (Springer International Publishing, 2021).10.1007/978-3-030-86549-8_16 ↗
21. Yue, X., Li, H., Fujikawa, Y. & Meng, L. Dynamic dataset augmentation for deep learning-based oracle bone inscriptions recognition. ACM J. Comput. Cult. Herit. 15, 1–20 (2022).10.1145/3532868 ↗
22. Tong, Z., Song, Y., Wang, J. & Wang, L. Videomae: masked autoencoders are data-efficient learners for self-supervised video pre-training. Adv. Neural Inf. Process. Syst. 35, 10078–10093 (2022).
23. Hou Z. et al. Graphmae: self-supervised masked graph autoencoders. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, 594–604 (2022).10.1145/3534678.3539321 ↗
24. Pang Y. et al. Masked autoencoders for point cloud self-supervised learning. In European Conference on Computer Vision, 604–621 (2022).10.1007/978-3-031-20086-1_35 ↗
25. Jacob D. et al. Bert: pre-training of deep bidirectional transformers for language understanding. In Proceedings of NAACL-HLT, 4171–4186 (2019).
26. He K. et al. Masked autoencoders are scalable vision learners. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 16000–16009 (2022).10.1109/CVPR52688.2022.01553 ↗
27. Bao, H., Dong, L., Piao, S. & Wei, F. Beit: Bert pre-training of image transformers. In International Conference on Learning Representations (ICLR, 2022).
28. Dosovitskiy, A. An image is worth 16x16 words: transformers for image recognition at scale. In International Conference on Learning Representations (ICLR, 2021).
29. Xie Z. et al. Simmim: A simple framework for masked image modeling. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 9653–9663 (2022).10.1109/CVPR52688.2022.00943 ↗
30. Xie J. et al. Masked frequency modeling for self-supervised visual pre-training. In International Conference on Learning Representations (ICLR, 2023).
31. Chen, T., Kornblith, S., Norouzi, M. & Hinton, G. A simple framework for contrastive learning of visual representations. In International Conference on Machine Learning, 1597–1607 (2020).
32. Chen, X. & He, K. Exploring simple Siamese representation learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 15750–15758 (2021).10.1109/CVPR46437.2021.01549 ↗
33. Chen, X., Xie, S. & He, K. An empirical study of training self-supervised vision transformers. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 9640–9649 (2021).10.1109/ICCV48922.2021.00950 ↗
34. Bardes, A., Ponce, J. & LeCun, Y. Vicreg: variance–invariance–covariance regularization for self-supervised learning. arXiv:2105.04906 (2021).
35. Zbontar, J., Jing, L., Misra, I., LeCun, Y. & Deny, S. Barlow twins: self-supervised learning via redundancy reduction. In International Conference on Machine Learning, 12310–12320 (2021).
36. Huang Z. et al. Contrastive masked autoencoders are stronger vision learners. IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 46, no. 4, 2506–2517 (2024).10.1109/TPAMI.2023.3336525 ↗
37. Zhang X. et al. Videolt: Large-scale long-tailed video recognition. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 7960–7969 (2021).10.1109/ICCV48922.2021.00786 ↗
38. Tsung Y. L. et al. Focal loss for dense object detection. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 2980–2988 (2017).
39. Hsieh, T., Robb, E., Chen, H. & Huang, J. Droploss for long-tail instance segmentation. In Proceedings of the AAAI Conference on Artificial Intelligence, 1549–1557 (2021).10.1609/aaai.v35i2.16246 ↗
40. Chen, J., Agarwal, A., Abdelkarim, S., Zhu, D. & Elhoseiny, M. Reltransformer: Balancing the visual relationship detection from local context, scene and memory. arXiv:2104.119341 (2021).
41. Wang, Y., Gan, W., Yang, J., Wu, W. & Yan, J. Dynamic curriculum learning for imbalanced data classification. In Proceedings of the IEEE/CVF International Conference Computer Vision, 5017–5026 (2019).10.1109/ICCV.2019.00512 ↗
42. Hu X. et al. Learning to segment the tail. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 14045–14054 (2020).10.1109/CVPR42600.2020.01406 ↗
43. Ren, J., Yu, C., Ma, X., Zhao, H. & Yi, S. Balanced meta-softmax for long-tailed visual recognition. Adv. Neural Inf. Process. Syst. 33, 4175–4186 (2020).
44. Wang J. et al. Seesaw loss for long-tailed instance segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 9695–9704 (2021).10.1109/CVPR46437.2021.00957 ↗
45. Kang B. et al. Decoupling representation and classifier for long-tailed recognition. In International Conference on Learning Representations (2020).
46. Zhong, Z., Cui, J., Liu, S. & Jia, J. Improving calibration for long-tailed recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 16489–16498 (2021).10.1109/CVPR46437.2021.01622 ↗
47. Oord, A. v. d., Li, Y. & Vinyals, O. Representation learning with contrastive predictive coding. arXiv:1807.03748 (2018).
48. Liu Z. et al. Large-scale long-tailed recognition in an open world. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2537–2546 (2019).10.1109/CVPR.2019.00264 ↗
49. Szegedy C. et al. Going deeper with convolutions. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 1–9 (2015).10.1109/CVPR.2015.7298594 ↗