A text image dual conditional stable diffusion model for oracle bone inscription decipherment
Source abstract
Abstract
Oracle Bone Inscriptions (OBI), the earliest systematic writing in China, are crucial for understanding early Chinese civilization. However, many inscriptions remain undeciphered due to limited data, complex glyphs, and weak semantic consistency. Although Generative Adversarial Networks (GAN) and Diffusion Models have introduced new possibilities to the field, most existing methods primarily focus on visual features and lack semantic integration. To address this challenge, we propose DCSD-OBI, a Dual-Conditional Stable Diffusion model that integrates OBI images and modern Chinese text during reverse diffusion to jointly learn structural and semantic features, thereby improving character generation accuracy. To reduce training cost while preserving generative quality, we adopt Low-Rank Adaptation (LoRA) to fine-tune only the U-Net’s cross-attention modules. Furthermore, we replace CLIP with Chinese-CLIP, a version tailored for the Chinese language, to improve cross-modal semantic consistency. Experiments results show DCSD-OBI improves decipherment accuracy by 11%, highlighting its effectiveness and potential for advancing OBI research.
Source: openalex_inverted_index · Quality: baseline
Research context
OpenAlex topics: Generative Adversarial Networks and Image Synthesis · Handwritten Text Recognition Techniques · Image Processing and 3D Reconstruction
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
References (37)
Show source-provided references
1. Christina, C. The chinese history that is written in bone. Am. Scientist 106, 133 (2018).10.1511/2018.106.3.133 ↗
2. Wang, M. & Deng, W. A dataset of oracle characters for benchmarking machine learning algorithms. Sci. Data 11, 87 (2024).10.1038/s41597-024-02933-w ↗
3. Li, Q., Yang, Y. & Wang, A. Recognition of inscriptions on bones or tortoise shells based on graph isomorphism. Jisuanji Gongcheng yu Yingyong(Computer Eng. Appl.) 47, 112–114 (2011).
4. Guo, J. et al. Building hierarchical representations for oracle character and sketch recognition. IEEE Trans. Image Process. 25, 104–118 (2016).10.1109/TIP.2015.2500019 ↗
5. Liu, J. et al. OBI-CMF: Self-supervised learning with contrastive masked frequency modeling for oracle bone inscription recognition. npj Herit. Sci. 13, 55 (2025).10.1038/s40494-025-01644-8 ↗
6. Huang, S. et al. Obc306: A large-scale oracle bone character recognition dataset. In 2019 International Conference on Document Analysis and Recognition (ICDAR), 681–688 (2019).10.1109/ICDAR.2019.00114 ↗
7. Wang, M., Deng, W. & Liu, C.-L. Unsupervised structure-texture separation network for oracle character recognition. IEEE Trans. Image Process. 31, 3137–3150 (2022).10.1109/TIP.2022.3165989 ↗
8. Yi-Kang, Z., Heng, Z., Yong-Ge, L. & Cheng-Lin, L. Oracle character recognition based on cross-modal deep metric learning. Acta Autom. Sin. 47, 791–800 (2021).
9. Li, J. et al. Towards better long-tailed oracle character recognition with adversarial data augmentation. Pattern Recogn. 140, 109534 (2023).10.1016/j.patcog.2023.109534 ↗
10. Wang, W. et al. Improving oracle bone characters recognition via a cyclegan-based data augmentation method. In International Conference on Neural Information Processing, 88–100 (Springer Nature Singapore, Singapore, 2022).10.1007/978-981-99-1645-0_8 ↗
11. Huang, H. et al. Agtgan: Unpaired image translation for photographic ancient character generation. In Proc. 30th ACM Int. Conf. Multimedia, 5456–5467 (2022).10.1145/3503161.3548338 ↗
12. Gao, J. et al. Oracle bone heritage data augmentation based on two-stage decomposition gans. npj Herit. Sci. 13, 230 (2025).10.1038/s40494-025-01774-z ↗
13. Zhang, G., Liu, D., Smyth, B. & Dong, R. Deciphering ancient chinese oracle bone inscriptions using case-based reasoning. In International Conference on Case-Based Reasoning, 309–324 (Springer, 2021).10.1007/978-3-030-86957-1_21 ↗
14. Li, F. & WOO, P.-Y. The coding principle and method for automatic recognition of jia gu wen characters. Int. J. Hum.-Computer Stud. 53, 289–299 (2000).10.1006/ijhc.2000.0388 ↗
15. Zhou, X.-L., Hua, X.-C. & Li, F. A method of jia gu wen recognition based on a two-level classification. In Proceedings of 3rd International Conference on Document Analysis and Recognition, vol. 2, 833–836 (IEEE, 1995).10.1109/ICDAR.1995.602030 ↗
16. Gao, F., Xiong, J. & Liu, Y. G. Recognition of fuzzy characters on oracle-bone inscriptions. In 2015 IEEE International Conference on Computer and Information Technology; Ubiquitous Computing and Communications; Dependable, Autonomic and Secure Computing; Pervasive Intelligence and Computing, 698–702 (IEEE, 2015).10.1109/CIT/IUCC/DASC/PICOM.2015.101 ↗
17. Chang, X., Chao, F., Shang, C. & Shen, Q. Sundial-gan: A cascade generative adversarial networks framework for deciphering oracle bone inscriptions. In Proceedings of the 30th ACM International Conference on Multimedia, MM ’22, 1195-1203 (Association for Computing Machinery, New York, NY, USA, 2022).10.1145/3503161.3547925 ↗
18. Guan, H. et al. Deciphering oracle bone language with diffusion models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics: Volume 1 (Long Papers), 15554–15567 (2024).10.18653/v1/2024.acl-long.831 ↗
19. Li, B. et al. Oracle bone inscriptions multi-modal dataset. (2024). 2407.03900.10.1007/978-981-97-3504-4_15 ↗
20. Rombach, R. et al. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 10684–10695 (2022).10.1109/CVPR52688.2022.01042 ↗
21. Hu, E. J. et al. LoRA: Low-rank adaptation of large language models. In International Conference on Learning Representations https://openreview.net/forum?id=nZeVKeeFYf9 (2022).
22. Radford, A. et al. Learning transferable visual models from natural language supervision. In Proceedings of the 38th International Conference on Machine Learning, vol. 139, 8748–8763 https://proceedings.mlr.press/v139/radford21a.html (2021).
23. Yang, A. et al. Chinese clip: Contrastive vision-language pretraining in chinese. (2023).
24. Zhang, L., Rao, A. & Agrawala, M. Adding conditional control to text-to-image diffusion models. In 2023 IEEE/CVF International Conference on Computer Vision (ICCV), 3813–3824 (2023).10.1109/ICCV51070.2023.00355 ↗
25. Kingma, D. P. & Welling, M. Auto-encoding variational bayes. In Bengio, Y. & LeCun, Y. (eds.) 2nd International Conference on Learning Representations, ICLR 2014, Banff, AB, Canada, April 14-16, 2014, Conference Track Proceedings (2014).
26. Zhao, J., Schäfer, F. & Anandkumar, A. Zero initialization: Initializing residual networks with only zeros and ones. (2022). 2110.12661.
27. Wang, P. et al. An open dataset for oracle bone script recognition and decipherment. (2024).10.1038/s41597-024-03807-x ↗
28. Guan, H. et al. An open dataset for the evolution of oracle bone characters: EVOBC. (2024).
29. Cui, C. et al. PaddleOCR 3.0 technical report. arXiv preprint arXiv:2507.05595 https://arxiv.org/abs/2507.05595. Code available at https://github.com/PaddlePaddle/PaddleOCR (2025).
30. Song, J., Meng, C. & Ermon, S. Denoising diffusion implicit models. In International Conference on Learning Representations https://openreview.net/forum?id=St1giarCHLP (2021).
31. Liu, L., Ren, Y., Lin, Z. & Zhao, Z. Pseudo numerical methods for diffusion models on manifolds. In International Conference on Learning Representations https://openreview.net/forum?id=PlKWVd2yBkY (2022).
32. Ho, J., Jain, A. & Abbeel, P. Denoising diffusion probabilistic models. Adv. neural Inf. Process. Syst. 33, 6840–6851 (2020).
33. Saharia, C. et al. Palette: Image-to-image diffusion models. In ACM SIGGRAPH 2022 Conference Proceedings, 1–10 (2022).10.1145/3528233.3530757 ↗
34. Zhu, J.-Y., Park, T., Isola, P. & Efros, A. A. Unpaired image-to-image translation using cycle-consistent adversarial networks. In 2017 IEEE International Conference on Computer Vision (ICCV), 2242–2251 (2017).10.1109/ICCV.2017.244 ↗
35. Li, B., Xue, K., Liu, B. & Lai, Y.-K. Bbdm: Image-to-image translation with brownian bridge diffusion models. In 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 1952–1961 (2023).10.1109/CVPR52729.2023.00194 ↗
36. Dhariwal, P. & Nichol, A. Diffusion models beat gans on image synthesis. In Advances in Neural Information Processing Systems, 8780–8794 (2021).
37. Xiao, B., Wang, P., He, Q. & Dong, M. Deterministic image-to-image translation via denoising brownian bridge models with dual approximators. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 28232–28241 (2025).10.1109/CVPR52734.2025.02629 ↗