Ladder-side mixture of experts adapters for bronze inscription recognition
Source abstract
Abstract
Bronze inscriptions (BI), engraved on ritual vessels, constitute a crucial stage of early Chinese writing and provide indispensable evidence for archeological and historical studies. However, automatic BI recognition is challenging due to visual degradation, cross-domain variability among photographs, rubbings, and tracings, and an extremely long-tailed character distribution. To address these challenges, we curate a large-scale BI dataset comprising 22,454 full-page images and 198,598 annotated characters spanning 6658 unique categories, enabling robust cross-domain evaluation. Building on this resource, we develop a two-stage detection-recognition pipeline. To handle heterogeneous domains and rare classes, we equip the pipeline with LadderMoE, which augments a pretrained CLIP encoder with ladder-style MoE adapters for dynamic expert specialization and enhanced robustness. Comprehensive experiments demonstrate that our method substantially outperforms the state-of-the-art scene text recognition baselines, achieving superior accuracy across head, mid, and tail categories as well as all acquisition modalities, establishing a strong foundation for downstream archeological analysis.
Source: openalex_inverted_index · Quality: baseline
Research context
OpenAlex topics: Image Processing and 3D Reconstruction · Handwritten Text Recognition Techniques · Currency Recognition and Detection
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
References (47)
Show source-provided references
1. Guo, R. A research on an intelligent recognition tool for bronze inscriptions of the Shang and Zhou dynasties. J. Chin. Writ. Syst. 4, 271–279 (2020).10.1177/2513850220967756 ↗
2. Egorov, A., Egorova, M. & Orlova, T. The use of a comparative analysis of the connection between ancient and modern Chinese languages in the process of teaching students Chinese characters. In Proc. 2nd International Conference on Education: Current Issues and Digital Technologies (ICECIDT 2022) 10–19 (Atlantis Press, 2022).10.2991/978-2-494069-02-2_3 ↗
3. Wolfgang, B. The language of the bronze inscriptions. Imprints of Kinship: Studies of Recently Discovered Bronze Inscriptions from Ancient China 9 (Chinese University Press, 2017).10.2307/j.ctv2n7r6s.6 ↗
4. Liu, Z. et al. Oracle character detection based on improved faster R-CNN. In Proc. International Conference on Intelligent Transportation, Big Data & Smart City (ICITBS) 697–700 (IEEE, 2021).10.1109/ICITBS53129.2021.00175 ↗
5. Fu, X., Zhou, R., Yang, X. & Li, C. Detecting oracle bone inscriptions via pseudo-category labels. npj Herit. Sci. 12, 107 (2024).10.1186/s40494-024-01221-5 ↗
6. Tao, Y., Fu, X., Pang, H., Yang, X. & Li, C. Clustering-based feature representation learning for oracle bone inscriptions detection. npj Herit. Sci. 13, 296 (2025).10.1038/s40494-025-01850-4 ↗
7. Meng, L. Recognition of oracle bone inscriptions by extracting line features on image processing. In Proc. 6th International Conference on Pattern Recognition Applications and Methods 606–611 (IEEE, 2017).10.5220/0006225706060611 ↗
8. Li, W.-Y., Cao, B., Cao, C.-S. & Huang, Y.-Z. A deep learning based method for bronze inscription recognition. Acta Autom. Sin. 44, 2023–2030 (2018).
9. Zhang, Y.-K., Zhang, H., Liu, Y.-G., Yang, Q. & Liu, C.-L. Oracle character recognition by nearest neighbor classification with deep metric learning. In Proc. International Conference on Document Analysis and Recognition (ICDAR) 309–314 (IEEE, 2019).10.1109/ICDAR.2019.00057 ↗
10. Zheng, Y., Chen, Y., Wang, X., Qi, D. & Yan, Y. Ancient chinese character recognition with improved swin-transformer and flexible data enhancement strategies. Sensors 24, 2182 (2024).10.3390/s24072182 ↗
11. Xia, G. & Shang, Z. Bronze inscription recognition method basedon automatic pruning strategy. Laser Optoelectron. Prog. 57, 257–264 (2020).
12. Bluche, T., Louradour, J. & Messina, R. Scan, attend and read: End-to-end handwritten paragraph recognition with MLDSTM attention. In 2017 14th IAPR International Conference on Document Analysis and Recognition (ICDAR) Vol. 1, 1050–1055 (IEEE, 2017).10.1109/ICDAR.2017.174 ↗
13. Cui, Y., Jia, M., Lin, T.-Y., Song, Y. & Belongie, S. Class-balanced loss based on effective number of samples. In Proc. Conference on Computer Vision and Pattern Recognition 9268–9277 (IEEE, 2019).10.1109/CVPR.2019.00949 ↗
14. Zhang, Y., Kang, B., Hooi, B., Yan, S. & Feng, J. Deep long-tailed learning: a survey. IEEE Trans. Pattern Anal. Mach. Intell. 45, 10795–10816 (2023).10.1109/TPAMI.2023.3268118 ↗
15. Bautista, D. & Atienza, R. Scene text recognition with permuted autoregressive sequence models. In European Conference on Computer Vision 178–196 (Springer, 2022).10.1007/978-3-031-19815-1_11 ↗
16. Baek, J. et al. What is wrong with scene text recognition model comparisons? dataset and model analysis. In Proc. International Conference on Computer Vision 4715–4723 (IEEE, 2019).10.1109/ICCV.2019.00481 ↗
17. Baek, J., Matsui, Y. & Aizawa, K. What if we only use real datasets for scene text recognition? toward scene text recognition with fewer labels. In Proc. Conference on Computer Vision and Pattern Recognition (CVPR) https://doi.org/10.1109/cvpr46437.2021.00313 (2021).10.1109/cvpr46437.2021.00313 ↗
18. Bhunia, A. K., Chowdhury, P. N., Sain, A. & Song, Y.-Z. Towards the unseen: iterative text recognition by distilling from errors. In Proc. International Conference on Computer Vision 14950–14959 (IEEE, 2021).10.1109/ICCV48922.2021.01468 ↗
19. Atienza, R. Vision transformer for fast and efficient scene text recognition. In Proc. International Conference on Document Analysis and Recognition 319–334 (Springer, 2021).10.1007/978-3-030-86549-8_21 ↗
20. Aberdam, A. et al. Sequence-to-sequence contrastive learning for text recognition. In Proc. Conference on Computer Vision and Pattern Recognition 15302–15312 (IEEE, 2021).10.1109/CVPR46437.2021.01505 ↗
21. Luo, C., Jin, L. & Chen, J. Siman: exploring self-supervised representation learning of scene text via similarity-aware normalization. In Proc. Conference on Computer Vision and Pattern Recognition 1039–1048 (IEEE, 2022).10.1109/CVPR52688.2022.00111 ↗
22. Graves, A., Fernández, S., Gomez, F. & Schmidhuber, J. Connectionist temporal classification: labelling unsegmented sequence data with recurrent neural networks. In Proc. 23rd International Conference On Machine Learning 369–376 (ACM, 2006).10.1145/1143844.1143891 ↗
23. He, P., Huang, W., Qiao, Y., Loy, C. C. & Tang, X. Reading scene text in deep convolutional sequences. In Proc. Thirtieth AAAI Conference on Artificial Intelligence (AAAI-16) 3501–3508 (AAAI, 2016).10.1609/aaai.v30i1.10465 ↗
24. Shi, B., Bai, X. & Yao, C. An end-to-end trainable neural network for image-based sequence recognition and its application to scene text recognition. IEEE Trans. pattern Anal. Mach. Intell. 39, 2298–2304 (2016).10.1109/TPAMI.2016.2646371 ↗
25. Borisyuk, F., Gordo, A. & Sivakumar, V. Rosetta: large scale system for text detection and recognition in images. In Proc. 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining 71–79 (ACM, 2018).10.1145/3219819.3219861 ↗
26. Liao, M. et al. Scene text recognition from two-dimensional perspective. In Proc. AAAI Conference on Artificial Intelligence Vol. 33, 8714–8721 (AAAI, 2019).10.1609/aaai.v33i01.33018714 ↗
27. Wan, Z., He, M., Chen, H., Bai, X. & Yao, C. Textscanner: reading characters in order for robust scene text recognition. In Proc. AAAI Conference on Artificial Intelligence Vol. 34, 12120–12127 (AAAI, 2020).10.1609/aaai.v34i07.6891 ↗
28. Cheng, Z. et al. Focusing attention: towards accurate text recognition in natural images. In Proc. International Conference on Computer Vision 5076–5084 (IEEE, 2017).10.1109/ICCV.2017.543 ↗
29. Shi, B. et al. Aster: an attentional scene text recognizer with flexible rectification. IEEE Trans. Pattern Anal. Mach. Intell. 41, 2035–2048 (2018).10.1109/TPAMI.2018.2848939 ↗
30. Fang, S., Xie, H., Wang, Y., Mao, Z. & Zhang, Y. Read like humans: autonomous, bidirectional and iterative language modeling for scene text recognition. In Proc. Conference on Computer Vision And Pattern Recognition 7098–7107 (IEEE, 2021).10.1109/CVPR46437.2021.00702 ↗
31. Wang, Z. et al. Symmetrical linguistic feature distillation with clip for scene text recognition. In Proc. 31st ACM International Conference on Multimedia 509–518 (ACM, 2023).10.1145/3581783.3611769 ↗
32. Zhao, S., Quan, R., Zhu, L. & Yang, Y. Clip4str: a simple baseline for scene text recognition with pre-trained vision-language model. IEEE Transactions on Image Processing (ACM, 2024).10.1109/TIP.2024.3512354 ↗
33. Litman, R. et al. Scatter: selective context attentional scene text recognizer. In Proc. Conference on Computer Vision and Pattern Recognition 11962–11972 (IEEE, 2020).10.1109/CVPR42600.2020.01198 ↗
34. Wang, L. et al. Parameter-efficient fine-tuning in large language models: a survey of methodologies. Artif. Intell. Rev. 58, 227 (2025).10.1007/s10462-025-11236-4 ↗
35. Zhang, Q. et al. Adaptive budget allocation for parameter-efficient fine-tuning. In Proc. The Eleventh International Conference on Learning Representations https://openreview.net/forum?id=lq62uWRJjiY (2023).
36. Li, X. L. & Liang, P. Prefix-tuning: optimizing continuous prompts for generation. In Proc. 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) 4582–4597 (Association for Computational Linguistics, 2021).10.18653/v1/2021.acl-long.353 ↗
37. Gao, P. et al. Clip-adapter: Better vision-language models with feature adapters. Int. J. Comput. Vis. 132, 581–595 (2024).10.1007/s11263-023-01891-x ↗
38. Hu, E. J. et al. LoRA: low-rank adaptation of large language models. In International Conference on Learning Representations https://openreview.net/forum?id=nZeVKeeFYf9 (2022).
39. Lester, B., Al-Rfou, R. & Constant, N. The power of scale for parameter-efficient prompt tuning. In Proc. Conference on Empirical Methods in Natural Language Processing, 3045–3059 (Association for Computational Linguistics, 2021).10.18653/v1/2021.emnlp-main.243 ↗
40. Jacobs, R. A., Jordan, M. I., Nowlan, S. J. & Hinton, G. E. Adaptive mixtures of local experts. Neural Comput. 3, 79–87 (1991).10.1162/neco.1991.3.1.79 ↗
41. Shi, C. et al. Unchosen experts can contribute too: Unleashing moe models’ power by self-contrast. Adv. Neural Inf. Process. Syst. 37, 136897–136921 (2024).
42. Zhou, Y. et al. Mixture-of-experts with expert choice routing. Adv. Neural Inf. Process. Syst. 35, 7103–7114 (2022).
43. Jiang, A. Q. et al. Mixtral of experts. Preprint at https://doi.org/10.48550/arXiv.2401.04088 (2024).10.48550/arXiv.2401.04088 ↗
44. Riquelme, C. et al. Scaling vision with sparse mixture of experts. Adv. Neural Inf. Process. Syst. 34, 8583–8595 (2021).
45. Fedus, W., Zoph, B. & Shazeer, N. Switch transformers: scaling to trillion parameter models with simple and efficient sparsity. J. Mach. Learn. Res. 23, 1–39 (2022).
46. Tian, Y., Ye, Q. & Doermann, D. YOLOv12: Attention-centric real-time object detectors. In: The Thirty-ninth Annual Conference on Neural Information Processing Systems (2025)
47. Shaughnessy, E. L. Sources of Western Zhou History: Inscribed Bronze Vessels (Univ. of California Press, 1992).10.1525/9780520910225 ↗