journal article·2025-03-20Open access

Text Alignment in the Service of Text Reuse Detection

Hadar Miller, Tsvi Kuflik, Moshe Lavee

Applied Sciences· Volume 15· Issue 6

Abstract

This study introduces a novel approach to text alignment tailored for ancient languages, with a focus on Hebrew and Aramaic, aimed at enhancing text reuse detection. Unlike previous methods, our approach integrates multiple NLP components into a specialized comparison pipeline, which is then incorporated into the Smith–Waterman algorithm. This integration enables improved alignment accuracy, particularly for historical texts characterized by fluctuations, orthographic changes, transcription variations, and word transpositions. Our key contributions include (1) a refined distance function that integrates fastText embeddings, allowing robust handling of out-of-vocabulary words; (2) a typological correction mechanism that can be integrated into automatic transcription pipelines to enhance text normalization; and (3) an evaluation of historical Hebrew texts, demonstrating an 11% improvement in the F1 score over existing approaches. These findings underscore the importance of computational methodologies in digital humanities and lay the groundwork for future multilingual extensions.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: Natural Language Processing Techniques · Digital Humanities and Scholarship · Web Data Mining and Analysis

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

Department of Information Systems, University of Haifa, 99 Aba Hushi St., Haifa 3498838, Israel

Department of Jewish History, University of Haifa, 99 Aba Hushi St., Haifa 3498838, Israel

Funding

Israel Ministry of Science and Technology (MOST)

References (29)

Show source-provided references
1. Shmidman, A., Koppel, M., and Porat, E. (2018). Identification of parallel passages across a large Hebrew/Aramaic corpus. J. Data Min. Digit. Humanit.10.46298/jdmdh.1388 ↗
2. Potthast, M., Hagen, M., Gollub, T., Tippmann, M., Kiesel, J., Rosso, P., Stamatatos, E., and Stein, B. (2013, January 23–26). Overview of the 5th international competition on plagiarism detection. Proceedings of the CLEF Conference on Multilingual and Multimodal Information Access Evaluation, CELCT, Valencia, Spain.
3. Language resources for Hebrew10.1007/s10579-007-9050-8 ↗
4. Reconstruction of the Mekhilta deuteronomy using philological and computational tools
5. FAST: Fast and accurate synoptic texts10.1093/llc/fqz029 ↗
6. A general method applicable to the search for similarities in the amino acid sequence of two proteins10.1016/0022-2836(70)90057-4 ↗
7. Distributed representations of words and phrases and their compositionality
8. Binary codes capable of correcting deletions, insertions, and reversals
9. Identification of common molecular subsequences10.1016/0022-2836(81)90087-5 ↗
10. Enriching word vectors with subword information10.1162/tacl_a_00051 ↗
11. A survey of text alignment visualization10.1109/TVCG.2020.3028975 ↗
12. Colavizza, G., Infelise, M., and Kaplan, F. (2014, January 11). Mapping the early modern news flow: An enquiry by robust text reuse detection. Proceedings of the Social Informatics: SocInfo 2014 International Workshops, Barcelona, Spain. Revised Selected Papers 6.10.1007/978-3-319-15168-7_31 ↗
13. Haentjens Dekker, R., and Middell, G. (2011, January 17–18). Computer-Supported Collation with CollateX. Proceedings of the Supporting Digital Humanities, Copenhagen, Denmark.
14. Smith, D.A., Cordel, R., Dillon, E.M., Stramp, N., and Wilkerson, J. (2014, January 8–12). Detecting and modeling local text reuse. Proceedings of the IEEE/ACM Joint Conference on Digital Libraries, London, UK.10.1109/JCDL.2014.6970166 ↗
15. Clough, P., Gaizauskas, R., Piao, S.S., and Wilks, Y. (2002, January 6–12). Measuring text reuse. Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, Philadelphia, PA, USA.
16. Part-of-speech tagging of Modern Hebrew text10.1017/S135132490700455X ↗
17. Joint transition-based models for morpho-syntactic parsing: Parsing strategies for MRLs and a case study from modern Hebrew10.1162/tacl_a_00253 ↗
18. Machine learning for ancient languages: A survey10.1162/coli_a_00481 ↗
19. Shmidman, S., Shmidman, A., Koppel, M., and Tsarfaty, R. (2024). MRL Parsing Without Tears: The Case of Hebrew. arXiv.10.18653/v1/2024.findings-acl.269 ↗
20. Computer-supported collation of modern manuscripts: CollateX and the Beckett Digital Manuscript Project10.1093/llc/fqu007 ↗
21. Benbaji, I., Doron, O., and Hénot-Mortier, A. (2022). Word-Embeddings Distinguish Denominal and Root-Derived Verbs in Semitic. arXiv.10.4204/EPTCS.366.6 ↗
22. Sprugnoli, R., Passarotti, M., and Moretti, G. (2019, January 13–15). Vir is to moderatus as mulier is to intemperans-lemma embeddings for Latin. Proceedings of the CLiC-it, Bari, Italy.
23. Techniques for automatically correcting words in text10.1145/146370.146380 ↗
24. Performance comparison of OCR tools10.5121/iju.2015.6303 ↗
25. Manning, C.D. (2009). An Introduction to Information Retrieval, Cambridge University Press.
26. Milikovski, H. (2023, February 15). Vayikra Rabba. Available online: https://www2.biu.ac.il/JS/midrash/VR/.
27. (2023, June 01). The Friedberg Jewish Manuscript Society. Available online: https://fjms.genizah.org/.
28. Sefaria, I. (2022, January 15). A Living Library of Torah Texts Online. Available online: https://www.sefaria.org/texts.
29. User’s guide to correlation coefficients10.1016/j.tjem.2018.08.001 ↗