journal article·2026-05-12Open access

A Multi-Stage Hybrid Retrieval Framework for the Scientific Literature with Cross-Encoder Re-Ranking

Walaa Al-Joofi, Alaa Sagheer, Hala Hamdoun

Applied Sciences· Volume 16· Issue 10

Abstract

Effective scientific literature retrieval requires moving beyond surface-level term matching toward structured semantic reasoning. This paper presents a controlled empirical study of multi-stage retrieval for scientific literature, integrating lexical matching, dense semantic modeling, hybrid fusion, and cross-encoder re-ranking within a unified evaluation framework. The study is designed to analyze the interactions, trade-offs, and failure modes of these components in claim-based scientific search. Experiments on the SciFact benchmark demonstrate that dense models capture semantic similarity but remain insufficient when used in isolation. Hybrid fusion broadens the candidate pool but does not consistently outperform the best standalone dense retriever, as RRF-based fusion can dilute strong dense rankings when lexical and semantic signals diverge. Cross-encoder re-ranking proves to be the primary driver of final performance gains, with the best configuration, Hybrid (SciNCL + BM25) + Cross-Encoder, reaching NDCG@10 of 0.523, MAP@10 of 0.479, Recall@10 of 0.642, and MRR@10 of 0.497. Ablation analysis shows that lexical pseudo-relevance feedback (RM3) introduces query drift in claim-focused retrieval, and that passage-level max pooling weakens effectiveness by fragmenting document-level evidence. Cross-domain evaluation on SciFact, PubMedQA, and SciDocs demonstrates that the relative ranking of retrieval paradigms remains stable across datasets with varying difficulty levels, while also revealing that the RRF dilution effect intensifies on harder retrieval tasks. These findings suggest that effective scientific retrieval benefits from integrated multi-stage pipelines, and that understanding component-level interactions is essential for designing robust retrieval systems.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: Information Retrieval and Search Behavior · Biomedical Text Mining and Ontologies · Topic Modeling

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

Department of Computer Science, College of Computer Sciences and Information Technology, King Faisal University, Hufof 36362, Al Ahsa, Saudi Arabia

Funding

King Faisal University

References (29)

Show source-provided references
1. An ontology-improved vector space model for semantic retrieval10.1108/EL-04-2020-0081 ↗
2. Salsabilla, N., and Wiharja, K. (2025, January 3–4). Implementation of Semantic Search Based on Vector Database for Personal Documents. Proceedings of the 2025 International Conference on Advancement in Data Science, E-Learning and Information System (ICADEIS), Bandung, Indonesia.10.1109/ICADEIS65852.2025.10933119 ↗
3. Prospects of Retrieval-Augmented Generation (RAG) for Academic Library Search and Retrieval
4. Wadden, D., Lin, S., Lo, K., Wang, L.L., van Zuylen, M., Cohan, A., and Hajishirzi, H. (2020). Fact or Fiction: Verifying Scientific Claims. Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), Association for Computational Linguistics.10.18653/v1/2020.emnlp-main.609 ↗
5. Thakur, N., Garg, N., Wallace, E., Roberts, A., Raffel, C., and Dehghani, M. (2021, January 6–14). BEIR: A Heterogeneous Benchmark for Zero-Shot Evaluation of Information Retrieval Models. Proceedings of the 35th Conference on Neural Information Processing Systems (NeurIPS), Online.
6. Manning, C.D., Raghavan, P., and Schütze, H. (2008). Introduction to Information Retrieval, Cambridge University Press.10.1017/CBO9780511809071 ↗
7. The probabilistic relevance framework: BM25 and beyond
8. Legal Information Retrieval and Entailment Based on BM25, Transformer and Semantic Thesaurus Methods10.1007/s12626-022-00103-1 ↗
9. Neural embedding-based indices for semantic search10.1016/j.ipm.2018.10.015 ↗
10. Yang, W., Lu, K., Yang, P., and Lin, J. (2019, January 21–25). Critically Examining the “Neural Hype”: Weak Baselines and the Additivity of Effectiveness Gains from Neural Ranking Models. Proceedings of the 42nd International ACM SIGIR Conference, Paris, France.10.1145/3331184.3331340 ↗
11. Advances in Transformer-Based Semantic Search: Techniques, Benchmarks, and Future Directions10.47000/tjmcs.1633092 ↗
12. Beltagy, I., Lo, K., and Cohan, A. (2019). SciBERT: A Pretrained Language Model for Scientific Text. arXiv.10.18653/v1/D19-1371 ↗
13. Cohan, A., Feldman, S., Beltagy, I., Downey, D., and Weld, D. (2020). SPECTER: Document-Level Representation Learning Using Citation-Informed Transformers. arXiv.10.18653/v1/2020.acl-main.207 ↗
14. Ostendorff, M., Rethmeier, N., Augenstein, I., Gipp, B., and Rehm, G. (2022). Neighborhood Contrastive Learning for Scientific Document Representations with Citation Embeddings. arXiv.10.18653/v1/2022.emnlp-main.802 ↗
15. Billion-Scale Similarity Search with GPUs10.1109/TBDATA.2019.2921572 ↗
16. Zhang, K., Tao, C., Shen, T., Xu, C., Geng, X., Jiao, B., and Jiang, D. (2022). LED: Lexicon-Enlightened Dense Retriever for Large-Scale Retrieval. arXiv.10.1145/3543507.3583294 ↗
17. Cormack, G.V., Clarke, C.L.A., and Buettcher, S. (2009, January 19–23). Reciprocal rank fusion outperforms condorcet and individual rank learning methods. Proceedings of the 32nd International ACM SIGIR Conference on Research and Development in Information Retrieval, Boston, MA, USA.10.1145/1571941.1572114 ↗
18. Pezzuti, F., MacAvaney, S., and Tonellotto, N. (2025). Exploring the Effectiveness of Multi-stage Fine-tuning for Cross-encoder Re-rankers. arXiv.10.1007/978-3-031-88714-7_45 ↗
19. Sager, P.J., Kamaraj, A., Grewe, B.F., and Stadelmann, T. (2025). Deep Retrieval at CheckThat! 2025: Identifying Scientific Papers from Implicit Social Media Mentions via Hybrid Retrieval and Re-Ranking. arXiv.10.1109/ResgenXAI64788.2025.11344015 ↗
20. Zhang, Y., Long, D., Xu, G., and Xie, P. (2022, January 22–27). HLATR: Enhance Multi-stage Text Retrieval with Hybrid List Aware Transformer Reranking. Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (ACL), Dublin, Ireland.
21. Reimers, N., and Gurevych, I. (2019). Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing (EMNLP). Association for Computational Linguistics, Association for Computational Linguistics.10.18653/v1/D19-1410 ↗
22. The Faiss Library10.1109/TBDATA.2025.3618474 ↗
23. MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformers
24. Wang, L., Yang, N., Huang, X., Jiao, B., Yang, L., Jiang, D., Majumder, R., and Wei, F. (2022). Text Embeddings by Weakly-Supervised Contrastive Pre-training. arXiv.
25. Xiao, S., Liu, Z., Zhang, P., and Muennighoff, N. (2023). C-Pack: Packaged Resources to Advance General Chinese Embedding. arXiv.
26. Li, Z., Zhang, X., Zhang, Y., Long, D., Xie, P., and Zhang, M. (2023). Towards General Text Embeddings with Multi-stage Contrastive Learning. arXiv.
27. Wolf, T., Debut, L., Sanh, V., Chaumond, J., Delangue, C., Moi, A., Cistac, P., Rault, T., Louf, R., and Funtowicz, M. (2020, January 16–20). Transformers: State-of-the-Art Natural Language Processing. Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, Online.10.18653/v1/2020.emnlp-demos.6 ↗
28. Dai, Z., and Callan, J. (2019, January 21–25). Deeper Text Understanding for IR with Contextual Neural Language Modeling. Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval, Paris, France.10.1145/3331184.3331303 ↗
29. Jin, Q., Dhingra, B., Liu, Z., Cohen, W.W., and Lu, X. (2019). PubMedQA: A Dataset for Biomedical Research Question Answering. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), Association for Computational Linguistics.10.18653/v1/D19-1259 ↗