A Scientific Document Retrieval and Reordering Method by Incorporating HFS and LSD
Source abstract
Abstract
Achieving scientific document retrieval by considering the wealth of mathematical expressions and the semantic text they contain has become an inescapable trend. Current scientific document matching models focus solely on the textual features of expressions and frequently encounter hurdles like proliferative parameters and sluggish reasoning speeds in the pursuit of improved performance. To solve this problem, this paper proposes a scientific document retrieval method founded upon hesitant fuzzy sets (HFS) and local semantic distillation (LSD). Concretely, in order to extract both spatial and semantic features for each symbol within a mathematical expression, this paper introduces an expression analysis module that leverages HFS to establish feature indices. Secondly, to enhance contextual semantic alignment, the method of knowledge distillation is employed to refine the pretrained language model and establish a twin network for semantic matching. Lastly, by amalgamating mathematical expressions with contextual semantic features, the retrieval results can be made more efficient and rational. Experiments were implemented on the NTCIR dataset and the expanded Chinese dataset. The average MAP for mathematical expression retrieval results was 83.0%, and the average nDCG for sorting scientific documents was 85.8%.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Mathematics, Computing, and Information Processing · Topic Modeling · Image Retrieval and Classification Techniques
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
School of Cyber Security and Computer, Hebei University, Baoding 071002, China
Institute of Intelligent Image and Document Information Processing, Hebei University, Baoding 071002, China
Hebei Machine Vision Engineering Research Center, Hebei University, Baoding 071002, China
Funding
Natural Science Foundation of Hebei Province of China
References (34)
Show source-provided references
1. Mansouri, B., Zanibbi, R., and Oard, D.W. (2021, January 11–15). Learning to rank for mathematical formula retrieval. Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, Virtual.10.1145/3404835.3462956 ↗
2. Nishizawa, G., Liu, J., Diaz, Y., Dmello, A., Zhong, W., and Zanibbi, R. (2020, January 14–17). MathSeer: A math-aware search interface with intuitive formula editing, reuse, and lookup. Proceedings of the Advances in Information Retrieval: 42nd European Conference on IR Research—ECIR 2020, Lisbon, Portugal.10.1007/978-3-030-45442-5_60 ↗
3. Mallia, A., Siedlaczek, M., and Suel, T. (2019, January 14–18). An experimental study of index compression and DAAT query processing methods. Proceedings of the Advances in Information Retrieval: 41st European Conference on IR Research, ECIR 2019, Cologne, Germany.10.1007/978-3-030-15712-8_23 ↗
4. Ni, J., Ábrego, G.H., Constant, N., Ma, J., Hall, K.B., Cer, D., and Yang, Y. (2021). Sentence-t5: Scalable sentence encoders from pre-trained text-to-text models. arXiv.10.18653/v1/2022.findings-acl.146 ↗
5. Mehta, S., Shah, D., Kulkarni, R., and Caragea, C. (2023). Semantic Tokenizer for Enhanced Natural Language Processing. arXiv.
6. Hinton, G., Vinyals, O., and Dean, J. (2015). Distilling the knowledge in a neural network. arXiv.
7. Hesitant fuzzy sets
8. Pfahler, L., and Morik, K. (2022). Self-Supervised Pretraining of Graph Neural Network for the Retrieval of Related Mathematical Expressions in Scientific Articles. arXiv.
9. Beltagy, I., Lo, K., and Cohan, A. (2019). SciBERT: A pretrained language model for scientific text. arXiv.10.18653/v1/D19-1371 ↗
10. BioBERT: A pre-trained biomedical language representation model for biomedical text mining10.1093/bioinformatics/btz682 ↗
11. Razdaibiedina, A., and Brechalov, A. (2023). MIReAD: Simple Method for Learning High-quality Representations from Scientific Documents. arXiv.10.18653/v1/2023.acl-short.46 ↗
12. Peng, S., Yuan, K., Gao, L., and Tang, Z. (2021). Mathbert: A pre-trained model for mathematical formula understanding. arXiv.
13. Embedding and generalization of formula with context in the retrieval of mathematical information
14. Hesitant fuzzy partitioned Maclaurin symmetric mean aggregation operators in multi-criteria decision-making10.1088/1402-4896/ac7589 ↗
15. Probabilistic hesitant bipolar fuzzy Hamacher prioritized aggregation operators and their application in multi-criteria group decision-making10.1007/s40314-023-02387-7 ↗
16. Fuzzy sets10.1016/S0019-9958(65)90241-X ↗
17. Multiattribute decision making based on Fermatean hesitant fuzzy sets and modified VIKOR method10.1016/j.ins.2022.06.037 ↗
18. Multi-objective reliability redundancy allocation using MOPSO under hesitant fuzziness10.1016/j.eswa.2022.116696 ↗
19. A novel high order hesitant fuzzy time series forecasting by using mean aggregated membership value with support vector machine10.1016/j.ins.2023.01.075 ↗
20. Scientific Documents Retrieval Based on Graph Convolutional Network and Hesitant Fuzzy Set10.1109/ACCESS.2023.3259234 ↗
21. Sanh, V., Debut, L., Chaumond, J., and Wolf, T. (2019). DistilBERT, a distilled version of BERT: Smaller, faster, cheaper and lighter. arXiv.
22. Sun, S., Cheng, Y., Gan, Z., and Liu, J. (2019). Patient knowledge distillation for bert model compression. arXiv.10.18653/v1/D19-1441 ↗
23. Wang, W., Bao, H., Huang, S., Dong, L., and Wei, F. (2020). Minilmv2: Multi-head self-attention relation distillation for compressing pretrained transformers. arXiv.10.18653/v1/2021.findings-acl.188 ↗
24. Romero, A., Ballas, N., Kahou, S.E., Chassang, A., Gatta, C., and Bengio, Y. (2014). Fitnets: Hints for thin deep nets. arXiv.
25. Jiao, X., Yin, Y., Shang, L., Jiang, X., Chen, X., Li, L., Wang, F., and Liu, Q. (2019). Tinybert: Distilling bert for natural language understanding. arXiv.10.18653/v1/2020.findings-emnlp.372 ↗
26. Chen, X., He, B., Hui, K., Sun, L., and Sun, Y. (April, January 28). Simplified tinybert: Knowledge distillation for document retrieval. Proceedings of the Advances in Information Retrieval: 43rd European Conference on IR Research, ECIR 2021, Virtual.10.1007/978-3-030-72240-1_21 ↗
27. Tian, X. (2017, January 26–28). A mathematical indexing method based on the hierarchical features of operators in formulae. Proceedings of the 2nd International Conference on Automatic Control and Information Engineering (ICACIE 2017), Hong Kong, China.10.2991/icacie-17.2017.11 ↗
28. Distance and similarity measures for hesitant fuzzy sets10.1016/j.ins.2011.01.028 ↗
29. Wang, H., Tian, X., Zhang, K., Cui, X., Shi, Q., and Li, X. (2019). A multi-membership evaluating method in ranking of mathematical retrieval results. Sci. Technol. Eng., 8.
30. Reimers, N., and Gurevych, I. (2019). Sentence-bert: Sentence embeddings using siamese bert-networks. arXiv.10.18653/v1/D19-1410 ↗
31. Learning short-text semantic similarity with word embeddings and external knowledge sources10.1016/j.knosys.2019.07.013 ↗
32. Oliveira, R.M., Gonzaga, F.B., Barbosa, V.C., and Xexéo, G.B. (2017). A distributed system for SearchOnMath based on the Microsoft BizSpark program. arXiv.10.5753/sbbd.2018.22245 ↗
33. Mansouri, B., Rohatgi, S., Oard, D.W., Wu, J., Giles, C.L., and Zanibbi, R. (2019, January 2–5). Tangent-CFT: An embedding model for mathematical formulas. Proceedings of the ACM SIGIR International Conference on Theory of Information Retrieval, Santa Clara, CA, USA.10.1145/3341981.3344235 ↗
34. Khattab, O., and Zaharia, M. (2020, January 25–30). Colbert: Efficient and effective passage search via contextualized late interaction over bert. Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval, Virtual.10.1145/3397271.3401075 ↗