Testing Pretrained Large Language Models to Set Up a Knowledge Hub of Heterogeneous Multisource Environmental Documents
Source abstract
Abstract
This contribution outlines the design of a Knowledge Hub of heterogeneous documents related to the UNEP/MAP Barcelona Convention system. The Knowledge Hub is intended to serve as a resource to assist public authorities and users with different backgrounds and needs in accessing information efficiently; users should be able to either formulate natural language queries or to navigate a knowledge graph that is automatically generated to find relevant documents. The ad hoc retrieval task and the Knowledge Hub creation are defined based on state-of-the-art Large Language Models (LLMs). Specifically, this contribution focuses on a user-evaluation experiment that tested publicly available pretrained foundation Large Language Models (LLMs) for retrieving a subset of documents with varying lengths and topics.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Topic Modeling · Semantic Web and Ontologies · Data Quality and Management
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Institute for Remote Sensing of Environment (IREA), National Research Council (CNR), Via A. Corti 12, 20133 Milano, Italy
INFO/RAC UNEP/MAP c/o ISPRA, DG-SINA, Via Vitaliano Brancati 48, 00144 Roma, Italy
Funding
ISPRA Istituto Superiore per la Protezione e la Ricerca Ambientale
References (35)
Show source-provided references
1. UNEP (2025, March 01). Mediterranean Action Plan (MAP)—Barcelona Convention. Available online: https://www.unep.org/mediterranean-action-plan-map-barcelona-convention.
2. Bordogna, G., Tagliolato, P., Lotti, A., Minelli, A., Oggioni, A., and Babbini, L. (2023). Report 2—Semantic Information Retrieval–Knowledge Hub. Zenodo.
3. Martino, S.D., Sansone, C., Masciari, E., Rossi, S., and Gravina, M. (2024, January 29–30). Design of a Knowledge Hub of Heterogeneous Multisource Documents to support Public Authorities. Proceedings of the Ital-IA Intelligenza Artificiale—Thematic Workshops Co-Located with the 4th CINI National Lab AIIS Conference on Artificial Intelligence (Ital-IA 2024), Naples, Italy. CEUR-WS.org, 2024, CEUR Workshop Proceedings.
4. Survey on supervised machine learning techniques for automatic text classification10.1007/s10462-018-09677-1 ↗
5. Oggioni, A., Bordogna, G., Lotti, A., Minelli, A., Tagliolato, P., and Babbini, L. (2023). Types of Users and Requirements—Report 0. Zenodo.
6. Manning, C.D., Raghavan, P., and Schütze, H. (2009). Introduction to Information Retrieval, Online Edition, Cambridge University Press.10.1017/CBO9780511809071 ↗
7. Zhou, C., Li, Q., Li, C., Yu, J., Liu, Y., Wang, G., Zhang, K., Ji, C., Yan, Q., and He, L. (2023). A Comprehensive Survey on Pretrained Foundation Models: A History from BERT to ChatGPT. arXiv.10.1007/s13042-024-02443-6 ↗
8. Bezdek, J.C., Dubois, D., and Prade, H. (1999). Fuzzy Set Techniques in Information Retrieval. Fuzzy Sets in Approximate Reasoning and Information Systems, Springer.10.1007/978-1-4615-5243-7 ↗
9. Utilizing BERT for Information Retrieval: Survey, Applications, Resources, and Challenges
10. REMPEC (2025, March 01). Regional Marine Pollution Emergency Response Centre for the Mediterranean Sea (REMPEC). Available online: https://www.rempec.org.
11. SPA/RAC (2025, March 01). Regional Activity Centre for Specially Protected Areas. Available online: https://www.rac-spa.org.
12. SCP/RAC (2025, March 01). Regional Activity Centre for Sustainable Consumption and Production. Available online: http://www.cprac.org.
13. PAP/RAC (2025, March 01). Priority Actions Programme/Regional Activity Centre. Available online: https://paprac.org.
14. UNEP/MAP (2025, March 01). UNEP/MAP: All Publications. Available online: https://www.unep.org/unepmap/resources/publications?/resources.
15. United Nations Environment Program (2025, March 01). United Nations Environment Program: Documents Repository. Available online: https://wedocs.unep.org/discover?filtertype=author&filter_relational_operator=equals&filter=UNEP%20MAP.
16. (2025, March 01). Scraping Library. Available online: https://github.com/INFO-RAC/KMP-library-scraping.
17. (2025, March 01). Inforac Ground Truth Data Used for the User Evaluation Experiment. Available online: https://github.com/IREA-CNR-MI/inforac_ground_truth.
18. Wolf, T., Debut, L., Sanh, V., Chaumond, J., Delangue, C., Moi, A., Cistac, P., Rault, T., Louf, R., and Funtowicz, M. (2020). HuggingFace’s Transformers: State-of-the-art Natural Language Processing. arXiv.10.18653/v1/2020.emnlp-demos.6 ↗
19. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, L., and Polosukhin, I. (2017, January 4–9). Attention is all you need. Proceedings of the 31st International Conference on Neural Information Processing Systems, Red Hook, NY, USA. NIPS’17.
20. Burstein, J., Doran, C., and Solorio, T. (2019, January 6). BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), Minneapolis, MI, USA.
21. CogLTX: Applying BERT to Long Texts
22. Reimers, N., and Gurevych, I. (2019, January 3–7). Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing, Association for Computational Linguistics, Hong Kong, China.10.18653/v1/D19-1410 ↗
23. Bird, S., Klein, E., and Loper, E. (2009). Natural Language Processing with Python, O’Reilly Media, Inc.. [1st ed.].
24. On the fuzzy cardinality of a fuzzy set10.1080/03081070500422729 ↗
25. Reimers, N., Espejel, O., Cuenca, P., and Aarsen, T. (2025, March 01). Msmarco-Distilbert-Cos-v5 Model Card. Available online: https://huggingface.co/sentence-transformers/msmarco-distilbert-cos-v5.
26. Reimers, N., Espejel, O., Cuenca, P., and Aarsen, T. (2025, March 01). All-MiniLM-L6-v2 Model Card. Available online: https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2.
27. Xiong, L., Xiong, C., Li, Y., Tang, K.F., Liu, J., Bennett, P., Ahmed, J., and Overwijk, A. (2020). Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrieval. arXiv.
28. Reimers, N., Espejel, O., Cuenca, P., and Aarsen, T. (2025, March 01). Msmarco-Roberta-Base-Ance-Firstp Model Card. Available online: https://huggingface.co/sentence-transformers/msmarco-roberta-base-ance-firstp.
29. Reimers, N., Espejel, O., Cuenca, P., and Aarsen, T. (2025, March 01). Msmarco-Bert-Base-Dot-v5 Model Card. Available online: https://huggingface.co/sentence-transformers/msmarco-bert-base-dot-v5.
30. Reimers, N., Espejel, O., Cuenca, P., and Aarsen, T. (2025, March 01). Msmarco-Distilbert-Base-Tas-b Model Card. Available online: https://huggingface.co/sentence-transformers/msmarco-distilbert-base-tas-b.
31. (2025, March 01). UNESCO Thesaurus. Available online: http://vocabularies.unesco.org/thesaurus.
32. (2025, March 01). Get-It Inforac Thesaurus—Linked Open Data Access. Available online: http://rdfdata.get-it.it/inforac/.
33. (2025, March 01). Get-It Inforac Thesaurus—Sparql Endpoint (Web Service). Available online: http://fuseki1.get-it.it/dataset.html.
34. Liu, L., and Özsu, M.T. (2009). MAP. Encyclopedia of Database Systems, Springer.10.1007/978-0-387-39940-9 ↗
35. Smucker, M.D., Allan, J., and Carterette, B. (2007;, January 6–10). A comparison of statistical significance tests for information retrieval evaluation. Proceedings of the Sixteenth ACM Conference on Conference on Information and Knowledge Management, New York, NY, USA.10.1145/1321440.1321528 ↗