journal article·2025-01-10Open access

Extracting Fruit Disease Knowledge from Research Papers Based on Large Language Models and Prompt Engineering

Yunqiao Fei, Jingchao Fan, Guomin Zhou

Applied Sciences· Volume 15· Issue 2

Abstract

In China, fruit tree diseases are a significant threat to the development of the fruit tree industry, and knowledge about fruit tree diseases is the most needed professional knowledge for fruit farmers and other practitioners in the fruit tree industry. Research papers are the primary sources of professional knowledge that represent the cutting-edge progress in fruit disease research. Traditional knowledge engineering methods for knowledge acquisition require extensive and cumbersome preparatory work, and they demand a high level of professional background and information technology skills from the handlers. This paper, from the perspective of fruit tree industry knowledge dissemination, aims at users such as fruit farmers, fruit tree experts, fruit tree knowledge communicators, and information gatherers. It proposes a fast, cost-effective, and low-technical-barrier method for extracting fruit tree disease knowledge from research paper abstracts—K-Extract, based on large language models (LLMs) and prompt engineering. Under zero-shot conditions, K-Extract utilizes conversational LLMs to automate the extraction of fruit tree disease knowledge. The K-Extract method has constructed a comprehensive classification system for fruit tree diseases and, through a series of optimized prompt questions, effectively overcomes the deficiencies of LLM models in providing factual accuracy. This paper tests multiple LLM models available in the Chinese market, and the results show that K-Extract can seamlessly integrate with any conversational LLM model, with the DeepSeek model and the Kimi model performing particularly well. The experimental results indicate that LLM models have a high accuracy rate in handling judgment tasks and simple knowledge Q&A tasks. The K-Extract method is simple, efficient, and accurate, and can serve as a convenient tool for knowledge extraction in the agricultural field.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: Advanced Text Analysis Techniques · Biomedical Text Mining and Ontologies

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

Agricultural Information Institute, Chinese Academy of Agricultural Sciences, Beijing 100081, China

China Agricultural Science and Technology Press, Beijing 100081, China

Key Laboratory of Knowledge Mining and Knowledge Services in Agricultural Converging Publishing, Agricultural Information Institute, Chinese Academy of Agricultural Sciences, Beijing 100081, China

National Agricultural Science Data Center, Beijing 100081, China

Nanjing Institute of Agricultural Mechanization, Ministry of Agriculture and Rural Affairs, Nanjing 210014, China

Funding

National Agricultural Science Data Center · Open Research Fund Program of the Key Laboratory of Knowledge Mining and Knowledge Services in Agricultural Converging Publishing, Agricultural Information Institute, Chinese Academy of Agricultural Sciences

References (51)

Show source-provided references
1. Current situation of main fruit tree industry in China and it is development countermeasure during the “14th five-year plan” period
2. A lightweight convolutional neural network for disease detection of fruit leaves10.1007/s00521-023-08496-y ↗
3. Liao, S., Wu, Y., Hu, X., Weng, S., Hu, Y., Zheng, L., Lei, Y., Tang, L., Wang, J., and Wang, H. (2023). Detection of apple fruit damages through Raman spectroscopy with cascade forest. Spectrochim. Acta Part Mol. Biomol., 296.10.1016/j.saa.2023.122668 ↗
4. Meng-Meng, Y., and Kichida, R. (2017, January 8–10). A study on classification of fruit type and fruit disease. Proceedings of the 3rd Annual International Conference on Electronics, Electrical Engineering and Information Science (EEEIS 2017), Guangzhou, China.
5. Pyramiding scab resistances in apple breeding
6. Combinatorial approach based on conventional and molecular methods for identification of scab resistance genes in apple germplasm of Jammu and Kashmir, India10.1016/j.pmpp.2023.101976 ↗
7. Xu, D., Chen, W., Peng, W., Zhang, C., Xu, T., Zhao, X., Wu, X., Zheng, Y., Wang, Y., and Chen, E. (2024). Large language models for generative information extraction: A survey. Front. Comput. Sci., 18.10.1007/s11704-024-40555-y ↗
8. Survey of Agricultural Knowledge Graph
9. Structured information extraction from scientific text with large language models10.1038/s41467-024-45563-x ↗
10. Knowledge graph survey: Representation, construction, reasoning and knowledge hypergraph theory
11. Agricultural pest and disease relation extraction based on multi-attention mechanism and distant supervision
12. Jie, Y. (2014). Research on Knowledge Modeling and Reasoning Ontology-Based of Citrus Disease and Pests. [Master’s Thesis, Central China Normal University].
13. Auto-generating databases of yield strength and grain size using chemdataextractor10.1038/s41597-022-01301-w ↗
14. ChemDataExtractor: A toolkit for automated extraction of chemical information from the scientific literature10.1021/acs.jcim.6b00207 ↗
15. Brown, T.B. (2020). Language models are few-shot learners. arXiv.
16. Training language models to follow instructions with human feedback
17. Workshop, B., Scao, T.L., Fan, A., Akiki, C., Pavlick, E., Ilić, S., Hesslow, D., Castagné, R., Luccioni, A.S., and Yvon, F. (2022). Bloom: A 176b-parameter open-access multilingual language model. arXiv.
18. Zhang, S., Roller, S., Goyal, N., Artetxe, M., Chen, M., Chen, S., Dewan, C., Diab, M., Li, X., and Lin, X.V. (2022). Opt: Open pre-trained transformer language models. arXiv.
19. Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.A., Lacroix, T., Rozière, B., Goyal, N., Hambro, E., and Azhar, F. (2023). Llama: Open and efficient foundation language models. arXiv.
20. He, Q., Wang, Y., and Rao, G. (2024, January 4–6). Generation and Validation of Teaching Examples Based on Large Language Models. Proceedings of the 2024 International Conference on Asian Language Processing (IALP), Hohhot, China.10.1109/IALP63756.2024.10661177 ↗
21. Large language model in electrocatalysis10.1016/S1872-2067(23)64612-1 ↗
22. Evolution and Prospects of Foundation Models: From Large Language Models to Large Multimodal Models
23. On finetuning large language models10.1017/pan.2023.36 ↗
24. Huang, D., Yan, C., Li, Q., and Peng, X. (2024). From Large Language Models to Large Multimodal Models: A Literature Review. Appl. Sci., 14.10.3390/app14125068 ↗
25. Dunn, A., Dagdelen, J., Walker, N., Lee, S., Rosen, A.S., Ceder, G., Persson, K., and Jain, A. (2022). Structured information extraction from complex scientific text with fine-tuned large language models. arXiv.
26. Reconstructing chromatic-dispersion relations and predicting refractive indices using text mining and machine learning10.1021/acs.jcim.2c00253 ↗
27. A rule-free workflow for the automated generation of databases from scientific literature10.1038/s41524-023-01171-9 ↗
28. Database, features, and machine learning model to identify thermally driven metal–insulator transition compounds10.1021/acs.chemmater.1c00905 ↗
29. Prompt engineering with ChatGPT: A guide for academic writers10.1007/s10439-023-03272-4 ↗
30. Prompt engineering: The next big skill in rheumatology research10.1111/1756-185X.15157 ↗
31. White, J., Fu, Q., Hays, S., Sandborn, M., Olea, C., Gilbert, H., Elnashar, A., Spencer-Smith, J., and Schmidt, D.C. (2023). A prompt pattern catalog to enhance prompt engineering with chatgpt. arXiv.
32. Arawjo, I., Swoopes, C., Vaithilingam, P., Wattenberg, M., and Glassman, E.L. (2024, January 11–16). ChainForge: A Visual Toolkit for Prompt Engineering and LLM Hypothesis Testing. Proceedings of the CHI Conference on Human Factors in Computing Systems, Honolulu, HI, USA.10.1145/3613904.3642016 ↗
33. Patil, R., Heston, T.F., and Bhuse, V. (2024). Prompt engineering in healthcare. Electronics, 13.10.3390/electronics13152961 ↗
34. Ramesh, A., Dhariwal, P., Nichol, A., Chu, C., and Chen, M. (2022). Hierarchical text-conditional image generation with clip latents. arXiv.
35. Rombach, R., Blattmann, A., Lorenz, D., Esser, P., and Ommer, B. (2022, January 18–24). High-resolution image synthesis with latent diffusion models. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.10.1109/CVPR52688.2022.01042 ↗
36. Prompting change: Exploring prompt engineering in large language model AI and its potential to transform education10.1007/s11528-023-00896-0 ↗
37. Sarmah, B., Mehta, D., Hall, B., Rao, R., Patel, S., and Pasquali, S. (2024, January 14–17). Hybridrag: Integrating knowledge graphs and vector retrieval augmented generation for efficient information extraction. Proceedings of the 5th ACM International Conference on AI in Finance, Brooklyn, NY, USA.10.1145/3677052.3698671 ↗
38. Wu, S., Zhao, S., Huang, Q., Huang, K., Yasunaga, M., Cao, K., Ioannidis, V.N., Subbian, K., Leskovec, J., and Zou, J. (2024). AvaTaR: Optimizing LLM Agents for Tool-Assisted Knowledge Retrieval. arXiv.
39. Agricultural technology knowledge intelligent question-answering system based on large language model
40. Large language models and agricultural extension services10.1038/s43016-023-00867-x ↗
41. Peng, R., Liu, K., Yang, P., Yuan, Z., and Li, S. (2023). Embedding-based retrieval with llm for effective agriculture information extracting from unstructured data. arXiv.
42. Prompt engineering or fine-tuning? A case study on phishing detection with large language models10.3390/make6010018 ↗
43. Xu, T., Gu, Y., Xue, M., Gu, R., Li, B., and Gu, X. (2024). Knowledge graph construction for heart failure using large language models with prompt engineering. Front. Comput. Neurosci., 18.10.3389/fncom.2024.1389475 ↗
44. Wang, X., Attal, M.I., Rafiq, U., and Hubner-Benz, S. (2022, January 13–17). Turning large language models into AI assistants for startups using prompt patterns. Proceedings of the International Conference on Agile Software Development, Copenhagen, Denmark.10.1007/978-3-031-48550-3_19 ↗
45. Extracting accurate materials data from research papers with conversational language models and prompt engineering10.1038/s41467-024-45914-8 ↗
46. Eldan, R., and Li, Y. (2023). Tinystories: How small can language models be and still speak coherent english?. arXiv.
47. Flexible, model-agnostic method for materials data extraction from text using general purpose language models10.1039/D4DD00016A ↗
48. Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., and Wen, J.R. (2023). Structgpt: A general framework for large language model to reason over structured data. arXiv.10.18653/v1/2023.emnlp-main.574 ↗
49. Bai, J., Bai, S., Chu, Y., Cui, Z., Dang, K., Deng, X., Fan, Y., Ge, W., Han, Y., and Huang, F. (2023). Qwen technical report. arXiv.
50. Bi, X., Chen, D., Chen, G., Chen, S., Dai, D., Deng, C., Ding, H., Dong, K., Du, Q., and Fu, Z. (2024). Deepseek llm: Scaling open-source language models with longtermism. arXiv.
51. Bang, Y., Cahyawijaya, S., Lee, N., Dai, W., Su, D., Wilie, B., Lovenia, H., Ji, Z., Yu, T., and Chung, W. (2023). A multitask, multilingual, multimodal evaluation of chatgpt on reasoning, hallucination, and interactivity. arXiv.10.18653/v1/2023.ijcnlp-main.45 ↗