DietNerd: A Nutrition Question-Answering System That Summarizes and Evaluates Peer-Reviewed Scientific Articles
Source abstract
Abstract
DietNerd is a large language model-based system designed to enhance public health education in diet and nutrition. The system responds to user questions with concise, evidence-based summaries and assesses the quality and potential biases of cited research. This paper describes the system’s workflow, back-end implementation, and the prompts used. Accuracy and quality-of-response results are presented based on an automated comparison against systematic surveys and against the responses of similar state-of-the-art systems through human feedback from registered dietitians. DietNerd is among the highest-evaluated of these systems and is unique in combining safety features with sophisticated source analysis. Thus, DietNerd could be a tool to bridge the gap between complex scientific literature and public understanding.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Topic Modeling
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Department of Computer Science, Courant Institute of Mathematical Sciences, New York University, New York, NY 10012, USA
Department of Linguistics, University of Illinois Urbana-Champaign, Champaign, IL 61820, USA
Funding
NYU Wireless
References (43)
Show source-provided references
1. Diet, Lifestyle, Biomarkers, Genetic Factors, and Risk of Cardiovascular Disease in the Nurses’ Health Studies10.2105/AJPH.2016.303316 ↗
2. The Effects of Dietary Improvement on Symptoms of Depression and Anxiety: A Meta-Analysis of Randomized Controlled Trials10.1097/PSY.0000000000000673 ↗
3. Seeking health- and nutrition-related information on the Internet in a large population of French adults: Results of the NutriNet-Santé study10.1017/S0007114516001355 ↗
4. Johnson, A. (2024, August 01). Here’s How To Use AI—Like ChatGPT And Bard—For Everyday Tasks Like Creating A Budget, Finding Airfare Or Planning Meals. Forbes, Available online: https://www.forbes.com/sites/ariannajohnson/2023/04/10/heres-how-to-use-ai-like-chatgpt-and-bard-for-everyday-tasks-like-creating-a-budget-finding-airfare-or-planning-meals.
5. Lozano, A., Fleming, S.L., Chiang, C.C., and Shah, N. (2024, January 3–7). Clinfo.ai: An open-source retrieval-augmented large language model system for answering medical questions using scientific literature. Proceedings of the Pacific Symposium on Biocomputing 2024, Kohala Coast, HI, USA.10.1142/9789811286421_0002 ↗
6. Lozano, A., Fleming, S.L., Chiang, C.C., and Shah, N. (2024, August 01). PubMedRS-200 Dataset. Available online: https://github.com/som-shahlab/Clinfo.AI/blob/main/PubMedRS-200/PubMedRS-200.csv.
7. Olson, E., and Salem, C. (2024, August 01). Consensus. Available at OpenAI’s GPT Marketplace. Available online: https://chatgpt.com/g/g-bo0FiWLY7-consensus.
8. Consensus (2024, August 01). How It Works & Consensus FAQ’s. Available online: https://consensus.app/home/blog/welcome-to-consensus/.
9. Awesomegpts.ai (2024, August 01). Scholar GPT. Available at OpenAI’s GPT Marketplace, March 2023. Available online: https://chatgpt.com/g/g-kZ0eYXlJe-scholar-gpt.
10. Nadler, D., Ziegler, Z., Wulff, J., Smith, M., Hernandez, E., Lehman, E., Xu, J., Ferreira, F., Esdaile, J., and Hu, H. (2024, August 01). OpenEvidence. Available online: https://www.openevidence.com/.
11. CB Insights (2024, August 01). AI 100: The Most Promising Artificial Intelligence Startups of 2024. Available online: https://www.cbinsights.com/research/report/artificial-intelligence-top-startups-2024/.
12. (2024, August 01). Business Insider. OpenEvidence AI Becomes the First AI in History to Score above 90% on the United States Medical Licensing Examination (USMLE), Available online: https://markets.businessinsider.com/news/stocks/openevidence-ai-becomes-the-first-ai-in-history-to-score-above-90-on-the-united-states-medical-licensing-examination-usmle-1032446924.
13. Hippocratic AI (2024, August 01). StatPearls Semantic Search. Available at Hippocratic Medical Questions Website, Available online: https://hippocratic-medical-questions.herokuapp.com/.
14. Elicit (2023, January 24). Elicit. Available at Elicit Website. Available online: https://elicit.com.
15. He, K., Mao, R., Lin, Q., Ruan, Y., Lan, X., Feng, M., and Cambria, E. (2024). A Survey of Large Language Models for Healthcare: From Data, Technology, and Applications to Accountability and Ethics. arXiv.10.2139/ssrn.4809363 ↗
16. Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., and Askell, A. (2020). Language Models are Few-Shot Learners. arXiv.
17. Lewis, P., Perez, E., Piktus, A., Petroni, F., Karpukhin, V., Goyal, N., Küttler, H., Lewis, M., Yih, W.t., and Rocktäschel, T. (2021). Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks. arXiv.
18. Li, J., Yuan, Y., and Zhang, Z. (2024). Enhancing LLM Factual Accuracy with RAG to Counter Hallucinations: A Case Study on Domain-Specific Queries in Private Knowledge-Bases. arXiv.
19. Accuracy and Interpretability: Struggling with the Epistemic Foundations of Machine Learning-Generated Medical Information and Their Practical Implications for the Doctor-Patient Relationship10.1007/s13347-022-00505-7 ↗
20. Thoppilan, R., De Freitas, D., Hall, J., Shazeer, N., Kulshreshtha, A., Cheng, H.-T., Jin, A., Bos, T., Baker, L., and Du, Y. (2022). LaMDA: Language Models for Dialog Applications. arXiv.
21. Glaese, A., McAleese, N., Trȩbacz, M., Aslanides, J., Firoiu, V., Ewalds, T., Rauh, M., Weidinger, L., Chadwick, M., and Thacker, P. (2022). Improving Alignment of Dialogue Agents via Targeted Human Judgements. arXiv.
22. Bar-Haim, R., Kantor, Y., Venezian, E., Katz, Y., and Slonim, N. (2021). Project Debater APIs: Decomposing the AI Grand Challenge. arXiv.10.18653/v1/2021.emnlp-demo.31 ↗
23. Shuster, K., Xu, J., Komeili, M., Ju, D., Smith, E.M., Roller, S., Ung, M., Chen, M., Arora, K., and Lane, J. (2022). BlenderBot 3: A Deployed Conversational Agent That Continually Learns to Responsibly Engage. arXiv.
24. Christiano, P., Leike, J., Brown, T.B., Martic, M., Legg, S., and Amodei, D. (2023). Deep Reinforcement Learning from Human Preferences. arXiv.
25. Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C.L., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., and Ray, A. (2022). Training Language Models to Follow Instructions with Human Feedback. arXiv.
26. Zhang, Y., Sun, S., Galley, M., Chen, Y.-C., Brockett, C., Gao, X., Gao, J., Liu, J., and Dolan, B. (2020). DialoGPT: Large-Scale Generative Pre-training for Conversational Response Generation. arXiv.10.18653/v1/2020.acl-demos.30 ↗
27. Sayers, E. (2022, November 17). A General Introduction to the E-Utilities, Available online: https://www.ncbi.nlm.nih.gov/books/NBK25497/.
28. National Library of Medicine (2024, September 11). The 9 E-Utilities and Associated Parameters Available at National Library of Medicine, Available online: https://www.nlm.nih.gov/dataguide/eutilities/utilities.html.
29. ‘First, do no harm’: Shifting the paradigm towards a culture of health10.35680/2372-0247.1189 ↗
30. PubMed Central (2024, August 01). PubMed Central: An Archive of Biomedical and Life Sciences Journal Literature. Available at National Center for Biotechnology Information, Available online: https://www.ncbi.nlm.nih.gov/pmc/.
31. National Library of Medicine (2024, August 01). Publication Types. Available at National Library of Medicine, Available online: https://www.nlm.nih.gov/mesh/pubtypes.html.
32. Zhang, T., Kishore, V., Wu, F., Weinberger, K.Q., and Artzi, Y. (2020). BERTScore: Evaluating Text Generation with BERT. arXiv.
33. Reimers, N., and Gurevych, I. (2019, January 3–7). Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing, Hong Kong, China.10.18653/v1/D19-1410 ↗
34. Anschütz, M., Lozano, D.M., and Groh, G. (2023). This is not correct! Negation-aware Evaluation of Language Generation Systems. arXiv.10.18653/v1/2023.inlg-main.12 ↗
35. Cer, D., Yang, Y., Kong, S.-Y., Hua, N., Limtiaco, N., St. John, R., Constant, N., Guajardo-Cespedes, M., Yuan, S., and Tar, C. (arXiv, 2018). Universal Sentence Encoder, arXiv.10.18653/v1/D18-2029 ↗
36. Papineni, K., Roukos, S., Ward, T., and Zhu, W.-J. (2002, January 6–12). BLEU: A method for automatic evaluation of machine translation. Proceedings of the 40th Annual Meeting on Association for Computational Linguistics, Philadelphia, PA, USA.10.3115/1073083.1073135 ↗
37. Lin, C.-Y. (2004). ROUGE: A Package for Automatic Evaluation of Summaries. Text Summarization Branches Out, Association for Computational Linguistics.
38. Ng, J.-P., and Abrecht, V. (2015, January 17–21). Better Summarization Evaluation with Word Embeddings for ROUGE. Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing 2015, Lisbon, Portugal.10.18653/v1/D15-1222 ↗
39. Learning similarity with cosine similarity ensemble10.1016/j.ins.2015.02.024 ↗
40. Katari, M.S., Shasha, D., and Tyagi, S. (2021). Statistics Is Easy: Case Studies on Real Scientific Datasets, Springer. [1st ed.]. Synthesis Lectures on Mathematics & Statistics.10.1007/978-3-031-02433-7 ↗
41. (2024, September 10). Allied Health Workforce Projections, 2016–2030: Registered Dieticians Allied Health Workforce Project, Available online: https://bhw.hrsa.gov/sites/default/files/bureau-health-workforce/data-research/registered-dieticians-2016-2030.pdf.
42. Health Coaching: A New Opportunity for Dietetics Professionals10.1016/j.jada.2006.04.003 ↗
43. Gharibi, M., Zachariah, A., and Rao, P. (2020). FoodKG: A Tool to Enrich Knowledge Graphs Using Machine Learning Techniques. Front. Big Data, 3.10.3389/fdata.2020.00012 ↗