journal article·2025-11-07Open access

Semi-Automatic Extraction and Analysis of Health Equity Covariates in Registered Research Projects

Navapat Nananukul, Mayank Kejriwal

Applied Sciences· Volume 15· Issue 22

Abstract

Advancing health equity requires rigorous analysis of how research initiatives incorporate and address structural disparities across populations. In this study, we apply large language models (LLMs) to systematically analyze research projects registered on the All of Us platform, with a focus on identifying patterns and institutional dynamics associated with health equity research. We examine the relationship between projects that explicitly pursue health equity goals and their use of available demographic data, their institutional composition (e.g., single- vs. multi-institutional teams), and the research tier of participating institutions (R1 vs. R2). Using the capabilities of an established LLM, we automate key tasks including the extraction of relevant attributes from unstructured project descriptions, classification of institutional affiliations, and the summarization of project content into standardized keywords from the Unified Medical Language System vocabulary. This LLM-assisted pipeline enabled scalable, replicable analysis of hundreds of projects with minimal manual overhead. Our findings suggest a strong association between the use of demographic data and health equity aims, and indicate nuanced differences in equity-oriented research participation by institution type and collaborative structure. More broadly, our approach demonstrates how LLMs can support equity-focused computational social science by transforming free-text administrative data into analyzable structures, enabling novel insights in public health, team science, and science-of-science studies.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: Health Policy Implementation Science · Health disparities and outcomes · Advanced Causal Inference Techniques

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

Daniel J. Epstein Department of Industrial & Systems Engineering, Information Sciences Institute, University of Southern California, Los Angeles, CA 90007, USA

References (57)

Show source-provided references
1. Health disparities and health equity: Concepts and measurement10.1146/annurev.publhealth.27.021405.102103 ↗
2. Baciu, A., Negussie, Y., Geller, A., Weinstein, J.N., National Academies of Sciences, Engineering, and Medicine, and Committee on Community-Based Solutions to Promote Health Equity in the United States (2017). The Need to Promote Health Equity. Communities in Action: Pathways to Health Equity, National Academies Press.
3. Achieving health equity: From root causes to fair outcomes10.1016/S0140-6736(07)61385-3 ↗
4. Advocacy for health equity: A synthesis review10.1111/1468-0009.12112 ↗
5. What is health equity?10.1177/237946151800400102 ↗
6. Whitehead, M. (1990). The Concepts and Principles of Equity in Health.
7. National Institute on Minority Health and Health Disparities (2025, October 11). NIMHD Research Framework, Available online: https://www.nimhd.nih.gov/resources/nimhd-research-framework.
8. National Institutes of Health (2024, April 06). All of Us: About, Available online: https://allofus.nih.gov/about.
9. National Institutes of Health (2024, April 06). All of Us: Research Projects Directory, Available online: https://allofus.nih.gov/protecting-data-and-privacy/research-projects-all-us-data.
10. Bogard, K., Murry, V., and Alexander, C. (2017). Perspectives on health equity and social determinants of health. NAM Perspectives, National Academy of Medicine.10.17226/27117 ↗
11. Social determinants of health and health equity policy research: Exploring the use, misuse, and nonuse of policy analysis theory10.1016/j.socscimed.2014.03.004 ↗
12. Measurement of health disparities, health inequities, and social determinants of health to support the advancement of health equity10.1097/PHH.0000000000000373 ↗
13. Carnegie Classification of Institutions of Higher Education (2024, April 06). Classification Methodology: Basic Classification. Available online: https://carnegieclassifications.acenet.edu/carnegie-classification/classification-methodology/basic-classification/.
14. Priorities for research to take forward the health equity policy agenda
15. Research on health equity in the SDG era: The urgent need for greater focus on implementation10.1186/s12939-016-0493-7 ↗
16. Toward a fourth generation of disparities research to achieve health equity10.1146/annurev-publhealth-031210-101136 ↗
17. Participation as a driver of health equity
18. Siiman, L.A., Rannastu-Avalos, M., Pöysä-Tarhonen, J., Häkkinen, P., and Pedaste, M. (2023, January 28–30). Opportunities and challenges for AI-assisted qualitative data analysis: An example from collaborative problem-solving discourse data. Proceedings of the International Conference on Innovative Technologies and Learning, Porto, Portugal.10.1007/978-3-031-40113-8_9 ↗
19. Gu, K., Shang, R., Althoff, T., Wang, C., and Drucker, S.M. (2024, January 11–16). How do analysts understand and verify ai-assisted data analyses?. Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems, Honolulu, HI, USA.10.1145/3613904.3642497 ↗
20. Dai, S.C., Xiong, A., and Ku, L.W. (2023). LLM-in-the-loop: Leveraging large language model for thematic analysis. arXiv.10.18653/v1/2023.findings-emnlp.669 ↗
21. Leveraging Large Language Models and Agent-Based Systems for Scientific Data Analysis: Validation Study10.2196/68135 ↗
22. The All of Us Research Program: Data quality, utility, and diversity10.1016/j.patter.2022.100570 ↗
23. The All of Us Research Program Genomics Investigators (2024). Genomic data in the all of us research program. Nature, 627, 340–346.
24. Predictive analytics for glaucoma using data from the all of us research program10.1016/j.ajo.2021.01.008 ↗
25. Expanding the All of Us Research Platform into the perioperative domain10.1001/jamasurg.2024.2343 ↗
26. Monitoring equity in health and healthcare: A conceptual framework
27. The health equity framework: A science-and justice-based model for public health researchers and practitioners10.1177/1524839920950730 ↗
28. A framework for digital health equity10.1038/s41746-022-00663-0 ↗
29. Leveraging large language models to foster equity in healthcare10.1093/jamia/ocae055 ↗
30. A toolbox for surfacing health equity harms and biases in large language models10.1038/s41591-024-03258-2 ↗
31. Iloanusi, N.J., and Chun, S.A. (2024, January 11–14). AI impact on health equity for marginalized, racial, and ethnic minorities. Proceedings of the 25th Annual International Conference on Digital Government Research, Taipei, Taiwan.10.1145/3657054.3657152 ↗
32. BioBERT: A pre-trained biomedical language representation model for biomedical text mining10.1093/bioinformatics/btz682 ↗
33. Beltagy, I., Lo, K., and Cohan, A. (2019). SciBERT: A pretrained language model for scientific text. arXiv.10.18653/v1/D19-1371 ↗
34. Han, Q., Tian, S., and Zhang, J. (2021). A PubMedBERT-based classifier with data augmentation strategy for detecting medication mentions in tweets. arXiv.
35. Groza, T., Caufield, H., Gration, D., Baynam, G., Haendel, M.A., Robinson, P.N., Mungall, C.J., and Reese, J.T. (2024). An evaluation of GPT models for phenotype concept recognition. BMC Med. Inform. Decis. Mak., 24.10.1186/s12911-024-02439-w ↗
36. Rouhizadeh, H., Yazdani, A., Zhang, B., Alvarez, D.V., Hüser, M., Vanobberghen, A., Yang, R., Li, I., Walter, A., and Teodoro, D. (2025). Large language models struggle to encode medical concepts—A multilingual benchmarking and comparative analysis. medRxiv.10.1101/2025.01.15.25320579 ↗
37. Chen, Q., Sun, H., Liu, H., Jiang, Y., Ran, T., Jin, X., Xiao, X., Lin, Z., Chen, H., and Niu, Z. (2023). An extensive benchmark study on biomedical text generation and mining with ChatGPT. Bioinformatics, 39.10.1093/bioinformatics/btad557 ↗
38. Benchmarking large language models for biomedical natural language processing applications and recommendations10.1038/s41467-025-56989-2 ↗
39. OpenAI (2024, April 06). Welcome to the OpenAI Developer Platform. Available online: https://platform.openai.com/docs/overview.
40. Content analysis in an era of big data: A hybrid approach to computational and manual methods10.1080/08838151.2012.761702 ↗
41. Analyzing open-ended questions by means of text analysis procedures10.1177/0759106315597389 ↗
42. The validity of sentiment analysis: Comparing manual annotation, crowd-coding, dictionary approaches, and machine learning algorithms10.1080/19312458.2020.1869198 ↗
43. Barbosa, N.M., and Chen, M. (2019, January 4–9). Rehumanized crowdsourcing: A labeling framework addressing bias and ethics in machine learning. Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, Glasgow, UK.10.1145/3290605.3300773 ↗
44. Tecimer, K.A., Aghajani, E., Bissyandé, T.F., Klein, J., and Le Traon, Y. (2021, January 21–24). Detection and elimination of systematic labeling bias in code reviewer recommendation systems. Proceedings of the 25th International Conference on Evaluation and Assessment in Software Engineering, Trondheim, Norway.10.1145/3463274.3463336 ↗
45. Richardson, L. (2024, April 06). Beautiful Soup Documentation. Available online: https://readthedocs.org/projects/beautiful-soup-4/downloads/pdf/latest/.
46. An unsupervised instance matcher for schema-free RDF data10.1016/j.websem.2015.07.002 ↗
47. The unified medical language system (UMLS): Integrating biomedical terminology10.1093/nar/gkh061 ↗
48. Holtz, Y. (2024, April 06). Chord Diagram. Available online: https://r-graph-gallery.com/chord-diagram.html.
49. Authorship in Global Mental Health Research: Recommendations for Collaborative Approaches to Writing and Publishing10.1016/j.aogh.2014.04.007 ↗
50. ‘It is always worth the extra effort’: Organizational structures and barriers to collaboration with consumers in mental health research: Perspectives of non-consumer researcher allies10.1111/inm.12757 ↗
51. Hispanic immigrants in the USA: Social and mental health perspectives10.1016/S2215-0366(16)30101-8 ↗
52. Challenges and Opportunities to Meet the Mental Health Needs of Underserved and Disenfranchised Populations in the United States10.1176/appi.focus.20190028 ↗
53. Pearman, A., Hughes, M.L., Smith, E.L., and Neupert, S.D. (2020). Mental Health Challenges of United States Healthcare Professionals During COVID-19. Front. Psychol., 11.10.3389/fpsyg.2020.02065 ↗
54. Rawte, V., Sheth, A., and Das, A. (2023). A survey of hallucination in large foundation models. arXiv.
55. Xu, Z., Jain, S., and Kankanhalli, M. (2024). Hallucination is inevitable: An innate limitation of large language models. arXiv.
56. Ji, Z., Lee, N., Frieske, R., Yu, T., Su, D., Xu, Y., Ishii, E., Bang, Y., Madotto, A., and Fung, P. (2023, January 6–10). Towards mitigating LLM hallucination via self reflection. Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, Singapore.10.18653/v1/2023.findings-emnlp.123 ↗
57. Martino, A., Iannelli, M., and Truong, C. (June, January 28). Knowledge injection to counter large language model (LLM) hallucination. Proceedings of the European Semantic Web Conference, Crete, Greece.