Applications of natural language processing and large language models in materials discovery
Source abstract
Abstract
The transformative impact of artificial intelligence (AI) technologies on materials science has revolutionized the study of materials problems. By leveraging well-characterized datasets derived from the scientific literature, AI-powered tools such as Natural Language Processing (NLP) have opened new avenues to accelerate materials research. The advances in NLP techniques and the development of large language models (LLMs) facilitate the efficient extraction and utilization of information. This review explores the application of NLP tools in materials science, focusing on automatic data extraction, materials discovery, and autonomous research. We also discuss the challenges and opportunities associated with utilizing LLMs and outline the prospects and advancements that will propel the field forward.
Source: openalex_inverted_index · Quality: baseline
Research context
OpenAlex topics: Machine Learning in Materials Science · Computational Drug Discovery Methods · X-ray Diffraction in Crystallography
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
References (100+)
Show source-provided references
1. Wang, W. Y. et al. Artificial intelligence enabled smart design and manufacturing of advanced materials: The endless Frontier in AI + era. Mater. Genome Eng. Adv. 2, e56 (2024).10.1002/mgea.56 ↗
2. Ramprasad, R., Batra, R., Pilania, G., Mannodi-Kanakkithodi, A. & Kim, C. Machine learning in materials informatics: recent applications and prospects. Npj Comput. Mater. 3, 54 (2017).10.1038/s41524-017-0056-5 ↗
3. Tshitoyan, V. et al. Unsupervised word embeddings capture latent knowledge from materials science literature. Nature 571, 95–98 (2019).10.1038/s41586-019-1335-8 ↗
4. Szymanski, N. J. et al. An autonomous laboratory for the accelerated synthesis of novel materials. Nature 624, 86–91 (2023).10.1038/s41586-023-06734-w ↗
5. Boiko, D. A., MacKnight, R., Kline, B. & Gomes, G. Autonomous chemical research with large language models. Nature 624, 570–578 (2023).10.1038/s41586-023-06792-0 ↗
6. Merchant, A. et al. Scaling deep learning for materials discovery. Nature 624, 80–85 (2023).10.1038/s41586-023-06735-9 ↗
7. Esterhuizen, J. A., Goldsmith, B. R. & Linic, S. Interpretable machine learning for knowledge generation in heterogeneous catalysis. Nat. Catal. 5, 175–184 (2022).10.1038/s41929-022-00744-z ↗
8. Hart, G. L. W., Mueller, T., Toher, C. & Curtarolo, S. Machine learning for alloys. Nat. Rev. Mater. 6, 730–755 (2021).10.1038/s41578-021-00340-w ↗
9. Gu, L. et al. Bond sensitive graph neural networks for predicting high temperature superconductors. Mater. Genome Eng. Adv. 2, e48 (2024).10.1002/mgea.48 ↗
10. Xie, J. Prospects of materials genome engineering frontiers. Mater. Genome Eng. Adv. 1, e17 (2023).10.1002/mgea.17 ↗
11. Li, S. et al. Optimal design of high‐performance rare‐earth‐free wrought magnesium alloys using machine learning. Mater. Genome Eng. Adv. 2, e45 (2024).10.1002/mgea.45 ↗
12. Jiang, X., Wang, Y., Jia, B., Qu, X. & Qin, M. Using machine learning to predict oxygen evolution activity for transition metal hydroxide electrocatalysts. ACS Appl. Mater. Interfaces 14, 41141–41148 (2022).10.1021/acsami.2c13435 ↗
13. Xue, D. et al. Accelerated search for materials with targeted properties by adaptive design. Nat. Commun. 7, 1–9 (2016).10.1038/ncomms11241 ↗
14. Jiang, X. et al. A strategy combining machine learning and multiscale calculation to predict tensile strength for pearlitic steel wires with industrial data. Scr. Mater. 186, 272–277 (2020).10.1016/j.scriptamat.2020.03.064 ↗
15. Wen, C. et al. Modeling solid solution strengthening in high entropy alloys using machine learning. Acta Mater. 212, 116917 (2021).10.1016/j.actamat.2021.116917 ↗
16. Wen, C. et al. Machine learning assisted design of high entropy alloys with desired property. Acta Mater. 170, 109–117 (2019).10.1016/j.actamat.2019.03.010 ↗
17. Zhang, Y. et al. Phase prediction in high entropy alloys with a rational selection of materials descriptors and machine learning models. Acta Mater. 185, 528–539 (2020).10.1016/j.actamat.2019.11.067 ↗
18. Yang, Z. et al. Scalable crystal structure relaxation using an iteration-free deep generative model with uncertainty quantification. Nat. Commun. 15, 8148 (2024).10.1038/s41467-024-52378-3 ↗
19. Olivetti, E. A. et al. Data-driven materials research enabled by natural language processing and information extraction. Appl. Phys. Rev. 7, 041317 (2020).10.1063/5.0021106 ↗
20. Jain, A. et al. Commentary: The Materials Project: A materials genome approach to accelerating materials innovation. APL Mater .1, 011002 (2013).10.1063/1.4812323 ↗
21. Kirklin, S. et al. The Open Quantum Materials Database (OQMD): assessing the accuracy of DFT formation energies. Npj Comput. Mater. 1, 1–15 (2015).10.1038/npjcompumats.2015.10 ↗
22. Liu, S. et al. An infrastructure with user-centered presentation data model for integrated management of materials data and services. Npj Comput. Mater. 7, 88 (2021).10.1038/s41524-021-00557-x ↗
23. Esters, M. et al. aflow. org: A web ecosystem of databases, software and tools. Comput. Mater. Sci. 216, 111808 (2023).10.1016/j.commatsci.2022.111808 ↗
24. Scheidgen, M. et al. NOMAD: A distributed web-based platform for managing materials science research data. J. Open Source Softw. 8, 5388 (2023).10.21105/joss.05388 ↗
25. Zhou, M., Duan, N., Liu, S. & Shum, H.-Y. Progress in neural NLP: modeling, learning, and reasoning. Engineering 6, 275–290 (2020).10.1016/j.eng.2019.12.014 ↗
26. Jessop, D. M., Adams, S. E., Willighagen, E. L., Hawizy, L. & Murray-Rust, P. OSCAR4: a flexible architecture for chemical text-mining. J. Cheminformatics 3, 41 (2011).10.1186/1758-2946-3-41 ↗
27. Mavračić, J., Court, C. J., Isazawa, T., Elliott, S. R. & Cole, J. M. ChemDataExtractor 2.0: Autopopulated Ontologies for Materials Science. J. Chem. Inf. Model. 61, 4280–4289 (2021).10.1021/acs.jcim.1c00446 ↗
28. Huang, S. & Cole, J. M. BatteryDataExtractor: battery-aware text-mining software embedded with BERT models. Chem. Sci. 13, 11487–11495 (2022).10.1039/D2SC04322J ↗
29. Swain, M. C. & Cole, J. M. ChemDataExtractor: A Toolkit For Automated Extraction Of Chemical Information From The Scientific Literature. J. Chem. Inf. Model. 56, 1894–1904 (2016).10.1021/acs.jcim.6b00207 ↗
30. Kononova, O. et al. Text-mined dataset of inorganic materials synthesis recipes. Sci. Data 6, 203 (2019).10.1038/s41597-019-0224-1 ↗
31. Vaucher, A. C. et al. Automated extraction of chemical synthesis actions from experimental procedures. Nat. Commun. 11, 3601 (2020).10.1038/s41467-020-17266-6 ↗
32. Huo, H. et al. Semi-supervised machine-learning classification of materials synthesis procedures. Npj Comput. Mater. 5, 62 (2019).10.1038/s41524-019-0204-1 ↗
33. Mehr, S. H. M., Craven, M., Leonov, A. I., Keenan, G. & Cronin, L. A universal system for digitization and automatic execution of the chemical synthesis literature. Science 370, 101–108 (2020).10.1126/science.abc2986 ↗
34. Cruse, K. et al. Text-mined dataset of gold nanoparticle synthesis procedures, morphologies, and size entities. Sci. Data 9, 234 (2022).10.1038/s41597-022-01321-6 ↗
35. Wang, W. et al. Automated pipeline for superalloy data by text mining. Npj Comput. Mater. 8, 9 (2022).10.1038/s41524-021-00687-2 ↗
36. Pfeiffer, O. P. et al. Aluminum alloy compositions and properties extracted from a corpus of scientific manuscripts and US patents. Sci. Data 9, 128 (2022).10.1038/s41597-022-01215-7 ↗
37. Wang, W. et al. Alloy synthesis and processing by semi-supervised text mining. Npj Comput. Mater. 9, 183 (2023).10.1038/s41524-023-01138-w ↗
38. Mikolov, T., Chen, K., Corrado, G. & Dean, J. Efficient estimation of word representations in vector space.Preprint at https://doi.org/10.48550/arXiv.1301.3781 (2013).10.48550/arXiv.1301.3781 ↗
39. Mikolov, T., Sutskever, I., Chen, K., Corrado, G. S. & Dean, J. Distributed representations of words and phrases and their compositionality. Adv. Neural Inf. Process. Syst .26, 3111–3119 (2013).
40. Pei, Z., Yin, J., Liaw, P. K. & Raabe, D. Toward the design of ultrahigh-entropy alloys via mining six million texts. Nat. Commun. 14, 54 (2023).10.1038/s41467-022-35766-5 ↗
41. OpenAI et al. GPT-4 Technical Report. Preprint at http://arxiv.org/abs/2303.08774 (2023).
42. Almazrouei, E. et al. The Falcon Series of Open Language Models. Preprint at http://arxiv.org/abs/2311.16867 (2023).
43. Devlin, J., Chang, M.-W., Lee, K. & Toutanova, K. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies. Vol. 1, 4171–4186. Preprint at http://arxiv.org/abs/1810.04805 (2019).
44. Thoppilan, R. et al. LaMDA: Language Models for Dialog Applications. Preprint at http://arxiv.org/abs/2201.08239 (2022).
45. Touvron, H. et al. LLaMA: Open and Efficient Foundation Language Models. Preprint at http://arxiv.org/abs/2302.13971 (2023).
46. Touvron, H. et al. Llama 2: Open Foundation and Fine-Tuned Chat Models. Preprint at http://arxiv.org/abs/2307.09288 (2023).
47. Chowdhery, A. et al. Palm: Scaling language modeling with pathways. J. Mach. Learn. Res. 24, 1–113 (2023).
48. Driess, D. et al. PaLM-E: An Embodied Multimodal Language Model. Preprint at http://arxiv.org/abs/2303.03378 (2023).
49. Vaswani, A. et al. Attention is all you need. Adv. Neural Inf. Process. Syst. 30, 5998–6008 (2017).
50. Kaplan, J. et al. Scaling Laws for Neural Language Models. Preprint at http://arxiv.org/abs/2001.08361 (2020).
51. Zhao, W. X. et al. A Survey of Large Language Models. Preprint at http://arxiv.org/abs/2303.18223 (2023).
52. Han, X. et al. Pre-trained models: Past, present and future. AI Open 2, 225–250 (2021).10.1016/j.aiopen.2021.08.002 ↗
53. Chen, M. et al. Evaluating Large Language Models Trained on Code. Preprint at http://arxiv.org/abs/2107.03374 (2021).
54. Brown, T. et al. Language models are few-shot learners. Adv. Neural Inf. Process. Syst. 33, 1877–1901 (2020).
55. Austin, J. et al. Program Synthesis with Large Language Models. Preprint at http://arxiv.org/abs/2108.07732 (2021).
56. Ahmad, W. U., Chakraborty, S., Ray, B. & Chang, K.-W. Unified Pre-training for Program Understanding and Generation. Preprint at http://arxiv.org/abs/2103.06333 (2021).10.18653/v1/2021.naacl-main.211 ↗
57. DeepSeek-AI et al. DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning. Preprint at https://doi.org/10.48550/arXiv.2501.12948 (2025).10.48550/arXiv.2501.12948 ↗
58. Pei, Z., Yin, J., Neugebauer, J. & Jain, A. Towards the holistic design of alloys with large language models. Nat. Rev. Mater. 9, 840–841 (2024).10.1038/s41578-024-00726-6 ↗
59. Zheng, Z., Zhang, O., Borgs, C., Chayes, J. T. & Yaghi, O. M. ChatGPT Chemistry Assistant for Text Mining and the Prediction of MOF Synthesis. J. Am. Chem. Soc. jacs.3c05819 https://doi.org/10.1021/jacs.3c05819 (2023).10.1021/jacs.3c05819 ↗
60. Kuenneth, C. & Ramprasad, R. polyBERT: a chemical language model to enable fully machine-driven ultrafast polymer informatics. Nat. Commun. 14, 4099 (2023).10.1038/s41467-023-39868-6 ↗
61. Kang, Y. ChatMOF: an artificial intelligence system for predicting and generating metal-organic frameworks using large language models. Nat. Commun. 15, 4705 (2024).10.1038/s41467-024-48998-4 ↗
62. Dong, L. et al. Unified language model pre-training for natural language understanding and generation. Adv. Neural Inf. Process 32, 13063–13075 (2019).
63. Cambria, E. & White, B. Jumping NLP curves: A review of natural language processing research. IEEE Comput. Intell. Mag. 9, 48–57 (2014).10.1109/MCI.2014.2307227 ↗
64. Bojanowski, P., Grave, E., Joulin, A. & Mikolov, T. Enriching word vectors with subword information. Trans. Assoc. Comput. Linguist. 5, 135–146 (2017).10.1162/tacl_a_00051 ↗
65. Pennington, J., Socher, R. & Manning, C. D. Glove: Global vectors for word representation. in Proceedings of the 2014 Conference on Empirical Methods In Natural Language Processing (EMNLP) 1532–1543 (2014).10.3115/v1/D14-1162 ↗
66. Erhan, D., Courville, A., Bengio, Y. & Vincent, P. Why does unsupervised pre-training help deep learning? in Proceedings of the 13th International Conference on Artificial Intelligence and Statistics 201–208 (JMLR Workshop and Conference Proceedings, 2010).
67. Song, K., Tan, X., Qin, T., Lu, J. & Liu, T.-Y. Mass: Masked sequence to sequence pre-training for language generation. https://doi.org/10.48550/arXiv.1905.02450 (2019).10.48550/arXiv.1905.02450 ↗
68. Peters, M. E. et al. Deep contextualized word representations. https://doi.org/10.48550/arXiv.1802.05365 (2018).10.48550/arXiv.1802.05365 ↗
69. Ethayarajh, K. How contextual are contextualized word representations? Comparing the geometry of BERT, ELMo, and GPT-2 embeddings. https://doi.org/10.48550/arXiv.1909.00512 (2019).10.48550/arXiv.1909.00512 ↗
70. Radford, A. et al. Language models are unsupervised multitask learners. OpenAI Blog 1, 9 (2019).
71. Radford, A., Narasimhan, K., Salimans, T. & Sutskever, I. Improving language understanding by generative pre-training. https://openai.com/index/language-unsupervised/ (2018).
72. Zhang, Y. & Yang, Q. A survey on multi-task learning. IEEE Trans. Knowl. Data Eng. 34, 5586–5609 (2021).10.1109/TKDE.2021.3070203 ↗
73. Wang, Y., Yao, Q., Kwok, J. T. & Ni, L. M. Generalizing from a few examples: A survey on few-shot learning. ACM Comput. Surv. Csur 53, 1–34 (2020).
74. Finn, C., Abbeel, P. & Levine, S. Model-agnostic meta-learning for fast adaptation of deep networks. in International Conference on Machine Learning 1126–1135 (PMLR, 2017).
75. Dong, Q. et al. A survey on in-context learning. https://doi.org/10.48550/arXiv.2301.00234 (2022).10.48550/arXiv.2301.00234 ↗
76. Ouyang, L. et al. Training language models to follow instructions with human feedback. Adv. Neural Inf. Process. Syst. 35, 27730–27744 (2022).
77. Lammey, R. CrossRef’s Text and Data Mining Services. Learn. Publ. 27, 245–250 (2014).10.1087/20140402 ↗
78. White, J. et al. A prompt pattern catalog to enhance prompt engineering with chatgpt. https://doi.org/10.48550/arXiv.2302.11382 (2023).10.48550/arXiv.2302.11382 ↗
79. Liu, Y. et al. Jailbreaking chatgpt via prompt engineering: An empirical study. https://doi.org/10.48550/arXiv.2305.13860 (2023).10.48550/arXiv.2305.13860 ↗
80. Dagdelen, J. et al. Structured information extraction from scientific text with large language models. Nat. Commun. 15, 1418 (2024).10.1038/s41467-024-45563-x ↗
81. Court, C. J. & Cole, J. M. Auto-generated materials database of Curie and Néel temperatures via semi-supervised relationship extraction. Sci. Data 5, 180111 (2018).10.1038/sdata.2018.111 ↗
82. Weston, L. et al. Named entity recognition and normalization applied to large-scale information extraction from the materials science literature. J. Chem. Inf. Model. 59, 3692–3702 (2019).10.1021/acs.jcim.9b00470 ↗
83. Shetty, P. et al. A general-purpose material property data extraction pipeline from large polymer corpora using natural language processing. Npj Comput. Mater. 9, 52 (2023).10.1038/s41524-023-01003-w ↗
84. Yan, R., Jiang, X., Wang, W., Dang, D. & Su, Y. Materials information extraction via automatically generated corpus. Sci. Data 9, 401 (2022).10.1038/s41597-022-01492-2 ↗
85. Hawizy, L., Jessop, D. M., Adams, N. & Murray-Rust, P. ChemicalTagger: A tool for semantic text-mining in chemistry. J. Cheminformatics 3, 1–13 (2011).10.1186/1758-2946-3-17 ↗
86. Beard, E. J. & Cole, J. M. Perovskite- and dye-sensitized solar-cell device databases auto-generated using ChemDataExtractor. Sci. Data 9, 329 (2022).10.1038/s41597-022-01355-w ↗
87. Dong, Q. & Cole, J. M. Auto-generated database of semiconductor band gaps using ChemDataExtractor. Sci. Data 9, 193 (2022).10.1038/s41597-022-01294-6 ↗
88. Kumar, P., Kabra, S. & Cole, J. M. Auto-generating databases of yield strength and grain size using ChemDataExtractor. Sci. Data 9, 292 (2022).10.1038/s41597-022-01301-w ↗
89. Zhao, J. & Cole, J. M. A database of refractive indices and dielectric constants auto-generated using ChemDataExtractor. Sci. Data 9, 192 (2022).10.1038/s41597-022-01295-5 ↗
90. Kim, E. et al. Materials synthesis insights from scientific literature via text extraction and machine learning. Chem. Mater. 29, 9436–9444 (2017).10.1021/acs.chemmater.7b03500 ↗
91. Kim, E. et al. Machine-learned and codified synthesis parameters of oxide materials. Sci. Data 4, 170127 (2017).10.1038/sdata.2017.127 ↗
92. Wilary, D. M. & Cole, J. M. ReactionDataExtractor: A Tool for Automated Extraction of Information from Chemical Reaction Schemes. J. Chem. Inf. Model. 61, 4962–4974 (2021).10.1021/acs.jcim.1c01017 ↗
93. He, T. et al. Similarity of Precursors in Solid-State Synthesis as Text-Mined from Scientific Literature. Chem. Mater. 32, 7861–7873 (2020).10.1021/acs.chemmater.0c02553 ↗
94. He, T. et al. Precursor recommendation for inorganic synthesis by machine learning materials similarity from scientific literature. Sci. Adv. 9, eadg8180 (2023).10.1126/sciadv.adg8180 ↗
95. Wang, Z. et al. Dataset of solution-based inorganic materials synthesis procedures extracted from the scientific literature. Sci. Data 9, 231 (2022).10.1038/s41597-022-01317-2 ↗
96. Kim, E. et al. Inorganic Materials Synthesis Planning with Literature-Trained Neural Networks. J. Chem. Inf. Model. 60, 1194–1201 (2020).10.1021/acs.jcim.9b00995 ↗
97. Kim, E., Huang, K., Jegelka, S. & Olivetti, E. Virtual screening of inorganic materials synthesis parameters with deep learning. Npj Comput. Mater. 3, 53 (2017).10.1038/s41524-017-0055-6 ↗
98. da Silva, V. T. et al. Automated, LLM enabled extraction of synthesis details for reticular materials from scientific literature. https://doi.org/10.48550/arXiv.2411.03484 (2024).10.48550/arXiv.2411.03484 ↗
99. Thway, M. et al. Harnessing GPT-3.5 for text parsing in solid-state synthesis–case study of ternary chalcogenides. Digit. Discov. 3, 328–336 (2024).10.1039/D3DD00202K ↗
100. Polak, M. P. & Morgan, D. Extracting accurate materials data from research papers with conversational language models and prompt engineering. Nat. Commun. 15, 1569 (2024).10.1038/s41467-024-45914-8 ↗