A general-purpose material property data extraction pipeline from large polymer corpora using natural language processing
Source abstract
Abstract
Abstract The ever-increasing number of materials science articles makes it hard to infer chemistry-structure-property relations from literature. We used natural language processing methods to automatically extract material property data from the abstracts of polymer literature. As a component of our pipeline, we trained MaterialsBERT, a language model, using 2.4 million materials science abstracts, which outperforms other baseline models in three out of five named entity recognition datasets. Using this pipeline, we obtained ~300,000 material property records from ~130,000 abstracts in 60 hours. The extracted data was analyzed for a diverse range of applications such as fuel cells, supercapacitors, and polymer solar cells to recover non-trivial insights. The data extracted through our pipeline is made available at polymerscholar.org which can be used to locate material property data recorded in abstracts. This work demonstrates the feasibility of an automatic pipeline that starts from published literature and ends with extracted material property information.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Machine Learning in Materials Science · Fuel Cells and Related Materials · Computational Drug Discovery Methods
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
References (71)
Show source-provided references
1. Kenton, J. D. M.-W. C. & Toutanova, L. K. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of NAACL-HLT, vol. 1, p. 2 (2019).
2. Vaswani, A. et al. Attention is all you need. Adv Neural Inf Process Syst 30 (2017).
3. Swain, M. C. & Cole, J. M. Chemdataextractor: a toolkit for automated extraction of chemical information from the scientific literature. J. Chem. Inf. Model 56, 1894–1904 (2016).10.1021/acs.jcim.6b00207 ↗
4. Rocktäschel, T., Weidlich, M. & Leser, U. Chemspot: a hybrid system for chemical named entity recognition. Bioinformatics 28, 1633–1640 (2012).10.1093/bioinformatics/bts183 ↗
5. Hawizy, L., Jessop, D. M., Adams, N. & Murray-Rust, P. Chemicaltagger: a tool for semantic text-mining in chemistry. J. Cheminformatics 3, 17 (2011).10.1186/1758-2946-3-17 ↗
6. Court, C. J. & Cole, J. M. Auto-generated materials database of curie and néel temperatures via semi-supervised relationship extraction. Sci. Data 5, 1–12 (2018).10.1038/sdata.2018.111 ↗
7. Court, C. J., Jain, A. & Cole, J. M. Inverse design of materials that exhibit the magnetocaloric effect by text-mining of the scientific literature and generative deep learning. Chem. Mater. 33, 7217–7231 (2021).10.1021/acs.chemmater.1c01368 ↗
8. Tchoua, R. B. et al. Blending education and polymer science: semiautomated creation of a thermodynamic property database. J. Chem. Educ. 93, 1561–1568 (2016).10.1021/acs.jchemed.5b01032 ↗
9. Tchoua, R. B. et al. Creating training data for scientific named entity recognition with minimal human effort, 398–411 (Springer, 2019).10.1007/978-3-030-22734-0_29 ↗
10. Kim, E. et al. Materials synthesis insights from scientific literature via text extraction and machine learning. Chem. Mater. 29, 9436–9444 (2017).10.1021/acs.chemmater.7b03500 ↗
11. Kononova, O. et al. Text-mined dataset of inorganic materials synthesis recipes. Sci. Data 6, 1–11 (2019).10.1038/s41597-018-0005-2 ↗
12. Kemp, N. & Lynch, M. Extraction of information from the text of chemical patents. 1. identification of specific chemical names. J. Chem. Inf. Comput Sci. 38, 544–551 (1998).10.1021/ci980324v ↗
13. Klinger, R., Kolářik, C., Fluck, J., Hofmann-Apitius, M. & Friedrich, C M. Detection of IUPAC and IUPAC-like chemical names. Bioinformatics 24, i268–i276 (2008).10.1093/bioinformatics/btn181 ↗
14. Weininger, D., Weininger, A. & Weininger, J. L. Smiles. 2. algorithm for generation of unique smiles notation. J. Chem. Inf. Comput Sci. 29, 97–101 (1989).10.1021/ci00062a008 ↗
15. Friedl, J. E. Mastering regular expressions (O’Reilly Media, Inc., 2006).
16. Schwalbe-Koda, D., Jensen, Z., Olivetti, E. & Gómez-Bombarelli, R. Graph similarity drives zeolite diffusionless transformations and intergrowth. Nat. Mater. 18, 1177–1181 (2019).10.1038/s41563-019-0486-1 ↗
17. Shetty, P. & Ramprasad, R. Automated knowledge extraction from polymer literature using natural language processing. iScience 24, 101922 (2020).10.1016/j.isci.2020.101922 ↗
18. Tshitoyan, V. et al. Unsupervised word embeddings capture latent knowledge from materials science literature. Nature 571, 95–98 (2019).10.1038/s41586-019-1335-8 ↗
19. Gu, Y. et al. Domain-specific language model pretraining for biomedical natural language processing. ACM Trans. Comput. Healthc. (HEALTH) 3, 1–23 (2021).
20. Gupta, T., Zaki, M. & Krishnan, N. A. Matscibert: A materials domain language model for text mining and information extraction. Npj Comput. Mater. 8, 102 (2022).10.1038/s41524-022-00784-w ↗
21. Yoshitake, M., Sato, F., Kawano, H. & Teraoka, H. Materialbert for natural language processing of materials science texts. Sci. Technol. Adv. Mater. 2, 372–380 (2022).
22. Lee, J. et al. Biobert: a pre-trained biomedical language representation model for biomedical text mining. Bioinformatics 36, 1234–1240 (2020).10.1093/bioinformatics/btz682 ↗
23. Guo, J. et al. Automated chemical reaction extraction from scientific literature. J. Chem. Inf. Model. 62, 2035–2045 (2021).10.1021/acs.jcim.1c00284 ↗
24. Weston, L. et al. Named entity recognition and normalization applied to large-scale information extraction from the materials science literature. J. Chem. Inf. Model 59, 3692–3702 (2019).10.1021/acs.jcim.9b00470 ↗
25. Fleiss, J. L. Measuring nominal scale agreement among many raters. Psychol. Bull. 76, 378 (1971).10.1037/h0031619 ↗
26. Tabassum, J., Lee, S., Xu, W. & Ritter, A. Wnut-2020 task 1 overview: Extracting entities and relations from wet lab protocols. arXiv preprint arXiv:2010.14576. https://arxiv.org/abs/2010.14576 (2020).10.18653/v1/2020.wnut-1.33 ↗
27. Beltagy, I., Peters, M. E. & Cohan, A. Longformer: The long-document transformer. arXiv preprint arXiv:2004.05150. https://arxiv.org/abs/2004.05150 (2020).
28. Trewartha, A. et al. Quantifying the advantage of domain-specific pre-training on named entity recognition tasks in materials science. Patterns 3, 100488 (2022).10.1016/j.patter.2022.100488 ↗
29. Huang, Z., Xu, W. & Yu, K. Bidirectional lstm-crf models for sequence tagging. arXiv preprint arXiv:1508.01991. https://arxiv.org/abs/1508.01991 (2015).
30. Otsuka, S., Kuwajima, I., Hosoya, J., Xu, Y. & Yamazaki, M. Polyinfo: Polymer database for polymeric materials design, In 2011 International Conference on Emerging Intelligent Data and Web Technologies, pp. 22–29 (2011).10.1109/EIDWT.2011.13 ↗
31. Shetty, P. & Ramprasad, R. Machine-guided polymer knowledge extraction using natural language processing: The example of named entity normalization. J. Chem. Inf. Model 61, 5377–5385 (2021).10.1021/acs.jcim.1c00554 ↗
32. Palomba, D., Vazquez, G. E. & Díaz, M. F. Prediction of elongation at break for linear polymers. Chemom. Intell. Lab Syst. 139, 121–131 (2014).10.1016/j.chemolab.2014.09.009 ↗
33. Doan Tran, H. et al. Machine-learning predictions of polymer properties with polymer genome. J. Appl. Phys. 128, 171104 (2020).10.1063/5.0023759 ↗
34. Zhang, M., Niu, H. & Wu, D. Polyimide fibers with high strength and high modulus: preparation, structures, properties, and applications. Macromol. Rapid Commun. 39, 1800141 (2018).10.1002/marc.201800141 ↗
35. Wang, C., Zhang, S., Zhang, L., Xu, Y. & Zhang, L. Evading the strength–ductility trade-off dilemma of rigid thermosets by incorporating triple cross-links of varying strengths. Polym. Chem. 11, 6281–6287 (2020).10.1039/D0PY00928H ↗
36. Zhang, Z.-G. & Li, Y. Polymerized small-molecule acceptors for high-performance all-polymer solar cells. Angew. Chem. Int 60, 4422–4433 (2021).10.1002/anie.202009666 ↗
37. Nagasawa, S., Al-Naamani, E. & Saeki, A. Computer-aided screening of conjugated polymers for organic solar cell: classification by random forest. J. Phys. Chem. 9, 2639–2646 (2018).
38. Zhang, M. et al. Metallated terpolymer donors with strongly absorbing iridium complex enables polymer solar cells with 16.71% efficiency. Chem. Eng. J. 430, 132832 (2022).10.1016/j.cej.2021.132832 ↗
39. Abdelkareem, M. A. et al. Environmental aspects of fuel cells: A review. Sci. Total Environ. 752, 141803 (2021).10.1016/j.scitotenv.2020.141803 ↗
40. Larminie, J., Dicks, A. & McDonald, M. S. Fuel cell systems explained Vol. 2 (J. Wiley Chichester, UK, 2003).10.1002/9781118878330 ↗
41. Shaari, N. et al. Enhanced proton conductivity and methanol permeability reduction via sodium alginate electrolyte-sulfonated graphene oxide bio-membrane. Nanoscale Res. Lett. 13, 1–16 (2018).10.1186/s11671-018-2493-6 ↗
42. Catenaro, E., Rizzo, D. M. & Onori, S. Experimental analysis and analytical modeling of enhanced-ragone plot. Appl. Energy 291, 116473 (2021).10.1016/j.apenergy.2021.116473 ↗
43. Shown, I., Ganguly, A., Chen, L.-C. & Chen, K.-H. Conducting polymer-based flexible supercapacitor. Energy Sci. Eng. 3, 2–26 (2015).10.1002/ese3.50 ↗
44. Uppugalla, S. & Srinivasan, P. Polyaniline nanofibers and porous ni [oh] 2 sheets coated carbon fabric for high performance super capacitor. J. Appl. Polym. Sci. 136, 48042 (2019).10.1002/app.48042 ↗
45. Li, Y. et al. Coupled and decoupled hierarchical carbon nanomaterials toward high-energy-density quasi-solid-state na-ion hybrid energy storage devices. Energy Stor. Mater. 23, 530–538 (2019).
46. Javed, M. S. et al. Achieving high rate and high energy density in an all-solid-state flexible asymmetric pseudocapacitor through the synergistic design of binder-free 3d znco 2 o 4 nano polyhedra and 2d layered ti 3 c 2 t x-mxenes. J. Mater. Chem. 7, 24543–24556 (2019).10.1039/C9TA08227A ↗
47. Fu, H., Wang, Z. & Sun, Y. Polymer donors for high-performance non-fullerene organic solar cells. Angew. Chem. Int 58, 4442–4453 (2019).10.1002/anie.201806291 ↗
48. Ciotti, M. et al. The covid-19 pandemic. Crit. Rev. Clin. Lab Sci. 57, 365–388 (2020).10.1080/10408363.2020.1783198 ↗
49. Gao, J., Yin, Y., Myers, K. R., Lakhani, K. R. & Wang, D. Potentially long-lasting effects of the pandemic on scientists. Nat. Commun. 12, 1–6 (2021).10.1038/s41467-021-26428-z ↗
50. Li, Y., Shetty, P., Liu, L., Zhang, C. & Song, L. BERTifying the hidden Markov model for multi-source weakly supervised named entity recognition, 6178–6190 (Association for Computational Linguistics, Online, 2021). https://aclanthology.org/2021.acl-long.482.
51. Ratner, A. et al. Snorkel: Rapid training data creation with weak supervision, Vol. 11, 269 (NIH Public Access, 2017).10.14778/3157794.3157797 ↗
52. Zhang, R., Yu, Y., Shetty, P., Song, L. & Zhang, C. Prompt-based rule discovery and boosting for interactive weakly-supervised learning, 745–758 (Association for Computational Linguistics, Dublin, Ireland, 2022). https://aclanthology.org/2022.acl-long.55.10.18653/v1/2022.acl-long.55 ↗
53. Rajan, K., Zielesny, A. & Steinbeck, C. Decimer: towards deep learning for chemical image recognition. J. Cheminformatics 12, 1–9 (2020).10.1186/s13321-020-00469-w ↗
54. Khokhlov, I., Krasnov, L., Fedorov, M. V. & Sosnin, S. Image2smiles: Transformer-based molecular optical recognition engine. Chem.-Methods 2, e202100069 (2022).10.1002/cmtd.202100069 ↗
55. Chen, L. et al. Polymer informatics: Current status and critical next steps. Mater. Sci. Eng. R. Rep. 144, 100595 (2021).10.1016/j.mser.2020.100595 ↗
56. Wu, C. et al. Dielectric polymers tolerant to electric field and temperature extremes: Integration of phenomenology, informatics, and experimental validation. ACS Appl Mater. Interfaces 13, 53416–53424 (2021).10.1021/acsami.1c11885 ↗
57. Wu, Y. et al. Google’s neural machine translation system: Bridging the gap between human and machine translation. arXiv preprint arXiv:1609.08144. https://arxiv.org/abs/1609.08144 (2016).
58. Song, X., Salcianu, A., Song, Y., Dopson, D. & Zhou, D. Fast WordPiece tokenization, 2089–2103 (Association for Computational Linguistics, Online and Punta Cana, Dominican Republic, 2021). https://aclanthology.org/2021.emnlp-main.160.10.18653/v1/2021.emnlp-main.160 ↗
59. Kingma, D. P. & Ba, J. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980. https://arxiv.org/abs/1412.6980 (2014).
60. Liang, C. et al. Bond: Bert-assisted open-domain named entity recognition with distant supervision, KDD ’20, 1054-1064 (Association for Computing Machinery, New York, NY, USA, 2020). https://doi.org/10.1145/3394486.3403149.10.1145/3394486.3403149 ↗
61. Zhu, Y. et al. Aligning books and movies: Towards story-like visual explanations by watching movies and reading books, In Proceedings of the IEEE international conference on computer vision, pp. 19–27 (2015).10.1109/ICCV.2015.11 ↗
62. Liu, Y. et al. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692. https://arxiv.org/abs/1907.11692 (2019).
63. Wolf, T. et al. Transformers: State-of-the-art natural language processing, 38–45 (Association for Computational Linguistics, Online, 2020). https://aclanthology.org/2020.emnlp-demos.6.
64. Huang, K., Altosaar, J. & Ranganath, R. Clinicalbert: Modeling clinical notes and predicting hospital readmission. arXiv preprint arXiv:1904.05342. https://arxiv.org/abs/1904.05342 (2019).
65. Araci, D. Finbert: Financial sentiment analysis with pre-trained language models. arXiv preprint arXiv:1908.10063. https://arxiv.org/abs/1908.10063 (2019).
66. Levenshtein, V. I. Binary codes capable of correcting deletions, insertions, and reversals. Dokl. Phys. 10, 707–710 (1966).
67. Mitkov, R. Anaphora resolution (Routledge, 2014).10.4324/9781315840086 ↗
68. Wang, L., Cao, Z., De Melo, G. & Liu, Z. Relation classification via multi-level attention CNNS, 1298–1307 (2016).10.18653/v1/P16-1123 ↗
69. Zhong, Z. & Chen, D. A frustratingly easy approach for entity and relation extraction, 50–61 (Association for Computational Linguistics, Online, 2021). https://aclanthology.org/2021.naacl-main.5.10.18653/v1/2021.naacl-main.5 ↗
70. Krallinger, M. et al. The Chemdner corpus of chemicals and drugs and its annotation principles. J. Cheminformatics 7, 1–17 (2015).10.1186/1758-2946-7-S1-S1 ↗
71. Mysore, S. et al. The materials science procedural text corpus: Annotating materials synthesis procedures with shallow semantic structures. arXiv preprint arXiv:1905.06939. https://arxiv.org/abs/1905.06939 (2019).10.18653/v1/W19-4007 ↗