Recent advances and applications of deep learning methods in materials science
Source abstract
Abstract
Abstract Deep learning (DL) is one of the fastest-growing topics in materials data science, with rapidly emerging applications spanning atomistic, image-based, spectral, and textual data modalities. DL allows analysis of unstructured data and automated identification of features. The recent development of large materials databases has fueled the application of DL methods in atomistic prediction in particular. In contrast, advances in image and spectral data have largely leveraged synthetic data enabled by high-quality forward models as well as by generative unsupervised DL methods. In this article, we present a high-level overview of deep learning methods followed by a detailed discussion of recent developments of deep learning in atomistic simulation, materials imaging, spectral analysis, and natural language processing. For each modality we discuss applications involving both theoretical and experimental data, typical modeling approaches with their strengths and limitations, and relevant publicly available software and datasets. We conclude the review with a discussion of recent cross-cutting work related to uncertainty quantification in this field and a brief perspective on limitations, challenges, and potential growth areas for DL methods in materials science.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Machine Learning in Materials Science · X-ray Diffraction in Crystallography · Electron and X-Ray Spectroscopy Techniques
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
References (100+)
Show source-provided references
1. Callister, W. D. et al. Materials Science and Engineering: An Introduction (Wiley, 2021).
2. Saito, T. Computational Materials Design, Vol. 34 (Springer Science & Business Media, 2013).
3. Choudhary, K. et al. The joint automated repository for various integrated simulations (jarvis) for data-driven materials design. npj Comput. Mater. 6, 1–13 (2020).10.1038/s41524-020-00440-1 ↗
4. Kirklin, S. et al. The open quantum materials database (oqmd): assessing the accuracy of dft formation energies. npj Comput. Mater. 1, 1–15 (2015).10.1038/npjcompumats.2015.10 ↗
5. Jain, A. et al. Commentary: The materials project: A materials genome approach to accelerating materials innovation. APL Mater. 1, 011002 (2013).10.1063/1.4812323 ↗
6. Curtarolo, S. et al. Aflow: An automatic framework for high-throughput materials discovery. Comput. Mater. Sci. 58, 218–226 (2012).10.1016/j.commatsci.2012.02.005 ↗
7. Ramakrishnan, R., Dral, P. O., Rupp, M. & Von Lilienfeld, O. A. Quantum chemistry structures and properties of 134 kilo molecules. Sci. Data 1, 1–7 (2014).10.1038/sdata.2014.22 ↗
8. Draxl, C. & Scheffler, M. Nomad: The fair concept for big data-driven materials science. MRS Bull. 43, 676–682 (2018).10.1557/mrs.2018.208 ↗
9. Wang, R., Fang, X., Lu, Y., Yang, C.-Y. & Wang, S. The pdbbind database: methodologies and updates. J. Med. Chem. 48, 4111–4119 (2005).10.1021/jm048957q ↗
10. Zakutayev, A. et al. An open experimental database for exploring inorganic materials. Sci. Data 5, 1–12 (2018).10.1038/sdata.2018.53 ↗
11. de Pablo, J. J. et al. New frontiers for the materials genome initiative. npj Comput. Mater. 5, 1–23 (2019).10.1038/s41524-019-0173-4 ↗
12. Wilkinson, M. D. et al. The fair guiding principles for sci. data management and stewardship. Sci. Data 3, 1–9 (2016).10.1038/sdata.2016.18 ↗
13. Friedman, J. et al. The Elements of Statistical Learning, Vol. 1 (Springer series in statistics New York, 2001).
14. Agrawal, A. & Choudhary, A. Perspective: Materials informatics and big data: Realization of the “fourth paradigm” of science in materials science. APL Mater. 4, 053208 (2016).10.1063/1.4946894 ↗
15. Vasudevan, R. K. et al. Materials science in the artificial intelligence age: high-throughput library generation, machine learning, and a pathway from correlations to the underpinning physics. MRS Commun. 9, 821–838 (2019).10.1557/mrc.2019.95 ↗
16. Schmidt, J., Marques, M. R., Botti, S. & Marques, M. A. Recent advances and applications of machine learning in solid-state materials science. npj Comput. Mater. 5, 1–36 (2019).10.1038/s41524-019-0221-0 ↗
17. Butler, K. T., Davies, D. W., Cartwright, H., Isayev, O. & Walsh, A. Machine learning for molecular and materials science. Nature 559, 547–555 (2018).10.1038/s41586-018-0337-2 ↗
18. Xu, Y. et al. Deep dive into machine learning models for protein engineering. J. Chem. Inf. Model. 60, 2773–2790 (2020).10.1021/acs.jcim.0c00073 ↗
19. Schleder, G. R., Padilha, A. C., Acosta, C. M., Costa, M. & Fazzio, A. From dft to machine learning: recent approaches to materials science–a review. J. Phys. Mater. 2, 032001 (2019).10.1088/2515-7639/ab084b ↗
20. Agrawal, A. & Choudhary, A. Deep materials informatics: applications of deep learning in materials science. MRS Commun. 9, 779–792 (2019).10.1557/mrc.2019.73 ↗
21. Goodfellow, I., Bengio, Y. & Courville, A. Deep Learning (MIT Press, 2016).
22. LeCun, Y., Bengio, Y. & Hinton, G. Deep learning. Nature 521, 436–444 (2015).10.1038/nature14539 ↗
23. McCulloch, W. S. & Pitts, W. A logical calculus of the ideas immanent in nervous activity. Bull. Math. Biophys. 5, 115–133 (1943).10.1007/BF02478259 ↗
24. Rosenblatt, F. The perceptron: A probabilistic model for information storage and organization in the brain. Psychol. Rev. 65, 386–408 (1958).10.1037/h0042519 ↗
25. Gibney, E. Google ai algorithm masters ancient game of go. Nat. News 529, 445 (2016).10.1038/529445a ↗
26. Ramos, S., Gehrig, S., Pinggera, P., Franke, U. & Rother, C. Detecting unexpected obstacles for self-driving cars: Fusing deep learning and geometric modeling. in 2017 IEEE Intelligent Vehicles Symposium (IV), 1025–1032 (IEEE, 2017).10.1109/IVS.2017.7995849 ↗
27. Buduma, N. & Locascio, N. Fundamentals of deep learning: Designing next-generation machine intelligence algorithms (O’Reilly Media, Inc., O’Reilly, 2017).
28. Kearnes, S., McCloskey, K., Berndl, M., Pande, V. & Riley, P. Molecular graph convolutions: moving beyond fingerprints. J. Computer Aided Mol. Des. 30, 595–608 (2016).10.1007/s10822-016-9938-8 ↗
29. Albrecht, T., Slabaugh, G., Alonso, E. & Al-Arif, S. M. R. Deep learning for single-molecule science. Nanotechnology 28, 423001 (2017).10.1088/1361-6528/aa8334 ↗
30. Ge, M., Su, F., Zhao, Z. & Su, D. Deep learning analysis on microscopic imaging in materials science. Mater. Today Nano 11, 100087 (2020).10.1016/j.mtnano.2020.100087 ↗
31. Agrawal, A., Gopalakrishnan, K. & Choudhary, A. In Handbook on Big Data and Machine Learning in the Physical Sciences: Volume 1. Big Data Methods in Experimental Materials Discovery World Scientific Series on Emerging Technologies, 205–230 (“World Scientific, 2020).10.1142/9789811204555_0006 ↗
32. Erdmann, M., Glombitza, J., Kasieczka, G. & Klemradt, U. Deep Learning for Physics Research (World Scientific, 2021).10.1142/12294 ↗
33. Chen, C., Ye, W., Zuo, Y., Zheng, C. & Ong, S. P. Graph networks as a universal machine learning framework for molecules and crystals. Chem. Mater. 31, 3564–3572 (2019).10.1021/acs.chemmater.9b01294 ↗
34. Jha, D. et al. Enhancing materials property prediction by leveraging computational and experimental data using deep transfer learning. Nat. Commun. 10, 1–12 (2019).10.1038/s41467-019-13297-w ↗
35. Cubuk, E. D., Sendek, A. D. & Reed, E. J. Screening billions of candidates for solid lithium-ion conductors: a transfer learning approach for small data. J. Chem. Phys. 150, 214701 (2019).10.1063/1.5093220 ↗
36. Chen, C., Zuo, Y., Ye, W., Li, X. & Ong, S. P. Learning properties of ordered and disordered materials from multi-fidelity data. Nat. Comput. Sci. 1, 46–53 (2021).10.1038/s43588-020-00002-x ↗
37. Artrith, N. et al. Best practices in machine learning for chemistry. Nat. Chem. 13, 505–508 (2021).10.1038/s41557-021-00716-z ↗
38. Holm, E. A. In defense of the black box. Science 364, 26–27 (2019).10.1126/science.aax0162 ↗
39. Mueller, T., Kusne, A. G. & Ramprasad, R. Machine learning in materials science: Recent progress and emerging applications. Rev. Comput. Chem. 29, 186–273 (2016).
40. Wei, J. et al. Machine learning in materials science. InfoMat 1, 338–358 (2019).10.1002/inf2.12028 ↗
41. Liu, Y. et al. Machine learning in materials genome initiative: a review. J. Mater. Sci. Technol. 57, 113–122 (2020).10.1016/j.jmst.2020.01.067 ↗
42. Wang, A. Y.-T. et al. Machine learning for materials scientists: an introductory guide toward best practices. Chem. Mater. 32, 4954–4965 (2020).10.1021/acs.chemmater.0c01907 ↗
43. Morgan, D. & Jacobs, R. Opportunities and challenges for machine learning in materials science. Annu. Rev. Mater. Res. 50, 71–103 (2020).10.1146/annurev-matsci-070218-010015 ↗
44. Himanen, L., Geurts, A., Foster, A. S. & Rinke, P. Data-driven materials science: status, challenges, and perspectives. Adv. Sci. 6, 1900808 (2019).10.1002/advs.201900808 ↗
45. Rajan, K. Informatics for materials science and engineering: data-driven discovery for accelerated experimentation and application (Butterworth-Heinemann, 2013).
46. Montáns, F. J., Chinesta, F., Gómez-Bombarelli, R. & Kutz, J. N. Data-driven modeling and learning in science and engineering. Comptes Rendus Mécanique 347, 845–855 (2019).10.1016/j.crme.2019.11.009 ↗
47. Aykol, M. et al. The materials research platform: defining the requirements from user stories. Matter 1, 1433–1438 (2019).10.1016/j.matt.2019.10.024 ↗
48. Stanev, V., Choudhary, K., Kusne, A. G., Paglione, J. & Takeuchi, I. Artificial intelligence for search and discovery of quantum materials. Commun. Mater. 2, 1–11 (2021).10.1038/s43246-021-00209-z ↗
49. Chen, C. et al. A critical review of machine learning of energy materials. Adv. Energy Mater. 10, 1903242 (2020).10.1002/aenm.201903242 ↗
50. Cybenko, G. Approximation by superpositions of a sigmoidal function. Math. Control Signals Syst. 2, 303–314 (1989).10.1007/BF02551274 ↗
51. Kidger, P. & Lyons, T. Universal approximation with deep narrow networks. in Conference on learning theory, 2306–2327 (PMLR, 2020).
52. Lin, H. W., Tegmark, M. & Rolnick, D. Why does deep and cheap learning work so well? J. Stat. Phys. 168, 1223–1247 (2017).10.1007/s10955-017-1836-5 ↗
53. Minsky, M. & Papert, S. A. Perceptrons: An introduction to computational geometry (MIT press, 2017).10.7551/mitpress/11301.001.0001 ↗
54. Paszke, A. et al. Pytorch: An imperative style, high-performance deep learning library. Adv. Neural Inf. Process. Syst. 32, 8026–8037 (2019).
55. Abadi et al., TensorFlow: A system for large-scale machine learning. arXiv:1605.08695, Preprint at https://arxiv.org/abs/1605.08695 (2006).
56. Chen, T. et al. Mxnet: A flexible and efficient machine learning library for heterogeneous distributed systems. arXiv. https://arxiv.org/abs/1512.01274 (2015).
57. Nwankpa, C., Ijomah, W., Gachagan, A. & Marshall, S. Activation functions: comparison of trends in practice and research for deep learning. arXiv. https://arxiv.org/abs/1811.03378 (2018).
58. Baydin, A. G., Pearlmutter, B. A., Radul, A. A. & Siskind, J. M. Automatic differentiation in machine learning: a survey. J. Machine Learn. Res. 18, 1–43 (2018).
59. Hinton, G. E., Srivastava, N., Krizhevsky, A., Sutskever, I. & Salakhutdinov, R. R. Improving neural networks by preventing co-adaptation of feature detectors. arXiv. https://arxiv.org/abs/1207.0580 (2012).
60. Breiman, L. Bagging predictors. Machine Learn. 24, 123–140 (1996).10.1007/BF00058655 ↗
61. LeCun, Y. et al. The Handbook of Brain Theory and Neural Networks vol. 3361 (MIT press Cambridge, MA, USA 1995).
62. Wilson, R. J. Introduction to Graph Theory (Pearson Education India, 1979).
63. West, D. B. et al. Introduction to Graph Theory Vol. 2 (Prentice hall Upper Saddle River, 2001).
64. Wang, M. et al. Deep graph library: A graph-centric, highly-performant package for graph neural networks. arXiv. https://arxiv.org/abs/1909.01315 (2019).
65. Choudhary, K. & DeCost, B. Atomistic line graph neural network for improved materials property predictions. npj Comput. Mater. 7, 1–8 (2021).10.1038/s41524-021-00650-1 ↗
66. Li, M. et al. Dgl-lifesci: An open-source toolkit for deep learning on graphs in life science. arXiv. https://arxiv.org/abs/2106.14232 (2021).
67. Xie, T. & Grossman, J. C. Crystal graph convolutional neural networks for an accurate and interpretable prediction of material properties. Phys. Rev. Lett. 120, 145301 (2018).10.1103/PhysRevLett.120.145301 ↗
68. Klicpera, J., Groß, J. & Günnemann, S. Directional message passing for molecular graphs. arXiv. https://arxiv.org/abs/2003.03123 (2020).
69. Schutt, K. et al. Schnetpack: A deep learning toolbox for atomistic systems. J. Chem. Theory Comput. 15, 448–455 (2018).10.1021/acs.jctc.8b00908 ↗
70. Kipf, T. N. & Welling, M. Semi-supervised classification with graph convolutional networks. arXiv. https://arxiv.org/abs/1609.02907 (2016).
71. Veličković, P. et al. Graph attention networks. arXiv. https://arxiv.org/abs/1710.10903 (2017).
72. Schlichtkrull, M. et al. Modeling relational data with graph convolutional networks. arXiv. https://arxiv.org/abs/1703.06103 (2017).
73. Song, L., Zhang, Y., Wang, Z. & Gildea, D. A graph-to-sequence model for AMR-to-text generation. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 1616–1626 (Association for Computational Linguistics, 2018).10.18653/v1/P18-1150 ↗
74. Xu, K., Hu, W., Leskovec, J. & Jegelka, S. How powerful are graph neural networks? arXiv. https://arxiv.org/abs/1810.00826 (2018).
75. Chen, Z., Li, X. & Bruna, J. Supervised community detection with line graph neural networks. arXiv. https://arxiv.org/abs/1705.08415 (2017).
76. Jing, Y., Bian, Y., Hu, Z., Wang, L. & Xie, X.-Q. S. Deep learning for drug design: an artificial intelligence paradigm for drug discovery in the big data era. AAPS J. 20, 1–10 (2018).10.1208/s12248-017-0160-y ↗
77. Devlin, J., Chang, M.-W., Lee, K. & Toutanova, K. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv. https://arxiv.org/abs/1810.04805 (2018).
78. De Cao, N. & Kipf, T. Molgan: An implicit generative model for small molecular graphs. arXiv. https://arxiv.org/abs/1805.11973 (2018).
79. Pereira, T., Abbasi, M., Ribeiro, B. & Arrais, J. P. Diversity oriented deep reinforcement learning for targeted molecule generation. J. Cheminformatics 13, 1–17 (2021).10.1186/s13321-021-00498-z ↗
80. Baker, N. et al. Workshop report on basic research needs for scientific machine learning: core technologies for artificial intelligence. Tech. Rep. https://doi.org/10.2172/1478744. (2019).10.2172/1478744 ↗
81. Chan, H. et al. Rapid 3d nanoscale coherent imaging via physics-aware deep learning. Appl. Phys. Rev. 8, 021407 (2021).10.1063/5.0031486 ↗
82. Pun, G. P., Batra, R., Ramprasad, R. & Mishin, Y. Physically informed artificial neural networks for atomistic modeling of materials. Nat. Commun. 10, 1–10 (2019).10.1038/s41467-019-10343-5 ↗
83. Onken, D. et al. A neural network approach for high-dimensional optimal control. arXiv. https://arxiv.org/abs/2104.03270 (2021).
84. Zunger, A. Inverse design in search of materials with target functionalities. Nat. Rev. Chem. 2, 1–16 (2018).10.1038/s41570-018-0121 ↗
85. Chen, L., Zhang, W., Nie, Z., Li, S. & Pan, F. Generative models for inverse design of inorganic solid materials. J. Mater. Inform. 1, 4 (2021).
86. Cranmer, M. et al. Discovering symbolic models from deep learning with inductive biases. arXiv. https://arxiv.org/abs/2006.11287 (2020).
87. Rupp, M., Tkatchenko, A., Müller, K.-R. & Von Lilienfeld, O. A. Fast and accurate modeling of molecular atomization energies with machine learning. Phys. Rev. Lett. 108, 058301 (2012).10.1103/PhysRevLett.108.058301 ↗
88. Bartók, A. P., Kondor, R. & Csányi, G. On representing chemical environments. Phys. Rev. B 87, 184115 (2013).10.1103/PhysRevB.87.184115 ↗
89. Faber, F. A. et al. Prediction errors of molecular machine learning models lower than hybrid dft error. J. Chem. Theory Comput. 13, 5255–5264 (2017).10.1021/acs.jctc.7b00577 ↗
90. Choudhary, K., DeCost, B. & Tavazza, F. Machine learning with force-field-inspired descriptors for materials: Fast screening and mapping energy landscape. Phys. Rev. Mater. 2, 083801 (2018).10.1103/PhysRevMaterials.2.083801 ↗
91. Choudhary, K., Garrity, K. F., Ghimire, N. J., Anand, N. & Tavazza, F. High-throughput search for magnetic topological materials using spin-orbit spillage, machine learning, and experiments. Phys. Rev. B 103, 155131 (2021).10.1103/PhysRevB.103.155131 ↗
92. Choudhary, K., Garrity, K. F. & Tavazza, F. Data-driven discovery of 3d and 2d thermoelectric materials. J. Phys. Condens. Matter 32, 475501 (2020).10.1088/1361-648X/aba06b ↗
93. Ward, L. et al. Including crystal structure attributes in machine learning models of formation energies via voronoi tessellations. Phys. Rev. B 96, 024104 (2017).10.1103/PhysRevB.96.024104 ↗
94. Isayev, O. et al. Universal fragment descriptors for predicting properties of inorganic crystals. Nat. Commun. 8, 1–12 (2017).10.1038/ncomms15679 ↗
95. Liu, C.-H., Tao, Y., Hsu, D., Du, Q. & Billinge, S. J. Using a machine learning approach to determine the space group of a structure from the atomic pair distribution function. Acta Crystallogr. Sec. A 75, 633–643 (2019).10.1107/S2053273319005606 ↗
96. Smith, J. S., Isayev, O. & Roitberg, A. E. Ani-1: an extensible neural network potential with dft accuracy at force field computational cost. Chem. Sci. 8, 3192–3203 (2017).10.1039/C6SC05720A ↗
97. Behler, J. Atom-centered symmetry functions for constructing high-dimensional neural network potentials. J. Chem. Phys. 134, 074106 (2011).10.1063/1.3553717 ↗
98. Behler, J. & Parrinello, M. Generalized neural-network representation of high-dimensional potential-energy surfaces. Phys. Rev. Lett. 98, 146401 (2007).10.1103/PhysRevLett.98.146401 ↗
99. Ko, T. W., Finkler, J. A., Goedecker, S. & Behler, J. A fourth-generation high-dimensional neural network potential with accurate electrostatics including non-local charge transfer. Nat. Commun. 12, 398 (2021).10.1038/s41467-020-20427-2 ↗
100. Weinreich, J., Romer, A., Paleico, M. L. & Behler, J. Properties of alpha-brass nanoparticles. 1. neural network potential energy surface. J. Phys. Chem C 124, 12682–12695 (2020).10.1021/acs.jpcc.0c00559 ↗