Representations of molecules and materials for interpolation of quantum-mechanical simulations via machine learning
Source abstract
Abstract
Abstract Computational study of molecules and materials from first principles is a cornerstone of physics, chemistry, and materials science, but limited by the cost of accurate and precise simulations. In settings involving many simulations, machine learning can reduce these costs, often by orders of magnitude, by interpolating between reference simulations. This requires representations that describe any molecule or material and support interpolation. We comprehensively review and discuss current representations and relations between them. For selected state-of-the-art representations, we compare energy predictions for organic molecules, binary alloys, and Al–Ga–In sesquioxides in numerical experiments controlled for data distribution, regression method, and hyper-parameter optimization.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Machine Learning in Materials Science · Computational Drug Discovery Methods · X-ray Diffraction in Crystallography
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
References (100+)
Show source-provided references
1. Blum, L. C. & Reymond, J.-L. 970 million druglike small molecules for virtual screening in the chemical universe database GDB-13. J. Am. Chem. Soc. 131, 8732 (2009).10.1021/ja902302h ↗
2. Ghahramani, Z. Probabilistic machine learning and artificial intelligence. Nature 521, 452 (2015).10.1038/nature14541 ↗
3. Jordan, M. I. & Mitchell, T. M. Machine learning: trends, perspectives, and prospects. Science 349, 255 (2015).10.1126/science.aaa8415 ↗
4. Rupp, M., Tkatchenko, A., Müller, K.-R. & von Lilienfeld, O. A. Fast and accurate modeling of molecular atomization energies with machine learning. Phys. Rev. Lett. 108, 058301 (2012).10.1103/PhysRevLett.108.058301 ↗
5. Behler, J. First principles neural network potentials for reactive simulations of large molecular and condensed systems. Angew. Chem. Int. Ed. 56, 12828 (2017).10.1002/anie.201703114 ↗
6. Ceriotti, M., Willatt, M. J. & Csányi, G. in Handbook of Materials Modeling. Methods: Theory and Modeling (eds. Andreoni, W. & Yip, S.) (Springer, 2018).
7. Huang, B. & von Lilienfeld, O. A. Ab initio machine learning in chemical compound space. Chem. Rev. 121, 10001 (2021).10.1021/acs.chemrev.0c01303 ↗
8. Behler, J. & Parrinello, M. Generalized neural-network representation of high-dimensional potential-energy surfaces. Phys. Rev. Lett. 98, 146401 (2007).10.1103/PhysRevLett.98.146401 ↗
9. Bartók, A. P., Payne, M. C., Kondor, R. & Csányi, G. Gaussian approximation potentials: the accuracy of quantum mechanics, without the electrons. Phys. Rev. Lett. 104, 136403 (2010).10.1103/PhysRevLett.104.136403 ↗
10. Caro, M. A., Deringer, V. L., Koskinen, J., Laurila, T. & Csányi, G. Growth mechanism and origin of high sp3 content in tetrahedral amorphous carbon. Phys. Rev. Lett. 120, 166101 (2018).10.1103/PhysRevLett.120.166101 ↗
11. Jinnouchi, R., Lahnsteiner, J., Karsai, F., Kresse, G. & Bokdam, M. Phase transitions of hybrid perovskites simulated by machine-learning force fields trained on the fly with Bayesian inference. Phys. Rev. Lett. 122, 225701 (2019).10.1103/PhysRevLett.122.225701 ↗
12. Kiyohara, S., Oda, H., Tsuda, K. & Mizoguchi, T. Acceleration of stable interface structure searching using a Kriging approach. Jpn. J. Appl. Phys. 55, 045502 (2016).10.7567/JJAP.55.045502 ↗
13. Bartók, A. P., Kermode, J., Bernstein, N. & Csányi, G. Machine learning a general-purpose interatomic potential for silicon. Phys. Rev. X 8, 041048 (2018).
14. Sendek, A. D. et al. Machine learning-assisted discovery of solid Li-ion conducting materials. Chem. Mater. 31, 342 (2018).10.1021/acs.chemmater.8b03272 ↗
15. Jinnouchi, R., Karsai, F. & Kresse, G. On-the-fly machine learning force field generation: application to melting points. Phys. Rev. B 100, 014105 (2019).10.1103/PhysRevB.100.014105 ↗
16. Schölkopf, B. & Smola, A. Learning with Kernels (MIT Press, 2002). https://mitpress.mit.edu/books/learning-kernels10.7551/mitpress/4175.001.0001 ↗
17. Hofmann, T., Schölkopf, B. & Smola, A. Kernel methods in machine learning. Ann. Stat. 36, 1171 (2008).10.1214/009053607000000677 ↗
18. Todeschini, R. & Consonni, V. Handbook of Molecular Descriptors 2nd edn (Wiley, 2009).10.1007/978-1-4020-9783-6_3 ↗
19. Gilmer, J., Schoenholz, S. S., Riley, P. F., Vinyals, O. & Dahl, G. E. Neural message passing for quantum chemistry. In: Proc. 34th International Conference on Machine Learning (ICML) 1263 (2017).
20. Schütt, K. T., Arbabzadah, F., Chmiela, S., Müller, K.-R. & Tkatchenko, A. Quantum-chemical insights from deep tensor neural networks. Nat. Commun. 8, 13890 (2017).10.1038/ncomms13890 ↗
21. Schütt, K. T. et al. SchNet: a continuous-filter convolutional neural network for modeling quantum interactions. In Advances in Neural Information Processing Systems 30 (NeurIPS) (2017).
22. Kondor, R. n-body networks: a covariant hierarchical neural network architecture for learning atomic potentials. Preprint at https://arxiv.org/abs/1803.01588 (2018).
23. Schütt, K. T., Sauceda, H. E., Kindermans, P.-J., Tkatchenko, A. & Müller, K.-R. : SchNet—a deep learning architecture for molecules and materials. J. Chem. Phys. 148, 241722 (2018).10.1063/1.5019779 ↗
24. Zhang, L. et al. End-to-end symmetry preserving inter-atomic potential energy model for finite and extended systems. In Advances in Neural Information Processing Systems 31 (NeurIPS), 4436 (2018).
25. Thomas, N. et al. Tensor field networks: rotation- and translation-equivariant neural networks for 3D point clouds. In NeurIPS Workshop on Machine Learning for Molecules and Materials (2018).
26. Kondor, R., Li, Z., Trivedi, S. Clebsch-Gordan nets: a fully Fourier space spherical convolutional neural network. In Advances in Neural Information Processing Systems 31 (NeurIPS) 10117 (2018).
27. Weiler, M., Geiger, M., Welling, M., Boomsma, W. & Cohen, T. S. 3D steerable CNNs: learning rotationally equivariant features in volumetric data. In Advances in Neural Information Processing Systems 31 (NeurIPS), 10381 (2018).
28. Unke, O. T. & Meuwly, M. PhysNet: a neural network for predicting energies, forces, dipole moments, and partial charges. J. Chem. Theor. Comput. 15, 3678 (2019).10.1021/acs.jctc.9b00181 ↗
29. Anderson, B., Hy, T.-S., Kondor, R.: Cormorant: covariant molecular neural networks. In Advances in Neural Information Processing Systems 32 (NeurIPS) 14537 (2019).
30. Zhang, Y., Hu, C. & Jiang, B. Embedded atom neural network potentials: efficient and accurate machine learning with a physically inspired representation. J. Phys. Chem. Lett. 10, 4962 (2019).10.1021/acs.jpclett.9b02037 ↗
31. Mailoa, J. P. et al. A fast neural network approach for direct covariant forces prediction in complex multi-element extended systems. Nat. Mach. Intell. 1, 471 (2019).10.1038/s42256-019-0098-0 ↗
32. Klicpera, J., Groß, J. & Günnemann, S. Directional message passing for molecular graphs. In Proc. 8th International Conference on Learning Representations (ICLR) (2020).
33. Miller, B. K., Geiger, M., Smidt, T. E. & Noé, F. Relevance of rotationally equivariant convolutions for predicting molecular properties. In NeurIPS Workshop on Machine Learning for Molecules (2020).
34. Fuchs, F. B., Worrall, D. E., Fischer, V. & Welling, M. SE(3)-transformers: 3D roto-translation equivariant attention networks. In Advances in Neural Information Processing Systems 33 (NeurIPS) (2020).
35. Qiao, Z., Welborn, M., Anandkumar, A., Manby, F. R. & Miller III, T. F. OrbNet: deep learning for quantum chemistry using symmetry-adapted atomic-orbital features. J. Chem. Phys. 153, 124111 (2020).10.1063/5.0021955 ↗
36. Chen, Y., Zhang, L., Wang, H. & E, W. Ground state energy functional with Hartree-Fock efficiency and chemical accuracy. J. Phys. Chem. A 124, 7155 (2020).10.1021/acs.jpca.0c03886 ↗
37. Hermann, J., Schätzle, Z. & Noé, F. Deep-neural-network solution of the electronic Schrödinger equation. Nat. Chem. 12, 891 (2020).10.1038/s41557-020-0544-y ↗
38. Ho, T.-S. & Rabitz, H. A general method for constructing multidimensional molecular potential energy surfaces from ab initio calculations. J. Chem. Phys. 104, 2584 (1996).10.1063/1.470984 ↗
39. Unke, O. T. & Meuwly, M. A toolkit for the construction of reproducing kernel-based representations of data: application to multi-dimensional potential energy surfaces. J. Chem. Inf. Model. 57, 1923 (2017).10.1021/acs.jcim.7b00090 ↗
40. Glielmo, A., Zeni, C. & Vita, A. D. Efficient non-parametric n-body force fields from machine learning. Phys. Rev. B 97, 184307 (2018).10.1103/PhysRevB.97.184307 ↗
41. Koner, D. & Meuwly, M. Permutationally invariant, reproducing kernel-based potential energy surfaces for polyatomic molecules: from formaldehyde to acetone. J. Chem. Theor. Comput. 16, 5474 (2020).10.1021/acs.jctc.0c00535 ↗
42. Glielmo, A., Zeni, C., Fekete, Á., De Vita, A.: Building nonparametric n-body force fields using Gaussian process regression. In Machine Learning Meets Quantum Physics, 67 (eds. Schütt, K. T. et al.) (Springer, 2020).10.1007/978-3-030-40245-7_5 ↗
43. Braun, M. L., Buhmann, J. M. & Müller, K.-R. On relevant dimensions in kernel feature spaces. J. Mach. Learn. Res. 9(Aug), 1875 (2008).
44. Bartók, A. P., Kondor, R. & Csányi, G. On representing chemical environments. Phys. Rev. B 87, 184115 (2013).10.1103/PhysRevB.87.184115 ↗
45. Rupp, M., Ramakrishnan, R. & von Lilienfeld, O. A. Machine learning for quantum mechanical properties of atoms in molecules. J. Phys. Chem. Lett. 6, 3309 (2015).10.1021/acs.jpclett.5b01456 ↗
46. Bowman, J. M., Bittman, J. S. & Harding, L. B. Ab initio calculations of electronic and vibrational energies of HCO and HOC. J. Chem. Phys. 85, 911 (1986).10.1063/1.451246 ↗
47. Darsey, J. A., Noid, D. W. & Upadhyaya, B. R. Application of neural network computing to the solution for the ground-state eigenenergy of two-dimensional harmonic oscillators. Chem. Phys. Lett. 177, 189 (1991).10.1016/0009-2614(91)90066-I ↗
48. Heo, H., Ho, T.-S., Lehmann, K. K. & Rabitz, H. Regularized inversion of diatomic vibration-rotation spectral data: a functional sensitivity analysis approach. J. Chem. Phys. 97, 852 (1992).10.1063/1.463188 ↗
49. Hollebeek, T., Ho, T.-S. & Rabitz, H. Constructing multidimensional molecular potential energy surfaces from ab initio data. Annu. Rev. Phys. Chem. 50, 537 (1999).10.1146/annurev.physchem.50.1.537 ↗
50. Li, G. et al. Random sampling-high dimensional model representation (RS-HDMR) and orthogonality of its different order component functions. J. Phys. Chem. A 110, 2474 (2006).10.1021/jp054148m ↗
51. Glielmo, A., Sollich, P. & De Vita, A. Accurate interatomic force fields via machine learning with covariant kernels. Phys. Rev. B 95, 214302 (2017).10.1103/PhysRevB.95.214302 ↗
52. Grisafi, A., Wilkins, D. M., Csányi, G. & Ceriotti, M. Symmetry-adapted machine-learning for tensorial properties of atomistic systems. Phys. Rev. Lett. 120, 036002 (2018).10.1103/PhysRevLett.120.036002 ↗
53. Drautz, R. Atomic cluster expansion of scalar, vectorial, and tensorial properties including magnetism and charge transfer. Phys. Rev. B 102, 024104 (2020).10.1103/PhysRevB.102.024104 ↗
54. Hy, T. S., Trivedi, S., Pan, H., Anderson, B. M. & Kondor, R. Covariant compositional networks for learning graphs. In Proc. International Workshop on Mining and Learning with Graphs (MLG) (2019).
55. Moussa, J. E. Comment on “Fast and accurate modeling of molecular atomization energies with machine learning”. Phys. Rev. Lett. 109, 059801 (2012).10.1103/PhysRevLett.109.059801 ↗
56. von Lilienfeld, O. A., Ramakrishnan, R., Rupp, M. & Knoll, A. Fourier series of atomic radial distribution functions: a molecular fingerprint for machine learning models of quantum chemical properties. Int. J. Quant. Chem. 115, 1084 (2015).10.1002/qua.24912 ↗
57. Pozdnyakov, S. N. et al. Incompleteness of atomic structure representations. Phys. Rev. Lett. 125, 166001 (2020).10.1103/PhysRevLett.125.166001 ↗
58. Kocer, E., Mason, J. K. & Erturk, H. Continuous and optimally complete description of chemical environments using spherical Bessel descriptors. AIP Adv. 10, 015021 (2020).10.1063/1.5111045 ↗
59. Le, H. M., Huynh, S. & Raff, L. M. Molecular dissociation of hydrogen peroxide (HOOH) on a neural network ab initio potential surface with a new configuration sampling method involving gradient fitting. J. Chem. Phys. 131, 014107 (2009).10.1063/1.3159748 ↗
60. Bartók, A. P. & Csányi, G. Gaussian approximation potentials: a brief tutorial introduction. Int. J. Quant. Chem. 116, 1051 (2015).10.1002/qua.24927 ↗
61. Chmiela, S., Sauceda, H. E., Müller, K.-R. & Tkatchenko, A. Towards exact molecular dynamics simulations with machine-learned force fields. Nat. Commun. 9, 3887 (2018).10.1038/s41467-018-06169-2 ↗
62. Sadeghi, A. et al. Metrics for measuring distances in configuration spaces. J. Chem. Phys. 139, 184118 (2013).10.1063/1.4828704 ↗
63. Welborn, M., Cheng, L. & Miller III, T. F. Transferability in machine learning for electronic structure via the molecular orbital basis. J. Chem. Theor. Comput. 14, 4772 (2018).10.1021/acs.jctc.8b00636 ↗
64. Ramakrishnan, R., Dral, P. O., Rupp, M. & von Lilienfeld, O. A. Big data meets quantum chemistry approximations: the Δ-machine learning approach. J. Chem. Theor. Comput. 11, 2087 (2015).10.1021/acs.jctc.5b00099 ↗
65. Schmitz, G., Godtliebsen, I. H. & Christiansen, O. Machine learning for potential energy surfaces: an extensive database and assessment of methods. J. Chem. Phys. 150, 244113 (2019).10.1063/1.5100141 ↗
66. Behler, J. Atom-centered symmetry functions for constructing high-dimensional neural network potentials. J. Chem. Phys. 134, 074106 (2011).10.1063/1.3553717 ↗
67. Collins, C. R., Gordon, G. J., von Lilienfeld, O. A. & Yaron, D. J. Constant size descriptors for accurate machine learning models of molecular properties. J. Chem. Phys. 148, 241718 (2018).10.1063/1.5020441 ↗
68. Goscinski, A., Fraux, G., Imbalzano, G. & Ceriotti, M. The role of feature space in atomistic learning. Mach. Learn. Sci. Tech. 2, 025028 (2021).10.1088/2632-2153/abdaf7 ↗
69. Jinnouchi, R., Karsai, F., Verdi, C., Asahi, R. & Kresse, G. Descriptors representing two- and three-body atomic distributions and their effects on the accuracy of machine-learned inter-atomic potentials. J. Chem. Phys. 152, 234102 (2020).10.1063/5.0009491 ↗
70. Willatt, M. J., Musil, F. & Ceriotti, M. Atom-density representations for machine learning. J. Chem. Phys. 150, 154110 (2019).10.1063/1.5090481 ↗
71. Nigam, J., Pozdnyakov, S. & Ceriotti, M. Recursive evaluation and iterative contraction of n-body equivariant features. J. Chem. Phys. 153, 121101 (2020).10.1063/5.0021116 ↗
72. Musil, F. et al. Physics-inspired structural representations for molecules and materials. Chem. Rev. 121, 9759 (2021).10.1021/acs.chemrev.1c00021 ↗
73. Huo, H. & Rupp, M. Unified representation of molecules and crystals for machine learning. Preprint athttps://arxiv.org/abs/1704.06439 (2017).
74. Shapeev, A. V. Moment tensor potentials: a class of systematically improvable interatomic potentials. Multiscale Model. Simul. 14, 1153 (2016).10.1137/15M1054183 ↗
75. Zaverkin, V. & Kästner, J. Gaussian moments as physically inspired molecular descriptors for accurate and scalable machine learning potentials. J. Chem. Theor. Comput. 16, 5410 (2020).10.1021/acs.jctc.0c00347 ↗
76. Schütt, K. T. et al. How to represent crystal structures for machine learning: towards fast prediction of electronic properties. Phys. Rev. B 89, 205118 (2014).10.1103/PhysRevB.89.205118 ↗
77. Smith, J. S., Isayev, O. & Roitberg, A. E. ANI-1: an extensible neural network potential with DFT accuracy at force field computational cost. Chem. Sci. 8, 3192 (2017).10.1039/C6SC05720A ↗
78. Gastegger, M., Schwiedrzik, L., Bittermann, M., Berzsenyi, F. & Marquetand, P. wACSF—weighted atom-centered symmetry functions as descriptors in machine learning potentials. J. Chem. Phys. 148, 241709 (2018).10.1063/1.5019667 ↗
79. Rostami, S., Amsler, M. & Ghasemi, S. A. Optimized symmetry functions for machine-learning interatomic potentials of multicomponent systems. J. Chem. Phys. 149, 124106 (2018).10.1063/1.5040005 ↗
80. Artrith, N., Urban, A. & Ceder, G. Constructing first-principles phase diagrams of amorphous LixSi using machine-learning-assisted sampling with an evolutionary algorithm. J. Chem. Phys. 148, 241711 (2018).10.1063/1.5017661 ↗
81. The DScribe code contains a local MBTR example of this. See https://github.com/SINGROUP/dscribe.
82. Wood, M. A. & Thompson, A. P. Extending the accuracy of the SNAP interatomic potential form. J. Chem. Phys. 148, 241721 (2018).10.1063/1.5017641 ↗
83. Caro, M. A. Optimizing many-body atomic descriptors for enhanced computational performance of machine learning based interatomic potentials. Phys. Rev. B 100, 024112 (2019).10.1103/PhysRevB.100.024112 ↗
84. Himanen, L. et al. DScribe: library of descriptors for machine learning in materials science. Comput. Phys. Commun. 247, 106949 (2020).10.1016/j.cpc.2019.106949 ↗
85. Kocer, E., Mason, J. K. & Erturk, H. A novel approach to describe chemical environments in high-dimensional neural network potentials. J. Chem. Phys. 150, 154102 (2019).10.1063/1.5086167 ↗
86. Rupp, M., Tkatchenko, A., Müller, K.-R. & von Lilienfeld, O. A. Reply to the comment by J.E. Moussa. Phys. Rev. Lett. 109, 059802 (2012).10.1103/PhysRevLett.109.059802 ↗
87. Barker, J., Bulin, J., Hamaekers, J. & Mathias, S. in Scientific Computing And Algorithms In Industrial Simulations (eds. Griebel, M. et al.) 25, Springer, 2017).10.1007/978-3-319-62458-7_2 ↗
88. Hansen, K. et al. Machine learning predictions of molecular properties: accurate many-body potentials and nonlocality in chemical space. J. Phys. Chem. Lett. 6, 2326 (2015).10.1021/acs.jpclett.5b00831 ↗
89. Huang, B. & von Lilienfeld, O. A. Communication: understanding molecular representations in machine learning: the role of uniqueness and target similarity. J. Chem. Phys. 145, 161102 (2016).10.1063/1.4964627 ↗
90. Pronobis, W., Tkatchenko, A. & Müller, K.-R. Many-body descriptors for predicting molecular properties with machine learning: analysis of pairwise and three-body interactions in molecules. J. Chem. Theor. Comput. 14, 2991 (2018).10.1021/acs.jctc.8b00110 ↗
91. Faber, F. A. et al. Prediction errors of molecular machine learning models lower than hybrid DFT error. J. Chem. Theor. Comput. 13, 5255 (2017).10.1021/acs.jctc.7b00577 ↗
92. Faber, F. A., Christensen, A. S., Huang, B. & von Lilienfeld, O. A. Alchemical and structural distribution based representation for universal quantum machine learning. J. Chem. Phys. 148, 241717 (2018).10.1063/1.5020710 ↗
93. Christensen, A. S., Bratholm, L. A., Faber, F. A. & von Lilienfeld, O. A. FCHL revisited: faster and more accurate quantum machine learning. J. Chem. Phys. 152, 044107 (2020).10.1063/1.5126701 ↗
94. Hirn, M., Poilvert, N. & Mallat, S. Quantum energy regression using scattering transforms. Preprint athttps://arxiv.org/abs/1502.02077 (2015).
95. Hirn, M., Mallat, S. & Poilvert, N. Wavelet scattering regression of quantum chemical energies. Multiscale Model. Simul. 15, 827 (2017).10.1137/16M1075454 ↗
96. Eickenberg, M., Exarchakis, G., Hirn, M. & Mallat, S. Solid harmonic wavelet scattering: predicting quantum molecular energy from invariant descriptors of 3D electronic densities. In Advances in Neural Information Processing Systems 30 (NeurIPS), 6522 (2017).
97. Brumwell, X., Sinz, P., Kim, K. J., Qi, Y. & Hirn, M. Steerable wavelet scattering for 3D atomic systems with application to Li-Si energy prediction. In NeurIPS Workshop on Machine Learning for Molecules and Materials (2018).
98. Eickenberg, M., Exarchakis, G., Hirn, M., Mallat, S. & Thiry, L. Solid harmonic wavelet scattering for predictions of molecule properties. J. Chem. Phys. 148, 241732 (2018).10.1063/1.5023798 ↗
99. Homer, E. R., Hensley, D. M., Rosenbrock, C. W., Nguyen, A. H. & Hart, G. L. W. Machine-learning informed representations for grain boundary structures. Front. Mater. 6, 168 (2019).10.3389/fmats.2019.00168 ↗
100. Sinz, P. et al. Wavelet scattering networks for atomistic systems with extrapolation of material properties. J. Chem. Phys. 153, 084109 (2020).10.1063/5.0016020 ↗