Transferable equivariant graph neural networks for the Hamiltonians of molecules and solids
Source abstract
Abstract
Abstract This work presents an E(3) equivariant graph neural network called HamGNN, which can fit the electronic Hamiltonian matrix of molecules and solids by a complete data-driven method. Unlike invariant models that achieve equivariance approximately through data augmentation, HamGNN employs E(3) equivariant convolutions to construct the Hamiltonian matrix, ensuring strict adherence to all equivariant constraints inherent in the physical system. In contrast to previous models with limited transferability, HamGNN demonstrates exceptional accuracy on various datasets, including QM9 molecular datasets, carbon allotropes, silicon allotropes, SiO2 isomers, and BixSey compounds. The trained HamGNN models exhibit accurate predictions of electronic structures for large crystals beyond the training set, including the Moiré twisted bilayer MoS2 and silicon supercells with dislocation defects, showcasing remarkable transferability and generalization capabilities. The HamGNN model, trained on small systems, can serve as an efficient alternative to density functional theory (DFT) for accurately computing the electronic structures of large systems.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Machine Learning in Materials Science · X-ray Diffraction in Crystallography · Inorganic Chemistry and Materials
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
References (81)
Show source-provided references
1. Choudhary, K. et al. High-throughput density functional perturbation theory and machine learning predictions of infrared, piezoelectric, and dielectric responses. npj Comput. Mater. 6, 64 (2020).10.1038/s41524-020-0337-2 ↗
2. Liu, Y. T., Zhou, Q. & Cui, G. L. Machine learning boosting the development of advanced lithium batteries. Small Methods 5, 2100442 (2021).10.1002/smtd.202100442 ↗
3. Zhang, N. et al. Machine learning in screening high performance electrocatalysts for CO2 reduction. Small Methods 5, 2100987 (2021).10.1002/smtd.202100987 ↗
4. Manzhos, S. & Carrington, T. Jr Neural network potential energy surfaces for small molecules and reactions. Chem. Rev. 121, 10187–10217 (2021).10.1021/acs.chemrev.0c00665 ↗
5. Choudhary, K. et al. Unified graph neural network force-field for the periodic table: solid state applications. Digit. Discov. 2, 346–355 (2023).10.1039/D2DD00096B ↗
6. Cheng, Z. et al. Building quantum mechanics quality force fields of proteins with the generalized energy-based fragmentation approach and machine learning. Phys. Chem. Chem. Phys. 24, 1326–1337 (2022).10.1039/D1CP03934B ↗
7. Unke, O. T. et al. SpookyNet: Learning force fields with electronic degrees of freedom and nonlocal effects. Nat. Commun. 12, 7273 (2021).
8. Cheng, G. J., Gong, X. G. & Yin, W. J. Crystal structure prediction by combining graph network and optimization algorithm. Nat. Commun. 13, 1492 (2022).
9. Ganea, O. et al. GeoMol: torsional geometric generation of molecular 3D conformer ensembles. Adv. Neural Inf. Process. Syst. 34, 13757–13769 (2021).
10. Xie, T., Fu, X., Ganea, O.-E., Barzilay, R. & Jaakkola, T. Crystal diffusion variational autoencoder for periodic material generation. In International Conference on Learning Representations (ICLR, 2019).
11. Kirkpatrick, J. et al. Pushing the frontiers of density functionals by solving the fractional electron problem. Science 374, 1385–1389 (2021).10.1126/science.abj6511 ↗
12. Nagai, R., Akashi, R. & Sugino, O. Machine-learning-based exchange correlation functional with physical asymptotic constraints. Phys. Rev. Res. 4, 013106 (2022).10.1103/PhysRevResearch.4.013106 ↗
13. Goringe, C. M., Bowler, D. R. & Hernandez, E. Tight-binding modelling of materials. Rep. Prog. Phys. 60, 1447–1512 (1997).10.1088/0034-4885/60/12/001 ↗
14. Papaconstantopoulos, D. A. & Mehl, M. J. The Slater-Koster tight-binding method: a computationally efficient and accurate approach. J. Phys. Condens. Matter 15, R413–R440 (2003).10.1088/0953-8984/15/10/201 ↗
15. Hegde, G. & Bowen, R. C. Machine-learned approximations to density functional theory Hamiltonians. Sci. Rep. 7, 42669 (2017).
16. Wang, Z. F. et al. Machine learning method for tight-binding Hamiltonian parameterization from ab-initio band structure. npj Comput. Mater. 7, 11 (2021).10.1038/s41524-020-00490-5 ↗
17. Ozaki, T. Variationally optimized atomic orbitals for large-scale electronic structures. Phys. Rev. B 67, 155108 (2003).10.1103/PhysRevB.67.155108 ↗
18. Ozaki, T. & Kino, H. Numerical atomic basis orbitals from H to Kr. Phys. Rev. B 69, 195113 (2004).10.1103/PhysRevB.69.195113 ↗
19. Artacho, E. et al. The SIESTA method; developments and applicability. J. Phys. 20, 064208 (2008).
20. Garcia, A. et al. Siesta: recent developments and applications. J. Chem. Phys. 152, 204108 (2020).10.1063/5.0005077 ↗
21. Morrison, M. A. & Parker, G. A. A guide to rotations in quantum-mechanics. Aust. J. Phys. 40, 465–497 (1987).10.1071/PH870465 ↗
22. Weinert, U. Spherical tensor representation. Arch. Ration. Mech. 74, 165–196 (1980).10.1007/BF00253328 ↗
23. Zhang, L. et al. Equivariant analytical mapping of first principles Hamiltonians to accurate and transferable materials models. npj Comput. Mater. 8, 158 (2022).10.1038/s41524-022-00843-2 ↗
24. Nigam, J., Willatt, M. J. & Ceriotti, M. Equivariant representations for molecular Hamiltonians and N-center atomic-scale properties. J. Chem. Phys. 156, 014115 (2022).10.1063/5.0072784 ↗
25. Unke, O. T. & Meuwly, M. A reactive, scalable, and transferable model for molecular energies from a neural network approach based on local information. J. Chem. Phys. 148, 241708 (2018).10.1063/1.5017898 ↗
26. Christensen, A. S., Bratholm, L. A., Faber, F. A. & Anatole von Lilienfeld, O. FCHL revisited: Faster and more accurate quantum machine learning. J. Chem. Phys. 152, 044107 (2020).10.1063/1.5126701 ↗
27. Faber, F. A., Christensen, A. S., Huang, B. & von Lilienfeld, O. A. Alchemical and structural distribution based representation for universal quantum machine learning. J. Chem. Phys. 148, 241717 (2018).10.1063/1.5020710 ↗
28. Bartok, A. P., Kondor, R. & Csanyi, G. On representing chemical environments. Phys. Rev. B 87, 184115 (2013).10.1103/PhysRevB.87.184115 ↗
29. Gilmer, J., Schoenholz, S. S., Riley, P. F., Vinyals, O. & Dahl, G. E. Neural message passing for quantum chemistry. Proc. 34th Int. Conf. Mach. Learn. 70, 1263–1272 (2017).
30. Xie, T. & Grossman, J. C. Crystal graph convolutional neural networks for an accurate and interpretable prediction of material properties. Phys. Rev. Lett. 120, 145301 (2018).10.1103/PhysRevLett.120.145301 ↗
31. Schutt, K. T., Sauceda, H. E., Kindermans, P. J., Tkatchenko, A. & Muller, K. R. SchNet - A deep learning architecture for molecules and materials. J. Chem. Phys. 148, 241722 (2018).10.1063/1.5019779 ↗
32. Choudhary, K. & DeCost, B. Atomistic line graph neural network for improved materials property predictions. npj Comput. Mater. 7, 185 (2021).10.1038/s41524-021-00650-1 ↗
33. Batzner, S. et al. E(3)-equivariant graph neural networks for data-efficient and accurate interatomic potentials. Nat. Commun. 13, 2453 (2022).10.1038/s41467-022-29939-5 ↗
34. Wang, Z. et al. Heterogeneous relational message passing networks for molecular dynamics simulations. npj Comput. Mater. 8, 53 (2022).10.1038/s41524-022-00739-1 ↗
35. Schutt, K. T., Gastegger, M., Tkatchenko, A., Muller, K. R. & Maurer, R. J. Unifying machine learning and quantum chemistry with a deep neural network for molecular wavefunctions. Nat. Commun. 10, 5024 (2019).10.1038/s41467-019-12875-2 ↗
36. Unke, O. et al. SE(3)-equivariant prediction of molecular wavefunctions and electronic densities. Adv. Neural Inf. Process. Syst. 34, 14434–14447 (2021).
37. Li, H. C., Collins, C., Tanha, M., Gordon, G. J. & Yaron, D. J. A density functional tight binding layer for deep learning of chemical Hamiltonians. J. Chem. Theory Comput. 14, 5764–5776 (2018).10.1021/acs.jctc.8b00873 ↗
38. Grisafi, A., Wilkins, D. M., Csanyi, G. & Ceriotti, M. Symmetry-adapted machine learning for tensorial properties of atomistic systems. Phys. Rev. Lett. 120, 036002 (2018).10.1103/PhysRevLett.120.036002 ↗
39. Jones, M. D. & Albers, R. C. Spin-orbit coupling in an f-electron tight-binding model: Electronic properties of Th, U, and Pu. Phys. Rev. B 79, 045107 (2009).10.1103/PhysRevB.79.045107 ↗
40. Hemstreet, L. A., Fong, C. Y. & Nelson, J. S. First-principles calculations of spin-orbit splittings in solids using nonlocal separable pseudopotentials. Phys. Rev. B: Condens. Matter 47, 4238–4243 (1993).10.1103/PhysRevB.47.4238 ↗
41. Klicpera, J., Groß, J. & Günnemann, S. Directional message passing for molecular graphs. In International Conference on Learning Representations (ICLR, 2019).
42. Unke, O. T. & Meuwly, M. PhysNet: A neural network for predicting energies, forces, dipole moments, and partial charges. J. Chem. Theory Comput. 15, 3678–3693 (2019).10.1021/acs.jctc.9b00181 ↗
43. Thomas, N. et al. Tensor field networks: Rotation- and translation-equivariant neural networks for 3D point clouds. Preprint at https://doi.org/10.48550/arXiv.1802.08219 (2018).10.48550/arXiv.1802.08219 ↗
44. Weiler, M., Geiger, M., Welling, M., Boomsma, W. & Cohen, T. S. 3d steerable cnns: Learning rotationally equivariant features in volumetric data. Adv. Neural Inf. Process. Syst. 31, 10381–10392 (2018).
45. Li, H. et al. Deep-learning density functional theory Hamiltonian for efficient ab initio electronic-structure calculation. Nat. Comput. Sci. 2, 367–377 (2022).10.1038/s43588-022-00265-6 ↗
46. Ruddigkeit, L., van Deursen, R., Blum, L. C. & Reymond, J. L. Enumeration of 166 billion organic small molecules in the chemical universe database GDB-17. J. Chem. Inf. Model. 52, 2864–2875 (2012).10.1021/ci300415d ↗
47. Ramakrishnan, R., Dral, P. O., Rupp, M. & von Lilienfeld, O. A. Quantum chemistry structures and properties of 134 kilo molecules. Sci. Data 1, 140022 (2014).10.1038/sdata.2014.22 ↗
48. Hoffmann, R., Kabanov, A. A., Golov, A. A. & Proserpio, D. M. Homo citans and carbon allotropes: for an ethics of citation. Angew. Chem. Int. Ed. 55, 10962–10976 (2016).10.1002/anie.201600655 ↗
49. Jain, A. et al. Commentary: The Materials Project: A materials genome approach to accelerating materials innovation. APL Mater. 1, 011002 (2013).10.1063/1.4812323 ↗
50. Fujii, Y., Maruyama, M., Cuong, N. T. & Okada, S. Pentadiamond: a hard carbon allotrope of a pentagonal network of sp2 and sp3 C atoms. Phys. Rev. Lett. 125, 016001 (2020).10.1103/PhysRevLett.125.016001 ↗
51. Puska, M. J. Point defects in silicon, first-principles calculations. Comput. Mater. Sci. 17, 365–373 (2000).10.1016/S0927-0256(00)00054-9 ↗
52. McCluskey, M. D. & Janotti, A. Defects in semiconductors. J. Appl. Phys. 127, 190401 (2020).10.1063/5.0012677 ↗
53. Blumenau, A. T. et al. Dislocations in diamond: Dissociation into partials and their glide motion. Phys. Rev. B 68, 014115 (2003).10.1103/PhysRevB.68.014115 ↗
54. Li, Z. & Picu, R. C. Shuffle-glide dislocation transformation in Si. J. Appl. Phys. 113, 083519 (2013).10.1063/1.4793635 ↗
55. Das, S. & Gavini, V. Electronic structure study of screw dislocation core energetics in Aluminum and core energetics informed forces in a dislocation aggregate. J. Mech. Phys. Solids 104, 115–143 (2017).10.1016/j.jmps.2017.03.010 ↗
56. Wang, J. W., Xu, W. W., Wang, R., Laref, A. & Wu, X. Z. Structural and electronic properties of 90 degrees dislocations in silicon nanorods: A first-principles calculation. Comput. Mater. Sci. 149, 243–249 (2018).10.1016/j.commatsci.2018.03.036 ↗
57. Wu, F., Lovorn, T., Tutuc, E., Martin, I. & MacDonald, A. H. Topological insulators in twisted transition metal dichalcogenide homobilayers. Phys. Rev. Lett. 122, 086402 (2019).10.1103/PhysRevLett.122.086402 ↗
58. Li, X. & Zhu, H. Two-dimensional MoS2: Properties, preparation, and applications. J. Mater. 1, 33–44 (2015).
59. Conley, H. J. et al. Bandgap engineering of strained monolayer and bilayer MoS2. Nano. Lett. 13, 3626–3630 (2013).10.1021/nl4014748 ↗
60. Xu, K. et al. The role of Anderson’s rule in determining electronic, optical and transport properties of transition metal dichalcogenide heterostructures. Phys. Chem. Chem. Phys. 20, 30351–30364 (2018).10.1039/C8CP05522J ↗
61. Naik, M. H. & Jain, M. Ultraflatbands and shear solitons in Moiré patterns of twisted bilayer transition metal dichalcogenides. Phys. Rev. Lett. 121, 266401 (2018).10.1103/PhysRevLett.121.266401 ↗
62. Devakul, T., Crepel, V., Zhang, Y. & Fu, L. Magic in twisted transition metal dichalcogenide bilayers. Nat. Commun. 12, 6730 (2021).10.1038/s41467-021-27042-9 ↗
63. Liu, Y. et al. Moiré superlattices and related Moiré excitons in twisted van der Waals heterostructures. Chem. Soc. Rev. 50, 6401–6422 (2021).10.1039/D0CS01002B ↗
64. Majhi, K. et al. Emergence of a weak topological insulator from the BixSey family. Appl. Phys. Lett. 110, 162102 (2017).10.1063/1.4981875 ↗
65. Goncalves, P. H. R. et al. Formation of BixSey phases upon annealing of the topological insulator Bi2Se3: Stabilization of in-depth bismuth bilayers. J. Phys. Chem. Lett. 9, 954–960 (2018).10.1021/acs.jpclett.7b03172 ↗
66. Yazyev, O. V., Moore, J. E. & Louie, S. G. Spin polarization and transport of surface states in the topological insulators Bi2Se3 and Bi2Te3 from first principles. Phys. Rev. Lett. 105, 266806 (2010).10.1103/PhysRevLett.105.266806 ↗
67. Zhang, Y. et al. Crossover of the three-dimensional topological insulator Bi2Se3 to the two-dimensional limit. Nat. Phys. 6, 584–588 (2010).10.1038/nphys1689 ↗
68. Crowley, J. M., Tahir-Kheli, J. & Goddard, W. A. 3rd accurate ab initio quantum mechanics simulations of Bi2Se3 and Bi2Te3 topological insulator surfaces. J. Phys. Chem. Lett. 6, 3792–3796 (2015).10.1021/acs.jpclett.5b01586 ↗
69. Mazumder, K. & Shirage, P. M. A brief review of Bi2Se3 based topological insulator: From fundamentals to applications. J. Alloy. Compd. 888, 161492 (2021).10.1016/j.jallcom.2021.161492 ↗
70. Shirali, K., Shelton, W. A. & Vekhter, I. Importance of van der Waals interactions for ab initio studies of topological insulators. J. Phys. Condens. Matter 33, 035702 (2020).10.1088/1361-648X/abbdbc ↗
71. Perdew, J. P., Burke, K. & Ernzerhof, M. Generalized gradient approximation made simple. Phys. Rev. Lett. 77, 3865–3868 (1996).10.1103/PhysRevLett.77.3865 ↗
72. Hirel, P. Atomsk: A tool for manipulating and converting atomic data files. Comput. Phys. Comm. 197, 212–219 (2015).10.1016/j.cpc.2015.07.012 ↗
73. Fan, Z. et al. GPUMD: A package for constructing accurate machine-learned potentials and performing highly efficient atomistic simulations. J. Chem. Phys. 157, 114801 (2022).10.1063/5.0106617 ↗
74. Kresse, G. & Furthmuller, J. Efficiency of ab-initio total energy calculations for metals and semiconductors using a plane-wave basis set. Comp. Mater. Sci. 6, 15–50 (1996).10.1016/0927-0256(96)00008-0 ↗
75. Kresse, G. & Joubert, D. From ultrasoft pseudopotentials to the projector augmented-wave method. Phys. Rev. B 59, 1758–1775 (1999).10.1103/PhysRevB.59.1758 ↗
76. Paszke, A. et al. Pytorch: An imperative style, high-performance deep learning library. Adv. Neural Inf. Process. Syst. 32, 8026–8037 (2019).
77. Fey, M. & Lenssen, J. E. Fast graph representation learning with pytorch geometric. In Workshop of International Conference on Learning Representations (ICLR, 2019).
78. Ong, S. P. et al. Python Materials Genomics (pymatgen): A robust, open-source python library for materials analysis. Comp. Mater. Sci. 68, 314–319 (2013).10.1016/j.commatsci.2012.10.028 ↗
79. Geiger M. et al. e3nn/e3nn: 2022-04-13 (0.5.0). Zenodo https://doi.org/10.5281/zenodo.6459381 (2022).10.5281/zenodo.6459381 ↗
80. Loshchilov, I. & Hutter, F. Decoupled weight decay regularization. In International Conference on Learning Representations (2021).
81. Reddi, S. J., Kale, S. & Kumar, S. On the convergence of adam and beyond. Preprint at https://doi.org/10.48550/arXiv.1904.09237 (2019).10.48550/arXiv.1904.09237 ↗