journal article·2024-07-26Open access

Higher-order equivariant neural networks for charge density prediction in materials

Teddy Koker, Keegan Quigley, Eric Taw, Kevin Tibbetts, Lin Li

npj Computational Materials· Volume 10· Issue 1

Abstract

Abstract The calculation of electron density distribution using density functional theory (DFT) in materials and molecules is central to the study of their quantum and macro-scale properties, yet accurate and efficient calculation remains a long-standing challenge. We introduce ChargE3Net, an E(3)-equivariant graph neural network for predicting electron density in atomic systems. ChargE3Net enables the learning of higher-order equivariant features to achieve high predictive accuracy and model expressivity. We show that ChargE3Net exceeds the performance of prior work on diverse sets of molecules and materials. When trained on the massive dataset of over 100K materials in the Materials Project database, our model is able to capture the complexity and variability in the data, leading to a significant 26.7% reduction in self-consistent iterations when used to initialize DFT calculations on unseen materials. Furthermore, we show that non-self-consistent DFT calculations using our predicted charge densities yield near-DFT performance on electronic and thermodynamic property prediction at a fraction of the computational cost. Further analysis attributes the greater predictive accuracy to improved modeling of systems with high angular variations. These results illuminate a pathway towards a machine learning-accelerated ab initio calculations for materials discovery.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: Machine Learning in Materials Science · X-ray Diffraction in Crystallography · Computational Drug Discovery Methods

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

References (77)

Show source-provided references
1. Hohenberg, P. & Kohn, W. Inhomogeneous electron gas. Phys. Rev. 136, B864–B871 (1964).10.1103/PhysRev.136.B864 ↗
2. Kohn, W. & Sham, L. J. Self-consistent equations including exchange and correlation effects. Phys. Rev. 140, A1133–A1138 (1965).10.1103/PhysRev.140.A1133 ↗
3. Prentice, J. C. A. et al. The ONETEP linear-scaling density functional theory program. J. Chem. Phys. 152, 174111 (2020).10.1063/5.0004445 ↗
4. Soler, J. M. et al. The SIESTA method for ab initio order-N materials simulation. J. Condens. Matter Phys. 14, 2745 (2002).10.1088/0953-8984/14/11/302 ↗
5. Mohr, S. et al. Accurate and efficient linear scaling DFT calculations with universal applicability. Phys. Chem. Chem. Phys. 17, 31360–31370 (2015).10.1039/C5CP00437C ↗
6. Riplinger, C. & Neese, F. An efficient and near linear scaling pair natural orbital based local coupled cluster method. J. Chem. Phys. 138, 034106 (2013).10.1063/1.4773581 ↗
7. Riplinger, C., Sandhoefer, B., Hansen, A. & Neese, F. Natural triple excitations in local coupled cluster calculations with pair natural orbitals. J. Chem. Phys. 139, 134101 (2013).10.1063/1.4821834 ↗
8. Witt, W. C., del Rio, B. G., Dieterich, J. M. & Carter, E. A. Orbital-free density functional theory for materials research. J. Mater. Res. 33, 777–795 (2018).10.1557/jmr.2017.462 ↗
9. Xie, T. & Grossman, J. C. Crystal graph convolutional neural networks for an accurate and interpretable prediction of material properties. Phys. Rev. Lett. 120, 145301 (2018).10.1103/PhysRevLett.120.145301 ↗
10. Chen, C., Ye, W., Zuo, Y., Zheng, C. & Ong, S. P. Graph networks as a universal machine learning framework for molecules and crystals. Chem. Mater. 31, 3564–3572 (2019).10.1021/acs.chemmater.9b01294 ↗
11. Louis, S.-Y. et al. Graph convolutional neural networks with global attention for improved materials property prediction. Phys. Chem. Chem. Phys. 22, 18141–18148 (2020).10.1039/D0CP01474E ↗
12. Dunn, A., Wang, Q., Ganose, A., Dopp, D. & Jain, A. Benchmarking materials property prediction methods: the Matbench test set and Automatminer reference algorithm. Npj Comput. Mater. 6, 1–10 (2020).10.1038/s41524-019-0267-z ↗
13. Choudhary, K. & DeCost, B. Atomistic line graph neural network for improved materials property predictions. Npj Comput. Mater. 7, 185 (2021).10.1038/s41524-021-00650-1 ↗
14. Koker, T., Quigley, K., Spaeth, W., Frey, N. C. & Li, L. Graph contrastive learning for materials. Preprint at https://arxiv.org/abs/2211.13408 (2022).
15. Drautz, R. Atomic cluster expansion for accurate and transferable interatomic potentials. Phys. Rev. B 99, 014104 (2019).10.1103/PhysRevB.99.014104 ↗
16. Batzner, S. et al. E (3)-Equivariant graph neural networks for data-efficient and accurate interatomic potentials. Nat. Commun. 13, 2453 (2022).10.1038/s41467-022-29939-5 ↗
17. Musaelian, A. et al. Learning local equivariant representations for large-scale atomistic dynamics. Nat. Commun. 14, 579 (2023).10.1038/s41467-023-36329-y ↗
18. Chmiela, S. et al. Machine learning of accurate energy-conserving molecular force fields. Sci. Adv. 3, e1603015 (2017).10.1126/sciadv.1603015 ↗
19. Du, X. et al. Machine-learning-accelerated simulations to enable automatic surface reconstruction. Nat. Comput. Sci. 3, 1034–1044 (2023).10.1038/s43588-023-00571-7 ↗
20. Chanussot, L. et al. Open Catalyst 2020 (OC20) dataset and community challenges. ACS Catal. 11, 6059–6072 (2021).10.1021/acscatal.0c04525 ↗
21. Tran, R. et al. The Open Catalyst 2022 (OC22) dataset and challenges for oxide electrocatalysts. ACS Catal. 13, 3066–3084 (2023).10.1021/acscatal.2c05426 ↗
22. Sriram, A.et al. The Open DAC 2023 dataset and challenges for sorbent discovery in direct air capture. arXiv https://doi.org/10.48550/arXiv.2311.00341 (2023).10.48550/arXiv.2311.00341 ↗
23. Pengmei, Z., Liu, J. & Shu, Y. Beyond MD17: the reactive xxMD dataset. Sci. Data 11, 222 (2024).10.1038/s41597-024-03019-3 ↗
24. Deng, B. et al. CHGNet as a pretrained universal neural network potential for charge-informed atomistic modelling. Nat. Mach. Intell. 5, 1031–1041 (2023).10.1038/s42256-023-00716-3 ↗
25. Batatia, I. et al. A foundation model for atomistic materials chemistry. Preprint at https://arxiv.org/abs/2401.00096 (2023).
26. Brockherde, F. et al. Bypassing the Kohn-Sham equations with machine learning. Nat. Commun. 8, 872 (2017).10.1038/s41467-017-00839-3 ↗
27. Grisafi, A. et al. Transferable machine-learning model of the electron density. ACS Cent. Sci. 5, 57–64 (2018).10.1021/acscentsci.8b00551 ↗
28. Fabrizio, A., Grisafi, A., Meyer, B., Ceriotti, M. & Corminboeuf, C. Electron density learning of non-covalent systems. Chem. Sci. 10, 9424–9432 (2019).10.1039/C9SC02696G ↗
29. Lewis, A. M., Grisafi, A., Ceriotti, M. & Rossi, M. Learning electron densities in the condensed phase. J. Chem. Theory Comput. 17, 7203–7214 (2021).10.1021/acs.jctc.1c00576 ↗
30. Qiao, Z. et al. Informing geometric deep learning with electronic interactions to accelerate quantum chemistry. Proc. Natl. Acad. Sci. USA 119, e2205221119 (2022).10.1073/pnas.2205221119 ↗
31. Rackers, J. A., Tecot, L., Geiger, M. & Smidt, T. E. A recipe for cracking the quantum scaling limit with machine learned electron densities. Mach. Learn.: Sci. Technol. 4, 015027 (2023).
32. del Rio, B. G., Phan, B. & Ramprasad, R. A deep learning framework to emulate density functional theory. Npj Comput. Mater. 9, 158 (2023).10.1038/s41524-023-01115-3 ↗
33. Li, H. et al. Deep-learning density functional theory Hamiltonian for efficient ab initio electronic-structure calculation. Nat. Comput. Sci. 2, 367–377 (2022).10.1038/s43588-022-00265-6 ↗
34. Unke, O. et al. SE (3)-equivariant prediction of molecular wavefunctions and electronic densities. Adv. Neural Inf. Process. Syst. 34, 14434–14447 (2021).
35. Nigam, J., Willatt, M. J. & Ceriotti, M. Equivariant representations for molecular Hamiltonians and N-center atomic-scale properties. J. Chem. Phys. 156, 14115 (2022).10.1063/5.0072784 ↗
36. Zhong, Y., Yang, J., Xiang, H. & Gong, X. Universal machine learning kohn-sham hamiltonian for materials. Chin. Phys. Lett. 41, 077103 (2024).10.1088/0256-307X/41/7/077103 ↗
37. Kresse, G. & Hafner, J. Ab initio molecular dynamics for liquid metals. Phys. Rev. B 47, 558–561 (1993).10.1103/PhysRevB.47.558 ↗
38. Kresse, G. & Furthmüller, J. Efficiency of ab-initio total energy calculations for metals and semiconductors using a plane-wave basis set. Comput. Mater. Sci. 6, 15–50 (1996).10.1016/0927-0256(96)00008-0 ↗
39. Kresse, G. & Furthmüller, J. Efficient iterative schemes for ab initio total-energy calculations using a plane-wave basis set. Phys. Rev. B 54, 11169–11186 (1996).10.1103/PhysRevB.54.11169 ↗
40. Giannozzi, P. et al. QUANTUM ESPRESSO: a modular and open-source software project for quantum simulations of materials. J. Condens. Matter Phys. 21, 395502 (2009).10.1088/0953-8984/21/39/395502 ↗
41. Gong, S. et al. Predicting charge density distribution of materials using a local-environment-based graph convolutional network. Phys. Rev. B 100, 184103 (2019).10.1103/PhysRevB.100.184103 ↗
42. Jørgensen, P. B. & Bhowmik, A. DeepDFT: Neural message passing network for accurate charge density prediction. Preprint at https://arxiv.org/abs/2011.03346 (2020).
43. Jørgensen, P. B. & Bhowmik, A. Equivariant graph neural networks for fast electron density estimation of molecules, liquids, and solids. Npj Comput. Mater. 8, 183 (2022).10.1038/s41524-022-00863-y ↗
44. Sunshine, E. M., Shuaibi, M., Ulissi, Z. W. & Kitchin, J. R. Chemical properties from graph neural network-predicted electron densities. J. Phys. Chem. C 127, 23459–23466 (2023).10.1021/acs.jpcc.3c06157 ↗
45. Focassio, B., Domina, M., Patil, U., Fazzio, A. & Sanvito, S. Linear jacobi-legendre expansion of the charge density for machine learning-accelerated electronic structure calculations. Npj Comput. Mater. 9, 87 (2023).10.1038/s41524-023-01053-0 ↗
46. Schütt, K. et al. Schnet: a continuous-filter convolutional neural network for modeling quantum interactions. Adv. Neural Inf. Process. Syst. 30, 991–1001 (2017).
47. Schütt, K., Unke, O. & Gastegger, M. Equivariant message passing for the prediction of tensorial properties and molecular spectra. In International Conference on Machine Learning, 9377–9388 (PMLR, 2021).
48. Pope, P. & Jacobs, D. Towards combinatorial generalization for catalysts: a kohn-sham charge-density approach. Adv. Neural Inf. Process. Syst. 36, 60585–60598 (2024).
49. Shen, J.-X. et al. A representation-independent electronic charge density database for crystalline materials. Sci. Data 9, 661 (2022).10.1038/s41597-022-01746-z ↗
50. Merchant, A. et al. Scaling deep learning for materials discovery. Nature 624, 80–85 (2023).10.1038/s41586-023-06735-9 ↗
51. Pozdnyakov, S. N. & Ceriotti, M. Incompleteness of graph neural networks for points clouds in three dimensions. Mach. Learn.: Sci. Technol. 3, 045020 (2022).
52. Thomas, N. et al. Tensor field networks: rotation-and translation-equivariant neural networks for 3d point clouds. Preprint at https://arxiv.org/abs/1802.08219 (2018).
53. Geiger, M. & Smidt, T. e3nn: Euclidean neural networks. Preprint at https://arxiv.org/abs/2207.09453 (2022).
54. Ruddigkeit, L., Van Deursen, R., Blum, L. C. & Reymond, J.-L. Enumeration of 166 billion organic small molecules in the chemical universe database GDB-17. J. Chem. Inf. Model 52, 2864–2875 (2012).10.1021/ci300415d ↗
55. Ramakrishnan, R., Dral, P. O., Rupp, M. & Von Lilienfeld, O. A. Quantum chemistry structures and properties of 134 kilo molecules. Sci. Data 1, 1–7 (2014).10.1038/sdata.2014.22 ↗
56. Jørgensen, P. B. & Bhowmik, A. QM9 Charge Densities And Energies Calculated With VASP. https://doi.org/10.11583/DTU.16794500.v1 (2022).10.11583/DTU.16794500.v1 ↗
57. Jørgensen, P. B. & Bhowmik, A. NMC li-ion Battery Cathode Energies And Charge Densities. https://doi.org/10.11583/DTU.16837721.v1 (2022).10.11583/DTU.16837721.v1 ↗
58. Jain, A.et al. Commentary: The Materials Project: a materials genome approach to accelerating materials innovation. APL Mater.1, 11002 (2013).10.1063/1.4812323 ↗
59. Sholl, D. S. & Steckel, J. A. Density Functional Theory (John Wiley & Sons, Ltd, 2009).10.1002/9780470447710 ↗
60. Lehtola, S. Assessment of initial guesses for self-consistent field calculations. superposition of atomic potentials: simple yet efficient. J. Chem. Theory Comput. 15, 1593–1604 (2019).10.1021/acs.jctc.8b01089 ↗
61. Wing, D. et al. Band gaps of crystalline solids from Wannier-localization–Based optimal tuning of a screened range-separated hybrid functional. Proc. Natl. Acad. Sci. USA 118, e2104556118 (2021).10.1073/pnas.2104556118 ↗
62. Passaro, S. & Zitnick, C. L. Reducing SO(3) convolutions to SO(2) for efficient equivariant gnns. In International Conference on Machine Learning, 27420–27438 (PMLR, 2023).
63. Horton, M. K., Montoya, J. H., Liu, M. & Persson, K. A. High-throughput prediction of the ground-state collinear magnetic order of inorganic materials using Density Functional Theory. Npj Comput. Mater. 5, 1–11 (2019).10.1038/s41524-019-0199-7 ↗
64. Lee, K., Youn, Y. & Han, S. Identification of ground-state spin ordering in antiferromagnetic transition metal oxides using the Ising model and a genetic algorithm. Sci. Technol. Adv. Mater. 18, 246–252 (2017).10.1080/14686996.2017.1300046 ↗
65. Hendrycks, D. & Gimpel, K. Gaussian error linear units (GELUs). Preprint at https://arxiv.org/abs/1606.08415 (2016).
66. Weiler, M., Geiger, M., Welling, M., Boomsma, W. & Cohen, T. S. 3d steerable cnns: learning rotationally equivariant features in volumetric data. Adv. Neural Inf. Process. Syst. 31, 10381–10392 (2018).
67. Larsen, A. H. et al. The atomic simulation Environment—a Python library for working with atoms. J. Condens. Matter Phys. 29, 273002 (2017).10.1088/1361-648X/aa680e ↗
68. Bentley, J. L. Multidimensional binary search trees used for associative searching. Commun. ACM 18, 509–517 (1975).10.1145/361002.361007 ↗
69. Virtanen, P. et al. SciPy 1.0: fundamental algorithms for scientific computing in python. Nat. Methods 17, 261–272 (2020).10.1038/s41592-019-0686-2 ↗
70. Kingma, D. P. & Ba, J. Adam: a method for stochastic optimization. Int. Conf. Learn. Represent. 3, 1–15 (2015).
71. Ong, S. P. et al. Python Materials Genomics (pymatgen): a robust, open-source python library for materials analysis. Comput. Mater. Sci. 68, 314–319 (2013).10.1016/j.commatsci.2012.10.028 ↗
72. Hafner, J. Ab-initio simulations of materials using VASP: density-functional theory and beyond. J. Comput. Chem. 29, 2044–2078 (2008).10.1002/jcc.21057 ↗
73. Kresse, G. & Joubert, D. From ultrasoft pseudopotentials to the projector augmented-wave method. Phys. Rev. B 59, 1758–1775 (1999).10.1103/PhysRevB.59.1758 ↗
74. Wang, L., Maxisch, T. & Ceder, G. Oxidation energies of transition metal oxides within the GGA+U framework. Phys. Rev. B 73, 195107 (2006).10.1103/PhysRevB.73.195107 ↗
75. Jain, A. et al. Formation enthalpies by mixing GGA and GGA+U calculations. Phys. Rev. B 84, 045115 (2011).10.1103/PhysRevB.84.045115 ↗
76. Togo, A., Chaput, L., Tadano, T. & Tanaka, I. Implementation strategies in phonopy and phono3py. J. Condens. Matter Phys. 35, 353001 (2023).10.1088/1361-648X/acd831 ↗
77. Togo, A. First-principles phonon calculations with phonopy and phono3py. J. Phys. Soc. Jpn 92, 012001 (2023).10.7566/JPSJ.92.012001 ↗