journal article·2025-03-02Open access

Rapid high-fidelity quantum simulations using multi-step nonlinear autoregression and graph embeddings

Akeel A. Shah, P. K. Leung, W. W. Xing

npj Computational Materials· Volume 11· Issue 1

Abstract

Abstract The design and high-throughput screening of materials using machine-learning assisted quantum-mechanical simulations typically requires the existence of a very large data set, often generated from simulations at a high level of theory or fidelity. A single simulation at high fidelity can take on the order of days for a complex molecule. Thus, although machine learning surrogate simulations seem promising at first glance, generation of the training data can defeat the original purpose. For this reason, the use of machine learning to screen or design materials remains elusive for many important applications. In this paper we introduce a new multi-fidelity approach based on a dual graph embedding to extract features that are placed inside a nonlinear multi-step autoregressive model. Experiments on five benchmark problems, with 14 different quantities and 27 different levels of theory, demonstrate the generalizability and high accuracy of the approach. It typically requires a few 10s to a few 1000’s of high-fidelity training points, which is several orders of magnitude lower than direct ML methods, and can be up to two orders of magnitude lower than other multi-fidelity methods. Furthermore, we develop a new benchmark data set for 860 benzoquinone molecules with up to 14 atoms, containing energy, HOMO, LUMO and dipole moment values at four levels of theory, up to coupled cluster with singles and doubles.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: Machine Learning in Materials Science · Quantum Computing Algorithms and Architecture · Spectroscopy and Quantum Chemical Studies

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

References (84)

Show source-provided references
1. Pham, T.-H., Qiu, Y., Zeng, J., Xie, L. & Zhang, P. A deep learning framework for high-throughput mechanism-driven phenotype compound screening and its application to covid-19 drug repurposing. Nat. Mach. Intell. 3, 247–257 (2021).10.1038/s42256-020-00285-9 ↗
2. Zerner, M. C., Loew, G. H., Kirchner, R. F. & Mueller-Westerhoff, U. T. An intermediate neglect of differential overlap technique for spectroscopy of transition-metal complexes. ferrocene. J. Am. Chem. Soc. 102, 589–599 (1980).10.1021/ja00522a025 ↗
3. Roothaan, C. C. J. New developments in molecular orbital theory. Rev. Mod. Phys. 23, 69 (1951).10.1103/RevModPhys.23.69 ↗
4. Sherrill, C. D. & Schaefer III, H. F. The configuration interaction method: advances in highly correlated approaches, Adv. Quantum Chem. 34, 143–269 (1999).10.1016/S0065-3276(08)60532-8 ↗
5. Møller, C. & Plesset, M. S. Note on an approximation treatment for many-electron systems. Phys. Rev. 46, 618 (1934).10.1103/PhysRev.46.618 ↗
6. Shavitt, I. & Bartlett, R. J. Many-body methods in chemistry and physics: MBPT and coupled-cluster theory (Cambridge university press, 2009).10.1017/CBO9780511596834 ↗
7. Aryasetiawan, F. & Gunnarsson, O. The gw method. Rep. Prog. Phys. 61, 237 (1998).10.1088/0034-4885/61/3/002 ↗
8. Parr, R. G., Gadre, S. R. & Bartolotti, L. J. Local density functional theory of atoms and molecules. Proc. Natl Acad. Sci. USA 76, 2522–2526 (1979).10.1073/pnas.76.6.2522 ↗
9. Hohenberg, P. & Kohn, W. Inhomogeneous electron gas. Phys. Rev. 136, 864–871 (1964).10.1103/PhysRev.136.B864 ↗
10. Parr, R. G. Density functional theory of atoms and molecules. in Horizons of quantum chemistry (Springer, 1980).10.1007/978-94-009-9027-2_2 ↗
11. Perdew, J. P., Burke, K. & Ernzerhof, M. Generalized gradient approximation made simple. Phys. Rev. Lett. 77, 3865 (1996).10.1103/PhysRevLett.77.3865 ↗
12. Stephens, P. J., Devlin, F. J., Chabalowski, C. F. & Frisch, M. J. Ab initio calculation of vibrational absorption and circular dichroism spectra using density functional force fields. J. Phys. Chem. 98, 11623–11627 (1994).10.1021/j100096a001 ↗
13. Curtiss, L. A., Redfern, P. C. & Raghavachari, K. Gaussian-4 theory using reduced order perturbation theory. J. Chem. Phys. 127, 124105 (2007).10.1063/1.2770701 ↗
14. Stewart, R. F. Small gaussian expansions of slater-type orbitals. J. Chem. Phys. 52, 431–438 (1970).10.1063/1.1672702 ↗
15. McGrath, M. P. & Radom, L. Extension of gaussian-1 (g1) theory to bromine-containing molecules. J. Chem. Phys. 94, 511–516 (1991).10.1063/1.460367 ↗
16. Dunning Jr, T. H. Gaussian basis sets for use in correlated molecular calculations. i. the atoms boron through neon and hydrogen. J. Chem. Phys. 90, 1007–1023 (1989).10.1063/1.456153 ↗
17. Rosen, A. S. et al. High-throughput predictions of metal–organic framework electronic properties: theoretical challenges, graph neural networks, and data exploration. npj Comput. Mater. 8, 112 (2022).10.1038/s41524-022-00796-6 ↗
18. Elton, D. C., Boukouvalas, Z., Butrico, M. S., Fuge, M. D. & Chung, P. W. Applying machine learning techniques to predict the properties of energetic materials. Sci. Rep. 8, 9059 (2018).10.1038/s41598-018-27344-x ↗
19. Reiser, P. et al. Graph neural networks for materials science and chemistry. Commun. Mater. 3, 93 (2022).10.1038/s43246-022-00315-6 ↗
20. Hansen, K. et al. Assessment and validation of machine learning methods for predicting molecular atomization energies. J. Chem. Theory Comput. 9, 3404–3419 (2013).10.1021/ct400195d ↗
21. Schmidt, J., Wang, H.-C., Schmidt, G. & Marques, M. A. Machine learning guided high-throughput search of non-oxide garnets. npj Comput. Mater. 9, 63 (2023).10.1038/s41524-023-01009-4 ↗
22. Kirkpatrick, J. et al. Pushing the frontiers of density functionals by solving the fractional electron problem. Science 374, 1385–1389 (2021).10.1126/science.abj6511 ↗
23. Margraf, J. T. & Reuter, K. Pure non-local machine-learned density functional theory for electron correlation. Nat. Commun. 12, 1–7 (2021).10.1038/s41467-020-20471-y ↗
24. Ellis, J. A. et al. Accelerating finite-temperature kohn-sham density functional theory with deep neural networks. Phys. Rev. B 104, 035120 (2021).10.1103/PhysRevB.104.035120 ↗
25. Wang, L. et al. Quantum chemical descriptors in quantitative structure-activity relationship models and their applications. Chemom. Intell. Lab. Syst. 217, 104384 (2021).10.1016/j.chemolab.2021.104384 ↗
26. Manzhos, S. & Carrington Jr, T. A random-sampling high dimensional model representation neural network for building potential energy surfaces. J. Chem. Phys. 125, 084109 (2006).10.1063/1.2336223 ↗
27. Rogers, D. & Hahn, M. Extended-connectivity fingerprints. J. Chem. Inf. Model. 50, 742–754 (2010).10.1021/ci100050t ↗
28. Faber, F. A. et al. Prediction errors of molecular machine learning models lower than hybrid dft error. J. Chem. Theory Comput. 13, 5255–5264 (2017).10.1021/acs.jctc.7b00577 ↗
29. Bartók, A. P., Kondor, R. & Csányi, G. On representing chemical environments. Phys. Rev. B-Condens. Matter Mater. Phys. 87, 184115 (2013).10.1103/PhysRevB.87.184115 ↗
30. Behler, J. Atom-centered symmetry functions for constructing high-dimensional neural network potentials. J. Chem. Phys. 134, 074106 (2011).10.1063/1.3553717 ↗
31. Himanen, L. et al. Dscribe: library of descriptors for machine learning in materials science. Comput. Phys. Commun. 247, 106949 (2020).10.1016/j.cpc.2019.106949 ↗
32. Jäger, M. O., Morooka, E. V., Federici Canova, F., Himanen, L. & Foster, A. S. Machine learning hydrogen adsorption on nanoclusters through structural descriptors. npj Comput. Mater. 4, 37 (2018).10.1038/s41524-018-0096-5 ↗
33. Lange, J. J. et al. Comparative analysis of chemical descriptors by machine learning reveals atomistic insights into solute-lipid interactions. Mol. Pharm. 21, 3343–3355 (2024).10.1021/acs.molpharmaceut.4c00080 ↗
34. Santiago, R., Vela, S., Deumal, M. & Ribas-Arino, J. Unlocking the predictive power of quantum-inspired representations for intermolecular properties in machine learning. Digital Discov. 3, 99–112 (2024).10.1039/D3DD00187C ↗
35. Xu, P., Ji, X., Li, M. & Lu, W. Small data machine learning in materials science. npj Comput. Mater. 9, 42 (2023).10.1038/s41524-023-01000-z ↗
36. Ward, L. et al. Including crystal structure attributes in machine learning models of formation energies via voronoi tessellations. Phys. Rev. B 96, 024104 (2017).10.1103/PhysRevB.96.024104 ↗
37. Dou, B. et al. Machine learning methods for small data challenges in molecular science. Chem. Rev. 123, 8736–8780 (2023).10.1021/acs.chemrev.3c00189 ↗
38. Yamada, H. et al. Predicting materials properties with little data using shotgun transfer learning. ACS Cent. Sci. 5, 1717–1730 (2019).10.1021/acscentsci.9b00804 ↗
39. Kong, S., Guevarra, D., Gomes, C. P. & Gregoire, J. M. Materials representation and transfer learning for multi-property prediction. Appl. Phys. Rev. 8, 021409 (2021).10.1063/5.0047066 ↗
40. Lee, J. & Asahi, R. Transfer learning for materials informatics using crystal graph convolutional neural network. Comput. Mater. Sci. 190, 110314 (2021).10.1016/j.commatsci.2021.110314 ↗
41. Hoffmann, N., Schmidt, J., Botti, S. & Marques, M. A. Transfer learning on large datasets for the accurate prediction of material properties. Digital Discov. 2, 1368–1379 (2023).10.1039/D3DD00030C ↗
42. Wu, S. et al. Machine-learning-assisted discovery of polymers with high thermal conductivity using a molecular design algorithm. Npj Comput. Mater. 5, 66 (2019).10.1038/s41524-019-0203-2 ↗
43. Zhang, Y. et al. Bayesian semi-supervised learning for uncertainty-calibrated prediction of molecular properties and active learning. Chem. Sci. 10, 8154–8163 (2019).10.1039/C9SC00616H ↗
44. Hayes, N., Merkurjev, E. & Wei, G.-W. Integrating transformer and autoencoder techniques with spectral graph algorithms for the prediction of scarcely labeled molecular data. Comput. Biol. Med. 153, 106479 (2023).10.1016/j.compbiomed.2022.106479 ↗
45. Chen, C., Ye, W., Zuo, Y., Zheng, C. & Ong, S. P. Graph networks as a universal machine learning framework for molecules and crystals. Chem. Mater. 31, 3564–3572 (2019).10.1021/acs.chemmater.9b01294 ↗
46. Schütt, K., Sauceda, H., Kindermans, P., Tkatchenko, A. & Müller, K. Schnet-a deep learning architecture for molecules and materials. J. Chem. Phys. 148, 241722 (2018).10.1063/1.5019779 ↗
47. Gasteiger, J., Groß, J. & Günnemann, S. Directional message passing for molecular graphs, arXiv:2003.03123 (2022).
48. Gasteiger, J., Yeshwanth, C. & Günnemann, S. Directional message passing on molecular graphs via synthetic coordinates. Adv. Neural Inf. Process. Syst. 34, 15421–15433 (2021).
49. Fung, V., Zhang, J., Juarez, E. & Sumpter, B. G. Benchmarking graph neural networks for materials chemistry. npj Comput. Mater. 7, 84 (2021).10.1038/s41524-021-00554-0 ↗
50. Peherstorfer, B., Willcox, K. & Gunzburger, M. Survey of multifidelity methods in uncertainty propagation, inference, and optimization. SIAM Rev. 60, 550–591 (2018).10.1137/16M1082469 ↗
51. Kennedy, M. C. & O’Hagan, A. Predicting the output from a complex computer code when fast approximations are available. Biometrika 87, 1–13 (2000).10.1093/biomet/87.1.1 ↗
52. Ramakrishnan, R., Dral, P. O., Rupp, M. & Von Lilienfeld, O. A. Big data meets quantum chemistry approximations: the δ-machine learning approach. J. Chem. theory Comput. 11, 2087–2096 (2015).10.1021/acs.jctc.5b00099 ↗
53. Huang, B., von Lilienfeld, O. A., Krogel, J. T. & Benali, A. Toward dmc accuracy across chemical space with scalable δ-qml. J. Chem. Theory Comput. 19, 1711–1721 (2023).10.1021/acs.jctc.2c01058 ↗
54. Chen, C., Zuo, Y., Ye, W., Li, X. & Ong, S. P. Learning properties of ordered and disordered materials from multi-fidelity data. Nat. Comput. Sci. 1, 46–53 (2021).10.1038/s43588-020-00002-x ↗
55. Zaspel, P., Huang, B., Harbrecht, H. & von Lilienfeld, O. A. Boosting quantum machine learning models with a multilevel combination technique: Pople diagrams revisited. J. Chem. Theory Comput. 15, 1546–1559 (2019).10.1021/acs.jctc.8b00832 ↗
56. Fare, C., Fenner, P., Benatan, M., Varsi, A. & Pyzer-Knapp, E. O. A multi-fidelity machine learning approach to high throughput materials screening. npj Comput. Mater. 8, 257 (2022).10.1038/s41524-022-00947-9 ↗
57. Perdikaris, P., Raissi, M., Damianou, A., Lawrence, N. & Karniadakis, G. E. Nonlinear information fusion algorithms for data-efficient multi-fidelity modelling. Proc. R. Soc. A Math. Phys. Eng. Sci. 473, 20160751 (2017).
58. Narayan, A., Gittelson, C. & Xiu, D. A stochastic collocation algorithm with multifidelity models. SIAM J. Sci. Comput. 36, A495–A521 (2014).10.1137/130929461 ↗
59. Egorova, O., Hafizi, R., Woods, D. C. & Day, G. M. Multifidelity statistical machine learning for molecular crystal structure prediction. J. Phys. Chem. A 124, 8065–8078 (2020).10.1021/acs.jpca.0c05006 ↗
60. Tran, A., Tranchida, J., Wildey, T. & Thompson, A. P. Multi-fidelity machine-learning with uncertainty quantification and bayesian optimization for materials design: application to ternary random alloys. J. Chem. Phys. 153, 074705 (2020).10.1063/5.0015672 ↗
61. Donoho, D., Chui, C., Coifman, R. R. & Lafon, S. Special issue: diffusion maps and Wavelets Diffusion maps. Appl. Comput. Harmonic Anal. 21, 5 – 30 (2006).
62. Coifman, R. R. et al. Geometric diffusions as a tool for harmonic analysis and structure definition of data: Diffusion maps. Proc. Natl Acad. Sci. USA 102, 7426–7431 (2005).10.1073/pnas.0500334102 ↗
63. Venkitaraman, A., Chatterjee, S. & Handel, P. Gaussian processes over graphs. In: ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) (IEEE, 2020).10.1109/ICASSP40776.2020.9053859 ↗
64. Borovitskiy, V. et al. Matérn gaussian processes on graphs. In International Conference on Artificial Intelligence and Statistics (PMLR, 2021).
65. Gao, P. et al. Graphical gaussian process regression model for aqueous solvation free energy prediction of organic molecules in redox flow batteries. Phys. Chem. Chem. Phys. 23, 24892–24904 (2021).10.1039/D1CP04475C ↗
66. Cramer, C. J. Essentials of computational chemistry: theories and models (John Wiley & Sons, 2013).
67. Montavon, G. et al. Machine learning of molecular electronic properties in chemical compound space. N. J. Phys. 15, 095003 (2013).10.1088/1367-2630/15/9/095003 ↗
68. Tkatchenko, A., DiStasio Jr, R. A., Car, R. & Scheffler, M. Accurate and efficient method for many-body van der waals interactions. Phys. Rev. Lett. 108, 236402 (2012).10.1103/PhysRevLett.108.236402 ↗
69. Adamo, C. & Barone, V. Toward reliable density functional methods without adjustable parameters: The pbe0 model. J. Chem. Phys. 110, 6158–6170 (1999).10.1063/1.478522 ↗
70. Ghahremanpour, M. M., Van Maaren, P. J. & Van Der Spoel, D. The alexandria library, a quantum-chemical database of molecular properties for force field development. Sci. Data 5, 1–10 (2018).10.1038/sdata.2018.62 ↗
71. Pople, J. A., Head-Gordon, M., Fox, D. J., Raghavachari, K. & Curtiss, L. A. Gaussian-1 theory: a general procedure for prediction of molecular energies. J. Chem. Phys. 90, 5622–5629 (1989).10.1063/1.456415 ↗
72. Curtiss, L. A., Raghavachari, K., Trucks, G. W. & Pople, J. A. Gaussian-2 theory for molecular energies of first-and second-row compounds. J. Chem. Phys. 94, 7221–7230 (1991).10.1063/1.460205 ↗
73. Barnes, E. C., Petersson, G. A., Montgomery Jr, J. A., Frisch, M. J. & Martin, J. M. Unrestricted coupled cluster and brueckner doubles variations of w1 theory. J. Chem. Theory Comput. 5, 2687–2693 (2009).10.1021/ct900260g ↗
74. Montgomery Jr, J. A., Frisch, M. J., Ochterski, J. W. & Petersson, G. A. A complete basis set model chemistry. vi. use of density functional geometries and frequencies. J. Chem. Phys. 110, 2822–2827 (1999).10.1063/1.477924 ↗
75. Montgomery Jr, J. A., Frisch, M. J., Ochterski, J. W. & Petersson, G. A. A complete basis set model chemistry. vii. use of the minimum population localization method. J. Chem. Phys. 112, 6532–6542 (2000).10.1063/1.481224 ↗
76. Gritsenko, O., van Leeuwen, R., van Lenthe, E. & Baerends, E. J. Self-consistent approximation to the kohn-sham exchange potential. Phys. Rev. A 51, 1944 (1995).10.1103/PhysRevA.51.1944 ↗
77. Heyd, J., Scuseria, G. E. & Ernzerhof, M. Hybrid functionals based on a screened coulomb potential. J. Chem. Phys. 118, 8207–8215 (2003).10.1063/1.1564060 ↗
78. Rupp, M., Tkatchenko, A., Müller, K.-R. & Von Lilienfeld, O. A. Fast and accurate modeling of molecular atomization energies with machine learning. Phys. Rev. Lett. 108, 058301 (2012).10.1103/PhysRevLett.108.058301 ↗
79. Durant, J. L., Leland, B. A., Henry, D. R. & Nourse, J. G. Reoptimization of mdl keys for use in drug discovery. J. Chem. Inf. Comput. Sci. 42, 1273–1280 (2002).10.1021/ci010132r ↗
80. Hansen, K. et al. Machine learning predictions of molecular properties: accurate many-body potentials and nonlocality in chemical space. J. Phys. Chem. Lett. 6, 2326–2331 (2015).10.1021/acs.jpclett.5b00831 ↗
81. Hall, L. H. & Kier, L. B. Electrotopological state indices for atom types: a novel combination of electronic, topological, and valence state information. J. Chem. Inf. Comput. Sci. 35, 1039–1045 (1995).10.1021/ci00028a014 ↗
82. Zou, H., Hastie, T. & Tibshirani, R. Sparse principal component analysis. J. Comput. Graph. Stat. 15, 265–286 (2006).10.1198/106186006X113430 ↗
83. Sun, Q. et al. Pyscf: the python-based simulations of chemistry framework. WIREs Comput. Mol. Sci. 8, e1340 (2018).10.1002/wcms.1340 ↗
84. Kingma, D. P. & Ba, J. Adam: a method for stochastic optimization, arXiv preprint arXiv:1412.6980 (2014).