Sampling thermodynamic ensembles of molecular systems with generative neural networks: Will integrating physics-based models close the generalization gap?
Source abstract
Abstract
If the promise of generative modeling techniques is realized, it may fundamentally change how we carry out molecular simulation. The suite of techniques and models collectively termed “generative AI” includes many different classes of models built for varied types of data, from natural language to images. Recent advances in the machine learning literature that construct ever better generative models, though, do not contend with the challenges unique to complex, molecular systems. To generate a statistically likely molecular configuration, many correlated degrees of freedom must be sampled together, while also satisfying the strong constraints of chemical physics. Recent efforts to develop generative models for biomolecular systems have shown spectacular results in some cases—nevertheless, some simple systems remain out of reach with our present methodology. Arguably, the central concern is data efficiency: we should aim to train models that can meaningfully generalize beyond their training data and hence facilitate discovery. In this review, we discuss methods and future directions for directly incorporating physics-based models into generative neural networks, which we believe is a crucial step for addressing the limitations of the current toolkit.
Source: expertdata · Quality: candidate unverified · Source record ↗
Research context
OpenAlex topics: Machine Learning in Materials Science · Protein Structure and Dynamics · Quantum many-body systems
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
References (85)
Show source-provided references
1. Albergo · 2023
2. Free energy calculation of crystalline solids using normalizing flows10.1088/1361-651X/ac7f4b ↗
3. Amodei · AI and Compute · 2018
4. Reverse-time diffusion equation models10.1016/0304-4149(82)90051-5 ↗
5. Michael S. Albergo, Eric Vanden-Eijnden, Building normalizing flows with stochastic interpolants. In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1–5, 2023. OpenReview.net, 2023.
6. Prediction of protein side-chain rotamers from a backbone-dependent rotamer library: A new homology modeling tool11Edited by B10.1006/jmbi.1997.0926 ↗
7. Yasaman Bahri, Ethan Dyer, Jared Kaplan, Jaehoon Lee, Utkarsh Sharma, Explaining Neural Scaling Laws, February 2021.
8. James Bradbury, Roy Frostig, Peter Hawkins, Matthew James Johnson, Chris Leary, Dougal Maclaurin, George Necula, Adam Paszke, Jake VanderPlas, Skye Wanderman-Milne, Qiao Zhang, JAX: Composable transformations of Python+NumPy programs, 2018.
9. Ilyes Batatia, David P. Kovacs, Gregor Simm, Christoph Ortner, Gabor Csanyi, MACE: Higher order equivariant message passing neural networks for fast and accurate force fields. In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, A. Oh, editors, Advances in Neural Information Processing Systems, volume 35, pages 11423–11436. Curran Associates Inc, 2022.
10. E(3)-equivariant graph neural networks for data-efficient and accurate interatomic potentials10.1038/s41467-022-29939-5 ↗
11. Michael Broughton, Guillaume Verdon, Trevor McCourt, Antonio J. Martinez, Jae Hyeon Yoo, Sergei V. Isakov, Philip Massey, Murphy Yuezhen Niu, Ramin Halavati, Evan Peters, Martin Leib, Andrea Skolik, Michael Streif, David Von Dollen, Jarrod R. McClean, Sergio Boixo, Dave Bacon, Alan K. Ho, Hartmut Neven, Masoud Mohseni, TensorFlow Quantum: A Software Framework for Quantum Machine Learning. arXiv:2003.02989 [cond-mat, physics:quant-ph], March 2020.
12. Machine learning and the physical sciences10.1103/RevModPhys.91.045002 ↗
13. The Amber biomolecular simulation programs10.1002/jcc.20290 ↗
14. Chandler · 1987
15. Data-Efficient Generation of Protein Conformational Ensembles with Backbone-to-Side-Chain Transformers10.1021/acs.jpcb.3c08195 ↗
16. Ricky T.Q. Chen, Yulia Rubanova, Jesse Bettencourt, and David K Duvenaud. Neural ordinary differential equations. In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett, editors, Advances in Neural Information Processing Systems, volume 31. Curran Associates Inc, 2018.
17. Entropy production fluctuation theorem and the nonequilibrium work relation for free energy differences10.1103/PhysRevE.60.2721 ↗
18. Ensuring thermodynamic consistency with invertible coarse-graining10.1063/5.0141888 ↗
19. Machine-learning-assisted Monte Carlo fails at sampling computationally hard problems
20. Juan Viguera Diez, Sara Romeo Atance, Ola Engkvist, Simon Olsson, Generation of conformational ensembles of small molecules via surrogate model-assisted molecular dynamics, November 2023.10.26434/chemrxiv-2023-sx61w ↗
21. Conor Durkan, Artur Bekasov, Iain Murray, George Papamakarios. Neural Spline Flows, in: H. Wallach, H. Larochelle, A. Beygelzimer, F. d’Alché-Buc, E. Fox, and R. Garnett, editors, Advances in Neural Information Processing Systems, volume 32. Curran Associates Inc, 2019.
22. Efficient modeling of trivializing maps for lattice ϕ4 theory using normalizing flows: A first look at scalability10.1103/PhysRevD.104.094507 ↗
23. Bayesian statistical analysis of protein side-chain rotamer preferences10.1002/pro.5560060807 ↗
24. An adaptive sequential Monte Carlo method for approximate Bayesian computation10.1007/s11222-011-9271-y ↗
25. Density Estimation Using Real NVP
26. Rotamer Libraries in the 21st Century10.1016/S0959-440X(02)00344-5 ↗
27. Loris Felardos, Jérôme Hénin, Guillaume Charpiat, Designing losses for data-free training of normalizing flows on Boltzmann distributions, January 2023.
28. New Monte Carlo method to compute the free energy of arbitrary solids. Application to the fcc and hcp phases of hard spheres10.1063/1.448024 ↗
29. Frenkel · 2002
30. Free-Form Continuous Dynamics for Scalable Reversible Generative Models
31. Davide Ghio, Yatin Dandi, Florent Krzakala, Lenka Zdeborová, Sampling with flows, diffusion and autoregressive neural networks: A spin-glass perspective, August 2023.10.1073/pnas.2311810121 ↗
32. Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, Yoshua Bengio, Generative Adversarial Nets, in: Z. Ghahramani, M. Welling, C. Cortes, N.D. Lawrence, and K.Q. Weinberger, editors, Advances in Neural Information Processing Systems 27, pages 2672–2680. Curran Associates Inc, 2014.
33. Efficient Bayesian Sampling Using Normalizing Flows to Assist Markov Chain Monte Carlo Methods
34. Adaptive Monte Carlo augmented with normalizing flows10.1073/pnas.2109420119 ↗
35. Coarse graining molecular dynamics with graph neural networks10.1063/5.0026133 ↗
36. Jonathan Ho, Ajay Jain, Pieter Abbeel, Denoising diffusion probabilistic models. In Hugo Larochelle, Marc’Aurelio Ranzato, Raia Hadsell, Maria-Florina Balcan, and Hsuan-Tien Lin, editors, Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6–12, 2020, Virtual, 2020.
37. A Stochastic Estimator of the Trace of the Influence Matrix for Laplacian Smoothing Splines10.1080/03610918908812806 ↗
38. Skipping the Replica Exchange Ladder with Normalizing Flows10.1021/acs.jpclett.2c03327 ↗
39. Highly accurate protein structure prediction with AlphaFold10.1038/s41586-021-03819-2 ↗
40. John M. Jumper, Karl F. Freed, Tobin R. Sosnick, Rapid calculation of side chain packing and free energy with applications to protein molecular dynamics, September 2017.10.1371/journal.pcbi.1006342 ↗
41. Bottom-up Coarse-Graining: Principles and Perspectives10.1021/acs.jctc.2c00643 ↗
42. Physics-informed machine learning10.1038/s42254-021-00314-5 ↗
43. Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B. Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. Scaling Laws for Neural Language Models, January 2020.
44. Fluctuation theorem for stochastic dynamics10.1088/0305-4470/31/16/003 ↗
45. Flow Matching for Generative Modeling
46. Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow
47. Generalizing Hamiltonian Monte Carlo with Neural Networks
48. Lelièvre · 2010
49. A Gallavotti–Cohen-Type Symmetry in the Large Deviation Functional for Stochastic Dynamics10.1023/A:1004589714161 ↗
50. Applications of artificial intelligence for chemical inference. I. Number of possible organic compounds. Acyclic structures containing carbon, hydrogen, oxygen, and nitrogen10.1021/ja01039a025 ↗
51. FP-Diffusion: Improving Score-based Diffusion Models by Enforcing the Underlying Score Fokker-Planck Equation
52. Deep Learning in Chemistry10.1021/acs.jcim.9b00266 ↗
53. Bypassing backmapping: Coarse-grained electronic property distributions using heteroscedastic Gaussian processes10.1063/5.0101038 ↗
54. Simulations of disordered matter in 3d with the morphological autoregressive protocol (map) and convolutional neural networks10.1063/5.0174615 ↗
55. Mehdi · 2023
56. The multiscale coarse-graining method. I. A rigorous bridge between atomistic and coarse-grained models10.1063/1.2938860 ↗
57. Boltzmann generators: Sampling equilibrium states of many-body systems with deep learning10.1126/science.aaw1147 ↗
58. Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang, Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, Soumith Chintala. PyTorch: An Imperative Style, High-Performance Deep Learning Library. In Advances in Neural Information Processing Systems, volume 32. Curran Associates Inc, 2019.
59. Reweighted autoencoded variational Bayes for enhanced sampling (RAVE)10.1063/1.5025487 ↗
60. Variational Inference with Normalizing Flows
61. Jascha Sohl-Dickstein, Benjamin J. Culpepper, Hamiltonian Annealed Importance Sampling for partition function estimation. arXiv:1205.1925 [physics], May 2012.
62. Molecular latent space simulators10.1039/D0SC03635H ↗
63. Yuyang Shi, Valentin De Bortoli, Andrew Campbell, Arnaud Doucet, Diffusion Schrødinger Bridge Matching, March 2023.
64. Arne Schneuing, Yuanqi Du, Charles Harris, Arian Jamasb, Ilia Igashov, Weitao Du, Tom Blundell, Pietro Lió, Carla Gomes, Max Welling, Michael Bronstein, and Bruno Correia. Structure-based Drug Design with Equivariant Diffusion Models, October 2022.
65. Salakhutdinov · Deep Boltzmann Machines. AISTATS · 2009
66. Victor Garcia Satorras, Emiel Hoogeboom, Fabian B. Fuchs, Ingmar Posner, Max Welling, E(n) equivariant normalizing flows for molecule generation in 3D. CoRR, abs/2105.09016, 2021.
67. Akiyoshi Sannai, Masaaki Imaizumi, Makoto Kawano, Improved generalization bounds of group invariant/ equivariant deep networks via quotient feature spaces. In Proceedings of the Thirty-Seventh Conference on Uncertainty in Artificial Intelligence, pages 771–780. PMLR, December 2021.
68. Restricted Boltzmann machines for collaborative filtering
69. SchNet – A deep learning architecture for molecules and materials10.1063/1.5019779 ↗
70. Score-Based Generative Modeling through Stochastic Differential Equations
71. Jiaming Song, Shengjia Zhao, and Stefano Ermon. A-nice-mc: Adversarial training for MCMC. In I. Guyon, U.V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, editors, Advances in Neural Information Processing Systems, volume 30. Curran Associates Inc, 2017.
72. Density estimation by dual ascent of the log-likelihood10.4310/CMS.2010.v8.n1.a11 ↗
73. Denoising Diffusion Samplers
74. Likelihood-based non-Markovian models from molecular dynamics10.1073/pnas.2117586119 ↗
75. CHARMM general force field: A force field for drug-like molecules compatible with the CHARMM all-atom additive biological force fields10.1002/jcc.21367 ↗
76. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is All you Need. In Advances in Neural Information Processing Systems, volume 30. Curran Associates Inc, 2017.
77. Targeted free energy estimation via learned mappings10.1063/5.0018903 ↗
78. Coarse-graining auto-encoders for molecular dynamics10.1038/s41524-019-0261-5 ↗
79. From data to noise to data for mixing physics across temperatures with generative artificial intelligence10.1073/pnas.2203656119 ↗
80. Estimating Gibbs free energies via isobaric-isothermal flows
81. De novo design of protein structure and function with RFdiffusion10.1038/s41586-023-06415-8 ↗
82. Guided diffusion for inverse molecular design
83. Machine Learning of Coarse-Grained Molecular Dynamics Force Fields10.1021/acscentsci.8b00913 ↗
84. Normalizing flows for atomic solids
85. Shuxin Zheng, Jiyan He, Chang Liu, Yu Shi, Ziheng Lu, Weitao Feng, Fusong Ju, Jiaxi Wang, Jianwei Zhu, Yaosen Min, He Zhang, Shidi Tang, Hongxia Hao, Peiran Jin, Chi Chen, Frank Noé, Haiguang Liu, Tie-Yan Liu, Towards predicting equilibrium distributions for molecular systems with deep learning, June 2023.10.1038/s42256-024-00837-3 ↗