Assessing Deep Generative Models in Chemical Composition Space
Source abstract
Abstract
The computational discovery of novel materials has been one of the main motivations behind research in theoretical chemistry for several decades. Despite much effort, this is far from a solved problem, however. Among other reasons, this is due to the enormous space of possible structures and compositions that could potentially be of interest. In the case of inorganic materials, this is exacerbated by the combinatorics of the periodic table since even a single-crystal structure can in principle display millions of compositions. Consequently, there is a need for tools that enable a more guided exploration of the materials design space. Here, generative machine learning models have recently emerged as a promising technology. In this work, we assess the performance of a range of deep generative models based on reinforcement learning, variational autoencoders, and generative adversarial networks for the prototypical case of designing Elpasolite compositions with low formation energies. By relying on the fully enumerated space of 2 million main-group Elpasolites, the precision, coverage, and diversity of the generated materials are rigorously assessed. Additionally, a hyperparameter selection scheme for generative models in chemical composition space is developed.
Source: openalex_inverted_index · Quality: baseline
Research context
OpenAlex topics: Machine Learning in Materials Science · Computational Drug Discovery Methods · Topic Modeling
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Fritz-Haber-Institut der Max-Planck-Gesellschaft, Faradayweg 4-6, D-14195Berlin, Germany
Chair for Theoretical Chemistry and Catalysis Research Center, Department of Chemistry, Technische Universität München, Lichtenbergstraße 4, 85748Garching, Germany
Funding
Deutsche Forschungsgemeinschaft
References (90)
Show source-provided references
1. Brock, A.; Donahue, J.; Simonyan, K.Large Scale GAN Training for High Fidelity Natural Image Synthesis. 2019. arXiv preprint arXiv:1809.11096
2. Clark, A.; Donahue, J.; Simonyan, K.Adversarial Video Generation on Complex Datasets. 2019. arXiv preprint arXiv:1907.06571
3. Nie, W.; Narodytska, N.; Patel, A.RelGAN: Relational Generative Adversarial Networks for Text Generation. International Conference on Learning Representations , 2019.
4. Dong, H.W.; Hsiao, W.Y.; Yang, L.C.; Yang, Y.H.MuseGAN: Multi-track Sequential Generative Adversarial Networks for Symbolic Music Generation and Accompaniment. Proceedings of the Annual AAAI Conference on Artificial Intelligence , 2018.10.1609/aaai.v32i1.11312 ↗
5. 10.1002/minf.20170015310.1002/minf.201700153 ↗
6. 10.1038/s41587-019-0224-x10.1038/s41587-019-0224-x ↗
7. 10.1126/science.aat266310.1126/science.aat2663 ↗
8. 10.1039/c9me00039a10.1039/c9me00039a ↗
9. 10.1021/acs.jcim.0c0138810.1021/acs.jcim.0c01388 ↗
10. 10.1186/s13321-019-0341-z10.1186/s13321-019-0341-z ↗
11. 10.1021/acs.jcim.8b0083910.1021/acs.jcim.8b00839 ↗
12. 10.3389/fphar.2020.56564410.3389/fphar.2020.565644 ↗
13. 10.1021/acs.jcim.0c0132810.1021/acs.jcim.0c01328 ↗
14. Flam-Shepherd, D.; Zhu, K.; Aspuru-Guzik, A.Keeping it Simple: Language Models can learn Complex Molecular Distributions. 2021. arXiv preprint arXiv:2112.0304110.1038/s41467-022-30839-x ↗
15. 10.1021/acs.jcim.0c0149610.1021/acs.jcim.0c01496 ↗
16. 10.1002/minf.20170011110.1002/minf.201700111 ↗
17. 10.1021/acscentsci.7b0051210.1021/acscentsci.7b00512 ↗
18. 10.1021/acs.jcim.0c0034310.1021/acs.jcim.0c00343 ↗
19. 10.1021/acs.jmedchem.9b0214710.1021/acs.jmedchem.9b02147 ↗
20. 10.1038/s42256-020-0160-y10.1038/s42256-020-0160-y ↗
21. 10.1021/acs.jcim.8b0026310.1021/acs.jcim.8b00263 ↗
22. 10.1039/c9sc04503a10.1039/c9sc04503a ↗
23. 10.1186/s13321-018-0287-610.1186/s13321-018-0287-6 ↗
24. 10.1038/s42256-020-0174-510.1038/s42256-020-0174-5 ↗
25. 10.1038/s41467-022-28526-y10.1038/s41467-022-28526-y ↗
26. 10.1186/s13321-018-0286-710.1186/s13321-018-0286-7 ↗
27. Guimaraes, G. L.; Sanchez-Lengeling, B.; Outeiral, C.; Farias, P. L. C.; Aspuru-Guzik, A.Objective-Reinforced Generative Adversarial Networks (ORGAN) for Sequence Generation Models. 2017. arXiv preprint arXiv:1705.10843
28. 10.1126/sciadv.aap788510.1126/sciadv.aap7885 ↗
29. 10.1186/s13321-017-0235-x10.1186/s13321-017-0235-x ↗
30. 10.1021/acs.jcim.0c0091510.1021/acs.jcim.0c00915 ↗
31. 10.1021/acscentsci.7b0057210.1021/acscentsci.7b00572 ↗
32. Jin, W.; Barzilay, R.; Jaakkola, T.Junction Tree Variational Autoencoder for Molecular Graph Generation. 2019. arXiv preprint arXiv:1802.0436410.1039/9781788016841-00228 ↗
33. 10.1039/c9sc04026a10.1039/c9sc04026a ↗
34. 10.1021/acs.jcim.0c0046410.1021/acs.jcim.0c00464 ↗
35. 10.1016/j.matt.2019.08.01710.1016/j.matt.2019.08.017 ↗
36. 10.1016/j.commatsci.2021.11043610.1016/j.commatsci.2021.110436 ↗
37. 10.1038/s41524-021-00526-410.1038/s41524-021-00526-4 ↗
38. 10.1021/acscentsci.0c0042610.1021/acscentsci.0c00426 ↗
39. 10.1038/s42256-020-00271-110.1038/s42256-020-00271-1 ↗
40. 10.1016/j.carbon.2020.07.01310.1016/j.carbon.2020.07.013 ↗
41. 10.1038/s41524-020-00352-010.1038/s41524-020-00352-0 ↗
42. 10.1039/d0cp03508d10.1039/d0cp03508d ↗
43. 10.1002/advs.20210056610.1002/advs.202100566 ↗
44. 10.20517/jmi.2021.0710.20517/jmi.2021.07 ↗
45. Hoffmann, J.; Maestrati, L.; Sawada, Y.; Tang, J.; Sellier, J. M.; Bengio, Y.Data-driven approach to encoding and decoding 3-d crystal structures. 2019. arXiv preprint arXiv:1909.00949
46. 10.1016/j.matt.2021.11.03210.1016/j.matt.2021.11.032 ↗
47. 10.1039/d0mh00881h10.1039/d0mh00881h ↗
48. Nouira, A.; Sokolovska, N.; Crivello, J.C.CrystalGAN: Learning to Discover Crystallographic Structures with Generative Adversarial Networks. 2019. arXiv preprint arXiv:1810.11203
49. Debnath, A.; Krajewski, A. M.; Sun, H.; Lin, S.; Ahn, M.; Li, W.; Priya, S.; Singh, J.; Shang, S.; Beese, A. M., Generative deep learning as a tool for inverse design of high-entropy refractory alloys. 2021. arXiv preprint arXiv:2108.1201910.20517/jmi.2021.05 ↗
50. 10.1364/oe.27.00587410.1364/oe.27.005874 ↗
51. 10.1016/j.eml.2020.10065110.1016/j.eml.2020.100651 ↗
52. 10.1016/j.ddtec.2020.09.00310.1016/j.ddtec.2020.09.003 ↗
53. 10.1038/s42256-021-00368-110.1038/s42256-021-00368-1 ↗
54. Sawada, Y.; Morikawa, K.; Fujii, M.Study of deep generative models for inorganic chemical compositions. 2019. arXiv preprint arXiv:1910.11499
55. 10.1103/physrevlett.117.13550210.1103/physrevlett.117.135502 ↗
56. 10.1016/j.commatsci.2019.10943610.1016/j.commatsci.2019.109436 ↗
57. 10.1038/s41586-019-1335-810.1038/s41586-019-1335-8 ↗
58. Huo, H.; Rupp, M.Unified Representation of Molecules and Crystals for Machine Learning. 2017. arXiv preprint arXiv:1704.06439
59. 10.1039/c8cp05921g10.1039/c8cp05921g ↗
60. 10.1073/pnas.180118111510.1073/pnas.1801181115 ↗
61. Metz, L.; Poole, B.; Pfau, D.; Sohl-Dickstein, J.Unrolled Generative Adversarial Networks. 2017. arXiv preprint arXiv:1611.02163
62. Yan, C.; Wang, S.; Yang, J.; Xu, T.; Huang, J.Re-balancing variational autoencoder loss for molecule sequence generation. Proceedings of the 11th ACM International Conference on Bioinformatics, Computational Biology and Health Informatics , 2020; pp 1–7.10.1145/3388440.3412458 ↗
63. Lucas J. · 2019
64. Maas, A. L.; Hannun, A.; Ng, A.Rectifier Nonlinearities Improve Neural Network Acoustic Models. Proceedings of the 30th International Conference on Machine Learning , 2013.
65. Kingma, D. P.; Ba, J.Adam: A Method for Stochastic Optimization. 2017. arXiv preprint arXiv:1412.6980
66. Bergstra J. · J. Mach. Learn. Res. · 2012 · 13 · 281
67. Kingma, D. P.; Welling, M.Auto-encoding variational bayes. 2013. arXiv preprint arXiv:1312.6114
68. Rezende, D. J.; Mohamed, S.; Wierstra, D.Stochastic backpropagation and approximate inference in deep generative models. Proceedings of the 31st International Conference on Machine Learning , 2014; pp 1278–1286.
69. 10.1214/aoms/117772969410.1214/aoms/1177729694 ↗
70. Kusner, M. J.; Paige, B.; Hernández-Lobato, J. M. Grammar Variational Autoencoder. International Conference on Machine Learning , 2017; pp 1945–1954.
71. Sohn, K.; Lee, H.; Yan, X.Learning Structured Output Representation using Deep Conditional Generative Models. Advances in Neural Information Processing Systems , 2015.
72. Karanasou, P.; Karlapati, S.; Moinet, A.; Joly, A.; Abbas, A.; Slangen, S.; Trueba, J. L.; Drugman, T.A learned conditional prior for the VAE acoustic space of a TTS system. 2021. arXiv preprint arXiv:2106.1022910.21437/Interspeech.2021-528 ↗
73. Ba, J. L.; Kiros, J. R.; Hinton, G. E.Layer Normalization. 2016. arXiv preprint arXiv:1607.06450
74. Goodfellow, I. J.; Pouget-Abadie, J.; Mirza, M.; Xu, B.; Warde-Farley, D.; Ozair, S.; Courville, A.; Bengio, Y.Generative Adversarial Networks. 2014. arXiv preprint arXiv:1406.2661
75. Salimans, T.; Goodfellow, I.; Zaremba, W.; Cheung, V.; Radford, A.; Chen, X.Improved Techniques for Training GANs. 2016. arXiv preprint arXiv:1606.03498
76. Gui, J.; Sun, Z.; Wen, Y.; Tao, D.; Ye, J.A Review on Generative Adversarial Networks: Algorithms, Theory, and Applications. 2020. arXiv preprint arXiv:2001.06937
77. Arjovsky, M.; Chintala, S.; Bottou, L.; Wasserstein GAN. 2017. arXiv preprint arXiv:1701.07875
78. Gulrajani, I.; Ahmed, F.; Arjovsky, M.; Dumoulin, V.; Courville, A.Improved Training of Wasserstein GANs. 2017. arXiv preprint arXiv:1704.00028
79. Mirza, M.; Osindero, S.Conditional Generative Adversarial Nets. 2014. arXiv preprint arXiv:1411.1784
80. Gauthier J. · 2014 · 2014 · 2
81. Simm, G. N.; Pinsler, R.; Hernández-Lobato, J. M.Reinforcement learning for molecular design guided by quantum mechanics. 37th International Conference on Machine Learning. ICML 2020 , 2020; PartF168147-12, pp 8906–8916.
82. 10.1186/s13321-020-00473-010.1186/s13321-020-00473-0 ↗
83. Meldgaard, S. A.; Köhler, J.; Mortensen, H. L.; Christiansen, M.P. V.; Noé, F.; Hammer, B.Generating stable molecules using imitation and reinforcement learning. 2021. arXiv preprint arXiv:2107.05007
84. Thiede, L. A.; Krenn, M.Curiosity in exploring chemical space : Intrinsic rewards for deep molecular reinforcement learning. 2020. arXiv preprint arXiv:2012.11293
85. Sutton R. · 2014
86. Mnih, V.; Kavukcuoglu, K.; Silver, D.; Graves, A.; Antonoglou, I.; Wierstra, D.; Riedmiller, M.Playing Atari with Deep Reinforcement Learning. 2013. arXiv preprint arXiv:1312.5602
87. 10.1007/978-3-319-56991-8_3210.1007/978-3-319-56991-8_32 ↗
88. Hessel, M.; Modayil, J.; Van Hasselt, H.; Schaul, T.; Ostrovski, G.; Dabney, W.; Horgan, D.; Piot, B.; Azar, M.; Silver, D.Rainbow: Combining improvements in deep reinforcement learning. 32nd AAAI Conference on Artificial Intelligence AAAI 2018 , 2018; pp 3215–3222.10.1609/aaai.v32i1.11796 ↗
89. Van Hasselt, H.; Guez, A.; Silver, D.Deep reinforcement learning with double Q-Learning. 30th AAAI Conference on Artificial Intelligence AAAI 2016 , 2016; pp 2094–2100.10.1609/aaai.v30i1.10295 ↗
90. 10.1039/d1dd00038a10.1039/d1dd00038a ↗