Towards understanding structure–property relations in materials with interpretable deep learning
Source abstract
Abstract
Abstract Deep learning (DL) models currently employed in materials research exhibit certain limitations in delivering meaningful information for interpreting predictions and comprehending the relationships between structure and material properties. To address these limitations, we propose an interpretable DL architecture that incorporates the attention mechanism to predict material properties and gain insights into their structure–property relationships. The proposed architecture is evaluated using two well-known datasets (the QM9 and the Materials Project datasets), and three in-house-developed computational materials datasets. Train–test–split validations confirm that the models derived using the proposed DL architecture exhibit strong predictive capabilities, which are comparable to those of current state-of-the-art models. Furthermore, comparative validations, based on first-principles calculations, indicate that the degree of attention of the atoms’ local structures to the representation of the material structure is critical when interpreting structure–property relationships with respect to physical properties. These properties encompass molecular orbital energies and the formation energies of crystals. The proposed architecture shows great potential in accelerating material design by predicting material properties and explicitly identifying crucial features within the corresponding structures.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Machine Learning in Materials Science · X-ray Diffraction in Crystallography · Computational Drug Discovery Methods
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
References (53)
Show source-provided references
1. Agrawal, A. & Choudhary, A. Perspective: materials informatics and big data: realization of the “fourth paradigm" of science in materials science. APL Mater. 4, 053208 (2016).10.1063/1.4946894 ↗
2. Ramprasad, R., Batra, R., Pilania, G., Mannodi-Kanakkithodi, A. & Kim, C. Machine learning in materials informatics: recent applications and prospects. Npj Comput. Mater. 3, 54 (2017).10.1038/s41524-017-0056-5 ↗
3. Butler, K. T., Davies, D. W., Cartwright, H., Isayev, O. & Walsh, A. Machine learning for molecular and materials science. Nature 559, 547–555 (2018).10.1038/s41586-018-0337-2 ↗
4. Siriwardane, E. M. D., Zhao, Y., Perera, I. & Hu, J. Generative design of stable semiconductor materials using deep learning and density functional theory. Npj Comput. Mater. 8, 164 (2022).10.1038/s41524-022-00850-3 ↗
5. Ward, L. & Wolverton, C. Atomistic calculations and materials informatics: a review. Curr. Opin. Solid State Mater. Sci. 21, 167–176 (2017).10.1016/j.cossms.2016.07.002 ↗
6. Ramakrishnan, R., Dral, P. O., Rupp, M. & Von Lilienfeld, O. A. Quantum chemistry structures and properties of 134 kilo molecules. Sci. data 1, 1–7 (2014).10.1038/sdata.2014.22 ↗
7. Himanen, L., Geurts, A., Foster, A. S. & Rinke, P. Data-driven materials science: status, challenges, and perspectives. Adv. Sci. 6, 1900808 (2019).10.1002/advs.201900808 ↗
8. Zhao, Y. et al. Physics guided deep learning for generative design of crystal materials with symmetry constraints. Npj Comput. Mater. 9, 38 (2023).10.1038/s41524-023-00987-9 ↗
9. Rupp, M., Tkatchenko, A., Müller, K.-R. & Von Lilienfeld, O. A. Fast and accurate modeling of molecular atomization energies with machine learning. Phys. Rev. Lett. 108, 058301 (2012).10.1103/PhysRevLett.108.058301 ↗
10. Schütt, K. T. et al. How to represent crystal structures for machine learning: towards fast prediction of electronic properties. Phys. Rev. B 89, 205118 (2014).10.1103/PhysRevB.89.205118 ↗
11. Karamad, M. et al. Orbital graph convolutional neural network for material property prediction. Phys. Rev. Mater. 4, 093801 (2020).10.1103/PhysRevMaterials.4.093801 ↗
12. Xie, T. & Grossman, J. C. Crystal graph convolutional neural networks for an accurate and interpretable prediction of material properties. Phys. Rev. Lett. 120, 145301 (2018).10.1103/PhysRevLett.120.145301 ↗
13. Rahaman, O. & Gagliardi, A. Deep learning total energies and orbital energies of large organic molecules using hybridization of molecular fingerprints. J. Chem. Inf. Model. 60, 5971–5983 (2020).10.1021/acs.jcim.0c00687 ↗
14. Anderson, B., Hy, T. S. & Kondor, R. Cormorant: covariant molecular neural networks. NeurIPS 32, 14537–14546 (2019).
15. Fuchs, F. B., Worrall, D. E., Fischer, V. & Welling, M. Se(3)-transformers: 3d roto-translation equivariant attention networks. NeurIPS 33, 1970–1981 (2020).
16. Vaswani, A. et al. Attention is all you need. NeurIPS 30, 6000–6010 (2017).
17. Duvenaud, D. K. et al. Convolutional networks on graphs for learning molecular fingerprints. NeurIPS 28, 2224–2232 (2015).
18. Wu, Z. et al. Moleculenet: a benchmark for molecular machine learning. Chem. Sci. 9, 513–530 (2018).10.1039/C7SC02664A ↗
19. Fung, V., Zhang, J., Juarez, E. & Sumpter, B. G. Benchmarking graph neural networks for materials chemistry. Npj Comput. Mater. 7, 84 (2021).10.1038/s41524-021-00554-0 ↗
20. Gilmer, J., Schoenholz, S. S., Riley, P. F., Vinyals, O. & Dahl, G. E. Neural message passing for quantum chemistry. ICML 70, 1263–1272 (2017).
21. Yang, K. et al. Analyzing learned molecular representations for property prediction. J. Chem. Inf. Model. 59, 3370–3388 (2019).10.1021/acs.jcim.9b00237 ↗
22. Chen, P., Jiao, R., Liu, J., Liu, Y. & Lu, Y. Interpretable graph transformer network for predicting adsorption isotherms of metal-organic frameworks. J. Chem. Inf. Model. 62, 5446–5456 (2022).10.1021/acs.jcim.2c00876 ↗
23. Cao, Z., Magar, R., Wang, Y. & Barati Farimani, A. MOFormer: self-supervised transformer model for metal-organic framework property prediction. J. Am. Chem. Soc. 145, 2958–2967 (2023).10.1021/jacs.2c11420 ↗
24. Kang, Y., Park, H., Smit, B. & Kim, J. A multi-modal pre-training transformer for universal transfer learning in metal–organic frameworks. Nat. Mach. Intell. 5, 309–318 (2023).10.1038/s42256-023-00628-2 ↗
25. Korolev, V. & Protsenko, P. Accurate, interpretable predictions of materials properties within transformer language models. Patterns 4, 100803 (2023).10.1016/j.patter.2023.100803 ↗
26. Das, K., Goyal, P., Lee, S.-C., Bhattacharjee, S. & Ganguly, N. Crysmmnet: multimodal representation for crystal property prediction. PMLR 216, 507–517 (2023).
27. Gunning, D. et al. XAI–Explainable artificial intelligence. Sci. Robot. 4, eaay7120 (2019).10.1126/scirobotics.aay7120 ↗
28. Moran, M., Gaultois, M. W., Gusev, V. V. & Rosseinsky, M. J. Site-net: using global self-attention and real-space supercells to capture long-range interactions in crystal structures. Digit. Discov. 2, 1297–1310 (2023).10.1039/D3DD00005B ↗
29. Pham, T.-L. et al. Learning materials properties from orbital interactions. J. Phys. Conf. Ser. 1290, 012012 (2019).10.1088/1742-6596/1290/1/012012 ↗
30. Schütt, K. T., Sauceda, H. E., Kindermans, P.-J., Tkatchenko, A. & Müller, K.-R. Schnet–a deep learning architecture for molecules and materials. J. Chem. Phys 148, 241722 (2018).10.1063/1.5019779 ↗
31. Wu, Z. et al. A comprehensive survey on graph neural networks. IEEE Trans. Neural Netw. Learn. Syst. 32, 4–24 (2021).10.1109/TNNLS.2020.2978386 ↗
32. Schweidtmann, A. M. et al. Physical pooling functions in graph neural networks for molecular property prediction. Comput. Chem. Eng. 172, 108202 (2023).10.1016/j.compchemeng.2023.108202 ↗
33. Keyulu, X., Weihua H., Leskovec, J. & Jegelka, S. How Powerful are Graph Neural Networks? In Proceedings of the 7th International Conference on Learning Representations (ICLR 2019), New Orleans, LA, USA, 2019.
34. Jain, A. et al. Commentary: the materials project: a materials genome approach to accelerating materials innovation. APL Mater. 1, 011002 (2013).10.1063/1.4812323 ↗
35. Chen, C., Ye, W., Zuo, Y., Zheng, C. & Ong, S. P. Graph networks as a universal machine learning framework for molecules and crystals. Chem. Mater. 31, 3564–3572 (2019).10.1021/acs.chemmater.9b01294 ↗
36. Choudhary, K. & DeCost, B. Atomistic line graph neural network for improved materials property predictions. Npj Comput. Mater. 7, 185 (2021).10.1038/s41524-021-00650-1 ↗
37. Vu, T.-S. & Chi, D. H. Fullerene structures and Pt absorbed on Graphene structures with HOMO, LUMO and Total energy properties. Zenodo, https://zenodo.org/record/7792716 (2023).
38. Nguyen, D.-N., Kino, H., Miyake, T. & Dam, H.-C. Explainable active learning in investigating structure-stability of SmFe12−α−βXαYβ structures X, Y {Mo, Zn, Co, Cu, Ti, Al, Ga}. MRS Bull. 48, 31–44 (2022).10.1557/s43577-022-00372-9 ↗
39. Hirn, M., Mallat, S. & Poilvert, N. Wavelet scattering regression of quantum chemical energies. Multiscale Model. Simul. 15, 827–863 (2017).10.1137/16M1075454 ↗
40. Faber, F. A. et al. Prediction errors of molecular machine learning models lower than hybrid dft error. J. Chem. Theory Comput. 13, 5255–5264 (2017).10.1021/acs.jctc.7b00577 ↗
41. Ohara, T. et al. Acrylic acid and derivatives. Ullmann’s Encyclopedia of Industrial Chemistry 1–21 (Wiley Online Library, 2020).10.1002/14356007.a01_161.pub4 ↗
42. Schulze-Topphoff, U. et al. Dimethyl fumarate treatment induces adaptive and innate immune modulation independent of Nrf2. PNAS 113, 4777–4782 (2016).10.1073/pnas.1603907113 ↗
43. Mantero, V. et al. Covid 19 in dimethyl fumarate treated patients with multiple sclerosis. J. Neurol. 268, 2023–2025 (2021).10.1007/s00415-020-10015-1 ↗
44. Jain, S. & Wallace, B. C. Wallace. Attention is not Explanation. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume1 (Long and Short Papers), 3543–3556, Association for Computational Linguistics, Minneapolis, Minnesota.
45. Wiegreffe, S. & Pinter, Y. Attention is not not Explanation. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), 11–20, Association for Computational Linguistics, Hong Kong, China.10.18653/v1/D19-1002 ↗
46. Grimsley, C., Mayfield, E. & R.S. Bursten, J. Why attention is not explanation: Surgical intervention and causal reasoning about neural models. LREC 12, 1780–1790 (2020).
47. O’Keeffe, M. A proposed rigorous definition of coordination number. Acta Crystallogr. A: Found. Adv. 35, 772–775 (1979).10.1107/S0567739479001765 ↗
48. Pham, T. L. et al. Machine learning reveals orbital interaction in materials. Sci. Technol. Adv. Mater. 18, 756 (2017).10.1080/14686996.2017.1378060 ↗
49. Glorot, X. & Bengio, Y. Understanding the difficulty of training deep feedforward neural networks. PMLR 9, 249–256 (2010).
50. Kingma, D. & Ba, J. Adam: A Method for Stochastic Optimization. In Proceedings of the 3rd International Conference on Learning Representations (ICLR 2015), San Diego, CA, USA, 2015.
51. Chi, D. H. et al. Electronic structures of Pt clusters adsorbed on (5,5) single wall carbon nanotube. Chem. Phys. Lett. 432, 213–217 (2006).10.1016/j.cplett.2006.10.063 ↗
52. Dam, H. C. et al. Substrate-mediated interactions of Pt atoms adsorbed on single-wall carbon nanotubes: Density functional calculations. Phys. Rev. B 79, 115426 (2009).10.1103/PhysRevB.79.115426 ↗
53. Vu, T. S. Python implementation of Self-Consistent Atention-based Neural Network - SCANN. GitHub, https://github.com/sinhvt3421/scann--material (2023).