journal article·2025-01-09Open access

Accelerating superconductor discovery through tempered deep learning of the electron-phonon spectral function

Jason B. Gibson, Ajinkya C. Hire, Philip M. Dee, Oscar Barrera, Benjamin Geisler, Peter J. Hirschfeld, Richard G. Hennig

npj Computational Materials· Volume 11· Issue 1

Abstract

Integrating deep learning with the search for new electron-phonon superconductors represents a burgeoning field of research, where the primary challenge lies in the computational intensity of calculating the electron-phonon spectral function, α2F(ω), the essential ingredient of Midgal-Eliashberg theory of superconductivity. To overcome this challenge, we adopt a two-step approach. First, we compute α2F(ω) for 818 dynamically stable materials. We then train a deep-learning model to predict α2F(ω), using a training strategy tailored for limited data to temper the model’s overfitting, enhancing predictions. Specifically, we train a Bootstrapped Ensemble of Tempered Equivariant graph neural NETworks (BETE-NET), obtaining an MAE of 0.21, 45 K, and 43 K for the moments derived from α2F(ω): λ, $${\omega }\log $$ , and ω2, respectively, yielding an MAE of 2.5 K for the critical temperature, Tc. Further, we incorporate domain knowledge of the site-projected phonon density of states to impose inductive bias into the model’s node attributes and enhance predictions. This methodological innovation decreases the MAE to 0.18, 29 K, and 28 K, respectively, yielding an MAE of 2.1 K for Tc. We illustrate the practical application of our model in high-throughput screening for high-Tc materials. The model demonstrates an average precision nearly five times higher than random screening, highlighting the potential of ML in accelerating superconductor discovery. BETE-NET accelerates the search for high-Tc superconductors while setting a precedent for applying ML in materials discovery, particularly when data is limited.
Source: openalex_inverted_index · Quality: baseline

Research context

OpenAlex topics: Machine Learning in Materials Science · High-pressure geophysics and materials · Nuclear Physics and Applications

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

References (79)

Show source-provided references
1. Materials Genome Initiative. https://obamawhitehouse.archives.gov/mgi Accessed: 2024-01-08 (2014).
2. Yao, C. & Ma, Y. Superconducting materials: Challenges and opportunities for large-scale applications. iScience 24, 102541 (2021).10.1016/j.isci.2021.102541 ↗
3. Belkin, M., Hsu, D., Ma, S. & Mandal, S. Reconciling modern machine-learning practice and the classical bias-variance trade-off. Proc. Natl Acad. Sci. 116, 15849–15854 (2019).10.1073/pnas.1903070116 ↗
4. Nakkiran, P. et al. Deep double descent: where bigger models and more data hurt*. J. Stat. Mech.: Theory Exp. 2021, 124003 (2021).10.1088/1742-5468/ac3a74 ↗
5. Wu, S. Q. et al. An adaptive genetic algorithm for crystal structure prediction. J. Phys.: Condens. Matter 26, 035402 (2013).
6. Jennings, P. C., Lysgaard, S., Hummelshøj, J. S., Vegge, T. & Bligaard, T. Genetic algorithms for computational materials discovery accelerated by machine learning. npj Comput. Mater. 5, 46 (2019).10.1038/s41524-019-0181-4 ↗
7. Sahinovic, A. & Geisler, B. Active learning and element-embedding approach in neural networks for infinite-layer versus perovskite oxides. Phys. Rev. Res. 3, L042022 (2021).10.1103/PhysRevResearch.3.L042022 ↗
8. Sahinovic, A. & Geisler, B. Quantifying transfer learning synergies in infinite-layer and perovskite nitrides, oxides, and fluorides. J. Phys.: Condens. Matter 34, 214003 (2022).
9. Charraud, J.-B., Geneste, G., Torrent, M. & Maillet, J.-B. Machine learning accelerated random structure searching: Application to yttrium superhydrides. The Journal of Chemical Physics156 (2022).10.1063/5.0085173 ↗
10. Li, C.-N., Liang, H.-P., Zhang, X., Lin, Z. & Wei, S.-H. Graph deep learning accelerated efficient crystal structure search and feature extraction. npj Comput. Mater. 9, 176 (2023).10.1038/s41524-023-01122-4 ↗
11. Gibson, J., Hire, A. & Hennig, R. G. Data-augmentation for graph neural network learning of the relaxed energies of unrelaxed structures. npj Comput. Mater. 8, 211 (2022).10.1038/s41524-022-00891-8 ↗
12. Xie, T. & Grossman, J. C. Crystal graph convolutional neural networks for an accurate and interpretable prediction of material properties. Phys. Rev. Lett. 120, 145301 (2018).10.1103/PhysRevLett.120.145301 ↗
13. Ward, L., Agrawal, A., Choudhary, A. & Wolverton, C. A general-purpose machine learning framework for predicting properties of inorganic materials. npj Comput. Mater. 2, 16028 (2016).10.1038/npjcompumats.2016.28 ↗
14. Duan, D. et al. Pressure-induced metallization of dense (H2S)2H2 with high-Tc superconductivity. Sci. Rep. 4, 6968 (2014).10.1038/srep06968 ↗
15. Liu, H., Naumov, I. I., Hoffmann, R., Ashcroft, N. W. & Hemley, R. J. Potential high-Tc superconducting lanthanum and yttrium hydrides at high pressure. Proc. Natl. Acad. Sci. 114, 6990–6995 (2017).10.1073/pnas.1704505114 ↗
16. Peng, F. et al. Hydrogen clathrate structures in rare earth hydrides at high pressures: Possible route to room-temperature superconductivity. Phys. Rev. Lett. 119, 107001 (2017).10.1103/PhysRevLett.119.107001 ↗
17. Ashcroft, N. W. Hydrogen dominant metallic alloys: High temperature superconductors? Phys. Rev. Lett. 92, 187002 (2004).10.1103/PhysRevLett.92.187002 ↗
18. Drozdov, A. P., Eremets, M. I., Troyan, I. A., Ksenofontov, V. & Shylin, S. I. Conventional superconductivity at 203 kelvin at high pressures in the sulfur hydride system. Nature 525, 73 (2015).10.1038/nature14964 ↗
19. Somayazulu, M. et al. Evidence for superconductivity above 260 K in lanthanum superhydride at megabar pressures. Phys. Rev. Lett. 122, 027001 (2019).10.1103/PhysRevLett.122.027001 ↗
20. Drozdov, A. P. et al. Superconductivity at 250 K in lanthanum hydride under high pressures. Nature 569, 528–531 (2019).10.1038/s41586-019-1201-8 ↗
21. Allen, P. B. & Dynes, R. C. Transition temperature of strong-coupled superconductors reanalyzed. Phys. Rev. B 12, 905–922 (1975).10.1103/PhysRevB.12.905 ↗
22. Xie, S. R. et al. Machine learning of superconducting critical temperature from eliashberg theory. npj Comput. Mater. 8, 14 (2022).10.1038/s41524-021-00666-7 ↗
23. Migdal, A. B. Interaction between electrons and lattice vibrations in a normal metal. Sov. Phys. JETP 7, 996–1001 (1958).
24. Eliashberg, G. M. Interactions between electrons and lattice vibrations in a superconductor. Sov. Phys. JETP 11, 696–702 (1960).
25. Eliashberg, G. M. Temperature Green’s Function For Electrons In a Supercondutor. Sov. Phys. JETP 12, 1000–1002 (1961).
26. Jain, A. et al. Commentary: The materials project: A materials genome approach to accelerating materials innovation. APL Mater. 1, 011002 (2013).10.1063/1.4812323 ↗
27. Kirklin, S. et al. The open quantum materials database (OQMD): assessing the accuracy of DFT formation energies. npj Comput. Mater. 1, 1–15 (2015).10.1038/npjcompumats.2015.10 ↗
28. Curtarolo, S. et al. Aflowlib.org: A distributed materials properties repository from high-throughput ab initio calculations. Comput. Mater. Sci. 58, 227–235 (2012).10.1016/j.commatsci.2012.02.002 ↗
29. Choudhary, K. et al. The joint automated repository for various integrated simulations (JARVIS) for data-driven materials design. npj Comput. Mater. 6, 173 (2020).10.1038/s41524-020-00440-1 ↗
30. Zeng, S. et al. Atom table convolutional neural networks for an accurate prediction of compounds properties. npj Comput. Mater. 5, 84 (2019).10.1038/s41524-019-0223-y ↗
31. Li, S. et al. Critical temperature prediction of superconductors based on atomic vectors and deep learning. Symmetry 12, 262 (2020).10.3390/sym12020262 ↗
32. Roter, B. & Dordevic, S. Predicting new superconductors and their critical temperatures using machine learning. Phys. C: Superconduct. Appl. 575, 1353689 (2020).10.1016/j.physc.2020.1353689 ↗
33. Konno, T. et al. Deep learning model for finding new superconductors. Phys. Rev. B 103, 014509 (2021).10.1103/PhysRevB.103.014509 ↗
34. Kim, E. & Dordevic, S. V. ScGAN: A Generative Adversarial Network to Predict Hypothetical Superconductors. arXiv e-prints arXiv:2209.03444 2209.03444 (2022).
35. Stanev, V. et al. Machine learning modeling of superconducting critical temperature. npj Comput. Mater. 4, 29 (2018).10.1038/s41524-018-0085-8 ↗
36. National Institute for Materials Science. SuperCon Database. https://mdr.nims.go.jp/collections/5712mb227 (2011).
37. Hamidieh, K. A data-driven statistical model for predicting the critical temperature of a superconductor. Comput. Mater. Sci. 154, 346–354 (2018).10.1016/j.commatsci.2018.07.052 ↗
38. Sommer, T., Willa, R., Schmalian, J. & Friederich, P. 3DSC - a dataset of superconductors including crystal structures. Sci. Data 10, 816 (2023).10.1038/s41597-023-02721-y ↗
39. Hosono, H. et al. Exploration of new superconductors and functional materials, and fabrication of superconducting tapes and wires of iron pnictides. Sci. Technol. Adv. Mater. 16, 033503 (2015).10.1088/1468-6996/16/3/033503 ↗
40. Foppiano, L. et al. Automatic extraction of materials and properties from superconductors scientific literature. Sci. Technol. Adv. Mater.: Methods 3, 2153633 (2023).
41. Saha, S. et al. Mapping superconductivity in high-pressure hydrides: The superhydra project https://arxiv.org/abs/2205.02554 (2022).
42. Hoffmann, N. et al. Searching for ductile superconducting Heusler X2YZ compounds. npj Comput. Mater. 9, 138 (2023).10.1038/s41524-023-01084-7 ↗
43. Choudhary, K. & Garrity, K. Designing high-Tc superconductors with BCS-inspired screening, density functional theory, and deep-learning. npj Comput. Mater. 8, 244 (2022).10.1038/s41524-022-00933-1 ↗
44. Cerqueira, T. F. T., Sanna, A. & Marques, M. A. L. Sampling the materials space for conventional superconducting compounds. Adv. Mat. https://doi.org/10.1002/adma.202307085 (2023).10.1002/adma.202307085 ↗
45. F. T. Cerqueira, T., Sanna, A. & L. Marques, M. A. Sampling the materials space for conventional superconducting compounds https://archive.materialscloud.org/record/2023.163 (2023).10.1002/adma.202307085 ↗
46. Choudhary, K. & DeCost, B. Atomistic line graph neural network for improved materials property predictions. npj Comput. Mater. 7, 185 (2021).10.1038/s41524-021-00650-1 ↗
47. Chen, Z. et al. Direct prediction of phonon density of states with Euclidean neural networks. Adv. Sci. 8, 2004214 (2021).10.1002/advs.202004214 ↗
48. Allen, P. B. Electron-phonon effects in the infrared properties of metals. Phys. Rev. B 3, 305–320 (1971).10.1103/PhysRevB.3.305 ↗
49. Allen, P. B. New method for solving Boltzmann’s equation for electrons in metals. Phys. Rev. B 17, 3725–3734 (1978).10.1103/PhysRevB.17.3725 ↗
50. Kratzer, P., Rettig, L., Sklyadneva, I. Y., Chulkov, E. V. & Bovensiepen, U. Relaxation of photoexcited hot carriers beyond multitemperature models: General theory description verified by experiments on Pb/Si(111). Phys. Rev. Res. 4, 033218 (2022).10.1103/PhysRevResearch.4.033218 ↗
51. Mallinar, N. et al. Benign, tempered, or catastrophic: A taxonomy of overfitting 2207.06569 (2022).
52. Pickett, W. E. Colloquium : Room temperature superconductivity: The roles of theory and materials design. Rev. Mod. Phys. 95, 021001 (2023).10.1103/RevModPhys.95.021001 ↗
53. Frey, N. C. et al. Neural scaling of deep chemical models. Nat. Mach. Intell. 5, 1297–1305 (2023).10.1038/s42256-023-00740-3 ↗
54. Rackers, J. A., Tecot, L., Geiger, M. & Smidt, T. E. A recipe for cracking the quantum scaling limit with machine learned electron densities. Mach. Learn.: Sci. Technol. 4, 015027 (2023).
55. Owen, C. J. et al. Complexity of many-body interactions in transition metals via machine-learned force fields from the TM23 data set 2302.12993 (2023).10.1038/s41524-024-01264-z ↗
56. Efron, B. Bootstrap Methods: Another Look at the Jackknife. Ann. Stat. 7, 1 – 26 (1979).10.1214/aos/1176344552 ↗
57. Ba, J., Erdogdu, M., Suzuki, T., Wu, D. & Zhang, T. Generalization of two-layer neural networks: An asymptotic viewpoint. In International Conference on Learning Representations (2020).
58. Yang, Z., Yu, Y., You, C., Steinhardt, J. & Ma, Y. Rethinking bias-variance trade-off for generalization of neural networks 2002.11328 (2020).
59. Li, H., Xu, Z., Taylor, G., Studer, C. & Goldstein, T. Visualizing the loss landscape of neural nets 1712.09913 (2018).
60. Rubner, Y., Tomasi, C. & Guibas, L. A metric for distributions with applications to image databases. Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271) (1998).
61. Xie, S. R., Rupp, M. & Hennig, R. G. Ultra-fast interpretable machine-learning potentials. npj Comput. Mater. 9, 162 (2023).10.1038/s41524-023-01092-7 ↗
62. Chen, C. & Ong, S. P. A universal graph deep learning interatomic potential for the periodic table. Nat. Comput. Sci. 2, 718–728 (2022).10.1038/s43588-022-00349-3 ↗
63. Giannozzi, P. et al. QUANTUM ESPRESSO: a modular and open-source software project for quantum simulations of materials. J. Phys.: Condens. Matter 21, 395502 (2009).
64. Giannozzi, P. et al. Quantum espresso toward the exascale. J. Chem. Phys. 152, 154105 (2020).10.1063/5.0005082 ↗
65. Giannozzi, P. et al. Advanced capabilities for materials modelling with quantum ESPRESSO. J. Phys.: Condens. Matter 29, 465901 (2017).
66. Perdew, J. P. et al. Restoring the density-gradient expansion for exchange in solids and surfaces. Phys. Rev. Lett. 100, 136406 (2008).10.1103/PhysRevLett.100.136406 ↗
67. Wierzbowska, M., de Gironcoli, S. & Giannozzi, P. Origins of low- and high-pressure discontinuities of Tc in niobium https://arxiv.org/abs/cond-mat/0504077 (2005).
68. Hamann, D. R. Optimized norm-conserving Vanderbilt pseudopotentials. Phys. Rev. B 88, 085117 (2013).10.1103/PhysRevB.88.085117 ↗
69. Schlipf, M. & Gygi, F. Optimization algorithm for the generation of ONCV pseudopotentials. Computer Phys. Commun. 196, 36–44 (2015).10.1016/j.cpc.2015.05.011 ↗
70. SG15 ONCV Pseudopotential. http://www.quantum-simulation.org/potentials/sg15_oncv/ (2023).
71. van Setten, M. et al. The pseudodojo: Training and grading a 85 element optimized norm-conserving pseudopotential table. Computer Phys. Commun. 226, 39–54 (2018).10.1016/j.cpc.2018.01.012 ↗
72. Ivashchenko, V. I., Turchi, P. E. A. & Olifan, E. I. Phase stability and mechanical properties of niobium nitrides. Phys. Rev. B 82, 054109 (2010).10.1103/PhysRevB.82.054109 ↗
73. Babu, K. R. & Guo, G.-Y. Electron-phonon coupling, superconductivity, and nontrivial band topology in NbN polytypes. Phys. Rev. B 99, 104508 (2019).10.1103/PhysRevB.99.104508 ↗
74. Dee, P. M. et al. Diboride compounds doped with transition metals: A route to superconductivity through structure stabilization as well as defects. Phys. Rev. B 109, 104520 (2024).10.1103/PhysRevB.109.104520 ↗
75. Hire, A. C. et al. High critical field superconductivity at ambient pressure in MoB2 stabilized in the P6/mmm structure via Nb substitution. Phys. Rev. B 106, 174515 (2022).10.1103/PhysRevB.106.174515 ↗
76. Paszke, A. et al. Pytorch: An imperative style, high-performance deep learning library. In Advances in Neural Information Processing Systems 32, 8024–8035 (Curran Associates, Inc. 2019). http://papers.neurips.cc/paper/9015-pytorch-an-imperative-style-high-performance-deep-learning-library.pdf.
77. Weiler, M., Geiger, M., Welling, M., Boomsma, W. & Cohen, T. S. 3D steerable CNNs: Learning rotationally equivariant features in volumetric data. In Bengio, S. et al. (eds.) Advances in Neural Information Processing Systems, vol. 31 (Curran Associates, Inc., 2018). https://proceedings.neurips.cc/paper_files/paper/2018/file/488e4104520c6aab692863cc1dba45af-Paper.pdf.
78. Thomas, N. et al. Tensor field networks: Rotation- and translation-equivariant neural networks for 3d point clouds 1802.08219 (2018).
79. Miller, B. K., Geiger, M., Smidt, T. E. & Noé, F. Relevance of rotationally equivariant convolutions for predicting molecular properties 2008.08461 (2020).