journal article·2025-12-17Open access

An efficient forgetting-aware fine-tuning framework for pretrained universal machine-learning interatomic potentials

Jisu Kim, Jiho Lee, Sangmin Oh, Yutack Park, Seungwoo Hwang, Seungwu Han, Sungwoo Kang, Youngho Kang

npj Computational Materials· Volume 12· Issue 1

Abstract

Pretrained universal machine-learning interatomic potentials (MLIPs) have revolutionized computational materials science by enabling rapid atomistic simulations as efficient alternatives to ab initio methods. Fine-tuning pretrained MLIPs offers a practical approach to improving accuracy for materials and properties where predictive performance is insufficient. However, this approach often induces catastrophic forgetting, undermining the generalizability that is a key advantage of pretrained MLIPs. Herein, we propose reEWC, an advanced fine-tuning strategy that integrates Experience Replay and Elastic Weight Consolidation (EWC) to effectively balance forgetting prevention with fine-tuning efficiency. Using Li 6 PS 5 Cl (LPSC), a sulfide-based Li solid-state electrolyte, as a fine-tuning target, we show that reEWC significantly improves the accuracy of a pretrained MLIP, resolving well-known issues of potential energy surface softening and overestimated Li diffusivities. Moreover, reEWC preserves the generalizability of the pretrained MLIP and enables knowledge transfer to chemically distinct systems, including other sulfide, oxide, nitride, and halide electrolytes. Compared to Experience Replay and EWC used individually, reEWC delivers clear synergistic benefits, mitigating their respective limitations while maintaining computational efficiency. These results establish reEWC as a robust and effective solution for continual learning in MLIPs, enabling universal models that can advance materials research through large-scale, high-throughput simulations across diverse chemistries.
Source: openalex_inverted_index · Quality: baseline

Research context

OpenAlex topics: Machine Learning in Materials Science · Inorganic Chemistry and Materials · Advanced Battery Materials and Technologies

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

References (80)

Show source-provided references
1. Deringer, V. L., Caro, M. A. & Csányi, G. Machine learning interatomic potentials as emerging tools for materials science. Adv. Mater. 31, 1902765 (2019).10.1002/adma.201902765 ↗
2. Behler, J. & Parrinello, M. Generalized neural-network representation of high-dimensional potential-energy surfaces. Phys. Rev. Lett. 98, 146401 (2007).10.1103/PhysRevLett.98.146401 ↗
3. Bartók, A. P., Payne, M. C., Kondor, R. & Csányi, G. Gaussian approximation potentials: the accuracy of quantum mechanics, without the electrons. Phys. Rev. Lett. 104, 136403 (2010).10.1103/PhysRevLett.104.136403 ↗
4. Batzner, S. et al. E(3)-equivariant graph neural networks for data-efficient and accurate interatomic potentials. Nat. Commun. 13, 2453 (2022).10.1038/s41467-022-29939-5 ↗
5. Batatia, I., Kovacs, D. P., Simm, G., Ortner, C. & Csanyi, G. MACE: higher order equivariant message passing neural networks for fast and accurate force fields. In Proc. 36th Conference on Neural Information Processing System (NeurIPS, 2022).
6. Griesemer, S. D., Xia, Y. & Wolverton, C. Accelerating the prediction of stable materials with machine learning. Nat. Comput. Sci. 3, 934–945 (2023).10.1038/s43588-023-00536-w ↗
7. Pilania, G., Wang, C., Jiang, X., Rajasekaran, S. & Ramprasad, R. Accelerating materials property predictions using machine learning. Sci. Rep. 3, 2810 (2013).10.1038/srep02810 ↗
8. Hwang, S. et al. Stability and equilibrium structures of unknown ternary metal oxides explored by machine-learned potentials. J. Am. Chem. Soc. 145, 19378–19386 (2023).10.1021/jacs.3c06210 ↗
9. Tao, K. et al. Transformer enables ion transport behavior evolution and conductivity regulation for solid electrolyte. Energy Storage Mater. 71, 103555 (2024).10.1016/j.ensm.2024.103555 ↗
10. Tao, K., He, W., Chen, A., Han, Y. & Li, J. Harnessing lateral transfer learning for pioneering solid electrolyte interphase innovation. Energy Storage Mater. 75, 104034 (2025).10.1016/j.ensm.2025.104034 ↗
11. Tao, K. et al. Unlocking potential of pyrochlore in energy systems via soft voting ensemble learning. Small 20, 2402756 (2024).10.1002/smll.202402756 ↗
12. Wang, Z. et al. AlphaMat: a material informatics hub connecting data, features, models and applications. npj Comput. Mater. 9, 130 (2023).10.1038/s41524-023-01086-5 ↗
13. Lee, J. et al. Disorder-dependent Li diffusion in Li6PS5Cl investigated by machine-learning potential. ACS Appl. Mater. Interfaces 16, 46442–46453 (2024).10.1021/acsami.4c08865 ↗
14. Hong, C. et al. Atomistic simulation of HF etching process of amorphous Si3N4 using machine learning potential. ACS Appl. Mater. Interfaces 16, 48457–48469 (2024).10.1021/acsami.4c07949 ↗
15. Choi, J. M. et al. Accelerated computation of lattice thermal conductivity using neural network interatomic potentials. Comput. Mater. Sci. 211, 111472 (2022).10.1016/j.commatsci.2022.111472 ↗
16. Kang, S., Han, S. & Kang, Y. First-principles calculations of luminescence spectra of real-scale quantum dots. ACS Mater. Au 2, 103–109 (2021).10.1021/acsmaterialsau.1c00034 ↗
17. Jung, J. et al. Electrochemical degradation of Pt3Co nanoparticles investigated by off-lattice kinetic Monte Carlo simulations with machine-learned potentials. ACS Catal. 13, 16078–16087 (2023).10.1021/acscatal.3c04964 ↗
18. Jain, A. et al. Commentary: the materials project: a materials genome approach to accelerating materials innovation. APL Mater. 1, 011002 (2013).10.1063/1.4812323 ↗
19. Schmidt, J. et al. Improving machine-learning models in materials science through large datasets. Mater. Today Phys. 48, 101560 (2024).10.1016/j.mtphys.2024.101560 ↗
20. Barroso-Luque, L. et al. Open materials 2024 (OMat24) inorganic materials dataset and models. Preprint at arXiv:2410.12771 (2024).
21. Park, Y., Kim, J., Hwang, S. & Han, S. Scalable parallel algorithm for graph neural network interatomic potentials in molecular dynamics simulations. J. Chem. Theory Comput. 20, 4857–4868 (2024).10.1021/acs.jctc.4c00190 ↗
22. Deng, B. et al. CHGNet as a pretrained universal neural network potential for charge-informed atomistic modelling. Nat. Mach. Intell. 5, 1031–1041 (2023).10.1038/s42256-023-00716-3 ↗
23. Batatia, I. et al. A foundation model for atomistic materials chemistry. J. Chem. Phys. 163, 184110 (2025).10.1063/5.0297006 ↗
24. Chen, C. & Ong, S. P. A universal graph deep learning interatomic potential for the periodic table. Nat. Comput. Sci. 2, 718–728 (2022).10.1038/s43588-022-00349-3 ↗
25. Fu, X. et al. Learning smooth and expressive interatomic potentials for physical property prediction. In Proc. 42nd International Conference on Machine Learning 17875–17893 (PMLR, 2025).
26. Bochkarev, A., Lysogorskiy, Y. & Drautz, R. Graph atomic cluster expansion for semilocal interactions beyond equivariant message passing. Phys. Rev. X 14, 021036 (2024).
27. Kang, S. How graph neural network interatomic potentials extrapolate: role of the message-passing algorithm. J. Chem. Phys. 161 (2024).10.1063/5.0234287 ↗
28. Zhang, Y.-W. et al. Roadmap for the development of machine learning-based interatomic potentials. Modell. Simul. Mater. Sci. Eng. 33, 023301 (2025).10.1088/1361-651X/ad9d63 ↗
29. Riebesell, J. et al. A framework to evaluate machine learning crystal stability predictions. Nat. Mach. Intell. 7, 836–847 (2025).10.1038/s42256-025-01055-1 ↗
30. Ju, S. et al. Application of pretrained universal machine-learning interatomic potential for physicochemical simulation of liquid electrolytes in Li-ion batteries. Digit. Discov. 4, 1544–1559 (2025).10.1039/D5DD00025D ↗
31. Deng, B. et al. Systematic softening in universal machine learning interatomic potentials. npj Comput. Mater. 11, 1–9 (2025).10.1038/s41524-024-01500-6 ↗
32. Focassio, B., M. Freitas, L. P. & Schleder, G. R. Performance assessment of universal machine learning interatomic potentials: challenges and directions for materials’ surfaces. ACS Appl. Mater. Interface 17, 13111–13121 (2025).10.1021/acsami.4c03815 ↗
33. Pitfield, J. et al. Augmentation of universal potentials for broad applications. Phys. Rev. Lett. 134, 056201 (2025).10.1103/PhysRevLett.134.056201 ↗
34. Radova, M., Stark, W. G., Allen, C. S., Maurer, R. J. & Bartók, A. P. Fine-tuning foundation models of materials interatomic potentials with frozen transfer learning. npj. Comput. Mater. 11, 237 (2025).10.1038/s41524-025-01727-x ↗
35. Kaur, H. et al. Data-efficient fine-tuning of foundational models for first-principles quality sublimation enthalpies. Faraday Discuss. 256, 120–138 (2025).10.1039/D4FD00107A ↗
36. Clausen, C. M., Rossmeisl, J. & Ulissi, Z. W. Adapting OC20-trained equiformerV2 models for high-entropy materials. J. Phys. Chem. C. 128, 11190–11195 (2024).10.1021/acs.jpcc.4c01704 ↗
37. Kirkpatrick, J. et al. Overcoming catastrophic forgetting in neural networks. Proc. Natl. Acad. Sci. USA 114, 3521–3526 (2017).10.1073/pnas.1611835114 ↗
38. Zenke, F., Poole, B. & Ganguli, S. Continual learning through synaptic intelligence. In Proc. 34th International Conference on Machine Learning 3987–3995 (PMLR, 2017).
39. Rolnick, D., Ahuja, A., Schwarz, J., Lillicrap, T. & Wayne, G. Experience replay for continual learning. In Proc. 33rd Conference on Neural Information Processing System (NeurIPS, 2019).
40. Mnih, V. et al. Human-level control through deep reinforcement learning. Nature 518, 529–533 (2015).10.1038/nature14236 ↗
41. Rusu, A. A. et al. Progressive neural networks. Preprint at arXiv:1606.04671 (2016).
42. Yoon, J., Yang, E., Lee, J. & Hwang, S. J. Lifelong learning with dynamically expandable networks. In Proc. 6th International Conference on Learning Representations (ICLR, 2018).
43. Eckhoff, M. & Reiher, M. Lifelong machine learning potentials. J. Chem. Theory Comput. 19, 3509–3525 (2023).10.1021/acs.jctc.3c00279 ↗
44. De Lange, M. et al. A continual learning survey: Defying forgetting in classification tasks. IEEE Trans. Pattern Anal. Mach. Intell. 44, 3366–3385 (2021).
45. Eckhoff, M. & Reiher, M. CoRe optimizer: an all-in-one solution for machine learning. Mach. Learn. Sci. Technol. 5, 015018 (2024).10.1088/2632-2153/ad1f76 ↗
46. Wang, R. et al. A pre-trained deep potential model for sulfide solid electrolytes with broad coverage and high accuracy. npj Comput. Mater. 11, 266 (2025).10.1038/s41524-025-01764-6 ↗
47. Zhao, Q., Stalin, S., Zhao, C.-Z. & Archer, L. A. Designing solid-state electrolytes for safe, energy-dense batteries. Nat. Rev. Mater. 5, 229–252 (2020).10.1038/s41578-019-0165-5 ↗
48. Bachman, J. C. et al. Inorganic solid-state electrolytes for lithium batteries: mechanisms and properties governing ion conduction. Chem. Rev. 116, 140–162 (2016).10.1021/acs.chemrev.5b00563 ↗
49. Manthiram, A., Yu, X. & Wang, S. Lithium battery chemistries enabled by solid-state electrolytes. Nat. Rev. Mater. 2, 1–16 (2017).10.1038/natrevmats.2016.103 ↗
50. Dohare, S. et al. Loss of plasticity in deep continual learning. Nature 632, 768–774 (2024).10.1038/s41586-024-07711-7 ↗
51. Ritter, H., Botev, A. & Barber, D. Online structured Laplace approximations for overcoming catastrophic forgetting. In Proc. Conference on Neural Information Processing Systems (NeurIPS, 2018).
52. Lee, S., Ha, J., Zhang, D. & Kim, G. A neural Dirichlet process mixture model for task-free continual learning. In Proc. 8th International Conference on Learning Representations (ICLR, 2020).
53. Yin, D., Farajtabar, M., Li, A., Levine, N. & Mott, A. Optimization and generalization of regularization-based continual learning: a loss approximation viewpoint. Preprint at arXiv:2006.10974 (2021).
54. MDIL-SNU. SevenNet GitHub Repository (2024). https://github.com/MDIL-SNU/SevenNet.
55. Zhou, L., Minafra, N., Zeier, W. G. & Nazar, L. F. Innovative approaches to Li-argyrodite solid electrolytes for all-solid-state lithium batteries. Acc. Chem. Res. 54, 2717–2728 (2021).10.1021/acs.accounts.0c00874 ↗
56. Yu, C. et al. Facile synthesis toward the optimal structure-conductivity characteristics of the argyrodite Li6PS5Cl solid-state electrolyte. ACS Appl. Mater. Interfaces 10, 33296–33306 (2018).10.1021/acsami.8b07476 ↗
57. Ding, N. et al. Parameter-efficient fine-tuning of large-scale pre-trained language models. Nat. Mach. Intell. 5, 220–235 (2023).10.1038/s42256-023-00626-4 ↗
58. An, H., Kim, J., Kim, G. & Han, S. Atomistic simulation of HF diffusion on ammonium fluorosilicate surface using neural network potential. Modell. Simul. Mater. Sci. Eng. (2025).10.1088/1361-651X/add283 ↗
59. Li, H. et al. Rethinking the hyperparameters for fine-tuning. In Proc. 8th International Conference on Learning Representations (ICLR, 2020).
60. Kolesnikov, A. et al. Big transfer (BiT): General visual representation learning. In Proc. European Conference on Computer Vision, 491–507 (IEEE, 2020).10.1007/978-3-030-58558-7_29 ↗
61. Wortsman, M. et al. Robust fine-tuning of zero-shot models. In Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition 7959–7971 (IEEE, 2022).10.1109/CVPR52688.2022.00780 ↗
62. Murugan, R., Thangadurai, V. & Weppner, W. Fast lithium ion conduction in garnet-type Li7La3Zr2O12. Angew. Chem. Int. Ed. 46, 7778 (2007).10.1002/anie.200701144 ↗
63. He, X., Zhu, Y., Epstein, A. & Mo, Y. Statistical variances of diffusional properties from ab initio molecular dynamics simulations. npj Comput. Mater. 4, 18 (2018).10.1038/s41524-018-0074-y ↗
64. Kamaya, N. et al. A lithium superionic conductor. Nat. Mater. 10, 682–686 (2011).10.1038/nmat3066 ↗
65. Seino, Y., Ota, T., Takada, K., Hayashi, A. & Tatsumisago, M. A sulphide lithium super ion conductor is superior to liquid ion conductors for use in rechargeable batteries. Energy Environ. Sci. 7, 627–631 (2014).10.1039/C3EE41655K ↗
66. Asano, T. et al. Solid halide electrolytes with high lithium-ion conductivity for application in 4 V class bulk-type all-solid-state batteries. Adv. Mater. 30, 1803075 (2018).10.1002/adma.201803075 ↗
67. Li, W. et al. Li+ ion conductivity and diffusion mechanism in α − Li3N and β − Li3N. Energy Environ. Sci. 3, 1524–1530 (2010)..10.1039/c0ee00052c ↗
68. Miara, L. J. et al. Li-ion conductivity in Li9S3N. J. Mater. Chem. A 3, 20338–20344 (2015).10.1039/C5TA05432J ↗
69. Matsuo, M. & Orimo, S.-i Lithium fast-ionic conduction in complex hydrides: review and prospects. Adv. Energy Mater. 1, 161–172 (2011).10.1002/aenm.201000012 ↗
70. Yoo, D. et al. Atomic energy mapping of neural network potential. Phys. Rev. Mater. 3, 093802 (2019).10.1103/PhysRevMaterials.3.093802 ↗
71. Kim, J. et al. Data-efficient multifidelity training for high-fidelity machine learning interatomic potentials. J. Am. Chem. Soc. 147, 1042–1054 (2025).10.1021/jacs.4c14455 ↗
72. Kraft, M. A. et al. Influence of lattice polarizability on the ionic conductivity in the lithium superionic argyrodites Li6PS5X (X=Cl, Br, I). J. Am. Chem. Soc. 139, 10909–10918 (2017).10.1021/jacs.7b06327 ↗
73. Evans, D. J. & Holian, B. L. The nosè–Hoover thermostat. J. Chem. Phys. 83, 4069–4074 (1985).10.1063/1.449071 ↗
74. Kresse, G. & Joubert, D. From ultrasoft pseudopotentials to the projector augmented-wave method. Phys. Rev. B 59, 1758–1775 (1999).10.1103/PhysRevB.59.1758 ↗
75. Blöchl, P. E. Projector augmented-wave method. Phys. Rev. B 50, 17953–17979 (1994).10.1103/PhysRevB.50.17953 ↗
76. Perdew, J. P., Burke, K. & Ernzerhof, M. Generalized gradient approximation made simple. Phys. Rev. Lett. 77, 3865 (1996).10.1103/PhysRevLett.77.3865 ↗
77. Larsen, A. H. et al. The atomic simulation environment-a Python library for working with atoms. J. Phys. Condens. Matter 29, 273002 (2017).10.1088/1361-648X/aa680e ↗
78. Thompson, A. P. et al. LAMMPS–a flexible simulation tool for particle-based materials modeling at the atomic, meso, and continuum scales. Comput. Phys. Commun. 271, 108171 (2022).10.1016/j.cpc.2021.108171 ↗
79. Hogrefe, K. et al. Opening diffusion pathways through site disorder: the interplay of local structure and ion dynamics in the solid electrolyte Li6+xP1−xGexS5I as probed by neutron diffraction and NMR. J. Am. Chem. Soc. 144, 1795–1812 (2022).10.1021/jacs.1c11571 ↗
80. Friauf, R. J. Correlation effects for diffusion in ionic crystals. J. Appl. Phys. 33, 494–505 (1962).10.1063/1.1777148 ↗