Deep learning framework for material design space exploration using active transfer learning and data augmentation
Source abstract
Abstract
Abstract Neural network-based generative models have been actively investigated as an inverse design method for finding novel materials in a vast design space. However, the applicability of conventional generative models is limited because they cannot access data outside the range of training sets. Advanced generative models that were devised to overcome the limitation also suffer from the weak predictive power on the unseen domain. In this study, we propose a deep neural network-based forward design approach that enables an efficient search for superior materials far beyond the domain of the initial training set. This approach compensates for the weak predictive power of neural networks on an unseen domain through gradual updates of the neural network with active transfer learning and data augmentation methods. We demonstrate the potential of our framework with a grid composite optimization problem that has an astronomical number of possible design configurations. Results show that our proposed framework can provide excellent designs close to the global optima, even with the addition of a very small dataset corresponding to less than 0.5% of the initial training dataset size.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Machine Learning in Materials Science · Topology Optimization in Engineering · Advanced Multi-Objective Optimization Algorithms
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
References (60)
Show source-provided references
1. Libonati, F., Gu, G. X., Qin, Z., Vergani, L. & Buehler, M. J. Bone‐inspired materials by design: toughness amplification observed using 3D printing and testing. Adv. Eng. Mater. 18, 1354–1363 (2016).10.1002/adem.201600143 ↗
2. Sigmund, O. & Torquato, S. Design of materials with extreme thermal expansion using a three-phase topology optimization method. J. Mech. Phys. Solids 45, 1037–1067 (1997).10.1016/S0022-5096(96)00114-7 ↗
3. Sigmund, O. & Torquato, S. Design of smart composite materials using topology optimization. Smart Mater. Struct. 8, 365 (1999).10.1088/0964-1726/8/3/308 ↗
4. Kim, Y., Kim, Y., Libonati, F. & Ryu, S. Designing tough isotropic structural composite using computation, 3D printing and testing. Compos. Part B: Eng. 167, 736–745 (2019).10.1016/j.compositesb.2019.03.039 ↗
5. Guan, A.-Y., Liu, C.-L., Sun, X.-F., Xie, Y. & Wang, M.-A. Discovery of pyridine-based agrochemicals by using intermediate derivatization methods. Bioorg. Med. Chem. 24, 342–353 (2016).10.1016/j.bmc.2015.09.031 ↗
6. Zhu, H. et al. Discovery of TaFeSb-based half-Heuslers with high thermoelectric performance. Nat. Commun. 10, 1–8 (2019).10.1038/s41467-018-07882-8 ↗
7. Carraturo, M. et al. Graded-material design based on phase-field and topology optimization. Comput. Mech. 64, 1589–1600 (2019).10.1007/s00466-019-01736-w ↗
8. Chen, C.-T., Chrzan, D. C. & Gu, G. X. Nano-topology optimization for materials design with atom-by-atom control. Nat. Commun. 11, 1–9 (2020).10.1038/s41467-019-13993-7 ↗
9. Holder, A. M. et al. Novel phase diagram behavior and materials design in heterostructural semiconductor alloys. Sci. Adv. 3, e1700270 (2017).10.1126/sciadv.1700270 ↗
10. Kim, Y., Jeong, H., Gu, G. X. & Ryu, S. A three-dimensional fracture pattern diagram of staggered platelet structures. Compos. Struct. 220, 769–775 (2019).10.1016/j.compstruct.2019.04.017 ↗
11. Radman, A., Huang, X. & Xie, Y. Topological design of microstructures of multi-phase materials for maximum stiffness or thermal conductivity. Comput. Mater. Sci. 91, 266–273 (2014).10.1016/j.commatsci.2014.04.064 ↗
12. de Kruijf, N., Zhou, S., Li, Q. & Mai, Y.-W. Topological design of structures and composite materials with multiobjectives. Int. J. Solids Struct. 44, 7092–7109 (2007).10.1016/j.ijsolstr.2007.03.028 ↗
13. Osanov, M. & Guest, J. K. Topology optimization for architected materials design. Annu. Rev. Mater. Res. 46, 211–233 (2016).10.1146/annurev-matsci-070115-031826 ↗
14. Pruksawan, S., Lambard, G., Samitsu, S., Sodeyama, K. & Naito, M. Prediction and optimization of epoxy adhesive strength from a small dataset through active learning. Sci. Technol. Adv. Mater. 20, 1010–1021 (2019).10.1080/14686996.2019.1673670 ↗
15. Lavecchia, A. Machine-learning approaches in drug discovery: methods and applications. Drug Discov. Today 20, 318–331 (2015).10.1016/j.drudis.2014.10.012 ↗
16. Butler, K. T., Davies, D. W., Cartwright, H., Isayev, O. & Walsh, A. Machine learning for molecular and materials science. Nature 559, 547–555 (2018).10.1038/s41586-018-0337-2 ↗
17. Liu, X., Athanasiou, C. E., Padture, N. P., Sheldon, B. W. & Gao, H. A machine learning approach to fracture mechanics problems. Acta Mater. 190, 105–112 (2020).10.1016/j.actamat.2020.03.016 ↗
18. Hanakata, P. Z., Cubuk, E. D., Campbell, D. K. & Park, H. S. Forward and inverse design of kirigami via supervised autoencoder. Phys. Rev. Res. 2, 042006 (2020).10.1103/PhysRevResearch.2.042006 ↗
19. Pilania, G., Wang, C., Jiang, X., Rajasekaran, S. & Ramprasad, R. Accelerating materials property predictions using machine learning. Sci. Rep. 3, 1–6 (2013).10.1038/srep02810 ↗
20. Ramprasad, R., Batra, R., Pilania, G., Mannodi-Kanakkithodi, A. & Kim, C. Machine learning in materials informatics: recent applications and prospects. npj Comput. Mater. 3, 1–13 (2017).10.1038/s41524-017-0056-5 ↗
21. Chen, C. T. & Gu, G. X. Effect of constituent materials on composite performance: exploring design strategies via machine learning. Adv. Theory Simul. 2, 1900056 (2019).10.1002/adts.201900056 ↗
22. Gu, G. X., Chen, C.-T., Richmond, D. J. & Buehler, M. J. Bioinspired hierarchical composite design using machine learning: simulation, additive manufacturing, and experiment. Mater. Horiz. 5, 939–945 (2018).10.1039/C8MH00653A ↗
23. Gu, G. X., Chen, C.-T. & Buehler, M. J. De novo composite design based on machine learning algorithm. Extrem. Mech. Lett. 18, 19–28 (2018).10.1016/j.eml.2017.10.001 ↗
24. Kim, Y., Yang, C., Kim, Y., Gu, G. X. & Ryu, S. Designing an adhesive pillar shape with deep learning-based optimization. ACS Appl. Mater. Inter. 12, 24458–24465 (2020).10.1021/acsami.0c04123 ↗
25. Mansouri Tehrani, A. et al. Machine learning directed search for ultraincompressible, superhard materials. J. Am. Chem. Soc. 140, 9844–9853 (2018).10.1021/jacs.8b02717 ↗
26. Noh, J., Gu, G. H., Kim, S. & Jung, Y. Machine-enabled inverse design of inorganic solid materials: promises and challenges. Chem. Sci. 11, 4871–4881 (2020).10.1039/D0SC00594K ↗
27. Janet, J. P., Chan, L. & Kulik, H. J. Accelerating chemical discovery with machine learning: simulated evolution of spin crossover complexes with an artificial neural network. J. Phys. Chem. Lett. 9, 1064–1071 (2018).10.1021/acs.jpclett.8b00170 ↗
28. Chen, W.-C., Schmidt, J. N., Yan, D., Vohra, Y. K. & Chen, C.-C. Machine learning and evolutionary prediction of superhard BCN compounds. npj Comput. Mater. 7, 1–8 (2021).10.1038/s41524-021-00585-7 ↗
29. Avery, P. et al. Predicting superhard materials via a machine learning informed evolutionary structure search. npj Comput. Mater. 5, 1–11 (2019).10.1038/s41524-019-0226-8 ↗
30. Simpson, T. W., Mauery, T. M., Korte, J. J. & Mistree, F. Kriging models for global approximation in simulation-based multidisciplinary design optimization. AIAA J. 39, 2233–2241 (2001).10.2514/2.1234 ↗
31. Jeong, S., Murayama, M. & Yamamoto, K. Efficient optimization design method using kriging model. J. Aircr. 42, 413–420 (2005).10.2514/1.6386 ↗
32. Li, Z., Gerdroodbary, M. B., Valipour, P., Moradi, R. & Babazadeh, H. The optimization via response surface method for micro hydrogen gas actuator. Int. J. Hydrog. Energy 44, 31633–31643 (2019).10.1016/j.ijhydene.2019.10.015 ↗
33. Peurifoy, J. et al. Nanophotonic particle simulation and inverse design using artificial neural networks. Sci. Adv. 4, eaar4206 (2018).10.1126/sciadv.aar4206 ↗
34. Sanchez-Lengeling, B. & Aspuru-Guzik, A. Inverse molecular design using machine learning: generative models for matter engineering. Science 361, 360–365 (2018).10.1126/science.aat2663 ↗
35. Chen, C. T. & Gu, G. X. Generative deep neural networks for inverse materials design using backpropagation and active learning. Adv. Sci. 7, 1902607 (2020).10.1002/advs.201902607 ↗
36. Kim, B., Lee, S. & Kim, J. Inverse design of porous materials using artificial neural networks. Sci. Adv. 6, eaax9324 (2020).10.1126/sciadv.aax9324 ↗
37. Hsu, Y.-C., Yu, C.-H. & Buehler, M. J. Tuning mechanical properties in polycrystalline solids using a deep generative framework. Adv. Eng. Mater. 23, 20201339 (2021).10.1002/adem.202001339 ↗
38. Lim, J., Ryu, S., Kim, J. W. & Kim, W. Y. Molecular generative model based on conditional variational autoencoder for de novo molecular design. J. Cheminformatics 10, 1–9 (2018).10.1186/s13321-018-0286-7 ↗
39. Kollmann, H. T., Abueidda, D. W., Koric, S., Guleryuz, E. & Sobh, N. A. Deep learning for topology optimization of 2D metamaterials. Mater. Des. 196, 109098 (2020).10.1016/j.matdes.2020.109098 ↗
40. Kim, K. et al. Deep-learning-based inverse design model for intelligent discovery of organic molecules. npj Comput. Mater. 4, 1–7 (2018).10.1038/s41524-018-0128-1 ↗
41. Noh, J. et al. Inverse design of solid-state materials via a continuous representation. Matter 1, 1370–1384 (2019).10.1016/j.matt.2019.08.017 ↗
42. Yao, Z. et al. Inverse design of nanoporous crystalline reticular materials with deep generative models. Nat. Mach. Intell. 3, 76–86 (2021).10.1038/s42256-020-00271-1 ↗
43. Marcus, G. Deep learning: A critical appraisal. Preprint at https://arxiv.org/abs/1801.00631 (2018).
44. Martius, G. & Lampert, C. H. Extrapolation and learning equations. Preprint at https://arxiv.org/abs/1610.02995 (2016).
45. Barnard, E. & Wessels, L. Extrapolation and interpolation in neural network classifiers. IEEE Control Syst. Mag. 12, 50–53 (1992).10.1109/37.158898 ↗
46. Mitchell, J., Minervini, P., Stenetorp, P. & Riedel, S. Extrapolation in NLP. Preprint at https://arxiv.org/abs/1805.06648 (2018).10.18653/v1/W18-1005 ↗
47. Greydanus, S., Dzamba, M. & Yosinski, J. Hamiltonian neural networks. Adv. Neural Inf. Process. Syst. 32, 15379–15389 (2019).
48. Long, Z., Lu, Y. & Dong, B. PDE-Net 2.0: learning PDEs from data with a numeric-symbolic hybrid deep network. J. Comput. Phys. 399, 108925 (2019).10.1016/j.jcp.2019.108925 ↗
49. Kloss, A., Schaal, S. & Bohg, J. Combining learned and analytical models for predicting action effects. Preprint at https://arxiv.org/abs/1710.04102 (2017).
50. Lookman, T., Balachandran, P. V., Xue, D. & Yuan, R. Active learning in materials science with emphasis on adaptive sampling using uncertainties for targeted design. npj Comput. Mater. 5, 1–17 (2019).10.1038/s41524-019-0153-8 ↗
51. Lopedoto, E. & Weyde, T. In International Conference on Innovative Techniques and Applications of Artificial Intelligence. 159-165 (Springer).
52. Xu, K. et al. How neural networks extrapolate: from feedforward to graph neural networks. Preprint at https://arxiv.org/abs/2009.11848 (2020).
53. Hooker, G. & Rosset, S. Prediction-based regularization using data augmented regression. Stat. Comput. 22, 237–249 (2012).10.1007/s11222-010-9220-1 ↗
54. He, K., Zhang, X., Ren, S. & Sun, J. In European conference on computer vision. 630–645 (Springer, 2016).10.1007/978-3-319-46493-0_38 ↗
55. He, K., Zhang, X., Ren, S. & Sun, J. In Proceedings of the IEEE conference on computer vision and pattern recognition. 770–778 (Springer, 2016).
56. Mozaffar, M. et al. Deep learning predicts path-dependent plasticity. Proc. Natl Acad. Sci. 116, 26414–26420 (2019).10.1073/pnas.1911815116 ↗
57. Abueidda, D. W., Koric, S., Sobh, N. A. & Sehitoglu, H. Deep learning for plasticity and thermo-viscoplasticity. Int. J. Plasticity 136, 102852 (2021).10.1016/j.ijplas.2020.102852 ↗
58. Li, L. & Ma, C. In IOP Conference Series: Earth and Environmental Science. 022095 (2019).10.1088/1755-1315/252/2/022095 ↗
59. Campagna, R. & Perracchione, E. Data-driven extrapolation via feature augmentation based on variably scaled thin plates plines. J. Sci. Comput. 88, 1–15 (2021).10.1007/s10915-021-01526-8 ↗
60. Jeong, H., Signetti, S., Han, T.-S. & Ryu, S. Phase field modeling of crack propagation under combined shear and tensile loading with hybrid formulation. Comput. Mater. Sci. 155, 483–492 (2018).10.1016/j.commatsci.2018.09.021 ↗