journal article·2020-11

Advances in neural networks and potential for their application to steel metallurgy

J. L. Smith

Materials Science and Technology· Volume 36· Issue 17
Publisher page

Abstract

This review provides a timely exploration of several novel neural network (NN) architectures and learning methods, following a concise overview of the fundamentals of NNs and some important associated challenges. There are many benefits to using NNs, including deep learning models, in scientific research and, by understanding novel techniques better suited to certain applications, this benefit can be maximised. Finally, a few developed and emerging alternative learning paradigms are surveyed for their potential benefit to future research. The reviewed literature and accompanying discussion are of generic value well beyond steel metallurgy, and there is much to be gained from assessing methods used in other areas of materials science and further afield in order to apply them to steel metallurgy.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: Mineral Processing and Grinding · Metallurgical Processes and Thermodynamics · Non-Destructive Testing Techniques

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

Department of Materials Science & Metallurgy, University of Cambridge, Cambridge, UK

References (100+)

Show source-provided references
1. 10.1002/wcms.112510.1002/wcms.1125 ↗
2. 10.1557/mrc.2019.7310.1557/mrc.2019.73 ↗
3. 10.2355/isijinternational.39.96610.2355/isijinternational.39.966 ↗
4. 10.1002/sam.1001810.1002/sam.10018 ↗
5. 10.1145/306538610.1145/3065386 ↗
6. Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition. arXiv:14091556 [cs]. 2015 Apr. Available from: http://arxiv.org/abs/1409.1556
7. Ze H, Senior A, Schuster M. Statistical parametric speech synthesis using deep neural networks. 2013 IEEE International Conference on Acoustics, Speech and Signal Processing; 2013. p. 7962–7966. ISSN: 2379-190X.
8. 10.21437/Interspeech.2014-44310.21437/Interspeech.2014-443 ↗
9. 10.1109/ICASSP.2015.717881610.1109/ICASSP.2015.7178816 ↗
10. 10.1179/1743280413Y.000000002110.1179/1743280413Y.0000000021 ↗
11. Odejobi OA, Umoru LE. Applications of soft computing techniques in materials engineering: a review. p. 28.
12. 10.1007/s11831-019-09340-010.1007/s11831-019-09340-0 ↗
13. 10.1080/1042691080261233810.1080/10426910802612338 ↗
14. 10.1109/IJCNN.2016.772752210.1109/IJCNN.2016.7727522 ↗
15. 10.1109/IJCNN.2012.625246810.1109/IJCNN.2012.6252468 ↗
16. 10.1007/s40192-018-0117-810.1007/s40192-018-0117-8 ↗
17. 10.1371/journal.pone.021649310.1371/journal.pone.0216493 ↗
18. 10.21105/joss.0074710.21105/joss.00747 ↗
19. 10.1080/13640461.2003.1181953710.1080/13640461.2003.11819537 ↗
20. 10.1515/zna-2010-051210.1515/zna-2010-0512 ↗
21. 10.1007/BF0264688210.1007/BF02646882 ↗
22. Häggblad H , Oldenburg M. Modelling and simulation of metal powder die pressing with use of explicit time integration. Model Simul Mater Sci Eng. 1994 Jul;2(4):893–911. Available from: https://doi.org/10.10882/F0965-03932F22F42F00610.1088/0965-0393/2/4/006 ↗
23. 10.1016/j.matdes.2006.09.00310.1016/j.matdes.2006.09.003 ↗
24. 10.1088/0965-0393/16/2/02500510.1088/0965-0393/16/2/025005 ↗
25. 10.1007/s11661-019-05170-810.1007/s11661-019-05170-8 ↗
26. 10.1162/neco.1992.4.3.44810.1162/neco.1992.4.3.448 ↗
27. 10.1162/neco.1992.4.3.41510.1162/neco.1992.4.3.415 ↗
28. Blundell C, Cornebise J, Kavukcuoglu K, et al. Weight uncertainty in neural networks. arXiv:150505424 [cs, stat]. 2015 May. Available from: http://arxiv.org/abs/1505.05424
29. Pearce T, Leibfried F, Brintrup A, et al. Uncertainty in neural networks: approximately Bayesian ensembling. arXiv:181005546 [cs, stat]. 2019 Oct. Available from: http://arxiv.org/abs/1810.05546
30. 10.1002/jcc.2476410.1002/jcc.24764 ↗
31. 10.1109/TASE.2016.264077810.1109/TASE.2016.2640778 ↗
32. Marcus G. Deep learning: a critical appraisal. arXiv:180100631 [cs, stat]. 2018 Jan. Available from: http://arxiv.org/abs/1801.00631
33. Krizhevsky A, Sutskever I, Hinton GE. ImageNet classification with deep convolutional neural networks. Pereira F, Burges CJC, Bottou L, Weinberger KQ, editors, Advances in neural information processing systems 25. Curran Associates, Inc.; 2012. p. 1097–1105. Available from: http://papers.nips.cc/paper/4824-imagenet-classification-with-deep-convolutional-neural-networks.pdf
34. 10.1109/CVPR.2015.729896510.1109/CVPR.2015.7298965 ↗
35. 10.1109/CVPR.2016.20710.1109/CVPR.2016.207 ↗
36. 10.1109/FCCM.2012.3210.1109/FCCM.2012.32 ↗
37. 10.1109/TIE.2006.88879110.1109/TIE.2006.888791 ↗
38. 10.1021/ci034247210.1021/ci0342472 ↗
39. Srivastava N · J Machine Learn Res · 2014 · 15 · 1929
40. 10.1109/TNNLS.2013.229363710.1109/TNNLS.2013.2293637 ↗
41. Bornschein J, Visin F, Osindero S. Small data, big decisions: model selection in the small-data regime. p. 10.
42. 10.1109/TKDE.2009.19110.1109/TKDE.2009.191 ↗
43. 10.1109/TMI.2016.252816210.1109/TMI.2016.2528162 ↗
44. 10.1186/s40537-019-0197-010.1186/s40537-019-0197-0 ↗
45. 10.1109/DICTA.2016.779709110.1109/DICTA.2016.7797091 ↗
46. 10.1113/jphysiol.1962.sp00683710.1113/jphysiol.1962.sp006837 ↗
47. 10.1109/5.72679110.1109/5.726791 ↗
48. 10.1016/j.commatsci.2018.03.07410.1016/j.commatsci.2018.03.074 ↗
49. 10.3390/app924551010.3390/app9245510 ↗
50. Jordan MI. Attractor dynamics and parallelism in a connectionist sequential machine. Artificial neural networks: concept learning. IEEE Press; 1990. p. 112–127.
51. 10.1207/s15516709cog1402_110.1207/s15516709cog1402_1 ↗
52. 10.1109/IJCNN.1989.11872410.1109/IJCNN.1989.118724 ↗
53. 10.1162/neco.1989.1.3.37210.1162/neco.1989.1.3.372 ↗
54. 10.1162/neco.1997.9.8.173510.1162/neco.1997.9.8.1735 ↗
55. 10.3115/v1/D14-117910.3115/v1/D14-1179 ↗
56. Olah C. Understanding LSTM networks – colah's blog. Available from: https://colah.github.io/posts/2015-08-Understanding-LSTMs/
57. 10.1111/1468-0394.0011510.1111/1468-0394.00115 ↗
58. Veeramani AS, Crews JH, Buckner GD. Hysteretic recurrent neural networks: a tool for modeling hysteretic materials and systems. Smart Mater Struct. 2009 Jun;18(7):075004. Available from: https://doi.org/10.1088/2F0964-17262F182F72F07500410.1088/0964-1726/18/7/075004 ↗
59. 10.1016/j.sna.2010.08.01310.1016/j.sna.2010.08.013 ↗
60. 10.1016/j.mfglet.2018.01.00210.1016/j.mfglet.2018.01.002 ↗
61. 10.1016/j.mfglet.2018.10.00210.1016/j.mfglet.2018.10.002 ↗
62. 10.1016/S0142-1123(98)00071-110.1016/S0142-1123(98)00071-1 ↗
63. 10.1016/S0029-5493(99)00252-610.1016/S0029-5493(99)00252-6 ↗
64. 10.1145/1081870.108195010.1145/1081870.1081950 ↗
65. 10.1145/2046684.204669210.1145/2046684.2046692 ↗
66. 10.1145/1014052.101406610.1145/1014052.1014066 ↗
67. Goodfellow IJ, Pouget-Abadie J, Mirza M, et al. Generative Adversarial Networks. arXiv:14062661 [cs, stat]. 2014 Jun. version: 1. Available from: http://arxiv.org/abs/1406.2661
68. 10.1109/CVPR.2019.0045310.1109/CVPR.2019.00453 ↗
69. 10.1155/2018/490742310.1155/2018/4907423 ↗
70. 10.1109/CVPR.2017.24110.1109/CVPR.2017.241 ↗
71. 10.1115/1.404137110.1115/1.4041371 ↗
72. 10.1021/acs.nanolett.8b0317110.1021/acs.nanolett.8b03171 ↗
73. 10.1145/3197517.320130410.1145/3197517.3201304 ↗
74. Miikkulainen BFaR. Using marker-based genetic encoding of neural networks to evolve finite-state behaviour. 1991. Available from: http://nn.cs.utexas.edu/?fullmer:ecal91
75. Miikkulainen DEMaR. Efficient reinforcement learning through symbiotic evolution. 1996. Available from: http://nn.cs.utexas.edu/?moriarty:mlj96
76. Gomez FJ, Miikkulainen R. Solving non-Markovian control tasks with neuroevolution. p. 6.
77. 10.1007/s12065-007-0002-410.1007/s12065-007-0002-4 ↗
78. 10.1002/wcms.7010.1002/wcms.70 ↗
79. 10.1038/s42256-018-0006-z10.1038/s42256-018-0006-z ↗
80. 10.1016/B978-0-12-394399-6.00005-910.1016/B978-0-12-394399-6.00005-9 ↗
81. 10.1080/1042691080260911010.1080/10426910802609110 ↗
82. 10.1162/10636560232016981110.1162/106365602320169811 ↗
83. 10.1007/978-3-030-36296-6_7610.1007/978-3-030-36296-6_76 ↗
84. Deb K. · 2001
85. 10.1179/09506600422502190910.1179/095066004225021909 ↗
86. 10.1016/j.asoc.2005.09.00110.1016/j.asoc.2005.09.001 ↗
87. 10.1007/BFb005686710.1007/BFb0056867 ↗
88. 10.1007/3-540-36970-8_1510.1007/3-540-36970-8_15 ↗
89. 10.1002/srin.20180012110.1002/srin.201800121 ↗
90. 10.1016/j.hydromet.2011.02.00810.1016/j.hydromet.2011.02.008 ↗
91. 10.1080/1042691080253976210.1080/10426910802539762 ↗
92. 10.1080/1042691080267938610.1080/10426910802679386 ↗
93. 10.1080/10426914.2019.166078610.1080/10426914.2019.1660786 ↗
94. 10.1080/10426914.2019.164347210.1080/10426914.2019.1643472 ↗
95. Mnih V, Kavukcuoglu K, Silver D, et al. Playing atari with deep reinforcement learning. arXiv:13125602 [cs]. 2013 Dec. Available from: http://arxiv.org/abs/1312.5602
96. Silver D, Hubert T, Schrittwieser J, et al. A general reinforcement learning algorithm that masters chess, shogi, and Go through self-play. Sci. 2018 Dec;362(6419):1140–1144. American Association for the Advancement of Science Section: Report. Available from: https://science.sciencemag.org/content/362/6419/114010.1126/science.aar6404 ↗
97. Silver D, Huang A, Maddison CJ, et al. Mastering the game of Go with deep neural networks and tree search. Nature. 2016 Jan;529(7587):484–489. Number: 7587 Publisher: Nature Publishing Group; Available from: https://www.nature.com/articles/nature1696110.1038/nature16961 ↗
98. 10.1038/s41586-019-1724-z10.1038/s41586-019-1724-z ↗
99. Nakajima K. Physical reservoir computing-an introductory perspective. Jpn J Appl Phys. 2020 May;59(6):060501. Publisher: IOP Publishing; Available from: https://doi.org/10.35848/2F1347-40652Fab8d4f10.35848/1347-4065/ab8d4f ↗
100. 10.1007/978-3-319-33924-5_2210.1007/978-3-319-33924-5_22 ↗