Improving Nanoparticle Size Estimation from Scanning Transmission Electron Micrographs with a Multislice Surrogate Model
Source abstract
Abstract
The computational cost of simulating scanning transmission electron microscopy (STEM) images limits the curation of large enough data sets to train accurate and robust machine learning networks for deep feature extraction from atomically resolved STEM images. For nanoparticle size estimation in particular, a diverse data set is essential due to the large variations in size, shape, crystallinity, orientation, and dynamical diffraction effects in experimental data. To address this, we train a 3D convolutional neural network to predict STEM images from voxelized atomic models, achieving a 100x speed-up compared to traditional multislice simulations while maintaining high image quality. We then generate a data set of 100.000 synthetic multislice images and investigate the performance of different size-estimator architectures as a function of training set size. A ResNet18-based model trained on 4000 real and 100.000 synthetic images is found to perform the best, reducing the median size-estimation error from 9.89% without synthetic data to 5.26%.
Source: openalex_inverted_index · Quality: baseline
Research context
OpenAlex topics: Electron and X-Ray Spectroscopy Techniques · Machine Learning in Materials Science · Enhanced Oil Recovery Techniques
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Electron Microscopy Center, Empa − Swiss Federal Laboratories for Materials Science and Technology, Überlandstrasse 129, 8600 Dübendorf, Switzerland
Department of Materials
ETH Zürich
Funding
Schweizerischer Nationalfonds zur F?rderung der Wissenschaftlichen Forschung
References (32)
Show source-provided references
1. 10.1038/s41578-021-00360-610.1038/s41578-021-00360-6 ↗
2. 10.1038/s41565-020-00799-810.1038/s41565-020-00799-8 ↗
3. 10.1021/la801367j10.1021/la801367j ↗
4. 10.1021/jacs.1c1246610.1021/jacs.1c12466 ↗
5. 10.1002/adma.20230799110.1002/adma.202307991 ↗
6. 10.1016/j.ultramic.2016.08.01810.1016/j.ultramic.2016.08.018 ↗
7. 10.1016/j.ultramic.2013.05.00310.1016/j.ultramic.2013.05.003 ↗
8. 10.1002/smtd.20240110810.1002/smtd.202401108 ↗
9. Zhu, J. Y.; Park, T.; Isola, P.; Efros, A. A. Unpaired Image-to-Image Translation Using Cycle-Consistent Adversarial Networks. In 2017 IEEE International Conference on Computer Vision (ICCV), 2017; pp 2242–2251.10.1109/ICCV.2017.244 ↗
10. 10.1016/S0304-3991(02)00249-810.1016/S0304-3991(02)00249-8 ↗
11. Uhrig, J.; Schneider, N.; Schneider, L.; Franke, U.; Brox, T.; Geiger, A. Sparsity Invariant CNNs. In 2017 International Conference on 3D Vision (3DV), 2017; pp 11–20.10.1109/3DV.2017.00012 ↗
12. Krizhevsky, A.; Sutskever, I.; Hinton, G. E. ImageNet classification with deep convolutional neural networks. In Proceedings of the 26th International Conference on Neural Information Processing Systems - Volume 1, Lake Tahoe, NV, 2012; pp 1097–1105.
13. 10.1016/j.micron.2021.10314110.1016/j.micron.2021.103141 ↗
14. 10.1186/s40679-017-0046-110.1186/s40679-017-0046-1 ↗
15. Ros, G.; Sellart, L.; Materzynska, J.; Vazquez, D.; Lopez, A. M. The SYNTHIA Dataset: A Large Collection of Synthetic Images for Semantic Segmentation of Urban Scenes. In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016; pp 3234–3243.10.1109/CVPR.2016.352 ↗
16. 10.1038/s41598-021-84499-w10.1038/s41598-021-84499-w ↗
17. 10.1038/s41524-024-01360-010.1038/s41524-024-01360-0 ↗
18. Richter, S. R.; Vineet, V.; Roth, S.; Koltun, V. Playing for data: Ground truth from computer games. In Computer Vision-ECCV 2016:14th European Conference, Amsterdam, The Netherlands, October 11–14, 2016, Proceedings, Part II, 2016; Springer: pp 102–118.10.1007/978-3-319-46475-6_7 ↗
19. 10.1103/PhysRevLett.100.20610110.1103/PhysRevLett.100.206101 ↗
20. 10.1186/s40679-019-0064-210.1186/s40679-019-0064-2 ↗
21. He, K.; Zhang, X.; Ren, S.; Sun, J. Deep Residual Learning for Image Recognition. In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016; pp 770–778.10.1109/CVPR.2016.90 ↗
22. Qi, C. R.; Yi, L.; Su, H.; Guibas, L. J. PointNet++: deep hierarchical feature learning on point sets in a metric space. In Proceedings of the 31st International Conference on Neural Information Processing Systems, Long Beach, California, USA, 2017; pp 5105–5114.
23. Çiçek, Ö.; Abdulkadir, A.; Lienkamp, S. S.; Brox, T.; Ronneberger, O. 3D U-Net: learning dense volumetric segmentation from sparse annotation. In Medical Image Computing and Computer-Assisted Intervention-MICCAI 2016:19th International Conference, Athens, Greece, October 17–21, 2016, Proceedings, Part II, 2016; Springer: pp 424–432.10.1007/978-3-319-46723-8_49 ↗
24. Multislice surrogate repository. https://github.com/benke97/MultisliceSurrogate (accessed 2025-01-18).
25. Zhang, R.; Isola, P.; Efros, A. A.; Shechtman, E.; Wang, O. The Unreasonable Effectiveness of Deep Features as a Perceptual Metric. In 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2018; pp 586–595.10.1109/CVPR.2018.00068 ↗
26. Iandola, F. N.; Han, S.; Moskewicz, M. W.; Ashraf, K.; Dally, W. J.; Keutzer, K. SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and < 0.5MB model size. In Proceedings of the International Conference on Learning Representations (ICLR), 2017.
27. 10.1016/j.ultramic.2018.06.00310.1016/j.ultramic.2018.06.003 ↗
28. 10.1016/0304-3991(91)90004-P10.1016/0304-3991(91)90004-P ↗
29. Kirkland E. J. · 2010 · 16310.1007/978-1-4419-6533-2_7 ↗
30. 10.1039/D3NR04701F10.1039/D3NR04701F ↗
31. 10.1088/1361-6528/acf93810.1088/1361-6528/acf938 ↗
32. 10.1017/S143192762100999510.1017/S1431927621009995 ↗