Machine Learning Refinement of In Situ Images Acquired by Low Electron Dose LC-TEM
Source abstract
Abstract
Abstract We have studied a machine learning (ML) technique for refining images acquired during in situ observation using liquid-cell transmission electron microscopy. Our model is constructed using a U-Net architecture and a ResNet encoder. For training our ML model, we prepared an original image dataset that contained pairs of images of samples acquired with and without a solution present. The former images were used as noisy images, and the latter images were used as corresponding ground truth images. The number of pairs of image sets was 1,204, and the image sets included images acquired at several different magnifications and electron doses. The trained model converted a noisy image into a clear image. The time necessary for the conversion was on the order of 10 ms, and we applied the model to in situ observations using the software Gatan DigitalMicrograph (DM). Even if a nanoparticle was not visible in a view window in the DM software because of the low electron dose, it was visible in a successive refined image generated by our ML model.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Advanced Electron Microscopy Techniques and Applications · Electron and X-Ray Spectroscopy Techniques · Force Microscopy Techniques and Applications
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Emerging Media Initiative, Kanazawa University , Kakuma-machi, Kanazawa, 920-1192 Ishikawa , Japan
Institute of Low Temperature Science, Hokkaido University , Kita-19, Nishi-8, Kita-ku, Sapporo, 060-0819 Hokkaido , Japan
Institute for Liberal Arts and Sciences, Kyoto University , 302 Konoe-kae, 69 Konoe-cho, Sakyo-ku, Kyoto, 606-8315 Kyoto , Japan
Institute for Chemical Reaction Design and Discovery, Hokkaido University , N21 W10, Kita-ku, Sapporo, 001-0021 Hokkaido , Japan
References (46)
Show source-provided references
1. High-dimensional dynamics of generalization error in neural networks10.1016/j.neunet.2020.08.022 ↗
2. Electron microscopy of gold nanoparticles at atomic resolution10.1126/science.1251959 ↗
3. Benign overfitting in linear regression10.1073/pnas.1907378117 ↗
4. Overfitting or perfect fitting? Risk bounds for classification and regression rules that interpolate
5. Bradski · 2000
6. Chen · 2018
7. Resolution and aberration correction in liquid cell transmission electron microscopy10.1038/s41578-018-0071-2 ↗
8. Electron microscopy of specimens in liquid10.1038/nnano.2011.161 ↗
9. Image super-resolution using deep convolutional networks10.1109/TPAMI.2015.2439281 ↗
10. Molecular nucleation mechanisms and control strategies for crystal polymorph selection10.1038/nature25971 ↗
11. U-Net: Deep learning for cell counting, detection, and morphometry10.1038/s41592-018-0261-2 ↗
12. Electron ptychographic microscopy for three-dimensional imaging10.1038/s41467-017-00150-1 ↗
13. Goodfellow · 2014
14. Generative adversarial networks10.1145/3422622 ↗
15. Electron microscopy image enhanced10.1038/33823 ↗
16. He · 2016
17. In situ electron energy-loss spectroscopy in liquids10.1017/S1431927613001505 ↗
18. Iakubovskii · 2019
19. Self-assembly of MoO 3 needles in gas current for cubic formation pathway10.1039/C7NR02613G ↗
20. Reversible disorder-order transitions in atomic crystal nucleation10.1126/science.aaz7555 ↗
21. Kalinin · 2023
22. Early detection of nucleation events from solution in LC-TEM by machine learning10.3389/fchem.2022.818230 ↗
23. Fast improvement of TEM images with low-dose electrons by deep learning10.1017/S1431927621013799 ↗
24. Katsuno · 2023
25. Katsuno · 2023
26. Possible embryos and precursors of crystalline nuclei of calcium carbonate observed by liquid-cell transmission electron microscopy10.1039/D1FD00125F ↗
27. Kingma · 2015
28. Machine learning: A review of classification and combining techniques10.1007/s10462-007-9052-3 ↗
29. Machine learning based de-noising of electron back scatter patterns of various crystallographic metallic materials fabricated using laser directed energy deposition10.1016/j.ultramic.2023.113703 ↗
30. Compressed sensing electron tomography10.1016/j.ultramic.2013.03.019 ↗
31. Multiple pathways of crystal nucleation in an extremely supersaturated aqueous potassium dihydrogen phosphate (KDP) solution droplet10.1073/pnas.1604938113 ↗
32. A deep learning approach to identify local structures in atomic-resolution transmission electron microscopy images10.1002/adts.201800037 ↗
33. Markov · 200310.1142/5172 ↗
34. Michely · 2004
35. Time-resolved observation of evolution of amyloid-β oligomer with temporary salt crystals10.1021/acs.jpclett.0c01487 ↗
36. Emergence of simple-cell receptive field properties by learning a sparse code for natural images10.1038/381607a0 ↗
37. Scikit-learn: Machine learning in Python
38. Ronneberger · 2015
39. In recognition of aberration-corrected TEM10.1038/s41565-020-0721-6 ↗
40. Electron–water interactions and implications for liquid cell electron microscopy10.1021/jp507400n ↗
41. The potential for Bayesian compressive sensing to significantly reduce electron dose in high-resolution stem images10.1093/jmicro/dft042 ↗
42. Image quality assessment: From error visibility to structural similarity10.1109/TIP.2003.819861 ↗
43. Radiolysis-induced crystallization of sodium chloride in acetone by electron beam irradiation10.1017/S1431927621000179 ↗
44. Machine learning to reveal nanoparticle dynamics from liquid-phase TEM videos10.1021/acscentsci.0c00430 ↗
45. Crystallization by particle attachment in synthetic, biogenic, and geologic environments10.1126/science.aaa6760 ↗
46. The influence of CS/CC correction in analytical imaging and spectroscopy in scanning and transmission electron microscopy10.1016/j.ultramic.2014.09.012 ↗