Dose Requirements for Fused Multi-Modal Electron Tomography

Electron tomography, used in combination with electron energy loss (EELS) or energy dispersive X-ray (EDX) spectroscopy, can characterize three-dimensional (3D) material chemistry at the nanoscale [1,2]. However, due to the high radiation doses required, especially for core excitation spectroscopy (e.g., > 107 e/Å2), the specimen limits are typically surpassed. Recently, advancements in multi-modal data fusion have opened new possibilities to lower the dose requirements for achieving high-resolution chemical imaging [3]. With data fusion, chemical imaging no longer strictly relies on dose-demanding spectroscopic maps but can leverage the high signal-to-noise provided by annular dark-field (ADF) modalities. For chemical tomography, the benefits are more significant—we can measure a material's 3D chemical distributions with two orders of magnitude less electron dose. To achieve the desired chemical accuracy in the new form of fused 3D spectroscopy, experimental strategies must be reconsidered. Now, optimized sampling from both the HAADF projections and spectroscopic maps should be determined. Here we present optimized sampling requirements, via large-scale simulations, for accurate chemical recovery with fused multi-modal electron tomography. Synthetic gold decorated CoO / CuO nanocubes (Fig. 1c) inspired by real experimental data [4] were used as a ground truth to evaluate the accuracy among several sampling configurations. Fig. 1a highlights the relationship between the number of tilts per modality and the average normalized root mean square error (NRMSE) among all elements. In all cases, we observe a 2-3-fold improvement in reconstruction accuracy when multi-modal tomography is used over traditional chemical tomography (bottom row). The advantages of multi-modal tomography are clearly visible in 2D slices (Fig. 1b) taken from the 3D reconstructions. Data fusion strongly benefits from exploiting structural information available in the HAADF modality (Fig. 1b, middle column). When comparing multi-modal data fusion to traditional and compressed sensing regularized algorithms, multi-modal tomography always outperforms both approaches (Fig. 2a & 2b). Notably, a large number of chemical projections are not required when many HAADF projections have also been measured. Increasing the ADF SNR provides dose-efficient improvements in reconstruction quality. We find advised experimental conditions exist when the SNR for the HAADF and chemical modality are above 50 and 4, respectively (Fig. 2c). To efficiently optimize solutions for the fused multi-modal optimization, we employed Bayesian optimization (BO) on supercomputing GPU clusters. Algorithmically, fused multi-modal tomography is an inverse problem whose regularization parameters can significantly influence the reconstruction quality and distort scientific conclusions. Using BO alongside Gaussian process regression enables autonomous optimization of reconstruction parameters with minimal evaluations [5]. When coupled with supercomputer GPU clusters, these simulations can be run in parallel, reducing phase diagram generation from 125 days to 17.5 hours [6]. Sampling Limits for Multi-Modal Tomography. a, NRMSE map representing the reconstruction error as a function of the number of HAADF and chemical tilts. Brighter pixels denote results containing incorrect reconstructions from the ground truth. b, Visualization of the three points on the phase diagram corresponding to conventional chemical tomography (without any HAADF projections), low and high dose fused multi-modal electron tomography. c, The 3D models used for generating synthetic chemical and ADF projections. Scale bar, 75 nm. SNR Dependency for Successful Fused Multi-Modal Reconstructions. a, SNR plot highlighting average reconstruction error as a function of chemical SNR. b, Visualization of the 2D slices from the reconstructions from each of the three curves. c, SNR plot highlighting average reconstruction error as a function of chemical SNR.