PID3Net: a deep learning approach for single-shot coherent X-ray diffraction imaging of dynamic phenomena
Source abstract
Abstract
This paper introduces a deep learning (DL)-based method for phase retrieval tailored to single-shot, multiple-frame coherent X-ray diffraction imaging (CXDI), designed specifically for visualizing local nanostructural dynamics within a larger sample. Current phase retrieval methods often struggle with achieving high spatiotemporal resolutions, handling dynamic imaging, and managing computational costs, which limits their applicability in observing nanostructural dynamics. This study addresses these gaps by developing a novel method that leverages a feedforward architecture with a physics-informed strategy utilizing measurement settings, enabling the reconstruction of dynamic “movies" from time-evolving diffraction images of the illuminated area. The method incorporates key enhancements, such as temporal convolution blocks to capture spatiotemporal correlations and a unified TV regularization applied to the reconstructed object, resulting in improved noise reduction and spatial smoothness. An expanded evaluation framework, including multiple metrics and systematic sensitivity analysis, is employed to comprehensively assess the method’s performance and robustness. Proof-of-concept experiments, including numerical simulations and imaging experiments of a moving Ta test chart and colloidal gold particles (dispersed in aqueous polyvinyl alcohol solutions) with synchrotron hard X-rays, validate the high imaging performance of this method. Experimental results demonstrate that structures in the sample have been successfully reconstructed at short exposure times, significantly outperforming both traditional methods and current DL-based methods. The proposed method provides efficient and reliable reconstruction of dynamic images with low computational costs, making it suitable for exploring fast-evolving phenomena in synchrotron- or free-electron laser-based applications requiring high spatiotemporal resolutions.
Source: openalex_inverted_index · Quality: baseline
Research context
OpenAlex topics: Advanced X-ray Imaging Techniques · Medical Imaging Techniques and Applications · Advanced X-ray and CT Imaging
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
References (70)
Show source-provided references
1. Wang, Y. et al. Ultrafast x-ray study of dense-liquid-jet flow dynamics using structure-tracking velocimetry. Nat. Phys. 4, 305–309 (2008).10.1038/nphys840 ↗
2. Withers, P. J. et al. X-ray computed tomography. Nat. Rev. Methods Prim. 1, 18 (2021).10.1038/s43586-021-00015-4 ↗
3. Johnson, A. S. et al. Ultrafast x-ray imaging of the light-induced phase transition in vo2. Nat. Phys. 19, 215–220 (2023).
4. Chapman, H. N. & Nugent, K. A. Coherent lensless x-ray imaging. Nat. Photonics 4, 833–839 (2010).10.1038/nphoton.2010.240 ↗
5. Keller, J. B. Diffraction by an aperture. J. Appl. Phys. 28, 426–444 (1957).10.1063/1.1722767 ↗
6. Miao, J., Ishikawa, T., Robinson, I. K. & Murnane, M. M. Beyond crystallography: Diffractive imaging using coherent x-ray light sources. Science 348, 530–535 (2015).10.1126/science.aaa1394 ↗
7. Chapman, H. N. et al. High-resolution ab initio three-dimensional x-ray diffraction microscopy. JOSA A 23, 1179–1200 (2006).10.1364/JOSAA.23.001179 ↗
8. Robinson, I. & Harder, R. Coherent x-ray diffraction imaging of strain at the nanoscale. Nat. Mater. 8, 291–298 (2009).10.1038/nmat2400 ↗
9. Sun, T., Jiang, Z., Strzalka, J., Ocola, L. & Wang, J. Three-dimensional coherent x-ray surface scattering imaging near total external reflection. Nat. Photonics 6, 586–590 (2012).10.1038/nphoton.2012.178 ↗
10. Abbey, B. From grain boundaries to single defects: A review of coherent methods for materials imaging in the x-ray sciences. JOM 65, 1183–1201 (2013).10.1007/s11837-013-0702-4 ↗
11. Ulvestad, A. et al. Topological defect dynamics in operando battery nanoparticles. Science 348, 1344–1347 (2015).10.1126/science.aaa1313 ↗
12. Yau, A., Cha, W., Kanan, M. W., Stephenson, G. B. & Ulvestad, A. Bragg coherent diffractive imaging of single-grain defect dynamics in polycrystalline films. Science 356, 739–742 (2017).10.1126/science.aam6168 ↗
13. Pedersen, A. F. et al. X-ray coherent diffraction imaging with an objective lens: Towards three-dimensional mapping of thick polycrystals. Phys. Rev. Res. 2, 033031 (2020).10.1103/PhysRevResearch.2.033031 ↗
14. Takazawa, S. et al. Demonstration of single-frame coherent x-ray diffraction imaging using triangular aperture: Towards dynamic nanoimaging of extended objects. Opt. Express 29, 14394–14402 (2021).10.1364/OE.419998 ↗
15. Marchesini, S. et al. X-ray image reconstruction from a diffraction pattern alone. Phys. Rev. B 68, 140101 (2003).10.1103/PhysRevB.68.140101 ↗
16. Latychevskaia, T. Iterative phase retrieval in coherent diffractive imaging: practical issues. Appl. Opt. 57, 7187–7197 (2018).10.1364/AO.57.007187 ↗
17. Fienup, J. R. Phase retrieval algorithms: a comparison. Appl. Opt. 21, 2758–2769 (1982).10.1364/AO.21.002758 ↗
18. Kliuiev, P., Latychevskaia, T., Osterwalder, J., Hengsberger, M. & Castiglioni, L. Application of iterative phase-retrieval algorithms to arpes orbital tomography. N. J. Phys. 18, 093041 (2016).10.1088/1367-2630/18/9/093041 ↗
19. Elser, V. Phase retrieval by iterated projections. J. Opt. Soc. Am. A 20, 40–55 (2003).10.1364/JOSAA.20.000040 ↗
20. Kourousias, G. et al. Monitoring dynamic electrochemical processes with in situ ptychography. Appl. Nanosci. 8, 627–636 (2018).10.1007/s13204-018-0703-2 ↗
21. Datta, K. et al. Computational requirements for real-time ptychographic image reconstruction. Appl. Opt. 58, B19–B27 (2019).10.1364/AO.58.000B19 ↗
22. Pfeiffer, F. X-ray ptychography. Nat. Photonics 12, 9–17 (2018).10.1038/s41566-017-0072-5 ↗
23. Maiden, A. M. & Rodenburg, J. M. An improved ptychographical phase retrieval algorithm for diffractive imaging. Ultramicroscopy 109, 1256–1262 (2009).10.1016/j.ultramic.2009.05.012 ↗
24. Cherukara, M. J. et al. Ai-enabled high-resolution scanning coherent diffraction imaging. Appl. Phys. Lett. 117, 044103 (2020).10.1063/5.0013065 ↗
25. Rodenburg, J. & Maiden, A. Ptychography. Springer Handbook of Microscopy 819–904 (2019).10.1007/978-3-030-00069-1_17 ↗
26. Nashed, Y. S. et al. Parallel ptychographic reconstruction. Opt. Express 22, 32082–32097 (2014).10.1364/OE.22.032082 ↗
27. Abbey, B. et al. Keyhole coherent diffractive imaging. Nat. Phys. 4, 394–398 (2008).10.1038/nphys896 ↗
28. Zhang, F. et al. Phase retrieval by coherent modulation imaging. Nat. Commun. 7, 13367 (2016).10.1038/ncomms13367 ↗
29. Khakurel, K. P. et al. Generation of apodized X-ray illumination and its application to scanning and diffraction microscopy. J. Synchrotron Radiat. 24, 142–149 (2017).10.1107/S1600577516017677 ↗
30. Lo, Y. H. et al. In situ coherent diffractive imaging. Nat. Commun. 9, 1826 (2018).10.1038/s41467-018-04259-9 ↗
31. Levitan, A. L., Keskinbora, K., Sanli, U. T., Weigand, M. & Comin, R. Single-frame far-field diffractive imaging with randomized illumination. Opt. Express 28, 37103–37117 (2020).10.1364/OE.397421 ↗
32. Takayama, Y. et al. Dynamic nanoimaging of extended objects via hard x-ray multiple-shot coherent diffraction with projection illumination optics. Commun. Phys. 4, 48 (2021).10.1038/s42005-021-00539-x ↗
33. Kang, J., Takazawa, S., Ishiguro, N. & Takahashi, Y. Single-frame coherent diffraction imaging of extended objects using triangular aperture. Opt. Express 29, 1441–1453 (2021).10.1364/OE.414341 ↗
34. Takazawa, S. et al. Coupling x-ray photon correlation spectroscopy and dynamic coherent x-ray diffraction imaging: Particle motion analysis from nano-to-micrometer scale. Phys. Rev. Res. 5, L042019 (2023).10.1103/PhysRevResearch.5.L042019 ↗
35. Wu, L. et al. Three-dimensional coherent x-ray diffraction imaging via deep convolutional neural networks. npj Comput. Mater. 7, 175 (2021).10.1038/s41524-021-00644-z ↗
36. Yao, Y. et al. AutoPhaseNN: unsupervised physics-aware deep learning of 3D nanoscale Bragg coherent diffraction imaging. npj Comput. Mater. 8, 124 (2022).10.1038/s41524-022-00803-w ↗
37. Bohra, P., Pham, T.A., Long, Y., Yoo, J. & Unser, M. Dynamic fourier ptychography with deep spatiotemporal priors. Inverse Probl. 39, 064005 (2023).10.1088/1361-6420/acca72 ↗
38. Welker, S., Peer, T., Chapman, H. N. & Gerkmann, T. Deep iterative phase retrieval for ptychography. In ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 1591–1595 (IEEE, 2022).10.1109/ICASSP43922.2022.9746811 ↗
39. Cha, E., Lee, C., Jang, M. & Ye, J. C. Deepphasecut: Deep relaxation in phase for unsupervised fourier phase retrieval. IEEE Trans. Pattern Anal. Mach. Intell. 44, 9931–9943 (2021).10.1109/TPAMI.2021.3138897 ↗
40. Lee, S. Y. et al. Denoising low-intensity diffraction signals using k-space deep learning: Applications to phase recovery. Phys. Rev. Res. 3, 043066 (2021).10.1103/PhysRevResearch.3.043066 ↗
41. Zhuang, Z., Yang, D., Hofmann, F., Barmherzig, D. & Sun, J. Practical phase retrieval using double deep image priors. Electron. Imaging 35, 1–6 (2023).10.2352/EI.2023.35.14.COIMG-153 ↗
42. Zhang, Y. et al. Phasegan: a deep-learning phase-retrieval approach for unpaired datasets. Opt. Express 29, 19593–19604 (2021).10.1364/OE.423222 ↗
43. Ye, Q., Wang, L.-W. & Lun, D. P. Towards practical single-shot phase retrieval with physics-driven deep neural network. Opt. Express 31, 35982–35999 (2023).10.1364/OE.496418 ↗
44. Dekel, S. & Gugel, L. PR-DAD: Phase Retrieval Using Deep Auto-Decoders. In: 2022 7th Int. Conf. Frontiers Signal Process. (ICFSP) 165–172 (IEEE, 2022).10.1109/ICFSP55781.2022.9924739 ↗
45. Cherukara, M. J., Nashed, Y. S. & Harder, R. J. Real-time coherent diffraction inversion using deep generative networks. Sci. Rep. 8, 16520 (2018).10.1038/s41598-018-34525-1 ↗
46. Ye, Q., Wang, L.-W. & Lun, D. P. Sisprnet: end-to-end learning for single-shot phase retrieval. Opt. Express 30, 31937–31958 (2022).10.1364/OE.464086 ↗
47. Kappeler, A., Ghosh, S., Holloway, J., Cossairt, O. & Katsaggelos, A. Ptychnet: Cnn based fourier ptychography. In 2017 IEEE International Conference on Image Processing (ICIP), 1712–1716 (IEEE, 2017).10.1109/ICIP.2017.8296574 ↗
48. Wengrowicz, O., Peleg, O., Zahavy, T., Loevsky, B. & Cohen, O. Deep neural networks in single-shot ptychography. Opt. Express 28, 17511–17520 (2020).10.1364/OE.393961 ↗
49. Shamshad, F., Abbas, F. & Ahmed, A. Deep ptych: Subsampled fourier ptychography using generative priors. In ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 7720–7724 (IEEE, 2019).10.1109/ICASSP.2019.8682179 ↗
50. Metzler, C., Schniter, P., Veeraraghavan, A. & Baraniuk, R. prdeep: Robust phase retrieval with a flexible deep network. In International Conference on Machine Learning, 3501–3510 (PMLR, 2018).
51. Bertasius, G., Wang, H. & Torresani, L. Is space-time attention all you need for video understanding? ArXivabs/2102.05095 https://api.semanticscholar.org/CorpusID:231861462 (2021).
52. Chen, M., Lin, P., Quan, Y., Pang, T. & Ji, H. Unsupervised phase retrieval using deep approximate mmse estimation. IEEE Trans. Signal Process. 70, 2239–2252 (2022).10.1109/TSP.2022.3170710 ↗
53. Tripathi, A., McNulty, I. & Shpyrko, O. G. Ptychographic overlap constraint errors and the limits of their numerical recovery using conjugate gradient descent methods. Opt. Express 22, 1452–1466 (2014).10.1364/OE.22.001452 ↗
54. Lucas, A., Iliadis, M., Molina, R. & Katsaggelos, A. K. Using deep neural networks for inverse problems in imaging: Beyond analytical methods. IEEE Signal Process. Mag. 35, 20–36 (2018).10.1109/MSP.2017.2760358 ↗
55. Thibault, P. & Menzel, A. Reconstructing state mixtures from diffraction measurements. Nature 494, 68–71 (2013).10.1038/nature11806 ↗
56. Tran, D. et al. A closer look at spatiotemporal convolutions for action recognition. In 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 6450–6459 (2018).10.1109/CVPR.2018.00675 ↗
57. Li, P., Edo, T., Batey, D., Rodenburg, J. & Maiden, A. Breaking ambiguities in mixed state ptychography. Opt. Express 24, 9038–9052 (2016).10.1364/OE.24.009038 ↗
58. Guizar-Sicairos, M. & Fienup, J. R. Understanding the twin-image problem in phase retrieval. JOSA A 29, 2367–2375 (2012).10.1364/JOSAA.29.002367 ↗
59. Bian, L. et al. Fourier ptychographic reconstruction using poisson maximum likelihood and truncated wirtinger gradient. Sci. Rep. 6, 27384 (2016).10.1038/srep27384 ↗
60. Chen, Y. & Candes, E. Solving random quadratic systems of equations is nearly as easy as solving linear systems 28,https://proceedings.neurips.cc/paper_files/paper/2015/file/7380ad8a673226ae47fce7bff88e9c33-Paper.pdf (2015).
61. Chambolle, A. An algorithm for total variation minimization and applications. J. Math. Imaging Vis. 20, 89–97 (2004).10.1023/B:JMIV.0000011321.19549.88 ↗
62. Miao, J. et al. Three-dimensional gan- ga 2 o 3 core shell structure revealed by x-ray diffraction microscopy. Phys. Rev. Lett. 97, 215503 (2006).10.1103/PhysRevLett.97.215503 ↗
63. Sekiguchi, Y., Oroguchi, T. & Nakasako, M. Classification and assessment of retrieved electron density maps in coherent x-ray diffraction imaging using multivariate analysis. J. Synchrotron Radiat. 23, 312–323 (2016).10.1107/S1600577515018202 ↗
64. Wakonig, K. et al. X-ray fourier ptychography. Sci. Adv. 5, eaav0282 (2019).10.1126/sciadv.aav0282 ↗
65. Hore, A. & Ziou, D. Image quality metrics: Psnr vs. ssim. In 2010 20th International Conference on Pattern Recognition, 2366–2369 (IEEE, 2010).10.1109/ICPR.2010.579 ↗
66. Butola, M., Rajora, S. & Khare, K. Robust phase retrieval with complexity-guidance for coherent x-ray imaging. Intell. Comput. 2022, https://doi.org/10.34133/2022/9819716 (2022).10.34133/2022/9819716 ↗
67. Takahashi, Y. et al. High-resolution and high-sensitivity X-ray ptychographic coherent diffraction imaging using the CITIUS detector. J. Synchrotron. Radiat. 30, 989–994 (2023).10.1107/S1600577523004897 ↗
68. Ozaki, K. et al. A 17400 frame/s X-ray imaging detector CITIUS with a linear response up to 945 Mcps/pixel (18 Tcps/cm2): evaluation in ptychography. Acta Crystallogr. Sect. A 79, C1240 (2023).10.1107/S205327332308381X ↗
69. Tien-Sinh Vu, T., Takazawa & Chi, D. H. Diffraction intensity for phase retrieval of Ta test chart and AuNP dynamics motion using single-shot coherrent X-ray diffraction imaging https://doi.org/10.5281/zenodo.12144981 (2024).10.5281/zenodo.12144981 ↗
70. Vu, T. S. Python implementation of Physics-Informed Deep learning Network for Dynamic Diffraction imaging (PID3Net). GitHub https://github.com/sinhvt3421/PID3Net (2025).