journal article·2025-10-14

Zero-Shot Autonomous Microscopy for Scalable and Intelligent Characterization of 2D Materials

Jingyun Yang, Ruoyan Avery Yin, Chi Jiang, Yuepeng Hu, Xiaokai Zhu, Xingjian Hu, Sutharsika Kumar, Samantha K. Holmes, Xiao Wang, Xiaohua Zhai, Keran Rong, Yunyue Zhu, Tianyi Zhang, Zongyou Yin, Yuan Cao, Haoning Tang, Aaron D. Franklin, Jing Kong, Neil Zhenqiang Gong, Zhichu Ren, Haozhe Wang

ACS Nano· Volume 19· Issue 40
Publisher page

Abstract

Characterization of atomic-scale materials traditionally requires human experts with months to years of specialized training. Even for trained human operators, accurate and reliable characterization remains challenging when examining newly discovered materials such as two-dimensional (2D) structures. This bottleneck drives demand for fully autonomous experimentation systems capable of comprehending research objectives without requiring large training data sets. In this work, we present ATOMIC (Autonomous Technology for Optical Microscopy & Intelligent Characterization), an end-to-end framework that integrates foundation models to enable fully autonomous, zero-shot characterization of 2D materials. Our system integrates the vision foundation model (i.e., Segment Anything Model), large language models (i.e., ChatGPT), unsupervised clustering, and topological analysis to automate microscope control, sample scanning, image segmentation, and intelligent analysis through prompt engineering, eliminating the need for additional training. When analyzing typical MoS 2 samples, our approach achieves 99.7% segmentation accuracy for single layer identification, which is equivalent to that of human experts. In addition, the integrated model is able to detect grain boundary slits that are challenging to identify with human eyes. Furthermore, the system retains robust accuracy despite variable conditions, including defocus, color-temperature fluctuations, and exposure variations. It is applicable to a broad spectrum of common 2D materials─including graphene, MoS 2, WSe 2, SnSe─regardless of whether they were fabricated via top-down or bottom-up methods. This work represents the implementation of foundation models to achieve autonomous analysis, providing a scalable and data-efficient characterization paradigm that transforms the approach to nanoscale materials research.
Source: openalex_inverted_index · Quality: baseline

Research context

OpenAlex topics: Machine Learning in Materials Science · Advanced Electron Microscopy Techniques and Applications · Electronic and Structural Properties of Oxides

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

Department of Electrical and Computer Engineering

Duke University

School of Computing

National University of Singapore

Google DeepMind

Department of Electrical Engineering and Computer Science

Massachusetts Institute of Technology

Research School of Chemistry

Australian National University

Department of Electrical Engineering and Computer Sciences

University of California, Berkeley

Department of Materials Science and Engineering

Funding

Swarthmore College · Duke University · Google · Semiconductor Research Corporation Center

References (40)

Show source-provided references
1. 10.1038/s41586-020-2098-y10.1038/s41586-020-2098-y ↗
2. 10.1038/s41563-023-01595-010.1038/s41563-023-01595-0 ↗
3. 10.1038/s41563-018-0055-z10.1038/s41563-018-0055-z ↗
4. 10.1038/nmat367310.1038/nmat3673 ↗
5. 10.1038/nature1743910.1038/nature17439 ↗
6. 10.1038/s41586-023-06393-x10.1038/s41586-023-06393-x ↗
7. 10.1103/PhysRevLett.120.14530110.1103/PhysRevLett.120.145301 ↗
8. 10.1038/s41586-025-08628-510.1038/s41586-025-08628-5 ↗
9. 10.1038/s41586-018-0307-810.1038/s41586-018-0307-8 ↗
10. 10.1038/ncomms1389010.1038/ncomms13890 ↗
11. 10.1038/s41467-018-05761-w10.1038/s41467-018-05761-w ↗
12. 10.1021/acsnano.1c0964410.1021/acsnano.1c09644 ↗
13. 10.1021/acsnano.7b0750410.1021/acsnano.7b07504 ↗
14. 10.1038/s41699-018-0084-010.1038/s41699-018-0084-0 ↗
15. 10.1038/s41524-019-0262-410.1038/s41524-019-0262-4 ↗
16. 10.1002/adma.20200095310.1002/adma.202000953 ↗
17. 10.1103/PhysRevApplied.13.06401710.1103/PhysRevApplied.13.064017 ↗
18. 10.1088/2632-2153/ad228710.1088/2632-2153/ad2287 ↗
19. 10.1021/acsnano.0c0968510.1021/acsnano.0c09685 ↗
20. 10.1038/s41598-024-79386-z10.1038/s41598-024-79386-z ↗
21. 10.1038/s41699-020-0137-z10.1038/s41699-020-0137-z ↗
22. 10.1021/acsnano.1c0210410.1021/acsnano.1c02104 ↗
23. 10.1038/s41467-023-40339-110.1038/s41467-023-40339-1 ↗
24. 10.1093/micmic/ozad067.96310.1093/micmic/ozad067.963 ↗
25. 10.1021/acsnano.4c1876010.1021/acsnano.4c18760 ↗
26. OpenAI Achiam, J.; Adler, S.; Agarwal, S.; Ahmad, L.; Akkaya, I.; Aleman, F. L.; Almeida, D.; Altenschmidt, J.; Altman, S.; Anadkat, S.; Avila, R.; Babuschkin, I.; Balaji, S.; Balcom, V.; Baltescu, P.; Bao, H.; Bavarian, M.; Belgum, J.; Bello, I. GPT-4 Technical Report 2024, 2303.08774. arXiv.https://arxiv.org/abs/2303.08774 (accessed Mar 7, 2025).
27. Radford, A.; Kim, J. W.; Hallacy, C.; Ramesh, A.; Goh, G.; Agarwal, S.; Sastry, G.; Askell, A.; Mishkin, P.; Clark, J.; Krueger, G.; Sutskever, I. Learning Transferable Visual Models From Natural Language Supervision; Proceedings of the 38th International Conference on Machine Learning, PMLR, 2021; pp 8748–8763.
28. Bommasani, R.; Hudson, D. A.; Adeli, E.; Altman, R.; Arora, S.; von Arx, S.; Bernstein, M. S.; Bohg, J.; Bosselut, A.; Brunskill, E.; Brynjolfsson, E.; Buch, S.; Card, D.; Castellon, R.; Chatterji, N.; Chen, A.; Creel, K.; Davis, J. Q.; Demszky, D.; Donahue, C. On the Opportunities and Risks of Foundation Models 2022, 2108.07258. arXiv.https://arxiv.org/abs/2108.07258 (accessed Mar 7, 2025).
29. 10.1038/s41592-024-02235-410.1038/s41592-024-02235-4 ↗
30. Wang, S.; Zhao, Z.; Ouyang, X.; Wang, Q.; Shen, D.; arXiv. ChatCAD: Interactive Computer-Aided Diagnosis on Medical Image using Large Language Models 2023, 2302.07257. arXiv. https://arxiv.org/abs/2302.07257 (accessed Mar 7, 2025).
31. Fu, D.; Li, X.; Wen, L.; Dou, M.; Cai, P.; Shi, B.; Qiao, Y. Drive Like a Human: Rethinking Autonomous Driving with Large Language Models; 2024 IEEE/CVF Winter Conference on Applications of Computer Vision Workshops (WACVW), IEEE, 2024; pp 910–919.10.1109/WACVW60836.2024.00102 ↗
32. Boiko, D. A.; MacKnight, R.; Gomes, G.; arXiv. Emergent autonomous scientific research capabilities of large language models 2023, 2304.05332. arXiv.https://arxiv.org/abs/2304.05332 (accessed Mar 7, 2025).
33. Kojima, T.; Gu, S. S.; Reid, M.; Matsuo, Y.; Iwasawa, Y. Large Language Models are Zero-Shot Reasoners; Advances in Neural Information Processing Systems, NeurIPS, 2022; pp 22199–22213.
34. 10.1088/2632-2153/ad52e910.1088/2632-2153/ad52e9 ↗
35. 10.1002/sstr.7004710.1002/sstr.70047 ↗
36. Kirillov, A.; Mintun, E.; Ravi, N.; Mao, H.; Rolland, C.; Gustafson, L.; Xiao, T.; Whitehead, S.; Berg, A. C.; Lo, W.Y.; Dollar, P.; Girshick, R. Segment Anything; Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), IEEE, 2023; pp 4015–4026.10.1109/ICCV51070.2023.00371 ↗
37. 10.1002/adma.20150135410.1002/adma.201501354 ↗
38. 10.1021/acs.nanolett.7b0209910.1021/acs.nanolett.7b02099 ↗
39. 10.1364/OME.8.00308210.1364/OME.8.003082 ↗
40. 10.1021/acs.nanolett.4c0131410.1021/acs.nanolett.4c01314 ↗