AI-Enhanced Nanoparticle Analysis: Integrating Single-Shot Object Detection and Vision Transformer for Rapid and Accurate Characterization

Nanoparticles (NPs) are integral to a wide range of scientific, technological, and industrial applications due to their unique properties.Accurate and efficient analysis of NP characteristics is essential for advancing our understanding of their structure-property relationship and facilitating their applications.In this study, we introduce a novel AI prompt engineering approach to NP analysis, utilizing state-of-the-art single-shot detection (SSD) and vision transformer (ViT) techniques, including Ultralytics YOLOv8 and Meta AI segment-anything model (SAM) frameworks, as illustrated in Figure 1 [1, 2].Our results demonstrate that AI-powered automated nanoparticle analysis is competitive with human-handcrafted manual methods, significantly reducing the processing and analysis time while maintaining high precision and accuracy.Our methodology begins with the acquisition of bright-field transmission electron microscopy (BF-TEM) and high-angle annular dark field scanning transmission electron microscopy (HAADF STEM) images containing NPs with varying shapes, sizes, and contrast levels on catalytic support materials.During inference, a custom-trained YOLOv8 object detection model identifies and localizes these individual NPs within the images and generates corresponding bounding boxes for each detection.Subsequently, these bounding boxes serve as location prompts for NP segmentation using SAM, a ViT model known for its exceptional zero-shot learning capability [2].This segmentation step enables the extraction of essential morphological and dimensional characteristics, such as NP size, shape, and area on the catalytic support material.Moreover, the segmentation process aids in distinguishing closely packed and occluded nanoparticles, providing valuable insights into their spatial arrangements.Figure 2 represents a set of object detection, segmentation, and NP analysis results from BF-TEM images of PtCo NPs on graphite and HAADF STEM images of Ru NPs on -Al 2 0 3 (alumina) support materials.To validate the efficacy of our approach, we conducted extensive experiments on diverse S/TEM datasets, encompassing various sizes and shapes, as well as NP-support material systems.Segmentation performance of the model is assessed using commonly employed semantic segmentation metrics, as detailed in Table 1, to measure the overall overlap between the predicted and ground truth segmentations.Our results demonstrate strong performance in NP segmentation, achieving an average F1 score exceeding 0.9 on validation datasets for PtCo NPs -graphite and Cu NPs -silica, and 0.89 on the test dataset for PtW NPs -alumina.These preliminary findings suggest that the proposed AI workflow can be effectively employed for similar downstream tasks, showcasing a remarkable zero-shot learning capability.Furthermore, we present a user-friendly workflow that integrates the YOLOv8-based NP analysis pipeline, enabling scientists and engineers to seamlessly apply this methodology to their NP datasets in a fully automated fashion without the need for retraining the model.The code, including the model weights and a demonstration script, is publicly available on GitHub [3].