Chirality Analysis of Complex Microparticles using Deep Learning on Realistic Sets of Microscopy Images
Source abstract
Abstract
Nanoscale chirality is an actively growing research field spurred by the giant chiroptical activity, enantioselective biological activity, and asymmetric catalytic activity of chiral nanostructures. Compared to chiral molecules, the handedness of chiral nano- and microstructures can be directly established via electron microscopy, which can be utilized for the automatic analysis of chiral nanostructures and prediction of their properties. However, chirality in complex materials may have multiple geometric forms and scales. Computational identification of chirality from electron microscopy images rather than optical measurements is convenient but is fundamentally challenging, too, because (1) image features differentiating left- and right-handed particles can be ambiguous and (2) three-dimensional structure essential for chirality is ‘flattened’ into two-dimensional projections. Here, we show that deep learning algorithms can identify twisted bowtie-shaped microparticles with nearly 100% accuracy and classify them as left- and right-handed with as high as 99% accuracy. Importantly, such accuracy was achieved with as few as 30 original electron microscopy images of bowties. Furthermore, after training on bowtie particles with complex nanostructured features, the model can recognize other chiral shapes with different geometries without retraining for their specific chiral geometry with 93% accuracy, indicating the true learning abilities of the employed neural networks. These findings indicate that our algorithm trained on a practically feasible set of experimental data enables automated analysis of microscopy data for the accelerated discovery of chiral particles and their complex systems for multiple applications.
Source: openalex_inverted_index · Quality: baseline
Research context
OpenAlex topics: Surface Chemistry and Catalysis · Advanced biosensing and bioanalysis techniques · Machine Learning in Materials Science
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Department of Chemical Engineering and Biointerfaces Institute, University of Michigan, Ann Arbor, Michigan 48109, United States
Beehive AI, San Francisco, California 94080, United States
Department of Aeronautics, Faculty of Engineering, Imperial College London, South Kensington Campus London, SW7 2AZ, United Kingdom
Funding
Division of Chemistry · Office of Naval Research
References (79)
Show source-provided references
1. 10.1126/science.aaz794910.1126/science.aaz7949 ↗
2. 10.1038/nmat419310.1038/nmat4193 ↗
3. 10.1126/science.aba098010.1126/science.aba0980 ↗
4. 10.1038/s41586-018-0034-110.1038/s41586-018-0034-1 ↗
5. 10.1039/C3NR06913C10.1039/C3NR06913C ↗
6. 10.1021/acsnano.1c0588810.1021/acsnano.1c05888 ↗
7. 10.1002/adma.20120517810.1002/adma.201205178 ↗
8. 10.1002/aic.1743810.1002/aic.17438 ↗
9. 10.1088/2632-2153/abd61410.1088/2632-2153/abd614 ↗
10. 10.1017/S143192761900148X10.1017/S143192761900148X ↗
11. 10.3390/nano1110270610.3390/nano11102706 ↗
12. 10.1021/acsnano.0c0680910.1021/acsnano.0c06809 ↗
13. 10.1039/D0NR04140H10.1039/D0NR04140H ↗
14. 10.1016/j.ultramic.2020.11312510.1016/j.ultramic.2020.113125 ↗
15. 10.1016/j.cmpb.2021.10595810.1016/j.cmpb.2021.105958 ↗
16. 10.1002/smtd.20210022310.1002/smtd.202100223 ↗
17. 10.3390/nano1007128510.3390/nano10071285 ↗
18. 10.1002/adts.20180003710.1002/adts.201800037 ↗
19. 10.1016/j.ultramic.2019.03.01710.1016/j.ultramic.2019.03.017 ↗
20. 10.1016/j.mee.2019.11105110.1016/j.mee.2019.111051 ↗
21. 10.1117/1.JMM.18.2.02400110.1117/1.JMM.18.2.024001 ↗
22. 10.1017/S143192761900168510.1017/S1431927619001685 ↗
23. 10.1186/s42649-020-00041-810.1186/s42649-020-00041-8 ↗
24. 10.1038/s41467-020-18952-110.1038/s41467-020-18952-1 ↗
25. 10.1038/s41592-019-0580-y10.1038/s41592-019-0580-y ↗
26. 10.1017/S143192762101267810.1017/S1431927621012678 ↗
27. 10.1038/s41592-021-01080-z10.1038/s41592-021-01080-z ↗
28. 10.1038/s41598-019-48444-210.1038/s41598-019-48444-2 ↗
29. 10.3390/ai101000110.3390/ai1010001 ↗
30. 10.1088/2632-2153/ab81e210.1088/2632-2153/ab81e2 ↗
31. 10.1109/ISBI.2019.875915310.1109/ISBI.2019.8759153 ↗
32. 10.1007/s12551-020-00669-610.1007/s12551-020-00669-6 ↗
33. 10.1016/j.jsb.2020.10749810.1016/j.jsb.2020.107498 ↗
34. 10.1126/sciadv.aaw194910.1126/sciadv.aaw1949 ↗
35. 10.1038/s41524-018-0130-710.1038/s41524-018-0130-7 ↗
36. 10.3390/molecules2406118110.3390/molecules24061181 ↗
37. 10.1109/JSTSP.2020.300531710.1109/JSTSP.2020.3005317 ↗
38. 10.1109/TMI.2016.261301910.1109/TMI.2016.2613019 ↗
39. 10.1038/s41598-019-49105-010.1038/s41598-019-49105-0 ↗
40. 10.1016/j.neunet.2019.08.02510.1016/j.neunet.2019.08.025 ↗
41. 10.1021/acsnano.1c1111810.1021/acsnano.1c11118 ↗
42. 10.1021/jacs.1c0309110.1021/jacs.1c03091 ↗
43. 10.1038/s41524-022-00822-710.1038/s41524-022-00822-7 ↗
44. 10.1002/adma.20150101510.1002/adma.201501015 ↗
45. 10.1038/s41586-023-05733-110.1038/s41586-023-05733-1 ↗
46. 10.1039/C4NR00330F10.1039/C4NR00330F ↗
47. 10.1039/C8SM01290C10.1039/C8SM01290C ↗
48. 10.1039/C3NR06913C10.1039/C3NR06913C ↗
49. 10.1126/science.117721810.1126/science.1177218 ↗
50. 10.1002/adma.20190597510.1002/adma.201905975 ↗
51. Deng, J.; Dong, W.; Socher, R.; Li, L.J.; Li, K.; Fei-Fei, L. Imagenet: A Large-Scale Hierarchical Image Database. 2009 IEEE conference on computer vision and pattern recognition Miami, FL, June 20–25, 2009; pp 248–255.10.1109/CVPR.2009.5206848 ↗
52. 10.1007/978-3-319-10602-1_4810.1007/978-3-319-10602-1_48 ↗
53. Koch, G.; Zemel, R.; Salakhutdinov, R. Siamese Neural Networks for One-Shot Image Recognition. Proceedings of the 32 nd International Conference on Machine Learning , Lille, France, July 6–11, 2015; Vol. 57.
54. Ranasinghe, T.; Orǎsan, C.; Mitkov, R. Semantic Textual Similarity with Siamese Neural Networks. Proceedings of the International Conference on Recent Advances in Natural Language Processing (RANLP 2019) , Varna, Bulgaria, September 2–4, 2019; pp 1004–1011.10.26615/978-954-452-056-4_116 ↗
55. 10.1093/bioinformatics/btz41110.1093/bioinformatics/btz411 ↗
56. 10.1016/j.compbiomed.2017.03.02410.1016/j.compbiomed.2017.03.024 ↗
57. 10.1103/PhysRevResearch.2.03349910.1103/PhysRevResearch.2.033499 ↗
58. 10.1038/s41746-020-0255-110.1038/s41746-020-0255-1 ↗
59. 10.1093/bioinformatics/btz41110.1093/bioinformatics/btz411 ↗
60. Chopra S. · 2005 · 1 · 539
61. Chechik G. · Journal of Machine Learning Research · 2010 · 11 · 1109
62. Schroff F. · 2015 · 815
63. 10.1073/pnas.08209929910.1073/pnas.082099299 ↗
64. 10.1142/S021800149300033910.1142/S0218001493000339 ↗
65. 10.1186/s40537-019-0197-010.1186/s40537-019-0197-0 ↗
66. Tan, M.; Le, Q. Efficientnet: Rethinking Model Scaling for Convolutional Neural Networks. International Conference on Machine Learning , Long Beach, CA, June 9–15, 2019; pp 6105–6114.
67. Hinton G. · Advances in neural information processing systems · 2002 · 15 · 833
68. Simonyan, K.; Vedaldi, A.; Zisserman, A. Deep inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps. arXiv . https://arxiv.org/abs/1312.6034 (accessed 2023-03-20).
69. Selvaraju, R. R.; Cogswell, M.; Das, A.; Vedantam, R.; Parikh, D.; Batra, D. Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization. Proceedings of the IEEE international conference on computer vision , Venice, Italy, October 22–29, 2017; pp 618–626.10.1109/ICCV.2017.74 ↗
70. Tremblay, J.; Prakash, A.; Acuna, D.; Brophy, M.; Jampani, V.; Anil, C.; To, T.; Cameracci, E.; Boochoon, S.; Birchfield, S. Training Deep Networks with Synthetic Data: Bridging the Reality Gap by Domain Randomization. In 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) , Salt Lake City, UT, June 18–22, 2018; pp 969–977.10.1109/CVPRW.2018.00143 ↗
71. 10.1038/s41598-019-54244-510.1038/s41598-019-54244-5 ↗
72. 10.1080/21681163.2016.114910410.1080/21681163.2016.1149104 ↗
73. 10.1038/s41598-021-84287-610.1038/s41598-021-84287-6 ↗
74. 10.1016/j.ultramic.2020.11312510.1016/j.ultramic.2020.113125 ↗
75. 10.5281/zenodo.556371510.5281/zenodo.5563715 ↗
76. Xie, E.; Wang, W.; Yu, Z.; Anandkumar, A.; Alvarez, J. M.; Luo, P. SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers. 35th Conference on Neural Information Processing Systems , Virtual, December 6–14, 2021.
77. 10.1021/acs.chemrev.6b0075510.1021/acs.chemrev.6b00755 ↗
78. Caron, M.; Touvron, H.; Misra, I.; Jégou, H.; Mairal, J.; Bojanowski, P.; Joulin, A. Emerging Properties in Self-Supervised Vision Transformers. 2021 IEEE/CVF International Conference on Computer Vision (ICCV) , Montreal, BC, Canada, October 11–17, 2021.10.1109/ICCV48922.2021.00951 ↗
79. Visheratina, A.; Visheratin, A.; Kumar, P.; Veksler, M.; Kotov, N. A. Chirality Analysis for Nanostructured Microparticles Using Deep Learning. ChemRxiv . https://chemrxiv.org/engage/chemrxiv/article-details/62d5748381efd03957b0a3a4 (accessed 2023-03-15).