A Review on Machine and Deep Learning for Semiconductor Defect Classification in Scanning Electron Microscope Images
Source abstract
Abstract
Continued advances in machine learning (ML) and deep learning (DL) present new opportunities for use in a wide range of applications. One prominent application of these technologies is defect detection and classification in the manufacturing industry in order to minimise costs and ensure customer satisfaction. Specifically, this scoping review focuses on inspection operations in the semiconductor manufacturing industry where different ML and DL techniques and configurations have been used for defect detection and classification. Inspection operations have traditionally been carried out by specialised personnel in charge of visually judging the images obtained with a scanning electron microscope (SEM). This scoping review focuses on inspection operations in the semiconductor manufacturing industry where different ML and DL methods have been used to detect and classify defects in SEM images. We also include the performance results of the different techniques and configurations described in the articles found. A thorough comparison of these results will help us to find the best solutions for future research related to the subject.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Industrial Vision Systems and Defect Detection · Electron and X-Ray Spectroscopy Techniques · Advancements in Photolithography Techniques
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Instituto de Investigación en Informática de Albacete, Universidad de Castilla-La Mancha, 02071 Albacete, Spain
Departamento de Sistemas Informáticos, Universidad de Castilla-La Mancha, 02071 Albacete, Spain
Departamento de Eléctrica, Electrónica, Automática y Comunicaciones, Universidad de Castilla-La Mancha, 02071 Albacete, Spain
Funding
Electronic Components and Systems for European Leadership · Agencia Estatal de Investigación
References (71)
Show source-provided references
1. STATISTA (2019). Monthly Semiconductor Sales Worldwide from 2012 to 2020 (in Billion U.S. Dollars), Statista GmbH.
2. Automated visual inspection in the semiconductor industry: A survey10.1016/j.compind.2014.10.006 ↗
3. Park, H., Choi, J.E., Kim, D., and Hong, S.J. (2021). Artificial Immune System for Fault Detection and Classification of Semiconductor Equipment. Electronics, 10.10.3390/electronics10080944 ↗
4. Recent advances in surface defect inspection of industrial products using deep learning techniques10.1007/s00170-021-06592-8 ↗
5. Artificial intelligence within the interplay between natural and artificial computation: Advances in data science, trends and applications10.1016/j.neucom.2020.05.078 ↗
6. O’Mahony, N., Campbell, S., Carvalho, A., Harapanahalli, S., Hernandez, G.V., Krpalkova, L., Riordan, D., and Walsh, J. (2019). Deep learning vs. traditional computer vision. Science and Information Conference, Springer Nature.10.1007/978-3-030-17795-9_10 ↗
7. A Review and Analysis of Automatic Optical Inspection and Quality Monitoring Methods in Electronics Industry10.1109/ACCESS.2020.3029127 ↗
8. Research on defect detection method of powder metallurgy gear based on machine vision10.1007/s00138-021-01177-7 ↗
9. A computational model of vision attention for inspection of surface quality in production line10.1007/s00138-012-0429-1 ↗
10. Schlosser, T., Beuth, F., Friedrich, M., and Kowerko, D. (2019, January 10–13). A novel visual fault detection and classification system for semiconductor manufacturing using stacked hybrid convolutional neural networks. Proceedings of the 2019 24th IEEE International Conference on Emerging Technologies and Factory Automation (ETFA), Zaragoza, Spain.10.1109/ETFA.2019.8869311 ↗
11. A survey of the advancing use and development of machine learning in smart manufacturing10.1016/j.jmsy.2018.02.004 ↗
12. Tomlinson, W., Halliday, B., Farrington, D., and Skumanich, A. (2000, January 12–14). In-line SEM based ADC for advanced process control. Proceedings of the 2000 IEEE/SEMI Advanced Semiconductor Manufacturing Conference and Workshop, Boston, MA, USA.
13. Avinun-Kalish, M., Sagy, O., Im, S.M., Lee, C., Oh, J., Lim, J., Yoo, H., and Kim, C. (2009, January 10–12). Novel SEM based imaging using secondary electron spectrometer for enhanced voltage contrast and bottom layer defect review. Proceedings of the 2009 IEEE/SEMI Advanced Semiconductor Manufacturing Conference, Berlin, Germany.10.1109/ASMC.2009.5155988 ↗
14. Becker, B., Porat, R., and Eschwege, H. (2010, January 11–13). Identification of yield loss sources in the outer dies using SEM based wafer bevel review. Proceedings of the 2010 IEEE/SEMI Advanced Semiconductor Manufacturing Conference, San Francisco, CA, USA.10.1109/ASMC.2010.5551435 ↗
15. Newell, T., Tillotson, B., Pearl, H., and Miller, A. (2016, January 16–19). Detection of electrical defects with SEMVision in semiconductor production mode manufacturing. Proceedings of the 2016 27th Annual SEMI Advanced Semiconductor Manufacturing Conference, Saratoga Springs, NY, USA.10.1109/ASMC.2016.7491149 ↗
16. Jain, A., Sheridan, J.G., Levitov, F., Aristov, V., Yasharzade, S., and Nguyen, H. (May, January 30). Inline SEM imaging of buried defects using novel electron detection system. Proceedings of the 2018 29th Annual SEMI Advanced Semiconductor Manufacturing Conference (ASMC), Saratoga Springs, NY, USA.10.1109/ASMC.2018.8373177 ↗
17. Zhou, W., Apkarian, R., Wang, Z.L., and Joy, D. (2006). Fundamentals of scanning electron microscopy (SEM). Scanning Microscopy for Nanotechnology, Springer Nature.10.1007/978-0-387-39620-0_1 ↗
18. Jain, A., Sheridan, J.G., Xing, R., Levitov, F., Yasharzade, S., and Nguyen, H. (2017, January 15–18). SEM imaging and automated defect analysis at advanced technology nodes. Proceedings of the 2017 28th Annual SEMI Advanced Semiconductor Manufacturing Conference (ASMC), Saratoga Springs, NY, USA.10.1109/ASMC.2017.7969238 ↗
19. Machine-learning for automatic prediction of flatness deviation considering the wear of the face mill teeth10.1007/s10845-020-01645-3 ↗
20. Machine learning algorithms: A review
21. Sánchez-Reolid, R., López, M.T., and Fernández-Caballero, A. (2020). Machine Learning for Stress Detection from Electrodermal Activity: A Scoping Review. Preprints.10.20944/preprints202011.0043.v1 ↗
22. Support vector regression machines
23. Convolutional neural network for wafer surface defect classification and the detection of unknown defect class10.1109/TSM.2019.2902657 ↗
24. Freund, Y., and Mason, L. (1999, January 27–30). The alternating decision tree learning algorithm. Proceedings of the Sixteenth International Conference on Machine Learning, Bled, Slovenia.
25. Ex-situ porosity classification in metallic components by laser metal deposition: A machine learning-based approach10.1016/j.jmapro.2020.12.048 ↗
26. Machine learning enabled advanced manufacturing in nuclear engineering applications10.1016/j.nucengdes.2020.110817 ↗
27. Lei, H., Teh, C., Li, H., Lee, P.H., and Fang, W. (2020, January 24–26). Automated Wafer Defect Classification using a Convolutional Neural Network Augmented with Distributed Computing. Proceedings of the 2020 31st Annual SEMI Advanced Semiconductor Manufacturing Conference, Saratoga Springs, NY, USA.10.1109/ASMC49169.2020.9185253 ↗
28. The distance-weighted k-nearest-neighbor rule10.1109/TSMC.1976.5408784 ↗
29. Rish, I. (2001, January 4–10). An empirical study of the naive Bayes classifier. Proceedings of the IJCAI 2001 Workshop on Empirical Methods in Artificial Intelligence, Seattle, DC, USA.
30. Mika, S., Ratsch, G., Weston, J., Scholkopf, B., and Mullers, K.R. (1999, January 25–25). Fisher discriminant analysis with kernels. Proceedings of the Neural Networks for Signal Processing IX, Madison, WI, USA.
31. Singh, A., Thakur, N., and Sharma, A. (2016, January 16–18). A review of supervised machine learning algorithms. Proceedings of the 2016 3rd International Conference on Computing for Sustainable Global Development, New Delhi, India.
32. Algorithm AS 136: A k-means clustering algorithm
33. Halder, S., Cerbu, D., Saib, M., and Leray, P. (May, January 30). SEM image analysis with K-means algorithm. Proceedings of the 2018 29th Annual SEMI Advanced Semiconductor Manufacturing Conference, Saratoga Springs, NY, USA.
34. Kaufman, L., and Rousseeuw, P.J. (1990). Partitioning Around Medoids (Program PAM). Finding Groups in Data: An Introduction to Cluster Analysis, Wiley.10.1002/9780470316801 ↗
35. The self-organizing map10.1109/5.58325 ↗
36. An unsupervised neural network approach for automatic semiconductor wafer defect inspection10.1016/j.eswa.2007.10.033 ↗
37. Deep learning for visual understanding: A review10.1016/j.neucom.2015.09.116 ↗
38. Aluminum Casting Inspection using Deep Object Detection Methods and Simulated Ellipsoidal Defects10.1007/s00138-021-01195-5 ↗
39. Deep learning for smart manufacturing: Methods and applications10.1016/j.jmsy.2018.01.003 ↗
40. Lei, C.W., Zhang, L., Tai, T.M., Tsai, C.C., Hwang, W.J., and Jhang, Y.J. (2021). Automated Surface Defect Inspection Based on Autoencoders and Fully Convolutional Neural Networks. Appl. Sci., 11.10.3390/app11177838 ↗
41. Neocognitron: A hierarchical neural network capable of visual pattern recognition10.1016/0893-6080(88)90014-7 ↗
42. Albawi, S., Mohammed, T.A., and Al-Zawi, S. (2017, January 21–23). Understanding of a convolutional neural network. Proceedings of the 2017 International Conference on Engineering and Technology, Antalya, Turkey.10.1109/ICEngTechnol.2017.8308186 ↗
43. Weld image deep learning-based on-line defects detection using convolutional neural networks for Al alloy in robotic arc welding10.1016/j.jmapro.2019.06.023 ↗
44. Rolling element bearing fault diagnosis for rotating machinery using vibration spectrum imaging and convolutional neural networks10.1007/s00170-019-04726-7 ↗
45. Vision based defects detection for Keyhole TIG welding using deep learning with visual explanation10.1016/j.jmapro.2020.05.033 ↗
46. Wang, J., and Lee, S. (2021). Data Augmentation Methods Applying Grayscale Images for Convolutional Neural Networks in Machine Vision. Appl. Sci., 11.10.3390/app11156721 ↗
47. Generative adversarial nets
48. Automated defect inspection system for metal surfaces based on deep learning and data augmentation10.1016/j.jmsy.2020.03.009 ↗
49. Milling chatter detection using scalogram and deep convolutional neural network10.1007/s00170-019-04807-7 ↗
50. Advanced deep-learning techniques for salient and category-specific object detection: A survey10.1109/MSP.2017.2749125 ↗
51. Inspecting method for defective casting products with convolutional neural network (CNN)10.1007/s40684-020-00197-4 ↗
52. Convolutional neural network: A review of models, methodologies and applications to object detection10.1007/s13748-019-00203-0 ↗
53. Imagenet classification with deep convolutional neural networks10.1145/3065386 ↗
54. Simonyan, K., and Zisserman, A. (2014). Very deep convolutional networks for large-scale image recognition. arXiv.
55. Yuan-Fu, Y., and Min, S. (2020, January 24–26). Double Feature Extraction Method for Wafer Map Classification Based on Convolution Neural Network. Proceedings of the 2020 31st Annual SEMI Advanced Semiconductor Manufacturing Conference, Saratoga Springs, NY, USA.10.1109/ASMC49169.2020.9185393 ↗
56. Deep Learning for Classification of the Chemical Composition of Particle Defects on Semiconductor Wafers10.1109/TSM.2019.2963656 ↗
57. Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., and Rabinovich, A. (2015, January 7–12). Going deeper with convolutions. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA.10.1109/CVPR.2015.7298594 ↗
58. Imoto, K., Nakai, T., Ike, T., Haruki, K., and Sato, Y. (2018, January 10–11). A CNN-based transfer learning method for defect classification in semiconductor manufacturing. Proceedings of the 2018 International Symposium on Semiconductor Manufacturing, Tokyo, Japan.10.1109/ISSM.2018.8651174 ↗
59. Monno, S., Kamada, Y., Miwa, H., Ashida, K., and Kaneko, T. (2018). Detection of Defects on SiC Substrate by SEM and Classification Using Deep Learning. International Conference on Intelligent Networking and Collaborative Systems, Springer Nature Switzerland AG.10.1007/978-3-319-98557-2_5 ↗
60. He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27–30). Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.10.1109/CVPR.2016.90 ↗
61. Howard, A.G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., Andreetto, M., and Adam, H. (2017). Mobilenets: Efficient convolutional neural networks for mobile vision applications. arXiv.
62. Tan, M., and Le, Q. (2019, January 9–15). Efficientnet: Rethinking model scaling for convolutional neural networks. Proceedings of the International Conference on Machine Learning, Long Beach, CA, USA.
63. Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., and Chen, L.C. (2018, January 18–23). Mobilenetv2: Inverted residuals and linear bottlenecks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00474 ↗
64. A neural-network approach for semiconductor wafer post-sawing inspection10.1109/66.999602 ↗
65. Choi, D., Shallue, C.J., Nado, Z., Lee, J., Maddison, C.J., and Dahl, G.E. (2019). On empirical comparisons of optimizers for deep learning. arXiv.
66. A stochastic approximation method10.1214/aoms/1177729586 ↗
67. Some methods of speeding up the convergence of iteration methods10.1016/0041-5553(64)90137-5 ↗
68. A method for solving the convex programming problem with convergence rate O (1/k2)
69. Lecture 6.5-rmsprop: Divide the gradient by a running average of its recent magnitude
70. Kingma, D.P., and Ba, J. (2014). Adam: A method for stochastic optimization. arXiv.
71. Online inspection of narrow overlap weld quality using two-stage convolution neural network image recognition10.1007/s00138-020-01158-2 ↗