CN116485779B - Adaptive wafer defect detection method and device, electronic equipment and storage medium - Google Patents

Adaptive wafer defect detection method and device, electronic equipment and storage medium Download PDF

Info

Publication number
CN116485779B
CN116485779B CN202310527789.5A CN202310527789A CN116485779B CN 116485779 B CN116485779 B CN 116485779B CN 202310527789 A CN202310527789 A CN 202310527789A CN 116485779 B CN116485779 B CN 116485779B
Authority
CN
China
Prior art keywords
image
wafer
defect
preset
defect detection
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202310527789.5A
Other languages
Chinese (zh)
Other versions
CN116485779A (en
Inventor
陈斌
王君逸
张元�
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shenzhen Graduate School Harbin Institute of Technology
Chongqing Research Institute of Harbin Institute of Technology
Original Assignee
Shenzhen Graduate School Harbin Institute of Technology
Chongqing Research Institute of Harbin Institute of Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shenzhen Graduate School Harbin Institute of Technology, Chongqing Research Institute of Harbin Institute of Technology filed Critical Shenzhen Graduate School Harbin Institute of Technology
Priority to CN202310527789.5A priority Critical patent/CN116485779B/en
Publication of CN116485779A publication Critical patent/CN116485779A/en
Application granted granted Critical
Publication of CN116485779B publication Critical patent/CN116485779B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0004—Industrial image inspection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0464—Convolutional networks [CNN, ConvNet]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/088—Non-supervised learning, e.g. competitive learning
    • G06T3/04—
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformation in the plane of the image
    • G06T3/40—Scaling the whole image or part thereof
    • G06T3/4007—Interpolation-based scaling, e.g. bilinear interpolation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/11—Region-based segmentation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30108—Industrial image inspection
    • G06T2207/30148—Semiconductor; IC; Wafer
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
    • Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
    • Y02P90/30—Computing systems specially adapted for manufacturing

Abstract

The application provides a self-adaptive wafer defect detection method, a device, electronic equipment and a storage medium, and relates to the technical field of defect detection. The method comprises the following steps: acquiring a camera image obtained by shooting a wafer through a camera; preprocessing a camera image to obtain an image to be detected; inputting an image to be detected into a trained preset defect detection model to obtain a detection result representing whether a wafer has defects, wherein the preset defect detection model is constructed based on an unsupervised algorithm PaDim; when the detection result is that the wafer has defects, an image representing the defect area is segmented from the camera image by a preset image segmentation strategy to be used as an image segmentation result. Therefore, the method for detecting the wafer defects based on machine vision can be improved, and the problems that the defect samples are difficult to collect, the labor cost is high, and the defect detection robustness in a complex environment is insufficient are solved.

Description

Adaptive wafer defect detection method and device, electronic equipment and storage medium
Technical Field
The present invention relates to the field of defect detection technologies, and in particular, to a method and apparatus for detecting a defect of a wafer in a self-adaptive manner, an electronic device, and a storage medium.
Background
Semiconductor inspection is a derivative service form of the semiconductor market under the development scale of the present day, wherein the key technology involved is wafer defect detection. Along with the increasing research and development investment of semiconductors, the tolerance to design defects is almost zero, so that strict test and verification are required to be carried out on chips, and the defects of pollution, scratch, heterogeneous conditions and the like, which have adverse effects on chip processes, are mainly detected in the surface of a wafer or a circuit structure.
The quality detection of traditional semiconductor products is often carried out manually, i.e. the quality detection is conventionally carried out by using human eyes, and a detection mode of subjective judgment is carried out manually. The manpower cost of manual quality inspection is higher, and detection accuracy gradually drops along with the increase of working time length and fatigue degree of workers, and the defects of slower speed and lower reliability exist. In the semiconductor manufacturing industry, the requirement of the error rate of 6 sigma (3.4 parts per million) is very difficult to reach such high reliability by manual detection under the premise of huge quantity to be detected. In order to solve the problems, an automatic optical detection technology (Automated Optical Inspection) has been developed, which applies a machine vision technology in artificial intelligence, compensates for the vision limitation of human eyes through imaging of electronic equipment, and then transfers the quality judgment of human brain to a computer to analyze and judge by using image processing and artificial intelligence technology, thereby completing the requirements on the quality inspection precision and speed of products in precision manufacturing work.
In recent years, the development of artificial intelligence algorithms based on deep neural networks has greatly promoted the development of AOI detection industry. The deep learning is based on the data-driven automatic feature extraction, has stronger adaptability and robustness, and the image classification, target detection and other algorithms based on the research are widely applied in the AOI detection industry.
However, the current machine vision detection algorithm based on deep learning needs to rely on a large amount of data with true value labels for learning, so that the data scale and the labeling quality for training a network model directly influence the quality of the algorithm, and in an actual wafer detection application scene, a defect sample which can be acquired is quite rare, even if enough defect samples can be acquired, a large amount of manual labeling cost is brought, and great obstruction is brought to application landing of the algorithm.
Disclosure of Invention
In view of the foregoing, an object of the embodiments of the present application is to provide a method, apparatus, electronic device and storage medium for detecting a wafer defect in a self-adaptive manner, which can improve the problems of difficult collection of a defect sample, high labor cost and insufficient defect detection robustness in a complex environment in the current method for detecting a wafer defect based on machine vision.
In order to achieve the technical purpose, the technical scheme adopted by the application is as follows:
in a first aspect, an embodiment of the present application provides an adaptive wafer defect detection method, where the method includes:
acquiring a camera image obtained by shooting a wafer through a camera;
preprocessing the camera image to obtain an image to be detected;
inputting the image to be detected into a trained preset defect detection model to obtain a detection result for representing whether the wafer has defects, wherein the preset defect detection model is constructed based on an unsupervised algorithm PaDim;
when the detection result is that the wafer has defects, an image representing a defect area is segmented from the camera image through a preset image segmentation strategy to serve as an image segmentation result.
With reference to the first aspect, in some optional embodiments, before acquiring the camera image obtained by photographing the wafer by the camera, the method further includes:
based on a PaDim algorithm, constructing a defect detection model by taking a pre-trained ResNet18 as a backup, and taking the pre-trained ResNet18 as the preset defect detection model;
acquiring a training set, wherein the training set comprises standard images of the wafer in a defect-free conventional state;
performing data enhancement processing on the training set to obtain a training set subjected to the data enhancement processing;
and training the preset defect detection model through the training set subjected to data enhancement processing to obtain the trained preset defect detection model.
With reference to the first aspect, in some optional embodiments, training the preset defect detection model through the training set subjected to data enhancement processing to obtain the trained preset defect detection model includes:
extracting a feature image of the standard image through the pre-trained ResNet18, and reducing the dimension of the feature image through a semi-orthogonal embedding strategy to generate a feature vector corresponding to the standard image;
and calculating the multi-element Gaussian distribution corresponding to the feature vector according to the feature vector, and the Markov distance between the feature vector and the multi-element Gaussian distribution.
With reference to the first aspect, in some optional embodiments, calculating, according to the feature vector, a multivariate gaussian distribution corresponding to the feature vector, and a mahalanobis distance between the feature vector and the multivariate gaussian distribution, where the method includes:
calculating to obtain a feature vector set of N standard images at the position (i, j):
calculating a multi-element Gaussian distribution N (mu) of N standard images at the position (i, j) ij Σij), wherein μ ij The sample mean is expressed as follows:
Σij is the sample covariance, expressed as follows:
in the formula, epsilon I is a regularization term;
calculating the feature vector x ij And the multivariate Gaussian distribution N (mu) ij Sigma ij) is defined as the distance M (x) ij )：
Determining the mahalanobis distance as a OOD (Out of Distribution) score of the standard image.
With reference to the first aspect, in some optional embodiments, preprocessing the camera image to obtain a to-be-detected image includes:
carrying out graying treatment on the camera image to obtain a gray image;
image segmentation is carried out on the gray level image so as to obtain a binary image;
performing single-crystal grain image segmentation on the binary image based on a preset template image to obtain a crystal grain image;
and determining the grain image as the image to be detected.
With reference to the first aspect, in some optional embodiments, inputting the to-be-inspected image into a trained preset defect detection model to obtain a detection result indicating whether the wafer has a defect, including:
acquiring an OOD score of the image to be detected through the preset defect detection model;
and when the OOD score is larger than a preset defect threshold, determining that the current image to be detected has defects according to the detection result.
With reference to the first aspect, in some optional embodiments, when the detection result is that the wafer is defective, an image characterizing a defective area is segmented from the camera image by a preset image segmentation policy, so as to be an image segmentation result, including:
obtaining a defect score of a position corresponding to the OOD score in the image to be detected through a bilinear interpolation up-sampling strategy according to the OOD score;
setting a mask of an area, in the grain image, of which the defect score is smaller than or equal to the preset defect threshold value to 0 based on a mask segmentation strategy;
and setting the image pixel value corresponding to the area with the mask of 0 to be 0 so as to obtain the image segmentation result.
In a second aspect, embodiments of the present application further provide an adaptive wafer defect detection apparatus, where the apparatus includes:
a first acquisition unit configured to acquire a camera image obtained by photographing a wafer with a camera;
the image preprocessing unit is used for preprocessing the camera image to obtain an image to be detected;
the detection unit is used for inputting the image to be detected into a trained preset defect detection model to obtain a detection result representing whether the wafer has defects, wherein the preset defect detection model is constructed based on an unsupervised algorithm PaDim;
and the segmentation unit is used for segmenting an image representing a defect area from the camera image by a preset image segmentation strategy when the detection result is that the wafer has defects, and the image is taken as an image segmentation result. The device comprises:
in a third aspect, an embodiment of the present application further provides an electronic device, where the electronic device includes a processor and a memory coupled to each other, where the memory stores a computer program, and when the computer program is executed by the processor, causes the electronic device to perform the method described above.
In a fourth aspect, embodiments of the present application further provide a computer readable storage medium, where a computer program is stored, which when run on a computer, causes the computer to perform the above-mentioned method.
The invention adopting the technical scheme has the following advantages:
in the technical scheme provided by the application, a camera image obtained by shooting a wafer through a camera is firstly obtained, the camera image is preprocessed to obtain an image to be detected, and then the image to be detected is input into a trained preset defect detection model to obtain a detection result for representing whether the wafer has defects, wherein the preset defect detection model is constructed based on an unsupervised algorithm PaDim, and when the detection result is that the wafer has defects, an image for representing a defect area is segmented from the camera image through a preset image segmentation strategy to serve as an image segmentation result. Therefore, the defect detection of the wafer is realized through the non-supervision detection model, the problems that a defect sample is difficult to collect and the labor cost is high when the defect detection of the wafer is performed are solved, meanwhile, the defect detection realized based on the non-supervision algorithm can be better applied to a complex environment, and the environment self-adaption capability of the detection model is improved.
Drawings
The present application may be further illustrated by the non-limiting examples given in the accompanying drawings. It is to be understood that the following drawings illustrate only certain embodiments of the present application and are therefore not to be considered limiting of its scope, for the person of ordinary skill in the art may derive other relevant drawings from the drawings without inventive effort.
Fig. 1 is a flow chart of a method for adaptive wafer defect detection according to an embodiment of the present application.
Fig. 2 is a block diagram of an adaptive wafer defect detection apparatus according to an embodiment of the present application.
Icon: 200-a self-adaptive wafer defect detection device; 210-a first acquisition unit; 220-an image preprocessing unit; 230-a detection unit; 240-segmentation unit.
Detailed Description
The present application will be described in detail below with reference to the drawings and the specific embodiments, and it should be noted that in the drawings or the description of the specification, similar or identical parts use the same reference numerals, and implementations not shown or described in the drawings are in a form known to those of ordinary skill in the art. In the description of the present application, the terms "first," "second," and the like are used merely to distinguish between descriptions and are not to be construed as indicating or implying relative importance.
The embodiment of the application provides an electronic device which can comprise a processing module and a storage module. The memory module stores a computer program that, when executed by the processing module, enables the electronic device to perform the respective steps in the adaptive wafer defect detection method described below.
In this embodiment, the electronic device may be a personal computer, a cloud server, a notebook computer, or the like, and is configured to construct a defect detection model based on a PaDim algorithm, perform defect detection on an image to be detected, and perform defect image segmentation on the image to be detected with a defect.
Referring to fig. 1, the present application further provides a method for detecting a wafer defect. The method can be applied to the electronic equipment, and the steps in the method are executed or realized by the electronic equipment. The adaptive wafer defect detection method may include the following steps:
step 110, obtaining a camera image obtained by shooting a wafer through a camera;
step 120, preprocessing the camera image to obtain an image to be detected;
130, inputting the image to be detected into a trained preset defect detection model to obtain a detection result representing whether the wafer has defects, wherein the preset defect detection model is constructed based on an unsupervised algorithm PaDim;
and 140, when the detection result is that the wafer has defects, segmenting an image representing a defect area from the camera image by a preset image segmentation strategy to serve as an image segmentation result.
In the above embodiment, firstly, a camera image obtained by photographing a wafer with a camera is obtained, and the camera image is preprocessed to obtain an image to be inspected, then the image to be inspected is input into a trained preset defect detection model to obtain a detection result for representing whether the wafer has defects, wherein the preset defect detection model is constructed based on an unsupervised algorithm PaDim, and when the detection result is that the wafer has defects, an image for representing a defect area is segmented from the camera image through a preset image segmentation strategy to be used as an image segmentation result. Therefore, the defect detection of the wafer is realized through the non-supervision detection model, the problems that a defect sample is difficult to collect and the labor cost is high when the defect detection of the wafer is performed are solved, meanwhile, the defect detection realized based on the non-supervision algorithm can be better applied to a complex environment, and the environment self-adaption capability of the detection model is improved.
The following will describe each step of the adaptive wafer defect detection method in detail, as follows:
prior to step 110, the method may include:
step 101, constructing a defect detection model by taking a pre-trained ResNet18 as a backup based on a PaDim algorithm, wherein the defect detection model is used as the preset defect detection model;
102, acquiring a training set, wherein the training set comprises standard images of the wafer in a defect-free conventional state;
step 103, carrying out data enhancement processing on the training set to obtain a training set subjected to the data enhancement processing;
and 104, training the preset defect detection model through the training set subjected to data enhancement processing to obtain the trained preset defect detection model.
It can be appreciated that due to uncertainty of wafer defect types and sparsity of defect samples, the supervised deep learning method is difficult to be applied to wafer defect detection, so that by adopting an unsupervised PaDim algorithm, a training set is formed by standard images of a plurality of wafers in a defect-free conventional state, and the collection difficulty of the training samples is reduced.
In addition, since training of the model typically requires a large number of training samples to support the optimization of the parameters of the model during the training process, the initial neural network in the model may also be identified as a unique individual object for slightly changing image class samples. Therefore, the purposes of increasing the sample data amount and diversity, improving the model anti-interference capability, reducing the false alarm rate and improving the recall rate are achieved by slightly changing the existing training samples (i.e. standard images), such as rotation, overturning, scaling, clipping, shifting, adding Gaussian noise and the like, and adding new images generated after the change to the training set.
In this embodiment, step 104, training the preset defect detection model through the training set subjected to the data enhancement processing to obtain the trained preset defect detection model includes:
step 1041, extracting a feature map of the standard image through the pre-trained res net18, and performing dimension reduction on the feature map through a semi-orthogonal embedding strategy to generate a feature vector corresponding to the standard image;
step 1042, calculating to obtain a multi-element gaussian distribution corresponding to the feature vector, and a mahalanobis distance between the feature vector and the multi-element gaussian distribution according to the feature vector.
In step 1041, the h×w×c feature map is multiplied by a c×k (K < C) semi-orthogonal matrix in the channel dimension to reduce the feature map to h×w×k, and a corresponding feature vector is generated. Where H represents the height of the feature map, W represents the width of the feature map, and C represents the number of channels of the feature map.
In step 1042, according to the feature vector, a multivariate gaussian distribution corresponding to the feature vector is calculated, and a mahalanobis distance between the feature vector and the multivariate gaussian distribution is calculated, including:
calculating to obtain a feature vector set of N standard images at the position (i, j):
calculating a multi-element Gaussian distribution N (mu) of N standard images at the position (i, j) ij Σij), wherein μ ij The sample mean is expressed as follows:
Σij is the sample covariance, expressed as follows:
in the formula, epsilon I is a regularization term;
calculating the feature vector x ij And the multivariate Gaussian distribution N (mu) ij Sigma ij) is defined as the distance M (x) ij )：
Determining the mahalanobis distance as a OOD (Out of Distribution) score of the standard image.
In this embodiment, the OOD score may be understood as an anomaly score that characterizes the anomaly of the standard image, and the OOD score may be an anomaly score map that characterizes the anomaly of the standard image, instead of a single numerical value.
In step 110, because of the different resolutions of the cameras, a complete wafer is often not carried completely and clearly by one camera image, so in order to ensure that each chip forming the wafer can be recorded clearly and completely in the camera image by the camera, the complete wafer may be divided into a plurality of camera images to be sent to the subsequent defect detection and classification process.
In step 120, preprocessing the camera image to obtain an image to be inspected, including:
carrying out graying treatment on the camera image to obtain a gray image;
image segmentation is carried out on the gray level image so as to obtain a binary image;
performing single-crystal grain image segmentation on the binary image based on a preset template image to obtain a crystal grain image;
and determining the grain image as the image to be detected.
In this embodiment, an adaptive threshold method may be used to perform image segmentation on a gray image, so as to obtain a binary image with clearer image presentation. The adaptive threshold may determine the threshold of the target pixel (may be a mean value, a median value, a gaussian weighted average value, etc. of the neighborhood block of the target pixel) according to the pixel distribution of the neighborhood block, that is, the image area with higher brightness is generally higher in threshold, and the image area with lower brightness is less adaptive, so that for the image with larger brightness distribution difference, the segmentation of the image may be clearer.
After the binary image is obtained, traversing the binary image according to a preset single-die image template, and matching the single-die image in the binary image, so that the binary image representing the appearance of the wafer is divided into a plurality of die images, and the set of the die images is the image to be detected.
In step 130, inputting the image to be inspected into a trained preset defect detection model to obtain a detection result indicating whether the wafer has a defect, including:
acquiring an OOD score of the image to be detected through the preset defect detection model;
and when the OOD score is larger than a preset defect threshold, determining that the current image to be detected has defects according to the detection result.
In this embodiment, the method of calculating the OOD score in step 1042 is described in detail, and will not be described here again. The preset defect threshold can be flexibly set according to actual conditions, and a strict detection threshold and a loose detection threshold can be generally set for aiming at different product process requirements. In practical application, a single lot of wafer inspection may take either a strict inspection threshold or a loose inspection threshold. In the present embodiment, the strict detection threshold is exemplified by 9, and the loose detection threshold is exemplified by 15.
The method includes the steps of obtaining an image to be detected through image preprocessing, inputting the image to be detected into a preset defect detection model, obtaining an OOD score map representing abnormal (defect) conditions of the image to be detected, comparing the OOD score of each area with a preset defect threshold (for example: 9), and determining that the detection result is that defects exist in the image to be detected and wafers corresponding to the image to be detected when the OOD score of any area is larger than 9.
In step 140, when the detection result is that the wafer has a defect, an image representing a defect area is segmented from the camera image by a preset image segmentation strategy, so as to be used as an image segmentation result, which includes:
obtaining a defect score of a position corresponding to the OOD score in the image to be detected through a bilinear interpolation up-sampling strategy according to the OOD score;
setting a mask of an area, in the grain image, of which the defect score is smaller than or equal to the preset defect threshold value to 0 based on a mask segmentation strategy;
and setting the image pixel value corresponding to the area with the mask of 0 to be 0 so as to obtain the image segmentation result.
It will be appreciated that the initial OOD score map calculated by the method described above generally characterizes feature map anomalies that have been downsampled (i.e., scaled down). Therefore, after confirming that the defect exists in the to-be-detected image, extracting the OOD score of the position of the defect, and restoring the defect score of the original image corresponding to the OOD score in the to-be-detected image in a bilinear difference up-sampling mode.
In this embodiment, an image characterizing the defect area is segmented from the image to be inspected by means of mask segmentation. First, the mask of the region of the grain image having a defect score equal to or less than a preset defect threshold (for example: 9) is set to 0, the mask of the region of the grain image having a defect score greater than 9 is set to 1, then the image pixel value corresponding to the region of the mask of 0 is set to 0, and the image original pixel value corresponding to the region of the mask of 0 is held. Therefore, the defect area is "scratched" in a mask segmentation mode, so that the original image to be detected is output as a defect image with white background filling.
Referring to fig. 2, the present application further provides an adaptive wafer defect detecting apparatus 200, where the adaptive wafer defect detecting apparatus 200 includes at least one software functional module that may be stored in a memory module in the form of software or Firmware (Firmware) or cured in an Operating System (OS) of an electronic device. The processing module is configured to execute executable modules stored in the storage module, such as software functional modules and computer programs included in the adaptive wafer defect detection apparatus 200.
The adaptive wafer defect detecting apparatus 200 includes a first acquiring unit 210, an image preprocessing unit 220, a detecting unit 230, and a dividing unit 240, and the functions of each unit may be as follows:
a first acquiring unit 210 for acquiring a camera image obtained by photographing a wafer by a camera;
an image preprocessing unit 220, configured to preprocess the camera image to obtain an image to be inspected;
the detecting unit 230 is configured to input the image to be detected into a trained preset defect detecting model to obtain a detecting result indicating whether the wafer has a defect, where the preset defect detecting model is constructed based on an unsupervised algorithm PaDim;
and a segmentation unit 240, configured to segment an image representing a defective area from the camera image by a preset image segmentation strategy when the detection result indicates that the wafer has a defect, as an image segmentation result.
Optionally, the adaptive wafer defect detection apparatus 200 may further include:
the construction unit is used for constructing a defect detection model by taking the pre-trained ResNet18 as a backup based on a PaDim algorithm, and taking the pre-trained ResNet18 as the preset defect detection model;
the second acquisition unit is used for acquiring a training set, wherein the training set comprises standard images of the wafer in a defect-free conventional state;
the data enhancement unit is used for carrying out data enhancement processing on the training set to obtain a training set subjected to the data enhancement processing;
the training unit is used for training the preset defect detection model through the training set subjected to data enhancement processing to obtain the trained preset defect detection model.
Optionally, the training unit may be further configured to:
extracting a feature image of the standard image through the pre-trained ResNet18, and reducing the dimension of the feature image through a semi-orthogonal embedding strategy to generate a feature vector corresponding to the standard image;
and calculating the multi-element Gaussian distribution corresponding to the feature vector according to the feature vector, and the Markov distance between the feature vector and the multi-element Gaussian distribution.
Optionally, according to the feature vector, calculating to obtain a multivariate gaussian distribution corresponding to the feature vector, and a mahalanobis distance between the feature vector and the multivariate gaussian distribution may include:
calculating to obtain a feature vector set of N standard images at the position (i, j):
calculating a multi-element Gaussian distribution N (mu) of N standard images at the position (i, j) ij Σij), wherein μ ij The sample mean is expressed as follows:
Σij is the sample covariance, expressed as follows:
in the formula, epsilon I is a regularization term;
calculating the feature vector x ij And the multivariate Gaussian distribution N (mu) ij Sigma ij) is defined as the distance M (x) ij )：
And determining the mahalanobis distance as the OOD score of the standard image.
Optionally, the image preprocessing unit 220 may be further configured to:
carrying out graying treatment on the camera image to obtain a gray image;
image segmentation is carried out on the gray level image so as to obtain a binary image;
performing single-crystal grain image segmentation on the binary image based on a preset template image to obtain a crystal grain image;
and determining the grain image as the image to be detected.
Optionally, the detection unit 230 may be further configured to:
acquiring an OOD score of the image to be detected through the preset defect detection model;
and when the OOD score is larger than a preset defect threshold, determining that the current image to be detected has defects according to the detection result.
Alternatively, the segmentation unit 240 may be further configured to:
obtaining a defect score of a position corresponding to the OOD score in the image to be detected through a bilinear interpolation up-sampling strategy according to the OOD score;
setting a mask of an area, in the grain image, of which the defect score is smaller than or equal to the preset defect threshold value to 0 based on a mask segmentation strategy;
and setting the image pixel value corresponding to the area with the mask of 0 to be 0 so as to obtain the image segmentation result.
In this embodiment, the processing module may be an integrated circuit chip with signal processing capability. The processing module may be a general purpose processor. For example, the processor may be a central processing unit (Central Processing Unit, CPU), digital signal processor (Digital Signal Processing, DSP), application specific integrated circuit (Application Specific Integrated Circuit, ASIC), field programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or may implement or perform the methods, steps, and logic blocks disclosed in embodiments of the present application.
The memory module may be, but is not limited to, random access memory, read only memory, programmable read only memory, erasable programmable read only memory, electrically erasable programmable read only memory, and the like. In this embodiment, the storage module may be configured to store a camera image, an image to be inspected, a preset defect detection model, a detection result, a preset image segmentation policy, an image segmentation result, a training set, an OOD score, a preset defect threshold, a bilinear interpolation upsampling policy, a mask segmentation policy, and the like. Of course, the storage module may also be used to store a program, and the processing module executes the program after receiving the execution instruction.
It should be noted that, for convenience and brevity of description, specific working processes of the electronic device described above may refer to corresponding processes of each step in the foregoing method, and will not be described in detail herein.
Embodiments of the present application also provide a computer-readable storage medium. The computer readable storage medium has stored therein a computer program which, when run on a computer, causes the computer to perform the adaptive wafer defect detection method as described in the above embodiments.
From the foregoing description of the embodiments, it will be apparent to those skilled in the art that the present application may be implemented in hardware, or by means of software plus a necessary general hardware platform, and based on this understanding, the technical solution of the present application may be embodied in the form of a software product, which may be stored in a non-volatile storage medium (may be a CD-ROM, a U-disc, a mobile hard disk, etc.), and includes several instructions to cause a computer device (may be a personal computer, a server, or a network device, etc.) to perform the methods described in the respective implementation scenarios of the present application.
In summary, the embodiments of the present application provide a method, an apparatus, an electronic device, and a storage medium for adaptive wafer defect detection. In the technical scheme, a camera image obtained by shooting a wafer through a camera is firstly obtained, the camera image is preprocessed to obtain an image to be detected, and then the image to be detected is input into a trained preset defect detection model to obtain a detection result for representing whether the wafer has defects, wherein the preset defect detection model is constructed based on an unsupervised algorithm PaDim, and when the detection result is that the wafer has defects, an image for representing a defect area is segmented from the camera image through a preset image segmentation strategy to be used as an image segmentation result. Therefore, the defect detection of the wafer is realized through the non-supervision detection model, the problems that a defect sample is difficult to collect and the labor cost is high when the defect detection of the wafer is performed are solved, meanwhile, the defect detection realized based on the non-supervision algorithm can be better applied to a complex environment, and the environment self-adaption capability of the detection model is improved.
In the embodiments provided in the present application, it should be understood that the disclosed apparatus, system, and method may be implemented in other manners as well. The above-described apparatus, systems, and method embodiments are merely illustrative, for example, flow charts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions. In addition, the functional modules in the embodiments of the present application may be integrated together to form a single part, or each module may exist alone, or two or more modules may be integrated to form a single part.
The foregoing is merely exemplary embodiments of the present application and is not intended to limit the scope of the present application, and various modifications and variations may be suggested to one skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present application should be included in the protection scope of the present application.

Claims (4)

1. An adaptive wafer defect detection method, the method comprising:
based on PaDim algorithm, constructing a defect detection model by taking the pre-trained ResNet18 as a backup, and taking the pre-trained ResNet18 as a preset defect detection model;
acquiring a training set, wherein the training set comprises standard images of the wafer in a defect-free conventional state;
performing data enhancement processing on the training set to obtain a training set subjected to the data enhancement processing;
training the preset defect detection model through the training set subjected to data enhancement processing to obtain a trained preset defect detection model;
acquiring a camera image obtained by shooting a wafer through a camera;
preprocessing the camera image to obtain an image to be detected;
inputting the image to be detected into a trained preset defect detection model to obtain a detection result for representing whether the wafer has defects, wherein the preset defect detection model is constructed based on an unsupervised algorithm PaDim;
when the detection result is that the wafer has defects, an image representing a defect area is segmented from the camera image through a preset image segmentation strategy to serve as an image segmentation result;
the training of the preset defect detection model through the training set subjected to data enhancement processing to obtain a trained preset defect detection model comprises the following steps:
extracting a feature image of the standard image through the pre-trained ResNet18, and reducing the dimension of the feature image through a semi-orthogonal embedding strategy to generate a feature vector corresponding to the standard image;
according to the feature vector, calculating to obtain a multi-element Gaussian distribution corresponding to the feature vector and a Markov distance between the feature vector and the multi-element Gaussian distribution;
according to the feature vector, calculating to obtain a multi-element Gaussian distribution corresponding to the feature vector, and a mahalanobis distance between the feature vector and the multi-element Gaussian distribution, wherein the method comprises the following steps:
calculating to obtain a feature vector set of N standard images at the position (i, j):
calculating a multi-element Gaussian distribution N (mu) of N standard images at the position (i, j) ij Σij), wherein μ ij The sample mean is expressed as follows:
Σij is the sample covariance, expressed as follows:
in the formula, epsilon I is a regularization term;
calculating the feature vector x ij And the multivariate Gaussian distribution N (mu) ij Sigma ij) is defined as the distance M (x) ij )：
Determining the mahalanobis distance as an OOD score of the standard image;
preprocessing the camera image to obtain an image to be detected, including:
carrying out graying treatment on the camera image to obtain a gray image;
image segmentation is carried out on the gray level image so as to obtain a binary image;
performing single-crystal grain image segmentation on the binary image based on a preset template image to obtain a crystal grain image;
determining the grain image as the image to be detected;
inputting the image to be detected into a trained preset defect detection model to obtain a detection result representing whether the wafer has defects or not, wherein the detection result comprises the following steps:
acquiring an OOD score of the image to be detected through the preset defect detection model;
when the OOD score is larger than a preset defect threshold, determining that the current image to be detected has defects according to the detection result;
when the detection result is that the wafer has defects, an image representing a defect area is segmented from the camera image by a preset image segmentation strategy to serve as an image segmentation result, wherein the image segmentation method comprises the following steps:
obtaining a defect score of a position corresponding to the OOD score in the image to be detected through a bilinear interpolation up-sampling strategy according to the OOD score;
setting a mask of an area, in the grain image, of which the defect score is smaller than or equal to the preset defect threshold value to 0 based on a mask segmentation strategy;
and setting the image pixel value corresponding to the area with the mask of 0 to be 0 so as to obtain the image segmentation result.
2. An adaptive wafer defect detection apparatus, the apparatus comprising:
the construction unit is used for constructing a defect detection model by taking the pre-trained ResNet18 as a backup based on a PaDim algorithm to serve as a preset defect detection model;
the second acquisition unit is used for acquiring a training set, wherein the training set comprises standard images of the wafer in a defect-free conventional state;
the data enhancement unit is used for carrying out data enhancement processing on the training set to obtain a training set subjected to the data enhancement processing;
the training unit is used for training the preset defect detection model through the training set subjected to the data enhancement processing to obtain a trained preset defect detection model;
a first acquisition unit configured to acquire a camera image obtained by photographing a wafer with a camera;
the image preprocessing unit is used for preprocessing the camera image to obtain an image to be detected;
the detection unit is used for inputting the image to be detected into a trained preset defect detection model to obtain a detection result representing whether the wafer has defects, wherein the preset defect detection model is constructed based on an unsupervised algorithm PaDim;
the segmentation unit is used for segmenting an image representing a defect area from the camera image through a preset image segmentation strategy when the detection result is that the wafer has defects, and the image is used as an image segmentation result;
the training unit is further configured to:
extracting a feature image of the standard image through the pre-trained ResNet18, and reducing the dimension of the feature image through a semi-orthogonal embedding strategy to generate a feature vector corresponding to the standard image;
according to the feature vector, calculating to obtain a multi-element Gaussian distribution corresponding to the feature vector and a Markov distance between the feature vector and the multi-element Gaussian distribution;
calculating to obtain a feature vector set of N standard images at the position (i, j):
calculating a multi-element Gaussian distribution N (mu) of N standard images at the position (i, j) ij Σij), wherein μ ij The sample mean is expressed as follows:
Σij is the sample covariance, expressed as follows:
in the formula, epsilon I is a regularization term;
calculating the feature vector x ij And the multivariate Gaussian distribution N (mu) ij Sigma ij) is defined as the distance M (x) ij )：
Determining the mahalanobis distance as an OOD score of the standard image;
the image preprocessing unit is further used for:
carrying out graying treatment on the camera image to obtain a gray image;
image segmentation is carried out on the gray level image so as to obtain a binary image;
performing single-crystal grain image segmentation on the binary image based on a preset template image to obtain a crystal grain image;
determining the grain image as the image to be detected;
the detection unit is also used for:
acquiring an OOD score of the image to be detected through the preset defect detection model;
when the OOD score is larger than a preset defect threshold, determining that the current image to be detected has defects according to the detection result;
the segmentation unit is further configured to:
obtaining a defect score of a position corresponding to the OOD score in the image to be detected through a bilinear interpolation up-sampling strategy according to the OOD score;
setting a mask of an area, in the grain image, of which the defect score is smaller than or equal to the preset defect threshold value to 0 based on a mask segmentation strategy;
and setting the image pixel value corresponding to the area with the mask of 0 to be 0 so as to obtain the image segmentation result.
3. An electronic device comprising a processor and a memory coupled to each other, the memory storing a computer program that, when executed by the processor, causes the electronic device to perform the method of claim 1.
4. A computer readable storage medium, characterized in that the computer readable storage medium has stored therein a computer program which, when run on a computer, causes the computer to perform the method of claim 1.
CN202310527789.5A 2023-05-11 2023-05-11 Adaptive wafer defect detection method and device, electronic equipment and storage medium Active CN116485779B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202310527789.5A CN116485779B (en) 2023-05-11 2023-05-11 Adaptive wafer defect detection method and device, electronic equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202310527789.5A CN116485779B (en) 2023-05-11 2023-05-11 Adaptive wafer defect detection method and device, electronic equipment and storage medium

Publications (2)

Publication Number Publication Date
CN116485779A CN116485779A (en) 2023-07-25
CN116485779B true CN116485779B (en) 2024-01-30

Family

ID=87216315

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202310527789.5A Active CN116485779B (en) 2023-05-11 2023-05-11 Adaptive wafer defect detection method and device, electronic equipment and storage medium

Country Status (1)

Country Link
CN (1) CN116485779B (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117036267A (en) * 2023-08-01 2023-11-10 广州伊索自动化科技有限公司 Curved surface printing detection method, system and storage medium

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111325713A (en) * 2020-01-21 2020-06-23 浙江省北大信息技术高等研究院 Wood defect detection method, system and storage medium based on neural network
CN113344886A (en) * 2021-06-11 2021-09-03 长江存储科技有限责任公司 Wafer surface defect detection method and equipment
CN113470024A (en) * 2021-09-02 2021-10-01 深圳市信润富联数字科技有限公司 Hub internal defect detection method, device, equipment, medium and program product
CN114723708A (en) * 2022-04-07 2022-07-08 浙大城市学院 Handicraft appearance defect detection method based on unsupervised image segmentation
CN115239719A (en) * 2022-09-22 2022-10-25 南昌昂坤半导体设备有限公司 Defect detection method, system, electronic device and storage medium
CN115423755A (en) * 2022-08-08 2022-12-02 深圳市格灵精睿视觉有限公司 Wafer micromachining structure defect detection method and device, equipment and medium thereof
CN115830004A (en) * 2022-12-27 2023-03-21 凌云光技术股份有限公司 Surface defect detection method, device, computer equipment and storage medium
CN115880298A (en) * 2023-03-02 2023-03-31 湖南大学 Glass surface defect detection method and system based on unsupervised pre-training

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114764765A (en) * 2020-12-30 2022-07-19 富泰华工业（深圳）有限公司 Image defect detection method and device, electronic equipment and storage medium

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111325713A (en) * 2020-01-21 2020-06-23 浙江省北大信息技术高等研究院 Wood defect detection method, system and storage medium based on neural network
CN113344886A (en) * 2021-06-11 2021-09-03 长江存储科技有限责任公司 Wafer surface defect detection method and equipment
CN113470024A (en) * 2021-09-02 2021-10-01 深圳市信润富联数字科技有限公司 Hub internal defect detection method, device, equipment, medium and program product
CN114723708A (en) * 2022-04-07 2022-07-08 浙大城市学院 Handicraft appearance defect detection method based on unsupervised image segmentation
CN115423755A (en) * 2022-08-08 2022-12-02 深圳市格灵精睿视觉有限公司 Wafer micromachining structure defect detection method and device, equipment and medium thereof
CN115239719A (en) * 2022-09-22 2022-10-25 南昌昂坤半导体设备有限公司 Defect detection method, system, electronic device and storage medium
CN115830004A (en) * 2022-12-27 2023-03-21 凌云光技术股份有限公司 Surface defect detection method, device, computer equipment and storage medium
CN115880298A (en) * 2023-03-02 2023-03-31 湖南大学 Glass surface defect detection method and system based on unsupervised pre-training

Also Published As

Publication number Publication date
CN116485779A (en) 2023-07-25

Similar Documents

Publication Publication Date Title
CN110060237B (en) Fault detection method, device, equipment and system
CN111681240B (en) Bridge surface crack detection method based on YOLO v3 and attention mechanism
CN110148130B (en) Method and device for detecting part defects
US20210192714A1 (en) Automated machine vision-based defect detection
CN114418957A (en) Global and local binary pattern image crack segmentation method based on robot vision
CN111968095B (en) Product surface defect detection method, system, device and medium
CN111861990B (en) Method, system and storage medium for detecting bad appearance of product
CN115272330B (en) Defect detection method, system and related equipment based on battery surface image
CN113936132B (en) Method and system for detecting water pollution of chemical plant based on computer vision
CN116485779B (en) Adaptive wafer defect detection method and device, electronic equipment and storage medium
CN117094916B (en) Visual inspection method for municipal bridge support
CN111598889A (en) Grading ring inclination fault identification method and device and computer equipment
CN115980050A (en) Water quality detection method and device for water outlet, computer equipment and storage medium
CN117094975A (en) Method and device for detecting surface defects of steel and electronic equipment
CN117392042A (en) Defect detection method, defect detection apparatus, and storage medium
CN117173187B (en) Intelligent valve fault detection system
CN114004858A (en) Method and device for identifying aviation cable surface code based on machine vision
CN116596875B (en) Wafer defect detection method and device, electronic equipment and storage medium
CN115797314B (en) Method, system, equipment and storage medium for detecting surface defects of parts
CN116824135A (en) Atmospheric natural environment test industrial product identification and segmentation method based on machine vision
CN110751623A (en) Joint feature-based defect detection method, device, equipment and storage medium
CN115423765A (en) Grain defect quantitative segmentation method based on template image
CN115239663A (en) Method and system for detecting defects of contact lens, electronic device and storage medium
CN111414895A (en) Face recognition method and device and storage equipment
CN111724352B (en) Patch LED flaw labeling method based on kernel density estimation

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant