CN116542908A - Wafer defect detection method and device, electronic equipment and storage medium - Google Patents
Wafer defect detection method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN116542908A CN116542908A CN202310365130.4A CN202310365130A CN116542908A CN 116542908 A CN116542908 A CN 116542908A CN 202310365130 A CN202310365130 A CN 202310365130A CN 116542908 A CN116542908 A CN 116542908A
- Authority
- CN
- China
- Prior art keywords
- image
- wafer
- defect
- detected
- data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000007547 defect Effects 0.000 title claims abstract description 263
- 238000001514 detection method Methods 0.000 title claims abstract description 145
- 238000003860 storage Methods 0.000 title claims abstract description 23
- 238000012549 training Methods 0.000 claims abstract description 64
- 238000003062 neural network model Methods 0.000 claims abstract description 39
- 238000000034 method Methods 0.000 claims abstract description 36
- 230000006870 function Effects 0.000 claims description 20
- 238000007689 inspection Methods 0.000 claims description 15
- 230000007246 mechanism Effects 0.000 claims description 13
- 238000004590 computer program Methods 0.000 claims description 11
- 230000009466 transformation Effects 0.000 claims description 11
- 238000002372 labelling Methods 0.000 claims description 8
- 238000012216 screening Methods 0.000 claims description 7
- 238000004364 calculation method Methods 0.000 claims description 5
- 230000002950 deficient Effects 0.000 claims description 4
- 239000004065 semiconductor Substances 0.000 abstract description 6
- 235000012431 wafers Nutrition 0.000 description 184
- 238000004891 communication Methods 0.000 description 11
- XUIMIQQOPSSXEZ-UHFFFAOYSA-N Silicon Chemical compound [Si] XUIMIQQOPSSXEZ-UHFFFAOYSA-N 0.000 description 6
- 238000012545 processing Methods 0.000 description 6
- 229910052710 silicon Inorganic materials 0.000 description 6
- 239000010703 silicon Substances 0.000 description 6
- 238000010586 diagram Methods 0.000 description 5
- 238000006243 chemical reaction Methods 0.000 description 3
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 238000005286 illumination Methods 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 239000013078 crystal Substances 0.000 description 2
- 239000000463 material Substances 0.000 description 2
- 230000003213 activating effect Effects 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000006399 behavior Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004422 calculation algorithm Methods 0.000 description 1
- 238000012790 confirmation Methods 0.000 description 1
- 239000000356 contaminant Substances 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- -1 etc.) Substances 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 238000009776 industrial production Methods 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 230000009191 jumping Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 230000005055 memory storage Effects 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 229910021421 monocrystalline silicon Inorganic materials 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 239000002245 particle Substances 0.000 description 1
- 230000000704 physical effect Effects 0.000 description 1
- 229910021420 polycrystalline silicon Inorganic materials 0.000 description 1
- 238000011176 pooling Methods 0.000 description 1
- 238000012552 review Methods 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000001131 transforming effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0004—Industrial image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/90—Determination of colour characteristics
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10024—Color image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30108—Industrial image inspection
- G06T2207/30148—Semiconductor; IC; Wafer
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30204—Marker
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/30—Computing systems specially adapted for manufacturing
Abstract
The embodiment of the application provides a wafer defect detection method and device, electronic equipment and storage medium, and belongs to the field of semiconductors. The method comprises the following steps: obtaining a wafer sample image, carrying out image annotation on the wafer sample image to obtain image training data, training a neural network model obtained in advance based on the image training data to obtain a wafer defect detection model, obtaining a wafer to-be-detected image to be detected to obtain image to-be-detected data, inputting the image to-be-detected data into the wafer defect detection model to carry out defect detection to obtain defect detection data, and judging whether the wafer to-be-detected image has defects according to the defect detection data. The method and the device can realize classification of the surface defects and the foreign matters of the wafer, improve the accuracy of classification of the surface defects and the foreign matters of the wafer, and reduce the workload of manual rechecking.
Description
Technical Field
The present invention relates to the field of semiconductors, and in particular, to a method and apparatus for detecting a wafer defect, an electronic device, and a storage medium.
Background
At present, in the semiconductor field, the automatic detection equipment still cannot realize the classification of wafer defects at present, because the image collected by the automatic detection equipment is usually a gray level image, no color component information exists, identification features are lacked, and even if the traditional comparison method is adopted, the feature identification cannot be realized, the manual recheck is still needed, the classification is carried out through naked eyes, the workload is large, the subjective factor of the manual recheck is strong, and unified standard and quantitative detection indexes cannot be set, so that the classification of the wafer surface defects and the foreign matters is realized, the accuracy of the classification of the wafer surface defects and the foreign matters is improved, and the manual recheck workload is reduced, so that the technical problem to be solved is urgent.
Disclosure of Invention
The embodiment of the application mainly aims to provide a wafer defect detection method and device, electronic equipment and storage medium, aiming at realizing classification of wafer surface defects and foreign matters and improving accuracy of classification of the wafer surface defects and the foreign matters, so that manual rechecking workload is reduced.
In order to achieve the above object, a first aspect of an embodiment of the present application provides a method for detecting a wafer defect, where the method includes:
Acquiring a wafer sample image, wherein the wafer sample image is a color image;
performing image annotation on the wafer sample image to obtain image training data, wherein the image training data comprises the wafer sample image and an image tag;
training a neural network model obtained in advance based on the image training data to obtain a wafer defect detection model, wherein the neural network model is constructed according to a preset target detection network and an attention mechanism;
acquiring a wafer to be detected image to be detected, and obtaining image to be detected data;
and inputting the image to-be-detected data into the wafer defect detection model to detect defects, obtaining defect detection data, and judging whether the wafer to-be-detected image has defects according to the defect detection data.
In some embodiments, the performing image labeling on the wafer sample image to obtain image training data includes:
marking the wafer sample image based on a preset marking tool to obtain the image tag, wherein the image tag comprises defect type and defect position information of the wafer sample image;
And obtaining the image training data based on the wafer sample image and the image tag.
In some embodiments, before the training the pre-acquired neural network model based on the image training data to obtain the wafer defect detection model, the detection method further includes:
performing color space transformation on the wafer sample image in the image training data;
before the data to be detected of the image is input to the wafer defect detection model to detect defects, the detection method further comprises the following steps:
and carrying out color space transformation on the wafer to-be-detected image in the image to-be-detected data.
In some embodiments, training the pre-acquired neural network model based on the image training data to obtain a wafer defect detection model includes:
performing defect detection on the image training data based on the neural network model to obtain sample detection data;
performing loss calculation on the neural network model based on a preset loss function, the sample detection data and the image tag to obtain model loss data, wherein the loss function is a GIOU function;
And updating parameters of the neural network model according to the model loss data to train the neural network model to obtain the wafer defect detection model.
In some embodiments, the determining whether the image to be inspected of the wafer has a defect according to the defect inspection data includes:
if the image label corresponding to the wafer to-be-detected image in the defect detection data is not empty, the wafer to-be-detected image has a defect;
and if the image label corresponding to the wafer to-be-detected image in the defect detection data is empty, the wafer to-be-detected image has no defect.
In some embodiments, the determining whether the image to be inspected of the wafer has a defect according to the defect inspection data includes:
if the image label corresponding to the wafer to be detected image in the defect detection data is not empty, extracting the confidence coefficient of each first defect in the wafer to be detected image from the defect detection data, and if the confidence coefficient of all the first defects is lower than a preset first threshold value, the wafer to be detected image has no defects; if the confidence coefficient of at least one first defect is not lower than the first threshold value, the image to be detected of the wafer is defective;
And if the image label corresponding to the wafer to-be-detected image in the defect detection data is empty, the wafer to-be-detected image has no defect.
In some embodiments, the determining whether the image to be inspected of the wafer has a defect according to the defect inspection data includes:
if the image label corresponding to the wafer to-be-detected image in the defect detection data is not empty, extracting confidence and position information of each first defect in the wafer to-be-detected image from the defect detection data, and screening out a second defect from the first defects according to the position information and a second threshold; if the confidence degrees of all the second defects are lower than a preset first threshold value, the image to be detected of the wafer is free of defects; if the confidence coefficient of the at least one second defect is not lower than the first threshold value, the image to be detected of the wafer is defective;
and if the image label corresponding to the wafer to-be-detected image in the defect detection data is empty, the wafer to-be-detected image has no defect.
To achieve the above object, a second aspect of the embodiments of the present application provides a wafer defect detection apparatus, including:
The sample image acquisition module is used for acquiring a wafer sample image, wherein the wafer sample image is a color image;
the image labeling module is used for carrying out image labeling on the wafer sample image to obtain image training data, wherein the image training data comprises the wafer sample image and an image label;
the training module is used for training a neural network model acquired in advance based on the image training data to obtain a wafer defect detection model, wherein the neural network model is constructed according to a preset target detection network and an attention mechanism;
the target image acquisition module is used for acquiring a wafer to be detected image to be detected to obtain image to-be-detected data;
and the defect detection module is used for inputting the image to-be-detected data into the wafer defect detection model to detect defects, obtaining defect detection data, and judging whether the wafer to-be-detected image has defects according to the defect detection data.
Optionally, the wafer defect detecting device further includes:
the data enhancement module is specifically used for:
performing color space transformation on the wafer sample image in the image training data;
Performing color space conversion on the wafer to-be-detected image in the image to-be-detected data;
to achieve the above object, a third aspect of the embodiments of the present application proposes an electronic device, which includes a memory and a processor, the memory storing a computer program, the processor implementing the method according to the first aspect when executing the computer program.
To achieve the above object, a fourth aspect of the embodiments of the present application proposes a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method of the first aspect.
According to the wafer defect detection method and device, the electronic equipment and the storage medium, the wafer image containing the color information is used as input data, the target detection network added with the attention mechanism is trained to serve as the wafer defect detection model, classification of wafer surface defects and foreign matters can be achieved, the accuracy of classification of the wafer surface defects and the foreign matters is improved, and the manual rechecking workload is reduced.
Additional features and advantages of the application will be set forth in the description which follows, and in part will be apparent from the description, or may be learned by practice of the disclosure. The objectives and other advantages of the application will be realized and attained by the structure particularly pointed out in the written description and claims hereof as well as the appended drawings.
Drawings
FIG. 1 is a flow chart of a method for detecting wafer defects according to one embodiment provided herein;
FIG. 2 is a particular form of image tag of one embodiment provided herein;
FIG. 3 is a block diagram of an object detection network of one embodiment provided herein;
FIG. 4 is a block diagram of a backhaul component of an object detection network according to one embodiment provided herein;
FIG. 5 is a block diagram of the attention mechanism of one embodiment provided herein;
FIG. 6 is a flow chart of step 150 of FIG. 1;
FIG. 7 is another flow chart of step 150 in FIG. 1;
FIG. 8 is another flow chart of step 150 in FIG. 1;
FIG. 9 is a flow chart of a method for detecting wafer defects according to another embodiment of the present application;
FIG. 10 is a schematic structural diagram of a wafer defect detection apparatus according to one embodiment of the present disclosure;
fig. 11 is a schematic hardware structure of an electronic device according to an embodiment provided in the present application.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the present application will be further described in detail with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the present application.
It should be noted that although functional block division is performed in a device diagram and a logic sequence is shown in a flowchart, in some cases, the steps shown or described may be performed in a different order than the block division in the device, or in the flowchart. The terms first, second and the like in the description and in the claims and in the above-described figures, are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order.
Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of the present application only and is not intended to be limiting of the present application.
First, several nouns referred to in this application are parsed:
and (3) wafer: wafer refers to a silicon wafer used for manufacturing silicon semiconductor circuits, the original material of which is silicon. The high-purity polycrystalline silicon is dissolved and then doped with silicon crystal seed, and then slowly pulled out to form cylindrical monocrystalline silicon. The silicon ingot is ground, polished, and sliced to form a silicon wafer, i.e., a wafer.
Wafer defect: the wafer surface has many defect types, which may be defects generated by the process or may be defects of the material itself. Different defect detection modes may be used to divide defects differently. Considering the physical properties of the defects and the pertinence of the following defect detection algorithm, the defects can be simply classified into surface redundancy (particles, contaminants, etc.), crystal defects (slip line defects, stacking faults), scratches, pattern defects (for pattern wafers).
At present, in the semiconductor field, the automatic detection equipment still cannot realize the classification of wafer defects at present, because the image collected by the automatic detection equipment is usually a gray level image, no color component information exists, identification features are lacked, and even if the traditional comparison method is adopted, the feature identification cannot be realized, the manual recheck is still needed, the classification is carried out through naked eyes, the workload is large, the subjective factor of the manual recheck is strong, and unified standard and quantitative detection indexes cannot be set, so that the classification of the wafer surface defects and the foreign matters is realized, the accuracy of the classification of the wafer surface defects and the foreign matters is improved, and the manual recheck workload is reduced, so that the technical problem to be solved is urgent.
Based on the above, the embodiments of the present application provide a method and apparatus for detecting a wafer defect, an electronic device, and a storage medium, which aim to implement classification of a wafer surface defect and a foreign object, and improve accuracy of classification of the wafer surface defect and the foreign object, so as to reduce workload of manual review.
The method and apparatus for detecting a wafer defect, an electronic device, and a storage medium provided in the embodiments of the present application are specifically described through the following embodiments, and the method for detecting a wafer defect in the embodiments of the present application is described first.
The embodiment of the application provides a method for detecting wafer defects, and relates to the field of semiconductors. The wafer defect detection method provided by the embodiment of the invention can be applied to a terminal, a server side and software running in the terminal or the server side. In some embodiments, the terminal may be a smart phone, tablet, notebook, desktop, etc.; the server side can be configured as an independent physical server, a server cluster or a distributed system formed by a plurality of physical servers, and a cloud server for providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDNs, basic cloud computing services such as big data and artificial intelligent platforms and the like; the software may be an application or the like for realizing the wafer defect detection method, but is not limited to the above form.
The subject application is operational with numerous general purpose or special purpose computer system environments or configurations. For example: personal computers, server computers, hand-held or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments that include any of the above systems or devices, and the like. The application may be described in the general context of computer-executable instructions, such as program modules, being executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types. The application may also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules may be located in both local and remote computer storage media including memory storage devices.
In the embodiments of the present application, when related processing is required according to user information, user behavior data, user history data, user location information, and other data related to user identity or characteristics, permission or consent of the user is obtained first, and the collection, use, processing, and the like of the data comply with related laws and regulations and standards of related countries and regions. In addition, when the embodiment of the application needs to acquire the sensitive personal information of the user, the independent permission or independent consent of the user is acquired through a popup window or a jump to a confirmation page or the like, and after the independent permission or independent consent of the user is explicitly acquired, necessary user related data for enabling the embodiment of the application to normally operate is acquired.
Fig. 1 is a flowchart of a method for detecting a wafer defect according to an embodiment of the present application, where the method in fig. 1 may include, but is not limited to, steps 110 to 150.
Step 110, obtaining a wafer sample image;
step 120, performing image annotation on the wafer sample image to obtain image training data;
step 130, training a pre-acquired neural network model based on image training data to obtain a wafer defect detection model;
Step 140, obtaining a wafer to be detected image to be detected, and obtaining image to be detected data;
and 150, inputting the image to-be-detected data into a wafer defect detection model to detect defects, obtaining defect detection data, and judging whether the wafer to-be-detected image has defects or not according to the defect detection data.
In step 110, the wafer sample image is a color image.
In step 110 of some embodiments, the wafer sample image is acquired by a color oblique illumination microscope, and color information in the wafer sample image may be used to enhance the feature information of the defect; the resolution of the wafer sample image obtained by the color oblique illumination microscope is doubled compared with that of the general illumination, and the detection of the subsequent defects is facilitated. The wafer sample image may also be acquired by other means, not limited thereto.
In step 120, the image training data includes a wafer sample image and an image tag. Referring to fig. 2, fig. 2 is a specific form of an image tag according to an embodiment provided herein, in some embodiments, a wafer sample image is marked based on a preset marking tool to obtain an image tag, where the image tag includes defect type and defect position information of the wafer sample image, and image training data is obtained based on the wafer sample image and the image tag. It should be noted that, one wafer sample image corresponds to one image tag, if one image tag is empty, it indicates that the wafer sample image corresponding to the image tag has no defect, otherwise, it indicates that the wafer sample image corresponding to the image tag has a defect. When the wafer sample image contains a defect, the first number represents the defect type in each-row data of the image tag corresponding to the wafer sample image; the second to fifth numbers represent defect position information. In one possible implementation, the defect is marked by a square target frame, the second number representing the center point abscissa value of the square target frame divided by the width of the wafer sample image, the third number representing the center point ordinate value of the square target frame divided by the height of the wafer sample image, the fourth number representing the width of the square target frame divided by the width of the wafer sample image, and the fifth number representing the height of the square target frame divided by the height of the wafer sample image. The image label of fig. 2 indicates that there are 6 defects in the wafer sample image corresponding to the image label, wherein 4 defects with defect type 0 and 2 defects with defect type 1.
In step 130, a neural network model is constructed according to a predetermined target detection network and attention mechanism. Referring to fig. 3, in some embodiments, a YOLO v5 network is selected as the target detection network, as shown in fig. 3, the YOLO v5 network includes four sub-components: input, backbone, neck and Prediction. The YOLO v5 network has the advantages of high detection speed, high accuracy and the like, and is suitable for real-time monitoring in the industrial production process. Referring to fig. 5, in one possible implementation manner, a SENet structure is selected as the attention mechanism, and as shown in fig. 5, the dimension of the feature map with the dimension H (height) xW (width) xC (channel number) is reduced to 1x1xC through Global pooling; reducing the dimension of the feature map to 1x1xC/r by a fully connected layer (FC), wherein r is a scaling parameter for reducing the calculation amount; then activating by a ReLU function; restoring the dimension of the feature map to 1x1xC through a full connection layer (FC); obtaining normalized weights through a Sigmoid function; finally, scale functions are used to weight the normalized weights to the features of each channel of the feature map. In the SENet structure, complex correlations among channels can be better fitted by reducing the dimension of the feature map and then recovering; and simultaneously, the parameter quantity and the calculated quantity are greatly reduced. As shown in fig. 4, fig. 4 is a specific structure of a backup subassembly of the YOLO v5 network, by replacing the CSP structure in the backup subassembly of the YOLO v5 network with a SENet structure, an attention mechanism is added to the YOLO v5, and the SENet structure learns the weights of the features according to the loss function, so that the effective features have high weights, ineffective or small feature weights, which is beneficial to extracting important features, and further, the accuracy of the model is improved. Therefore, the YOLO v5 network added with the SENet structure can achieve better detection effect as a neural network model. In one possible implementation, the neural network model uses a loss function that is a GIOU function that can well represent the distance and overlap between the predicted location of the defect and the true location of the defect. Preferably, in the present embodiment, the neural network model is constructed according to the YOLO v5 network and the SENet attention mechanism, the loss function is set to the GIOU function, the number of pictures (batch-size) fed to the neural network each time is set to 8, the training iteration number (epochs) is set to 50, and the training picture size (img-size) is set to 1280.
In a specific embodiment, the loss function is set to a GIOU function, and step 130 includes, but is not limited to, the steps of:
performing defect detection on the image training data based on the neural network model to obtain sample detection data;
performing loss calculation on the neural network model based on the loss function, the sample detection data and the image label to obtain model loss data;
and updating parameters of the neural network model according to the model loss data to train the neural network model to obtain the wafer defect detection model.
In a specific implementation of this embodiment, defect detection is performed on each wafer sample image in the image training data by using a YOLO v5 network and a SENet structure of the neural network model, so as to obtain sample detection data corresponding to each wafer sample image, where the sample detection data includes detection defect type and detection position information of the wafer sample image. Further, based on the GIOU function, loss calculation is performed based on the detected defect type, the detected position information and the defect type and the defect position information in the image label of each wafer sample image, so that model loss data are obtained. And finally, continuously updating parameters of the neural network model according to the model loss data until the model loss data accords with preset iteration conditions, stopping updating the parameters of the neural network model, taking the parameters of the neural network model at the moment as final parameters, and taking the neural network model at the moment as a wafer defect detection model obtained through training.
In step 140, the image to be detected of the wafer to be detected is obtained to obtain image to be detected data, wherein the image to be detected of the wafer is a color image, and the obtaining manner is similar to that of the wafer sample image, and will not be described herein.
In step 150, the image to-be-detected data is input to the wafer defect detection model for defect detection, so as to obtain defect detection data, and whether the wafer to-be-detected image has defects is judged according to the defect detection data.
Referring to fig. 6, in some embodiments, step 150 may include, but is not limited to, steps 610 through 640:
step 610: inputting the image data to be detected into a wafer defect detection model for defect detection to obtain defect detection data;
step 620: if the image tag corresponding to the image to be detected of the wafer in the defect detection data is empty, step 630 is skipped; otherwise, step 640 is skipped;
step 630: the image to be detected of the wafer has no defect;
step 640: the image to be detected of the wafer has defects.
Through the steps 610 to 640, it can be effectively determined whether the image to be inspected of the wafer has defects, and manual operation is reduced.
Referring to fig. 7, in some embodiments, step 150 may include, but is not limited to, steps 710 through 760:
Step 710: inputting the image data to be detected into a wafer defect detection model for defect detection to obtain defect detection data;
step 720: if the image tag corresponding to the image to be detected of the wafer in the defect detection data is empty, step 730 is skipped; otherwise, step 740 is skipped;
step 730: the image to be detected of the wafer has no defect;
step 740: extracting the confidence coefficient of each first defect in the wafer to-be-detected image from the defect detection data;
step 750: if the confidence levels of all the first defects are lower than the preset first threshold value, step 730 is skipped; otherwise jump to step 760;
step 760: the image to be detected of the wafer has defects.
In some possible implementations, the first threshold is determined when training the wafer defect detection model, and the accuracy of the defect detection data may be improved by screening out the first defects with a confidence below the first threshold.
Referring to fig. 8, in some embodiments, step 150 may include, but is not limited to, steps 810 through 870:
step 810: inputting the image data to be detected into a wafer defect detection model for defect detection to obtain defect detection data;
step 820: if the image tag corresponding to the image to be detected of the wafer in the defect detection data is empty, step 830 is skipped; otherwise, step 840 is skipped;
Step 830: the image to be detected of the wafer has no defect;
step 840: extracting confidence and position information of each first defect in an image to be detected of the wafer from the defect detection data;
step 850: screening a second defect from the first defects according to the position information and a second threshold;
step 860: if the confidence levels of all the second defects are lower than the preset first threshold value, jumping to step 830; otherwise, step 880 is skipped;
step 870: the image to be detected of the wafer has defects.
In some possible embodiments, the second threshold represents a possible position of the defect, and in step 850, screening the second defect from the first defect according to the position information and the second threshold refers to screening the defect having an intersection between the position of the defect in the first defect and the possible position of the defect, so as to obtain the second defect; in some possible implementations, the first threshold is determined when training a wafer defect detection model; the accuracy of the defect detection data can be further improved by screening and filtering the first defect and the second defect by using the second threshold and the first threshold, respectively.
In some embodiments, prior to step 130, the method of detecting wafer defects further comprises performing a color space transformation on the wafer sample image in the image training data; before step 150, the method for detecting a wafer defect further includes performing color space transformation on a wafer to be detected image in the image to be detected data.
Referring to fig. 9, prior to step 130, the wafer defect detection method may include, but is not limited to, step 910; prior to step 150, the wafer defect detection method may include, but is not limited to, steps 920:
step 910: performing color space transformation on the wafer sample image in the image training data;
step 920: performing color space conversion on the wafer to-be-detected image in the image to-be-detected data;
in some possible implementations, in step 910, the wafer sample image in the image training data is transformed into an HSV color space; in step 920, transforming the wafer to be detected image in the image to be detected data into HSV color space; the HSV color space uses Hue (Hue), saturation (Saturation), brightness (Value) to represent color, can separate brightness (image intensity) from chromaticity (color information), and is more suitable for image processing; in some possible implementations, when the HSV color space transformation is performed on the wafer sample image in the image training data and the wafer to be detected image in the image to be detected data, the disturbance can be adjusted or increased for H (hue), S (saturation) and V (brightness) so as to achieve the purpose of data enhancement. Through carrying out color space transformation on the wafer sample image in the image training data and the wafer to-be-detected image in the image to-be-detected data, the color information in the image can be better utilized, the purpose of data enhancement is achieved, and the accuracy of the wafer defect detection model is further improved.
Referring to fig. 10, an embodiment of the present application further provides a wafer defect detection apparatus, which can implement the above wafer defect detection method, where the apparatus includes:
a sample image acquisition module 1010 for acquiring a wafer sample image, wherein the wafer sample image is a color image;
the image labeling module 1020 is configured to perform image labeling on the wafer sample image to obtain image training data, where the image training data includes the wafer sample image and an image tag;
the training module 1030 is configured to train the neural network model acquired in advance based on the image training data to obtain a wafer defect detection model, where the neural network model is constructed according to a preset target detection network and an attention mechanism;
the target image acquisition module 1040 is used for acquiring an image to be detected of the wafer to be detected, and obtaining image data to be detected;
the defect detection module 1050 is configured to input the image to-be-detected data to the wafer defect detection model for performing defect detection, obtain defect detection data, and determine whether the wafer to-be-detected image has a defect according to the defect detection data.
Optionally, the wafer defect detecting device further includes:
A data enhancement module (not shown), in particular for:
performing color space transformation on the wafer sample image in the image training data;
performing color space conversion on the wafer to-be-detected image in the image to-be-detected data;
the specific implementation manner of the wafer defect detection device is basically the same as that of the specific embodiment of the wafer defect detection method, and is not repeated here.
The embodiment of the application also provides electronic equipment, which comprises a memory and a processor, wherein the memory stores a computer program, and the processor realizes the wafer defect detection method when executing the computer program. The electronic equipment can be any intelligent terminal including a tablet personal computer, a vehicle-mounted computer and the like.
Referring to fig. 11, fig. 11 is a schematic hardware structure of an electronic device according to an embodiment of the present application, where the electronic device includes:
the processor 1110 may be implemented by a general-purpose CPU (central processing unit), a microprocessor, an application-specific integrated circuit (ApplicationSpecificIntegratedCircuit, ASIC), or one or more integrated circuits, etc. for executing related programs to implement the technical solutions provided by the embodiments of the present application;
The memory 1120 may be implemented in the form of read-only memory (ReadOnlyMemory, ROM), static storage, dynamic storage, or random access memory (RandomAccessMemory, RAM). The memory 1120 may store an operating system and other application programs, and when the technical solutions provided in the embodiments of the present application are implemented by software or firmware, relevant program codes are stored in the memory 1120, and the processor 1110 invokes a method for detecting a wafer defect in the embodiments of the present application;
an input/output interface 1130 for implementing information input and output;
the communication interface 1140 is configured to implement communication interaction between the present device and other devices, and may implement communication in a wired manner (e.g. USB, network cable, etc.), or may implement communication in a wireless manner (e.g. mobile network, WIFI, bluetooth, etc.);
a bus 1150 for transferring information between various components of the device (e.g., processor 1110, memory 1120, input/output interface 1130, and communication interface 1140);
wherein processor 1110, memory 1120, input/output interface 1130, and communication interface 1140 implement communication connections among each other within the device via bus 1150.
The embodiment of the application also provides a computer readable storage medium, wherein the computer readable storage medium stores a computer program, and the computer program realizes the wafer defect detection method when being executed by a processor.
The memory, as a non-transitory computer readable storage medium, may be used to store non-transitory software programs as well as non-transitory computer executable programs. In addition, the memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one magnetic disk storage device, flash memory device, or other non-transitory solid state storage device. In some embodiments, the memory optionally includes memory remotely located relative to the processor, the remote memory being connectable to the processor through a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
According to the wafer defect detection method and device, the electronic equipment and the storage medium, through the acquisition of the wafer sample image and the image marking of the wafer sample image, the image training data are obtained, and the wafer image containing color information can be used as input data. Further, training a pre-acquired neural network model based on image training data to obtain a wafer defect detection model, wherein the neural network model is constructed according to a preset target detection network and an attention mechanism, and the accuracy of classifying defects and foreign matters on the surface of a wafer by the model can be improved by taking the target detection network with the attention mechanism added in training as the wafer defect detection model. Finally, obtaining a wafer to be detected image to be detected, and obtaining image to be detected data; the image to-be-detected data are input into the wafer defect detection model to detect the defects, so that the defect detection data are obtained, whether the wafer to-be-detected image has defects or not is judged according to the defect detection data, the classification of the wafer surface defects and the foreign matters can be realized, the accuracy of the classification of the wafer surface defects and the foreign matters is improved, and the manual re-inspection workload is reduced.
The embodiments described in the embodiments of the present application are for more clearly describing the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided by the embodiments of the present application, and as those skilled in the art can know that, with the evolution of technology and the appearance of new application scenarios, the technical solutions provided by the embodiments of the present application are equally applicable to similar technical problems.
It will be appreciated by those skilled in the art that the technical solutions shown in the figures do not constitute limitations of the embodiments of the present application, and may include more or fewer steps than shown, or may combine certain steps, or different steps.
The above described apparatus embodiments are merely illustrative, wherein the units illustrated as separate components may or may not be physically separate, i.e. may be located in one place, or may be distributed over a plurality of network elements. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
Those of ordinary skill in the art will appreciate that all or some of the steps of the methods, systems, functional modules/units in the devices disclosed above may be implemented as software, firmware, hardware, and suitable combinations thereof.
The terms "first," "second," "third," "fourth," and the like in the description of the present application and in the above-described figures, if any, are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate such that embodiments of the present application described herein may be implemented in sequences other than those illustrated or otherwise described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
It should be understood that in this application, "at least one" means one or more, and "a plurality" means two or more. "and/or" for describing the association relationship of the association object, the representation may have three relationships, for example, "a and/or B" may represent: only a, only B and both a and B are present, wherein a, B may be singular or plural. The character "/" generally indicates that the context-dependent object is an "or" relationship. "at least one of" or the like means any combination of these items, including any combination of single item(s) or plural items(s). For example, at least one (one) of a, b or c may represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", wherein a, b, c may be single or plural.
In the several embodiments provided in this application, it should be understood that the disclosed apparatus and method may be implemented in other ways. For example, the above-described apparatus embodiments are merely illustrative, and for example, the above-described division of units is merely a logical function division, and there may be another division manner in actual implementation, for example, a plurality of units or components may be combined or may be integrated into another system, or some features may be omitted, or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be an indirect coupling or communication connection via some interfaces, devices or units, which may be in electrical, mechanical or other form.
The units described above as separate components may or may not be physically separate, and components shown as units may or may not be physical units, may be located in one place, or may be distributed over a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
In addition, each functional unit in each embodiment of the present application may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units may be implemented in hardware or in software functional units.
The integrated units, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present application may be embodied essentially or in part or all of the technical solution or in part in the form of a software product stored in a storage medium, including multiple instructions to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods of the various embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (Random Access Memory, RAM), a magnetic disk, or an optical disk, or other various media capable of storing a program.
Preferred embodiments of the present application are described above with reference to the accompanying drawings, and thus do not limit the scope of the claims of the embodiments of the present application. Any modifications, equivalent substitutions and improvements made by those skilled in the art without departing from the scope and spirit of the embodiments of the present application shall fall within the scope of the claims of the embodiments of the present application.
Claims (10)
1. The wafer defect detection method is characterized by comprising the following steps:
acquiring a wafer sample image, wherein the wafer sample image is a color image;
performing image annotation on the wafer sample image to obtain image training data, wherein the image training data comprises the wafer sample image and an image tag;
training a neural network model obtained in advance based on the image training data to obtain a wafer defect detection model, wherein the neural network model is constructed according to a preset target detection network and an attention mechanism;
acquiring a wafer to be detected image to be detected, and obtaining image to be detected data;
and inputting the image to-be-detected data into the wafer defect detection model to detect defects, obtaining defect detection data, and judging whether the wafer to-be-detected image has defects according to the defect detection data.
2. The method according to claim 1, wherein the performing image labeling on the wafer sample image to obtain image training data includes:
marking the wafer sample image based on a preset marking tool to obtain the image tag, wherein the image tag comprises defect type and defect position information of the wafer sample image;
And obtaining the image training data based on the wafer sample image and the image tag.
3. The inspection method of claim 1, further comprising, prior to training a pre-acquired neural network model based on the image training data to obtain a wafer defect inspection model:
performing color space transformation on the wafer sample image in the image training data;
before the data to be detected of the image is input to the wafer defect detection model to detect defects, the detection method further comprises the following steps:
and carrying out color space transformation on the wafer to-be-detected image in the image to-be-detected data.
4. The inspection method according to claim 1, wherein the training the pre-acquired neural network model based on the image training data to obtain the wafer defect inspection model comprises:
performing defect detection on the image training data based on the neural network model to obtain sample detection data;
performing loss calculation on the neural network model based on a preset loss function, the sample detection data and the image tag to obtain model loss data, wherein the loss function is a GIOU function;
And updating parameters of the neural network model according to the model loss data to train the neural network model to obtain the wafer defect detection model.
5. The inspection method according to any one of claims 1 to 4, wherein the determining whether the image to be inspected of the wafer has a defect according to the defect inspection data includes:
if the image label corresponding to the wafer to-be-detected image in the defect detection data is not empty, the wafer to-be-detected image has a defect;
and if the image label corresponding to the wafer to-be-detected image in the defect detection data is empty, the wafer to-be-detected image has no defect.
6. The inspection method according to any one of claims 1 to 4, wherein the determining whether the image to be inspected of the wafer has a defect according to the defect inspection data includes:
if the image label corresponding to the wafer to be detected image in the defect detection data is not empty, extracting the confidence coefficient of each first defect in the wafer to be detected image from the defect detection data, and if the confidence coefficient of all the first defects is lower than a preset first threshold value, the wafer to be detected image has no defects; if the confidence coefficient of at least one first defect is not lower than the first threshold value, the image to be detected of the wafer is defective;
And if the image label corresponding to the wafer to-be-detected image in the defect detection data is empty, the wafer to-be-detected image has no defect.
7. The inspection method according to any one of claims 1 to 4, wherein the determining whether the image to be inspected of the wafer has a defect according to the defect inspection data includes:
if the image label corresponding to the wafer to-be-detected image in the defect detection data is not empty, extracting confidence and position information of each first defect in the wafer to-be-detected image from the defect detection data, and screening out a second defect from the first defects according to the position information and a second threshold; if the confidence degrees of all the second defects are lower than a preset first threshold value, the image to be detected of the wafer is free of defects; if the confidence coefficient of the at least one second defect is not lower than the first threshold value, the image to be detected of the wafer is defective;
and if the image label corresponding to the wafer to-be-detected image in the defect detection data is empty, the wafer to-be-detected image has no defect.
8. A wafer defect inspection apparatus, the apparatus comprising:
The sample image acquisition module is used for acquiring a wafer sample image, wherein the wafer sample image is a color image;
the image labeling module is used for carrying out image labeling on the wafer sample image to obtain image training data, wherein the image training data comprises the wafer sample image and an image label;
the training module is used for training a neural network model acquired in advance based on the image training data to obtain a wafer defect detection model, wherein the neural network model is constructed according to a preset target detection network and an attention mechanism;
the target image acquisition module is used for acquiring a wafer to be detected image to be detected to obtain image to-be-detected data;
and the defect detection module is used for inputting the image to-be-detected data into the wafer defect detection model to detect defects, obtaining defect detection data, and judging whether the wafer to-be-detected image has defects according to the defect detection data.
9. An electronic device comprising a memory storing a computer program and a processor that when executing the computer program implements the method of detecting a wafer defect according to any one of claims 1 to 7.
10. A computer readable storage medium storing a computer program, characterized in that the computer program, when executed by a processor, implements the method of detecting a wafer defect according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310365130.4A CN116542908A (en) | 2023-03-30 | 2023-03-30 | Wafer defect detection method and device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310365130.4A CN116542908A (en) | 2023-03-30 | 2023-03-30 | Wafer defect detection method and device, electronic equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116542908A true CN116542908A (en) | 2023-08-04 |
Family
ID=87453223
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310365130.4A Pending CN116542908A (en) | 2023-03-30 | 2023-03-30 | Wafer defect detection method and device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116542908A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117455897A (en) * | 2023-11-30 | 2024-01-26 | 魅杰光电科技(上海)有限公司 | Wafer scratch detection method, device, equipment and storage medium |
| CN117558645A (en) * | 2024-01-09 | 2024-02-13 | 武汉中导光电设备有限公司 | Big data Wafer defect determination method, device, equipment and storage medium |
| CN117558660A (en) * | 2024-01-10 | 2024-02-13 | 深圳市华拓半导体技术有限公司 | Management method and system for control workstation of semiconductor wafer detection equipment |
| CN117558660B (en) * | 2024-01-10 | 2024-04-12 | 深圳市华拓半导体技术有限公司 | Management method and system for control workstation of semiconductor wafer detection equipment |
-
2023
- 2023-03-30 CN CN202310365130.4A patent/CN116542908A/en active Pending
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117455897A (en) * | 2023-11-30 | 2024-01-26 | 魅杰光电科技(上海)有限公司 | Wafer scratch detection method, device, equipment and storage medium |
| CN117558645A (en) * | 2024-01-09 | 2024-02-13 | 武汉中导光电设备有限公司 | Big data Wafer defect determination method, device, equipment and storage medium |
| CN117558660A (en) * | 2024-01-10 | 2024-02-13 | 深圳市华拓半导体技术有限公司 | Management method and system for control workstation of semiconductor wafer detection equipment |
| CN117558660B (en) * | 2024-01-10 | 2024-04-12 | 深圳市华拓半导体技术有限公司 | Management method and system for control workstation of semiconductor wafer detection equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN116542908A (en) | Wafer defect detection method and device, electronic equipment and storage medium | |
| CN110059694B (en) | Intelligent identification method for character data in complex scene of power industry | |
| CN109002824B (en) | OpenCV-based building drawing label information detection method | |
| CN109977191B (en) | Problem map detection method, device, electronic equipment and medium | |
| CN112381788B (en) | Part surface defect increment detection method based on double-branch matching network | |
| CN110929607A (en) | Remote sensing identification method and system for urban building construction progress | |
| CN110826429A (en) | Scenic spot video-based method and system for automatically monitoring travel emergency | |
| CN109993138A (en) | A kind of car plate detection and recognition methods and device | |
| US10373316B2 (en) | Images background subtraction for dynamic lighting scenarios | |
| CN114169381A (en) | Image annotation method and device, terminal equipment and storage medium | |
| CN109977750A (en) | Seal true and false method of calibration, device and computer readable storage medium | |
| CN114581419A (en) | Transformer insulating sleeve defect detection method, related equipment and readable storage medium | |
| CN111461260A (en) | Target detection method, device and equipment based on feature fusion and storage medium | |
| CN116311205A (en) | License plate recognition method, license plate recognition device, electronic equipment and storage medium | |
| CN114627435B (en) | Intelligent light adjusting method, device, equipment and medium based on image recognition | |
| CN110674827A (en) | Equipment state visual detection method integrating deep learning and morphological filtering | |
| CN116129100A (en) | Truck part positioning and fault identifying method, device, equipment and medium | |
| CN115760854A (en) | Deep learning-based power equipment defect detection method and device and electronic equipment | |
| CN111931721B (en) | Method and device for detecting color and number of annual inspection label and electronic equipment | |
| CN112784189A (en) | Method and device for identifying page image | |
| CN111950475A (en) | Yalhe histogram enhancement type target recognition algorithm based on yoloV3 | |
| CN112131418A (en) | Target labeling method, target labeling device and computer-readable storage medium | |
| CN117315368B (en) | Intelligent operation and maintenance inspection method for large-scale data center | |
| CN114821484B (en) | Airport runway FOD image detection method, system and storage medium | |
| CN115424095B (en) | Quality analysis method and device based on waste materials |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |