CN118038151B - Wafer defect classification detection method, system, electronic equipment and storage medium - Google Patents

Wafer defect classification detection method, system, electronic equipment and storage medium Download PDF

Info

Publication number
CN118038151B
CN118038151B CN202410184264.0A CN202410184264A CN118038151B CN 118038151 B CN118038151 B CN 118038151B CN 202410184264 A CN202410184264 A CN 202410184264A CN 118038151 B CN118038151 B CN 118038151B
Authority
CN
China
Prior art keywords
wafer
defect
result
classification
point cloud
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202410184264.0A
Other languages
Chinese (zh)
Other versions
CN118038151A (en
Inventor
彭广德
王睿
李卫铳
李卫燊
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Guangzhou Ligong Industrial Co ltd
Original Assignee
Guangzhou Ligong Industrial Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Guangzhou Ligong Industrial Co ltd filed Critical Guangzhou Ligong Industrial Co ltd
Priority to CN202410184264.0A priority Critical patent/CN118038151B/en
Publication of CN118038151A publication Critical patent/CN118038151A/en
Application granted granted Critical
Publication of CN118038151B publication Critical patent/CN118038151B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
    • G06F16/58—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
    • G06F16/5866—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using information manually generated, e.g. tags, keywords, comments, manually generated location and time information
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0464—Convolutional networks [CNN, ConvNet]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0475—Generative networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/094—Adversarial learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0004—Industrial image inspection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30108—Industrial image inspection
    • G06T2207/30148—Semiconductor; IC; Wafer
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
    • Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
    • Y02P90/30—Computing systems specially adapted for manufacturing

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Evolutionary Computation (AREA)
  • Computing Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Software Systems (AREA)
  • General Health & Medical Sciences (AREA)
  • Data Mining & Analysis (AREA)
  • General Engineering & Computer Science (AREA)
  • Biophysics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Computational Linguistics (AREA)
  • Mathematical Physics (AREA)
  • Biomedical Technology (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Databases & Information Systems (AREA)
  • Molecular Biology (AREA)
  • Medical Informatics (AREA)
  • Multimedia (AREA)
  • Library & Information Science (AREA)
  • Quality & Reliability (AREA)
  • Investigating Materials By The Use Of Optical Means Adapted For Particular Applications (AREA)
  • Testing Or Measuring Of Semiconductors Or The Like (AREA)

Abstract

The application provides a wafer defect classification detection method, a wafer defect classification detection system, electronic equipment and a storage medium. The method comprises the following steps: firstly, three-dimensional point cloud data and a single-pixel image of a wafer are acquired; then performing defect classification detection on the three-dimensional point cloud data according to the three-dimensional point cloud defect classification detection model to obtain a point cloud classification result; the point cloud classification result comprises the defect type, the defect position and the defect size of the wafer; then, carrying out feature retrieval on the single-pixel image to obtain a feature retrieval result; the feature retrieval result comprises the defect type of the wafer; performing defect classification detection on the single-pixel image according to the wafer multi-label classification detection model to obtain a multi-label classification result; the multi-label classification result comprises multi-class defects and probabilities of the wafer; and finally, weighting the point cloud classification result, the feature retrieval result and the multi-label classification result to obtain a wafer defect classification detection result, so that the accuracy and the efficiency of wafer defect classification can be improved.

Description

Wafer defect classification detection method, system, electronic equipment and storage medium
Technical Field
The present application relates to the field of semiconductor inspection technologies, and in particular, to a wafer defect classification inspection method, a wafer defect classification inspection system, an electronic device, and a storage medium.
Background
In actual production, defects on the surface of the wafer are various, the shape is uneven, and the difficulty of detecting the defects of the wafer is increased. Among the types of wafer defects, no-pattern wafer defects and patterned wafer defects are two major forms of wafer defects, which are the major causes of chip failure. Defects on the wafer surface not only affect the aesthetics and comfort of the product, but also can have adverse effects on its performance. Therefore, surface defect detection of wafer products is highly valued, and accurate prediction of wafer yield is of great importance for improving the production process and reducing wafer manufacturing loss.
However, the quality inspection of the wafer still needs to be performed manually in a large amount, and the defects on the surface need to be observed and found out under high light for a long time, so that the quality inspection efficiency and quality are low, and the cost is high. In addition, considering the distribution of the probability density of defects, the related technology also indicates that an image detection algorithm can be used for detecting the defects of the wafer, but the traditional image detection algorithm generally adopts a CCD or CMOS sensor to collect the information of the wafer, so that the requirement of intelligent quality inspection of the current defects cannot be met, and the accuracy is low.
Disclosure of Invention
The embodiment of the application mainly aims to provide a wafer defect classification detection method, a system, electronic equipment and a storage medium, which can improve the efficiency and accuracy of wafer defect classification detection.
In order to achieve the above object, an aspect of an embodiment of the present application provides a wafer defect classification detection method, which includes:
acquiring three-dimensional point cloud data and a single-pixel image of a wafer;
Performing defect classification detection on the three-dimensional point cloud data according to a three-dimensional point cloud defect classification detection model to obtain a point cloud classification result; the point cloud classification result comprises a defect type, a defect position and a defect size of the wafer;
performing feature retrieval on the single-pixel image to obtain a feature retrieval result; wherein, the characteristic search result comprises the defect type of the wafer;
performing defect classification detection on the single-pixel image according to a wafer multi-label classification detection model to obtain a multi-label classification result; the multi-label classification result comprises multi-class defects and probabilities of the wafer;
And weighting the point cloud classification result, the feature retrieval result and the multi-label classification result to obtain a wafer defect classification detection result.
In some embodiments, the acquiring three-dimensional point cloud data and single-pixel images of a wafer includes the steps of:
acquiring a wafer according to a three-dimensional camera to obtain three-dimensional point cloud data;
And carrying out normalization and projection conversion on the three-dimensional point cloud data to obtain a single-pixel image.
In some embodiments, the acquiring three-dimensional point cloud data and single-pixel images of a wafer includes the steps of:
acquiring a wafer according to a three-dimensional camera to obtain three-dimensional point cloud data;
And carrying out normalization and projection conversion on the three-dimensional point cloud data to obtain a single-pixel image.
In some embodiments, the wafer multi-label classification detection model is obtained by:
acquiring an information set of a wafer; wherein the information set of the wafer comprises physical characteristics and process parameters of the wafer;
constructing a convolutional neural network model according to the information set of the wafer; the convolutional neural network model is used for carrying out defect classification detection on the wafer;
and performing parameter tuning on the convolutional neural network model according to the fourth data set until the convolutional neural network model converges to obtain the wafer multi-label classification detection model.
In some embodiments, the fourth data set is obtained by:
Performing defect class labeling on the single-pixel image according to the multi-label classification labeling standard to respectively obtain defect class labels of all wafers;
forming a first data set by the single-pixel image and defect category marks of each wafer, and forming a second data set by the three-dimensional point cloud data;
Selecting a certain number of first data sets and second data sets, and generating new data sets from the selected first data sets and second data sets through the countermeasure generation network;
updating and synthesizing defect category labels of the new data set, and forming a third data set by the new data set and the updated defect category labels;
and merging the first data set and the third data set to obtain a fourth data set.
In some embodiments, the feature retrieval is performed on the single-pixel image to obtain a feature retrieval result, and the method includes the following steps:
Extracting features of the single-pixel image according to a backbone network in the wafer multi-label classification detection model to obtain modal features of the wafer;
And searching in a defect template library according to the modal characteristics to obtain characteristic search results.
In some embodiments, the searching in the defect template library according to the modal feature to obtain a feature searching result includes the following steps:
acquiring a first keyword of the modal feature and a second keyword of the defect template library;
A plurality of first keywords and second keywords are taken out, and the first keywords and the second keywords are combined to obtain a feature set;
Respectively calculating word frequencies of the modal features and the defect template library in the feature set to obtain a first word frequency vector corresponding to the modal features and a second word frequency vector corresponding to the defect template library;
And calculating cosine similarity of the first word frequency vector and the second word frequency vector to obtain a feature retrieval result.
In some embodiments, the weighting the point cloud classification result, the feature retrieval result and the multi-label classification result to obtain a wafer defect classification detection result includes the following steps:
Acquiring a first weight parameter of the point cloud classification result, a second weight parameter of the feature retrieval result and a third weight parameter of the multi-label classification result;
obtaining a first classification detection result according to the point cloud classification result and the first weight parameter;
Obtaining a second classification detection result according to the characteristic retrieval result and the second weight parameter;
obtaining a third classification detection result according to the multi-label classification result and the third weight parameter;
and adding the first classification detection result, the second classification detection result and the third classification detection result to obtain a wafer defect classification detection result.
To achieve the above object, another aspect of the embodiments of the present application provides a wafer defect classification and inspection system, including:
the first module is used for acquiring three-dimensional point cloud data and single-pixel images of the wafer;
The second module is used for carrying out defect classification detection on the three-dimensional point cloud data according to the three-dimensional point cloud defect classification detection model to obtain a point cloud classification result; the point cloud classification result comprises a defect type, a defect position and a defect size of the wafer;
the third module is used for carrying out feature retrieval on the single-pixel image to obtain a feature retrieval result; wherein, the characteristic search result comprises the defect type of the wafer;
A fourth module, configured to perform defect classification detection on the single-pixel image according to a wafer multi-label classification detection model, to obtain a multi-label classification result; the multi-label classification result comprises multi-class defects and probabilities of the wafer;
and a fifth module, configured to weight the point cloud classification result, the feature retrieval result and the multi-label classification result to obtain a wafer defect classification detection result.
To achieve the above object, another aspect of the embodiments of the present application provides an electronic device, including a memory and a processor, where the memory stores a computer program, and the processor implements the method described above when executing the computer program.
To achieve the above object, another aspect of the embodiments of the present application proposes a computer-readable storage medium storing a computer program which, when executed by a processor, implements the method described above.
The embodiment of the application at least comprises the following beneficial effects: the application provides a wafer defect classification detection method, a system, electronic equipment and a storage medium, according to the scheme, the wafer multi-label classification detection model, the three-dimensional point cloud defect classification detection model and the feature retrieval are combined to carry out defect classification detection on the wafer, so that the accuracy of wafer defect classification can be improved. In addition, through the detection of the wafer multi-label classification detection model, a multi-label classification result can be provided, so that the richness of wafer defect detection is improved.
Drawings
FIG. 1 is a flowchart of a wafer defect classification detection method according to an embodiment of the present application;
Fig. 2 is a flowchart of step S101 in fig. 1;
Fig. 3 is a flowchart of step S104 in fig. 1;
fig. 4 is a flowchart of step S303 in fig. 1;
fig. 5 is a flowchart of step S103 in fig. 1;
Fig. 6 is a flowchart of step S502 in fig. 1;
Fig. 7 is a flowchart of step S105 in fig. 1;
fig. 8 is a schematic diagram of a hardware structure of an electronic device according to an embodiment of the present application;
FIG. 9 is a schematic diagram of an exemplary structure of a wafer multi-label classification test model in accordance with an embodiment of the present application;
Fig. 10 is an exemplary diagram of converting three-dimensional point cloud data into a single-pixel image according to an embodiment of the present application.
Detailed Description
The present application will be described in further detail with reference to the drawings and examples, in order to make the objects, technical solutions and advantages of the present application more apparent. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the application. When the following description refers to the accompanying drawings, the same numbers in different drawings refer to the same or similar elements, unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with embodiments of the application, but are merely examples of systems and methods consistent with aspects of embodiments of the application as detailed in the accompanying claims.
It is to be understood that the terms "first," "second," and the like, as used herein, may be used to describe various concepts, but are not limited by these terms unless otherwise specified. These terms are only used to distinguish one concept from another. For example, the first information may also be referred to as second information, and similarly, the second information may also be referred to as first information, without departing from the scope of embodiments of the present application. The words "if", as used herein, may be interpreted as "at … …" or "at … …" or "in response to a determination", depending on the context.
Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of the application only and is not intended to be limiting of the application.
Referring to fig. 1, an alternative flowchart of a method for classifying wafer defects according to an embodiment of the present application is shown, where the method in fig. 1 may include, but is not limited to, steps S101 to S105.
Step S101, three-dimensional point cloud data and a single-pixel image of a wafer are obtained;
step S102, performing defect classification detection on three-dimensional point cloud data according to a three-dimensional point cloud defect classification detection model to obtain a point cloud classification result; the point cloud classification result comprises the defect type, the defect position and the defect size of the wafer;
Step S103, carrying out feature retrieval on the single-pixel image to obtain a feature retrieval result; the feature retrieval result comprises the defect type of the wafer;
Step S104, performing defect classification detection on the single-pixel image according to the wafer multi-label classification detection model to obtain a multi-label classification result; the multi-label classification result comprises multi-class defects and probabilities of the wafer;
step S105, weighting the point cloud classification result, the feature retrieval result and the multi-label classification result to obtain a wafer defect classification detection result.
In step S101 of some embodiments, the single-pixel image refers to a two-dimensional single-pixel image obtained after normalization and projection conversion processing on the acquired three-dimensional point cloud data.
In step S102 of some embodiments, the three-dimensional point cloud defect classification detection model is constructed by the following steps: constructing a multi-layer neural network and training by using a large amount of point cloud data (point cloud data of the existing wafer); in the training process, the constructed neural network learns how to extract useful features from the point cloud data and how to judge defects on the surface of the wafer according to the features; and finally, obtaining the three-dimensional point cloud defect classification detection model. Further, the obtained three-dimensional point cloud data are input into a constructed three-dimensional point cloud defect classification detection model, so that the wafer defects can be automatically identified, and a point cloud classification result is obtained.
Illustratively, the point cloud classification result may be expressed as: (class, x, z, w, h, l). Wherein class represents the defect class; (x, y, z) represents a defect position; (w, h, l) represents defect size, i.e. w represents width, h represents height, and l represents length.
In step S103 of some embodiments, feature retrieval refers to retrieving features extracted from a single-pixel image to obtain a feature retrieval result of a wafer (i.e., a defect type of the wafer). Illustratively, the feature retrieval results may be expressed as: (class). Wherein class represents the defect class.
In step S104 of some embodiments, the wafer multi-label classification detection model is constructed by the following steps: and constructing a convolutional neural network model according to the information set (namely the WM-811K data set) of the wafer, training the information set of the wafer, and finally obtaining the wafer multi-label classification detection model. Further, inputting the acquired single-pixel image into a constructed wafer multi-label classification detection model to obtain a multi-label classification result.
Illustratively, the multi-label classification result may be expressed as: (class 1, class 2.; p1, p 2.).
Wherein class1 represents a first defect class, class2 represents a second defect class, p1 represents a probability corresponding to the first defect class, and p2 represents a probability corresponding to the second defect class.
In step S105 of some embodiments, the wafer defect classification detection result includes a maximum defect class and a multi-label class of the wafer. Illustratively, the wafer defect classification detection result may be expressed as; (class, class1, class 2).
In the steps S101 to S105 shown in the embodiment of the present application, the defect classification detection is performed on the wafer by combining the wafer multi-label classification detection model, the three-dimensional point cloud defect classification detection model and the feature search, so that the accuracy of the wafer defect classification can be improved. In addition, through the detection of the wafer multi-label classification detection model, a multi-label classification result can be provided, so that the richness of wafer defect detection is improved.
Referring to fig. 2, in some embodiments, step S101 includes, but is not limited to, steps S201 to S202, where three-dimensional point cloud data and single-pixel images of a wafer are acquired:
Step S201, collecting a wafer according to a three-dimensional camera to obtain three-dimensional point cloud data;
Step S202, carrying out normalization and projection conversion on the three-dimensional point cloud data to obtain a single-pixel image.
In step S201 of some embodiments, three-dimensional cameras include, but are not limited to, TOF cameras, binocular vision cameras, and structured light depth cameras, as examples, the present invention is not particularly limited.
The embodiment of the invention takes a TOF camera as an example to describe the process of acquiring three-dimensional point cloud data:
firstly, continuous near infrared pulses are transmitted to a wafer, then pulse waves reflected by the wafer are received through a sensor, and finally, the flight (round trip) time of the light pulses is calculated, so that the distance between the wafer and a TOF camera is obtained, and three-dimensional point cloud data can be obtained.
In step S202 of some embodiments, please refer to fig. 10, normalization refers to converting three-dimensional point cloud data from an original coordinate system to a unified coordinate system, so as to facilitate subsequent data processing and analysis. In the embodiment of the invention, p (a set of all coordinate points of a three-dimensional point cloud point) is projected onto a two-dimensional plane by using the points coordinates and a depth value d (single pixel assignment is performed after normalization), the projected point is expressed as p and the mismatch position between the sparse point and the two-dimensional plane is set as 0. The projected feature image may be expressed as: { p x, y| (x, y) ∈D }, where D is a two-dimensional planar region of the image, x is a horizontal axis coordinate, y is a vertical axis coordinate, and (x, y) is a coordinate point. After normalization and projective transformation, a single-pixel image as shown in fig. 10 can be obtained. The single pixel image may be an image of 1×512×512 specification, for example.
In the steps S201 to S202 shown in the embodiment of the present application, three-dimensional point cloud data is converted into a single-pixel image, so that a wafer multi-label classification detection model can be conveniently constructed subsequently, thereby effectively improving the accuracy of wafer defect detection, and providing a multi-label classification result to improve the richness of defect detection.
Referring to fig. 3, in some embodiments, the wafer multi-label classification detection model in step S104 may be obtained by the following steps, including but not limited to steps S301 to S303:
step S301, acquiring an information set of a wafer; wherein the information set of the wafer comprises physical characteristics and process parameters of the wafer;
Step S302, constructing a convolutional neural network model according to an information set of a wafer; the convolutional neural network model is used for carrying out defect classification detection on the wafer;
And step S303, performing parameter tuning on the convolutional neural network model according to the fourth data set until the convolutional neural network model converges to obtain the wafer multi-label classification detection model.
In step S301 of some embodiments, the information set of the wafer may be, for example, a wafer WM-811K data set. The wafer WM-811K dataset is a dataset used in semiconductor manufacturing and research that gathers a large amount of information about the wafer, including the physical characteristics and process parameters of the wafer. Specifically, the method comprises physical characteristics such as size, shape, material, thickness and the like of the wafer, and process parameters such as temperature, pressure, atmosphere, uniformity of the film and the like in the manufacturing process.
In steps S302 to S303 of some embodiments, the construction of the wafer multi-label classification detection model may be obtained by:
s1, preprocessing and normalizing an information set of a wafer to obtain a training set, a verification set and a test set.
S2, establishing a convolutional neural network model for performing defect classification detection on the wafer.
S3, inputting the training set into the convolutional neural network model to perform learning training, and performing verification by using a corresponding verification set.
And S4, testing the trained convolutional neural network model according to the test set, and generating a test result.
S5, comparing the test result with the real result in the test set, determining the accuracy of the test result according to the comparison result, and performing parameter tuning on the convolutional neural network model through the fourth data set to improve the accuracy of the convolutional neural network model until the accuracy of the evaluation index of the test set reaches 90%, so as to obtain the wafer multi-label classification detection model. The training set is used for training the convolutional neural network model, so that the trained convolutional neural network model can identify the defects of the wafer; the test set is used for testing the trained convolutional neural network model so as to verify the accuracy of the identification of the trained convolutional neural network model; the fourth data set is used for performing parameter tuning on the trained convolutional neural network model so that the trained convolutional neural network model can accurately identify the defects of the wafer, namely, the accuracy of model identification is improved.
In addition, the convolutional neural network model includes an input layer, a hidden layer, and an output layer. The wafer multi-label classification detection model constructed in the embodiment of the invention can refer to fig. 9.
In the steps S301 to S303 shown in the embodiment of the application, the convolutional neural network model of the wafer can be established by training and verifying the WM-811K data set of the wafer so as to predict the performance and quality of the wafer, thereby improving the quality of the wafer product. And meanwhile, the convolutional neural network model is optimized through the fourth data set, so that the accuracy of the convolutional neural network model (namely the wafer multi-label classification detection model) is improved.
Referring to fig. 4, in some embodiments, the fourth data set in step S303 may be obtained by the following steps, including but not limited to step S401 to step S405:
Step S401, marking the defect type of the single pixel image according to the multi-label classification marking standard, and respectively obtaining defect type marks of each wafer;
Step S402, forming a first data set by the single pixel image and defect type marks of each wafer, and forming a second data set by the three-dimensional point cloud data;
Step S403, selecting a certain number of first data sets and second data sets, and generating new data sets from the selected first data sets and second data sets through the countermeasure generation network;
Step S404, updating and synthesizing defect type marks of the new data set, and forming a third data set by the new data set and the updated defect type marks;
step S405, merging the first data set and the third data set to obtain a fourth data set.
In step S401 of some embodiments, illustratively, defect class labels include, but are not limited to, defect shape, defect volume, defect size, and scratches.
In step S403 of some embodiments, the countermeasure generation network may be GAN, for example, and the present invention is not particularly limited. By synthesizing new data sets against the generation network (i.e., by randomly changing the size, direction, color saturation, etc., generating a wafer map), a balance of numbers between each defect type sample of the wafer can be maintained.
In steps S404 to S405 of some embodiments, defect class labels are performed on the synthesized new data set by the multi-label classification labeling criteria, resulting in new defect class labels. Then, the original defect class mark is updated according to the new defect class mark, and the new data set and the updated defect class mark form a third data set. And finally, merging the first data set and the third data set to obtain a fourth data set. Illustratively, merging refers to merging images of the same type or defect class marks of the same type in the first data set and the third data set, i.e., similar to stacking two identical sheets.
In the steps S401 to S405 shown in the embodiment of the present application, the accuracy of the wafer multi-label classification detection model can be improved by performing operations such as labeling on the single-pixel image and synthesizing a new data set using the countermeasure generation network to obtain a fourth data set for parameter tuning.
Referring to fig. 5, in some embodiments, the feature retrieval is performed on the single-pixel image in step S103 to obtain a feature retrieval result, which may include, but is not limited to, steps S501 to S502:
step S501, extracting features of single-pixel images according to a backbone network in a wafer multi-label classification detection model to obtain modal features of a wafer;
Step S502, searching in a defect template library according to the modal characteristics to obtain characteristic searching results.
In step S501 of some embodiments, the backbone network may be convnextv, for example. The modal characteristics of the single-pixel image can be extracted through convnextv < 2 >, so that the defect type retrieval can be conveniently carried out subsequently. The modal feature refers to a feature vector of a defect type in the single-pixel image.
In the steps S501 to S502 shown in the embodiment of the application, the mode features extracted from the single-pixel images are searched and compared by using the defect template library, so that the accuracy of classifying and detecting the wafer defects can be improved.
Referring to fig. 6, in some embodiments, in step S502, the feature retrieval result is obtained by retrieving the feature from the defect template library according to the modal feature, which may include, but is not limited to, steps S601 to S604:
step S601, obtaining a first keyword of a modal feature and a second keyword of a defect template library;
Step S602, a plurality of first keywords and second keywords are taken out, and the first keywords and the second keywords are combined to obtain a feature set;
step S603, calculating word frequencies of the modal features and the defect template library in the feature set respectively to obtain a first word frequency vector corresponding to the modal features and a second word frequency vector corresponding to the defect template library;
Step S604, calculating cosine similarity of the first word frequency vector and the second word frequency vector to obtain a feature retrieval result.
In step S603 of some embodiments, the term frequency refers to the number of occurrences of a term in the feature set, and the more occurrences, the more important the term is explained, i.e. the higher the likelihood of its defect class. The first term frequency vector refers to the number of times a first keyword appears in the feature set, and the second term frequency vector refers to the number of times a second keyword appears in the feature set.
In step S604 of some embodiments, the formula for calculating cosine similarity may be expressed as:
Where x represents a first word frequency vector, y represents a second word frequency vector, n represents a number, x i represents an i-th first word frequency vector, and y i represents an i-th second word frequency vector.
Specifically, when the cosine value is close to 1 and the included angle is close to 0, the more similar the two vectors are (namely, the most possible defect class); the cosine value is close to 0 and the angle tends to 90 °, the more dissimilar the two vectors are.
By taking any two texts as an example, the flow of the feature retrieval result in the embodiment of the invention is described:
S1, finding out keywords corresponding to a text a and a text b;
Text a: this/defect/shape/circle.
Text b: this/defect/type/score/scratch.
Namely, the first keyword is: this, defect, shape, and circle; the second key words are: this, defect, type, score and scratch.
S2, combining the first keywords and the second keywords to obtain a feature set.
Namely, the feature set is as follows: this, defect, shape, be, circle, type and scratch.
S3, calculating word frequencies of the text a and the text b in the feature set respectively to obtain a first word frequency vector and a second word frequency vector.
Text a: 1 of this 1, defect 1, shape 1, circle 1, type 0, scratch 0;
Text b: 1, defect 1, shape 0, 1, circle 0, type 1, scratch 1.
Namely, the first word frequency vector is: (1,1,1,1,1,1,0,0); the second word frequency vector is: (1,1,1,0,1,0,1,1).
And S4, calculating cosine similarity of the first word frequency vector and the second word frequency vector to obtain a feature retrieval result.
Substituting the first word frequency vector and the second word frequency vector into a cosine similarity formula, and obtaining a feature retrieval result.
In steps S601 to S604 shown in the embodiment of the present application, the similarity information between the defect template library and the single-pixel image is searched through cosine similarity matching to output the most possible defect type (i.e. feature search result), so that the accuracy of wafer defect classification detection can be improved.
Referring to fig. 7, in some embodiments, the weighting of the point cloud classification result, the feature retrieval result and the multi-label classification result in step S105 to obtain the wafer defect classification detection result may include, but is not limited to, steps S701 to S705:
Step S701, acquiring a first weight parameter of a point cloud classification result, a second weight parameter of a feature retrieval result and a third weight parameter of a multi-label classification result;
step S702, obtaining a first classification detection result according to a point cloud classification result and a first weight parameter;
Step S703, obtaining a second classification detection result according to the feature retrieval result and the second weight parameter;
Step S704, obtaining a third classification detection result according to the multi-label classification result and the third weight parameter;
step S705, adding the first classification detection result, the second classification detection result and the third classification detection result to obtain a wafer defect classification detection result.
In steps S701 to S705 of some embodiments, the wafer defect classification detection result is obtained by the following expression:
Wherein Y represents the wafer defect classification detection result, w 1 represents the first weight parameter, w 2 represents the second weight parameter, w 3 represents the third weight parameter, The result of the classification of the point cloud is represented,The result of the feature retrieval is represented,The multi-label classification result is represented,A first classification test result is indicated,A second classification test result is indicated,And represents the third classification detection result.
In the steps S701 to S705 shown in the embodiment of the present application, the accuracy of classifying wafer defects can be greatly improved by weighting the point cloud classification result, the feature retrieval result and the multi-label classification result.
The embodiment of the application also provides a wafer defect classification detection system, which can realize the method, and comprises the following steps:
the first module is used for acquiring three-dimensional point cloud data and single-pixel images of the wafer;
The second module is used for carrying out defect classification detection on the three-dimensional point cloud data according to the three-dimensional point cloud defect classification detection model to obtain a point cloud classification result; the point cloud classification result comprises the defect type, the defect position and the defect size of the wafer;
The third module is used for carrying out feature retrieval on the single-pixel image to obtain a feature retrieval result; the feature retrieval result comprises the defect type of the wafer;
A fourth module, configured to perform defect classification detection on the single-pixel image according to the wafer multi-label classification detection model, to obtain a multi-label classification result; the multi-label classification result comprises multi-class defects and probabilities of the wafer;
and a fifth module, configured to weight the point cloud classification result, the feature retrieval result and the multi-label classification result to obtain a wafer defect classification detection result.
It can be understood that the content in the above method embodiment is applicable to the system embodiment, and the functions specifically implemented by the system embodiment are the same as those of the above method embodiment, and the achieved beneficial effects are the same as those of the above method embodiment.
The embodiment of the application also provides electronic equipment, which comprises a memory and a processor, wherein the memory stores a computer program, and the processor realizes the wafer defect classification detection method when executing the computer program. The electronic equipment can be any intelligent terminal including a tablet personal computer, a vehicle-mounted computer and the like.
It can be understood that the content in the above method embodiment is applicable to the embodiment of the present apparatus, and the specific functions implemented by the embodiment of the present apparatus are the same as those of the embodiment of the above method, and the achieved beneficial effects are the same as those of the embodiment of the above method.
Referring to fig. 8, fig. 8 illustrates a hardware structure of an electronic device according to another embodiment, the electronic device includes:
The processor 801 may be implemented by a general-purpose CPU (central processing unit), a microprocessor, an application-specific integrated circuit (ApplicationSpecificIntegratedCircuit, ASIC), or one or more integrated circuits, etc. for executing related programs to implement the technical solution provided by the embodiments of the present application;
Memory 802 may be implemented in the form of read-only memory (ReadOnlyMemory, ROM), static storage, dynamic storage, or random access memory (RandomAccessMemory, RAM), among others. The memory 802 may store an operating system and other application programs, and when the technical solutions provided in the embodiments of the present disclosure are implemented by software or firmware, relevant program codes are stored in the memory 802, and the processor 801 invokes a wafer defect classification detection method for executing the embodiments of the present disclosure;
an input/output interface 803 for implementing information input and output;
The communication interface 804 is configured to implement communication interaction between the device and other devices, and may implement communication in a wired manner (e.g., USB, network cable, etc.), or may implement communication in a wireless manner (e.g., mobile network, WIFI, bluetooth, etc.);
A bus 805 that transfers information between the various components of the device (e.g., the processor 801, the memory 802, the input/output interface 803, and the communication interface 804);
wherein the processor 801, the memory 802, the input/output interface 803, and the communication interface 804 implement communication connection between each other inside the device through a bus 805.
The embodiment of the application also provides a computer readable storage medium, wherein the computer readable storage medium stores a computer program, and the computer program realizes the wafer defect classification detection method when being executed by a processor.
It can be understood that the content of the above method embodiment is applicable to the present storage medium embodiment, and the functions of the present storage medium embodiment are the same as those of the above method embodiment, and the achieved beneficial effects are the same as those of the above method embodiment.
The memory, as a non-transitory computer readable storage medium, may be used to store non-transitory software programs as well as non-transitory computer executable programs. In addition, the memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one magnetic disk storage device, flash memory device, or other non-transitory solid state storage device. In some embodiments, the memory optionally includes memory remotely located relative to the processor, the remote memory being connectable to the processor through a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The wafer defect classification detection method, the wafer defect classification detection system, the electronic equipment and the storage medium provided by the embodiment of the application acquire three-dimensional point cloud data and single-pixel images of a wafer; then performing defect classification detection on the three-dimensional point cloud data according to the three-dimensional point cloud defect classification detection model to obtain a point cloud classification result; the point cloud classification result comprises the defect type, the defect position and the defect size of the wafer; then, carrying out feature retrieval on the single-pixel image to obtain a feature retrieval result; the feature retrieval result comprises the defect type of the wafer; performing defect classification detection on the single-pixel image according to the wafer multi-label classification detection model to obtain a multi-label classification result; the multi-label classification result comprises multi-class defects and probabilities of the wafer; and finally, weighting the point cloud classification result, the feature retrieval result and the multi-label classification result to obtain a wafer defect classification detection result, so that the accuracy and the efficiency of wafer defect classification can be improved.
The embodiments described in the embodiments of the present application are for more clearly describing the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided by the embodiments of the present application, and those skilled in the art can know that, with the evolution of technology and the appearance of new application scenarios, the technical solutions provided by the embodiments of the present application are equally applicable to similar technical problems.
It will be appreciated by persons skilled in the art that the embodiments of the application are not limited by the illustrations, and that more or fewer steps than those shown may be included, or certain steps may be combined, or different steps may be included.
Those of ordinary skill in the art will appreciate that all or some of the steps of the methods, systems, functional modules/units in the devices disclosed above may be implemented as software, firmware, hardware, and suitable combinations thereof.
The terms "first," "second," "third," "fourth," and the like in the description of the application and in the above figures, if any, are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate such that the embodiments of the application described herein may be implemented in sequences other than those illustrated or otherwise described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
The preferred embodiments of the present application have been described above with reference to the accompanying drawings, and are not thereby limiting the scope of the claims of the embodiments of the present application. Any modifications, equivalent substitutions and improvements made by those skilled in the art without departing from the scope and spirit of the embodiments of the present application shall fall within the scope of the claims of the embodiments of the present application.

Claims (6)

1. A wafer defect classification detection method, the method comprising the steps of:
acquiring a wafer according to a three-dimensional camera to obtain three-dimensional point cloud data;
Normalizing and projective converting the three-dimensional point cloud data to obtain a single-pixel image;
Performing defect classification detection on the three-dimensional point cloud data according to a three-dimensional point cloud defect classification detection model to obtain a point cloud classification result; the point cloud classification result comprises a defect type, a defect position and a defect size of the wafer;
Extracting features of the single-pixel image according to a backbone network in the wafer multi-label classification detection model to obtain modal features of the wafer;
Searching in a defect template library according to the modal characteristics to obtain characteristic searching results, wherein the method comprises the following steps:
acquiring a first keyword of the modal feature and a second keyword of the defect template library;
A plurality of first keywords and second keywords are taken out, and the first keywords and the second keywords are combined to obtain a feature set;
Respectively calculating word frequencies of the modal features and the defect template library in the feature set to obtain a first word frequency vector corresponding to the modal features and a second word frequency vector corresponding to the defect template library;
Calculating cosine similarity of the first word frequency vector and the second word frequency vector to obtain a feature retrieval result;
wherein, the characteristic search result comprises the defect type of the wafer;
performing defect classification detection on the single-pixel image according to a wafer multi-label classification detection model to obtain a multi-label classification result; the multi-label classification result comprises multi-class defects and probabilities of the wafer;
And weighting the point cloud classification result, the feature retrieval result and the multi-label classification result to obtain a wafer defect classification detection result.
2. The method of claim 1, wherein the wafer multi-label classification test model is obtained by:
acquiring an information set of a wafer; wherein the information set of the wafer comprises physical characteristics and process parameters of the wafer;
constructing a convolutional neural network model according to the information set of the wafer; the convolutional neural network model is used for carrying out defect classification detection on the wafer;
performing parameter tuning on the convolutional neural network model according to a fourth data set until the convolutional neural network model converges to obtain a wafer multi-label classification detection model;
Wherein the fourth dataset is obtained by:
Performing defect class labeling on the single-pixel image according to the multi-label classification labeling standard to respectively obtain defect class labels of all wafers;
forming a first data set by the single-pixel image and defect category marks of each wafer, and forming a second data set by the three-dimensional point cloud data;
Selecting a certain number of first data sets and second data sets, and generating new data sets from the selected first data sets and second data sets through the countermeasure generation network;
updating and synthesizing defect category labels of the new data set, and forming a third data set by the new data set and the updated defect category labels;
and merging the first data set and the third data set to obtain a fourth data set.
3. The method according to claim 1, wherein the weighting the point cloud classification result, the feature retrieval result, and the multi-label classification result to obtain a wafer defect classification detection result comprises the following steps:
Acquiring a first weight parameter of the point cloud classification result, a second weight parameter of the feature retrieval result and a third weight parameter of the multi-label classification result;
obtaining a first classification detection result according to the point cloud classification result and the first weight parameter;
Obtaining a second classification detection result according to the characteristic retrieval result and the second weight parameter;
obtaining a third classification detection result according to the multi-label classification result and the third weight parameter;
and adding the first classification detection result, the second classification detection result and the third classification detection result to obtain a wafer defect classification detection result.
4. A wafer defect classification inspection system, comprising:
The first module is used for acquiring the wafer according to the three-dimensional camera to obtain three-dimensional point cloud data;
Normalizing and projective converting the three-dimensional point cloud data to obtain a single-pixel image;
The second module is used for carrying out defect classification detection on the three-dimensional point cloud data according to the three-dimensional point cloud defect classification detection model to obtain a point cloud classification result; the point cloud classification result comprises a defect type, a defect position and a defect size of the wafer;
The third module is used for extracting the characteristics of the single-pixel image according to a backbone network in the wafer multi-label classification detection model to obtain the modal characteristics of the wafer;
Searching in a defect template library according to the modal characteristics to obtain characteristic searching results, wherein the method comprises the following steps:
acquiring a first keyword of the modal feature and a second keyword of the defect template library;
A plurality of first keywords and second keywords are taken out, and the first keywords and the second keywords are combined to obtain a feature set;
Respectively calculating word frequencies of the modal features and the defect template library in the feature set to obtain a first word frequency vector corresponding to the modal features and a second word frequency vector corresponding to the defect template library;
Calculating cosine similarity of the first word frequency vector and the second word frequency vector to obtain a feature retrieval result;
wherein, the characteristic search result comprises the defect type of the wafer;
A fourth module, configured to perform defect classification detection on the single-pixel image according to a wafer multi-label classification detection model, to obtain a multi-label classification result; the multi-label classification result comprises multi-class defects and probabilities of the wafer;
and a fifth module, configured to weight the point cloud classification result, the feature retrieval result and the multi-label classification result to obtain a wafer defect classification detection result.
5. An electronic device, comprising:
At least one processor;
at least one memory for storing at least one program;
the at least one program, when executed by the at least one processor, causes the at least one processor to implement the method of any of claims 1-3.
6. A computer readable storage medium storing a computer program, characterized in that the computer program, when executed by a processor, implements the method of any one of claims 1 to 3.
CN202410184264.0A 2024-02-19 2024-02-19 Wafer defect classification detection method, system, electronic equipment and storage medium Active CN118038151B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202410184264.0A CN118038151B (en) 2024-02-19 2024-02-19 Wafer defect classification detection method, system, electronic equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202410184264.0A CN118038151B (en) 2024-02-19 2024-02-19 Wafer defect classification detection method, system, electronic equipment and storage medium

Publications (2)

Publication Number Publication Date
CN118038151A CN118038151A (en) 2024-05-14
CN118038151B true CN118038151B (en) 2024-08-09

Family

ID=90983661

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202410184264.0A Active CN118038151B (en) 2024-02-19 2024-02-19 Wafer defect classification detection method, system, electronic equipment and storage medium

Country Status (1)

Country Link
CN (1) CN118038151B (en)

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113822842A (en) * 2021-04-30 2021-12-21 聚时科技(上海)有限公司 Industrial defect detection method based on multi-task learning
CN116539619A (en) * 2023-04-19 2023-08-04 广州里工实业有限公司 Product defect detection method, system, device and storage medium

Family Cites Families (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP5006520B2 (en) * 2005-03-22 2012-08-22 株式会社日立ハイテクノロジーズ Defect observation apparatus and defect observation method using defect observation apparatus
CN111044525B (en) * 2019-12-30 2021-10-29 歌尔股份有限公司 Product defect detection method, device and system
CN114648480A (en) * 2020-12-17 2022-06-21 杭州海康威视数字技术股份有限公司 Surface defect detection method, device and system
CN113205176B (en) * 2021-04-19 2022-09-06 重庆创通联达智能技术有限公司 Method, device and equipment for training defect classification detection model and storage medium
CN115699082A (en) * 2021-05-21 2023-02-03 京东方科技集团股份有限公司 Defect detection method and device, storage medium and electronic equipment
CN116542908A (en) * 2023-03-30 2023-08-04 深圳市格灵精睿视觉有限公司 Wafer defect detection method and device, electronic equipment and storage medium
CN117437455A (en) * 2023-09-20 2024-01-23 上海朋熙半导体有限公司 Method, device, equipment and readable medium for determining wafer defect mode

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113822842A (en) * 2021-04-30 2021-12-21 聚时科技(上海)有限公司 Industrial defect detection method based on multi-task learning
CN116539619A (en) * 2023-04-19 2023-08-04 广州里工实业有限公司 Product defect detection method, system, device and storage medium

Also Published As

Publication number Publication date
CN118038151A (en) 2024-05-14

Similar Documents

Publication Publication Date Title
US11282295B2 (en) Image feature acquisition
CN106096561B (en) Infrared pedestrian detection method based on image block deep learning features
CN108960140B (en) Pedestrian re-identification method based on multi-region feature extraction and fusion
US11631238B1 (en) Method for recognizing distribution network equipment based on raspberry pi multi-scale feature fusion
CN108171184B (en) Method for re-identifying pedestrians based on Simese network
WO2022213879A1 (en) Target object detection method and apparatus, and computer device and storage medium
EP2124159B1 (en) Image learning, automatic annotation, retrieval method, and device
CN110569901A (en) Channel selection-based countermeasure elimination weak supervision target detection method
CN110929080B (en) Optical remote sensing image retrieval method based on attention and generation countermeasure network
CN112085072B (en) Cross-modal retrieval method of sketch retrieval three-dimensional model based on space-time characteristic information
JP7430243B2 (en) Visual positioning method and related equipment
CN104298992B (en) A kind of adaptive scale pedestrian recognition methods again based on data-driven
US20140222783A1 (en) Systems and methods for automatically determining an improved view for a visual query in a mobile search
CN112633382A (en) Mutual-neighbor-based few-sample image classification method and system
CN110751027B (en) Pedestrian re-identification method based on deep multi-instance learning
WO2023273572A1 (en) Feature extraction model construction method and target detection method, and device therefor
CN111008576A (en) Pedestrian detection and model training and updating method, device and readable storage medium thereof
CN116595208B (en) Classification method and device for hyperspectral images and electronic equipment
TW201546636A (en) Annotation display assistance device and method of assisting annotation display
TWI812888B (en) Image recognition method and image recognition system
CN111325237A (en) Image identification method based on attention interaction mechanism
CN110059212A (en) Image search method, device, equipment and computer readable storage medium
CN113259883A (en) Multi-source information fusion indoor positioning method for mobile phone user
CN107452003A (en) A kind of method and device of the image segmentation containing depth information
CN112015937B (en) Picture geographic positioning method and system

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant