CN115527072A - Chip surface defect detection method based on sparse space perception and meta-learning - Google Patents

Abstract

The invention discloses a chip surface defect detection method based on sparse space perception and meta-learning, which comprises the steps of firstly, acquiring data and carrying out image preprocessing operation, secondly, enhancing an image by selecting a similarity contrast learning enhancement network algorithm, and adding a transfer learning module before inputting image features after enhancement transformation into a sparse space alignment network of cross transformation, so that the model can more easily identify feature information in classes on the fine granularity, and the convergence of the model is accelerated. And finally, training and testing the model by adopting an N-way K-shot task detection method, and finally realizing the detection of the chip defects. The invention greatly reduces the calculation amount required by the model during learning, thereby achieving the effect of light weight; the generalization ability of the model is improved by introducing meta-learning, and a small amount of data sets are used for enhancing the neural network, so that information outside the class of the image label is learned, and the accuracy of detecting the defects on the surface of the chip is improved.

Description

A Chip Surface Defect Detection Method Based on Sparse Spatial Awareness and Meta-Learning

technical field

The invention relates to the technical field of chip surface defect detection, in particular to a chip surface defect detection method based on sparse space perception and meta-learning.

Background technique

Chips play an irreplaceable role in people's daily life, but chips with surface defects will directly affect the performance and service life of electronic products. During the chip production process, it is necessary to inspect the surface defects of the manufactured chips, such as: scratches, Bump component defects (bumps, dislocations or missing), metallic pollutants, and etchant residues. With the continuous progress of the manufacturing industry, the quality of the chips produced is getting better and better, so the data set of chips with surface defects is very limited.

Small-sample learning is designed to solve the problem of a small number of data sets, but the network model based on small-sample learning is difficult to extract the characteristic information of surface defects in the process of training defect detection tasks, and the structure of the network is relatively simple, which may lead to loss. Important surface defect feature information is not learned from the defect features in new samples; secondly, complex models often require a lot of computation and time costs. Therefore, in view of the above two difficulties, this patent proposes to further combine the sparse space based on cross-transformation and meta-learning, so that the model can achieve a lightweight effect and improve the calculation speed, so as to complete the defect detection of chips with surface defects.

The surface defect detection of chips is one of the key links in the production line, and the surface defect detection of chips refers to the accurate classification by using common surface defect features. At present, the more common surface defect detection methods are:

Traditional classification: For the classification of whether the surface of the product has defects, the traditional surface inspection method is to manually detect the surface of the product, but some defects will cause visual fatigue, and it is easy to receive external interference, and the detection efficiency is difficult to obtain. Guarantee leads to many misjudgments. This method of surface defect detection is convenient to operate, but has disadvantages such as low efficiency, inconsistent standards, and high cost.

Machine learning classification: Classify the defect types on the chip surface mainly through the different defect features reflected by the defect image information input on the chip surface. The surface defect detection algorithm here is mainly a combination of a feature selection algorithm based on artificially designed features and a pattern recognition classification algorithm. The traditional classification algorithms include K nearest neighbor algorithm for image classification, BP neural network algorithm for image classification, and Bayesian algorithm. Implement image classification, etc.

Deep learning classification: The successful application of deep learning models represented by convolutional neural networks in the field of computer vision provides a new development direction for defect detection. The use of deep learning for classification can achieve amazing results to some extent, and find important intrinsic feature information of chip surface defect images. In recent years, with the rapid development of computer technology and the rapid development of artificial intelligence, deep learning has been widely used, involving all aspects of people's lives. Amazing accuracy is also achieved in industrial product classification. The target detection network model based on deep learning is currently divided into those that do not extract candidate regions independently and those that extract candidate regions independently. For network models that do not extract candidate regions independently, target boxes are directly generated for target detection. The target detection model has a fast detection speed, but the detection accuracy is not high. Such representative algorithms include Yolo series, SSD, etc. For independent extraction of candidate regions, the candidate boxes are first generated, and then the potential targets in these generated candidate boxes are selected as the final candidate boxes. The network model with independently extracted candidate regions has higher accuracy, but its detection speed is slower than that without independently extracted candidate regions. Representative algorithms include R-CNN, Faster RCNN, etc. The above-mentioned network models all require a large amount of data sets, which are difficult to meet the application in product production.

Currently commonly used deep learning network models are: Lenet, Alxnet, VGG series, Resnet series, Inception series, Densenet series, Googlenet, Nasnet, Xception, Senet,

The lightweight network models in deep learning are: Mobilenet v1, v2, Shufflenet v1, v2, Squeezenet

The lightweight models described above are mostly used in project applications, and they have different advantages and disadvantages:

Advantages: (1) The parameter model is small and easy to deploy; (2) The calculation amount is small and the speed is fast;

Disadvantages: (1) The accuracy of the lightweight model is not as high as that of the Resnet series, Inception series, Densenet series, and Senet.

The above classification algorithms using computers all require a large number of samples to obtain higher classification accuracy.

Small sample learning. It is to train the model through a large number of tasks. These tasks are composed of randomly selected pictures from the chip data set and each task is different, and then it can be quickly learned on a small number of new unseen chip samples. . Among them, meta-learning is to learn how to learn. At present, it is mainly used to solve the problem of small sample learning.

The development of meta-learning in recent years mainly includes:

1: Metric-based meta-learning: the training model does not need to be adjusted for the test task; but when the categories of the test and training on the task set are large, the effect is not very good;

2: Model-based meta-learning: Due to the flexibility of the internal dynamics of its system, it has wider applicability than most metric-based meta-learning; it is not as good as metric learning on many supervised tasks; when the amount of data increases, The effect becomes worse; when the category difference between tasks is large, the effect of the model is not as good as the optimization-based meta-learning method;

3: Optimization-based meta-learning: Compared with model-based meta-learning, they can achieve better performance in the case of a wider distribution of tasks; optimization-based techniques need to optimize learning on the base learner for each task, resulting in Computationally expensive.

Existing methods include a spatial alignment network based on small-sample cross-transformation, through data enhancement for small samples, so that it can obtain higher class information within and outside the class. This allows for better generalization on new tasks, which are then fed into a cross-transformed spatially aligned network, which has better classification accuracy.

For the method of manual classification: the efficiency is not high, and the labor intensity is high. The human factor accounts for the largest proportion.

For the method of machine learning classification: its disadvantage is that for chips with defects, the classification algorithm needs to be designed according to different prior knowledge, and features are extracted and selected according to the characteristics of chip surface defects, which makes the algorithm less robust. Low, it is difficult to complete the classification task under complex tasks. At the same time, the classification algorithm based on machine learning has high requirements for images. All images must have a uniform background, and the feature part is only a certain position in the normal image. For chips of different sizes, the collected backgrounds are different, and the defect positions on the chip surface are different. These will make the classification accuracy relatively low. In addition, it is usually difficult to obtain the information characteristics of the image and detect the chip by relying on machine learning classification algorithms. Whether the surface has defects or not, the classification results are also easily disturbed by the material itself and other factors. Therefore, it is difficult for traditional machine vision technology to fully and effectively extract defect features, which is inefficient and cannot accurately distinguish whether there are defects on the chip surface. For electronic equipment equipped with chips, there are potential safety hazards, so it is not suitable for high-precision chip quality defect detection.

For deep learning, deep learning is more suitable for accurate classification of whether chips have defects. The biggest difference between the classification algorithm of deep learning and the traditional method is: deep learning uses convolutional neural network to extract image features, and extracts image features through the inner product of data in different size windows and convolution kernel. However, deep learning has poor robustness, poor generalization, and relies heavily on massive data. A very large amount of data is required to achieve effective learning. If there is not enough data, it may lead to poor training results of the deep learning network, or even underfitting, making it difficult for the model to converge. Therefore, general-purpose object detection algorithms are not suitable for direct application to chip surface defect detection tasks, and new solutions need to be proposed.

Meta-learning can effectively solve the classification problem of a small number of chip samples with defects, and only need a small amount of image data to learn some specific tasks, which is different from general neural network algorithms. However, due to learning a small number of samples, in the process of meta-learning training tasks, less feature information is extracted, and the structure of the network is relatively simple, so it is easy to lose some information that is not needed during training, resulting in information needed in new tasks or new fields. It is also lost; in this way, when the defect location information, defect type, defect appearance, etc. of the chip change, the accuracy rate may be greatly reduced. Secondly, complex models often require a lot of calculation and time costs, and cannot be classified in real time, so it is difficult to meet the needs of actual production. Therefore, it is also very important to improve the operation speed of small sample classification at the same time.

Due to its huge parameter problem, the spatial alignment network based on the cross-transformation of small samples leads to a large computational time cost, which is difficult to meet the real-time requirements.

Contents of the invention

Purpose of the invention: This invention aims at the problem of insufficient information and huge amount of calculation outside the small sample learning category, and provides a chip surface defect detection method based on sparse space perception and meta-learning, and a cross-transformed sparse space alignment network, so that the model The amount of calculation required during learning is greatly reduced, achieving a lightweight effect, thereby adapting to the requirements of real-time defect detection in the industry; on the other hand, the introduction of meta-learning improves the generalization ability of the model, and a small amount of data sets To enhance the neural network, so as to learn information other than the image label category, and improve the accuracy of defect detection on the chip surface.

The technical solution adopted in the present invention: a chip surface defect detection method based on sparse space perception and meta-learning. The technology used in this patent is a fusion of cross-transformed sparse space alignment network and meta-learning, so as to detect chip surface defects: first, carry out Data collection and image preprocessing operations, and secondly, choose the similarity contrast learning enhancement network algorithm to enhance the picture, before inputting the enhanced transformed image features into the cross-transformed sparse space alignment network, add the migration learning module, It makes it easier for the model to identify the feature information within the class at a fine-grained level and speed up the convergence of the model. Finally, the N-way K-shot task detection method is adopted to train and test the model, and finally realize the detection of chip defects. Specific steps are as follows:

Step 1, data collection and processing: First, prepare the chip training data set, collect the chip data set with defects, and divide this data set into training set, verification set and test set according to the model training method; Sampling to form many tasks, these tasks are disjoint, each set is composed of many tasks, each task contains a support set and a query set, where the support set is a category label, and the query set is There is no label; the support set and the new query set are used when training the image with the method of similarity contrast learning enhancement. These new query sets randomly extract some data from the support set, so that the new query set image can be obtained The same number of categories, the above is to divide the data set;

Step 2, pre-training model: The similarity-contrast learning enhancement network used in this patent is used to enhance the transformation of data, which is mainly used for unsupervised learning, and can also improve the basic model and embedded feature information, which can greatly improve the migration. Obtain the information needed by the model while learning. Using similarity-contrastive learning to augment the network for training can obtain better image embeddings, which will not be affected by different image transformations of the same class. Because during training, random data enhancement is performed on the input picture, so that the network can learn more image information, and there is no need to learn the color of the picture or the position information of the target in the picture. Therefore, when the image embedding is pre-trained, the image is randomly enhanced to make it more difficult for the network model to learn, so that it can have better generalization ability in the future model.

Step 3, model selection: This patent uses a cross-transformed spatially sparse network to detect chip surface defects. This model is specially designed for classification of small objects, and at the same time achieves the purpose of reducing network parameter calculation and training time. Since the defects on the surface of the chip are relatively small, the network model converts the picture features into a three-dimensional feature space through the self-attention head, so as to obtain more feature information, and obtain the size of the attention value through the operation of the self-attention mechanism. Among them, the larger The value indicates that a higher semantic information is obtained, and a smaller value indicates that a small amount of semantic information is obtained; in order to reduce the time and information redundancy required for pixel traversal calculations, a sparse semantic alignment network module is added, and the semantic correlation is relatively large. Calculation, if the attention value is small, no calculation is required, and the finally obtained semantically aligned feature map and the map in the query set are used for measurement calculation.

Step 4, transfer learning: Generally, during the training process, the training parameters are randomly initialized. In order to obtain good parameters, a large number of pictures need to be trained. However, some parameters of feature extraction in small samples account for a lot; in order to make up for The disadvantage of the small number of samples is that the transfer learning module is added in the process of meta-learning; firstly, the previously divided training data is put into the similarity contrast enhancement network for training, and the network training weight is obtained; The trained model weights are used for transfer learning to enhance the ability to support centralized image feature extraction in the meta-learning test set; reduce the number of model iterations and speed up the rapid convergence of the model.

Step 5, meta-learning: In the meta-training stage, a task randomly adopts the N-way K-shot task classification method, where N is the number of randomly selected categories, and K is the number of pictures corresponding to each selected category.

For the meta-training set, this patent uses the 5-way 1-shot classification method to put the data into the network for training;

For the meta-test set, this patent uses a 5-way 1-shot classification method to put data into the network for testing.

First, input the support set of meta-training data into the similarity-contrast learning enhancement network, perform two different random enhancement transformations on the input image, and then send the enhanced images into the residual network to extract features, and then use The nonlinear fully connected layer based on the multi-layer perceptron obtains two embedding vectors, and uses the cosine similarity to calculate the similarity between the two enhanced images of the image. The similarity between enhanced images will be high, while the similarity between images of different categories will be low. Then all the remaining images in each batch are treated as dissimilar category images (that is, as negative samples), and then the positions between each two batches are exchanged, and the losses of all pairs are summed And take the average as the loss function. The formula is as follows:

in,

l(i,j) is the loss between two enhanced image features, i and j are the two enhanced image features of the original image.

After training the similarity contrast learning enhanced network model, it is only necessary to use the trained similarity contrast learning enhanced network to obtain the features in the support set through the transfer learning method. Similarly, transfer learning is also done for the query set. The network will get some positively related samples, and each picture will undergo similarity comparison learning and enhancement transformation to obtain the features of the two pictures for feature averaging operation, and then the feature map corresponding to each category can be obtained, and the same operation is done for the query set. The formula is as follows:

For the c-th category in the support set, it is represented as s ^c , |s ^c | represents the number of pictures contained in category c, x represents an original picture, and Φ(x) represents the feature vector obtained through migration learning.

生成键K_s和值V_s，投影头K:

和值投影头V:

进行特征维度的变换。类似地，使用一个线性投影为查询集特征

生成特征Q_q，投影头Q:

进行特征维度的变换。分别得到支持集和查询集的特征空间后，将他们在各自的维度对应点之间进行点乘，就可以得到一系列的查询图像与各支持类之间的语义关系矩阵。Then transform the obtained support set image and query set image from two-dimensional form to three-dimensional tensor feature form. In the N-way K-shot task, two independent linear projections are used as support set features

Generate key K _s and value V _s , projection head K:

and value projection head V:

Transform feature dimensions. Similarly, using a linear projection for the queryset features

Generate feature Q _q , projection head Q:

Transform feature dimensions. After obtaining the feature spaces of the support set and the query set, dot product them between the corresponding points of their respective dimensions, a series of semantic relationship matrices between the query image and each support class can be obtained.

If the semantic distance between the query set and the spatial corresponding points in the support set is similar, that is, the spatial points in the support set and the spatial corresponding points in the query set have a large attention value, then they are likely to have similar local features, otherwise the distance between them Semantic relationships are also relatively weak. First calculate the semantic relationship matrix between the query image and the spatial corresponding points on each support class, and get R _n :

Each row in _Rn represents the semantic similarity of each point in the query image to all points in all images in the support set. A sparse spatial cross-attention algorithm is employed to find task-relevant point features in the query image.

After collecting all the attention points related to the task, you can use the mask m=[m ₁ ;...;m _k ] to get the features with large attention points, and delete them if the value of the attention points is small, here you need to set in advance Set the threshold. If the value in the semantic relationship matrix is greater than the threshold, _mi is equal to 1, otherwise it is 0. The threshold here is set to 0.5. Multiplying the mask m with the semantic relationship matrix R _n , we can get the sparse attention map a _n , and use it to semantically align with the key value V _s of each support set to obtain the space corresponding to the query image set position, resulting in a task-specific prototype vector t, which can be computed as:

a _n =m*R _n

进行特征维度的变换，将其变换为与原型向量t相同的大小，进行度量计算：Also do a projection head V on the queryset:

Perform feature dimension transformation, transform it to the same size as the prototype vector t, and perform metric calculation:

经过投影头V:

变换得到的。如果距离相近的那么就是同一个类别，否则就不是同一个类别。Among them, H' and W' are the height and width of the original image respectively, and w ^p represents the query set feature

Through projection head V:

Transformed to get. If the distance is similar then it is the same category, otherwise it is not the same category.

Beneficial effects: the present invention uses a cross-transformed sparse space-aware alignment network as a meta-learning model of the framework, and a classification method that combines the similarity-contrast learning enhancement network with meta-learning, which improves the feasibility of small-sample classification and reduces the number of training samples. The number and the number of training iterations speed up the training iteration cycle and greatly reduce the amount of parameter calculations. A neural network that augments a dataset to learn information beyond the labeled categories. A small amount of data sets are used for network training, and image enhancement transformation is performed on the data sets, and feature extraction is performed using the enhanced images, so that fine-grained and coarse-grained category information can be obtained well during learning. It makes it possible to learn the data information of non-label categories even in the case of small samples, and solve the problem of good accuracy when the data set on the test is different from the training category. After learning information beyond label categories, semantic alignment is then performed through a sparse spatially aware network. Through the two network models, more feature information in small sample learning can be improved, and the accuracy of image classification under a small number of samples can be improved. When the amount of data is difficult to obtain, compared with the existing classification methods, the classification method of the present invention can effectively solve the classification problem of small samples, and has certain adaptability.

Description of drawings

Fig. 1 is a flow chart of the chip surface defect detection method based on sparse space perception and meta-learning in the present invention.

Fig. 2 is a schematic diagram of the sparse space-aware classification model of the present invention.

detailed description

The technical solutions in the entire embodiment will be described in detail below in conjunction with the drawings and specific implementation methods of the present invention.

As shown in Figure 1, a chip surface defect detection method based on sparse space perception and meta-learning, firstly, collect data and perform image preprocessing operations; secondly, select similarity contrast learning enhancement network algorithm to enhance image and improve features The ability to extract speeds up the convergence of the model. During meta-training, the data is input into the trained similarity-contrast learning enhancement network, and the obtained features are transferred to the cross-transformed sparse alignment network through the transfer learning module, and transformed into three-dimensional spatial features to make it have more abundant features. Information, in the face of unknown chip data sets with or without defects, the model can learn more feature information on the data set and improve the classification accuracy of the model. In both training and testing, the N-way K-shot task data classification method is adopted to train and test the network model, and finally achieve accurate classification of whether a small amount of chip data sets have surface defects.

The specific process includes the following steps:

Step 1: Collect Experimental Dataset

Step 2: Data partitioning

Step 3: Data Augmentation

Step 4: Spatial Sparse Semantic Alignment Network

Step 5: Meta-learning

Step 6: Metric Learning

The present invention produces a data set in tfrecord format that can be read by tensorflow, uses the LabelImg tool to label the pictures in the chip data set, and generates an xml file. Collect a dataset of common defect categories including annotation information. Divide common data sets according to types, some common data sets are used as data for migration learning training, and the other part of data sets are divided into training sets and test sets. The original image of the chip is divided into a chip element training set and a chip element test set according to the position of the defect calibration to intercept the local defect map. Perform image preprocessing operations on all collected images, scale all collected images to the same size, perform image enhancement operations, random image inversion, and horizontal rotation operations.

Divide the data set: There are many tasks in the meta-training set, each task is composed of a support set and a query set, and each task is different, and they are randomly extracted from the chip data set. The meta-test is also composed of multiple different tasks and the tasks in the meta-test set are disjoint from the tasks in the meta-training set. Then randomly select n tasks from the processed data set as the meta-training data set, and send the tasks to the entire network model for training and update parameters in turn, and finally save the updated parameters.

Model selection: This patent builds models by combining a cross-transformed spatially aligned network with a sparse space. The similarity-contrast learning enhancement technology is an unsupervised learning model. By randomly enhancing the data set, the feature information within the class can be well learned, and it has the advantages of quickly adapting to other new tasks. See Figure 2 for the sparse space-aware network model method. Among them, the backbone network enhanced by similarity contrast learning uses the residual network Resnet-34 to extract the feature map, and compares the similarity through the string function. Since chip defect locations are generally relatively small, the enhanced network loses less class-independent information when learning intra-class information through similarity comparison learning. The enhanced and transformed features are sent to the sparse spatial perception network for training. In this network model, the two enhanced and transformed features obtained by transfer learning are weighted and summed to obtain a feature map of this class, and then such two-dimensional features are transformed into three-dimensional feature dimensions, and the query set Multiply with the three-dimensional feature vector in the support set to get multiple spatial corresponding points, get the semantic relationship matrix between the image in each query set and all the images in each support set, and get the spatial perception attention value, in the three-dimensional This may take a lot of time in the tensor space. In order to reduce the weight of this model, the attention matrix obtained by the support set and the query set is selected from the top n values with the largest attention, which means that the query image is more related to the support image. pixels are associated. If the semantic distance between the query set and the spatial corresponding point in the support set is similar, that is, the spatial point in the support set is more correlated with the value of the corresponding point in the query set space, then they are likely to have similar local features, otherwise the semantics between them Relationships are also weak. In this way, category information irrelevant to the query set can be discarded. Then align the semantic feature matrix with the support set image to obtain an image feature on the support set, and multiply the obtained semantic alignment feature map with the support set feature transformed by the key value V to obtain the query set The aligned network feature map above, and then convert the query set to the same size as the aligned network feature map for measurement operations. If the distance is close, then it is the same category, otherwise it is not the same category.

Carry out meta-learning: In the meta-training stage, the N-way K-shot task classification method is randomly adopted in a task, N is the number of categories contained in each task, and K represents the number of images contained in each category. During the meta-training, the data set uses the 5-way 1-shot partition method to put the data into the network for training, and then test it in the meta-training through the query set in the training set. Similarly, for the meta-test set, the 5-way 1-shot classification method is also used to put the chip data on the network for testing.

Metric learning is to learn a distance function in a specific task, so that the distance function can achieve better performance between categories. Metric learning is a more common method, so that the calculation distance of similar objects in the embedding space is relatively close, and the distance between different objects is relatively long.

It should be pointed out that those skilled in the art can make some improvements and modifications without departing from the principle of the present invention, and these improvements and modifications should also be regarded as the protection scope of the present invention.

Claims (2)

1. A chip surface defect detection method based on sparse space perception and meta-learning, characterized in that it includes: firstly, collecting data and performing image preprocessing operations; secondly, selecting similarity contrast learning to enhance the network algorithm to pair pictures For enhancement, before inputting the enhanced transformed image features into the cross-transformed sparse space alignment network, a transfer learning module is added to make it easier for the model to identify the feature information in the class at a fine-grained level and speed up the convergence of the model; finally, adopt The N-way K-shot task detection method performs model training and testing, and finally realizes the detection of chip surface defects; the specific steps are as follows: Step 1, data collection and processing: first, prepare the chip training data set, collect the chip data set with defects, and divide this data set into training set, verification set and meta-test set according to the model training method; from the chip data set Sampling is performed to form many tasks. These tasks are disjoint. Each set is composed of many tasks. Each task contains a support set and a query set. The support set is a category label, and the query set There is no label; the support set and the new query set are used when training the image with the method of similarity contrast learning enhancement, and these new query sets are randomly extracted from the support set. The category with the same number of images, the above is to divide the data set; Step 2, pre-training model: use similarity and contrastive learning to enhance the network to enhance the transformation of the data for unsupervised learning, and also improve the basic model and embedded feature information, so as to improve the performance required to obtain the model when performing transfer learning. Information; use similarity contrast learning to enhance the network to train to obtain better image embedding, so that it will not be affected by different image transformations of the same class; because during training, random data enhancement is performed on the input image to make the network more Learn more image information, and do not need to learn the color of the picture or the position information of the target in the picture; therefore, when the image embedding is pre-trained, the image is randomly enhanced to make it difficult for the network model to learn, so that Have better generalization ability in future models; Step 3, model selection: use the cross-transformed spatial sparse network to detect chip surface defects. This model is specially designed for classification of small objects, and at the same time achieves the purpose of reducing network parameter calculation and training time; The existing defects are relatively small. The network model transforms the picture features into a three-dimensional feature space through the self-attention head, so as to obtain more feature information, and obtain the size of the attention value through the operation of the self-attention mechanism. Among them, a larger value indicates the obtained Higher semantic information, a smaller value means that a small amount of semantic information is obtained; in order to reduce the time and information redundancy required for pixel traversal calculations, a sparse semantic alignment network module is added, and the semantically related ones are calculated. Note If the force value is small, no calculation is required, and the finally obtained semantically aligned feature map and the map in the query set are measured and calculated; Step 4, transfer learning: Generally, during the training process, the training parameters are randomly initialized. In order to obtain good parameters, a large number of pictures need to be trained. However, some parameters of feature extraction in small samples account for a lot; in order to make up for The disadvantage of the small number of samples is that the transfer learning module is added in the process of meta-learning; firstly, the previously divided training data is put into the similarity contrast enhancement network for training, and the network training weight is obtained; The trained model weights are used for transfer learning to enhance the ability to support centralized image feature extraction in the meta-learning test set; reduce the number of model iterations and speed up the rapid convergence of the model; Step 5, meta-learning: In the meta-training stage, a task randomly adopts the N-way K-shot task classification method, where N is the number of randomly selected categories, and K is the number of pictures corresponding to each selected category; For the meta-training set, the 5-way 1-shot classification method is used to put the data into the network for training; For the meta-test set, a 5-way 1-shot classification method is used to put the data into the network for testing. 2. A chip surface defect detection method based on sparse space perception and meta-learning according to claim 1, characterized in that, in the step five, the specific steps are: First, input the support set of meta-training data into the similarity-contrast learning enhancement network, perform two different random enhancement transformations on the input image, and then send the enhanced images into the residual network to extract features, and then use The nonlinear fully connected layer based on the multi-layer perceptron obtains two embedding vectors, and uses the cosine similarity to calculate the similarity between the two enhanced images of the image. The similarity between images after enhancement will be high, while the similarity between images of different categories will be low; then all remaining images in each batch are regarded as dissimilar category images, and then every two The positions between batches are swapped, and the losses of all pairs are summed and averaged as the loss function; the formula is as follows:

in,

l(i,j) is the loss between the two enhanced image features, i and j are the two enhanced image features of the original image; After training the similarity contrast learning enhanced network model, it is only necessary to use the trained similarity contrast learning enhanced network to obtain the features in the support set through the transfer learning method. Similarly, transfer learning is also done for the query set. The network will get some positively correlated samples, and each picture will undergo similarity comparison learning enhancement transformation to obtain the features of the two pictures for feature averaging operation, and then get the feature map corresponding to each category, and do the same for the query set; the formula is as follows :

For the c-th category in the support set, it is represented as s ^c , |s ^c | represents the number of pictures contained in category c, x represents an original picture, and Φ(x) represents the feature vector obtained through migration learning; Then transform the obtained support set image and query set image from two-dimensional form to three-dimensional tensor feature form. In the N-way K-shot task, two independent linear projections are used as support set features

Generate key K _s and value V _s , projection head

sum value projection head

Transform the feature dimension; similarly, use a linear projection for the query set feature

To generate features Q _q , the projection head

Transform the feature dimension; after obtaining the feature spaces of the support set and the query set respectively, perform point multiplication between the corresponding points of their respective dimensions, and obtain a series of semantic relationship matrices between the query image and each support class; If the semantic distance between the query set and the spatial corresponding points in the support set is similar, that is, the spatial points in the support set and the spatial corresponding points in the query set have a large attention value, then they are likely to have similar local features, otherwise the distance between them The semantic relationship is also relatively weak; first calculate the semantic relationship matrix between the query image and the spatial corresponding points on each support class, and get R _n :

Each row in R _n represents the semantic similarity of each point in the query image to all points in all images in the support set; a sparse spatial cross-attention algorithm is used to find task-relevant points in the query image feature; After collecting all the attention points related to the task, use the mask m=[m ₁ ;...;m _k ] to get the features with large attention points, and delete them if the value of the attention points is small, which needs to be set in advance Good threshold, if the value in the semantic relationship matrix is greater than the threshold, mi is equal to 1, otherwise it is 0, the threshold here is set to 0.5; use the mask _{m to multiply the semantic relationship matrix R n to} get a sparse attention map a _n , and use it to semantically align with the key value V _s of each support set to obtain the spatial location corresponding to the query image set, resulting in a task-specific prototype vector t, calculated as: a _n =m*R _n

Also do a projection head on the queryset

Perform feature dimension transformation, transform it to the same size as the prototype vector t, and perform metric calculation:

Among them, H' and W' are the height and width of the original image respectively, and w ^p represents the query set feature

through projection head

Transformed; if the distance is similar, then it is the same category, otherwise it is not the same category.

Images