CN113591948A - Defect pattern recognition method and device, electronic equipment and storage medium - Google Patents
Defect pattern recognition method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN113591948A CN113591948A CN202110808135.0A CN202110808135A CN113591948A CN 113591948 A CN113591948 A CN 113591948A CN 202110808135 A CN202110808135 A CN 202110808135A CN 113591948 A CN113591948 A CN 113591948A
- Authority
- CN
- China
- Prior art keywords
- model
- defect
- image
- training
- product
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2413—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on distances to training or reference patterns
- G06F18/24147—Distances to closest patterns, e.g. nearest neighbour classification
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Computation (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computational Linguistics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Evolutionary Biology (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Image Analysis (AREA)
Abstract
The disclosure provides a defect pattern recognition method, a defect pattern recognition device, an electronic device and a storage medium, and belongs to the technical field of defect detection and unsupervised learning. Wherein the method comprises the following steps: collecting a product image in the production process; identifying a defect mode of a product in the product image according to a preset defect mode identification model; and the preset defect mode identification model outputs the probability of each defect mode corresponding to the product in the product image, and selects the defect mode with the maximum probability as the defect mode identification result of the product. The method and the device effectively improve the identification precision and efficiency of the defect mode, can greatly reduce the labor cost, and have great application prospect in the aspect of industrial production.
Description
Technical Field
The present disclosure relates to defect detection technologies and unsupervised learning technologies, and in particular, to a defect pattern recognition method and apparatus, an electronic device, and a storage medium.
Background field of the invention
In industrial production, since the production process is a complex process with coupled multiple factors and is very susceptible to interference from external environments and human factors, various defect modes often occur on products on a production line, which not only affects the appearance of the products, but also greatly affects the quality and the service life of the products. Any abnormity in the production process can cause product defects, and the generation of some typical defect modes can be associated with production line fault sources, so that various defect modes of the product can be accurately identified, the fault sources existing on the production line can be monitored in an auxiliary manner, corresponding production parameters can be adjusted and improved in time, and huge loss caused by the occurrence of large batches of defective products is avoided. The defect mode of the abnormal product is identified in time, which is an effective way for improving the production quality and the production efficiency, so that the defect mode identification method has very important research significance.
Early product defect pattern recognition was mainly performed by machine learning methods, such as support vector machines, back propagation networks, etc. Compared with the method for directly identifying the product defects by naked eyes, the method greatly reduces the workload. However, these early methods have the following disadvantages: the recognition accuracy is low and a large amount of label data is needed to train the model. With the development of deep learning in recent years, a large number of convolutional neural network-based algorithms are highlighted in the visual task. Because the convolutional neural network has very strong feature extraction capability, the convolutional neural network achieves remarkable achievement in the field of image recognition. Meanwhile, the convolutional neural network is widely applied to the defect detection task.
Compared with the traditional machine learning method, the deep learning method based on the convolutional neural network has higher recognition accuracy and working efficiency in the field of defect recognition. However, this method has the same disadvantage as the machine learning method, that model training requires a lot of label data, whereas images with defective mode labels are not easily obtainable, since acquiring images with product defect mode labels requires manual supervision, i.e. manual labeling by a lot of experienced professionals, which is very expensive and time consuming. In addition, when a large number of product defect modes are marked manually, the influence is easily caused by subjective factors, so that the model is trained under wrong labels, and the accuracy of the model in identifying the product defect modes is influenced. In addition to the difficulty in acquiring a large number of labels, in some highly automated production scenarios, the yield of products is extremely high, and it is very time-consuming to collect defect samples, while most of the current deep learning methods for defect detection build models based on defect samples, and the lack of defect samples causes the models to be difficult to be online. In addition to the difficulty in collecting defect samples caused by high product yield, in a multi-model small-batch production scenario (each model product is produced for only a few days) of some industries such as the automobile industry, a certain model product is not produced before the defect sample collection is completed, and a technical route based on defect sample modeling cannot be used in the scenario. In addition, since defects are generated by uncontrolled factors in the production process, the forms of the defects are various, and samples of various forms are difficult to collect completely, which also causes that the method for establishing a model based on the defect samples is difficult to control the omission of the defects to be 0.
Disclosure of Invention
The invention aims to solve the problems that an existing defect mode identification method is low in efficiency and accuracy, poor in model generalization capability, and needs a large amount of time-consuming and labor-consuming manual labeling images as training data, and the like, and provides a defect mode identification method, a defect mode identification device, electronic equipment and a storage medium. The method and the device effectively improve the identification precision and efficiency of the defect mode, can greatly reduce the labor cost, and have great application prospect in the aspect of industrial production.
An embodiment of a first aspect of the present disclosure provides a defect pattern identification method, including:
collecting a product image in the production process;
identifying a defect mode of a product in the product image according to a preset defect mode identification model; and the preset defect mode identification model outputs the probability of each defect mode corresponding to the product in the product image, and selects the defect mode with the maximum probability as the defect mode identification result of the product.
In one embodiment of the present disclosure,
before the identifying the defect mode of the product in the product image according to the preset defect mode identification model, the method further comprises the following steps:
training the defect pattern recognition model;
wherein the training the defect pattern recognition model comprises:
acquiring an image composition data set of the product in a production process, wherein the image comprises a defect image and a normal image of the product;
dividing the dataset into a label-free dataset and a label dataset; wherein the label dataset comprises an image and a defect pattern of the product in the image;
constructing an unsupervised pre-training model, and training the unsupervised pre-training model by using the unlabeled data set to obtain parameters of the unsupervised pre-training model;
and constructing a classification fine-tuning model, taking the parameters of the unsupervised pre-training model as initialization parameters of the classification fine-tuning model, training the classification fine-tuning model by using the label data set, and taking the trained classification fine-tuning model as a defect pattern recognition model.
In one embodiment of the present disclosure, the training of the classification fine-tuning model using the tag data set includes:
dividing the label data set into a training set, a verification set and a test set according to a set proportion;
setting a loss function of the classification fine tuning model;
training the classification fine-tuning model by using a training set in the label data set according to the loss function to reach the set iteration number, and obtaining a trained classification fine-tuning model;
and detecting the accuracy of the trained classification fine tuning model by using the verification set, and selecting the classification fine tuning model with the highest identification accuracy on the verification set as a defect mode identification model.
In one embodiment of the present disclosure, before the training the unsupervised pre-training model with the unlabeled dataset, further comprising performing image enhancement on images in the unlabeled dataset;
before training the classification fine-tuning model by using the training set in the label data set, performing image enhancement on images in the training set in the label data set;
the image enhancement includes: one or more of translation, flipping, random noise addition, rotation, and cropping.
In one embodiment of the present disclosure, the training the unsupervised pre-training model with the unlabeled dataset includes:
respectively taking each image in the unlabeled data set as a target sample;
according to a K nearest neighbor recursive transfer algorithm, mapping the label-free data set to a low dimension to obtain a corresponding low-dimension data set, and finding out samples belonging to the same defect mode with the target sample from the low-dimension data set to form a positive sample set of the target sample;
acquiring a difficult positive sample set and a difficult negative sample set of the target sample from the low-dimensional data set by sampling by using the positive sample set of the target sample;
constructing a loss function of each image in the unlabeled data set according to the difficult positive sample set and the difficult negative sample set;
and inputting the label-free data set into the classification fine-tuning model for training according to the loss function to reach the set iteration times, so as to obtain a trained unsupervised pre-training model.
In one embodiment of the present disclosure, the K-nearest neighbor recursive transfer algorithm includes:
recording the unlabeled dataset as A ═ aiMapping the unlabeled dataset to a low-dimensional set B ═ B }i-i ═ 1,2, ·, n }, where n is the total number of images in the unlabeled dataset;
any image a in the unlabeled datasetiThe probability of being identified as an r-th class defect pattern is:
wherein the content of the first and second substances,
is characteristic of the r-th type of defect pattern, biIs the ith image a in the unlabeled datasetiThe data after being mapped to the low dimension is,
is the defect characteristics of the current batch of images, and tau is the temperature coefficient of the unsupervised model;
a is to be describediAs a target sample, obtaining a sample composition a with the same type as the defect mode of the target sample from the low-dimensional set B according to the calculated probabilityiThe positive sample set of (a), (b);
the sampling method comprises the following specific steps:
selecting the target sample a from the positive sample set H (i)iQ sample compositions a with lowest similarity under the same defect modeiDifficult positive sample set Hp(i) (ii) a Selecting sample composition a similar to the target sample characteristics in other defect modes from the low-dimensional set BiDifficult negative sample set Hn(i) (ii) a Selecting B from the low-dimensional set BiZ adjacent samples of (a) constitute the target sample aiAdjacent sample set H ofz(i) Wherein said H isz(i) Comprising the above-mentioned Hn(i) A middle negative sample; the target sample aiIs expressed as: hn(i)＝Hz(i)-H(i)。
In one embodiment of the present disclosure, the loss function for each image in the unlabeled dataset is:
wherein the content of the first and second substances,
is a set Hp(i) The characteristics of the image of the medium sample,
is a set Hn(i)∪Hp(i) The image characteristics of the middle sample.
An embodiment of a second aspect of the present disclosure provides a defect pattern recognition apparatus, including:
the acquisition module is used for acquiring a product image in the production process;
the recognition module is used for recognizing the defect mode of the product in the product image according to a preset defect mode recognition model; and the preset defect mode identification model outputs the probability of each defect mode corresponding to the product in the product image, and selects the defect mode with the maximum probability as the defect mode identification result of the product.
An embodiment of a third aspect of the present disclosure provides an electronic device, including:
at least one processor; and a memory communicatively coupled to the at least one processor;
wherein the memory stores instructions executable by the at least one processor, the instructions being configured to perform one of the defect pattern recognition methods described above.
A fourth aspect of the present disclosure is directed to a computer-readable storage medium storing computer instructions for causing a computer to execute a defect pattern recognition method as described above.
The characteristics and the beneficial effects of the disclosure are as follows:
the method is based on unsupervised learning, and an unsupervised pre-training model and a defect classification fine-tuning model are respectively established in the learning process. By using the unlabeled images to perform pre-training on the unsupervised pre-training model, rich product feature representations can be obtained, so that the easily obtained unlabeled images are fully utilized. Parameters of the unsupervised and pre-trained model can be used as initialization parameters of the defect classification fine tuning model, so that product characteristics learned by the unsupervised and pre-trained model can be widely transferred to the downstream defect classification fine tuning model. This is a new strategy for defect pattern recognition with minimal human supervision cost.
The method uses the unsupervised pre-training model based on the invariance propagation method, does not need a professional to label the image in a large quantity, and can obviously reduce the artificial influence and the labor cost brought by manual supervision. In addition, the unsupervised model is used for pre-training, so that the image characteristics can be better extracted, and the accuracy of defect mode identification can be effectively improved.
The method uses various data enhancement methods, and can avoid the phenomena that the generalization capability of the model is insufficient for defect categories with small occupation ratio and overfitting phenomenon occurs for defect categories with large occupation ratio due to the unbalanced category in the data set. The universality of the model is improved, and the robustness of the model is enhanced
The invention aims to provide a defect pattern recognition method and device based on unsupervised learning, which can be practically applied to an industrial field, effectively improve recognition precision and efficiency, greatly reduce labor cost and have great application prospect in the aspect of industrial production.
Drawings
Fig. 1 is an overall flowchart of a defect pattern recognition method according to an embodiment of the present disclosure.
Fig. 2 is a diagram of an 18-layer residual neural network in an embodiment of the present disclosure.
Fig. 3 is a process diagram for visualizing recursive transfer of the K-nearest neighbor algorithm in the embodiment of the present disclosure.
Detailed Description
The present disclosure provides a defect pattern recognition method, apparatus, electronic device and storage medium, which are further described in detail below with reference to the accompanying drawings and specific embodiments.
An embodiment of a first aspect of the present disclosure provides a defect pattern recognition method, an overall flow of which is shown in fig. 1, and the method includes the following steps:
step 1, acquiring a data set: the image data of the product is collected on the actual production line of any product, and comprises various defect images and normal images of the product, wherein the normal images are not defective.
And 2, dividing the data set obtained in the step 1: dividing the product images in the data set into a label-free data set and a label data set according to a set proportion (in one embodiment of the disclosure, the proportion is set to be 8: 2, and the label-free data should be more than the label data), wherein the label data set manually marks a defect mode (including various defects and no defects) corresponding to each product image. Wherein, the label-free data set is used as a training set of the pre-training model. Using the label data set for training, verifying and testing the classified fine-tuning model, wherein the label data set is divided into a training set, a verifying set and a testing set according to a set proportion (the proportion is set to be 8: 1: 1 in one embodiment of the disclosure);
in the actual production process, the occurrence probability of various defect modes is different, so that the quantity of various defect mode images in the collected data set is greatly different, the serious imbalance problem of the data set is caused, the generalization capability of the model to small-proportion image categories is insufficient, and the overfitting phenomenon is caused to large-proportion image categories. Aiming at the problem, data enhancement is carried out on the images of the two training sets in step 2, so that the problem that the proportion difference of the number of the images of each type of defect mode in the data set is large is solved, and the robustness of the model is improved. The data enhancement algorithm expands the image for a particular defect mode while preserving the high-level semantic features of the image. To avoid the model being too dependent on homogeneous data, one embodiment of the present disclosure uses 5 data enhancement methods: respectively translation, inversion, random noise addition, rotation and clipping. Image translation is movement in either the vertical or horizontal direction (or both directions at the same time). Flipping is a mirror-like spatial flipping, as opposed to 180 degrees of rotation. Since the model is easy to over-fit when learning the high-frequency features, noise is randomly added into the data set to eliminate the over-fit problem caused by the high-frequency features. Rotation is the random clockwise or counterclockwise rotation of the image. Cropping is the random selection of a portion of the image to be cropped off and then resized to the original image.
Step 3, constructing an unsupervised pre-training model based on an invariance propagation method and setting a loss function of the model;
the constructed unsupervised pre-training model structure mainly comprises the following steps:
taking the 18 layers of residual error neural networks shown in fig. 2 as a backbone network, the convolution kernel size of the 1 st convolution layer is 3 × 3, the number of channels is 3, the step size is 1, and the input of the convolution layer is the image in the unlabeled data set in step 2; a maximum pooling layer is arranged behind the 1 st convolution layer, the convolution kernel size is 3 x 3, and the step length is 2; and 2-17, setting a residual error module for every two convolutional layers, wherein every two residual error modules form a convolution group, and the total number of the convolution groups is 4. The convolution kernel size of the 1 st convolution group is 3 x 3, the number of channels is 64, and the step size is 1; the 2 nd convolution group convolution kernel size is 3 x 3, the number of channels is 128, and the step size is 2; the convolution kernel size of the 3 rd convolution group is 3 x 3, the number of channels is 256, and the step size is 2; the 4 th convolution group convolution kernel size is 3 x 3, the number of channels is 521, and the step size is 2. An averaging pooling layer and a full-link layer are sequentially disposed behind the 17 th convolutional layer.
The unsupervised pre-training model obtains image characteristics through a K-nearest neighbor recursive transfer algorithm in a learning process, any image sample is taken as a target sample in a characteristic space of image samples of the unlabeled data set input by the model, and if K nearest image samples of the target sample belong to the same defect mode, the target sample can also be regarded as the image sample belonging to the defect mode; through the process, the feature representation of the defect mode in different image samples can be widely found, and the specific implementation process is as follows: defining an unlabeled dataset a ═ aiThen map the unlabeled dataset to a low-dimensional set B ═ B }iH (2-dimensional in this embodiment), i ═ 1, 2.., n, where n is the total number of images in the unlabeled dataset; after dimension reduction, the graph with the same semantic features of the defect modeLike the samples are aggregated, the image samples of different semantic features are separated. The probability that any image in the unlabeled dataset is identified as a class r defect mode is defined as:
herein, the
Is a characteristic of the r-th type defect pattern, biIs the ith image a in the unlabeled datasetiThe data after being mapped to the low dimension is,
is the defect feature of the current batch image, and tau is the temperature coefficient of the unsupervised model.
Will any one of aiAs a target sample, the defect mode probability calculation is performed on the low-dimensional mapping set B of the data set a by the method, and a sample with the same defect mode type as the target sample is obtained from the low-dimensional mapping set B, so that an image with the same defect mode type as the target sample in the data set a can be found. Mixing the low-dimensional set with biSample composition target sample a in the same defect modeiThe K nearest neighbor algorithm finds more positive samples belonging to h (i) in the feature space, and also sets the K value in the model to be small to ensure higher semantic consistency of the image. Different examples of the same defect mode can be found in the continuous recursion transfer process of the K-nearest neighbor algorithm, more different image samples can enable the pre-training model to better learn defect representation characteristics, and the learned intra-class semantics are richer without being influenced by intra-class image changes.
Fig. 3 is a process diagram for visualizing recursive transfer of the K-nearest neighbor algorithm in the embodiment of the present disclosure. The whole process is visualized in fig. 3, and the points in fig. 3 represent all samples in the low-dimensional mapping set B, wherein the black points are the target samples, and the gray points are the same samples as the target sample defect patterns. As shown in fig. 3(a), first, a target sample is randomly selected from the set B; as shown in fig. 3(b), after the K-nearest neighbor algorithm, the same sample as the target sample defect pattern begins to be found; as shown in fig. 3(c), the K-nearest neighbor algorithm finds more samples that are the same as the target sample defect pattern. In the K-nearest neighbor recursive pass of each step, the same image sample found as the present defect pattern (i.e., the defect pattern of the target sample) is added to the positive sample set h (i) of the target sample. The product image of the same defect pattern stays in the same high density area, and the same defect pattern sample is more consistent.
The present disclosure uses a new sampling method for defect image data features. In the sampling process, the similar defect mode sample with lower similarity with the target sample image is regarded as a difficult positive sample, and the target sample is more consistent with the difficult positive sample of the similar defect mode. Through the hard sampling process, the model can fully acquire the consistency representation characteristics among different images in the same defect mode.
Images belonging to the same type of defect mode in the label-free data set can be found out through a K nearest neighbor algorithm;
after H (i) of each target sample is obtained through a K nearest neighbor algorithm, further sampling is carried out in a low-dimensional set, and a difficult positive sample set and a difficult negative sample set corresponding to the target samples are obtained; in the embodiment of the present disclosure, the specific implementation process of the sampling method is as follows:
constructing a difficult positive sample set H corresponding to a target samplep(i) (the difficult positive sample set Hp(i) And belongs to a positive sample set H (i) which comprises Q samples with the lowest similarity under the same defect mode as the target sample. The samples in the difficult positive sample set have characteristics that are very different from the target samples, so they can provide more different representation characteristics within the same defect pattern, which is the learning of the pre-trained model. Correspondingly, selecting samples similar to the target sample characteristics in other defect modes from the low-dimensional set B to form a target sample aiDifficult negative sample set Hn(i) In that respect B in low dimensioniZ adjacent samples of (a) constitute a target sample aiIs close toSet of samples Hz(i) Here, Z is set to be large enough that Hz(i) Partial negative examples are included. So that the target specimen aiThe difficult negative sample set of (2) can be expressed as: hn(i)＝Hz(i) -H (i). Under the sampling method, each image a in the label-free data set can be definediLoss function (i.e. the loss function of the unsupervised pre-trained model):
where tau is the temperature coefficient and,
is a set Hp(i) The characteristics of the image of the medium sample,
is a set Hn(i)∪Hp(i) The image characteristics of the middle sample.
Step 4, inputting the label-free data set into the constructed unsupervised pre-training model for training, and obtaining the parameters of the unsupervised pre-training model and the unsupervised pre-training model after training, wherein the set iteration times are reached (all training set samples are trained by the model in each iteration);
and 5, constructing a classification fine-tuning model, wherein the initialization parameters of the classification fine-tuning model are the parameters of the unsupervised pre-training model, and then setting the loss function of the classification fine-tuning model. Training the classification fine-tuning model by using a training set in a label data set, inputting a verification set after training for a fixed number of times to detect the accuracy of the classification fine-tuning model, and selecting the classification fine-tuning model with the highest identification accuracy on the verification set as a defect mode identification model;
the backbone network of the classification fine-tuning model uses an 18-layer residual neural network. The network has the following specific structure: the convolution kernel size of the 1 st convolution layer is 3 x 3, the number of channels is 3, the step length is 1, and the input of the convolution layer is the label image in the step 2; a maximum pooling layer is arranged behind the 1 st convolution layer, the convolution kernel size is 3 x 3, and the step length is 2; and 2-17, setting a residual error module for every two convolutional layers, wherein every two residual error modules form a convolution group, and the total number of the convolution groups is 4. The convolution kernel size of the 1 st convolution group is 3 x 3, the number of channels is 64, and the step size is 1; the 2 nd convolution group convolution kernel size is 3 x 3, the number of channels is 128, and the step size is 2; the convolution kernel size of the 3 rd convolution group is 3 x 3, the number of channels is 256, and the step size is 2; the 4 th convolution group convolution kernel size is 3 x 3, the number of channels is 521, and the step size is 2. An average pooling layer and a full-connection layer are arranged behind the 17 th convolution layer; and arranging a softmax layer behind the full connection layer, wherein the softmax layer has the main function of mapping the output of a plurality of neurons into a (0, 1) interval, namely the probability of each type of defect mode corresponding to the input image, and the softmax function is as follows:
w hereiniRepresenting the ith image in the label dataset.
Then defining a loss function of the classification fine tuning model, giving a training set with m different defect mode labels, and using a multi-classification cross entropy loss function as a loss function L of the classification fine tuning model:
where p denotes the distribution of the true tags and q is the distribution of the model predicted tags.
In order to prevent the model from being over-fitted, dropout is set at the full connection layer of the classification fine tuning model, and the fixed probability is 0.5. Generally, the known training data set is small. When training an artificial neural network with a limited training data set, the neural network model can suffer from an overfitting problem. dropout is an effective technique to help reduce model overfitting;
and inputting the training set in the label data set into the constructed classification fine-tuning model for training, so as to achieve the set iteration times (all training set samples are trained through the model in each iteration), and obtaining the trained classification fine-tuning model after the training is finished. And then, detecting the accuracy of the classification fine-tuning model by using the verification set, and selecting the classification fine-tuning model with the highest identification accuracy on the verification set as a defect mode identification model. And 6, using the defect mode identification model obtained in the step 5 to perform identification test on various defect mode images and normal non-defective images of the test set, and obtaining the identification accuracy of the model on the defect mode of the product.
And 7, acquiring the product image in the actual production process, inputting the product image into a defect mode identification model, outputting the probability of each defect mode corresponding to the product in the image by the model, and automatically selecting the defect mode with the highest probability by the model as the defect mode identification result of the product in the input image. Where non-defective is also a category of defective modes in this disclosure.
In order to achieve the above embodiments, a second aspect of the present disclosure provides a defect pattern recognition apparatus, including: the acquisition module is used for acquiring a product image in the production process; the recognition module is used for recognizing the defect mode of the product in the product image according to a preset defect mode recognition model; and the preset defect mode identification model outputs the probability of each defect mode corresponding to the product in the product image, and selects the defect mode with the maximum probability as the defect mode identification result of the product.
In order to achieve the above embodiments, an embodiment of a third aspect of the present disclosure provides an electronic device, including: at least one processor; and a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being configured to perform a method of defect pattern recognition of the above embodiments.
In order to implement the above embodiments, a fourth aspect embodiment of the present disclosure proposes a computer-readable storage medium, on which a computer program is stored, the program being executed by a processor for executing a defect pattern recognition method of the above embodiments.
It should be noted that the computer readable medium in the present disclosure can be a computer readable signal medium or a computer readable storage medium or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present disclosure, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In contrast, in the present disclosure, a computer readable signal medium may comprise a propagated data signal with computer readable program code embodied therein, either in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: electrical wires, optical cables, RF (radio frequency), etc., or any suitable combination of the foregoing.
The computer readable medium may be embodied in the electronic device; or may exist separately without being assembled into the electronic device. The computer readable medium carries one or more programs which, when executed by the electronic device, cause the electronic device to perform a defect pattern recognition method of the above embodiments.
Computer program code for carrying out operations for aspects of the present disclosure may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C + +, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider).
In the description herein, reference to the description of the term "one embodiment," "some embodiments," "an example," "a specific example," or "some examples," etc., means that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the application. In this specification, the schematic representations of the terms used above are not necessarily intended to refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Furthermore, various embodiments or examples and features of different embodiments or examples described in this specification can be combined and combined by one skilled in the art without contradiction.
Furthermore, the terms "first", "second" and "first" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one such feature. In the description of the present application, "plurality" means at least two, e.g., two, three, etc., unless specifically limited otherwise.
Any process or method descriptions in flow charts or otherwise described herein may be understood as representing modules, segments, or portions of code which include one or more executable instructions for implementing specific logical functions or steps of the process, and the scope of the preferred embodiments of the present application includes other implementations in which functions may be executed out of order from that shown or discussed, including substantially concurrently or in reverse order, depending on the functionality involved, as would be understood by those reasonably skilled in the art of the present application.
The logic and/or steps represented in the flowcharts or otherwise described herein, e.g., an ordered listing of executable instructions that can be considered to implement logical functions, can be embodied in any computer-readable medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, processor-containing system, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions. For the purposes of this description, a "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of the computer-readable medium would include the following: an electrical connection (electronic device) having one or more wires, a portable computer diskette (magnetic device), a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disc read-only memory (CDROM). Additionally, the computer-readable medium could even be paper or another suitable medium upon which the program is printed, as the program can be electronically captured, via for instance optical scanning of the paper or other medium, then compiled, interpreted or otherwise processed in a suitable manner if necessary, and then stored in a computer memory.
It should be understood that portions of the present application may be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, the various steps or methods may be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, any one or combination of the following techniques, which are known in the art, may be used: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application specific integrated circuit having an appropriate combinational logic gate circuit, a Programmable Gate Array (PGA), a Field Programmable Gate Array (FPGA), or the like.
It will be understood by those skilled in the art that all or part of the steps carried by the method for implementing the above embodiments may be implemented by hardware that is related to instructions of a program, and the program may be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.
In addition, functional units in the embodiments of the present application may be integrated into one processing module, or each unit may exist alone physically, or two or more units are integrated into one module. The integrated module can be realized in a hardware mode, and can also be realized in a software functional module mode. The integrated module, if implemented in the form of a software functional module and sold or used as a separate product, may also be stored in a computer readable storage medium. The storage medium mentioned above may be a read-only memory, a magnetic or optical disk, etc. Although embodiments of the present application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present application, and that variations, modifications, substitutions and alterations may be made to the above embodiments by those of ordinary skill in the art within the scope of the present application.
Claims (10)
1. A method for defect pattern recognition, comprising:
collecting a product image in the production process;
identifying a defect mode of a product in the product image according to a preset defect mode identification model; and the preset defect mode identification model outputs the probability of each defect mode corresponding to the product in the product image, and selects the defect mode with the maximum probability as the defect mode identification result of the product.
2. The method according to claim 1, before the identifying the defect pattern of the product in the product image according to the preset defect pattern identification model, further comprising:
training the defect pattern recognition model;
wherein the training the defect pattern recognition model comprises:
acquiring an image composition data set of the product in a production process, wherein the image comprises a defect image and a normal image of the product;
dividing the dataset into a label-free dataset and a label dataset; wherein the label dataset comprises an image and a defect pattern of the product in the image;
constructing an unsupervised pre-training model, and training the unsupervised pre-training model by using the unlabeled data set to obtain parameters of the unsupervised pre-training model;
and constructing a classification fine-tuning model, taking the parameters of the unsupervised pre-training model as initialization parameters of the classification fine-tuning model, training the classification fine-tuning model by using the label data set, and taking the trained classification fine-tuning model as a defect pattern recognition model.
3. The method of claim 2, wherein training the classification fine-tuning model using the set of tag data comprises:
dividing the label data set into a training set, a verification set and a test set according to a set proportion;
setting a loss function of the classification fine tuning model;
training the classification fine-tuning model by using a training set in the label data set according to the loss function to reach the set iteration number, and obtaining a trained classification fine-tuning model;
and detecting the accuracy of the trained classification fine tuning model by using the verification set, and selecting the classification fine tuning model with the highest identification accuracy on the verification set as a defect mode identification model.
4. The method of claim 3, further comprising, prior to said training the unsupervised pre-training model with the unlabeled dataset, image enhancing images in the unlabeled dataset;
before training the classification fine-tuning model by using the training set in the label data set, performing image enhancement on images in the training set in the label data set;
the image enhancement includes: one or more of translation, flipping, random noise addition, rotation, and cropping.
5. The method of any of claims 2-4, wherein training the unsupervised pre-training model using the unlabeled dataset comprises:
respectively taking each image in the unlabeled data set as a target sample;
according to a K nearest neighbor recursive transfer algorithm, mapping the label-free data set to a low dimension to obtain a corresponding low-dimension data set, and finding out samples belonging to the same defect mode with the target sample from the low-dimension data set to form a positive sample set of the target sample;
acquiring a difficult positive sample set and a difficult negative sample set of the target sample from the low-dimensional data set by sampling by using the positive sample set of the target sample;
constructing a loss function of each image in the unlabeled data set according to the difficult positive sample set and the difficult negative sample set;
and inputting the label-free data set into the classification fine-tuning model for training according to the loss function to reach the set iteration times, so as to obtain a trained unsupervised pre-training model.
6. The method of claim 5, wherein the K-nearest neighbor recursive transfer algorithm comprises:
recording the unlabeled dataset as A ═ aiMapping the unlabeled dataset to a low-dimensional set B ═ B }i-i ═ 1,2, ·, n }, where n is the total number of images in the unlabeled dataset;
any image a in the unlabeled datasetiThe probability of being identified as an r-th class defect pattern is:
wherein the content of the first and second substances,
is characteristic of the r-th type of defect pattern, biIs the ith image a in the unlabeled datasetiThe data after being mapped to the low dimension is,
is the defect characteristics of the current batch of images, and tau is the temperature coefficient of the unsupervised model;
a is to be describediAs a target sample, obtaining a sample composition a with the same type as the defect mode of the target sample from the low-dimensional set B according to the calculated probabilityiThe positive sample set of (a), (b);
the sampling method comprises the following specific steps:
selecting the target sample a from the positive sample set H (i)iQ sample compositions a with lowest similarity under the same defect modeiDifficult positive sample set Hp(i) (ii) a Selecting other defect mode inner and target from the low-dimensional set BSample composition a with similar characteristics of standard sampleiDifficult negative sample set Hn(i) (ii) a Selecting B from the low-dimensional set BiZ adjacent samples of (a) constitute the target sample aiAdjacent sample set H ofz(i) Wherein said H isz(i) Comprising the above-mentioned Hn(i) A middle negative sample; the target sample aiIs expressed as: hn(i)＝Hz(i)-H(i)。
7. The method of claim 6, wherein the loss function for each image in the unlabeled dataset is:
wherein the content of the first and second substances,
is a set Hp(i) The characteristics of the image of the medium sample,
is a set Hn(i)∪Hp(i) The image characteristics of the middle sample.
8. A defect pattern recognition apparatus, comprising:
the acquisition module is used for acquiring a product image in the production process;
the recognition module is used for recognizing the defect mode of the product in the product image according to a preset defect mode recognition model; and the preset defect mode identification model outputs the probability of each defect mode corresponding to the product in the product image, and selects the defect mode with the maximum probability as the defect mode identification result of the product.
9. An electronic device, comprising:
at least one processor; and a memory communicatively coupled to the at least one processor;
wherein the memory stores instructions executable by the at least one processor, the instructions being arranged to perform the method of any of the preceding claims 1-7.
10. A computer-readable storage medium storing computer instructions for causing a computer to perform the method of any one of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110808135.0A CN113591948A (en) | 2021-07-16 | 2021-07-16 | Defect pattern recognition method and device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110808135.0A CN113591948A (en) | 2021-07-16 | 2021-07-16 | Defect pattern recognition method and device, electronic equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113591948A true CN113591948A (en) | 2021-11-02 |
Family
ID=78247745
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110808135.0A Pending CN113591948A (en) | 2021-07-16 | 2021-07-16 | Defect pattern recognition method and device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113591948A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20220091576A1 (en) * | 2020-09-24 | 2022-03-24 | International Business Machines Corporation | Detection of defect in edge device manufacturing by artificial intelligence |
| CN115129019A (en) * | 2022-08-31 | 2022-09-30 | 合肥中科迪宏自动化有限公司 | Training method of production line fault analysis model and production line fault analysis method |
| CN115660596A (en) * | 2022-11-03 | 2023-01-31 | 创启科技(广州)有限公司 | Data interaction method of mobile terminal and mobile terminal |
| CN115908407A (en) * | 2023-01-05 | 2023-04-04 | 佰聆数据股份有限公司 | Power equipment defect detection method and device based on infrared image temperature value |
-
2021
- 2021-07-16 CN CN202110808135.0A patent/CN113591948A/en active Pending
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20220091576A1 (en) * | 2020-09-24 | 2022-03-24 | International Business Machines Corporation | Detection of defect in edge device manufacturing by artificial intelligence |
| US11493901B2 (en) * | 2020-09-24 | 2022-11-08 | International Business Machines Corporation | Detection of defect in edge device manufacturing by artificial intelligence |
| CN115129019A (en) * | 2022-08-31 | 2022-09-30 | 合肥中科迪宏自动化有限公司 | Training method of production line fault analysis model and production line fault analysis method |
| CN115660596A (en) * | 2022-11-03 | 2023-01-31 | 创启科技(广州)有限公司 | Data interaction method of mobile terminal and mobile terminal |
| CN115660596B (en) * | 2022-11-03 | 2023-07-04 | 创启科技(广州)有限公司 | Data interaction method of mobile terminal and mobile terminal |
| CN115908407A (en) * | 2023-01-05 | 2023-04-04 | 佰聆数据股份有限公司 | Power equipment defect detection method and device based on infrared image temperature value |
| CN115908407B (en) * | 2023-01-05 | 2023-05-02 | 佰聆数据股份有限公司 | Power equipment defect detection method and device based on infrared image temperature value |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113591948A (en) | Defect pattern recognition method and device, electronic equipment and storage medium | |
| CN111444939B (en) | Small-scale equipment component detection method based on weak supervision cooperative learning in open scene of power field | |
| CN116188475B (en) | Intelligent control method, system and medium for automatic optical detection of appearance defects | |
| CN112766110A (en) | Training method of object defect recognition model, object defect recognition method and device | |
| CN112949408B (en) | Real-time identification method and system for target fish passing through fish channel | |
| CN111008641B (en) | Power transmission line tower external force damage detection method based on convolutional neural network | |
| CN116030237A (en) | Industrial defect detection method and device, electronic equipment and storage medium | |
| CN114155213A (en) | Chip defect detection method and device based on active learning | |
| CN112419268A (en) | Method, device, equipment and medium for detecting image defects of power transmission line | |
| CN111476307A (en) | Lithium battery surface defect detection method based on depth field adaptation | |
| CN113781483B (en) | Industrial product appearance defect detection method and device | |
| CN114781520A (en) | Natural gas behavior abnormity detection method and system based on improved LOF model | |
| CN113191419B (en) | Sag homologous event detection and type identification method based on track key point matching and region division | |
| CN113628252A (en) | Method for detecting gas cloud cluster leakage based on thermal imaging video | |
| CN112396580A (en) | Circular part defect detection method | |
| CN110704678B (en) | Evaluation sorting method, evaluation sorting system, computer device and storage medium | |
| CN115761359A (en) | Photovoltaic image defect classification method based on transfer learning and unsupervised learning method | |
| CN115511798A (en) | Pneumonia classification method and device based on artificial intelligence technology | |
| CN112949634A (en) | Bird nest detection method for railway contact network | |
| CN111369508A (en) | Defect detection method and system for metal three-dimensional lattice structure | |
| CN116883390B (en) | Fuzzy-resistant semi-supervised defect detection method, device and storage medium | |
| Balaram et al. | Software Fault Detection using Multi-Distinguished-Features Sampling with Ensemble Random Forest Classifier. | |
| Huang et al. | Quality control on manufacturing computer keyboards using multilevel deep neural networks | |
| CN113808079B (en) | Industrial product surface defect self-adaptive detection method based on deep learning model AGLNet | |
| CN116486178B (en) | Defect detection method and device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |