CN113362313A - Defect detection method and system based on self-supervision learning - Google Patents
Defect detection method and system based on self-supervision learning Download PDFInfo
- Publication number
- CN113362313A CN113362313A CN202110679316.8A CN202110679316A CN113362313A CN 113362313 A CN113362313 A CN 113362313A CN 202110679316 A CN202110679316 A CN 202110679316A CN 113362313 A CN113362313 A CN 113362313A
- Authority
- CN
- China
- Prior art keywords
- image
- distance matrix
- model
- self
- sub
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 230000007547 defect Effects 0.000 title claims abstract description 80
- 238000001514 detection method Methods 0.000 title claims abstract description 37
- 239000011159 matrix material Substances 0.000 claims abstract description 74
- 239000013598 vector Substances 0.000 claims abstract description 24
- 238000000034 method Methods 0.000 claims abstract description 20
- 238000001914 filtration Methods 0.000 claims abstract description 8
- 238000009499 grossing Methods 0.000 claims abstract description 8
- 238000005070 sampling Methods 0.000 claims abstract description 8
- 230000009466 transformation Effects 0.000 claims abstract description 8
- 238000012549 training Methods 0.000 claims description 20
- 238000013528 artificial neural network Methods 0.000 claims description 7
- 230000002950 deficient Effects 0.000 claims description 7
- 238000000605 extraction Methods 0.000 claims description 6
- 238000004364 calculation method Methods 0.000 claims description 5
- 238000009826 distribution Methods 0.000 claims description 5
- 230000000052 comparative effect Effects 0.000 claims description 3
- 230000008602 contraction Effects 0.000 claims description 3
- 238000013527 convolutional neural network Methods 0.000 claims description 2
- 238000003062 neural network model Methods 0.000 claims description 2
- 230000000306 recurrent effect Effects 0.000 claims description 2
- 230000008569 process Effects 0.000 description 7
- 238000004519 manufacturing process Methods 0.000 description 6
- 238000011179 visual inspection Methods 0.000 description 6
- 238000010586 diagram Methods 0.000 description 4
- 238000011161 development Methods 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 230000003449 preventive effect Effects 0.000 description 3
- 238000005457 optimization Methods 0.000 description 2
- 230000001737 promoting effect Effects 0.000 description 2
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 239000000470 constituent Substances 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 230000008451 emotion Effects 0.000 description 1
- 238000009776 industrial production Methods 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000010606 normalization Methods 0.000 description 1
- 230000008520 organization Effects 0.000 description 1
- 238000012805 post-processing Methods 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0004—Industrial image inspection
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/84—Systems specially adapted for particular applications
- G01N21/88—Investigating the presence of flaws or contamination
- G01N21/8851—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2411—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on the proximity to a decision surface, e.g. support vector machines
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/084—Backpropagation, e.g. using gradient descent
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/84—Systems specially adapted for particular applications
- G01N21/88—Investigating the presence of flaws or contamination
- G01N21/8851—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
- G01N2021/8887—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges based on image processing techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20172—Image enhancement details
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30108—Industrial image inspection
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Computation (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Computational Linguistics (AREA)
- Biophysics (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Signal Processing (AREA)
- Chemical & Material Sciences (AREA)
- Analytical Chemistry (AREA)
- Biochemistry (AREA)
- Immunology (AREA)
- Pathology (AREA)
- Quality & Reliability (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a defect detection method and a system based on self-supervision learning, wherein the method comprises the following steps: dividing the image into a plurality of sub-images with overlapping; extracting a feature vector of each sub-image by using a feature model of self-supervision learning; calculating the distance between the feature vector of each sub-image and the first type of model of the corresponding sub-image to obtain a distance matrix; performing up-sampling transformation on the distance matrix to obtain a distance matrix with the size consistent with that of the original image; smoothing the distance matrix by adopting filtering and normalizing to obtain a normalized distance matrix; judging the relationship between the maximum value in the normalized distance matrix and a preset image-level defect threshold value to determine whether defects exist or not; the relationship between all values of the normalized distance matrix and a predetermined pixel level defect threshold is determined to determine the location and size of the presence of a defect. The invention can solve the problem of poor robustness of a defect detection system under the conditions of multiple object defect types, few or no defect samples.
Description
Technical Field
The invention relates to a defect detection technology, in particular to a defect detection method and system based on self-supervision learning.
Background
The industrial internet is a brand new industrial ecology, key infrastructure and novel application mode which are deeply integrated by a new generation of information communication technology and industrial economy. The system takes the network as a basis, the platform as a center, the data as elements and the safety as a guarantee, comprehensively connects people, machines and objects, changes the traditional manufacturing mode, the production organization mode and the industrial form, constructs a novel industrial production manufacturing and service system with all elements, all industrial chains, all value chains and comprehensive connection, and has very important significance for supporting and manufacturing the construction of the strong country and the strong network country, promoting the modernization level of the industrial chains, promoting the development of economic high quality and constructing new development pattern.
In the industrial internet, the preventive detection of products is a crucial link, defects generated by the products and the change trend of the defects can be found in time through the preventive detection, and the production process is optimized by combining with on-site real-time process parameters, so that the defective rate is reduced, the production cost is reduced, and the profits of companies are improved. The preventive detection of the product is mainly divided into three stages: real-time process parameter acquisition, product defect detection and closed-loop optimization of the process.
In the product defect detection stage, the common practice in the industry is to adopt a visual inspection mode of professionals. The following problems exist in the manual visual inspection mode: (1) professional visual inspection personnel need to be trained; (2) meanwhile, the manual visual inspection has certain subjectivity, and the judgment capability of visual inspection personnel is influenced by factors such as self emotion, physical condition and the like; (3) the manual visual inspection is post-processing, and has no positive influence on the real-time whole process flow optimization.
With the development of defect detection technology, it has become a trend to predict product defects by using machine learning and artificial intelligence techniques. In a real scene, the defect types are more, samples of defect products are less or even no, and the robustness of a defect detection system is poor. To solve this problem, the method comprises: dividing the image into a plurality of sub-images with overlapping according to a preset mode; extracting a feature vector of each sub-image by using a feature model of self-supervision learning; calculating the distance between the feature vector of each sub-image and the class of model of the corresponding sub-image to obtain a distance matrix; performing up-sampling transformation on the distance matrix according to a preset mode to obtain the distance matrix with the size consistent with that of the original image; smoothing the distance matrix by adopting filtering, and normalizing to obtain a normalized distance matrix; determining whether a defect exists by determining a relationship between a maximum value in the normalized distance matrix and a predetermined image-level defect threshold; the location and size of the presence of a defect is determined by determining the relationship between all values of the normalized distance matrix and a predetermined pixel level defect threshold.
Disclosure of Invention
The invention aims to provide a defect detection method and system based on self-supervision learning, which are used for solving the problem of poor robustness of a defect detection system due to multiple defect types, few or no samples of defective products.
In order to achieve the purpose, the invention provides the following technical scheme:
a defect detection method based on self-supervision learning comprises the following steps:
s1, dividing the image into a plurality of sub-images;
s2: extracting a feature vector of each sub-image by using a feature model of self-supervision learning;
s3: calculating the distance between the feature vector of each sub-image and the class of model of the corresponding sub-image to obtain a distance matrix;
s4, performing up-sampling transformation on the distance matrix to obtain a distance matrix with the size consistent with that of the original image;
s5: smoothing the distance matrix by adopting filtering, and normalizing to obtain a normalized distance matrix;
s6: determining whether a defect exists by determining a relationship between a maximum value in the normalized distance matrix and a predetermined image-level defect threshold;
s7: the location and size of the presence of a defect is determined by determining the relationship between all values of the normalized distance matrix and a predetermined pixel level defect threshold.
The further technical scheme is that the feature model of the self-supervision learning is formed by a deep neural network, and the deep neural network is selected from one or more of a convolutional neural network, a recurrent neural network and a time-delay neural network.
The further technical scheme is that the one-class model is selected from any one of a clustering model, a multivariate Gaussian model or a single-class SVM.
The further technical scheme is that the multivariate Gaussian model comprises the following steps:
wherein x ∈ RnDenotes the feature vector, μ ∈ RnRepresents the mean value of the sample,. sigma.. din*nA covariance matrix representing the samples;
because the number of training samples is rare, the characteristic dimensionality of the samples is high, and the covariance matrix needs to be shrunk to more accurately fit the distribution of the samples, therefore,
where ρ denotes the contraction factor, tr (×) denotes the traces of the matrix, InRepresenting an identity matrix.
The further technical scheme is that the distance between the feature vector of each sub-image and the model of the corresponding sub-image is selected from any one of mahalanobis distance, Euclidean distance or cosine distance.
The further technical scheme is that the Mahalanobis distance is calculated in the following manner:
the further technical scheme is that the training method of the feature model for the self-supervision learning comprises the following steps:
s00: dividing the non-defective training set image into a plurality of sub-images;
s01: inputting the subimages in the S00 into a feature model of the self-supervision learning, and extracting feature vectors of the subimages; and outputs the total error Losstotal；
Wherein, the comparative learning error is:
Losscontrast＝(1-cos_sim(fθ(p1)，fθ(p2)))-(1-cos_sim(fθ(p1)，fθ(p3)))
neighborhood position classification error:
Lossposition＝Cross_entropy(y，Cφ(fθ(p1)，fθ(p3)))
thus, the overall error of the network is:
Losstotal＝Losscontrast+Lossposition
wherein p is1For selected subimages, p2Is a reaction of with p1Subimages with a certain intersection or p1Image enhanced, p3Is a reaction of with p1Uncorrelated subimages, cos _ sim is the cosine similarity, fθFor the parameters of the feature model for the supervised learning, Cross _ entropy is the Cross entropy, CφFor the classifier, y is p3Relative to p1Y e {0, 1, 2, 3, 4, 5, 6, 7 }.
S02: updating weights
Wherein ε is the learning rate;
s03: repeating the steps of S00 to S02 until the feature model of the self-supervised learning converges.
The invention provides a defect detection system based on self-supervision learning, which comprises the following modules:
a sub-image dividing module for dividing the image into a plurality of sub-images having an overlap;
the characteristic extraction module is used for extracting a characteristic vector of each sub-image by using a characteristic model of self-supervision learning;
the distance matrix calculation module is used for calculating the pixel-level distance of the image to be measured, and comprises: calculating the distance between the feature vector of each sub-image and the class of model of the corresponding sub-image to obtain a distance matrix; performing up-sampling transformation on the distance matrix according to a preset obtaining mode to obtain the distance matrix with the size consistent with that of the original image; smoothing the distance matrix by adopting filtering, and normalizing to obtain a normalized distance matrix;
the defect detection module is used for detecting whether defects exist, positions and sizes, and comprises: determining whether a defect exists by determining a relationship between a maximum value in the normalized distance matrix and a predetermined image-level defect threshold; the location and size of the presence of a defect is determined by determining the relationship between all values of the normalized distance matrix and a predetermined pixel level defect threshold.
The further technical scheme is that the feature extraction module further comprises a deep neural network model for self-supervision learning and a model training component.
The further technical scheme is that the distance matrix calculation module also comprises a class of model building components.
Compared with the prior art, the invention has the following beneficial effects: the defect detection method and the defect detection system provided by the invention can still ensure the robustness of the product defect detection system under the conditions of more defect types, less samples of defective products and even no samples.
Drawings
FIG. 1 is a schematic diagram of a defect detection method based on self-supervised learning;
FIG. 2 is a schematic diagram of feature model training of the auto-supervised learning in the defect detection method based on the auto-supervised learning;
FIG. 3 is a schematic diagram of a multivariate Gaussian distribution model establishment in a defect detection method based on self-supervised learning;
fig. 4 is a schematic diagram of a defect detection system based on the self-supervised learning.
Detailed Description
The invention will be further explained and explained with reference to the drawings and the embodiments.
Example 1
As shown in fig. 1, fig. 2 and fig. 3, the present invention provides a defect detection method based on self-supervised learning, specifically:
in step S101, the method is used for dividing an image into a plurality of sub-images with overlapping in a predetermined manner;
in step S102, extracting a feature vector of each sub-image in step S101 using a feature model obtained by the self-supervised learning;
in step S103, a distance between the feature vector of each sub-image and the class of models of the corresponding sub-image is calculated to obtain a distance matrix. The Mahalanobis distance is calculated in the following manner:
in step S104, performing up-sampling transformation on the distance matrix in a predetermined manner to obtain a distance matrix having the same size as the original image;
in step S105, smoothing the distance matrix by filtering and performing normalization processing to obtain a normalized distance matrix;
in step S106, determining whether there is a defect by judging a relationship between the maximum value in the normalized distance matrix and a predetermined image-level defect threshold;
in step S107, the position and size of the presence of a defect are determined by judging the relationship between all values of the normalized distance matrix and a predetermined pixel-level defect threshold.
Meanwhile, the defect detection method based on the self-supervision learning further comprises the characteristic model training of the self-supervision learning.
In step S201, a defect-free training set image is collected;
in step S202, dividing the non-defective training set image into a plurality of sub-images according to a predetermined manner;
in step S203, the sub-image in S202 is input into the feature model of the self-supervised learning, the feature vector of the sub-image is extracted, and the prediction result of the sub-image is obtained.
In step S204, the self-supervised learning only needs defect-free training samples, and the sum of the contrast learning error and the neighborhood position classification error is used as the error for training the whole network. Wherein, the comparative learning error is:
Losscontrast＝(1-cos_sim(fθ(p1)，fθ(p2)))-(1-cos_sim(fθ(p1)，fθ(p3)))
neighborhood position classification error:
Lossposition＝Cross_entropy(y，Cφ(fθ(p1)，fθ(p3)))
thus, the overall error of the network is:
Losstotal＝Losscontrast+Lossposition
wherein p is1For selected subimages, p2Is a reaction of with p1Subimages with a certain intersection or p1Image enhanced image, p3Is a reaction of with p1Uncorrelated subimages, cos _ sim is the cosine similarity, fθCross _ entry is the Cross entropy, a parameter of a feature model for unsupervised learning，CφFor the classifier, y is p3Relative to p1Y e {0, 1, 2, 3, 4, 5, 6, 7 }.
According to the prediction result of the sub-image S203, the contrast learning error and the neighborhood position classification error are calculated to obtain the total error Losstotal。
In step S205, it is determined whether convergence occurs or a set maximum number of iterations or a set minimum error is reached, and if not, the process goes to step S206 to continue training, otherwise, the process goes to step S207 to end training.
In step S206, the parameters of the feature model of the unsupervised learning are updated by means of back propagation, and the process goes to step S203. The updating mode of the parameters is as follows:
where ε is the learning rate.
Meanwhile, the defect detection method based on the self-supervision learning also comprises the establishment of a class of models. The multivariate Gaussian distribution model comprises the following components:
wherein x ∈ RnDenotes the feature vector, μ ∈ RnRepresents the mean value of the sample,. sigma.. din*nA covariance matrix of the samples is represented.
Because the number of training samples is rare, the characteristic dimensionality of the samples is high, and the covariance matrix needs to be shrunk to more accurately fit the distribution of the samples, therefore,
where ρ denotes the contraction factor, tr (×) denotes the traces of the matrix, InRepresenting an identity matrix.
The embodiment of the invention provides a defect detection method based on self-supervision learning, which solves the problems of poor robustness of a defect detection system caused by multiple defect types, few or no samples of defective products.
Example 2
As shown in fig. 4, the present invention provides a defect detection system based on self-supervised learning, and fig. 4 shows the constituent modules of the system. Referring to fig. 4, the system includes a sub-image division module 401, a feature extraction module 402, a distance matrix calculation module 403, a defect detection module 404, a feature model training component 405 for self-supervised learning, and a class model building component 406.
A sub-image dividing module 401, configured to divide an image into a plurality of overlapping sub-images according to a predetermined manner;
a feature extraction module 402, configured to extract a feature vector of each sub-image using a feature model of self-supervised learning;
a distance matrix calculation module 403, configured to calculate a pixel-level distance of the image to be measured, including: calculating the distance between the feature vector of each sub-image and the class of model of the corresponding sub-image to obtain a distance matrix; performing up-sampling transformation on the distance matrix according to a preset obtaining mode to obtain the distance matrix with the size consistent with that of the original image; smoothing the distance matrix by adopting filtering, and normalizing to obtain a normalized distance matrix;
the defect detecting module 404 is configured to detect whether a defect exists, a position, and a size, and includes: determining whether a defect exists by determining a relationship between a maximum value in the normalized distance matrix and a predetermined image-level defect threshold; determining the position and size of the defect by judging the relation between all values of the normalized distance matrix and a preset pixel-level defect threshold;
a feature model training component 405 for self-supervised learning, the training mode referring to example 1;
a class model building component 406, wherein the multivariate gaussian model building method is described with reference to example 1.
Although the present invention has been described herein with reference to the illustrated embodiments thereof, which are intended to be preferred embodiments of the present invention, it is to be understood that the invention is not limited thereto, and that numerous other modifications and embodiments can be devised by those skilled in the art that will fall within the spirit and scope of the principles of this disclosure.
Claims (10)
1. A defect detection method based on self-supervision learning is characterized by comprising the following steps:
s1: dividing the image into a plurality of sub-images;
s2: extracting a feature vector of each sub-image by using a feature model of self-supervision learning;
s3: calculating the distance between the feature vector of each sub-image and the class of model of the corresponding sub-image to obtain a distance matrix;
s4: performing up-sampling transformation on the distance matrix to obtain a distance matrix with the size consistent with that of the original image;
s5: smoothing the distance matrix by adopting filtering, and normalizing to obtain a normalized distance matrix;
s6: determining whether a defect exists by determining a relationship between a maximum value in the normalized distance matrix and a predetermined image-level defect threshold;
s7: the location and size of the presence of a defect is determined by determining the relationship between all values of the normalized distance matrix and a predetermined pixel level defect threshold.
2. The method for defect detection based on self-supervised learning as recited in claim 1, wherein the feature model of self-supervised learning is composed of a deep neural network selected from one or more of a convolutional neural network, a recurrent neural network and a time-delay neural network.
3. The method of claim 1, wherein the class model is selected from any one of a clustering model, a multivariate Gaussian model and a single-class SVM.
4. The method of claim 3, wherein the multivariate Gaussian model is as follows:
wherein x ∈ RnDenotes the feature vector, μ ∈ RnRepresents the mean value of the sample,. sigma.. din*nA covariance matrix representing the samples;
because the number of training samples is rare, the characteristic dimensionality of the samples is high, and the covariance matrix needs to be shrunk to more accurately fit the distribution of the samples, therefore,
where ρ denotes the contraction factor, tr (×) denotes the traces of the matrix, InRepresenting an identity matrix.
5. The method of claim 1, wherein the distance between the feature vector of each sub-image and the model of the corresponding sub-image is selected from any one of mahalanobis distance, euclidean distance, or cosine distance.
6. The method of claim 5, wherein the Mahalanobis distance is calculated as follows:
7. the method for defect detection based on self-supervised learning as recited in claim 1, wherein the method for training the feature model of self-supervised learning comprises the following steps:
s00: dividing the non-defective training set image into a plurality of sub-images;
s01: inputting the subimages in the S00 into a feature model of the self-supervision learning, and extracting feature vectors of the subimages; and outputs the total error Losstotal；
Wherein, the comparative learning error is:
Losscontrast＝(1-cos_sim(fθ(p1)，fθ(p2)))-(1-cos_sim(fθ(p1)，fθ(p3)))
neighborhood position classification error:
Lossposition＝Cross_entropy(y，Cφ(fθ(p1)，fθ(p3)))
thus, the overall error of the network is:
Losstotal＝Losscontrast+Lossposition
wherein p is1For selected subimages, p2Is a reaction of with p1Subimages with a certain intersection or p1Image enhanced, p3Is a reaction of with p1Uncorrelated subimages, cos _ sim is the cosine similarity, fθFor the parameters of the feature model for the supervised learning, Cross _ entropy is the Cross entropy, CφFor the classifier, y is p3Relative to p1Y e {0, 1, 2, 3, 4, 5, 6, 7 }.
S02: updating weights
Wherein ε is the learning rate;
s03: repeating the steps of S00 to S02 until the feature model of the self-supervised learning converges.
8. A defect detection system based on self-supervision learning is characterized by comprising the following modules:
a sub-image dividing module for dividing the image into a plurality of sub-images having an overlap;
the characteristic extraction module is used for extracting a characteristic vector of each sub-image by using a characteristic model of self-supervision learning;
the distance matrix calculation module is used for calculating the pixel-level distance of the image to be measured, and comprises: calculating the distance between the feature vector of each sub-image and the class of model of the corresponding sub-image to obtain a distance matrix; performing up-sampling transformation on the distance matrix according to a preset obtaining mode to obtain the distance matrix with the size consistent with that of the original image; smoothing the distance matrix by adopting filtering, and normalizing to obtain a normalized distance matrix;
the defect detection module is used for detecting whether defects exist, positions and sizes, and comprises: determining whether a defect exists by determining a relationship between a maximum value in the normalized distance matrix and a predetermined image-level defect threshold; the location and size of the presence of a defect is determined by determining the relationship between all values of the normalized distance matrix and a predetermined pixel level defect threshold.
9. The system of claim 8, wherein the feature extraction module further comprises a deep neural network model for self-supervised learning and a model training component.
10. The system of claim 8, wherein the distance matrix computation module further comprises a class of model building components.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110679316.8A CN113362313B (en) | 2021-06-18 | 2021-06-18 | Defect detection method and system based on self-supervised learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110679316.8A CN113362313B (en) | 2021-06-18 | 2021-06-18 | Defect detection method and system based on self-supervised learning |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113362313A true CN113362313A (en) | 2021-09-07 |
| CN113362313B CN113362313B (en) | 2024-03-15 |
Family
ID=77535080
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110679316.8A Active CN113362313B (en) | 2021-06-18 | 2021-06-18 | Defect detection method and system based on self-supervised learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113362313B (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114723751A (en) * | 2022-06-07 | 2022-07-08 | 中国空气动力研究与发展中心设备设计与测试技术研究所 | Unsupervised strip steel surface defect online detection method |
| CN115100390A (en) * | 2022-08-24 | 2022-09-23 | 华东交通大学 | Image emotion prediction method combining contrast learning and self-supervision region positioning |
| CN115690561A (en) * | 2022-04-01 | 2023-02-03 | 北京优创新港科技股份有限公司 | Road surface abnormity monitoring method |
| CN117274148A (en) * | 2022-12-05 | 2023-12-22 | 魅杰光电科技(上海)有限公司 | Unsupervised wafer defect detection method based on deep learning |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20130338965A1 (en) * | 2012-06-14 | 2013-12-19 | International Business Machines Corporation | Anomaly Detection Method, Program, and System |
| CN104809732A (en) * | 2015-05-07 | 2015-07-29 | 山东鲁能智能技术有限公司 | Electrical equipment appearance abnormity detection method based on image comparison |
| US20190188212A1 (en) * | 2016-07-27 | 2019-06-20 | Anomalee Inc. | Prioritized detection and classification of clusters of anomalous samples on high-dimensional continuous and mixed discrete/continuous feature spaces |
| CN111046855A (en) * | 2019-06-21 | 2020-04-21 | 腾讯科技（深圳）有限公司 | Picture identification method, identification model training method and device and electronic device |
| CN111080627A (en) * | 2019-12-20 | 2020-04-28 | 南京航空航天大学 | 2D +3D large airplane appearance defect detection and analysis method based on deep learning |
| CN112069992A (en) * | 2020-09-04 | 2020-12-11 | 西安西图之光智能科技有限公司 | Face detection method, system and storage medium based on multi-supervision dense alignment |
| CN112288741A (en) * | 2020-11-23 | 2021-01-29 | 四川长虹电器股份有限公司 | Product surface defect detection method and system based on semantic segmentation |
| US20210150757A1 (en) * | 2019-11-20 | 2021-05-20 | Nvidia Corporation | Training and inferencing using a neural network to predict orientations of objects in images |
| US20210150710A1 (en) * | 2019-11-15 | 2021-05-20 | Arizona Board Of Regents On Behalf Of Arizona State University | Systems, methods, and apparatuses for implementing a self-supervised chest x-ray image analysis machine-learning model utilizing transferable visual words |
| CN112862799A (en) * | 2021-02-24 | 2021-05-28 | 上海交通大学 | Image attribute recovery-based image anomaly detection method and system |
-
2021
- 2021-06-18 CN CN202110679316.8A patent/CN113362313B/en active Active
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20130338965A1 (en) * | 2012-06-14 | 2013-12-19 | International Business Machines Corporation | Anomaly Detection Method, Program, and System |
| CN104809732A (en) * | 2015-05-07 | 2015-07-29 | 山东鲁能智能技术有限公司 | Electrical equipment appearance abnormity detection method based on image comparison |
| US20190188212A1 (en) * | 2016-07-27 | 2019-06-20 | Anomalee Inc. | Prioritized detection and classification of clusters of anomalous samples on high-dimensional continuous and mixed discrete/continuous feature spaces |
| CN111046855A (en) * | 2019-06-21 | 2020-04-21 | 腾讯科技（深圳）有限公司 | Picture identification method, identification model training method and device and electronic device |
| US20210150710A1 (en) * | 2019-11-15 | 2021-05-20 | Arizona Board Of Regents On Behalf Of Arizona State University | Systems, methods, and apparatuses for implementing a self-supervised chest x-ray image analysis machine-learning model utilizing transferable visual words |
| US20210150757A1 (en) * | 2019-11-20 | 2021-05-20 | Nvidia Corporation | Training and inferencing using a neural network to predict orientations of objects in images |
| CN111080627A (en) * | 2019-12-20 | 2020-04-28 | 南京航空航天大学 | 2D +3D large airplane appearance defect detection and analysis method based on deep learning |
| CN112069992A (en) * | 2020-09-04 | 2020-12-11 | 西安西图之光智能科技有限公司 | Face detection method, system and storage medium based on multi-supervision dense alignment |
| CN112288741A (en) * | 2020-11-23 | 2021-01-29 | 四川长虹电器股份有限公司 | Product surface defect detection method and system based on semantic segmentation |
| CN112862799A (en) * | 2021-02-24 | 2021-05-28 | 上海交通大学 | Image attribute recovery-based image anomaly detection method and system |
Non-Patent Citations (8)
| Title |
|---|
| CARL DOERSCH等: "Unsupervised Visual Representation Learning by Context Prediction", 《2015 IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION》, pages 1422 - 1430 * |
| JIHUN YI ET AL: "Patch SVDD:Patch-Level SVDD for Anomaly Detection and Segmentation", 《COMPUTER VISION-ACCV 2020》, pages 3 * |
| JUNJIE XU等: "Anomaly Detection on Electroencephalography with Self-supervised Learning", 《2020 IEEE INTERNATIONAL CONFERENCE ON BIOINFORMATICS AND BIOMEDICINE》, pages 363 - 368 * |
| OLIVER RIPPEL ET AL: "Modeling the Distribution of Normal Data in Pre-Trained Deep Features for Anomaly Detection", 《2020 25TH INTERNATIONAL CONFERENCE ON PATTERN RECOGNITION》, pages 3 * |
| THOMAS DEFARD ET AL: "PaDiM：A Patch Distribution Modeling Framework for Anomaly Detection and Localization", 《PATTERN RECOGNITION.ICPR INTERNATIONAL WORKSHOPS AND CHALLENGES》, pages 3 * |
| XIAHAIFENG1995: "PaDiM-Anomaly-Detection-Localization-master", pages 170 - 173, Retrieved from the Internet <URL:《https://github.com/xiahaifeng1995/PaDiM-Anomaly-Detection-Localization-master/blob/main/main.py》> * |
| 刘金海等: "基于漏磁内检测的自监督缺陷检测方法", 《仪器仪表学报》, vol. 41, no. 9, pages 180 - 187 * |
| 陈涵深: "基于深度学习的路面破损检测研究", 《中国博士学位论文全文数据库（工程科技Ⅱ辑）》, no. 2, pages 034 - 2 * |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115690561A (en) * | 2022-04-01 | 2023-02-03 | 北京优创新港科技股份有限公司 | Road surface abnormity monitoring method |
| CN114723751A (en) * | 2022-06-07 | 2022-07-08 | 中国空气动力研究与发展中心设备设计与测试技术研究所 | Unsupervised strip steel surface defect online detection method |
| CN114723751B (en) * | 2022-06-07 | 2022-09-23 | 中国空气动力研究与发展中心设备设计与测试技术研究所 | Unsupervised strip steel surface defect online detection method |
| CN115100390A (en) * | 2022-08-24 | 2022-09-23 | 华东交通大学 | Image emotion prediction method combining contrast learning and self-supervision region positioning |
| CN117274148A (en) * | 2022-12-05 | 2023-12-22 | 魅杰光电科技(上海)有限公司 | Unsupervised wafer defect detection method based on deep learning |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113362313B (en) | 2024-03-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113362313A (en) | Defect detection method and system based on self-supervision learning | |
| CN111079747B (en) | Railway wagon bogie side frame fracture fault image identification method | |
| CN108961217B (en) | Surface defect detection method based on regular training | |
| CN108305243B (en) | Magnetic shoe surface defect detection method based on deep learning | |
| Martins et al. | Automatic detection of surface defects on rolled steel using computer vision and artificial neural networks | |
| CN111815601A (en) | Texture image surface defect detection method based on depth convolution self-encoder | |
| Mathavan et al. | Use of a self-organizing map for crack detection in highly textured pavement images | |
| CN101604384B (en) | Individualized fingerprint identification method | |
| CN111091544B (en) | Method for detecting breakage fault of side integrated framework of railway wagon bogie | |
| CN112801962B (en) | Semi-supervised industrial product flaw detection method and system based on positive sample learning | |
| CN112580693A (en) | Petrochemical process fault diagnosis method based on self-help resampling neighborhood preserving embedding | |
| Lee et al. | Review on the recent welding research with application of CNN-based deep learning part II: Model evaluation and visualizations | |
| CN114332008A (en) | Unsupervised defect detection and positioning method based on multi-level feature reconstruction | |
| CN115995056A (en) | Automatic bridge disease identification method based on deep learning | |
| CN113421223B (en) | Industrial product surface defect detection method based on deep learning and Gaussian mixture | |
| CN117422935B (en) | Motorcycle fault non-contact diagnosis method and system | |
| CN115375635A (en) | Wafer defect identification method | |
| KR20210122429A (en) | Method and System for Artificial Intelligence based Quality Inspection in Manufacturing Process using Machine Vision Deep Learning | |
| Aydin et al. | Defect classification of railway fasteners using image preprocessing and alightweight convolutional neural network | |
| CN113838040A (en) | Detection method for defect area of color texture fabric | |
| Hridoy et al. | A framework for industrial inspection system using deep learning | |
| CN116309493A (en) | Method and system for detecting defects of textile products | |
| Mansano et al. | Inspection of metallic surfaces using local binary patterns | |
| CN114897863A (en) | Defect detection method, device and equipment | |
| CN115082416A (en) | Lens flaw detection method, device, equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |