CN113935947A - Industrial visual defect detection method based on multi-point position double-branch model - Google Patents

Industrial visual defect detection method based on multi-point position double-branch model Download PDF

Info

Publication number
CN113935947A
CN113935947A CN202111055665.9A CN202111055665A CN113935947A CN 113935947 A CN113935947 A CN 113935947A CN 202111055665 A CN202111055665 A CN 202111055665A CN 113935947 A CN113935947 A CN 113935947A
Authority
CN
China
Prior art keywords
sample
branch
point
model
positive sample
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Withdrawn
Application number
CN202111055665.9A
Other languages
Chinese (zh)
Inventor
宋艳枝
汪方军
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Hefei Gauss Intelligent Technology Co ltd
Original Assignee
Hefei Gauss Intelligent Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Hefei Gauss Intelligent Technology Co ltd filed Critical Hefei Gauss Intelligent Technology Co ltd
Priority to CN202111055665.9A priority Critical patent/CN113935947A/en
Publication of CN113935947A publication Critical patent/CN113935947A/en
Withdrawn legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0004—Industrial image inspection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/243—Classification techniques relating to the number of classes
    • G06F18/2431—Multiple classes
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/25—Fusion techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/047—Probabilistic or stochastic networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • General Physics & Mathematics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • General Engineering & Computer Science (AREA)
  • Evolutionary Computation (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Molecular Biology (AREA)
  • Software Systems (AREA)
  • Mathematical Physics (AREA)
  • Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • General Health & Medical Sciences (AREA)
  • Computing Systems (AREA)
  • Evolutionary Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Probability & Statistics with Applications (AREA)
  • Quality & Reliability (AREA)
  • Image Analysis (AREA)
  • Image Processing (AREA)

Abstract

The invention discloses an industrial visual defect detection method based on a multi-point double-branch model, and relates to the technical field of artificial intelligence. The invention comprises the following steps: dividing data according to point positions: dividing data according to point location information to obtain a plurality of positive sample pools and negative sample pools, and aligning samples; multipoint sampling strategy: respectively selecting a positive sample pair and a negative sample pair from a positive sample pool and a negative sample pool according to a multi-point sampling strategy; feature extraction: taking a ResNet-50 model as a feature extractor to extract the features of input data; two-branch prediction: and inputting the extracted features into a plurality of fully-connected layers containing attention mechanisms to perform defect prediction. According to the invention, by utilizing the point location information, the data sets can be effectively divided, and the complex and various data sets are converted into subdata sets in a plurality of point locations, so that the difficulty of network learning is reduced, and the defect identification capability of the model is improved.

Description

Industrial visual defect detection method based on multi-point position double-branch model
Technical Field
The invention belongs to the technical field of artificial intelligence, and particularly relates to an industrial visual defect detection method based on a multi-point double-branch model.
Background
Quality control is a key link of an industrial production line, and industrial visual defect classification is an indispensable means for ensuring the quality of a sample. The industrial defect classification task comprises a plurality of application scenes such as SMT component defect classification, industrial product surface defect classification, cloth defect classification and the like. Therefore, the method has important practical significance for classifying the defects of the industrial products on the production line.
In recent years, deep learning techniques have been widely used in the field of industrial visual defect classification, and a large amount of research work has also appeared. In an overview of these studies, they mostly neglected the fact that: the quality of the product is extremely high in the industry, especially in the component manufacturing and assembling industry. This requires a high accuracy of the model for positive samples and a high recall for negative samples. The missing detection cost of the defects in industrial production is much higher than the misjudgment cost of normal samples. The existing methods do not take into account the actual needs of the industry. And by utilizing the point location information, the data sets can be effectively divided, and the complex and various data sets are converted into subdata sets in a plurality of point locations, so that the difficulty of network learning is reduced, and the defect identification capability of the model is improved.
Disclosure of Invention
The invention aims to provide an industrial visual defect detection method based on a multi-point position double-branch model, which divides a data set through point position information acquired from a production line, provides a multi-point position sampling strategy to extract training data, utilizes a ResNet-50 model to extract input data characteristics, inputs the input data characteristics into a double-branch structure to perform double-branch prediction, combines prediction results and solves the problems of high message difficulty and low defect identification capability of the conventional network model.
In order to solve the technical problems, the invention is realized by the following technical scheme:
the invention relates to an industrial visual defect detection method based on a multi-point double-branch model, which comprises the following steps:
step S01, dividing the data according to the point location: dividing data according to point location information to obtain a plurality of positive sample pools and negative sample pools, and aligning samples;
step S02, multi-point sampling strategy: respectively selecting a positive sample pair and a negative sample pair from a positive sample pool and a negative sample pool according to a multi-point sampling strategy;
step S03, feature extraction: taking a ResNet-50 model as a feature extractor to extract the features of input data;
step S04, two-branch prediction: and inputting the extracted features into a plurality of fully-connected layers containing attention mechanisms to perform defect prediction.
As a preferred technical solution, in step S01, dividing the original data D into I parts according to point location information, where each part is provided with a positive sample pool and a negative sample pool; the positive sample pool contains a positive sample of the point; the negative sample pool contains the negative samples of the point, and each sample is aligned and used for ensuring that the height of the sample image is larger than or equal to the width of the sample image when the sample rotates.
As a preferred technical solution, in step S02, the multi-point bit sampling strategy includes the following steps:
step P01: device for placing
Figure BDA0003254533590000021
Step P02: by probability
Figure BDA0003254533590000022
Selecting a data subset index i;
step P03: if D isiIf there is no positive sample, go to step P02;
step P04: random slave DiIn which a positive sample s is selected1；
Step P05: generating a random number between 0 and 1
Figure BDA0003254533590000031
Step P06: if it is not
Figure BDA0003254533590000032
Greater than a specified threshold p and DiWith negative samples, then from DiRandomly choosing a negative sample s2(ii) a Otherwise, from DiRandomly choosing a positive sample s2；
Step P07: sample pairs(s) to be generated1,s2) Placing D';
step P08: a data set for training is obtained.
As a preferable technical solution, in the step S03, the feature extraction model FwExtracting embedded features of the input sample using a convolutional neural network; the feature extraction model generates two embedded features respectively
Figure BDA0003254533590000033
And
Figure BDA0003254533590000034
in the formula, H ', W' and M are height, width and channel number of the embedding feature respectively, i represents the ith point position, and j and a are subscripts of the sample to be detected and the standard sample respectively.
As a preferred technical solution, in step S04, the model prediction includes a similarity branch BoS and a multi-classification branch BoM; the similarity branch BoS is used for predicting the similarity between the sample to be tested and the standard sample; the multi-class branch BoM is used to predict the specific defect class of the sample to be tested, and only the embedded features from the sample to be tested are received by the multi-class branch BoM.
As a preferred technical solution, the similarity branch BoS and the multi-classification branch BoM use an attention mechanism to perform weighted fusion on the embedded features, the result obtained by the fusion is input to a plurality of full-connected layers, and two outputs are obtained, which are respectively the similarity branch outputs
Figure BDA0003254533590000035
Multiple classification branch output
Figure BDA0003254533590000036
Wherein K is the number of multi-classification categories, and i is the subscript of the ith point.
As a preferred technical solution, the similarity branch output
Figure BDA0003254533590000037
And multi-classification branch output
Figure BDA0003254533590000038
And respectively calculating the mapping of each value to the conditional probability through the softmax layer, and calculating to obtain the following results:
Figure BDA0003254533590000039
Figure BDA0003254533590000041
only when PiAnd QiMeanwhile, when the input image is predicted to be OK, the classification result is OK.
The invention has the following beneficial effects:
according to the invention, by utilizing the point location information, the data sets can be effectively divided, and the complex and various data sets are converted into subdata sets in a plurality of point locations, so that the difficulty of network learning is reduced, and the defect identification capability of the model is improved.
Of course, it is not necessary for any product in which the invention is practiced to achieve all of the above-described advantages at the same time.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art that other drawings can be obtained according to the drawings without creative efforts.
FIG. 1 is a schematic diagram of an industrial visual defect detection method based on a multi-point dual-branch model according to the present invention;
FIG. 2 is a diagram of the multi-point sampling strategy steps of the present invention;
FIG. 3 is a schematic diagram of feature extraction and dual-branch structure prediction according to the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Referring to fig. 1, the present invention is a method for detecting industrial vision defects based on a multi-point dual-branch model, comprising the following steps:
step S01, dividing the data according to the point location: dividing data according to point location information to obtain a plurality of positive sample pools and negative sample pools, and aligning samples;
in step S01, dividing the original data D into I parts according to point location information, each part having a positive sample cell and a negative sample cell; the positive sample pool contains a positive sample of the point; the negative sample pool contains the negative samples of the point, and each sample is aligned and used for ensuring that the height of the sample image is larger than or equal to the width of the sample image when the sample rotates.
Step S02, multi-point sampling strategy: respectively selecting a positive sample pair and a negative sample pair from a positive sample pool and a negative sample pool according to a multi-point sampling strategy;
referring to fig. 2, in step S02, the multi-point sampling strategy includes the following steps:
step P01: device for placing
Figure BDA0003254533590000051
Step P02: by probability
Figure BDA0003254533590000052
Selecting a data subset index i;
step P03: if D isiIf there is no positive sample, go to step P02;
step P04: random slave DiIn which a positive sample s is selected1；
Step P05: generating a random number between 0 and 1
Figure BDA0003254533590000053
Step P06: if it is not
Figure BDA0003254533590000054
Greater than a specified threshold p and DiWith negative samples, then from DiRandomly choosing a negative sample s2(ii) a Otherwise, from DiRandomly choosing a positive sample s2；
Step P07: sample pairs(s) to be generated1,s2) Placing D';
step P08: a data set for training is obtained.
Step S03, feature extraction: taking a ResNet-50 model as a feature extractor to extract the features of input data;
referring to fig. 3, in step S03, the feature extraction model FwExtracting embedded features of the input sample using a convolutional neural network; the feature extraction model generates two embedded features respectively
Figure BDA0003254533590000061
Figure BDA0003254533590000062
And
Figure BDA0003254533590000063
in the formula, H ', W' and M are height, width and channel number of the embedding feature respectively, i represents the ith point position, and j and a are subscripts of the sample to be detected and the standard sample respectively.
Step S04, two-branch prediction: and inputting the extracted features into a plurality of fully-connected layers containing attention mechanisms to perform defect prediction.
In step S04, the model prediction includes a similarity branch BoS and a multi-class branch BoM; the similarity branch BoS is used for predicting the similarity between the sample to be detected and the standard sample; the multi-class branch BoM is used to predict the specific defect class of the sample to be tested, and only receives the embedded features from the sample to be tested.
The similarity branch BoS and the multi-classification branch BoM use an attention mechanism to carry out weighted fusion on the embedded features, the fusion result is input into a plurality of full-connection layers, and two outputs are obtained, namely the similarity branch output
Figure BDA0003254533590000064
Multiple classification branch output
Figure BDA0003254533590000065
Wherein K is the number of multi-classification categories, and i is the subscript of the ith point.
Similarity branch output
Figure BDA0003254533590000066
And multi-classification branch output
Figure BDA0003254533590000067
And respectively calculating the mapping of each value to the conditional probability through the softmax layer, and calculating to obtain the following results:
Figure BDA0003254533590000068
Figure BDA0003254533590000069
only when PiAnd QiMeanwhile, when the input image is predicted to be OK, the classification result is OK.
It should be noted that, in the above system embodiment, each included unit is only divided according to functional logic, but is not limited to the above division as long as the corresponding function can be implemented; in addition, specific names of the functional units are only for convenience of distinguishing from each other, and are not used for limiting the protection scope of the present invention.
In addition, it is understood by those skilled in the art that all or part of the steps in the method for implementing the embodiments described above may be implemented by a program instructing associated hardware, and the corresponding program may be stored in a computer-readable storage medium.
The preferred embodiments of the invention disclosed above are intended to be illustrative only. The preferred embodiments are not intended to be exhaustive or to limit the invention to the precise embodiments disclosed. Obviously, many modifications and variations are possible in light of the above teaching. The embodiments were chosen and described in order to best explain the principles of the invention and the practical application, to thereby enable others skilled in the art to best utilize the invention. The invention is limited only by the claims and their full scope and equivalents.

Claims (7)

1. An industrial visual defect detection method based on a multi-point position double-branch model is characterized by comprising the following steps:
step S01, dividing the data according to the point location: dividing data according to point location information to obtain a plurality of positive sample pools and negative sample pools, and aligning samples;
step S02, multi-point sampling strategy: respectively selecting a positive sample pair and a negative sample pair from a positive sample pool and a negative sample pool according to a multi-point sampling strategy;
step S03, feature extraction: taking a ResNet-50 model as a feature extractor to extract the features of input data;
step S04, two-branch prediction: and inputting the extracted features into a plurality of fully-connected layers containing attention mechanisms to perform defect prediction.
2. The industrial visual defect detection method based on the multi-point-location double-branch model as claimed in claim 1, wherein in step S01, the original data D is divided into I parts according to point location information, each part is provided with a positive sample pool and a negative sample pool; the positive sample pool contains a positive sample of the point; the negative sample pool contains the negative samples of the point, and each sample is aligned and used for ensuring that the height of the sample image is larger than or equal to the width of the sample image when the sample rotates.
3. The industrial visual defect detection method based on the multi-point bit double-branch model as claimed in claim 1, wherein in the step S02, the multi-point bit sampling strategy comprises the following steps:
step P01: device for placing
Figure FDA0003254533580000011
Step P02: by probability
Figure FDA0003254533580000012
Selecting a data subset index i;
step P03: if D isiIf there is no positive sample, go to step P02;
step P04: random slave DiIn which a positive sample s is selected1；
Step P05: generating a random number between 0 and 1
Figure FDA0003254533580000013
Step P06: if it is not
Figure FDA0003254533580000021
Greater than a specified threshold p and DiWith negative samples, then from DiRandomly choosing a negative sample s2(ii) a Otherwise, from DiRandomly choosing a positive sample s2；
Step P07: sample pairs(s) to be generated1,s2) Placing D';
step P08: a data set for training is obtained.
4. The method for detecting industrial vision defects based on the multi-point potential double-branch model as claimed in claim 1, wherein in the step S03, the feature extraction model F is usedwExtracting embedded features of the input sample using a convolutional neural network; the feature extraction model generates two embedded features respectively
Figure FDA0003254533580000022
And
Figure FDA0003254533580000023
in the formula, H ', W' and M are height, width and channel number of the embedding feature respectively, i represents the ith point position, and j and a are subscripts of the sample to be detected and the standard sample respectively.
5. The method for detecting industrial vision defects based on the multi-point potential double-branch model as claimed in claim 1, wherein in the step S04, the model prediction includes a similarity branch BoS and a multi-classification branch BoM; the similarity branch BoS is used for predicting the similarity between the sample to be tested and the standard sample; the multi-class branch BoM is used to predict the specific defect class of the sample to be tested, and only the embedded features from the sample to be tested are received by the multi-class branch BoM.
6. The industrial visual defect detection method based on the multi-point-position double-branch model as claimed in claim 1, wherein the similarity branch BoS and the multi-classification branch BoM use an attention mechanism to perform weighted fusion on the embedded features, the fusion result is input into a plurality of full-connected layers, and two outputs are obtained, wherein the two outputs are respectively the similarity branch outputGo out
Figure FDA0003254533580000024
Multiple classification branch output
Figure FDA0003254533580000025
Wherein K is the number of multi-classification categories, and i is the subscript of the ith point.
7. The method as claimed in claim 6, wherein the similarity branch is output
Figure FDA0003254533580000026
And multi-classification branch output
Figure FDA0003254533580000027
And respectively calculating the mapping of each value to the conditional probability through the softmax layer, and calculating to obtain the following results:
Figure FDA0003254533580000031
Figure FDA0003254533580000032
only when PiAnd QiMeanwhile, when the input image is predicted to be OK, the classification result is OK.
CN202111055665.9A 2021-09-09 2021-09-09 Industrial visual defect detection method based on multi-point position double-branch model Withdrawn CN113935947A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202111055665.9A CN113935947A (en) 2021-09-09 2021-09-09 Industrial visual defect detection method based on multi-point position double-branch model

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202111055665.9A CN113935947A (en) 2021-09-09 2021-09-09 Industrial visual defect detection method based on multi-point position double-branch model

Publications (1)

Publication Number Publication Date
CN113935947A true CN113935947A (en) 2022-01-14

Family

ID=79275271

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202111055665.9A Withdrawn CN113935947A (en) 2021-09-09 2021-09-09 Industrial visual defect detection method based on multi-point position double-branch model

Country Status (1)

Country Link
CN (1) CN113935947A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114648077A (en) * 2022-05-18 2022-06-21 合肥高斯智能科技有限公司 Method and device for multi-point industrial data defect detection

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114648077A (en) * 2022-05-18 2022-06-21 合肥高斯智能科技有限公司 Method and device for multi-point industrial data defect detection

Similar Documents

Publication Publication Date Title
CN109949317B (en) Semi-supervised image example segmentation method based on gradual confrontation learning
CN104573669B (en) Image object detection method
CN108229588B (en) Machine learning identification method based on deep learning
CN107330396A (en) A kind of pedestrian's recognition methods again based on many attributes and many strategy fusion study
CN105574550A (en) Vehicle identification method and device
CN110298343A (en) A kind of hand-written blackboard writing on the blackboard recognition methods
CN106446896A (en) Character segmentation method and device and electronic equipment
CN101221623B (en) Object type on-line training and recognizing method and system thereof
CN104915643A (en) Deep-learning-based pedestrian re-identification method
CN112633149B (en) Domain-adaptive foggy-day image target detection method and device
CN105590099A (en) Multi-user behavior identification method based on improved convolutional neural network
CN111860193B (en) Text-based pedestrian retrieval self-supervision visual representation learning system and method
CN115631365A (en) Cross-modal contrast zero sample learning method fusing knowledge graph
CN110599459A (en) Underground pipe network risk assessment cloud system based on deep learning
CN105654054A (en) Semi-supervised neighbor propagation learning and multi-visual dictionary model-based intelligent video analysis method
CN109446897B (en) Scene recognition method and device based on image context information
CN115830531A (en) Pedestrian re-identification method based on residual multi-channel attention multi-feature fusion
CN113128410A (en) Weak supervision pedestrian re-identification method based on track association learning
CN113935947A (en) Industrial visual defect detection method based on multi-point position double-branch model
CN108229692B (en) Machine learning identification method based on dual contrast learning
CN114898290A (en) Real-time detection method and system for marine ship
CN115188031A (en) Fingerprint identification method, computer program product, storage medium and electronic device
CN114550153A (en) Terminal block image detection and identification method
CN111242114B (en) Character recognition method and device
CN114445693A (en) Knowledge distillation-based sustainable learning water obstacle detection system and method

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
WW01 Invention patent application withdrawn after publication

Application publication date: 20220114

WW01 Invention patent application withdrawn after publication