CN111915593A - Model establishing method and device, electronic equipment and storage medium - Google Patents
Model establishing method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN111915593A CN111915593A CN202010775330.3A CN202010775330A CN111915593A CN 111915593 A CN111915593 A CN 111915593A CN 202010775330 A CN202010775330 A CN 202010775330A CN 111915593 A CN111915593 A CN 111915593A
- Authority
- CN
- China
- Prior art keywords
- flaw
- image
- defect
- detection
- network
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 47
- 238000003860 storage Methods 0.000 title claims abstract description 15
- 238000001514 detection method Methods 0.000 claims abstract description 108
- 230000007547 defect Effects 0.000 claims abstract description 106
- 238000012549 training Methods 0.000 claims abstract description 31
- 238000010586 diagram Methods 0.000 claims description 59
- 238000000605 extraction Methods 0.000 claims description 13
- 230000006870 function Effects 0.000 claims description 11
- 238000011176 pooling Methods 0.000 claims description 8
- 238000004590 computer program Methods 0.000 claims description 7
- 238000012545 processing Methods 0.000 claims description 7
- 238000013459 approach Methods 0.000 claims description 4
- 238000002372 labelling Methods 0.000 claims description 3
- 238000013527 convolutional neural network Methods 0.000 description 13
- 239000004744 fabric Substances 0.000 description 9
- 238000013528 artificial neural network Methods 0.000 description 4
- 238000012360 testing method Methods 0.000 description 4
- 238000013135 deep learning Methods 0.000 description 3
- 230000035945 sensitivity Effects 0.000 description 3
- 230000003247 decreasing effect Effects 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000004927 fusion Effects 0.000 description 2
- 238000004519 manufacturing process Methods 0.000 description 2
- 238000012216 screening Methods 0.000 description 2
- 206010063385 Intellectualisation Diseases 0.000 description 1
- 235000014676 Phragmites communis Nutrition 0.000 description 1
- 208000003464 asthenopia Diseases 0.000 description 1
- 238000013145 classification model Methods 0.000 description 1
- 230000002950 deficient Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000000227 grinding Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000001737 promoting effect Effects 0.000 description 1
- 238000005096 rolling process Methods 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 230000001629 suppression Effects 0.000 description 1
- 238000011179 visual inspection Methods 0.000 description 1
- 230000037303 wrinkles Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0004—Industrial image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10004—Still image; Photographic image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30108—Industrial image inspection
- G06T2207/30124—Fabrics; Textile; Paper
Landscapes
- Engineering & Computer Science (AREA)
- Quality & Reliability (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Image Analysis (AREA)
- Investigating Materials By The Use Of Optical Means Adapted For Particular Applications (AREA)
Abstract
A model establishing method, a device, electronic equipment and a storage medium for flaw detection are applied to the technical field of detection and comprise the following steps: establishing an image set, wherein the image set comprises defects of different types; constructing a flaw detection network model based on the image set and the Faster R-CNN; training the flaw detection network model to obtain a classification detection model; the classification detection model is used for detecting flaws of an image to be detected and outputting types, shapes and positions of the flaws in the image to be detected. The method can be used for detecting the flaw of the extreme size.
Description
Technical Field
The present disclosure relates to the field of image detection technologies, and in particular, to a method and an apparatus for model building for defect detection, an electronic device, and a storage medium.
Background
In recent years, with the development of science and technology, the automation of product quality detection becomes one of the main trends of modern production development, and the automatic flaw detection has important significance for reducing labor force, improving production efficiency and promoting the intellectualization of the industry.
The traditional manual visual inspection method has the problems of low detection efficiency, low detection speed, low detection precision, inconsistent detection standards and the like. Because the detection working time is long, the inspectors are easy to generate visual fatigue, the false detection rate and the omission factor are high, and the damage to human eyes is also great. Therefore, flaw detection methods based on machine vision and deep learning have been developed.
The defect detection deep learning algorithm is widely researched and mainly applied to surface defect detection of fabrics, blankets, floor tiles and the like. In the prior art, some image classification models based on deep learning are established by using a basic network, a regional proposal network and a Fast R-CNN detection network, and feature extraction is performed on input data in each iteration in a training process without manually designing a fussy image feature extractor, but preliminary image data screening is only completed aiming at defects with conventional shapes, and feature enhancement of defects such as fabrics is not performed. Some methods search and locate the flaw area by a fast circular convolution neural network method, combine features and detectors into a frame, and automatically detect the cloth flaws. The RPN network is used for generating suggestion windows, 300 suggestion windows are generated in each picture, and the number of the suggestion windows is very large, so that the detection speed is very slow.
Disclosure of Invention
The present disclosure is directed to a method and an apparatus for model building for defect detection, an electronic device and a storage medium, which have high detection accuracy and can be used for defect detection of extreme dimensions.
In order to achieve the above object, a first aspect of the embodiments of the present disclosure provides a method for modeling fault detection, including:
establishing an image set, wherein the image set comprises defects of different types;
constructing a flaw detection network model based on the image set and the Faster R-CNN;
training the flaw detection network model to obtain a classification detection model;
the classification detection model is used for detecting flaws of an image to be detected and outputting types, shapes and positions of the flaws in the image to be detected.
Optionally, the creating an image set includes:
collecting at least one image, and labeling flaws in the image by adopting a label to obtain an image set;
wherein the label is marked with at least one of type, shape and position information of the flaw.
Optionally, the constructing a flaw detection network model based on the image set and the Faster R-CNN includes:
inputting the image set into a feature pyramid network to obtain a first flaw feature map, wherein the feature pyramid network adopts a ResNet-50 feature extraction network;
and inputting the first flaw characteristic diagram into a prior anchor generation network, and constructing a flaw detection network model based on Faster R-CNN.
Optionally, the inputting the image set into the feature pyramid network to obtain a first defect feature map includes:
inputting the set of images into the feature pyramid network;
convolving the marked flaws in the image set from bottom to top to obtain a primary flaw characteristic diagram with sequentially reduced sizes;
performing 1 × 1 convolution dimensionality reduction on all the initial flaw feature maps to obtain intermediate flaw feature maps;
and all the intermediate defect characteristic diagrams are up-sampled from top to bottom and are fused with the adjacent intermediate defect characteristic diagram of the next size, so that a first defect characteristic diagram with the size being sequentially reduced is obtained.
Optionally, the inputting the first flaw feature map into a prior anchor generation network, and the constructing a flaw detection network model based on fast R-CNN includes:
generating candidate sub-regions in the first defect feature map with sequentially reduced size;
obtaining a true value frame according to the position of the flaw marked by the label in each image;
comparing the candidate subarea of each image with the truth value frame to screen out candidate anchors;
adjusting the shape of the candidate anchor through regression to enable the candidate anchor of each image to approach to a true value frame, and obtaining a prior anchor corresponding to the first defect characteristic diagram with the size reduced in sequence;
and inputting the first flaw characteristic diagram with the sequentially reduced size and the prior anchor corresponding to the first flaw characteristic diagram into a preset classification network, and determining parameters of the flaw detection network model.
Optionally, the classification network includes 1 pooling layer and 4 full-link layers, the size of the first defect feature map sequentially reduced and the prior anchor corresponding to the first defect feature map are input into a preset classification network, and determining the parameters of the defect detection network model includes:
inputting the first defect characteristic diagram with the sequentially reduced size and the prior anchor corresponding to the first defect characteristic diagram into the pooling layer to obtain characteristic diagrams with the same size and the prior anchor corresponding to the characteristic diagrams;
taking the feature maps with the same size and the prior anchors corresponding to the feature maps as the input of a first full-connection layer to obtain the output of the first full-connection layer;
taking the output of the first full connection layer as the input of a second full connection layer to obtain the output of the second full connection layer;
taking the output of the second fully-connected layer as the input of a third fully-connected layer, and enabling the third fully-connected layer to output the position and shape information of the flaw in each image through regression;
and taking the output of the second fully-connected layer as the input of a fourth fully-connected layer, and enabling the fourth fully-connected layer to output the type information of the flaws in each image through Softmax classification.
Optionally, in the process of training the fault detection network model, a cross entropy loss function and a shape prediction loss function of the prior anchor are used to constrain the training process of the fault detection network model.
A second aspect of the embodiments of the present disclosure provides a model building apparatus for flaw detection, including:
the image processing system comprises an establishing module, a processing module and a processing module, wherein the establishing module is used for establishing an image set, and the image set comprises defects of different types;
the building module is used for building a flaw detection network model based on the image set and the Faster R-CNN;
the training module is used for training the flaw detection network model to obtain a classification detection model;
the classification detection model is used for detecting flaws of an image to be detected and outputting types, shapes and positions of the flaws in the image to be detected.
A third aspect of the embodiments of the present disclosure provides an electronic device, including:
the method for fault detection is characterized in that the processor executes the program to implement the method for model establishment for fault detection provided by the first aspect of the embodiment of the present disclosure.
A fourth aspect of the embodiments of the present disclosure provides a computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, implements the model building method for flaw detection provided in the first aspect of the embodiments of the present disclosure.
It can be known from the foregoing embodiments of the present disclosure that, according to the model establishing method, apparatus, electronic device and storage medium for flaw detection provided by the present disclosure, the feature of the current layer is fused with the feature of the next layer adjacent to the current layer by using the multi-size feature fusion algorithm of the feature pyramid network, so that the problem of flaw feature information loss caused by only selecting the feature map of the uppermost layer when extracting features in a multilayer convolution in a conventional neural network is solved, the detailed features of small flaws are enhanced, the flaw feature extraction is more accurate, and the sensitivity and detection capability for extreme shape flaws such as small flaws and long and thin flaws are improved. The method has the advantages that a small number of candidate anchors are obtained through selection of candidate sub-regions on the first flaw characteristic diagram with various sizes, the sizes of the candidate anchors are adjusted through regression, and the prior anchors closest to the flaw characteristic sizes are finally obtained.
Drawings
In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present disclosure, and other drawings can be obtained by those skilled in the art without creative efforts.
FIG. 1 is a schematic flow chart diagram illustrating a method for modeling fault detection according to an embodiment of the present disclosure;
fig. 2 is a schematic diagram of a feature pyramid network according to an embodiment of the disclosure;
fig. 3 is a schematic diagram of an a priori anchor generation network provided by an embodiment of the present disclosure;
FIG. 4 is a flowchart illustrating a training process of a classification detection model according to an embodiment of the present disclosure;
FIG. 5 is a schematic structural diagram of a model building apparatus for defect detection according to an embodiment of the present disclosure;
fig. 6 shows a hardware structure diagram of an electronic device.
Detailed Description
In order to make the objects, features and advantages of the present disclosure more apparent and understandable, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present disclosure, and it is apparent that the described embodiments are only a part of the embodiments of the present disclosure, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments disclosed herein without making any creative effort, shall fall within the protection scope of the present disclosure.
The classification detection model comprises a feature extraction network, an interested region generation network, an R-CNN classification and positioning network, wherein the feature extraction network is a feature pyramid network, so that the loss of fine feature information is avoided, and the defect feature is supplemented and enhanced, so that the defect feature is extracted more accurately.
Referring to fig. 1, fig. 1 is a schematic flow chart of a model building method for defect detection according to an embodiment of the present disclosure, the method mainly includes the following steps:
s101, establishing an image set, wherein the image set comprises defects of different types;
specifically, the number of images for each defect type should be multiple, e.g., 100, 200, etc., and the image size is not required. Because the fabric flaw covers more extreme shapes, the fabric flaw is taken as an example in the disclosure. The types of the defects of the fabric are defect-free, hole-broken, stain, laddering, knot, pattern jump, rib, thick warp, thick weft, broken warp, broken weft, thin and dense grade, grinding mark, rolling mark, dead wrinkle, double weft, double warp, reed way, poor weft and the like.
S102, constructing a flaw detection network model based on the image set and the Faster R-CNN;
s103, training the flaw detection network model to obtain a classification detection model;
the classification detection model is used for detecting flaws of an image to be detected and outputting types, shapes and positions of the flaws in the image to be detected.
In one embodiment of the present disclosure, step S101 includes:
collecting at least one image, and labeling flaws in the image by adopting a label to obtain an image set;
wherein the label is marked with at least one of type, shape and position information of the flaw.
Optionally, a label library may be established based on labels, a label file in the label library is established according to a COCO format, and by taking a fabric defect as an example, all collected fabric image data with a defect form an image set.
In the embodiment, the classification detection model is obtained by constructing the flaw detection network model based on the image set and the Faster R-CNN and then training the flaw detection network model. The method can effectively improve the positioning precision and the detection speed, and still has remarkable effect on the flaws of the shapes of the extreme ends.
In one embodiment of the present disclosure, step S102 includes:
s1021, inputting the image set into a feature pyramid network to obtain a first defect feature map, wherein the feature pyramid network adopts a ResNet-50 feature extraction network;
s1022, inputting the first flaw characteristic diagram into a prior anchor generation network, and constructing a flaw detection network model based on Faster R-CNN.
Specifically, the ResNet-50 feature extraction network comprises 49 convolution layers and a full-connection layer, defect feature extraction with sequentially reduced size is performed by performing convolution on the collected fabric images from bottom to top, except that the first layer is 7 × 7 convolution, the rest are 1 × 1 convolution and 3 × 3 convolution to obtain a series of feature maps with different sizes (with sequentially reduced size), and exemplarily, as shown in fig. 2, the 10 th layer, the 40 th layer and the 49 th layer of preliminary defect feature maps C1, C2 and C3 can be selected from the feature maps and are subjected to 1 × 1 convolution de-scaling and/or up-sampling. It should be noted that, the number and the number of layers of the selected preliminary defect feature map are not limited in the present disclosure, and the above is only an illustrative description.
In the embodiment, the feature extraction network is selected as the feature pyramid network, so that not only is the loss of fine feature information avoided, but also the defect feature is supplemented and enhanced, so that the defect feature is extracted more accurately, the prior anchor generation network is adopted for the region-of-interest generation network, the more accurate position shape of the prior anchor can be provided for the training of the defect detection network, meanwhile, the number of candidate anchors is greatly reduced, the positioning precision and the detection speed are effectively improved, and the defect of the extreme shape is still obviously affected.
In one embodiment of the present disclosure, referring to fig. 2, step S1021 includes:
inputting the image set into the feature pyramid network;
convolving the marked flaws in the image set from bottom to top to obtain a primary flaw characteristic diagram with sequentially reduced sizes;
performing 1 × 1 convolution dimensionality reduction on all the initial flaw feature maps to obtain intermediate flaw feature maps;
and all the intermediate defect characteristic diagrams are up-sampled from top to bottom and are fused with the adjacent intermediate defect characteristic diagram of the next size, so that a first defect characteristic diagram with the size being sequentially reduced is obtained.
It will be appreciated that, referring to fig. 2, C1, C2, and C3 in fig. 2 are preliminary defect feature maps of successively decreasing size, and F1, F2, and F3 are first defect feature maps of successively decreasing size, wherein the corresponding preliminary defect feature maps, intermediate defect feature maps, and first defect feature maps are the same size (intermediate defect feature maps are not shown in fig. 2), e.g., C1 is the same size as F1, C2 is the same size as F2, and C3 is the same size as F3.
Understandably, the first defect feature map comprises all defect feature information in the preliminary defect feature map and the preliminary defect feature map of the next size adjacent to the preliminary defect feature map. Specifically, referring to fig. 2, the next size of the preliminary defect map adjacent to C1 is C2, and the next size of the preliminary defect map adjacent to C2 is C3. F1 includes fault signature information for C1 and C2, and F2 includes signature information for C2 and C3, wherein if C3 is the last preliminary fault signature, F3 includes only all fault signature information for C3 since C3 of the last size does not have a preliminary fault signature of the next size adjacent, and thus only C3 is subjected to 1 × 1 convolution dimensionality reduction, without subsequent top-down upsampling. If C3 has a preliminary defect signature of the next size adjacent, then normal 1 x 1 convolution dimensionality reduction, and top-down upsampling, is performed. It should be noted that fig. 2 is only a schematic illustration, and the number of the preliminary defect feature maps with successively reduced sizes obtained by performing convolution is not limited by the present disclosure, and may be 1, 2, 3, 4, and so on.
In this embodiment, the feature of the current layer is fused with the feature of the next adjacent layer, so that the problem of defect feature information loss caused by only selecting the feature map of the uppermost layer when extracting features by multilayer convolution in the conventional neural network is solved, the detail features of small defects are enhanced, the defect feature extraction is more accurate, and the sensitivity and the detection capability of extreme shape defects such as the small defects, the long and thin defects and the like are improved.
In one embodiment of the present disclosure, referring to fig. 3, step S1022 includes:
generating candidate sub-regions in the first defect feature map with sequentially reduced size;
obtaining a true value frame according to the position of the flaw marked by the label in each image;
comparing the candidate subarea of each image with the truth value frame to screen out candidate anchors;
adjusting the shape of the candidate anchor through regression to enable the candidate anchor of each image to approach to a true value frame, and obtaining a prior anchor corresponding to the first defect characteristic diagram with the size reduced in sequence;
and inputting the first flaw characteristic diagram with the size reduced in sequence and the prior anchor corresponding to the first flaw characteristic diagram into a preset classification network, and determining parameters of the flaw detection network model.
Specifically, the prior anchor generation network comprises two steps of position prediction and shape prediction. And (3) predicting the central point of the candidate anchor, and subtracting the image mean value of all channels corresponding to the first defect feature map from the image of each channel (the number of the channels can be set by self) in the first defect feature map with the sequentially reduced size, and obtaining a probability map with the same size as the first defect feature map through 1-by-1 convolution and sigmoid functions (the probability map comprises the probability that each pixel in the first defect feature map is a defect). And comparing a predefined threshold delta L with the probability that each pixel is defective, determining the pixels exceeding the threshold to obtain a plurality of candidate sub-regions, screening the candidate sub-regions containing the central point of the true value frame to be candidate anchors, and taking the central point of the candidate sub-regions as the central point of the candidate anchors. The shape prediction is to predict the shape and size of the prior anchor, map the flaw true value frame in the original image onto the first flaw feature map with successively reduced size to obtain the true value frame on the first flaw feature map, and adjust the width and height of the candidate anchor on the first flaw feature map with successively reduced size by regression to finally make the intersection ratio of the adjusted candidate anchor and the true value frame on the corresponding first flaw feature map be the maximum, namely the prior anchor corresponding to the first flaw feature map with the size.
The threshold Δ L may be set according to actual conditions, such as 0.5, 0.6,. 065, and so on. The present disclosure is not so limited.
In the embodiment, the position shape of the prior anchor is determined by comparing the candidate anchor with the true value box on the first flaw characteristic diagram with sequentially reduced sizes, so that a more accurate position shape of the prior anchor is provided for training a flaw detection network model, the number of the candidate anchors is greatly reduced, the positioning precision and the detection speed are effectively improved, the extreme-shaped flaw is still remarkably treated, the problem of fixed shape of the anchor in an Faster R-CNN network is solved, and the condition that the difference between the shape of the anchor and the size of the flaw is large is effectively avoided.
In one embodiment of the present disclosure, the classification network includes 1 pooling layer and 4 fully-connected layers, the sequentially reducing the size of the first defect feature map and the prior anchor corresponding to the first defect feature map are input into a preset classification network, and determining the parameters of the defect detection network model includes:
inputting the first flaw characteristic diagram with the sequentially reduced size and the prior anchor corresponding to the first flaw characteristic diagram into the pooling layer to obtain characteristic diagrams with the same size and the prior anchor corresponding to the characteristic diagrams;
taking the feature maps with the same size and the prior anchors corresponding to the feature maps as the input of a first full-connection layer to obtain the output of the first full-connection layer;
taking the output of the first full connection layer as the input of a second full connection layer to obtain the output of the second full connection layer;
taking the output of the second fully-connected layer as the input of a third fully-connected layer, and enabling the third fully-connected layer to output the position and shape information of the flaw in each image through regression;
and taking the output of the second fully-connected layer as the input of a fourth fully-connected layer, and enabling the fourth fully-connected layer to output the type information of the flaws in each image through Softmax classification.
In one embodiment of the disclosure, in the process of training the fault detection network model, a cross entropy loss function and a shape prediction loss function of a prior anchor are adopted to constrain the training process of the fault detection network model.
Specifically, the image set in S101 may be divided into a training set and a test set, the training set is input into the fault detection network model for training, and in the process of training the fault detection network model, the cross entropy loss function and the shape prediction loss function of the prior anchor are used to constrain the training process of the fault detection network model. And after the training is finished, testing the flaw detection network model by using the test set.
Wherein, the training set and the testing set are divided according to the ratio of 9: 1, 8: 2 or 7: 3, which is not limited by the present disclosure.
More specifically, in one example, the initialization model used for training is a model trained on the COCO target detection dataset of microsoft, the parameter updating mode of the model is Momentum, the initial learning rate is 0.001, after the iteration reaches 30000 times, the learning rate is 0.0001, the Momentum coefficient is 0.9, the batch size is 16, and the IoU threshold for non-maximum suppression is 0.7.
Referring to fig. 4, fig. 4 is a flowchart illustrating a training process of a classification detection model according to an embodiment of the present disclosure. This disclosure compares advantage with prior art and lies in: the method has the advantages that the characteristic of the current layer is fused with the adjacent characteristic of the next layer through a multi-size characteristic fusion algorithm of the characteristic pyramid network, the problem that defect characteristic information is lost due to the fact that only the uppermost characteristic diagram is selected when the characteristics are extracted through multilayer convolution in the traditional neural network is solved, the detail characteristics of small defects are enhanced, the defect characteristic extraction is more accurate, and the sensitivity and the detection capability of extreme shape defects such as the small defects, the long and thin defects and the like are improved. The method has the advantages that a small number of candidate anchors are obtained through selection of candidate sub-regions on the first flaw characteristic diagram with various sizes, the sizes of the candidate anchors are adjusted through regression, and the prior anchors closest to the flaw characteristic sizes are finally obtained.
Referring to fig. 5, fig. 5 is a schematic structural diagram of a model building apparatus for defect detection according to an embodiment of the present disclosure, which may be embedded in an electronic device, and the apparatus mainly includes:
an establishing module 501, configured to establish an image set, where the image set includes defects of different types;
a building module 502 for building a flaw detection network model based on the image set and Faster R-CNN;
a training module 503, configured to train the flaw detection network model to obtain a classification detection model;
the classification detection model is used for detecting flaws of an image to be detected and outputting types, shapes and positions of the flaws in the image to be detected.
The establishing module 501 comprises:
an acquisition sub-module for acquiring at least one image;
the marking submodule is used for marking the flaws in the image by adopting a label to obtain the image set;
wherein the label is marked with at least one of type, shape and position information of the flaw.
In one embodiment of the present disclosure, the building module 502 includes:
the first input submodule is used for inputting the image set into a characteristic pyramid network to obtain a first flaw characteristic image, and the characteristic pyramid network adopts a ResNet-50 characteristic extraction network;
and the second input submodule is used for inputting the first flaw characteristic diagram into a prior anchor generation network and constructing a flaw detection network model based on Faster R-CNN.
In one embodiment of the present disclosure, the first input submodule is specifically configured to: inputting the image set into the feature pyramid network; convolving the marked flaws in the image set from bottom to top to obtain a primary flaw characteristic diagram with sequentially reduced sizes; performing 1 × 1 convolution dimensionality reduction on all the initial flaw feature maps to obtain intermediate flaw feature maps; and all the intermediate defect characteristic diagrams are up-sampled from top to bottom and are fused with the adjacent intermediate defect characteristic diagram of the next size, so that a first defect characteristic diagram with the size being sequentially reduced is obtained.
In one embodiment of the present disclosure, the second input submodule is specifically configured to: generating candidate sub-regions in the first defect feature map with sequentially reduced size; obtaining a true value frame according to the position of the flaw marked by the label in each image; comparing the candidate subarea of each image with the truth value frame to screen out candidate anchors; adjusting the shape of the candidate anchor through regression to enable the candidate anchor of each image to approach to a true value frame, and obtaining a prior anchor corresponding to the first defect characteristic diagram with the size reduced in sequence; and inputting the first flaw characteristic diagram with the size reduced in sequence and the prior anchor corresponding to the first flaw characteristic diagram into a preset classification network, and determining parameters of the flaw detection network model.
In one embodiment of the present disclosure, the classification network includes 1 pooling layer and 4 fully-connected layers, the sequentially reducing the size of the first defect feature map and the prior anchor corresponding to the first defect feature map are input into a preset classification network, and determining the parameters of the defect detection network model includes: inputting the first flaw characteristic diagram with the sequentially reduced size and the prior anchor corresponding to the first flaw characteristic diagram into the pooling layer to obtain characteristic diagrams with the same size and the prior anchor corresponding to the characteristic diagrams; taking the feature maps with the same size and the prior anchors corresponding to the feature maps as the input of a first full-connection layer to obtain the output of the first full-connection layer; taking the output of the first full connection layer as the input of a second full connection layer to obtain the output of the second full connection layer; taking the output of the second fully-connected layer as the input of a third fully-connected layer, and enabling the third fully-connected layer to output the position and shape information of the flaw in each image through regression; and taking the output of the second fully-connected layer as the input of a fourth fully-connected layer, and enabling the fourth fully-connected layer to output the type information of the flaws in each image through Softmax classification.
In one embodiment of the disclosure, in the process of training the fault detection network model, a cross entropy loss function and a shape prediction loss function of a prior anchor are adopted to constrain the training process of the fault detection network model.
For details of the above embodiments, please refer to the related description of the embodiments shown in fig. 1 to 4, which will not be described herein.
Referring to fig. 6, fig. 6 shows a hardware structure diagram of an electronic device.
The electronic device described in this embodiment includes:
a memory 61, a processor 62 and a computer program stored in the memory 61 and executable on the processor, the processor implementing the method for modeling defect detection described in the embodiment of fig. 1.
Further, the electronic device further includes:
at least one input device 63; at least one output device 64.
The memory 61, processor 62 input device 63 and output device 64 are connected by a bus 65.
The input device 63 may be a camera, a touch panel, a physical button, a mouse, or the like. The output device 64 may specifically be a display screen.
The Memory 61 may be a high-speed Random Access Memory (RAM) Memory or a non-volatile Memory (non-volatile Memory), such as a magnetic disk Memory. The memory 61 is used for storing a set of executable program codes, and the processor 62 is coupled to the memory 61.
Further, the embodiment of the present disclosure also provides a computer-readable storage medium, where the computer-readable storage medium may be an electronic device provided in the foregoing embodiments, and the computer-readable storage medium may be the electronic device in the foregoing embodiment shown in fig. 5. The computer readable storage medium has stored thereon a computer program which, when executed by a processor, implements the model building method for flaw detection described in the foregoing embodiment shown in fig. 1. Further, the computer-readable storage medium may be various media that can store program codes, such as a usb disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk.
It should be noted that each functional module in each embodiment of the present disclosure may be integrated into one processing module, or each module may exist alone physically, or two or more modules are integrated into one module. The integrated module can be realized in a hardware mode, and can also be realized in a software functional module mode.
The integrated module, if implemented in the form of a software functional module and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention may be substantially or partially embodied in the form of a software product, or all or part of the technical solution that contributes to the prior art.
It should be noted that, for the sake of simplicity, the above-mentioned method embodiments are described as a series of acts or combinations, but those skilled in the art should understand that the present invention is not limited by the described order of acts, as some steps may be performed in other orders or simultaneously according to the present invention. Further, those skilled in the art will appreciate that the embodiments described in the specification are presently preferred and that no acts or modules are necessarily required of the invention.
In the above embodiments, the descriptions of the respective embodiments have respective emphasis, and for parts that are not described in detail in a certain embodiment, reference may be made to related descriptions of other embodiments.
In view of the above description of the method, apparatus, electronic device and storage medium for defect detection provided by the present invention, those skilled in the art will recognize that the concepts of the embodiments of the present invention may be modified in the specific implementation manners and application ranges.
Claims (10)
1. A method of modeling for fault detection, comprising:
establishing an image set, wherein the image set comprises defects of different types;
constructing a flaw detection network model based on the image set and the Faster R-CNN;
training the flaw detection network model to obtain a classification detection model;
the classification detection model is used for detecting flaws of an image to be detected and outputting types, shapes and positions of the flaws in the image to be detected.
2. The model building method according to claim 1, wherein the building an image set comprises:
collecting at least one image, and labeling flaws in the image by adopting a label to obtain an image set;
wherein the label is marked with at least one of type, shape and position information of the flaw.
3. The model building method according to claim 1, wherein the building a flaw detection network model based on the image set and Faster R-CNN comprises:
inputting the image set into a feature pyramid network to obtain a first flaw feature map, wherein the feature pyramid network adopts a ResNet-50 feature extraction network;
and inputting the first flaw characteristic diagram into a prior anchor generation network, and constructing a flaw detection network model based on Faster R-CNN.
4. The method of modeling according to claim 3, wherein said inputting said set of images into a feature pyramid network to obtain a first defect feature map comprises:
inputting the set of images into the feature pyramid network;
convolving the marked flaws in the image set from bottom to top to obtain a primary flaw characteristic diagram with sequentially reduced sizes;
performing 1 × 1 convolution dimensionality reduction on all the initial flaw feature maps to obtain intermediate flaw feature maps;
and all the intermediate defect characteristic diagrams are up-sampled from top to bottom and are fused with the adjacent intermediate defect characteristic diagram of the next size, so that a first defect characteristic diagram with the size being sequentially reduced is obtained.
5. The method of claim 4, wherein inputting the first defect feature map into a prior anchor generation network, and wherein constructing a defect detection network model based on Faster R-CNN comprises:
generating candidate sub-regions in the first defect feature map with sequentially reduced size;
obtaining a true value frame according to the position of the flaw marked by the label in each image;
comparing the candidate subarea of each image with the truth value frame to screen out candidate anchors;
adjusting the shape of the candidate anchor through regression to enable the candidate anchor of each image to approach to a true value frame, and obtaining a prior anchor corresponding to the first defect characteristic diagram with the size reduced in sequence;
and inputting the first flaw characteristic diagram with the sequentially reduced size and the prior anchor corresponding to the first flaw characteristic diagram into a preset classification network, and determining parameters of the flaw detection network model.
6. The model building method of claim 5, wherein the classification network comprises 1 pooling layer and 4 full-link layers, the sequentially reducing the size of the first defect feature map and the prior anchor corresponding to the first defect feature map are input into a preset classification network, and determining the parameters of the defect detection network model comprises:
inputting the first defect characteristic diagram with the sequentially reduced size and the prior anchor corresponding to the first defect characteristic diagram into the pooling layer to obtain characteristic diagrams with the same size and the prior anchor corresponding to the characteristic diagrams;
taking the feature maps with the same size and the prior anchors corresponding to the feature maps as the input of a first full-connection layer to obtain the output of the first full-connection layer;
taking the output of the first full connection layer as the input of a second full connection layer to obtain the output of the second full connection layer;
taking the output of the second fully-connected layer as the input of a third fully-connected layer, and enabling the third fully-connected layer to output the position and shape information of the flaw in each image through regression;
and taking the output of the second fully-connected layer as the input of a fourth fully-connected layer, and enabling the fourth fully-connected layer to output the type information of the flaws in each image through Softmax classification.
7. The model building method according to any one of claims 1 to 6, wherein in the process of training the fault detection network model, a cross entropy loss function and a shape prediction loss function of a priori anchor are used to constrain the training process of the fault detection network model.
8. A modeling apparatus for fault detection, comprising:
the image processing system comprises an establishing module, a processing module and a processing module, wherein the establishing module is used for establishing an image set, and the image set comprises defects of different types;
the building module is used for building a flaw detection network model based on the image set and the Faster R-CNN;
the training module is used for training the flaw detection network model to obtain a classification detection model;
the classification detection model is used for detecting flaws of an image to be detected and outputting types, shapes and positions of the flaws in the image to be detected.
9. An electronic device, comprising: memory, processor and computer program stored on the memory and executable on the processor, characterized in that the processor implements the steps of the method for model building for defect detection according to any of claims 1 to 7 when executing the computer program.
10. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the steps of the method for modeling flaw detection according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010775330.3A CN111915593A (en) | 2020-08-04 | 2020-08-04 | Model establishing method and device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010775330.3A CN111915593A (en) | 2020-08-04 | 2020-08-04 | Model establishing method and device, electronic equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111915593A true CN111915593A (en) | 2020-11-10 |
Family
ID=73288034
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010775330.3A Pending CN111915593A (en) | 2020-08-04 | 2020-08-04 | Model establishing method and device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111915593A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112614101A (en) * | 2020-12-17 | 2021-04-06 | 广东道氏技术股份有限公司 | Polished tile flaw detection method based on multilayer feature extraction and related equipment |
| CN113610848A (en) * | 2021-10-09 | 2021-11-05 | 阿里巴巴（中国）有限公司 | Digital cloth processing system, cloth flaw detection method, device and medium |
| CN114219807A (en) * | 2022-02-22 | 2022-03-22 | 成都爱迦飞诗特科技有限公司 | Mammary gland ultrasonic examination image grading method, device, equipment and storage medium |
| CN115082386A (en) * | 2022-06-07 | 2022-09-20 | 华南理工大学 | Injection molding part flaw detection method and device based on normal sample auxiliary feature extraction and medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109598287A (en) * | 2018-10-30 | 2019-04-09 | 中国科学院自动化研究所 | The apparent flaws detection method that confrontation network sample generates is generated based on depth convolution |
| WO2019091464A1 (en) * | 2017-11-12 | 2019-05-16 | 北京市商汤科技开发有限公司 | Target detection method and apparatus, training method, electronic device and medium |
| CN110647906A (en) * | 2019-08-02 | 2020-01-03 | 杭州电子科技大学 | Clothing target detection method based on fast R-CNN method |
| CN110889838A (en) * | 2019-11-26 | 2020-03-17 | 武汉纺织大学 | Fabric defect detection method and device |
| CN111179239A (en) * | 2019-12-24 | 2020-05-19 | 浙江大学 | Tire X-ray flaw detection method for performing re-ranking by using background features |
-
2020
- 2020-08-04 CN CN202010775330.3A patent/CN111915593A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2019091464A1 (en) * | 2017-11-12 | 2019-05-16 | 北京市商汤科技开发有限公司 | Target detection method and apparatus, training method, electronic device and medium |
| CN109598287A (en) * | 2018-10-30 | 2019-04-09 | 中国科学院自动化研究所 | The apparent flaws detection method that confrontation network sample generates is generated based on depth convolution |
| CN110647906A (en) * | 2019-08-02 | 2020-01-03 | 杭州电子科技大学 | Clothing target detection method based on fast R-CNN method |
| CN110889838A (en) * | 2019-11-26 | 2020-03-17 | 武汉纺织大学 | Fabric defect detection method and device |
| CN111179239A (en) * | 2019-12-24 | 2020-05-19 | 浙江大学 | Tire X-ray flaw detection method for performing re-ranking by using background features |
Non-Patent Citations (1)
| Title |
|---|
| 李宗洪等: "改进Faster R-CNN模型的汽车喷油器 阀座瑕疵检测算法", 广西科技大学学报, vol. 31, no. 1, 31 March 2020 (2020-03-31), pages 1 - 9 * |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112614101A (en) * | 2020-12-17 | 2021-04-06 | 广东道氏技术股份有限公司 | Polished tile flaw detection method based on multilayer feature extraction and related equipment |
| CN112614101B (en) * | 2020-12-17 | 2024-02-20 | 广东道氏技术股份有限公司 | Polished tile flaw detection method based on multilayer feature extraction and related equipment |
| CN113610848A (en) * | 2021-10-09 | 2021-11-05 | 阿里巴巴（中国）有限公司 | Digital cloth processing system, cloth flaw detection method, device and medium |
| CN113610848B (en) * | 2021-10-09 | 2022-04-12 | 阿里巴巴（中国）有限公司 | Digital cloth processing system, cloth flaw detection method, device and medium |
| CN114219807A (en) * | 2022-02-22 | 2022-03-22 | 成都爱迦飞诗特科技有限公司 | Mammary gland ultrasonic examination image grading method, device, equipment and storage medium |
| CN115082386A (en) * | 2022-06-07 | 2022-09-20 | 华南理工大学 | Injection molding part flaw detection method and device based on normal sample auxiliary feature extraction and medium |
| CN115082386B (en) * | 2022-06-07 | 2024-04-26 | 华南理工大学 | Injection molding flaw detection method, device and medium based on normal sample auxiliary feature extraction |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111915593A (en) | Model establishing method and device, electronic equipment and storage medium | |
| CN111402203B (en) | Fabric surface defect detection method based on convolutional neural network | |
| JP7004145B2 (en) | Defect inspection equipment, defect inspection methods, and their programs | |
| KR102150673B1 (en) | Inspection method for appearance badness and inspection system for appearance badness | |
| CN108918536B (en) | Tire mold surface character defect detection method, device, equipment and storage medium | |
| CN109671071B (en) | Underground pipeline defect positioning and grade judging method based on deep learning | |
| CN111402226A (en) | Surface defect detection method based on cascade convolution neural network | |
| CN110889838A (en) | Fabric defect detection method and device | |
| JP7547069B2 (en) | Information processing device, information processing method and program for information processing device | |
| CN111382785A (en) | GAN network model and method for realizing automatic cleaning and auxiliary marking of sample | |
| CN112906794A (en) | Target detection method, device, storage medium and terminal | |
| JP7496008B2 (en) | Information processing device, method for controlling information processing device, and program | |
| CN114743102A (en) | Furniture board oriented flaw detection method, system and device | |
| CN113538603B (en) | Optical detection method and system based on array product and readable storage medium | |
| CN115829995A (en) | Cloth flaw detection method and system based on pixel-level multi-scale feature fusion | |
| CN111507972A (en) | Tunnel surface defect detection method combining convolutional neural network and support vector machine | |
| CN114255212A (en) | FPC surface defect detection method and system based on CNN | |
| CN110706224A (en) | Optical element weak scratch detection method, system and device based on dark field image | |
| CN112330594A (en) | Paper tube defect detection method and device, electronic equipment and storage medium | |
| CN112669300A (en) | Defect detection method and device, computer equipment and storage medium | |
| CN115049619A (en) | Efficient flaw detection method for complex scene | |
| CN116342536A (en) | Aluminum strip surface defect detection method, system and equipment based on lightweight model | |
| CN107895371A (en) | Textile flaw detection method based on peak value coverage values and Gabor characteristic | |
| CN117067112B (en) | Water cutting machine and control method thereof | |
| CN116823775A (en) | Display screen defect detection method based on deep learning |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |