CN111915593A - Model building method, device, electronic device and storage medium - Google Patents

Abstract

A model establishing method, a device, electronic equipment and a storage medium for flaw detection are applied to the technical field of detection and comprise the following steps: establishing an image set, wherein the image set comprises defects of different types; constructing a flaw detection network model based on the image set and the Faster R-CNN; training the flaw detection network model to obtain a classification detection model; the classification detection model is used for detecting flaws of an image to be detected and outputting types, shapes and positions of the flaws in the image to be detected. The method can be used for detecting the flaw of the extreme size.

Description

Model building method, device, electronic device and storage medium

technical field

The present disclosure relates to the technical field of image detection, and in particular, to a model establishment method, device, electronic device and storage medium for defect detection.

Background technique

In recent years, with the development of science and technology, the automation of product quality inspection has become one of the main trends in the development of modern production. Automated inspection of defects is of great significance to reduce labor, improve production efficiency, and promote industry intelligence.

The traditional manual visual inspection method has problems such as low detection efficiency, slow detection speed, low detection accuracy, and inconsistent detection standards. Due to the long inspection work time, inspectors are prone to visual fatigue, not only the false detection rate and missed inspection rate are high, but also the damage to the human eye is great. Therefore, defect detection methods based on machine vision and deep learning emerge as the times require.

Deep learning algorithms for defect detection have been widely studied, and are mainly used in surface defect detection such as fabrics, blankets, and floor tiles. In the prior art, some use the basic network, the region proposal network and the Fast R-CNN detection network to establish an image classification model based on deep learning, and perform feature extraction on the input data in each iteration during the training process, without the need for manual design of cumbersome images. The image feature extractor only completes the preliminary image data screening for conventional shape defects, and does not enhance the characteristics of defects such as fabrics. In addition, the convolution process will also lose smaller defect features, so the detection accuracy is not high. Some search and locate defect areas through fast recurrent convolutional neural network methods, and combine features and detectors into a framework to automatically detect cloth defects. Among them, the RPN network is used to generate suggestion windows, and each image generates 300 suggested windows. The number of suggested windows is very large, resulting in very slow detection speed.

SUMMARY OF THE INVENTION

The main purpose of the present disclosure is to provide a model establishment method, device, electronic device and storage medium for flaw detection, which has high detection accuracy and can be used for flaw detection of extreme sizes.

In order to achieve the above object, a first aspect of the embodiments of the present disclosure provides a method for establishing a model for defect detection, including:

creating a set of images that includes different types of imperfections;

Build a defect detection network model based on the image set and Faster R-CNN;

Train the defect detection network model to obtain a classification detection model;

Wherein, the classification detection model is used to perform flaw detection on the image to be detected, and output the type, shape and position of the flaw in the image to be detected.

Optionally, the establishing an image set includes:

collecting at least one image, and labeling the defects in the image with a label to obtain the image set;

Wherein, the label is marked with at least one of the type, shape and position information of the defect.

Optionally, the building a defect detection network model based on the image set and Faster R-CNN includes:

Inputting the image set into a feature pyramid network to obtain a first defect feature map, and the feature pyramid network adopts a ResNet-50 feature extraction network;

The first defect feature map is input into the prior anchor generation network, and a defect detection network model based on Faster R-CNN is constructed.

Optionally, the inputting the image set into the feature pyramid network to obtain the first defect feature map includes:

inputting the image set into the feature pyramid network;

Convolving the marked flaws in the image set from bottom to top to obtain a preliminary flaw feature map with decreasing sizes;

Perform 1*1 convolutional dimension reduction on all preliminary defect feature maps to obtain intermediate defect feature maps;

All the intermediate defect feature maps are up-sampled from top to bottom, and are fused with the adjacent intermediate defect feature maps of the next size to obtain the first defect feature map with decreasing size in turn.

Optionally, inputting the first defect feature map into a priori anchor generation network, and constructing a defect detection network model based on Faster R-CNN includes:

generating candidate sub-regions in the first defect feature map with successively decreasing sizes;

According to the position of the defect marked by the label in each image, the ground-truth box is obtained;

Compare the candidate sub-regions of each image with the ground-truth box to filter out candidate anchors;

Adjust the shape of the candidate anchor by regression, so that the candidate anchor of each image is close to the ground truth frame, and obtain the prior anchor corresponding to the first defect feature map with the size decreasing in turn;

The size is sequentially reduced to the first defect feature map and the prior anchors corresponding to the first defect feature map are input into a preset classification network to determine the parameters of the defect detection network model.

Optionally, the classification network includes one pooling layer and four fully-connected layers, and the first defect feature map and the prior anchor input corresponding to the first defect feature map are sequentially reduced in size. The set classification network, the parameters to determine the defect detection network model include:

The first defect feature map and the prior anchors corresponding to the first defect feature map are sequentially reduced in size and input to the pooling layer to obtain feature maps with the same size and prior anchors corresponding to the feature maps. Anchor test;

Using the feature map with the same size and the prior anchor corresponding to the feature map as the input of the first fully connected layer to obtain the output of the first fully connected layer;

Using the output of the first fully connected layer as the input of the second fully connected layer to obtain the output of the second fully connected layer;

The output of the second fully connected layer is used as the input of the third fully connected layer, so that the third fully connected layer outputs the position and shape information of the flaws in each image through regression;

The output of the second fully-connected layer is used as the input of the fourth fully-connected layer, so that the fourth fully-connected layer outputs the type information of defects in each image through Softmax classification.

Optionally, in the process of training the defect detection network model, a cross-entropy loss function and a shape prediction loss function of a priori anchor are used to constrain the training process of the defect detection network model.

A second aspect of the embodiments of the present disclosure provides a model building apparatus for defect detection, including:

a building module for building an image set, the image set including different types of flaws;

Building a module for building a defect detection network model based on the image set and Faster R-CNN;

a training module for training the defect detection network model to obtain a classification detection model;

Wherein, the classification detection model is used to perform flaw detection on the image to be detected, and output the type, shape and position of the flaw in the image to be detected.

A third aspect of the embodiments of the present disclosure provides an electronic device, including:

A memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, when the processor executes the program, the model for defect detection provided by the first aspect of the embodiments of the present disclosure is implemented method.

A fourth aspect of the embodiments of the present disclosure provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, implements the model establishment for defect detection provided in the first aspect of the embodiments of the present disclosure method.

It can be seen from the above embodiments of the present disclosure that the model building method, device, electronic device and storage medium for defect detection provided by the present disclosure, through the multi-scale feature fusion algorithm of the feature pyramid network, the features of this layer and the adjacent next Layer feature fusion solves the problem of loss of defect feature information caused by only selecting the uppermost layer feature map when extracting features by multi-layer convolution in traditional neural networks, enhances the details of small defects, makes defect feature extraction more accurate, and improves accuracy. Sensitivity and detection capabilities for extreme shape flaws such as small and elongated flaws. By selecting the candidate sub-regions on the first defect feature map of each size, a smaller number of candidate anchors are obtained, and then the size of the candidate anchors is adjusted by regression, and finally the prior anchor closest to the size of the defect feature is obtained. The prior anchor is not only suitable for flaw detection of extreme sizes, but also solves the problem of the fixed shape of the anchor in the Faster R-CNN network. It also greatly reduces the number of candidate anchors, effectively improving the detection speed.

Description of drawings

In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, the following briefly introduces the accompanying drawings that need to be used in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only These are some embodiments of the present disclosure, and for those skilled in the art, other drawings can also be obtained from these drawings without creative effort.

FIG. 1 is a schematic flowchart of a model building method for defect detection according to an embodiment of the present disclosure;

2 is a schematic diagram of a feature pyramid network provided by an embodiment of the present disclosure;

3 is a schematic diagram of a prior anchor generation network provided by an embodiment of the present disclosure;

FIG. 4 is a training flowchart of a classification detection model provided by an embodiment of the present disclosure;

FIG. 5 is a schematic structural diagram of a model building apparatus for defect detection according to an embodiment of the present disclosure;

FIG. 6 shows a schematic diagram of the hardware structure of an electronic device.

Detailed ways

In order to make the disclosed purposes, features and advantages of the present disclosure more obvious and understandable, the technical solutions in the embodiments of the present disclosure will be described clearly and completely below with reference to the accompanying drawings in the embodiments of the present disclosure. The embodiments described above are only a part of the embodiments of the present disclosure, but not all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by those skilled in the art without creative efforts shall fall within the protection scope of the present disclosure.

The classification and detection model disclosed in the present disclosure includes a feature extraction network, a region of interest generation network, and an R-CNN classification and positioning network. The feature extraction network is a feature pyramid network, which not only avoids the loss of small feature information, but also supplements and enhances flawed features. Thereby, the defect features can be extracted more accurately. The said region of interest generation network adopts a priori anchor generation network, and the position and shape of the prior anchor is determined by comparing the candidate anchor and the ground-truth frame on the feature maps of different sizes, which provides more information for training. Accurate the position and shape of the prior anchor, while greatly reducing the number of candidate anchors, effectively improving the positioning accuracy and detection speed, and still has a significant effect on the flaws of extreme shapes.

Please refer to FIG. 1. FIG. 1 is a schematic flowchart of a method for establishing a model for defect detection according to an embodiment of the present disclosure. The method mainly includes the following steps:

S101, establish an image set, the image set includes different types of defects;

Specifically, there should be multiple images of each defect type, for example, 100 images, 200 images, etc., and the image size is not required. Because fabric flaws cover many extreme shapes, the present disclosure takes fabric flaws as an example. The types of fabric defects are divided into no defects, holes, stains, snagging, knots, pattern jumps, 100 feet, coarse warp, coarse weft, broken warp, broken weft, thin and dense files, wear marks, rolling marks, dead. Wrinkle, double weft, double warp, reed road, poor weft and so on.

S102, build a defect detection network model based on the image set and Faster R-CNN;

S103, train the defect detection network model to obtain a classification detection model;

Wherein, the classification detection model is used to perform flaw detection on the image to be detected, and output the type, shape and position of the flaw in the image to be detected.

In one of the embodiments of the present disclosure, step S101 includes:

Collect at least one image, and label the defects in the image to obtain the image set;

Wherein, the label is marked with at least one of the type, shape and position information of the defect.

Optionally, a label library may be established based on the labels, and the label files in the label library are established according to the COCO format. Taking fabric defects as an example, an image set is formed from all the collected fabric image data with defects.

In this embodiment, a classification detection model is obtained by building a defect detection network model based on the image set and Faster R-CNN, and then training the defect detection network model. It can effectively improve positioning accuracy and detection speed, and still has a significant effect on flaws with extreme shapes.

In one embodiment of the present disclosure, step S102 includes:

S1021, input the image set into a feature pyramid network to obtain a first defect feature map, and the feature pyramid network adopts a ResNet-50 feature extraction network;

S1022 , input the first defect feature map into the prior anchor generation network, and construct a defect detection network model based on Faster R-CNN.

Specifically, the ResNet-50 feature extraction network consists of 49 convolutional layers and a fully connected layer. The collected fabric images are convolved from bottom to top to extract defect features with decreasing sizes, except that the first layer is 7 Except for *7 convolution, the rest are 1*1 convolution and 3*3 convolution, and a series of feature maps of different sizes are obtained (the size decreases in turn). Preliminary defect feature maps C1, C2, C3 of layers 10, 40, and 49 are subjected to 1*1 convolution for dimensionality reduction and/or upsampling. It should be noted that the present disclosure does not limit the number and quantity of the selected preliminary defect feature maps, and the above is only a schematic illustration.

In this embodiment, the feature extraction network is selected as the feature pyramid network, which not only avoids the loss of small feature information, but also supplements and enhances the defect features, so as to extract the defect features more accurately. The anchor generation network can provide a more accurate position and shape of the prior anchor for the training of the defect detection network, and at the same time greatly reduce the number of candidate anchors, effectively improve the positioning accuracy and detection speed, and still have significant defects in extreme shapes. Effect.

In one embodiment of the present disclosure, referring to FIG. 2 , step S1021 includes:

Input the image set into the feature pyramid network;

Convolve the annotated defects in the image set from bottom to top to obtain preliminary defect feature maps with decreasing sizes;

Perform 1*1 convolutional dimension reduction on all preliminary defect feature maps to obtain intermediate defect feature maps;

All the intermediate defect feature maps are up-sampled from top to bottom, and are fused with the adjacent intermediate defect feature maps of the next size to obtain the first defect feature map with decreasing size in turn.

Understandably, referring to Fig. 2, C1, C2, and C3 in Fig. 2 are preliminary defect feature maps with decreasing sizes, F1, F2, and F3 are first defect feature maps with decreasing sizes, wherein the corresponding preliminary defect features The size of the image, the intermediate flaw feature map and the first flaw feature map are the same (the intermediate flaw feature map is not shown in Figure 2), for example, the size of C1 is the same as that of F1, the size of C2 is the same as that of F2, and the size of C3 is the same as that of F3 .

It is understandable that the first defect feature map includes the preliminary defect feature map and all the defect feature information in the preliminary defect feature map of the next size adjacent to the preliminary defect feature map. Specifically, please refer to FIG. 2 , the preliminary defect feature map of the next size adjacent to C1 is C2, and the preliminary defect feature map of the next size adjacent to C2 is C3. F1 includes the defect feature information of C1 and C2, and F2 includes the feature information of C2 and C3. If C3 is the last preliminary defect feature map, since C3 of the last size has no adjacent preliminary defect feature map of the next size, Therefore, only 1*1 convolution dimension reduction is performed on C3, and the subsequent top-down upsampling is not performed, so F3 only includes all the defect feature information of C3. If C3 has an adjacent preliminary flaw feature map of the next size, normal 1*1 convolutional dimension reduction and top-down upsampling are performed. It should be noted that FIG. 2 is only a schematic illustration, and the present disclosure does not limit the number of preliminary defect feature maps whose sizes are successively reduced by convolution, which may be 1, 2, 3, 4 one and so on.

In this embodiment, the features of this layer are fused with the features of the adjacent next layer, which solves the problem of loss of defect feature information caused by only selecting the uppermost layer feature map when extracting features by multi-layer convolution in the traditional neural network, and enhances the The detailed features of small flaws make the extraction of flaw features more accurate, and improve the sensitivity and detection ability of extreme shape flaws such as small flaws and slender flaws.

In one embodiment of the present disclosure, referring to FIG. 3 , step S1022 includes:

generating candidate sub-regions in the first defect feature map with successively decreasing sizes;

According to the position of the defect marked by the label in each image, the ground-truth box is obtained;

Compare the candidate sub-regions of each image with the ground-truth box to filter out candidate anchors;

Adjust the shape of the candidate anchor by regression, so that the candidate anchor of each image is close to the ground truth frame, and obtain the prior anchor corresponding to the first defect feature map with the size decreasing in turn;

The size is sequentially reduced to the first defect feature map and the prior anchors corresponding to the first defect feature map are input to a preset classification network to determine the parameters of the defect detection network model.

Specifically, the prior anchor generation network includes two steps: position prediction and shape prediction. The position prediction is to predict the center point of the candidate anchor, and the images of each channel (the number of channels can be set by yourself) in the first defect feature map with decreasing sizes are subtracted from the images of all channels corresponding to the first defect feature map respectively. Average and obtain a probability map with the same size as the first defect feature map through 1*1 convolution and sigmoid function (the probability map includes the probability that each pixel in the first defect feature map is a defect). Compare the predefined threshold ΔL with the probability that each pixel is a defect, determine the pixels that exceed the threshold, and obtain several candidate sub-regions, and filter the candidate sub-regions containing the center point of the ground truth box to be the candidate anchor, the candidate sub-region The center point of is the center point of the candidate anchor. Shape prediction is to predict the shape and size of the prior anchor, map the true value frame of the defect in the original image to the first defect feature map with decreasing size, and obtain the true value box on the first defect feature map, and adjust the size through regression. The width and height of the candidate anchors on the first defect feature map are successively reduced, and finally the intersection ratio of the adjusted candidate anchor and the ground truth box on the corresponding first defect feature map is the largest, which is the first defect feature map of this size. the corresponding prior anchors.

The threshold ΔL can be set according to the actual situation, such as 0.5, 0.6, .065 and so on. The present disclosure does not limit this.

In this embodiment, the position and shape of the prior anchor is determined by comparing the candidate anchor on the first defect feature map with decreasing size and the ground truth frame, so as to provide a more accurate position and shape of the prior anchor for training the defect detection network model At the same time, the number of candidate anchors is greatly reduced, the positioning accuracy and detection speed are effectively improved, and it still has a significant effect on the defects of extreme shapes. It also solves the problem of the fixed shape of the anchors in the Faster R-CNN network, effectively avoiding the A situation where the shape of the anchor differs greatly from the size of the flaw.

In one embodiment of the present disclosure, the classification network includes one pooling layer and four fully-connected layers, and the size is sequentially reduced by the first defect feature map and the prior anchor input corresponding to the first defect feature map. The preset classification network, the parameters for determining the defect detection network model include:

Decrease the size of the first flaw feature map and the prior anchors corresponding to the first flaw feature map in turn and input them to the pooling layer to obtain feature maps with the same size and a priori anchors corresponding to the feature maps;

The feature map with the same size and the prior anchor corresponding to the feature map are used as the input of the first fully connected layer, and the output of the first fully connected layer is obtained;

The output of the first fully connected layer is used as the input of the second fully connected layer, and the output of the second fully connected layer is obtained;

The output of the second fully connected layer is used as the input of the third fully connected layer, so that the third fully connected layer outputs the position and shape information of the defects in each image through regression;

The output of the second fully-connected layer is used as the input of the fourth fully-connected layer, so that the fourth fully-connected layer outputs the type information of defects in each image through Softmax classification.

In one of the embodiments of the present disclosure, in the process of training the defect detection network model, a cross-entropy loss function and a shape prediction loss function of a priori anchor are used to constrain the training process of the defect detection network model.

Specifically, the image set in S101 can be divided into a training set and a test set, and the training set can be input into the defect detection network model for training. In the process of training the defect detection network model, the cross entropy loss function and the prior anchor are used. The shape prediction loss function constrains the training process of the flaw detection network model. After the training is completed, use the test set to test the flaw detection network model.

The training set and the test set are divided according to the ratio of 9:1, 8:2 or 7:3, which is not limited in the present disclosure.

More, in one example, the initialization model used for training is a model trained on Microsoft's COCO target detection dataset, the parameter update method of the model is Momentum, the initial learning rate is 0.001, and after 30,000 iterations , the learning rate is 0.0001, the momentum coefficient is 0.9, the batch size is 16, and the IoU threshold for non-maximum suppression is 0.7.

Please refer to FIG. 4 above. FIG. 4 is a training flowchart of a classification detection model provided by an embodiment of the present disclosure. Compared with the prior art, the advantage of the present disclosure is that: through the multi-dimensional feature fusion algorithm of the feature pyramid network, the features of this layer are fused with the features of the adjacent next layer, which solves the problem of multi-layer convolution in the traditional neural network when extracting features. Only the defect feature information loss caused by the top-level feature map is selected, which enhances the detail features of small defects, makes the extraction of defect features more accurate, and improves the sensitivity and detection ability of extreme shape defects such as small defects and slender defects. By selecting the candidate sub-regions on the first defect feature map of each size, a smaller number of candidate anchors are obtained, and then the size of the candidate anchors is adjusted by regression, and finally the prior anchor closest to the size of the defect feature is obtained. The prior anchor is not only suitable for flaw detection of extreme sizes, but also solves the problem of the fixed shape of the anchor in the Faster R-CNN network. It also greatly reduces the number of candidate anchors, effectively improving the detection speed.

Please refer to FIG. 5. FIG. 5 is a schematic structural diagram of a model building apparatus for defect detection provided by an embodiment of the present disclosure. The apparatus can be built in an electronic device, and the apparatus mainly includes:

establishing module 501, for establishing an image set, the image set includes different types of defects;

A building module 502 is used to build a defect detection network model based on the image set and Faster R-CNN;

A training module 503 is used to train the defect detection network model to obtain a classification detection model;

Wherein, the classification detection model is used to perform flaw detection on the image to be detected, and output the type, shape and position of the flaw in the image to be detected.

The establishment module 501 includes:

the acquisition sub-module, due to the acquisition of at least one image;

The labeling sub-module is used to label the defects in the image to obtain the image set;

Wherein, the label is marked with at least one of the type, shape and position information of the defect.

In one embodiment of the present disclosure, the building module 502 includes:

The first input sub-module is used to input the image set into the feature pyramid network to obtain the first defect feature map, and the feature pyramid network adopts the ResNet-50 feature extraction network;

The second input sub-module is used to input the first defect feature map into the prior anchor generation network to construct a defect detection network model based on Faster R-CNN.

In one of the embodiments of the present disclosure, the first input sub-module is specifically configured to: input the image set into the feature pyramid network; perform convolution of the labeled flaws in the image set from bottom to top to obtain the successive reduction in size The initial defect feature map of the The intermediate defect feature maps of one size are fused to obtain the first defect feature map with decreasing size in turn.

In one of the embodiments of the present disclosure, the second input sub-module is specifically configured to: generate candidate sub-regions in the first defect feature map with decreasing sizes in sequence; obtain the true value according to the position of the defect marked by the label in each image box; compare the candidate sub-region of each image with the ground-truth box, and filter out candidate anchors; adjust the shape of the candidate anchor by regression to make the candidate anchor of each image approach the ground-truth box, and obtain the same size in order Decrease the prior anchor corresponding to the first flaw feature map; reduce the size in turn to the first flaw feature map and the prior anchor corresponding to the first flaw feature map and input the preset classification network to determine the flaw detection network model. parameter.

In one embodiment of the present disclosure, the classification network includes one pooling layer and four fully-connected layers, and the size is sequentially reduced by the first defect feature map and the prior anchor input corresponding to the first defect feature map. For the preset classification network, determining the parameters of the defect detection network model includes: sequentially reducing the size of the first defect feature map and inputting the prior anchors corresponding to the first defect feature map to the pooling layer, and obtaining the same size The feature map and the prior anchor corresponding to the feature map; the feature map with the same size and the prior anchor corresponding to the feature map are used as the input of the first fully connected layer, and the first fully connected layer is obtained. The output of a fully connected layer; the output of the first fully connected layer is used as the input of the second fully connected layer, and the output of the second fully connected layer is obtained; the output of the second fully connected layer is used as the first fully connected layer. The input of the three fully connected layers makes the third fully connected layer output the location and shape information of defects in each image through regression; the output of the second fully connected layer is used as the input of the fourth fully connected layer, Make this fourth fully connected layer output the type information of the defects in each image through Softmax classification.

In one of the embodiments of the present disclosure, in the process of training the defect detection network model, a cross-entropy loss function and a shape prediction loss function of a priori anchor are used to constrain the training process of the defect detection network model.

For details that are not described in the above embodiments, please refer to the related descriptions of the embodiments shown in FIGS. 1 to 4 , which will not be repeated here.

Please refer to FIG. 6, which shows a hardware structure diagram of an electronic device.

The electronic equipment described in this embodiment includes:

The memory 61 , the processor 62 and the computer program stored in the memory 61 and executable on the processor, when the processor executes the program, implements the model building method for defect detection described in the embodiment shown in FIG. 1 .

Further, the electronic device also includes:

At least one input device 63 ; at least one output device 64 .

The above-mentioned memory 61 , processor 62 input device 63 and output device 64 are connected through a bus 65 .

The input device 63 may specifically be a camera, a touch panel, a physical button, a mouse, or the like. The output device 64 may specifically be a display screen.

The memory 61 may be a high-speed random access memory (RAM, Random Access Memory) memory, or may be a non-volatile memory (non-volatile memory), such as a disk memory. Memory 61 is used to store a set of executable program codes, and processor 62 is coupled to memory 61 .

Further, an embodiment of the present disclosure further provides a computer-readable storage medium, and the computer-readable storage medium may be provided in the electronic device in the above-mentioned embodiments, and the computer-readable storage medium may be the one shown in FIG. 5 above. the electronic device in the example. The computer-readable storage medium stores a computer program, and when the program is executed by the processor, implements the model building method for defect detection described in the embodiment shown in FIG. 1 . Further, the computer-storable medium can also be a USB flash drive, a removable hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, etc. medium of program code.

It should be noted that each functional module in each embodiment of the present disclosure may be integrated into one processing module, or each module may exist physically alone, or two or more modules may be integrated into one module. The above-mentioned integrated modules can be implemented in the form of hardware, and can also be implemented in the form of software function modules.

If the integrated modules are implemented in the form of software functional modules and sold or used as independent products, they may be stored in a computer-readable storage medium. Based on such understanding, the technical solutions of the present invention can be embodied in the form of software products in essence or in part that contributes to the prior art, or all or part of the technical solutions.

It should be noted that, for the convenience of description, the foregoing method embodiments are all expressed as a series of action combinations, but those skilled in the art should know that the present invention is not limited by the described action sequence. As in accordance with the present invention, certain steps may be performed in other orders or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily all necessary to the present invention.

In the above-mentioned embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference may be made to the relevant descriptions of other embodiments.

The above is a description of a model establishment method, device, electronic device and storage medium for defect detection provided by the present invention. There will be changes in the above, and in conclusion, the content of this specification should not be construed as a limitation to the present invention.

Claims (10)

1. a model building method for flaw detection, is characterized in that, comprises: creating a set of images that includes different types of imperfections; Build a defect detection network model based on the image set and Faster R-CNN; Train the defect detection network model to obtain a classification detection model; Wherein, the classification detection model is used to perform flaw detection on the image to be detected, and output the type, shape and position of the flaw in the image to be detected. 2. The method for establishing a model according to claim 1, wherein the establishing an image set comprises: collecting at least one image, and labeling the defects in the image with a label to obtain the image set; Wherein, the label is marked with at least one of the type, shape and position information of the defect. 3. model building method according to claim 1, is characterized in that, described based on described image set and FasterR-CNN builds flaw detection network model and comprises: Inputting the image set into a feature pyramid network to obtain a first defect feature map, and the feature pyramid network adopts a ResNet-50 feature extraction network; The first defect feature map is input into the prior anchor generation network, and a defect detection network model based on Faster R-CNN is constructed. 4. The method for establishing a model according to claim 3, wherein the inputting the image set into a feature pyramid network to obtain the first flaw feature map comprises: inputting the image set into the feature pyramid network; Convolving the marked flaws in the image set from bottom to top to obtain a preliminary flaw feature map with decreasing sizes; Perform 1*1 convolutional dimension reduction on all preliminary defect feature maps to obtain intermediate defect feature maps; All the intermediate defect feature maps are up-sampled from top to bottom, and are fused with the adjacent intermediate defect feature maps of the next size to obtain the first defect feature map with decreasing size in turn. 5. The method for establishing a model according to claim 4, wherein the described first flaw feature map is input into a priori anchor generation network, and the construction of the flaw detection network model based on Faster R-CNN comprises: generating candidate sub-regions in the first defect feature map with successively decreasing sizes; According to the position of the defect marked by the label in each image, the ground-truth box is obtained; Compare the candidate sub-regions of each image with the ground-truth box to filter out candidate anchors; Adjust the shape of the candidate anchor by regression, so that the candidate anchor of each image is close to the ground truth frame, and obtain the prior anchor corresponding to the first defect feature map with the size decreasing in turn; The size is sequentially reduced to the first defect feature map and the prior anchors corresponding to the first defect feature map are input into a preset classification network to determine the parameters of the defect detection network model. 6 . The model building method according to claim 5 , wherein the classification network comprises 1 pooling layer and 4 fully connected layers, and the size is sequentially reduced by the first defect feature map and all the The prior anchor corresponding to the first defect feature map is input to a preset classification network, and the parameters for determining the defect detection network model include: Inputting the first defect feature map and the prior anchor corresponding to the first defect feature map in turn to the pooling layer to obtain a feature map with the same size and a prior corresponding to the feature map anchor; Using the feature map with the same size and the prior anchor corresponding to the feature map as the input of the first fully connected layer to obtain the output of the first fully connected layer; Using the output of the first fully connected layer as the input of the second fully connected layer to obtain the output of the second fully connected layer; The output of the second fully connected layer is used as the input of the third fully connected layer, so that the third fully connected layer outputs the position and shape information of the flaws in each image through regression; The output of the second fully-connected layer is used as the input of the fourth fully-connected layer, so that the fourth fully-connected layer outputs the type information of defects in each image through Softmax classification. 7. The model building method according to any one of claims 1 to 6, wherein, in the process of training the defect detection network model, a cross-entropy loss function and a shape prediction loss function of a priori anchor are used to determine the accuracy of the defect detection network model. The training process of the flaw detection network model described above is constrained. 8. A model establishment device for defect detection, characterized in that, comprising: a building module for building an image set, the image set including different types of flaws; Building a module for building a defect detection network model based on the image set and Faster R-CNN; a training module for training the defect detection network model to obtain a classification detection model; Wherein, the classification detection model is used to perform flaw detection on the image to be detected, and output the type, shape and position of the flaw in the image to be detected. 9. An electronic device, comprising: a memory, a processor and a computer program stored on the memory and running on the processor, characterized in that, when the processor executes the computer program, claims 1 to 7 are realized Each step in the model building method for defect detection according to any one of the above. 10. A computer-readable storage medium on which a computer program is stored, characterized in that, when the computer program is executed by a processor, the method for defect detection according to any one of claims 1 to 7 is implemented. The various steps in the model building method.

Images