CN113888542A - Product defect detection method and device - Google Patents

Product defect detection method and device Download PDF

Info

Publication number
CN113888542A
CN113888542A CN202111488051.XA CN202111488051A CN113888542A CN 113888542 A CN113888542 A CN 113888542A CN 202111488051 A CN202111488051 A CN 202111488051A CN 113888542 A CN113888542 A CN 113888542A
Authority
CN
China
Prior art keywords
network
defect detection
product
gradient
defect
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202111488051.XA
Other languages
Chinese (zh)
Other versions
CN113888542B (en
Inventor
卞庆林
郭骏
潘正颐
侯大为
倪文渊
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Changzhou Weiyizhi Technology Co Ltd
Original Assignee
Changzhou Weiyizhi Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Changzhou Weiyizhi Technology Co Ltd filed Critical Changzhou Weiyizhi Technology Co Ltd
Priority to CN202111488051.XA priority Critical patent/CN113888542B/en
Publication of CN113888542A publication Critical patent/CN113888542A/en
Application granted granted Critical
Publication of CN113888542B publication Critical patent/CN113888542B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0004—Industrial image inspection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20092—Interactive image processing based on input by user
    • G06T2207/20104—Interactive definition of region of interest [ROI]

Abstract

The invention provides a product defect detection method and a device, wherein the method comprises the following steps: constructing a deformable convolution defect detection model; acquiring image data of a product to be detected; processing the image data in a balanced manner to obtain sample data; training a deformable convolution defect detection model according to sample data; and performing defect detection on the product to be detected by adopting the trained deformable convolution defect detection model. The method can ensure the balance of the sample data, thereby preventing the over-training of the deformable convolution defect detection model, enhancing the generalization capability and the robustness of the deformable convolution defect detection model, and further improving the cross-platform performance of the deformable convolution defect detection model.

Description

Product defect detection method and device
Technical Field
The invention relates to the technical field of defect detection, in particular to a product defect detection method and a product defect detection device.
Background
The deep learning is applied to the training process of the surface defect detection of the mobile phone, and not only the imbalance of different defect sample types but also the generalization capability of the model are considered. Different mobile phone projects generally need to construct a new detection model and train again, resulting in weak generalization ability of the model, and in addition, different mobile phone surface textures and colors are different, so how to establish a model with strong generalization ability faces a great challenge.
Disclosure of Invention
The invention provides a product defect detection method for solving the technical problems, which can ensure the balance of sample data, thereby preventing the overtraining of a deformable convolution defect detection model, enhancing the generalization capability and robustness of the deformable convolution defect detection model and further improving the cross-platform performance of the deformable convolution defect detection model.
The technical scheme adopted by the invention is as follows:
a product defect detection method comprises the following steps: constructing a deformable convolution defect detection model; acquiring image data of a product to be detected; the image data is processed in an equalizing mode to obtain sample data; training the deformable convolution defect detection model according to the sample data; and carrying out defect detection on the product to be detected by adopting the trained deformable convolution defect detection model.
According to an embodiment of the present invention, the deformable convolution defect detection model includes a first detection module, a second detection module, a third detection module and a fourth detection module, the second detection module is respectively connected to the first detection module and the third detection module, the second detection module and the third detection module are connected to the fourth detection module through a ROI Align network, wherein the first detection module is a residual error network, and specifically includes a first gradient, a second gradient, a third gradient and a fourth gradient, and the first gradient, the second gradient, the third gradient and the fourth gradient are all composed of stacked residual error networks; the second detection module is an FPN (Feature Pyramid) network; the third detection module is an RPN (Region abstraction Network) Network; the fourth detection module is a Head network.
According to an embodiment of the present invention, the stacked residual error network includes a first type stacked residual error network and a second type stacked residual error network, wherein the first type stacked residual error network includes a DCL (variable convolutional Layer) network, and the second type stacked residual error network does not include a DCL network.
According to an embodiment of the invention, said first gradient comprises three said second type of stacked residual networks; the second gradient comprises four of the first type of stacked residual networks; the third gradient comprises six stacked residual networks of the first type; the fourth gradient comprises three stacked residual networks of the first type.
According to one embodiment of the invention, the product to be detected comprises a plurality of project platform products, and the image data comprises image data of the plurality of project platform products.
According to an embodiment of the present invention, the equalizing the image data to obtain sample data specifically includes the following steps: and carrying out balanced processing on the image data by adopting a cycleGAN network to obtain sample data.
According to an embodiment of the present invention, training the deformable convolution defect detection model according to the sample data specifically includes the following steps: performing label distribution on the sample data to obtain a training set, a verification set and a test set; performing on-line data enhancement operation on the training set to obtain a training set to be input; TTA operation is respectively carried out on the test set and the verification set so as to correspondingly obtain a test set to be input and a verification set to be input; and training the deformable convolution defect detection model according to the training set to be input, the test set to be input and the verification set to be input.
A product defect detection apparatus, comprising: a modeling module for constructing a deformable convolution defect detection model; the acquisition module is used for acquiring image data of a product to be detected; the equalization processing module is used for performing equalization processing on the image data to obtain sample data; a training module for training the deformable convolution defect detection model according to the sample data; and the detection module is used for carrying out defect detection on the product to be detected by adopting the trained deformable convolution defect detection model.
A computer device comprises a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor executes the computer program to implement the product defect detection method of the above embodiment.
A non-transitory computer-readable storage medium, on which a computer program is stored, which, when executed by a processor, implements the product defect detection method of the above-described embodiments.
The invention has the beneficial effects that:
the method can ensure the balance of the sample data, thereby preventing the over-training of the deformable convolution defect detection model, enhancing the generalization capability and the robustness of the deformable convolution defect detection model, and further improving the cross-platform performance of the deformable convolution defect detection model.
Drawings
FIG. 1 is a flow chart of a product defect detection method according to an embodiment of the present invention;
FIG. 2 is a block diagram of a deformable convolution defect detection model in accordance with one embodiment of the present invention;
FIG. 3(a) is a block diagram of a first type of stacked residual network in accordance with one embodiment of the present invention;
FIG. 3(b) is a block diagram of a second type of stacked residual network according to an embodiment of the present invention;
FIG. 3(c) is a block diagram of the CB, DCBR, CBR structure according to an embodiment of the present invention;
FIG. 4 is a block diagram of a first detection module in accordance with one embodiment of the present invention;
FIG. 5 is a block diagram of a third detection module in accordance with one embodiment of the present invention;
FIG. 6(a) is a diagram of a process for migrating image data of a project platform product A to a project platform product B according to an embodiment of the present invention;
FIG. 6(B) is a diagram of the process of migrating B project platform product image data to A project platform product according to one embodiment of the present invention;
FIG. 7 is a block diagram of a builder of the cycleGAN network of one embodiment of the present invention;
FIG. 8 is a block diagram of an arbiter for a cycleGAN network in accordance with one embodiment of the present invention;
FIG. 9(a) is a schematic diagram of a CycleGAN network X- > Y according to an embodiment of the present invention;
FIG. 9(b) is a schematic diagram of a CycleGAN network Y- > X according to an embodiment of the present invention;
fig. 10 is a block diagram of a product defect detecting apparatus according to an embodiment of the invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
FIG. 1 is a flowchart of a product defect detection method according to an embodiment of the present invention.
As shown in fig. 1, the method for detecting product defects of the embodiment of the present invention includes the following steps:
and S1, constructing a deformable convolution defect detection model.
In an embodiment of the invention, as shown in fig. 2, the deformable convolution defect detection model may include a first detection module BlockI, a second detection module BlockII, a third detection module BlockIII, and a fourth detection module BlockIV, where the second detection module BlockII is respectively connected to the first detection module BlockI and the third detection module BlockIII, and the second detection module BlockII and the third detection module BlockIII are connected to the fourth detection module BlockIV through an ROI Align network.
Specifically, as shown in fig. 2, the first detection module BlockI may be a residual network, and specifically may include a first gradient stage1, a second gradient stage2, a third gradient stage3, and a fourth gradient stage4, and each of the first gradient stage1, the second gradient stage2, the third gradient stage3, and the fourth gradient stage4 is composed of a stacked residual network; the second detection module block ii may be an FPN network (Feature Pyramid network); the third detection module blockack iii may be an RPN Network (Region forward Network); the fourth detection module BlockIV may be a Head network.
It should be noted that the stacked residual network may include a first type stacked residual network and a second type stacked residual network, where the first type stacked residual network may include a DCL (Deformable convergence Layer) network, as specifically shown in fig. 3 (a); the second type of stacked residual network does not include a DCL network, as shown in fig. 3 (b). Specifically, the first type of stacked residual network, i.e., the stacked residual network including DCLs, is a network including DCBRs, for example, including 3 × 3 DCBRs; the second type of stacked residual network, i.e. the stacked residual network not containing DCL, is a network containing CBR, e.g. containing 3 × 3CBR, wherein, as shown in fig. 3(c), DCBR is composed of Dconv + BN + ReLu, and Dconv is a variable convolution, and CBR is composed of Conv + BN + ReLu. Furthermore, as shown in fig. 3(c), the CBs in the first and second types of stacked residual networks may be composed of Conv + BN, where Conv is a convolutional layer, BN is BatchNorm, i.e., batch normalization, and ReLu is an activation function.
As can be further seen from fig. 3(a) and fig. 3(b), each of the first and second stacked residual networks includes four mapping channels, i.e., X1- > Y1, X2- > Y2, X3- > Y3, and X4- > Y4 mapping channels, and since the DCBR or CBR is stacked, the mapping relationships of the four mapping channels are as follows:
Figure 935032DEST_PATH_IMAGE001
where i denotes the number of channels, for example 1, 2, 3, 4,srepresents the maximum number of mapping channels (here 4), KiAnd the output mapping relation of the input image passing through the 3 × 3DCBR or the 3 × 3CBR is shown.
Further referring to fig. 3(b), 3 × 3CBR networks are respectively disposed in the mapping channels X2- > Y2, X3- > Y3, and X4- > Y4, wherein the output terminal of the 3 × 3CBR network in the mapping channel X2- > Y2 is further connected to the input terminal of the 3 × 3CBR network in the mapping channel X3- > Y3, and the output terminal of the 3 × 3CBR network in the mapping channel X3- > Y3 is further connected to the input terminal of the 3 × 3CBR network in the mapping channel X4- > Y4. Furthermore, as shown in fig. 3(b), the input terminals of the mapping channels X1- > Y1, X2- > Y2, X3- > Y3, and X4- > Y4 are connected to the output terminals of the 1 × 1CBR network, and 1 × 1CB networks are further provided at the output terminals of the mapping channels X1- > Y1, X2- > Y2, X3- > Y3, and X4- > Y4, and the output terminals of the 1 × 1CB networks are further connected to the input terminals of the 1 × 1CBR network, and the output terminals of the 1 × 1CB networks are further connected to the ReLu network.
As can be further seen from fig. 3(a) and 3(b), in the first type of stacked residual error network, a 3 × 3DCBR network may be used to replace a 3 × 3CBR network in the second type of stacked residual error network. Therefore, the method can adapt to model training of irregular defects, and particularly can adapt to detection of geometric deformation defects of different targets by dynamically adjusting the size of the DCL convolution kernel. In addition, by setting the stacked residual error network, various characteristic receptive fields can be obtained in a single basic unit, and the mode of blocking and re-fusing is favorable for extracting effective information from the context.
In one embodiment of the invention, as shown in FIG. 4, the first gradient stage1 may include three second-type stacked residual networks; the second gradient stage2 may include four stacked residual networks of the first type; the third gradient stage3 may include six stacked residual networks of the first type; the fourth gradient stage4 may include three stacked residual networks of the first type.
As further shown in fig. 2 and fig. 4, the first detection module block i may further include a MaxPool network and a 7 × 7CBR network, wherein one end of the 7 × 7CBR network is used for receiving Input data, that is, the other end of the 7 × 7CBR network is connected to one end of the MaxPool network, and the other end of the MaxPool network is connected to the first gradient stage 1. Wherein, the MaxPool network is a maximum pooling network.
In an embodiment of the present invention, as shown in fig. 2, the second detection module BlockII, i.e., the FPN network, may have four 1 × 1 consts, and the four 1 × 1 consts may be respectively connected to the first gradient stage1, the second gradient stage2, the third gradient stage3, and the fourth gradient stage4 in a one-to-one correspondence manner, and the output of the fourth gradient stage4 may obtain P5 after passing through 1 × 1 const, the output of the third gradient stage3 may obtain P4 after passing through 1 × 1 const and fusing with P5, the output of the second gradient stage2 may obtain P3 after passing through 1 × 1 const and fusing with P4, and the output of the first gradient stage1 may obtain P2 after passing through 1 × 1 const and fusing with P3. Further, as shown in FIG. 2, P2, P3, P4 and P5 obtained in the above steps can also obtain RPN/P2, RPN/P3, RPN/P4, RPN/P5 and RPN/P6 through a sliding window operation.
In an embodiment of the invention, as shown in fig. 2, the third detection module BlockIII, i.e., the RPN network, may process RPN/P2, RPN/P3, RPN/P4, RPN/P5, and RPN/P6 layer by layer, so as to obtain the propusals (candidate box).
Specifically, as shown in fig. 5, the third detection module block iii, i.e., the RPN network, may be provided with 3 × 3Conv and two 1 × 1Conv, and the inputs of the 3 × 3Conv may be used to receive RPN/P2, RPN/P3, RPN/P4, RPN/P5, and RPN/P6, the outputs of the 3 × 3Conv may be respectively connected to the inputs of the two 1 × 1Conv, and the outputs of the two 1 × 1Conv may be output to provide propusals.
In one embodiment of the present invention, as shown in fig. 2, the fourth detection module BlockIV, i.e., the Head network, may be provided with an FCN (full volumetric network) and a dual FC (full Connected Layer), and inputs of the FCN and the dual FC may receive the prosusals through ROI Align, and the RPN/P2, RPN/P3, RPN/P4 and RPN/P5, wherein the FCN may output a Mask, and the dual FC may output a Class Label for representing segmentation of each defective pixel level, the Class Label for representing classification of each defect, and a Boundary Box for representing a location of each defect.
And S2, acquiring image data of the product to be detected.
In an embodiment of the present invention, the product to be detected may include a plurality of project platform products, for example, may include an a project platform product, a B project platform product, and a C project platform product, and the primary color of each project platform product is different, for example, the primary color of the a project platform product may be red, the primary color of the B project platform product may be black, and the primary color of the C project platform product may be white; in addition, the A project platform product, the B project platform product and the C project platform product can share one production line and one quality inspection line, so that when an industrial camera is used for acquiring image data of a product to be detected, the image data of the A project platform product, the B project platform product and the C project platform product can be obtained.
S3, the image data is equalized to obtain sample data.
It should be noted that, the image data of the a project platform product, the B project platform product and the C project platform product obtained in step S2 are different in amount due to different quantities of each project platform product, for example, if the quantity of the a project platform product is the largest and the sum of the quantity of the B project platform product and the quantity of the C project platform product is the smallest, the image data of each project platform product may be different, and further, the defect samples of each project platform product may be different, so that the defect samples of different project platforms are unbalanced.
It should be noted that, the image data of the item platform product a, the item platform product B and the item platform product C obtained in step S2 are different from each other in defect samples of different defect types in each item platform product, for example, if there are 7 defect types in the item platform product a, namely, defect type 1, defect type 2, defect type 3, defect type 4, defect type 5, defect type 6 and defect type 7, and the proportion of the 7 defect types in the item platform product image data a is 40%, 25%, 15%, 10%, 5%, 3% and 2%, respectively, then it can be seen that the defect samples of different types are in an unbalanced state, and if the defect detection module is trained by using the defect sample, the characteristics of the defect type 1 and the defect type 2 are over-learned, and the defect type 5, the defect type 6, or the defect detection module is over-trained by using the defect sample, The feature learning of defect class 7 is weak.
Therefore, the image data obtained in the step S2 may be specifically processed by a CycleGAN network to obtain sample data, and mapping between two domains may be implemented through the CycleGAN network instead of mapping between two specific image data, so that mapping between different project platform products may be implemented, and thus, the sample data of each project platform product may be balanced. More specifically, taking defect type 3 as an example, the CycleGAN network equalization processing operations shown in fig. 6(a) and 6(B) can be performed separately, and for example, defect type 3 of the a project platform product, i.e., input 3-a (input data 3-a), and defect type 3 of the B project platform product, i.e., input 3-B (input data 3-B), can be input as inputs to the CycleGAN network.
As shown in fig. 6(a), the input defect type 3 of the a project platform product, i.e. input 3-a, may first pass through the generator network G, i.e. the image data of the a project platform product to the generator network G of the image data of the B project platform product, to obtain Generated-B (sample data B) of the B project platform product. Further, as shown in fig. 6(a), the Generated-B of the B project platform product obtained by the generator network G needs to be determined whether it meets the sample data standard of the B project platform product by the Discriminator B (Discriminator B), and if so, the determination is score 1, and if not, the determination is score 0.
As shown in fig. 6(B), the input defect type 3 of the B project platform product, i.e., input 3-B, may first pass through the generator network F, i.e., the B project platform product image data, to the generator network F of the a project platform product image data, to obtain Generated-a (sample data a) of the a project platform product. Further, as shown in fig. 3(b), the Generated-a of the a project platform product obtained by the generator network F needs to be judged whether it meets the sample data standard of the a project platform product by the Discriminator a (Discriminator a), and if so, the Decision is score 1, and if not, the Decision is score 0.
It should be noted that the CycleGAN network adopted in the above embodiment includes a generator structure as shown in fig. 7, including three CBR (Conv 2D-BatchNorm-Relu) networks, nine Residual Block networks, two CTBR (Conv 2D transit-BatchNorm-Relu) networks, and a Conv2D network and a Tanh network, and specific connection relationships thereof may refer to fig. 7; the structure of the arbiter included in the CycleGAN network used in the above embodiment is shown in fig. 8, and includes a Conv2D network and three Conv2D-InstanceNorm-leak networks-The ReLu network, the specific connection relationship of which can be referred to fig. 8. Wherein, Residual Block is a Residual module, Conv2D Transpose is a transposed convolution, Conv2D is a two-dimensional convolution, Tanh is a hyperbolic function, InstanceNorm is a normalization layer, Leaky-ReLu is a nonlinear activation function.
It should be further noted that the CycleGAN network used in the above embodiments also introduces resistance loss and cycle consistency loss, wherein the resistance loss is the difference between G (input 3-a) and Generated-B, or the difference between F (input 3-B) and Generated-a; the cyclic-consistency loss is specifically shown in fig. 9(a) and fig. 9(b), wherein fig. 9(a) is a forward cyclic consistency loss, the corresponding process is x — > g (x) — > F (g (x)) approximately ≈ x, and the loss is F (g (x)) x; fig. 9(b) shows backward loop consistency loss, y — > f (y) — > G (f (y)) approximately equal to y, and the loss is G (f (y)) y, which is more directly understood as: for each image x in the x domain, x can be brought back to the original image in an image translation cycle; for each image y in the y domain, y can be brought back to the original image in an image translation cycle. It should be noted that cycle3-a (cycle data 3-a) and cycle3-B (cycle data 3-B) generated in fig. 6(a) and 6(B) can be used for the calculation of the forward cycle consistency loss shown in fig. 9 (a).
Training overfitting of the generator network F and the generator network G can be effectively avoided by introducing cycle consistency loss, and therefore required sample data is obtained.
It should be noted that, for the input defect type 3 of the a project platform product, i.e. input 3-a, and the input defect type 3 of the B project platform product, i.e. input 3-B, the selection principle shown in table 1 can also be set:
TABLE 1
Figure 861924DEST_PATH_IMAGE002
Based on the above process, the project platform product data A can be migrated to the project platform product B, so that the project platform product data B can be increased, and data migration between other project platform products can be performed as well, which is not described in detail herein; in addition, data migration of different defect types can be performed between the same project platform product, so that the sample data imbalance can be improved. Certainly, data migration can also be performed between the project platform product A and the project platform product C, or between the project platform product B and the project platform product C; in addition, data migration may also be performed for other defect types 4 to 7, and specific steps may refer to the above embodiments, which are not described in detail here.
Through balanced processing image data, the unbalance phenomenon between different defect sample data can be improved, so that the over-training of the deformable convolution defect detection model can be prevented, and the defects between different project platform products can be trained in a coordinated manner, so that the generalization capability and robustness of the deformable convolution defect detection model can be enhanced, and the cross-platform performance of the deformable convolution defect detection model can be improved.
And S4, training a deformable convolution defect detection model according to the sample data.
Specifically, the sample data may be labeled and distributed to obtain a training set, a verification set and a test set, the training set may be subjected to online data enhancement operation to obtain a training set to be input, the test set and the verification set may be subjected to TTA operation respectively to obtain a test set to be input and a verification set to be input correspondingly, and then the deformable convolution defect detection model may be trained according to the training set to be input, the test set to be input and the verification set to be input. Wherein, the sample data can be labeled and distributed according to the proportion of 7:2:1, so as to obtain a 70% training set, a 20% verification set and a 10% test set.
More specifically, the sample data may be labeled and allocated to a specific data set format, for example, a CoCo or VOC format, and then the training set may be sequentially rotated, vertically flipped, horizontally mirrored, changed in brightness, center clipped, gaussian noise, and randomly erased to obtain a training set to be input, where the occurrence probabilities of the rotation, vertical flipping, horizontal mirroring, changed in brightness, center clipping, gaussian noise, and random erasure are all 0.5, the rotation angle of the rotation operation may be set to a random value in the middle of 0-360 degrees, and the factor of the brightness operation is changed, that is, the factor may be set to 0.5-1.5. Therefore, the influence of high-frequency components in the sample data on model training can be effectively reduced through Gaussian noise operation, the global capability of feature learning can be effectively improved through random erasing operation, and the over-fitting problem caused by local feature over-learning can be avoided.
In addition, TTA (Test-Time Augmentation) operations, such as multi-scale transform (mle) and flip (flip) operations, may be performed on the verification set and the Test set, respectively, wherein the multi-scale operation may adjust the short side sizes of the data of the verification set and the Test set (the aspect ratio of the data is fixed), such as adjusting the data to [300, 400, 500, 600, 700, 800, 900, 1000, 1100, 1200], and the flip operation may include left-flip (left-flip) and right-flip (right-flip), so that each input of the data of the verification set and the Test set may be subjected to augmented transformation, one input may be changed into a plurality of inputs, and then the input may be fused and input into the trained deformable convolution defect detection model for verification and testing.
In a specific embodiment of the present invention, a training set to be input may be used to train a deformable convolution defect detection model, and an mPA index may be calculated on a verification set to be input for the deformable convolution defect detection model obtained by training, where mPA is a ratio between the number of pixels in each category that are correctly classified and the number of all pixels in the category, and then is averaged, where mPA has a formula:
Figure 353952DEST_PATH_IMAGE003
wherein k is the total defect category number of the image segmentation task,
Figure 849043DEST_PATH_IMAGE004
indicating that the i-class defect class is predicted as i-class,
Figure 268830DEST_PATH_IMAGE005
indicating that the i-type defect class is predicted as the j-type. If mPA is greater than the value required for actual production, model training is complete.
And S5, adopting the trained deformable convolution defect detection model to detect the defects of the product to be detected.
Specifically, the deformable convolution defect detection model obtained in the above steps and used for completing the input training of the test set to be input can be input, and then a Mask, a Class Label and a Boundary Box corresponding to the test set can be obtained, so that the defect detection of the product to be detected can be realized.
According to the product defect detection method provided by the embodiment of the invention, the image data of the product to be detected is acquired, the image data is processed in a balanced manner to obtain sample data, the deformable convolution defect detection model is trained according to the sample data, and finally the defect detection is carried out on the product to be detected according to the trained deformable convolution defect detection model, so that the balance of the sample data can be ensured, the over-training of the deformable convolution defect detection model can be prevented, in addition, the generalization capability and the robustness of the deformable convolution defect detection model can be enhanced, and the cross-platform performance of the deformable convolution defect detection model can be further improved.
Corresponding to the embodiment, the invention further provides a product defect detection device.
As shown in fig. 10, the product defect detecting apparatus according to the embodiment of the present invention includes a modeling module 10, an obtaining module 20, an equalization processing module 30, a training module 40, and a detecting module 50. The modeling module 10 is used for constructing a deformable convolution defect detection model; the acquisition module 20 is used for acquiring image data of a product to be detected; the equalization processing module 30 is configured to perform equalization processing on the image data to obtain sample data; the training module 40 is used for training the deformable convolution defect detection model according to the sample data; the detection module 50 is configured to perform defect detection on a product to be detected by using the trained deformable convolution defect detection model.
In an embodiment of the invention, as shown in fig. 2, the deformable convolution defect detection model may include a first detection module BlockI, a second detection module BlockII, a third detection module BlockIII, and a fourth detection module BlockIV, where the second detection module BlockII is respectively connected to the first detection module BlockI and the third detection module BlockIII, and the second detection module BlockII and the third detection module BlockIII are connected to the fourth detection module BlockIV through an ROI Align network.
Specifically, as shown in fig. 2, the first detection module BlockI may be a residual network, and specifically may include a first gradient stage1, a second gradient stage2, a third gradient stage3, and a fourth gradient stage4, and each of the first gradient stage1, the second gradient stage2, the third gradient stage3, and the fourth gradient stage4 is composed of a stacked residual network; the second detection module block ii may be an FPN network (Feature Pyramid network); the third detection module blockack iii may be an RPN Network (Region forward Network); the fourth detection module BlockIV may be a Head network.
It should be noted that the stacked residual network may include a first type stacked residual network and a second type stacked residual network, where the first type stacked residual network may include a DCL (Deformable convergence Layer) network, as specifically shown in fig. 3 (a); the second type of stacked residual network does not include a DCL network, as shown in fig. 3 (b). Specifically, the first type of stacked residual network, i.e., the stacked residual network including DCLs, is a network including DCBRs, for example, including 3 × 3 DCBRs; the second type of stacked residual network, i.e. the stacked residual network not containing DCL, is a network containing CBR, e.g. containing 3 × 3CBR, wherein, as shown in fig. 3(c), DCBR is composed of Dconv + BN + ReLu, and Dconv is a variable convolution, and CBR is composed of Conv + BN + ReLu. Further, as shown in fig. 3(c), the CBs in the first-type and second-type laminated residual error networks may be composed of Conv + BN. Where Conv is the convolution layer, BN is BatchNorm, i.e., batch normalization, and ReLu is the activation function.
As can be further seen from fig. 3(a) and fig. 3(b), each of the first and second stacked residual networks includes four mapping channels, i.e., X1- > Y1, X2- > Y2, X3- > Y3, and X4- > Y4 mapping channels, and since the DCBR or CBR is stacked, the mapping relationships of the four mapping channels are as follows:
Figure 799081DEST_PATH_IMAGE006
where i denotes the number of channels, for example 1, 2, 3, 4,srepresents the maximum number of mapping channels (here 4), KiAnd the output mapping relation of the input image passing through the 3 × 3DCBR or the 3 × 3CBR is shown.
Further referring to fig. 3(b), 3 × 3CBR networks are respectively disposed in the mapping channels X2- > Y2, X3- > Y3, and X4- > Y4, wherein the output terminal of the 3 × 3CBR network in the mapping channel X2- > Y2 is further connected to the input terminal of the 3 × 3CBR network in the mapping channel X3- > Y3, and the output terminal of the 3 × 3CBR network in the mapping channel X3- > Y3 is further connected to the input terminal of the 3 × 3CBR network in the mapping channel X4- > Y4. Furthermore, as shown in fig. 3(b), the input terminals of the mapping channels X1- > Y1, X2- > Y2, X3- > Y3, and X4- > Y4 are connected to the output terminals of the 1 × 1CBR network, and 1 × 1CB networks are further provided at the output terminals of the mapping channels X1- > Y1, X2- > Y2, X3- > Y3, and X4- > Y4, and the output terminals of the 1 × 1CB networks are further connected to the input terminals of the 1 × 1CBR network, and the output terminals of the 1 × 1CB networks are further connected to the ReLu network.
As can be further seen from fig. 3(a) and 3(b), in the first type of stacked residual error network, a 3 × 3DCBR network may be used to replace a 3 × 3CBR network in the second type of stacked residual error network. Therefore, the method can adapt to model training of irregular defects, and particularly can adapt to detection of geometric deformation defects of different targets by dynamically adjusting the size of the DCL convolution kernel. In addition, by setting the stacked residual error network, various characteristic receptive fields can be obtained in a single basic unit, and the mode of blocking and re-fusing is favorable for extracting effective information from the context.
In one embodiment of the invention, as shown in FIG. 4, the first gradient stage1 may include three second-type stacked residual networks; the second gradient stage2 may include four stacked residual networks of the first type; the third gradient stage3 may include six stacked residual networks of the first type; the fourth gradient stage4 may include three stacked residual networks of the first type.
As further shown in fig. 2 and fig. 4, the first detection module block i may further include a MaxPool network and a 7 × 7CBR network, wherein one end of the 7 × 7CBR network is used for receiving Input data, that is, the other end of the 7 × 7CBR network is connected to one end of the MaxPool network, and the other end of the MaxPool network is connected to the first gradient stage 1. Wherein, the MaxPool network is a maximum pooling network.
In an embodiment of the present invention, as shown in fig. 2, the second detection module BlockII, i.e., the FPN network, may have four 1 × 1 const, and the four 1 × 1 const may be connected to the first gradient stage1, the second gradient stage2, the third gradient stage3, and the fourth gradient stage4 in a one-to-one correspondence, and the output of the fourth gradient stage4 may obtain P5 after passing through 1 × 1 const, the output of the third gradient stage3 may obtain P4 after passing through 1 × 1 const and fusing with P5, the output of the second gradient stage2 may obtain P3 after passing through 1 × 1 const and fusing with P4, and the output of the first gradient stage1 may obtain P2 after passing through 1 × 1 const and fusing with P3. Further, as shown in FIG. 2, P2, P3, P4 and P5 obtained in the above steps can also obtain RPN/P2, RPN/P3, RPN/P4, RPN/P5 and RPN/P6 through a sliding window operation.
In an embodiment of the invention, as shown in fig. 2, the third detection module BlockIII, i.e., the RPN network, may process RPN/P2, RPN/P3, RPN/P4, RPN/P5, and RPN/P6 layer by layer, so as to obtain the propusals (candidate box).
Specifically, as shown in fig. 5, the third detection module block iii, i.e., the RPN network, may be provided with 3 × 3Conv and two 1 × 1Conv, and the inputs of the 3 × 3Conv may be used to receive RPN/P2, RPN/P3, RPN/P4, RPN/P5, and RPN/P6, the outputs of the 3 × 3Conv may be respectively connected to the inputs of the two 1 × 1Conv, and the outputs of the two 1 × 1Conv may be output to provide propusals.
In one embodiment of the present invention, as shown in fig. 2, the fourth detection module, block iv, i.e., the Head network, may be provided with an FCN and a dual FC, and inputs of the FCN and the dual FC may receive prosassals through ROI Align, and RPN/P2, RPN/P3, RPN/P4, and RPN/P5, wherein the FCN may output a Mask, and the dual FC may output a Class Label and a Boundary Box, wherein the Mask is used to represent segmentation of each defect pixel level, the Class Label is used to represent classification of each defect, and the Boundary Box is used to represent a location of each defect.
In an embodiment of the present invention, the product to be detected may include a plurality of project platform products, for example, may include an a project platform product, a B project platform product, and a C project platform product, and the primary color of each project platform product is different, for example, the primary color of the a project platform product may be red, the primary color of the B project platform product may be black, and the primary color of the C project platform product may be white; in addition, the a project platform product, the B project platform product and the C project platform product may share one production line and one quality inspection line, so that when the image data of the product to be detected is acquired by the acquisition module 20, for example, an industrial camera, the image data of the a project platform product, the B project platform product and the C project platform product may be obtained.
It should be noted that, the image data of the a project platform product, the B project platform product and the C project platform product obtained by the obtaining module 20 are different in amount due to different quantities of each project platform product, for example, if the quantity of the a project platform product is the largest, and the sum of the quantity of the B project platform product and the quantity of the C project platform product is the smallest, the image data of each project platform product may be different, and further, the defect samples of each project platform product may be different, so that the defect samples of different project platforms are unbalanced.
It should be noted that, the image data of the item platform product a, the item platform product B and the item platform product C obtained by the obtaining module 20 are different in defect samples of different defect types in each item platform product, for example, if there are 7 defect types in the item platform product a, namely, defect type 1, defect type 2, defect type 3, defect type 4, defect type 5, defect type 6 and defect type 7, and the proportion of the 7 defect types in the item platform product image data a is 40%, 25%, 15%, 10%, 5%, 3% and 2%, respectively, it can be seen that the defect samples of different types are in an unbalanced state, and if the defect detecting module is trained by using the defect sample, the characteristics of the defect type 1 and the defect type 2 are over-learned, and the defect type 5 and the defect type 6 are over-learned, The feature learning of defect class 7 is weak.
Therefore, the balance processing module 30 may specifically adopt a CycleGAN network to balance the image data acquired in step S2 to obtain sample data, and may implement mapping between two domains, instead of mapping between two specific image data, through the CycleGAN network, thereby implementing mapping between different project platform products, and thus may balance the sample data of each project platform product. More specifically, taking defect type 3 as an example, the CycleGAN network equalization processing operations shown in fig. 6(a) and 6(B) can be performed separately, and for example, defect type 3 of the a project platform product, i.e., input 3-a (input data 3-a), and defect type 3 of the B project platform product, i.e., input 3-B (input data 3-B), can be input as inputs to the CycleGAN network.
As shown in fig. 6(a), the input defect type 3 of the a project platform product, i.e. input 3-a, may first pass through the generator network G, i.e. the image data of the a project platform product to the generator network G of the image data of the B project platform product, to obtain Generated-B (sample data B) of the B project platform product. Further, as shown in fig. 6(a), the Generated-B of the B project platform product obtained by the generator network G needs to be determined whether it meets the sample data standard of the B project platform product by the Discriminator B (Discriminator B), and if so, the determination is score 1, and if not, the determination is score 0.
As shown in fig. 6(B), the input defect type 3 of the B project platform product, i.e., input 3-B, may first pass through the generator network F, i.e., the B project platform product image data, to the generator network F of the a project platform product image data, to obtain Generated-a (sample data a) of the a project platform product. Further, as shown in fig. 3(b), the Generated-a of the a project platform product obtained by the generator network F needs to be judged whether it meets the sample data standard of the a project platform product by the Discriminator a (Discriminator a), and if so, the Decision is score 1, and if not, the Decision is score 0.
It should be noted that the CycleGAN network adopted in the above embodiment includes a generator structure as shown in fig. 7, including three CBR (Conv 2D-BatchNorm-Relu) networks, nine Residual Block networks, two CTBR (Conv 2D transit-BatchNorm-Relu) networks, and a Conv2D network and a Tanh network, and specific connection relationships thereof may refer to fig. 7; the structure of the arbiter included in the CycleGAN network used in the above embodiment is shown in fig. 8, and includes a Conv2D network and three Conv2D-InstanceNorm-leak networks-The ReLu network, the specific connection relationship of which can be referred to fig. 8. Wherein, Residual Block is a Residual module, Conv2D Transpose is a transposed convolution, Conv2D is a two-dimensional convolution, Tanh is a hyperbolic function, InstanceNorm is a normalization layer, Leaky-ReLu is a nonlinear activation function.
It should be further noted that the CycleGAN network used in the above embodiments also introduces resistance loss and cycle consistency loss, wherein the resistance loss is the difference between G (input 3-a) and Generated-B, or the difference between F (input 3-B) and Generated-a; the cyclic-consistency loss is specifically shown in fig. 9(a) and fig. 9(b), wherein fig. 9(a) is a forward cyclic consistency loss, the corresponding process is x — > g (x) — > F (g (x)) approximately ≈ x, and the loss is F (g (x)) x; fig. 9(b) shows backward loop consistency loss, y — > f (y) — > G (f (y)) approximately equal to y, and the loss is G (f (y)) y, which is more directly understood as: for each image x in the x domain, x can be brought back to the original image in an image translation cycle; for each image y in the y domain, y can be brought back to the original image in an image translation cycle. It should be noted that cycle3-a (cycle data 3-a) and cycle3-B (cycle data 3-B) generated in fig. 6(a) and 6(B) can be used for the calculation of the forward cycle consistency loss shown in fig. 9 (a).
Training overfitting of the generator network F and the generator network G can be effectively avoided by introducing cycle consistency loss, and therefore required sample data is obtained.
It should be noted that, for the input defect type 3 of the a project platform product, i.e. input 3-a, and the input defect type 3 of the B project platform product, i.e. input 3-B, the selection principle shown in table 1 can also be set:
TABLE 1
Figure 65721DEST_PATH_IMAGE007
Based on the above process, the project platform product data A can be migrated to the project platform product B, so that the project platform product data B can be increased, and data migration between other project platform products can be performed as well, which is not described in detail herein; in addition, data migration of different defect types can be performed between the same project platform product, so that the sample data imbalance can be improved. Certainly, data migration can also be performed between the project platform product A and the project platform product C, or between the project platform product B and the project platform product C; in addition, data migration may also be performed for other defect types 4 to 7, and specific steps may refer to the above embodiments, which are not described in detail here.
Through balanced processing image data, the unbalance phenomenon between different defect sample data can be improved, so that the over-training of the deformable convolution defect detection model can be prevented, and the defects between different project platform products can be trained in a coordinated manner, so that the generalization capability and robustness of the deformable convolution defect detection model can be enhanced, and the cross-platform performance of the deformable convolution defect detection model can be improved.
In an embodiment of the present invention, the training module 40 may be specifically configured to perform label assignment on sample data to obtain a training set, a verification set, and a test set, perform an online data enhancement operation on the training set to obtain a training set to be input, perform a TTA operation on the test set and the verification set to obtain a test set to be input and a verification set to be input, and then train the deformable defect detection model according to the training set to be input, the test set to be input, and the verification set to be input. Wherein, the sample data can be labeled and distributed according to the proportion of 7:2:1, so as to obtain a 70% training set, a 20% verification set and a 10% test set.
More specifically, the sample data may be labeled and distributed to obtain a training set, a verification set, and a test set, for example, the sample data may be labeled and distributed to a specific data set format, such as a CoCo or VOC format, and then the training set may be sequentially subjected to rotation, vertical flipping, horizontal mirroring, brightness change, center clipping, gaussian noise, and random erasure operation to obtain a training set to be input, where occurrence probabilities of the rotation, vertical flipping, horizontal mirroring, brightness change, center clipping, gaussian noise, and random erasure are all 0.5, a rotation angle of the rotation operation may be set to a random value in the middle of 0 to 360 degrees, and a factor of the brightness operation is changed, that is, the factor may be set to 0.5 to 1.5. Therefore, the influence of high-frequency components in the sample data on model training can be effectively reduced through Gaussian noise operation, the global capability of feature learning can be effectively improved through random erasing operation, and the over-fitting problem caused by local feature over-learning can be avoided.
Furthermore, the verification set and the Test set may be subjected to TTA (Test-Time Augmentation) operations, such as multi-scale transform (mle) and flip (flip) operations, respectively, wherein the multi-scale operation may adjust the short side sizes of the data of the verification set and the Test set (the aspect ratio of the data is fixed), such as adjusting the data to [300, 400, 500, 600, 700, 800, 900, 1000, 1100, 1200], and the flip operation may include left-flip (left-flip) and right-flip (right-flip), so that each input of the data of the verification set and the Test set may be subjected to augmented transformation, one input may be changed into multiple inputs, and then the input may be fused and input into the trained deformable convolution defect detection model for verification and testing.
In a specific embodiment of the present invention, the training module 40 may specifically adopt a training set to be input to train the deformable convolution defect detection model, and may calculate the mPA index of the trained deformable convolution defect detection model on a verification set to be input, where mPA is a ratio of the number of pixels in each category that is calculated to be correct and the number of all pixels in the category, and then averages, where mPA has a formula:
Figure 540169DEST_PATH_IMAGE008
wherein k is the total defect category number of the image segmentation task,
Figure 711169DEST_PATH_IMAGE009
indicating that the i-class defect class is predicted as i-class,
Figure 163884DEST_PATH_IMAGE010
indicating that the i-type defect class is predicted as the j-type. If mPA is greater than the value required for actual production, model training is complete.
In an embodiment of the present invention, the detection module 50 may be specifically configured to input the obtained test set to be input into the trained deformable convolution defect detection model, and further obtain a Mask, a Class Label and a Boundary Box corresponding to the test set, so that defect detection of a product to be detected can be achieved.
According to the product defect detection device provided by the embodiment of the invention, the image data of the product to be detected is acquired through the acquisition module, the image data is processed through the equalization processing module in an equalization mode to obtain sample data, the deformable convolution defect detection model is trained through the training module according to the sample data, and finally the defect detection is carried out on the product to be detected through the detection module by adopting the trained deformable convolution defect detection model.
The invention further provides a computer device corresponding to the embodiment.
The computer device of the embodiment of the invention comprises a memory, a processor and a computer program which is stored on the memory and can run on the processor, and when the processor executes the program, the product defect detection method of the embodiment is realized.
According to the computer equipment provided by the embodiment of the invention, the sample data can be ensured to be balanced, so that the over-training of the deformable convolution defect detection model can be prevented, in addition, the generalization capability and robustness of the deformable convolution defect detection model can be enhanced, and the cross-platform performance of the deformable convolution defect detection model can be further improved.
The invention also provides a non-transitory computer readable storage medium corresponding to the above embodiment.
The non-transitory computer-readable storage medium of the embodiment of the present invention stores thereon a computer program that, when executed by a processor, implements the product defect detection method of the above-described embodiment.
According to the non-transitory computer-readable storage medium provided by the embodiment of the invention, the balance of sample data can be ensured, so that the over-training of the deformable convolution defect detection model can be prevented, and in addition, the generalization capability and robustness of the deformable convolution defect detection model can be enhanced, and the cross-platform performance of the deformable convolution defect detection model can be further improved.
In the description of the present invention, the terms "first" and "second" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implying any number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. The meaning of "plurality" is two or more unless specifically limited otherwise.
In the present invention, unless otherwise expressly stated or limited, the terms "mounted," "connected," "secured," and the like are to be construed broadly and can, for example, be fixedly connected, detachably connected, or integrally formed; can be mechanically or electrically connected; either directly or indirectly through intervening media, either internally or in any other relationship. The specific meanings of the above terms in the present invention can be understood by those skilled in the art according to specific situations.
In the present invention, unless otherwise expressly stated or limited, the first feature "on" or "under" the second feature may be directly contacting the first and second features or indirectly contacting the first and second features through an intermediate. Also, a first feature "on," "over," and "above" a second feature may be directly or diagonally above the second feature, or may simply indicate that the first feature is at a higher level than the second feature. A first feature being "under," "below," and "beneath" a second feature may be directly under or obliquely under the first feature, or may simply mean that the first feature is at a lesser elevation than the second feature.
In the description herein, references to the description of the term "one embodiment," "some embodiments," "an example," "a specific example," or "some examples," etc., mean that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the invention. In this specification, the schematic representations of the terms used above are not necessarily intended to refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Furthermore, various embodiments or examples and features of different embodiments or examples described in this specification can be combined and combined by one skilled in the art without contradiction.
Any process or method descriptions in flow charts or otherwise described herein may be understood as representing modules, segments, or portions of code which include one or more executable instructions for implementing specific logical functions or steps of the process, and alternate implementations are included within the scope of the preferred embodiment of the present invention in which functions may be executed out of order from that shown or discussed, including substantially concurrently or in reverse order, depending on the functionality involved, as would be understood by those reasonably skilled in the art of the present invention.
The logic and/or steps represented in the flowcharts or otherwise described herein, e.g., an ordered listing of executable instructions that can be considered to implement logical functions, can be embodied in any computer-readable medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, processor-containing system, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions. For the purposes of this description, a "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of the computer-readable medium would include the following: an electrical connection (electronic device) having one or more wires, a portable computer diskette (magnetic device), a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disc read-only memory (CDROM). Additionally, the computer-readable medium could even be paper or another suitable medium upon which the program is printed, as the program can be electronically captured, via for instance optical scanning of the paper or other medium, then compiled, interpreted or otherwise processed in a suitable manner if necessary, and then stored in a computer memory.
It should be understood that portions of the present invention may be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, the various steps or methods may be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, any one or combination of the following techniques, which are known in the art, may be used: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application specific integrated circuit having an appropriate combinational logic gate circuit, a Programmable Gate Array (PGA), a Field Programmable Gate Array (FPGA), or the like.
It will be understood by those skilled in the art that all or part of the steps carried by the method for implementing the above embodiments may be implemented by hardware related to instructions of a program, which may be stored in a computer readable storage medium, and when the program is executed, the program includes one or a combination of the steps of the method embodiments.
In addition, functional units in the embodiments of the present invention may be integrated into one processing module, or each unit may exist alone physically, or two or more units are integrated into one module. The integrated module can be realized in a hardware mode, and can also be realized in a software functional module mode. The integrated module, if implemented in the form of a software functional module and sold or used as a stand-alone product, may also be stored in a computer readable storage medium.
The storage medium mentioned above may be a read-only memory, a magnetic or optical disk, etc. Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention, and that variations, modifications, substitutions and alterations can be made to the above embodiments by those of ordinary skill in the art within the scope of the present invention.

Claims (10)

1. A product defect detection method is characterized by comprising the following steps:
constructing a deformable convolution defect detection model;
acquiring image data of a product to be detected;
the image data is processed in an equalizing mode to obtain sample data;
training the deformable convolution defect detection model according to the sample data;
and carrying out defect detection on the product to be detected by adopting the trained deformable convolution defect detection model.
2. The product defect detection method of claim 1, wherein the deformable convolution defect detection model comprises a first detection module, a second detection module, a third detection module and a fourth detection module, the second detection module is connected with the first detection module and the third detection module respectively, the second detection module and the third detection module are connected with the fourth detection module through a ROI Align network, wherein,
the first detection module is a residual error network, and specifically comprises a first gradient, a second gradient, a third gradient and a fourth gradient, and the first gradient, the second gradient, the third gradient and the fourth gradient are all composed of stacked residual error networks;
the second detection module is an FPN network;
the third detection module is an RPN network;
the fourth detection module is a Head network.
3. The method according to claim 2, wherein the stacked residual networks include a first type stacked residual network and a second type stacked residual network, wherein the first type stacked residual network includes a DCL network, and the second type stacked residual network does not include a DCL network.
4. The product defect detecting method according to claim 3,
the first gradient comprises three of the second-type stacked residual networks;
the second gradient comprises four of the first type of stacked residual networks;
the third gradient comprises six stacked residual networks of the first type;
the fourth gradient comprises three stacked residual networks of the first type.
5. The product defect detection method of claim 4, wherein the products to be detected comprise a plurality of project platform products, and the image data comprises image data of the plurality of project platform products.
6. The method according to claim 5, wherein the step of performing equalization processing on the image data to obtain sample data comprises the following steps:
and carrying out balanced processing on the image data by adopting a cycleGAN network to obtain sample data.
7. The method of claim 6, wherein the training of the deformable convolution defect detection model according to the sample data comprises the following steps:
performing label distribution on the sample data to obtain a training set, a verification set and a test set;
performing on-line data enhancement operation on the training set to obtain a training set to be input;
TTA operation is respectively carried out on the test set and the verification set so as to correspondingly obtain a test set to be input and a verification set to be input;
and training the deformable convolution defect detection model according to the training set to be input, the test set to be input and the verification set to be input.
8. A product defect detecting apparatus, comprising:
a modeling module for constructing a deformable convolution defect detection model;
the acquisition module is used for acquiring image data of a product to be detected;
the equalization processing module is used for performing equalization processing on the image data to obtain sample data;
a training module for training the deformable convolution defect detection model according to the sample data;
and the detection module is used for carrying out defect detection on the product to be detected by adopting the trained deformable convolution defect detection model.
9. Computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor, when executing the computer program, implements the method for product defect detection according to any of claims 1-7.
10. A non-transitory computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, implements a product defect detection method according to any one of claims 1 to 7.
CN202111488051.XA 2021-12-08 2021-12-08 Product defect detection method and device Active CN113888542B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202111488051.XA CN113888542B (en) 2021-12-08 2021-12-08 Product defect detection method and device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202111488051.XA CN113888542B (en) 2021-12-08 2021-12-08 Product defect detection method and device

Publications (2)

Publication Number Publication Date
CN113888542A true CN113888542A (en) 2022-01-04
CN113888542B CN113888542B (en) 2022-03-22

Family

ID=79016529

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202111488051.XA Active CN113888542B (en) 2021-12-08 2021-12-08 Product defect detection method and device

Country Status (1)

Country Link
CN (1) CN113888542B (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115797349A (en) * 2023-02-07 2023-03-14 广东奥普特科技股份有限公司 Defect detection method, device and equipment
CN115810011A (en) * 2023-02-07 2023-03-17 广东奥普特科技股份有限公司 Training method, device and equipment for anomaly detection network and anomaly detection method, device and equipment

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2020007095A1 (en) * 2018-07-02 2020-01-09 北京百度网讯科技有限公司 Display screen quality inspection method and apparatus, electronic device, and storage medium
CN112070746A (en) * 2020-09-09 2020-12-11 深兰人工智能芯片研究院（江苏）有限公司 Steel strip defect detection method and device
CN112115972A (en) * 2020-08-14 2020-12-22 河南大学 Depth separable convolution hyperspectral image classification method based on residual connection
CN112581430A (en) * 2020-12-03 2021-03-30 厦门大学 Deep learning-based aeroengine nondestructive testing method, device, equipment and storage medium
CN113034476A (en) * 2021-03-30 2021-06-25 广东工业大学 Leather flaw detection method and system, storage medium and computer equipment
CN113496480A (en) * 2021-05-13 2021-10-12 西安数合信息科技有限公司 Method for detecting weld image defects
CN113674247A (en) * 2021-08-23 2021-11-19 河北工业大学 X-ray weld defect detection method based on convolutional neural network
CN113763384A (en) * 2021-11-10 2021-12-07 常州微亿智造科技有限公司 Defect detection method and defect detection device in industrial quality inspection

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2020007095A1 (en) * 2018-07-02 2020-01-09 北京百度网讯科技有限公司 Display screen quality inspection method and apparatus, electronic device, and storage medium
CN112115972A (en) * 2020-08-14 2020-12-22 河南大学 Depth separable convolution hyperspectral image classification method based on residual connection
CN112070746A (en) * 2020-09-09 2020-12-11 深兰人工智能芯片研究院（江苏）有限公司 Steel strip defect detection method and device
CN112581430A (en) * 2020-12-03 2021-03-30 厦门大学 Deep learning-based aeroengine nondestructive testing method, device, equipment and storage medium
CN113034476A (en) * 2021-03-30 2021-06-25 广东工业大学 Leather flaw detection method and system, storage medium and computer equipment
CN113496480A (en) * 2021-05-13 2021-10-12 西安数合信息科技有限公司 Method for detecting weld image defects
CN113674247A (en) * 2021-08-23 2021-11-19 河北工业大学 X-ray weld defect detection method based on convolutional neural network
CN113763384A (en) * 2021-11-10 2021-12-07 常州微亿智造科技有限公司 Defect detection method and defect detection device in industrial quality inspection

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
朱超平等: "基于改进的Faster-RCNN模型的汽车轮毂表面缺陷在线检测算法研究", 《表面技术》 *
潘晓霞: "《虚拟现实与人工智能技术的综合应用》", 30 December 2018 *
雷军明: "基于改进U-Net的视网膜血管图像分割", 《信息技术与信息化》 *

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115797349A (en) * 2023-02-07 2023-03-14 广东奥普特科技股份有限公司 Defect detection method, device and equipment
CN115810011A (en) * 2023-02-07 2023-03-17 广东奥普特科技股份有限公司 Training method, device and equipment for anomaly detection network and anomaly detection method, device and equipment
CN115797349B (en) * 2023-02-07 2023-07-07 广东奥普特科技股份有限公司 Defect detection method, device and equipment

Also Published As

Publication number Publication date
CN113888542B (en) 2022-03-22

Similar Documents

Publication Publication Date Title
CN113888542B (en) Product defect detection method and device
CN107123117B (en) IC pin welding spot quality detection method and device based on deep learning
CN110060237B (en) Fault detection method, device, equipment and system
CN109934810B (en) Defect classification method based on improved particle swarm wavelet neural network
CN107274419A (en) A kind of deep learning conspicuousness detection method based on global priori and local context
CN111126433A (en) Positive and negative sample data balancing method in factory PCB defect detection
CN111275660A (en) Defect detection method and device for flat panel display
CN104202583A (en) Image processing device and method
CN113240673B (en) Defect detection method, defect detection device, electronic equipment and storage medium
CN109740606A (en) A kind of image-recognizing method and device
CN114021704B (en) AI neural network model training method and related device
CN110458789A (en) A kind of image definition evaluating method, device and electronic equipment
CN109376782B (en) Support vector machine cataract classification method and device based on eye image features
CN114820541A (en) Defect detection method based on reconstructed network
CN109409305A (en) A kind of facial image clarity evaluation method and device
CN109801224A (en) A kind of image processing method, device, server and storage medium
CN111598807B (en) Automobile part detection data sharing system and method based on block chain
CN112818774A (en) Living body detection method and device
CN106530286A (en) Method and device for determining definition level
CN110310341A (en) The generation method of default parameters, device, equipment and storage medium in color algorithm
CN105354833A (en) Shadow detection method and apparatus
CN115546141A (en) Small sample Mini LED defect detection method and system based on multi-dimensional measurement
CN110263842A (en) For the neural network training method of target detection, device, equipment, medium
CN109191440A (en) Glass blister detection and method of counting
CN112560929B (en) Oil spilling area determining method and device and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant