CN111415353A - Detection structure and detection method for fastener burr defects based on ResNet58 network - Google Patents
Detection structure and detection method for fastener burr defects based on ResNet58 network Download PDFInfo
- Publication number
- CN111415353A CN111415353A CN202010277370.5A CN202010277370A CN111415353A CN 111415353 A CN111415353 A CN 111415353A CN 202010277370 A CN202010277370 A CN 202010277370A CN 111415353 A CN111415353 A CN 111415353A
- Authority
- CN
- China
- Prior art keywords
- image
- layer
- network
- module
- fastener
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000007547 defect Effects 0.000 title claims abstract description 63
- 238000001514 detection method Methods 0.000 title claims abstract description 60
- 238000012549 training Methods 0.000 claims abstract description 55
- 238000000034 method Methods 0.000 claims abstract description 29
- 238000012360 testing method Methods 0.000 claims abstract description 22
- 238000013528 artificial neural network Methods 0.000 claims abstract description 17
- 238000007781 pre-processing Methods 0.000 claims abstract description 15
- 238000000605 extraction Methods 0.000 claims abstract description 8
- 230000006870 function Effects 0.000 claims description 39
- 238000010606 normalization Methods 0.000 claims description 23
- 238000011176 pooling Methods 0.000 claims description 14
- 230000008569 process Effects 0.000 claims description 11
- 238000013519 translation Methods 0.000 claims description 11
- 238000001914 filtration Methods 0.000 claims description 9
- 230000002159 abnormal effect Effects 0.000 claims description 8
- 238000012545 processing Methods 0.000 claims description 8
- 230000004913 activation Effects 0.000 claims description 7
- 238000002372 labelling Methods 0.000 claims description 6
- 238000005457 optimization Methods 0.000 claims description 5
- 230000003044 adaptive effect Effects 0.000 claims description 3
- 238000004422 calculation algorithm Methods 0.000 claims description 3
- 230000008602 contraction Effects 0.000 claims description 3
- 230000009467 reduction Effects 0.000 claims description 3
- 238000005070 sampling Methods 0.000 claims description 3
- 230000009466 transformation Effects 0.000 claims description 3
- 230000005428 wave function Effects 0.000 claims description 3
- GNFTZDOKVXKIBK-UHFFFAOYSA-N 3-(2-methoxyethoxy)benzohydrazide Chemical compound COCCOC1=CC=CC(C(=O)NN)=C1 GNFTZDOKVXKIBK-UHFFFAOYSA-N 0.000 claims description 2
- 208000037170 Delayed Emergence from Anesthesia Diseases 0.000 claims description 2
- 238000011161 development Methods 0.000 abstract description 4
- 238000012795 verification Methods 0.000 abstract description 2
- 238000011068 loading method Methods 0.000 description 10
- 238000001994 activation Methods 0.000 description 6
- 238000009795 derivation Methods 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 238000004519 manufacturing process Methods 0.000 description 3
- 238000004364 calculation method Methods 0.000 description 2
- 238000009826 distribution Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000003062 neural network model Methods 0.000 description 2
- 102000008579 Transposases Human genes 0.000 description 1
- 108010020764 Transposases Proteins 0.000 description 1
- 230000001133 acceleration Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 238000009776 industrial production Methods 0.000 description 1
- 230000009191 jumping Effects 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0004—Industrial image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20024—Filtering details
- G06T2207/20032—Median filtering
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
Landscapes
- Engineering & Computer Science (AREA)
- Quality & Reliability (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Image Analysis (AREA)
- Image Processing (AREA)
Abstract
The invention discloses a detection structure and a detection method for fastener burr defects based on a ResNet58 network, which comprises an input module, a training module and an output module, wherein the input module, the training module and the output module are used for manually marking images of fastener products in a data source so as to obtain class information about qualified products and the burr defects; integrating the fastener product image and the marking information thereof to form a complete data set; performing image preprocessing and image enhancement on the data set; dividing the fastener product image enhanced by the data in the step 3 to obtain a training set, a testing set and a verification set; inputting data into a constructed network for training to obtain a burr defect detection model; and inputting the image of the fastener product to be detected into the burr defect detection model for detection to obtain a burr defect detection result. According to the method, the residual neural network of ResNet58 is used as a basic network for feature extraction, and the pyroch is used as a development frame, so that the accuracy of burr defect detection and identification is improved.
Description
Technical Field
The invention relates to the field of computer vision and deep learning, in particular to a structure and a method for detecting fastener burr defects based on a ResNet58 network.
Background
At present, science and technology develops more and more fast, and more industrial manufacturers have proposed higher requirement to production efficiency when pursuing the quality, and product quality and production efficiency hold the development of producer, and industrial product detects then to guarantee product quality, reflects the important link of production technology problem.
Currently, there are two main ways of detecting product defects commonly adopted in the industry: (1) manual detection; (2) machine intelligent detection method. Among them, there are many defects in manual detection, such as low efficiency, high cost, easy fatigue caused by manual work, etc.; in mass industrial production, the intelligent industrial defect detection method is superior to manual detection in the aspects of accuracy, speed, cost and the like, so that the adoption of an intelligent detection means based on machine vision to replace a manual method is an inevitable trend of industrial quality detection.
However, the intelligent detection method for the machine used in the industry at present has the following defects: due to the fact that imaging quality levels of the equipment are not uniform, a series of industrial products cannot be detected in a standard mode, the problems that images are not clear, key parts are not shot in place and the like are encountered, and therefore the intelligent industrial defect detection system brings about small challenges. In conclusion, the existing industrial defect detection method has the problems of low detection efficiency and low accuracy.
Disclosure of Invention
The invention aims to provide a detection structure and a detection method for fastener burr defects based on a ResNet58 network, wherein a residual neural network of the ResNet58 is used as a basic network for feature extraction, and a pyrroch is used as a development frame, so that the accuracy of burr defect detection and identification is improved.
In order to realize the purpose of the invention, the invention adopts the following technical scheme: a detection structure for fastener burr defects based on a ResNet58 network comprises an input module, a training module and an output module, wherein the input module and the output module are respectively connected with the training module, the training module comprises a plurality of convolution modules which are sequentially connected, each convolution module comprises a residual error module and a short connecting layer, and the convolution modules and the residual error modules are connected through the short connecting layers;
the input module is used for: the image processing device is used for carrying out normalization preprocessing and image enhancement on the image;
the convolution module: the system is used for performing convolution and pooling on sample data for multiple times to realize down-sampling and obtain multi-dimensional feature maps under different scales;
the residual error module: the system is used for connecting and integrating image features under different scales;
the output module: after the characteristic extraction, the burr defect is predicted and output.
Preferably, the convolution module includes a convolution layer, a batch normalization layer, a Re L U activation function layer, an adaptive average pooling layer, and a full connection layer, which are connected in sequence, specifically as follows:
a first layer: conv1, 7 × 7 convolution layers with step size of 2 and output size of 1 × 1;
a second layer: bn1, batch normalization layer and regularization with a l2 penalty term;
and a third layer: a Relu layer as an activation function;
a fourth layer: maximum pooling layer maxpool, size 3 × 3, step size 2, padding 1;
fifth to eighth layer: residual block with sizes of 64 × 64, 128 × 128, 256 × 256, 512 × 512;
a ninth layer: average pooling layer avgpool, size 1 x 1;
the above is a complete convolution module, the neural network structure is composed of 58 convolution modules described above, and after the last convolution module, a 512 × 512 full-connected layer fn is connected.
Preferably, the residual error module includes a convolution layer and a batch normalization layer, and specifically includes the following steps:
the first layer is a conv1, 1 × 1 convolution layer and is used for image dimensionality reduction and feature extraction;
a second layer: bn1, batch normalization layer;
and a third layer: conv2, 3 × 3 convolution layer, and entrain a shortcut;
a fourth layer: bn2, batch normalization layer;
and a fifth layer: conv3, 1 × 1 convolution;
a sixth layer: bn3, batch normalization layer;
the residual error module is a complete residual error module, and four residual error modules are arranged in each convolution module and used for superposing the characteristics extracted by the upper layer network.
A method of inspecting a ResNet58 network based fastener flash defect detection structure according to claim 1, comprising the steps of: step 1, manually marking a fastener product image in a data source to obtain category information about qualified products and burr defects;
step 2, integrating the fastener product image and the marking information thereof to be used as a complete data set;
step 3, image preprocessing and image enhancement are carried out on the data set;
step 4, dividing the fastener product image enhanced by the data in the step 3 to obtain a training set and a testing set;
step 5, inputting the data into the constructed network for training to obtain a burr defect detection model;
and 6, inputting the image of the fastener product to be detected into the burr defect detection model for detection to obtain a burr defect detection result.
Preferably, in step 3, the image preprocessing and image enhancement of the data set comprises the following steps: step a, converting marking information corresponding to the fastener product image into the name of the image to achieve the purpose of direct classification, labeling qualified products as F (fine), and labeling burr defects as M (mao);
b, normalizing the format and the size of the fastener product image, wherein the original image format is tif format, automatically changing tif into jpg format by compiling a program for modifying file format in batches, and simultaneously normalizing the image size to 256 × 256;
c, performing data enhancement on the normalized fastener product image, judging the color format and the pixel interval of the image, converting the non-RGB format into the RGB format, and then performing gray processing; a pixel difference of less than 50 can be considered as an abnormal picture, and the image is named again in sequence after being discarded.
Preferably, in step 4, a random seed is generated according to step 3, according to 8: 2 the data enhanced fastener product image is divided into a training set and a testing set.
Preferably, in the step C, the data enhancement comprises image deformation, image scaling, median filtering denoising and contrast enhancement, and comprises the following steps of (1) realizing the image deformation and scaling by a resize function, a rotate function and a transposase function in opencv, step (2) realizing the median filtering by a median blur function in opencv, replacing the value of a central pixel by the median of all signals in a neighborhood, and better retaining texture information under the condition of filtering an abnormal value, and step (3) realizing the contrast enhancement by a C L AHE algorithm to widen the gray scale range of the image;
step (4), the processed image still has more or less noise, the image is subjected to centralized denoising by using the wave function of opencv, and due to the multi-resolution characteristic, the signal is filtered by using band-pass filters with different central frequencies, scale coefficients mainly reflecting noise frequencies are removed, and then coefficients of the residual scales are combined to perform inverse transformation, so that the noise is well suppressed, and the formula is as follows:
the scale a controls the expansion and contraction of the wavelet function, the translation amount tau controls the translation of the wavelet function, the scale corresponds to the frequency, and the translation amount tau corresponds to the time;
and (5) the product image can enter a training module for model training after image preprocessing and image enhancement.
Preferably, in step 5, the loss function loss adopted in the training process of the glitch defect detection network is cross entropy:
preferably, in step 5, the training process of the glitch defect detection network uses a random gradient descent SGD containing momentum terms as an optimizer, and includes the following steps: e, outputting a score value score after the image passes through the neural network structure;
f, inputting the score value into a loss function and then calculating an error loss;
step g, then starting SGD optimization, including back propagation by deriving a neural network and forward propagation by repeating the above operation of the network structure;
and h, circularly reciprocating until the specified times or the error is smaller than a set value, and finishing training.
Preferably, after the tested precision reaches the standard, testing is performed, the operation is to input a marked image which does not participate in training, so that the model is judged, if the image is a qualified product, F is output, otherwise, M is output, and the test is finished.
Compared with the prior art, the detection structure and the detection method for the fastener burr defect based on the ResNet58 network have the following beneficial effects that: by adopting the detection structure and the detection method of the fastener burr defect based on the ResNet58 network, the residual neural network of the ResNet58 is used as a basic network for feature extraction, the pyrrch is used as a development frame, and the input module, the training module and the output module are arranged to detect the defect of a fastener product, so that the detection efficiency is improved, and the accuracy of burr defect detection and identification is improved.
Drawings
FIG. 1 is a schematic flow chart of an embodiment of a detection method of a fastener burr defect detection structure based on a ResNet58 network;
FIG. 2 is a structural diagram of a detection structure of a fastener burr defect based on a ResNet58 network in the embodiment;
FIG. 3 is a schematic flow chart of image preprocessing in the present embodiment;
FIG. 4 is a schematic flow chart of model training in this embodiment;
FIG. 5 is a schematic flow chart illustrating a model test in the present embodiment;
fig. 6 is a graph illustrating the precision P and the recall ratio R in the present embodiment.
Detailed Description
The invention is further described below with reference to the accompanying drawings.
Fig. 1 is a flow chart showing a detection method of a fastener burr defect detection structure based on a ResNet58 network, which comprises the following steps: step 1, manually marking a fastener product image in a data source to obtain category information about qualified products and burr defects;
step 2, integrating the fastener product image and the marking information thereof to be used as a complete data set;
step 3, performing image preprocessing and image enhancement on the data set, as shown in fig. 3, which is a schematic flow chart of image preprocessing, wherein the image preprocessing and image enhancement on the data set comprises the following steps: step a, converting marking information corresponding to a fastener product image into a name of the image, achieving the purpose of direct classification, and labeling qualified products as F (fine), such as F0001.jpg and F0002.jpg, and labeling burr defects as M (mao), such as M0001.jpg and M0002. jpg; b, normalizing the format and the size of the fastener product image, wherein the original image format is the tif format, and automatically changing the tif into the jpg format by compiling a program for modifying the file format in batches;
c, performing data enhancement on the normalized fastener product image, judging the color format and the pixel interval of the image, reading the image by using the imread in opencv, and judging whether the image is in an RGB format or not, wherein the non-RGB format is converted into the RGB format; performing GRAY level processing on the converted RGB image by using an RGB2GRAY function of opencv to convert the converted RGB image into a GRAY level image; calculating the difference between the maximum pixel value and the minimum pixel value of each image, wherein the pixel difference is less than 50, the image can be regarded as an abnormal image, the abnormal image is set as an abnormal value and deleted, and the sorted images are renamed in sequence after being discarded; storing a copy of the processed image in a backup by using imwrite so as to facilitate quick search in the future; and uses a resize function on the image data to normalize its size to 256X 256.
The data enhancement comprises image deformation, image scaling, median filtering denoising and contrast enhancement, and comprises the following steps: step (1), image deformation and scaling: the normalized image is sequentially realized through a resize function, a rotate function and a transpose function in opencv, so that the image is deformed, the randomness of data is increased, and the generalization capability training of a model is facilitated;
step (2), median filtering is realized through a media blur function in opencv, and by replacing the median of all signals in the neighborhood of the pixel value of a certain point with the value of a central pixel, texture information can be well reserved under the condition of filtering an abnormal value, so that some noises in an image are filtered, and more critical feature points are reserved;
step (3), contrast enhancement is realized through a C L AHE algorithm, the gray scale range of the image is widened, the darker pixel value is darker, and the opposite brighter brightness value is brighter, so that the effect of improving the contrast is achieved;
step (4), the processed image still has more or less noise, the image is subjected to centralized denoising by using the wave function of opencv, and due to the multi-resolution characteristic, the signal is filtered by using band-pass filters with different central frequencies, scale coefficients mainly reflecting noise frequencies are removed, and then coefficients of the residual scales are combined to perform inverse transformation, so that the noise is well suppressed, and the formula is as follows:
the scale a controls the expansion and contraction of the wavelet function, the translation amount tau controls the translation of the wavelet function, the scale corresponds to the frequency, and the translation amount tau corresponds to the time;
and (5) the product image can enter a training module for model training after image preprocessing and image enhancement.
And 4, dividing the fastener product image enhanced by the data in the step 3 to obtain a training set, a testing set and a verification set, generating a random seed according to the ratio of 8: 2 the data enhanced fastener product image is divided into a training set and a testing set.
And 5, inputting the data into the constructed network for training, and performing the training process of the neural network for detecting the burr defects, wherein the neural network is used for fitting a judgment rule which can best meet the actual situation and extracting the characteristics which meet the rule.
Fig. 4 is a schematic flow chart of model training, which specifically includes the following steps: step 5-1, creating a configuration file config, wherein the configuration file config comprises initialization of constants such as a data path root, a data list, a product type class, the number of single-group data batch size, a learning rate lr, a batch number epoch, a stop identifier early _ stopping and the like; step 5-2, loading a GPU and a neural network model, initializing relevant parameters, initializing model hyper-parameters, a loss function and an optimizer, and judging whether the purpose of program execution is training or testing; step 5-3, loading network pre-training weight, converting image data into a Tensor format, facilitating calculation and acceleration processing, performing random rotation and translation operations on Tensor size resize to 224 × 224, and normalizing the numerical value of Tensor, namely reducing the equal proportion to 0-1; step 5-4, reading and loading image data to a model input position, and loading configuration parameter constants;
step 5-5, entering an epoch cycle, training data in batches, presetting the optimal precision to be 80%, replacing when the actual precision is greater than the preset precision, starting training, initializing time, and inputting data; 5-6, loading the cuda driver, accelerating the training process, and obtaining an output result after the data is accessed into the network and is subjected to the processes of convolution, batch normalization, pooling and relu activation; step 5-7, sending output to a loss function loss, solving an error, and updating the error; step 5-8, performing back propagation on the loss function, performing derivation on the neural network, initializing a weight parameter by the optimizer, and recording the parameter obtained after the derivation; step 5-9, updating batch and time, and outputting specific information in the training process, including training batch, loss value and learning rate; step 5-10, repeating the steps 5-5 to 5-9 until one batch is finished, and jumping out of the cycle; step 5-11, updating the weight parameter of the model and the weight parameter of the optimizer, recording the weight parameter of the model with the minimum error at this time and storing the weight parameter of the model with the minimum error to checkpoint; 5-12, repeating the steps 5-5 to 5-11 until the epoch cycle is finished; and 5-13, comparing and finding out the model weight with the highest precision, and saving the model weight to the file best _ checkpoint.
Convolution operation is a means for mapping high-dimensional features to low-dimensional features, and aims to reduce interference of multi-dimensional information (i.e. prevent overfitting), extract and reserve features most critical to a target object
In the field of image processing, it is understood that a picture is mapped onto another smaller or larger image after feature extraction.
The neural network can carry out back propagation and optimization while training, and the training process of the glitch defect detection network uses a random gradient descent SGD (generalized minimum deviation) containing momentum items as an optimizer, and comprises the following steps: e, outputting a score value score after the image passes through the neural network structure;
f, inputting the score value into a loss function and then calculating an error loss; the loss function loss adopted in the training process of the burr defect detection network is cross entropy:
the above formula is used as a loss function in a neural network, p represents the distribution of real markers, q is the distribution of predicted markers of the trained model, and the similarity of p and q can be measured by the cross entropy loss function.
And g, starting SGD optimization, including back propagation and forward propagation, wherein the back propagation is realized by derivation of the neural network, the forward propagation is realized by repeating the operation of the network structure, and an SGD optimization equation is as follows:
α, called the step size, controls the magnitude of the change in theta each iteration in the direction that J (theta) (J (theta) represents the loss function) becomes smaller the partial derivative of J (theta) to theta represents the direction in which J (theta) changes the most, since the minimum is sought, the direction of the gradient is the opposite of the partial derivative.
Step h, the operation is repeated in a circulating mode until the specified times or the error is smaller than the set value, and the burr defect detection model is obtained after training is finished
And 6, inputting the image of the fastener product to be detected into a burr defect detection model for detection, wherein as shown in FIG. 5, the method is a schematic flow chart of model test and comprises the following steps: step 6-1, loading a GPU and a neural network model, and initializing relevant parameters;
step 6-2, loading the optimal weight parameters to the model, and reading a test image path;
step 6-3, converting the image data into a Tensor format, facilitating calculation and accelerated processing,
step 6-4, performing pre-operation on the Tensor, enabling the dimension resize of the Tensor to be 224, executing random rotation and translation operations, and normalizing the numerical value of the Tensor, namely reducing the numerical value of the Tensor to 0-1 in an equal proportion;
step 6-5, reading and loading image data to a model input position, loading configuration parameter constants, entering an epoch cycle, and testing data in batches;
6-6, loading a cuda driver, accelerating the training process, and performing convolution, batch normalization, pooling and relu activation processes when data is accessed to a network;
6-7, starting testing, and obtaining a result output2 after model prediction;
step 6-8, sending output2 into a cal _ acc function to obtain precision;
and 6-9, obtaining the test precision of all batches from the step 6-6 to the step 6-8, calculating the average value to obtain the final average precision Acc, and obtaining the burr defect detection result.
And after the tested precision reaches the standard, testing, namely inputting a marked image which does not participate in training, judging the model, outputting F if the model is a qualified product, and outputting M if the model is not a qualified product, and finishing the test. And (3) evaluating the model performance and outputting a detection result, wherein f1score, mAP, AUC and the like are commonly used as model performance evaluation indexes. In the present embodiment, the most commonly used method for evaluating the mep is based on a PR curve, the ordinate of the PR curve is the accuracy P, the abscissa of the PR curve is the recall ratio R, as shown in fig. 6, the graph of the accuracy P and the recall ratio R is a schematic diagram, the average accuracy of the AP single category is the area of the PR curve forming the shape with the R axis, i.e. the integral of P to R, and the mep is the average of the average accuracies of all categories.
After the model test is finished, classifying the product images in all test data according to the model prediction result, and finally evaluating the accuracy of the model by using mAP, so far, the fastener product burr defect detection method based on the ResNet58 network is completely explained, and the variables in each step are explained in Table 1, such as the variable description table in Table 1:
as shown in fig. 2, the structure diagram of the detection structure for the fastener burr defect based on the ResNet58 network comprises an input module, a training module and an output module, wherein the input module and the output module are respectively connected with the training module, the training module comprises a plurality of convolution modules which are sequentially connected, each convolution module comprises a residual error module and a short connecting layer, and the convolution modules are connected with the residual error modules through the short connecting layers.
An input module: the method is used for carrying out normalization preprocessing and image enhancement on the image. An output module: after the method is used for extracting features, the burr defects are predicted and output, and the output module comprises a convolution layer with the convolution kernel size of 1 x 1 and a full connection layer.
The convolution module is used for carrying out convolution and pooling on sample data for multiple times to realize down-sampling to obtain multi-dimensional feature maps under different scales, and comprises a convolution layer, a batch normalization layer, a Re L U activation function layer, an adaptive average pooling layer and a full connection layer which are sequentially connected, wherein the convolution module comprises a conv1 convolution layer, a 7 convolution layer with the step size of 2 and the output size of 1 x 1, a bn1 batch normalization layer and regularization with the penalty of l2, a Relu layer serving as an activation function, a fourth maximum pooling layer maxpool with the size of 3 x 3 and the step size of 2 and padding of 1, a fifth layer to an eighth layer, a block with the size of 64 x 64, 128 x 128, 256x256 and 512 x, a ninth layer, an average pooling layer avgpool with the size of 1 x 1, a residual error layer with the size of 64 x 128, 256x256 and 512 x, and a complete convolution module is formed by connecting more than one complete neural network 58 convolutional layer after the convolution module is connected.
A residual module: the system is used for connecting and integrating image features under different scales; the residual module comprises a convolution layer and a batch normalization layer, and comprises the following specific steps: the first layer is a conv1, 1 × 1 convolution layer and is used for image dimensionality reduction and feature extraction; a second layer: bn1, batch normalization layer; and a third layer: conv2, 3 × 3 convolution layer, and entrain a shortcut; a fourth layer: bn2, batch normalization layer; and a fifth layer: conv3, 1 × 1 convolution; a sixth layer: bn3, batch normalization layer; the residual error module is a complete residual error module, and four residual error modules are arranged in each convolution module and used for superposing the characteristics extracted by the upper layer network.
The foregoing is a preferred embodiment of the present invention, and it will be apparent to those skilled in the art that various changes and modifications may be made without departing from the spirit of the invention, and these should be considered to be within the scope of the invention.
Claims (10)
1. A detection structure of fastener burr defect based on ResNet58 network, characterized in that: the training system comprises an input module, a training module and an output module, wherein the input module and the output module are respectively connected with the training module, the training module comprises a plurality of convolution modules which are sequentially connected, each convolution module comprises a residual error module and a short connecting layer, and the convolution modules and the residual error modules are connected through the short connecting layers;
the input module is used for: the image processing device is used for carrying out normalization preprocessing and image enhancement on the image;
the convolution module: the system is used for performing convolution and pooling on sample data for multiple times to realize down-sampling and obtain multi-dimensional feature maps under different scales;
the residual error module: the system is used for connecting and integrating image features under different scales;
the output module: after the method is used for extracting the features, identifying, predicting and outputting the burr defects.
2. The structure for detecting the fastener burr defect based on the ResNet58 network as claimed in claim 1, wherein said convolution module comprises a convolution layer, a batch normalization layer, a Re L U activation function layer, an adaptive average pooling layer and a full connection layer which are connected in sequence as follows:
a first layer: conv1, 7 × 7 convolution layers with step size of 2 and output size of 1 × 1;
a second layer: bn1, batch normalization layer and regularization with a l2 penalty term;
and a third layer: a Relu layer as an activation function;
a fourth layer: maximum pooling layer maxpool, size 3 × 3, step size 2, padding 1;
fifth to eighth layer: residual block with sizes of 64 × 64, 128 × 128, 256 × 256, 512 × 512;
a ninth layer: average pooling layer avgpool, size 1 x 1;
the above is a complete convolution module, the neural network structure is composed of 58 convolution modules described above, and after the last convolution module, a 512 × 512 full-connected layer fn is connected.
3. The structure of claim 2 for detecting fastener flash defects based on a ResNet58 network, wherein: the residual error module comprises a convolution layer and a batch normalization layer, and specifically comprises the following steps:
the first layer is a conv1, 1 × 1 convolution layer and is used for image dimensionality reduction and feature extraction;
a second layer: bn1, batch normalization layer;
and a third layer: conv2, 3 × 3 convolution layer, and entrain a shortcut;
a fourth layer: bn2, batch normalization layer;
and a fifth layer: conv3, 1 × 1 convolution;
a sixth layer: bn3, batch normalization layer;
the residual error module is a complete residual error module, and four residual error modules are arranged in each convolution module and used for superposing the characteristics extracted by the upper layer network.
4. The method for detecting the structure of the fastener burr defect based on the ResNet58 network in claim 1, wherein: the method comprises the following steps: step 1, manually marking a fastener product image in a data source to obtain category information about qualified products and burr defects;
step 2, integrating the fastener product image and the marking information thereof to be used as a complete data set;
step 3, image preprocessing and image enhancement are carried out on the data set;
step 4, dividing the fastener product image enhanced by the data in the step 3 to obtain a training set and a testing set;
step 5, inputting the data into the constructed network for training to obtain a burr defect detection model;
and 6, inputting the image of the fastener product to be detected into the burr defect detection model for detection to obtain a burr defect detection result.
5. The method for detecting the structure of the fastener burr defect based on the ResNet58 network according to claim 4, wherein: in step 3, the image pre-processing and image enhancement of the data set comprises the following steps: step a, converting marking information corresponding to the fastener product image into the name of the image to achieve the purpose of direct classification, labeling qualified products as F (fine), and labeling burr defects as M (mao);
b, normalizing the format and the size of the fastener product image, wherein the original image format is tif format, automatically changing tif into jpg format by compiling a program for modifying file format in batches, and simultaneously normalizing the image size to 256 × 256;
c, performing data enhancement on the normalized fastener product image, judging the color format and the pixel interval of the image, converting the non-RGB format into the RGB format, and then performing gray processing; a pixel difference of less than 50 can be considered as an abnormal picture, and the image is named again in sequence after being discarded.
6. The method for detecting the structure of the fastener burr defect based on the ResNet58 network according to claim 5, wherein: in step 4, a random seed is generated according to step 3, and the seed is generated according to the following steps 8: 2 the data enhanced fastener product image is divided into a training set and a testing set.
7. The method for detecting the fastener burr defect structure based on the ResNet58 network is characterized in that in the step C, the data enhancement comprises image deformation, image scaling, median filtering denoising and contrast enhancement, and the method comprises the following steps of (1) realizing the image deformation and scaling through a reset function, a rotate function and a transpose function in opencv, step (2) realizing the median filtering through a media blur function in opencv, replacing the value of a central pixel with the median of all signals in a neighborhood, and better retaining texture information under the condition of filtering abnormal values, and step (3) realizing the contrast enhancement through a C L AHE algorithm to widen the gray scale range of the image;
step (4), the processed image still has more or less noise, the image is subjected to centralized denoising by using the wave function of opencv, and due to the multi-resolution characteristic, the signal is filtered by using band-pass filters with different central frequencies, scale coefficients mainly reflecting noise frequencies are removed, and then coefficients of the residual scales are combined to perform inverse transformation, so that the noise is well suppressed, and the formula is as follows:
the scale a controls the expansion and contraction of the wavelet function, the translation amount tau controls the translation of the wavelet function, the scale corresponds to the frequency, and the translation amount tau corresponds to the time;
and (5) the product image can enter a training module for model training after image preprocessing and image enhancement.
8. The method for detecting the structure of the fastener burr defect based on the ResNet58 network according to claim 4, wherein: in step 5, the loss function loss adopted in the training process of the glitch defect detection network is cross entropy:
9. the method for detecting the structure of the fastener burr defect based on the ResNet58 network as claimed in claim 8, wherein: in step 5, the training process of the glitch defect detection network uses a random gradient descent SGD containing momentum terms as an optimizer, and includes the following steps: e, outputting a score value score after the image passes through the neural network structure;
f, inputting the score value into a loss function and then calculating an error loss;
step g, then starting SGD optimization, including back propagation by deriving a neural network and forward propagation by repeating the above operation of the network structure;
and h, circularly reciprocating until the specified times or the error is smaller than a set value, and finishing training.
10. The method for detecting the structure of the fastener burr defect based on the ResNet58 network as claimed in claim 8, wherein: and after the tested precision reaches the standard, testing, namely inputting a marked image which does not participate in training, judging the model, outputting F if the model is a qualified product, and outputting M if the model is not a qualified product, and finishing the test.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010277370.5A CN111415353A (en) | 2020-04-10 | 2020-04-10 | Detection structure and detection method for fastener burr defects based on ResNet58 network |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010277370.5A CN111415353A (en) | 2020-04-10 | 2020-04-10 | Detection structure and detection method for fastener burr defects based on ResNet58 network |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111415353A true CN111415353A (en) | 2020-07-14 |
Family
ID=71493527
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010277370.5A Pending CN111415353A (en) | 2020-04-10 | 2020-04-10 | Detection structure and detection method for fastener burr defects based on ResNet58 network |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111415353A (en) |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111986170A (en) * | 2020-08-14 | 2020-11-24 | 湖南理工学院 | Defect detection algorithm based on Mask R-CNN (deep neural network) |
| CN112270404A (en) * | 2020-11-24 | 2021-01-26 | 南通大学 | Detection structure and method for bulge defect of fastener product based on ResNet64 network |
| CN112381165A (en) * | 2020-11-20 | 2021-02-19 | 河南爱比特科技有限公司 | Intelligent pipeline defect detection method based on RSP model |
| CN112446438A (en) * | 2020-12-16 | 2021-03-05 | 常州微亿智造科技有限公司 | Intelligent model training method under industrial Internet of things |
| CN113436138A (en) * | 2021-03-31 | 2021-09-24 | 成都飞机工业（集团）有限责任公司 | Image preprocessing method for aviation structural part identification based on digital image |
| CN114827384A (en) * | 2022-04-01 | 2022-07-29 | 杭州友勤信息技术有限公司 | Endoscope image tuning system |
| CN118150598A (en) * | 2024-05-13 | 2024-06-07 | 浙江智泓科技有限公司 | Burr removal quality detection method and system for automobile part shell |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105871413A (en) * | 2016-06-13 | 2016-08-17 | 哈尔滨工业大学 | Low SNR direct sequence spread spectrum signal detection method |
| CN109285139A (en) * | 2018-07-23 | 2019-01-29 | 同济大学 | A kind of x-ray imaging weld inspection method based on deep learning |
| CN109376792A (en) * | 2018-11-07 | 2019-02-22 | 河北工业大学 | Photovoltaic cell open defect classification method based on multichannel residual error neural network |
| CN110348376A (en) * | 2019-07-09 | 2019-10-18 | 华南理工大学 | A kind of pedestrian's real-time detection method neural network based |
| CN110728654A (en) * | 2019-09-06 | 2020-01-24 | 台州学院 | Automatic pipeline detection and classification method based on deep residual error neural network |
-
2020
- 2020-04-10 CN CN202010277370.5A patent/CN111415353A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105871413A (en) * | 2016-06-13 | 2016-08-17 | 哈尔滨工业大学 | Low SNR direct sequence spread spectrum signal detection method |
| CN109285139A (en) * | 2018-07-23 | 2019-01-29 | 同济大学 | A kind of x-ray imaging weld inspection method based on deep learning |
| CN109376792A (en) * | 2018-11-07 | 2019-02-22 | 河北工业大学 | Photovoltaic cell open defect classification method based on multichannel residual error neural network |
| CN110348376A (en) * | 2019-07-09 | 2019-10-18 | 华南理工大学 | A kind of pedestrian's real-time detection method neural network based |
| CN110728654A (en) * | 2019-09-06 | 2020-01-24 | 台州学院 | Automatic pipeline detection and classification method based on deep residual error neural network |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111986170A (en) * | 2020-08-14 | 2020-11-24 | 湖南理工学院 | Defect detection algorithm based on Mask R-CNN (deep neural network) |
| CN112381165A (en) * | 2020-11-20 | 2021-02-19 | 河南爱比特科技有限公司 | Intelligent pipeline defect detection method based on RSP model |
| CN112381165B (en) * | 2020-11-20 | 2022-12-20 | 河南爱比特科技有限公司 | Intelligent pipeline defect detection method based on RSP model |
| CN112270404A (en) * | 2020-11-24 | 2021-01-26 | 南通大学 | Detection structure and method for bulge defect of fastener product based on ResNet64 network |
| CN112446438A (en) * | 2020-12-16 | 2021-03-05 | 常州微亿智造科技有限公司 | Intelligent model training method under industrial Internet of things |
| CN113436138A (en) * | 2021-03-31 | 2021-09-24 | 成都飞机工业（集团）有限责任公司 | Image preprocessing method for aviation structural part identification based on digital image |
| CN113436138B (en) * | 2021-03-31 | 2023-07-21 | 成都飞机工业（集团）有限责任公司 | Image preprocessing method for aviation structure identification based on digital image |
| CN114827384A (en) * | 2022-04-01 | 2022-07-29 | 杭州友勤信息技术有限公司 | Endoscope image tuning system |
| CN118150598A (en) * | 2024-05-13 | 2024-06-07 | 浙江智泓科技有限公司 | Burr removal quality detection method and system for automobile part shell |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111415353A (en) | Detection structure and detection method for fastener burr defects based on ResNet58 network | |
| US10964004B2 (en) | Automated optical inspection method using deep learning and apparatus, computer program for performing the method, computer-readable storage medium storing the computer program, and deep learning system thereof | |
| CN111179251B (en) | Defect detection system and method based on twin neural network and by utilizing template comparison | |
| CN108765412B (en) | Strip steel surface defect classification method | |
| CN110060237A (en) | A kind of fault detection method, device, equipment and system | |
| CN111260640B (en) | Tree generator network gear pitting image measuring method and device based on cyclean | |
| CN116258707A (en) | PCB surface defect detection method based on improved YOLOv5 algorithm | |
| CN112907561A (en) | Notebook appearance flaw detection method based on deep learning | |
| CN112819748B (en) | Training method and device for strip steel surface defect recognition model | |
| CN116468663A (en) | Method for detecting surface micro defects based on improved YOLOv5 | |
| Chen et al. | A rotation-invariant convolutional neural network for image enhancement forensics | |
| CN116883393B (en) | Metal surface defect detection method based on anchor frame-free target detection algorithm | |
| CN117523394A (en) | SAR vessel detection method based on aggregation characteristic enhancement network | |
| CN117237599A (en) | Image target detection method and device | |
| CN111639697A (en) | Hyperspectral image classification method based on non-repeated sampling and prototype network | |
| CN114841992A (en) | Defect detection method based on cyclic generation countermeasure network and structural similarity | |
| CN117197146A (en) | Automatic identification method for internal defects of castings | |
| CN117456230A (en) | Data classification method, system and electronic equipment | |
| CN112270404A (en) | Detection structure and method for bulge defect of fastener product based on ResNet64 network | |
| Yu et al. | Research on CNN Algorithm for Monochromatic Fabric Defect Detection | |
| CN116597275A (en) | High-speed moving target recognition method based on data enhancement | |
| CN115661042A (en) | Hierarchical classification defect detection method based on attention mechanism guidance | |
| CN112052881B (en) | Hyperspectral image classification model device based on multi-scale near-end feature splicing | |
| JP7345764B2 (en) | Inspection system and inspection program | |
| CN117392440B (en) | Textile fabric retrieval method and system based on tissue structure and color classification |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200714 |