CN109685030A - A kind of mug rim of a cup defects detection classification method based on convolutional neural networks - Google Patents

Abstract

The mug rim of a cup defects detection classification method based on convolutional neural networks that the invention discloses a kind of, comprising the following steps: mug rim of a cup image information A, is acquired by image capturing system；B, the noise of image is collected using opencv removal and data set expands；C, flaw labeling is carried out to mug rim of a cup image using LabelImg, the training set image marked is uniformly formatted as fixed size: 2M*2M；D, training convolutional neural networks are treated using formatted training set to be trained；E, image characteristics extraction is carried out to the image after flaw labeling using convolutional neural networks model after training；F, region recommendation network generates the positive sample candidate frame and negative sample candidate frame of identical quantity according to characteristics of image；G, detection target in region is recommended to classify target.So far, whole system, which completes, classifies to mug rim of a cup defects detection.The present invention can be effectively used for the classification of mug rim of a cup defects detection, improve detection degree of automation and efficiency and reduce the labor intensity of influence and worker of the human factor to detection process.

Description

A detection and classification method of mug mouth defect based on convolutional neural network

technical field

The invention relates to the technical field of cup mouth defect detection and classification, in particular to a mug cup mouth defect detection and classification method based on a convolutional neural network.

Background technique

With the continuous development of China's manufacturing industry and the continuous improvement of people's living standards, mugs have become a part of people's daily life. The defects of the mug mouth mainly refer to the spots, gaps and scratches on the mouth of the cup, which directly affect the product. Sales and corporate image. Therefore, it is particularly important to adopt appropriate defect detection and classification methods. Traditional manual detection methods have disadvantages such as low detection efficiency, high work intensity and low precision. Other researchers combine computer vision and image processing to detect defects in mugs through defect comparison. This kind of defect image recognition The algorithm needs to manually construct, select the main features of the target, and select an appropriate classifier for identification, which has great limitations. For example, the candidate region discrimination, that is, the segmented candidate regions are identified based on shape features, grayscale features and Hu invariant moment features, which requires people to participate in the design of some main features of the defect, which has a problem: based on manual design The feature is not very robust to the diverse changes of defects, and is only suitable for specific defect detection.

In recent years, with the emergence of millions of labeled training sets and the emergence of GPU-based training algorithms, training complex convolutional network models has become a luxury. Compared with the traditional method of manually extracting features, the convolutional neural network can not only automatically learn the features of the target, but also be suitable for the processing of data sets, and can also perform end-to-end learning, and most of the predictions are completed in the GPU, greatly Improved the speed and accuracy of object detection.

Therefore, based on the above, it has become a feasible solution to introduce convolutional neural network into the detection and classification of mug cup mouth defects, which improves the speed and accuracy of target detection compared with defect comparison, and improves the speed and accuracy of target detection compared with traditional manual detection. In order to detect the degree of automation and efficiency and reduce the impact of human factors on the detection process and the labor intensity of workers.

SUMMARY OF THE INVENTION

The purpose of the present invention is to design a method for detecting and classifying defects of a mug mouth based on a convolutional neural network, so as to solve the problems raised in the above background art.

In order to achieve the above purpose, the present invention provides the following technical solutions: a method for detecting and classifying defects of a mug mouth based on a convolutional neural network, comprising the following steps:

A. The image information of the mug mouth is collected by the image acquisition system;

B. Use opencv to remove the noise of the collected images and expand the dataset;

C. Use labelImg to mark the defects of the mug cup mouth image, and uniformly format the marked training set images to a fixed size: 2M*2M;

D. Use the formatted training set to train the convolutional neural network to be trained;

E. Use the post-training convolutional neural network model to extract image features from the images marked with defects;

F. The regional recommendation network generates the same number of positive sample candidate frames and negative sample candidate frames according to image features;

G. Classify the detection targets in the target recommendation area.

Preferably, the weight update in the training network in the step D includes the following parts:

A. Add weight decay to the traditional Adam method;

B. The weight decay is not added to the loss function to participate in the gradient calculation, but an additional weight decay process will be performed each time the parameters are updated;

C. The parameter update step of weight decay is:

in, is the learning rate, is the weight decay coefficient, is the additional weight decay term, is a small constant for numerical stability, the default , the deviation of the corrected first moment , the deviation of the corrected second moment , and in the interval , the recommended defaults are: 0.9 and 0.999, respectively.

Preferably, generating positive and negative sample frames in step F includes the following parts:

A. Generate a feature map on the (default is to take ) small sliding window;

B. Predict while generating each small window recommended region candidate boxes;

C. Each small sliding window is mapped to a 256-dimensional feature vector under the ZF network, and the vector is sent to two parallel fully connected layers, including the target box classification layer and the target box regression layer, of which the target box classification layer outputs two Score, i.e. the probability that each target box corresponds to the target and non-target. The target box regression layer outputs four regression parameters , respectively represent the center coordinates and width and height of the target frame, which are used to correct the sliding window to obtain a corrected target frame;

D. The candidate frame whose intersection ratio with the target real frame is greater than 0.7 is regarded as a positive sample, and the candidate frame whose intersection ratio with the target real frame is less than 0.3 is regarded as a negative sample, and the rest of the candidate frames will be discarded.

Preferably, the classification of the detection targets in the target recommendation area in the step G includes the following parts:

A. Map the positive and negative candidate frames generated in step F to the original feature map finally extracted by the convolutional neural network;

B. It is uniformly formatted through the ROI pooling layer as default n=7;

C. The network uses the obtained recommended region features The classifier judges the target category, and uses the regression layer of the network to make a second adjustment to the regional candidate frame belonging to a certain category to further correct its position;

D. Get the final classification result.

Compared with the prior art, the beneficial effects of the present invention are:

(1) The mug defect detection and classification method provided by the present invention can effectively improve the degree of automation, accuracy and efficiency of detection and classification;

(2) The detection method can accurately reflect the defect type of the mug, which is convenient for a more accurate evaluation of the cup mouth defect of the mug, which is beneficial to the manufacturer to improve the process according to the defect type and improve the production efficiency.

Description of drawings

Fig. 1 is a flow chart of the present invention.

Figure 2 shows the workflow of the ROI pooling layer.

FIG. 3 is a general network structure diagram of the present invention.

Detailed ways

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

Please refer to Fig. 1 , Fig. 2 and Fig. 3 , the present invention provides a system solution: a method for detecting and classifying cup mouth defects based on convolutional neural network. The present invention is described in further detail below in conjunction with the accompanying drawings and specific embodiments: wherein accompanying drawing 1 is the overall flow chart of the system, accompanying drawing 2 is the working flow chart of the ROI pooling layer, and accompanying drawing 3 is the overall network structure diagram of the present invention:

Include the following steps:

A. The image information of the mug mouth is collected by the image acquisition system;

B. Use opencv to remove the noise of the collected images and expand the dataset;

C. Use labelImg to mark the defects of the mug cup mouth image, and uniformly format the marked training set images to a fixed size: 2M*2M;

D. Use the formatted training set to train the convolutional neural network to be trained;

E. Use the post-training convolutional neural network model to extract image features from the images marked with defects;

F. The regional recommendation network generates the same number of positive sample candidate frames and negative sample candidate frames according to image features;

G. Classify the detection targets in the target recommendation area.

In the present invention, the image acquisition system in step A includes the following steps:

A. Fix the CCD camera on the operating platform;

B. After the position is adjusted properly, the CCD camera obtains the image of the cup mouth of the mug;

C. The image is collected and saved by the host computer for subsequent operations.

The invention uses a CCD camera to replace the work of human eyes in traditional manual detection. The CCD camera is fixed on the operating platform to ensure the stability of detection, improve the automation degree of the detection process, and reduce errors caused by human factors.

In the present invention, in step B, image denoising and data set expansion include the following steps:

A. Install opencv3.3.0 and visual studio2013 and configure the environment;

B. Use the medianBlur function in opencv to filter the collected image;

C. Use functions such as resize and flip in opencv to expand the data set.

The present invention uses the medianBlur function in opencv for denoising, which has a more suitable denoising effect compared to other traditional denoising methods. The image geometric transformation is used to expand the data set, so that the trained convolutional neural network has better effect and increases the results. accuracy.

In the present invention, the defect marking in step C comprises the following steps:

A. Install python2.7.17 and labelImg and configure the environment;

B. Use notepad++ to open predefined_classes.txt in the source code folder, and modify the default category;

C. "Open Dir" to open the picture folder, select the pictures in turn to start marking, use "Create RectBox" to start drawing the frame, and mark the corresponding defect category;

D. Format the labeled training set images into a fixed size: 2M*2M.

The labelImg adopted in the present invention is a convenient image labeling tool, which can mark a designated area as a designated defect category, so as to facilitate the training of the convolutional neural network.

In the present invention, the convolutional neural network training in step D includes the following steps:

A. Select the convolutional neural network structure and determine the initial parameters;

B. The iterative process of training includes forward propagation of each small batch of data and back propagation of errors in turn. After all data back propagation is completed, the weight parameters are uniformly updated for the next iteration. When When the expected number of iterations is reached, the iteration is stopped, and the final accuracy rate of the network is calculated;

C. Continuously adjust the network structure and related parameters according to the training situation, until the training accuracy rate reaches the requirements;

D. Data such as the trained convolutional neural network model and weight parameters are used for subsequent operations.

The convolutional neural network adopted in the present invention needs to first use the data set marked in step C to train the network, and the trained network model can be used to detect and classify the collected images, and this scheme improves the intelligence of detection The degree of transformation, while reducing the labor intensity of workers.

In the present invention, the image feature extraction in step E includes the following steps:

A. The convolution layer perceives the image based on the principle of the local receptive field, and obtains the image features as a feature map for subsequent operations;

B. The pooling layer performs a secondary extraction of the feature map based on the principle of local correlation of the image, and further reduces the number of parameters required for training and the risk of overfitting on the premise of retaining the image features as much as possible.

In the image feature extraction network adopted in the present invention, the convolution layer can obtain features that are more in line with the actual image, reducing the complexity of the network and preventing over-fitting; extraction, reducing the risk of overfitting.

In the present invention, the image feature extraction in step F includes the following steps:

A. Generate a feature map on the (default is to take ) small sliding window;

B. Predict while generating each small window recommended region candidate boxes;

C. Each small sliding window is mapped to a 256-dimensional feature vector under the ZF network, and the vector is sent to two parallel fully connected layers, including the target box classification layer and the target box regression layer, of which the target box classification layer outputs two Score, i.e. the probability that each target box corresponds to the target and non-target. The target box regression layer outputs four regression parameters , respectively represent the center coordinates and width and height of the target frame, which are used to correct the sliding window to obtain a corrected target frame;

D. The candidate frame whose intersection ratio with the target real frame is greater than 0.7 is regarded as a positive sample, and the candidate frame whose intersection ratio with the target real frame is less than 0.3 is regarded as a negative sample, and the rest of the candidate frames will be discarded.

The region recommendation network used in the present invention generates positive and negative sample frames, which is faster than the previous method for obtaining candidate frames, and can be easily combined with subsequent target classification.

In the present invention, the target classification in step G comprises the following steps:

A. Map the positive and negative candidate frames generated in step F to the original feature map finally extracted by the convolutional neural network;

B. It is uniformly formatted through the ROI pooling layer as default n=7;

C. The network uses the obtained recommended region features The classifier judges the target category, and uses the regression layer of the network to make a second adjustment to the regional candidate frame belonging to a certain category to further correct its position;

D. Get the final classification result.

The invention adopts the classification of the detection target in the target recommendation area, and at the same time of classification, the classification result of the regression parameter correction is obtained, so that the classification result is more accurate.

Claims (3)

1. A method for detecting and classifying defects of mug cup mouth based on convolutional neural network, characterized in that: using the improved Adam to update the weights of the convolutional neural network network, according to the gradient descent in the update process, appropriate learning The rate is modified to adapt to different gradients and improve training efficiency. 2. A method for detection and classification of mug mouth defects based on convolutional neural network, characterized in that: a small sliding window is generated on the feature map and a candidate frame of the recommended region is predicted at the same time, and the intersection and union ratio with the target real frame is greater than 0.7 IOU The candidate frame of 1 is used as a positive sample, and the candidate frame that intersects with the target ground-truth frame and is less than 0.3 IOU is regarded as a negative sample, and the rest of the candidate frames are discarded. 3. A method for detection and classification of cup mouth defects based on convolutional neural network, characterized in that: using the softmax classifier to determine the target category for the obtained recommended region features, and using the network's method for the region candidate frame belonging to a certain category. The regression layer makes a second adjustment to further correct its position until the effect is the best, and finally the optimal classification is achieved.