CN108427969A - A kind of paper sheet defect sorting technique of Multiscale Morphological combination convolutional neural networks - Google Patents

Abstract

A paper defect classification method based on multi-scale morphology combined with convolutional neural network, using structural elements of different scales to perform morphological operations on the original image, weighted and fused gradient images at different scales to obtain a multi-scale morphological gradient image, in order to enhance Paper defect contrast, highlighting defect gradient features and defect edge information features, weighted fusion of multi-scale morphological gradient images and original defect images, to achieve defect image enhancement, input to convolutional neural network for feature extraction and classification, convolution The neural network is used in the classification of paper defects, which can quickly realize the accurate classification of paper defects, and has the characteristics of simple method, short time consumption and high recognition accuracy.

Description

A paper defect classification method based on multi-scale morphology combined with convolutional neural network

technical field

The invention belongs to the technical field of image processing and pattern recognition, in particular to a paper defect classification method based on multi-scale morphology combined with convolutional neural network.

Background technique

Feature extraction is a key step in pattern recognition and has important applications in image analysis and pattern recognition. The traditional feature extraction method of image classification is to pre-define a feature, and then perform feature extraction and classification according to the defined feature. In practical applications, paper images are easily affected by factors such as light and environment, which makes defect detection, feature extraction and classification become hot spots in the paper industry. At present, scholars have proposed a variety of paper defect classification algorithms. Yuan Hao et al proposed to apply support vector machine to the actual paper defect classification by performing fixed feature selection on the gray image of paper defects. There may be variations in brightness that lead to suboptimal classification. Therefore, Hu Muyi et al. proposed to use dynamic double thresholds to segment paper defect regions based on the grayscale features of different paper defect images, and extract defect features for classification. However, threshold segmentation needs to set different thresholds for different paper defects, which leads to difficulty in parameter setting. In order to reduce the complexity of parameter setting, Zhou Qiang et al. used the method of Hough transform to detect linear features to classify paper defects. This method has a better recognition effect when the shape of the defect is linear, but it is not suitable for most paper with non-linear shapes. Defect classification and complex calculation are not conducive to later online implementation. Based on the above problems, Yang Yannan et al. proposed to use fuzzy fuser to fuse various eigenvalues of paper defects, and use radial basis neural network to classify paper defect images, which expanded the scope of paper defect identification, but the extracted features Single and shallow features, resulting in low classification accuracy. For this reason, Luo Lei et al. proposed to extract the LBP (Local Binary Pattern, Local Binary Pattern) feature of the paper grayscale image for defect recognition. However, since the LBP method has higher requirements on the definition of paper surface image texture, complex preprocessing algorithms need to be added. Therefore, Wu Yiquan and others proposed a paper defect recognition method based on Krawtchouk moment invariants and wavelet support vector machines. By calculating the Krawtchouk moment invariants of paper defect images, the feature vectors of paper defect images were constructed. According to the feature vectors of training samples A support vector machine is constructed to classify paper defect images, but the complex calculation steps in the early stage will reduce the recognition speed. In order to improve the accuracy of defect identification, Zhou Qiang et al proposed to use two-dimensional wavelet transform to denoise and singular value decomposition to extract paper defect features and identify defect categories. However, there are still problems such as complex feature calculation in extracting paper defect features. And the existing paper defect classification methods rely on traditional feature descriptors and classifier selection, complex feature calculation and other issues.

Aiming at the problem that traditional image classification methods rely on feature descriptors, the deep learning proposed by Hinton et al. can learn image features hierarchically, effectively avoiding the problem that traditional image classification methods rely on artificial feature descriptors. has been widely used in the field. As a representative model of deep learning, convolutional neural network (CNN) can automatically learn image features, extract and classify deep feature information of shape features, texture features, color features and spatial relationship features of complex images. Usually better classification results can be obtained than traditional feature extraction methods. The texture of paper defect images is simple, the background is single, and paper defect images such as black spots, holes, pad marks, and wrinkles are small and a small number of defects. Traditional methods can only extract a single underlying feature, resulting in unsatisfactory classification results. Therefore, how to apply deep convolutional neural networks to paper defect images to achieve fast and accurate classification is a hot issue.

Contents of the invention

In order to overcome the shortcomings of the above-mentioned prior art, the purpose of the present invention is to provide a paper defect classification method based on multi-scale morphology combined with convolutional neural network. Using convolutional neural network in paper defect classification can quickly realize paper defect classification. Accurate classification has the characteristics of simple method, short time consumption and high recognition accuracy.

In order to achieve the above object, the technical scheme that the present invention takes is:

A paper defect classification method based on multi-scale morphology combined with convolutional neural network, comprising the following steps:

Step 1: Prepare paper defect image data, and randomly divide training set and test set;

Step 2: Initialize, given the operating parameters of the Matlab program, input the image of the paper defect training set;

Step 3: Perform different-scale morphological gradient operations on the input image to obtain morphological gradient images at multiple scales;

Step 4: Weighted fusion of gradient images at different scales, taking the mean value to obtain the final multi-scale morphological gradient image;

Step 5: Fuse the multi-scale morphological gradient image with the paper defect image to achieve image enhancement, and obtain the input image of the convolutional neural network;

Step 6: The enhanced training image is input to the convolutional neural network training model;

Step 7: Repeat steps 3-5 for the test set to obtain gradient enhanced images, and input them to the trained model for defect feature extraction and classification;

Step 8: Compare the predicted label with the actual label, and calculate the correct rate of defect classification.

The specific implementation process of the step 3 is:

Calculate the morphological gradient image G(i) of the paper defect image f(x,y) using mathematical morphology method:

(a) Calculate the gradient image g(i) corresponding to the structural elements se _i of different scales of f, the formula is as follows:

(b) Corrosion operation is performed on g(i) and se _i-1 to obtain the morphological gradient image G(i) of f, the formula is as follows:

G(i)=g(i)! se _i-1 .

The beneficial effects of the present invention compared with prior art are:

Aiming at the problems that traditional paper defect classification methods rely on the selection of feature descriptors and classifiers, the calculation steps of feature description are complicated, and the uneven illumination of paper defect images affects the classification accuracy, etc., using deep convolutional neural network to extract and classify paper defect images , to solve the problems existing in the traditional classification method, combined with the characteristics of paper defects, using the multi-scale morphology method to achieve gradient enhancement on the defect image, and then using the advantages of convolutional neural network (CNN) that can automatically learn image features to characterize the paper defect image Extract and classify. Compared with the existing traditional characteristic paper defect classification method, the present invention does not need to perform defect feature extraction and feature description for various defects, can quickly realize accurate classification of paper defects, and at the same time solves the problem that the classification is affected by uneven illumination of paper defect images The problem of precision has the advantages of low time consumption and high precision.

Description of drawings

Fig. 1 (a) is the paper defect image "black spot", "hole", "pad mark", "crease" in the experiment of the present invention;

Fig. 1 (b) is the image enhancement result of the present invention utilizing the preprocessing comparison method Canny operator to four types of paper defect images;

Fig. 1 (c) is the image enhancement result of the present invention utilizing the preprocessing comparison method Sobel operator to four types of paper defect images;

Fig. 1 (d) is the image enhancement result of the present invention utilizing the preprocessing contrast method Prewitt operator to four types of paper defect images;

Fig. 1(e) is the result of image enhancement of four types of paper defect images using the method of the present invention.

Fig. 2 (a) is the visualization feature map of each convolutional layer of the convolutional neural network using the paper defect "hole" image in the present invention;

Fig. 2(b) is the visualized feature map of each convolutional layer of the convolutional neural network of the "hole" image of the paper defect that is gradient enhanced by the method of the present invention.

Detailed ways

The present invention will be described in further detail below in conjunction with the accompanying drawings and embodiments.

The morphological operation is performed on the original image with structural elements of different scales, and the gradient images at different scales are weighted and fused to obtain a multi-scale morphological gradient image. In order to enhance the contrast of paper defects and highlight the defect gradient features and defect edge information features, the multi-scale morphological gradient image is weighted and fused with the original defect image to achieve defect image enhancement, and input to the convolutional neural network for feature extraction and classification. The implementation steps are as follows:

Step 1: Prepare paper defect image data, and randomly divide training set and test set;

Step 2: Initialization, given the operating parameters of the program, se _i is the structural element corresponding to the i-th scale, where the circular structural element with a radius of 1, 3, 5, 7 is taken, l=0, m=0 ,k=length(training set);

Among them, l represents the number equal to the predicted label and the actual label, m is a constant, k represents the number of test set images, and the input paper defect training set image;

Step 3: Calculate the morphological gradient image G(i) of the paper defect image f(x,y) using mathematical morphology method:

(a) Calculate the gradient image g(i) corresponding to the structural elements se _i of different scales of f, the formula is as follows:

(b) Corrosion operation is performed on g(i) and se _i-1 to obtain the morphological gradient image G(i) of f, the formula is as follows:

G(i)=g(i)! se _i-1 ;

Step 4: The weighted fusion of the morphological gradient images at different scales is averaged to obtain the final multi-scale morphological gradient image MG. The formula is as follows:

Among them, n represents the number of scales;

Step 5: Calculate the gradient enhanced image f _MG , the formula is as follows:

fMG = (f+ _MG );

Step 6: Input the gradient enhanced image f _MG corresponding to the training set to the convolutional neural network training model;

Step 7: Repeat steps 3-5 for the test set to obtain the gradient enhanced image f _MG of the test set, input it to the trained model for defect feature extraction and classification, and obtain the prediction label prel _k corresponding to the training set data;

Step 8: Compare the predicted label prel _k of the training set with the actual label l _k ;

(a) if prel _k =l _k , l=l+1, m=m+1;

(b) Otherwise, l remains unchanged, and the termination condition is: m=k;

(c) Calculate the defect classification recognition rate, the formula is as follows:

Embodiment one:

In order to test the effectiveness and superiority of the present invention to paper defect image classification, the simulation experiment is carried out under the hardware environment of CPU: Intel (TM) i7-6700U, 3.3GHz, memory 16GB, NVIDIA Quadro K620 graphics card and the software environment of MATLABR2017a of.

Three comparison methods are used: the traditional method extracts the HOG features of paper defects to classify, and obtains the classification results (HOG+SVM), extracts the LBP features and classifies to obtain the classification results (LBP+SVM), and the preprocessing method Canny operator to classify the four types of paper defects The classification results (Canny+CNN) obtained by inputting the image to the CNN model after image enhancement, the classification results (Sobel+CNN) obtained by the Sobel operator for the four types of paper defect image enhancement and input to the CNN model, the Prewitt operator for the four types of paper defects After the image is enhanced, the classification result (Prewitt+CNN) obtained by inputting to the CNN model and the result of feature extraction and classification of the four types of paper defect images by the method of the present invention are shown in Table 1 for the experimental results.

Table 1 shows the results of classifying paper defect images used in the present invention and comparing the time used by each method in the table.

Table 1

As can be seen from Table 1, the correct rate of paper defect image classification obtained by the method of the present invention is higher than that of several other traditional methods, and the advantages are obvious; although the time used for classification is not the shortest, it is already significantly lower than most other traditional methods. The superiority status, therefore, can show that the method of the present invention is a good paper defect classification method.

Table 2 is the result of classifying and comparing the time spent on the Caltech101 image object recognition data set and the KTH-TIPS texture image data set using the methods in the table to prove the feasibility and practicability of the present invention.

Table 2

Table 2 shows the results of gradient enhancement of each operator on the paper defect image. The contrast of the edge contour is improved compared with the original image, but it is more dependent on the directionality of the defect edge. Canny operator and Prewitt operator have stronger denoising ability than Sobel operator, and it is easy to smooth out some edge information. Sobel operator can enhance the edge of the image, but the operator used is a fixed-size structural element, which is not suitable for weak boundaries. Gradient enhancement of paper defect images. However, the present invention can enhance the edge contour information and gradient features in the paper defect image, while retaining the background information of the original image and the surrounding information of the defect target, taking into account that the small-scale structural elements in the morphology are beneficial to the detection of gradient details, and the large-scale structural elements are beneficial to suppress The advantages of noise can effectively highlight the defect gradient features and edge information features, and solve the problem that the classification of paper defects depends on the selection of feature descriptors and classifiers, and the feature calculation is complicated.

Claims (2)

1. a kind of paper sheet defect sorting technique of Multiscale Morphological combination convolutional neural networks, which is characterized in that including as follows Step：

Step 1：Prepare paper sheet defect image data, and random division training set and test set；

Step 2：Initialization gives Matlab program running parameters, inputs paper sheet defect training set image；

Step 3：Different scale Morphological Gradient operation is carried out to input picture, obtains the Morphological Gradient figure under multiple scales Picture；

Step 4：To the gradient image Weighted Fusion under different scale, takes and be worth to final multiscale morphological gradient figure Picture；

Step 5：Multiscale morphological gradient image with paper sheet defect image merge and realizes image enhancement, obtains convolution god Through network inputs image；

Step 6：Enhanced training image is input to convolutional neural networks training pattern；

Step 7：Test set repeats step 3-5 and obtains grad enhancement image, is input on trained model and carries out defect characteristic It extracts and classifies；

Step 8：Prediction label and physical tags are compared, defect classification accuracy rate is calculated.

2. a kind of paper sheet defect sorting technique of Multiscale Morphological combination convolutional neural networks according to claim 1, It is characterized in that, the step 3 specific implementation process is：

The Morphological Gradient image G (i) of paper defect image f (x, y) is calculated using Mathematical Morphology Method：

(a) the structural elements se of the different scale size of f is calculated_iCorresponding gradient image g (i), formula are as follows：

(b) to g (i) and se_i-1Erosion operation is carried out, obtains the Morphological Gradient image G (i) of f, formula is as follows：

G (i)=g (i)！se_i-1。