Mobile phone screen MURA defect detection method based on convolutional neural network pruning algorithm
The invention belongs to the field of target detection and identification, and relates to a method for detecting a specific target from an image, in particular to a method for detecting defects of a mobile phone screen.
The traditional method for detecting the defects of the screen manually has a plurality of defects, and the method cannot meet the requirements of high efficiency and accuracy of the current industrial production at present when the industrial production is developed at a high speed. For mobile phone screen manufacturers, finding an efficient and accurate automatic detection device to replace a manual detection link is an urgent need. With the development of the fields of computer vision, image processing and the like, an automatic detection system based on machine vision becomes a good solution. According to the scheme, the high-resolution industrial camera is used for collecting the mobile phone screen image, and then the image information is processed in real time through the image analysis module, so that whether the mobile phone screen is qualified or not is judged.
The traditional screen defect detection algorithm based on machine vision is mostly designed aiming at one or more types of screen defects, and has no universality, so that a special algorithm needs to be written for special defects. Moire is inevitable on imaging when photographing due to the particularity of the screen, and the Moire problem cannot be well solved by the traditional algorithm. In addition, although the traditional algorithm can detect more obvious linear and point defects, the accuracy rate of the traditional algorithm is low for the imaging light-weight massive MURA defects. Finally, the conventional screen defect detection algorithm needs to adjust a large number of parameters, and especially when the screen product is remodeled, time is wasted by adjusting the large number of parameters. Therefore, it is of great practical value to design an algorithm with good generalization.
In recent years, deep learning methods have had a great impact in the field of computer vision. Deep learning adopts a multilayer network structure to simulate the hierarchical relationship and the transmission mode of a nervous system in a human brain, and the method is widely applied to a plurality of fields of pattern recognition and achieves good results. The method adopts a classification algorithm based on a deep convolutional neural network, uses a preprocessed local image block of a mobile phone screen as the input of a classifier, then extracts a feature map of the convolutional neural network as a detection result, and converts a defect detection problem into an image block classification problem. In addition, compared with the traditional algorithm, the convolutional neural network algorithm has less parameter setting, and the algorithm has good universality, so that the method is particularly suitable for rapid model changing of screen products, the model changing time is shortened, and the production line efficiency is improved.
Disclosure of Invention
The invention provides a mobile phone screen MURA defect detection method based on a convolutional neural network pruning algorithm, aiming at overcoming the defects of a screen defect detection algorithm based on machine vision.
In order to achieve the purpose, the mobile phone screen MURA defect detection method based on the convolutional neural network pruning algorithm comprises the following steps:
1) self-defining a convolutional neural network, and training the network through training data until convergence is achieved and the accuracy is high;
2) pruning the convolutional neural network by a self-adaptive template matching method, and reducing the network scale and network parameters;
3) collecting mobile phone screen picture data, generating a picture pyramid, dividing the picture pyramid into picture blocks, generating test phase data, and sending the test phase data into the pruned convolutional neural network obtained in the step 2) for operation;
4) and taking the sum of the characteristic graphs of the intermediate hidden layers as a response graph, and obtaining and enclosing the final position of the defect by adopting a threshold segmentation method, wherein the method is particularly used for detecting the MURA defect.
The pruning matched with the self-adaptive template in the step 2) is specifically as follows: corresponding the part with response of the characteristic diagram of the hidden layer in the middle of the network to an original image, taking the part as a foreground, taking the rest as a background, and calculating the average brightness difference between the foreground and the background; taking a plurality of feature maps with the largest difference for reservation; the remaining feature maps and their associated convolution kernels are pruned from the network.
The picture pyramid in step 3) is specifically: the original high resolution picture is reduced to pictures of different scales, which are collectively referred to as a set of picture pyramids. The purpose of using the picture pyramid is to detect flaws of different sizes on different scales.
The response graph in step 4) is specifically: the output of the 4 th convolutional layer is adopted, and the sum of the characteristic graphs of the layer of the network after pruning is used as a response graph of the defect.
The invention has the following beneficial effects:
the invention discloses a mobile phone screen MURA defect detection method based on a convolutional neural network pruning algorithm. Based on a convolutional neural network algorithm of deep learning, a self-adaptive template matching method is adopted for network pruning, and a network model and parameters are compressed, so that the effect close to real time is achieved. In addition, the convolution neural network algorithm can automatically learn background texture information, so that moire interference in imaging can be well processed. When the screen product is remodeled, only a small amount of training pictures are needed to carry out network fine tuning, and the accuracy of the network can be guaranteed to reach the standard again.
Compared with the traditional method, the method can more effectively detect the defects with light imaging, such as the nodular MURA defects. In addition, the traditional method needs to set a large number of parameters, and the accuracy of the algorithm can be ensured only by adjusting the parameters every time the model is changed. The neural network algorithm adopted by the invention has good self-adaption and generalization characteristics, can quickly change the screen product, and saves the deployment time of a production line.
FIG. 1 is a framework diagram of the adaptive template matching pruning algorithm of the present invention.
FIG. 2 is a graphical representation of the inventive pyramid-to-tile segmentation of a picture.
The technical solution of the present invention is clearly and completely explained and described below.
The invention provides a mobile phone screen MURA defect detection method based on a convolutional neural network pruning algorithm.
The invention discloses a mobile phone screen MURA defect detection method based on a convolutional neural network pruning algorithm, which comprises the following steps:
step 1, data acquisition in a training phase: and respectively collecting small blocks containing flaws and normal pictures, and marking (1 represents that the pictures contain flaws, and 0 represents that the pictures are normal). According to the following steps of 9: 1 into a training set and a validation set; and (3) self-defining the convolutional neural network, training the convolutional neural network until convergence by using the training data, and achieving higher classification accuracy on the verification set. By observing the feature map of the hidden layer in the middle of the visual convolutional neural network, whether the network well learns the flaw feature can be analyzed.
And 2, pruning the convolutional neural network by using a self-adaptive template matching method, and reducing the network scale and network parameters. Specifically, all N feature maps on the L convolutional layer will be the ith (c)i < N) the response position learned by the feature map corresponds to the original image, the partial position is used as a foreground template, the rest position of the original image is used as a background template, and the average brightness difference D between the foreground template and the background template is calculatedi. For all DiSorting in descending order, and taking the first K (K is less than N) average brightness differences DiAnd taking the sum of the maximum feature maps as a response map. The remaining feature map and its corresponding convolution kernel are pruned directly as shown in fig. 1. And the whole network is retrained after pruning, so that the network after pruning still has higher accuracy.
Step 3, generating test stage data: and acquiring the picture data of the mobile phone screen, wherein the picture comprises and only comprises the complete mobile phone screen position. And spreading the picture into an image pyramid by using the scaling scales of different proportions for multi-scale detection. For each scale of picture, the picture is divided into picture blocks with fixed sizes, and all the picture blocks are used as a group to serve as primary input data of the convolutional neural network, as shown in fig. 2.
And 4, selecting the sum of all feature maps of the 4 th convolution layer as a response map for each picture block, re-corresponding the response map to the original map, obtaining the final position of the flaw by a threshold segmentation method, and drawing out the flaw.