CN107481231A - A kind of handware defect classifying identification method based on depth convolutional neural networks - Google Patents

Abstract

A hardware defect classification and recognition method based on a deep convolutional neural network, comprising the following steps: establishing a network to construct a deep convolutional neural network; training the network, and dividing the collected images into two categories, namely training sets and testing set, the training set accounts for 70% of the total number of images collected, and the test set accounts for 30% of the total number of images collected; for defect identification, the hardware images in the test set are input into the trained network, and the output results are checked. The recognition result is compared with the label of the image, and the correct recognition rate and false recognition rate are counted. In this algorithm, a deep convolutional neural network is used, which eliminates the need for complex image processing algorithms. By increasing the depth of the network, more abstract features of defects are extracted, so that different defect categories are more distinguishable and the recognition rate is higher.

Description

A hardware defect classification and recognition method based on deep convolutional neural network

technical field

The invention relates to the technical field of image processing based on artificial intelligence, in particular to a hardware defect classification and recognition method based on a deep convolutional neural network.

Background technique

Machine vision, also known as computer vision, is a science that studies the use of cameras and computers to imitate human eyes and brains, so that machines can replace humans for detection and judgment, and complete tasks such as target recognition and industrial inspection. Machine vision technology is an applied technical discipline that integrates digital image processing, artificial intelligence, computer graphics and other disciplines, and is widely used in automated production. In recent years, with the advancement of computer technology and the continuous improvement of neural network theory, the rapid development of computer vision has been promoted. The rapid development of my country's machine vision industry occupies a very important position in the field of automated production inspection.

Due to the advantages of easy molding, light weight, easy to obtain materials, and suitable for mass production, hardware parts are widely used in household appliances, machinery, chemicals, aviation and other fields. With the increasing application of hardware and the rapid development of rapid prototyping technology, people's requirements for the quality of hardware are also getting higher and higher. The quality of hardware mainly has requirements in terms of size and appearance. Appearance is an important link to ensure the quality of hardware, but manual inspection is usually used in actual production. The manual detection method is inefficient and the degree of automation is not high, and its accuracy is often related to the working experience and attitude of the testing personnel. At present, hardware product manufacturers are paying more and more attention to improving the level of production automation, and the requirements for production efficiency are getting higher and higher, and manual inspection methods are increasingly unable to meet the demand. In addition, in the process of production and processing, due to factors such as changes in the physical parameters of raw materials, unreasonable process parameters, and poor processing machinery performance, hardware products will have bruises, sand holes, scratches, lack of materials, deformation, pitting, oil stains, etc. on the surface defect. These surface defects will not only destroy the appearance of hardware products, but also affect its performance and make it unusable. At present, the detection and identification of surface defects of hardware products are mainly done manually, with low efficiency and low degree of automation.

The development and wide application of machine vision technology can just solve the above problems. Compared with the manual detection method, the detection system based on machine vision technology has the following advantages:

(1) High precision

The measurement accuracy of the machine vision system can reach 0.01mm accuracy level, which is much higher than the human vision limited by physical conditions.

(2) Repeatability

Machine vision systems can efficiently and accurately complete inspection tasks repeatedly without feeling fatigued like inspectors. When the human eye repeatedly checks the product, it will feel subtle differences due to the influence of various factors, which will affect the accuracy rate.

(3) Real-time

Machine vision systems use computers to efficiently capture, store and process images. The system automatically transmits image data, which can reflect the status of the production site in real time.

(4) Non-contact

The machine vision system does not have to be in contact with the workpiece during inspection, so it generally does not cause deformation of the workpiece to adversely affect it. In addition, the system can replace inspectors working in toxic, high-temperature and other harsh environments.

(5) Low cost

The machine vision system can work for a long time without interruption, and can complete the tasks equivalent to multiple inspectors. Now that labor costs are getting higher and higher, machine vision systems can greatly reduce production costs.

Artificial neural network is the technical reproduction of biological neural network in a simplified sense. As a discipline, its main task is to build a practical artificial neural network according to the principle of biological neural network and the needs of practical applications, and design corresponding learning Algorithms, which simulate certain intelligent activities of the human brain, are then implemented technically to solve practical problems. There are also many applications in computer vision, mainly used to classify and identify the extracted defects.

A brief flow chart of the traditional defect detection and recognition algorithm is shown in Figure 1. It first needs to perform image processing on the input image to segment the defect area from the image. Then analyze various defect features and select appropriate features with high discrimination. Next, manual extraction of features is carried out, and these extracted features are input into common classifiers such as BP neural network or SVM (Support Vector Machine) for classification, and finally the classification and recognition results are given at the output end. It can be seen that the above-mentioned traditional defect recognition algorithm is very dependent on the accuracy of defect region segmentation, and needs to manually select and extract defect features. However, for the image of hardware products in this paper, due to the serious noise interference, the accurate segmentation of defects requires a complex image processing process, and the amount of calculation is very large. In addition, it is often difficult to effectively select and describe highly discriminative features, requiring very specialized knowledge and good prior knowledge.

Convolutional neural network is a deep neural network proposed by LeCun, which can directly take a two-dimensional image as input without complex image preprocessing on the original image data. The convolutional neural network automatically extracts and combines features from images, recognizes visual patterns based on the extracted features, and then gives classification results at the output. The brief model of image recognition using convolutional neural network is shown in Figure 2 below. Compared with traditional defect recognition algorithms, it does not need to manually select and describe features, and avoids a lot of calculations. In addition, convolutional neural networks can recognize changing patterns, are robust to geometric deformation, and can tolerate image distortion.

Contents of the invention

The object of the present invention is to solve the above problems and propose a method for classifying and identifying hardware defects based on a deep convolutional neural network.

In order to achieve this goal, the present invention adopts the following technical solutions:

A method for classifying and identifying hardware defects based on a deep convolutional neural network, comprising the following steps:

A. Establish a network and construct a deep convolutional neural network;

B. train the network, divide the defect images collected into two categories, i.e. a training set and a test set, the training set accounts for 70% of the total number of images collected, and the test set accounts for 30% of the total number of images collected;

C. Defect recognition, input the hardware defect images in the test set into the trained network, check the output results, compare the recognition results with the labels of the images, and count the correct recognition rate and wrong recognition rate.

More preferably, the training network algorithm described in step B includes the following steps:

Step 1. First, initialize the weights of the network, and make the weight distribution obey the Gaussian distribution with a mean value of 0 and a variance of 0.01, and at the same time make the number of weights greater than 0 approximately equal to the number of less than 0;

Step 2, remove the mean value of the training sample, the training sample is a color picture, and each pixel has three components of R, G, and B to sum and calculate the average value. When the hardware sample is input, the three components of all pixels of the sample are Subtract the mean and feed into the network;

Step 3, network training, the network training adopts stochastic gradient descent method.

More preferably, the stochastic gradient descent method includes the following steps:

Step a, forward propagation, the forward propagation inputs the training samples into the network one by one, outputs the calculation results through the convolution layer, the excitation layer and the classifier, compares the labels, and calculates the output error;

Step b, feedback calculation error, according to the error described in step a, from the output layer to forward, calculate the error of each layer of the network, according to the error of each layer, calculate the weight update amount, update the weight w and deviation b;

Step c, square the errors obtained from all sample training, sum them up, and take the square root as the total error of the network output. If the total error of the network is greater than the set threshold, restore the initial value of the error and counter, and retrain the samples until the error is less than the set threshold Set the threshold.

More optimally, in the step a, it is assumed that there are m pairs of training samples, and each training error is

More preferably, the expression of the total error is

More preferably, the training network process uses GPU to accelerate calculation.

More preferably, the structure of the deep convolutional neural network described in step A includes six convolutional layers, six excitation layers, three pooling layers, and one fully connected layer;

An excitation layer is connected behind the convolutional layer, and a pooling layer is connected after two consecutive convolutional layers and the excitation layer, and the last layer of the deep convolutional neural network is connected to a classifier.

More preferably, each of the convolutional layers is composed of 3×3 convolutional units.

More preferably, the image collected in step B is a color image, the image is collected by an industrial camera, and the defect image is input into the network.

More optimally, the classifier is a softmax classifier.

Beneficial effects of the present invention:

1. A defect classification and recognition algorithm, which can be suitable for various hardware and other parts;

2. Using convolutional neural network and softmax classifier for classification and recognition, no other complex image preprocessing algorithms are required;

3. Remove the average value before the sample training, use GPU to accelerate the calculation during the training process, reduce the training time, and can process hardware defect images with more pixels;

4. Using deep convolutional neural network to identify, you can increase the number of layers according to the actual situation, extract more abstract features of defects, and have a higher recognition accuracy;

5. Using the Leaky ReLU excitation function, it will not fit during the training process, the calculation is simple and effective, and the convergence speed is fast;

6. Using the stochastic gradient descent method, the calculation amount is smaller than the standard gradient method, and it can avoid falling into the local minimum.

Description of drawings

Figure 1 is a brief flowchart of a traditional defect detection and recognition algorithm;

Figure 2 is a brief model diagram of image recognition using a convolutional neural network;

Figure 3 is a structural diagram of a convolutional neural network for identifying defects;

Fig. 4 is a two-dimensional matrix diagram of a convolutional neural network;

Figure 5 is a Leaky ReLU function diagram;

Figure 6 is a two-dimensional matrix diagram of max-pooling;

Figure 7 is a flowchart of the training network algorithm.

detailed description

The technical solutions of the present invention will be further described below in conjunction with the accompanying drawings and through specific embodiments.

A method for classifying and identifying hardware defects based on a deep convolutional neural network, comprising the following steps:

A. Establish a network and construct a deep convolutional neural network;

B. train the network, divide the defect images collected into two categories, i.e. a training set and a test set, the training set accounts for 70% of the total number of images collected, and the test set accounts for 30% of the total number of images collected;

C. Defect recognition, input the hardware images in the test set into the trained network, check the output results, compare the recognition results with the labels of the images, and count the correct recognition rate and wrong recognition rate.

To further illustrate, the training network algorithm described in step B includes the following steps:

Step 1. First, initialize the weights of the network, and make the weight distribution obey the Gaussian distribution with a mean value of 0 and a variance of 0.01, and at the same time make the number of weights greater than 0 approximately equal to the number of less than 0;

Step 2, remove the mean value of the training sample, the training sample is a color picture, and each pixel has three components of R, G, and B to sum and calculate the average value. When the hardware sample is input, the three components of all pixels of the sample are Subtract the mean and feed into the network;

Step 3, network training, the network training adopts stochastic gradient descent method.

Further description, the stochastic gradient descent method includes the following steps:

Step a, forward propagation, the forward propagation inputs the training samples into the network one by one, outputs the calculation results through the convolutional layer, the excitation layer and the classifier, compares the labels, and calculates the output error

Step b, feedback calculation error, according to the error described in step a, from the output layer to forward, calculate the error of each layer of the network, according to the error of each layer, calculate the weight update amount, update the weight w and deviation b;

Step c, square the errors obtained from all sample training, sum them up, and take the square root as the total error of the network output. If the total error of the network is greater than the set threshold, restore the initial value of the error and counter, and retrain the samples until the error is less than the set threshold Set the threshold.

As a further illustration, in the step a, it is assumed that there are m pairs of training samples, and each training error is

Further illustration, the expression of the total error is

To further illustrate, the process of training the network uses GPU to accelerate calculation. GPU parallel acceleration calculation reduces training time and can process hardware images with more pixels.

Further description, it is characterized in that: the structure of deep convolutional neural network described in step A includes six convolutional layers, six excitation layers, three pooling layers, and one fully connected layer; After two consecutive convolutional layers and excitation layers, a pooling layer will be connected, and the last layer of the deep convolutional neural network is connected to the classifier. The convolutional neural network structure diagram for identifying defects is shown in Figure 3. The convolutional neural network mainly consists of convolutional layers, excitation layers, and pooling layers. The excitation layer is connected behind the convolutional layer, and a pooling layer is connected after two consecutive convolutional layers and the excitation layer. After repeating this structure three times, a deep convolutional network is obtained. In the last layer of the network, the pixels obtained by each neuron are rasterized, that is, all the pixels obtained by each are arranged in a row. Then, a classifier is connected to identify defect categories.

The input of the neurons in the excitation layer is similar to the input of the hidden layer. The hidden layer data x is multiplied by the weight w and the bias value b is added to obtain the input of the excitation layer, that is, y=∑ _i w _i x _i +b. Input the value of y into the activation function, here select the Leaky ReLU function, as shown in Figure 5, and calculate the output structure.

The role of the pooling layer is to simplify the output of the convolutional layer. For example, each neuron in a pooling layer might sum over neurons in a 2×2 region of the previous layer. Another commonly used max-pooling, the pooling unit simply outputs the maximum excitation in a 2×2 input domain, as shown in Figure 6, this method uses the max-pooling method.

To further illustrate, each of the convolutional layers is composed of 3×3 convolutional units. Usually, the input layer in a neural network is represented by a series of neurons, and it is more visual and intuitive to use a two-dimensional matrix in a convolutional neural network. Like a regular neural network, neurons in the input layer need to be connected to neurons in the hidden layer. However, instead of connecting every input neuron to every hidden neuron, convolutional neural networks only create connections in local regions of an image. Taking an image of size 7X7 as an example, suppose the neurons of the first hidden layer are connected to a 3X3 area of the input layer, as shown in Figure 4. This 3X3 area is called the local perception domain. The 9 neurons of the local perceptual domain are connected to the same neuron of the first hidden layer, and each connection has a weight, so the local perceptual domain has a total of 3X3 weights. If the local perceptual domain is slid from left to right and from top to bottom, different neurons in the corresponding hidden layer will be obtained. Figure 4 only shows the first neuron and input of the first hidden layer. Layer connectivity. The local perceptual domain slides to the right for a compensation (here set to 2), and the input layer data is sent to the second hidden layer neuron. Carrying out in sequence, the transmission of data from the input layer to the hidden layer can be completed. The 3X3 neurons in the first hidden layer obtained above all use the same 3X3 weights, which is called the weight sharing principle. In addition, each neuron in the hidden layer shares a bias value b, which is called a shared bias.

To further illustrate, the image collected in step B is a color image, the image is collected by an industrial camera, and the entire image is input into the network.

To further illustrate, the classifier is a softmax classifier. In one embodiment of the present invention, the classifier is a softmax classifier. The Softmax classifier has only two layers, namely the input layer and the output layer, and lacks the hidden layer compared with the neural network. According to the types of defects to be identified, there are several output units for setting softmax. The present invention needs to identify six types of defects such as trachoma, scratch, lack of material, deformation, pitting and oil stain. So softmax has 6 output units, one for each defect category. After the image is input, after the network operation, each output unit of Softmax will output a value, representing the probability that the input belongs to each category. We think that the maximum value of the output is the defect category of the input sample.

The above describes the technical principles of the present invention in conjunction with specific embodiments. These descriptions are only for explaining the principles of the present invention, and cannot be construed as limiting the protection scope of the present invention in any way. Based on the explanations herein, those skilled in the art can think of other specific implementation modes of the present invention without creative work, and these modes will all fall within the protection scope of the present invention.

Claims (10)

1. a kind of handware defect classifying identification method based on depth convolutional neural networks, it is characterised in that including following step Suddenly：

A. network is established, constructs a depth convolutional neural networks；

B. training network, the image collected is divided into two major classes, i.e. training set and test set, the training set, which accounts for, collects figure As the 70% of sum, the test set, which accounts for, collects the 30% of total number of images；

C. defect recognition, the network that the handware defect image input in test set has been trained, checks output result, will Recognition result is compareed with the label of image, counts correct recognition rata and error recognition rate.

2. a kind of handware defect classifying identification method based on depth convolutional neural networks according to claim 1, its It is characterised by：Training network algorithm comprises the following steps described in step B：

Step 1: first, the weights of network are initialized, and it is 0 weight distribution is submitted to average, variance is 0.01 Gaussian Profile, while number of the weights more than 0 is approximately equal to the number less than 0；

Step 2: training sample goes average, the training sample is colour picture, each pixel have tri- components of R, G, B add and, Average, when handware sample inputs, three components of sample all pixels are subtracted into average, then input network；

Step 3: network training, the network training uses stochastic gradient descent method.

3. a kind of handware defect classifying identification method based on depth convolutional neural networks according to claim 2, its It is characterised by：The stochastic gradient descent method comprises the following steps：

Training sample is inputted network by step a, propagated forward, the propagated forward one by one, by convolutional layer, excitation layer and is divided Class device exports result of calculation, contrasts label, calculates output error

Step b, calculation error is fed back, the error according to step a, calculates the mistake of each layer network from output layer successively forward Difference, according to each layer error, right value update amount is calculated, updates weight w and deviation b；

Step c, after the square-error for obtaining whole sample trainings, summation, then evolution export overall error as network, if Network overall error is more than given threshold, error and counter is recovered into initial value, re -training sample, until error is less than setting Threshold value.

4. a kind of handware defect classifying identification method based on depth convolutional neural networks according to claim 3, its It is characterised by：Assume that shared m is to training sample, each training error in the step a

5. a kind of handware defect classifying identification method based on depth convolutional neural networks according to claim 3, its It is characterised by：The expression formula of the overall error is

6. a kind of handware defect classifying identification method based on depth convolutional neural networks according to claim 2, its It is characterised by：The training network process GPU speed-up computations.

7. a kind of handware defect classifying identification method based on depth convolutional neural networks according to claim 1, its It is characterised by：The structure of depth convolutional neural networks described in step A includes six convolutional layers, six excitation layers, three ponds Layer, a full articulamentum；

Excitation layer is connected behind convolutional layer, a pond layer, the depth can be connect after continuous two convolutional layers and excitation layer Spend last layer of link sort device of convolutional neural networks.

8. a kind of handware defect classifying identification method based on depth convolutional neural networks according to claim 7, its It is characterised by：Each convolutional layer forms by 3 × 3 convolution unit.

9. a kind of handware defect classifying identification method based on depth convolutional neural networks according to claim 1, its It is characterised by：The image collected described in step B is coloured image, and the acquisition mode of image shoots for industrial camera, and will Defect image inputs network.

10. a kind of handware defect classifying identification method based on depth convolutional neural networks according to claim 7, its It is characterised by：The grader is softmax graders.