CN108846415A - The Target Identification Unit and method of industrial sorting machine people - Google Patents

Abstract

The invention relates to a target recognition device and method for a sorting robot. The device includes a camera, a processor and a memory, and also includes a program for performing the following steps: obtaining image information of the target to be measured through the camera; loading a convolutional neural network Framework and its training model; use several candidate anchorboxes to generate a bounding box in the feature map; predict the category and category score corresponding to the bounding box; obtain the bounding box with the largest category score through non-maximum value suppression. The training model of loading the convolutional neural network is a program of the following steps: pre-training the network; obtaining the label data set of the target object; performing cluster analysis on the target frame in the label data set; obtaining each layer by forward propagation The output of the network is obtained to obtain the error between the output and the label; the gradient of the weights and biases of each layer is reversely calculated according to the error, and the weights and biases of each layer are adjusted. The detection speed of the invention is very fast, and real-time performance can be realized while ensuring accuracy.

Description

Object recognition device and method for industrial sorting robot

technical field

The invention relates to an industrial sorting robot, in particular to an object recognition device and method for the industrial sorting robot.

Background technique

Nowadays, automated production is becoming more and more popular, robots are widely used in various industrial assembly lines, and mechanical sorting of workpieces is a common task in industrial processes. The identification and positioning of moving objects is the basis of sorting operations. Most of the early sorting robots used the method of teaching programming. Although they can complete some fixed operations, they cannot complete more intelligent operations; while general machine vision technology , such as pure edge extraction, template matching, image enhancement and other processing technologies, although the target object can be detected, it is difficult to accurately sort the target object in a complex scene.

Convolutional neural network (CNN) is mainly used to identify two-dimensional graphics with distortion invariance. It is a mathematical model that can simulate animal brains. As long as the convolutional network is trained with known patterns, it can learn a large number of The mapping relationship between input and output has the ability to map between input and output pairs. Using the convolutional neural network algorithm for sorting robots can solve the need for sorting accuracy in complex scenes, but when there are many distracting objects and high density in the scene, multiple target objects need to be processed at the same time, and It is easily affected by factors such as light, and the speed of target recognition and positioning is slow, which is prone to missed and false detections, and cannot meet the real-time requirements of the gripper at the end of the machine for sorting.

This is the weak point of prior art.

Contents of the invention

Aiming at the deficiencies of the prior art, the technical problem to be solved by the present invention is to provide an object recognition device and method for an industrial sorting robot, which can help realize real-time detection of object objects.

The target identification device of industrial sorting robot of the present invention comprises camera, processor and memory, it is characterized in that: this device also comprises the program that carries out following steps:

Obtain the picture information of the target to be tested through the camera;

Load the convolutional neural network framework and its training model to obtain several candidate anchor boxes;

Use anchor boxes to generate several bounding boxes in the feature map;

Predict the corresponding categories and category scores for the above-mentioned several bounding boxes;

The bounding box with the largest category score is obtained by non-maximum value suppression, and its corresponding category is used as the category of the target.

The training model of the described loading convolutional neural network is the program of following steps:

First, pre-train the network on the ImageNet dataset to obtain the pre-training model of the network;

Then obtain the data set of the target object through the camera, perform manual labeling, and obtain the label data set;

The width and height of the target box in the label data set are clustered and analyzed by k-means, and several sets of candidate anchor boxes are obtained;

Obtain the network output of each layer through forward propagation, and obtain the error between the output and the label;

Calculate the gradients of the weights and biases of each layer in reverse order according to the error, and adjust the weights and biases of each layer.

The k=2.

In pre-training, multi-scale training is used, and the size of the input image is changed every 10 rounds.

In the pre-training, the size of the input image is kept between 320 and 608 with 32 as the downsampling factor.

The convolutional neural network is an improvement based on Darkne-19. The last convolutional layer is removed, and 3 convolutional layers with a kernel size of 3x3 and a channel number of 1024 are added, and a convolutional layer A convolutional layer with a kernel size of 1x1.

The target identification method of industrial sorting robot of the present invention is characterized in that comprising the steps:

Obtain the picture information of the target to be tested through the camera;

Load the convolutional neural network framework and its training model to obtain several candidate anchor boxes;

Use anchor boxes to generate several bounding boxes in the feature map;

Predict the corresponding categories and category scores for the above-mentioned several bounding boxes;

The bounding box with the largest category score is obtained by non-maximum value suppression, and its corresponding category is used as the category of the target.

The training model of described loading convolutional neural network is the following steps:

First, pre-train the network on the ImageNet dataset to obtain the pre-training model of the network;

Then obtain the data set of the target object through the camera, perform manual labeling, and obtain the label data set;

The width and height of the target box in the label data set are clustered and analyzed by k-means, and several sets of candidate anchor boxes are obtained;

Obtain the network output of each layer through forward propagation, and obtain the error between the output and the label;

Calculate the gradients of the weights and biases of each layer in reverse order according to the error, and adjust the weights and biases of each layer.

The beneficial effects of the present invention are as follows: firstly, the model trained by the convolutional neural network can identify and locate the target in complex scenes, and since the whole picture is used to train the model, the way of the whole target detection is a single volume Integrated neural network, the detection performance can be optimized end-to-end, so the detection speed is very fast, and the real-time requirements can be achieved while ensuring the accuracy. Second, the convolutional neural network of the present invention is an improvement on the basis of the basic network Draknet-19, adding 3 convolution kernels of 3*3, a convolution layer with a channel number of 1024, and a convolution kernel size It is a 1*1 convolutional layer. While deepening the network, it reduces the training parameters of the network, enables the network to extract more abundant feature information, and improves the accuracy of the network's recognition of target objects. Third, due to the multi-scale training of the network, the network is robust to input images of different sizes. Finally, by performing cluster analysis on the manually marked bounding boxes in the data set, the statistical rules of the bounding boxes are found, and it is found that when k=2, it has a better sorting effect.

Description of drawings

Fig. 1 is a flow chart of the object detection of the industrial sorting robot of the present invention.

Fig. 2 is a flow chart of the convolutional neural network training stage of the industrial sorting robot of the present invention.

Fig. 3 is a network structure table of the convolutional neural network of the industrial sorting robot of the present invention.

Fig. 4 is a relationship diagram between the number of iterations and the loss function of the convolutional neural network training of the present invention.

Fig. 5 is a graph of k value and cost function of the industrial sorting robot of the present invention.

Detailed ways

The present invention will now be described in further detail in conjunction with the accompanying drawings and embodiments.

The invention applies the improved YOLOv2 convolutional neural network algorithm to the sorting robot for processing parts, and realizes the sorting operation based on machine vision. The system obtains picture information through the camera on the conveyor belt, identifies and locates the target parts in real time from different types of parts, and grabs them through the gripper at the end of the machine to achieve neat placement.

Referring to Figure 1 and Figure 2, the YOLOv2 convolutional neural network algorithm treats the target detection problem as a regression problem, and can predict the position and category of multiple target borders in real time at one time, and complete the positioning of the target object in one network at the same time And classification, the whole detection process is completely unified.

Due to relying on the grid to generate the bounding box, the recall rate is low. YOLOv2 uses the k-means method to perform cluster analysis on the manually marked bounding boxes in the data set, find the statistical rules of the bounding boxes, and use these types of wide and high anchor boxes in the feature Bounding boxes are generated in the graph, which improves the recall and localization accuracy.

Multiple bounding boxes may be generated for a target object, and each bounding box contains variables such as x, y, h, w and confidence scores and category scores; where x, y represent the center coordinates of the bounding box relative to the origin of the coordinates, w, h represents the width and height of the bounding box relative to the entire image; the confidence score reflects the accuracy of the bounding box prediction;

It is necessary to leave the bounding box with the highest confidence score through non-maximum suppression to achieve precise positioning of the object;

By inputting the feature information extracted by the convolutional layer into the softmax classifier, for the category probabilities of multiple objects predicted by each bounding box, the prediction with the highest category probability score is also left through non-maximum value suppression to achieve classification of objects;

The YOLO v2 convolutional neural network algorithm is used for the sorting of machined parts. Since the network is not trained by sliding windows or candidate regions, the entire image is directly used to train the model. The entire target detection method is a single With the convolutional neural network, the detection performance can be optimized end-to-end, so the detection speed is very fast, and the real-time requirements can be achieved while ensuring the accuracy.

Due to the small amount of label data and often low resolution, if the produced label data set is directly used for network training, the accuracy is often low and the positioning is poor. Therefore, in the actual training, the pre-training method is used. The ImageNet data set has more than 14 million pictures, covering more than 20,000 categories. The images are clear and high-resolution, and most of the pictures have clear category label information. Therefore, pre-training on the ImageNet data set can get the network Pre-training model to enable the network to obtain a general understanding of the target object;

Then obtain the data set of the target object through the camera, manually mark it, and obtain the label data set; for example, use the 1000 hand-labeled nuts and gasket data sets we collected to retrain the network, and then adjust the width and height of the label data set K-means is used for cluster analysis, and the optimal k value is obtained through the relationship between the k value and the cost function. As shown in Figure 5, when k=2, the sorting effect is better. Then cluster analysis is performed using the optimal k value to obtain several sets of anchor boxes.

The training phase of the network is mainly divided into forward propagation and back propagation. The forward propagation is mainly to calculate the output value of each layer in turn, and the back propagation is mainly to calculate the gradient of the weight and bias of each layer in reverse order based on the error, and After the calculation is complete, adjust the weights and biases of each layer. In the training process of the network, the network output of each layer is first obtained through forward propagation, and then the error between the output and the label is obtained. Through back propagation, the parameters are continuously updated to reduce the loss function until the network completely converges. By visualizing the loss function during the training process of the convolutional neural network, analyze whether the model training is convergent and whether the accuracy meets the preset requirements.

Referring to Figure 4, when the number of training iterations reaches 40,000, the training model has converged, and the average loss function reaches 1.0. The trained model can accurately identify and locate target objects in complex scenes.

The trained model reaches a speed of 20 frames per second on the conveyor belt, which meets the real-time requirements. Through the test of a single picture, it can be seen that for any input picture containing the target object, the model trained by the convolutional neural network can detect the target object in complex scenes, and can accurately classify and predict the location.

In order to make the network robust to pictures of different input sizes, a multi-scale training method is adopted, and the size of the input pictures is changed every 10 rounds of training. During the training process, the size of the input image is kept between 320 and 608 with a downsampling factor of 32.

Referring to Fig. 3, the network structure of the present invention is realized by a convolutional neural network (CNN). CNN simulates feature distinction through convolution, and reduces the order of magnitude of network parameters through convolution weight sharing and pooling. Finally, through traditional Neural networks perform tasks such as classification. The basic model Darkne-19 based on YOLO has 19 convolutional layers, the first layer is a convolutional layer, the number of channels is 32, and the size of the convolution kernel is 3x3; the second layer is the maximum pooling layer, and the size of the convolution kernel is 2x2, the step size is 2; the third layer is a convolutional layer, the number of channels is 64, and the convolution kernel size is 3x3; the fourth layer is the maximum pooling layer, the convolution kernel size is 2x2, and the step size is 2; the fifth , The sixth and seventh layers are convolutional layers, the convolution kernel sizes are 3x3, 1x1, 3x3, and the number of channels is 128, 64, and 128 respectively; the eighth layer is the maximum pooling layer, the convolution kernel size is 2x2, and the step size is 2; the ninth, tenth, and eleventh are convolution layers, the convolution kernel sizes are 3x3, 1x1, 3x3, and the number of channels is 256, 128, and 256 respectively; the twelfth layer is the largest pooling layer, and the convolution kernel The size is 2x2, the step size is 2; the 13th, 14th, 15th, 16th, and 17th layers are convolution layers, the convolution kernel sizes are 3x3, 1x1, 3x1, 1x1, 3x3, and the number of channels is 512 , 256, 512, 256, 512; the eighteenth layer is the maximum pooling layer, the convolution kernel size is 2x2, and the step size is 2; is a convolution layer, the convolution kernel sizes are 3x3, 1x1, 3x3, 1x1, 3x3, and the number of channels is 1024, 512, 1024, 512, 1024 respectively; the twenty-fourth layer is a convolution layer, and the convolution kernel size is 1x1, the number of channels is 1000. In Darkne-19, a large number of convolutional cascade structures are used. The size of the convolution kernel mainly includes two sizes of convolution kernels: 3×3 and 1×1. Drawing on the idea of Network in network, a 1x1 convolution kernel is added between the 3x3 convolution kernels. The convolution layer is responsible for extracting the feature information of the target object, and the maximum pooling layer extracts the key information in the target object. , reducing redundant information and reducing network training parameters.

This article improves on the basis of Darkne-19, removes the last convolutional layer, adds 3 convolutional layers with a kernel size of 3x3, and a convolutional layer with a channel number of 1024, and a convolutional layer with a kernel size of 1x1 Floor. While deepening the network, the training parameters of the network are reduced, so that the network can extract more abundant feature information, and the accuracy of the network's recognition of the target object is improved. Use anchor boxes to predict bounding boxes on feature maps.

The above descriptions are only preferred embodiments of the present invention, and do not limit the present invention in any form. Although the present invention has been disclosed as above with preferred embodiments, it is not intended to limit the present invention. Anyone familiar with this professional technology Personnel, without departing from the scope of the technical solution of the present invention, when the technical content disclosed above can be used to make some changes or modifications to equivalent embodiments with equivalent changes, but all the content that does not depart from the technical solution of the present invention, according to the present invention Any simple modifications, equivalent changes and modifications made to the above embodiments by the technical essence still belong to the scope of the technical solutions of the present invention.

Claims (8)

1. A target recognition device of an industrial sorting robot, comprising a camera, a processor and a memory, is characterized in that: the device also includes a program that performs the following steps: Obtain the picture information of the target to be tested through the camera; Load the convolutional neural network framework and its training model to obtain several candidate anchor boxes; Use anchor boxes to generate several bounding boxes in the feature map; Predict the corresponding categories and category scores for the above-mentioned several bounding boxes; The bounding box with the largest category score is obtained by non-maximum value suppression, and its corresponding category is used as the category of the target. 2. the target identification device of industrial sorting robot as claimed in claim 1, is characterized in that: the training model of described loading convolutional neural network is the program of following steps: First, pre-train the network on the ImageNet dataset to obtain the pre-training model of the network; Then obtain the data set of the target object through the camera, perform manual labeling, and obtain the label data set; The width and height of the target box in the label data set are clustered and analyzed by k-means, and several sets of candidate anchor boxes are obtained; Obtain the network output of each layer through forward propagation, and obtain the error between the output and the label; Calculate the gradients of the weights and biases of each layer in reverse order according to the error, and adjust the weights and biases of each layer. 3. The object recognition device of an industrial sorting robot according to claim 2, characterized in that: said k=2. 4. The object recognition device of an industrial sorting robot according to claim 2, characterized in that: in the pre-training, multi-scale training is adopted, and the size of the input picture is changed every 10 rounds. 5. The object recognition device of an industrial sorting robot according to claim 4, characterized in that: in the pre-training, 32 is used as the downsampling factor to keep the size of the input picture between 320 and 608. 6. The object recognition device for industrial sorting robots according to any one of claims 1 to 5, characterized in that: the convolutional neural network is an improvement on the basis of Darkne-19, and the last convolutional layer is removed , adding three convolutional layers with a kernel size of 3x3 and a channel number of 1024, and a convolutional layer with a kernel size of 1x1. 7. A target recognition method for an industrial sorting robot, characterized in that it comprises the steps: Obtain the picture information of the target to be tested through the camera; Load the convolutional neural network framework and its training model to obtain several candidate anchor boxes; Use anchor boxes to generate several bounding boxes in the feature map; Predict the corresponding categories and category scores for the above-mentioned several bounding boxes; The bounding box with the largest category score is obtained by non-maximum value suppression, and its corresponding category is used as the category of the target. 8. the target recognition method of industrial sorting robot as claimed in claim 7 is characterized in that: the training model of described loading convolutional neural network is the following steps: First, pre-train the network on the ImageNet dataset to obtain the pre-training model of the network; Then obtain the data set of the target object through the camera, perform manual labeling, and obtain the label data set; The width and height of the target box in the label data set are clustered and analyzed by k-means, and several sets of candidate anchor boxes are obtained; Obtain the network output of each layer through forward propagation, and obtain the error between the output and the label; Calculate the gradients of the weights and biases of each layer in reverse order according to the error, and adjust the weights and biases of each layer.