TW202129593A - Convolutional neural network detection method and system for animals - Google Patents

Abstract

A convolutional neural network detecting system and method for animals are disclosed. The method includes the steps of: generating an image by an image capturing device, and then sequentially performing the first object detection and positioning process on the image by using a processor, and it extracts the first object information and the first sub-image. The first object type identification process extracts the probability values of the plurality of the first object types. If the maximum value of the plurality of the first object types is less than the first predetermined value, performing the second object detection and positioning process on the first sub-image, and it extracts the second object information and the second sub-image. To execute a weighting calculation to the first object type probability value and the second object type probability value, and generating the final object type probability values.

Description

Convolutional neural network detection method and system applied to animals

The present invention relates to a detection method and system of a convolutional neural network, and particularly relates to a detection method and system of a convolutional neural network applied to animals.

The traditional monitoring system broadcasts the field of view to be monitored directly from the monitoring screen, or records it on the hard disk in the form of pictures or videos. This monitoring method cannot provide a warning before the occurrence of the event, and only monitors when a specific event occurs. Only then began to read a large amount of information to do one-by-one comparison, search for interest or specific targets, such as pet dogs or stray dogs. This monitoring method not only wastes a lot of storage space, but also causes a lot of burden on the monitor.

In view of the above-mentioned conventional problems, the present invention provides a convolutional neural network (convolutional neural network) detection method and system applied to animals, which is applied to the positioning and type identification of pet dogs or stray dogs, so as to achieve an automated monitoring system .

The present invention provides a convolutional neural network detection method applied to animals. The method uses an image capture device and a processor. First, an image is generated by an image capture device. Then, the processor performs a first object detection and positioning process on the image, and extracts the first object information and the first sub-image corresponding to the first object information from the image. The processor performs the first object type identification process on the first sub-image to extract a plurality of first object type probability values. If the maximum value of the plurality of first object type probability values is greater than or equal to a first preset value, then Combine the first object information and the maximum value of the probability value of the first object type to generate the first final object information. If the maximum value of the probability values of the plurality of first object types is less than the first preset value, the processor performs a second object detection and positioning process on the first sub-image to extract the second object information and the second sub-image . The processor performs a second object type identification process on the second sub-image to extract a plurality of second object type probability values. The processor performs a weighted calculation on a plurality of first object type probability values and corresponding plural first weight values, and a plurality of second object type probability values and corresponding plural second weight values to generate corresponding pluralities The final object type probability value, and the first object information, the second object information, and the maximum value of the plurality of final object type probability values are combined to generate the second final object information.

Preferably, the first object detection and location process and the second object detection and location process are applied to the detection and location of dogs, and the first object type identification process and the second object type identification process are applied to dog type identification .

Preferably, the convolutional neural network detection method applied to animals further includes the following steps: the processor merges the first final object information or the second final object information into the image to generate the detection image. The above-mentioned image and the detected image are stored by the storage device. The detection image is output by the image output device.

Preferably, the first object detection and location process and the second object detection and location process include five-layer max pooling layer operation, and twenty-two convolution layer operations with five-layer max. The value pooling layer operation, the twenty-two layer convolutional layer operation and the corresponding one of the leaky rectified linear activation function operation (leaky rectified linear activation function).

Preferably, the first object type identification procedure and the second object type identification procedure include a three-layer convolution kernel (Inception) operation, a six-layer convolution layer operation, and a two-layer maximum pooling layer operation.

Preferably, the first object type recognition program and the second object type recognition program include a type recognition method, which includes a recall function, a precision function, and a mean intersection over union function.

Preferably, the first object detection and location process and the second object detection and location process include a loss function.

Preferably, the present invention also provides a convolutional neural network detection system applied to animals. The system can include an image capture device, a processor, a storage device, and an image output device. The image capture device generates an image, and the processor is connected to the image capture device. The processor includes an object detection and locator and an object type recognizer. The object detection and locator receives the image generated by the image capture device and outputs the first object information And the first sub-image, and the object detection and locator inputs the first sub-image and outputs the second object information. The object type identifier receives the first object information and the second object information, and outputs the first final object information and the second final object information. The processor merges the first final object information or the second final object information into the image to generate a detection image. The storage device is connected to the processor, and the storage device stores images and detects images. The image output device is connected to the storage device, and the image output device outputs the detected image.

In summary, the convolutional neural network detection method and system of the present invention can have one or more of the following advantages:

(1) Establish an automated identification system for animals, which can provide real-time information on the location and type of animals to be identified.

(2) Using a single neural network for object detection can achieve a certain degree of effect with less computing resources.

(3) The use of object detection on the whole body and head of the animal to be identified, together with the neural network for identifying the type of animal, can increase the identification success rate.

In order to facilitate the reviewers to understand the technical features, content and advantages of the present invention and the effects that can be achieved, the present invention is described in detail with the accompanying drawings and in the form of embodiment expressions as follows. The drawings used therein are as follows: The subject matter is only for the purpose of illustration and auxiliary description, and may not be the true proportions and precise configuration after the implementation of the invention. Therefore, it should not be interpreted in terms of the proportions and configuration relationships of the accompanying drawings, and should not limit the scope of rights of the invention in actual implementation. Hexian stated.

Please refer to Figure 1, which shows the flow of steps of the convolutional neural network detection method applied to animals according to the embodiment of the present invention. As shown in the figure, the convolutional neural network detection method applied to animals includes the following steps (S1~S6):

Step S1: Use the image capturing device 100 to generate an image 10 for object detection and positioning. The image capturing device 100 can be an electronic device such as a webcam, a digital camera, a smart phone, a driving recorder, and the like, and its image resolution is 416x416.

Please refer to FIG. 2, which is a schematic diagram of an image to be detected of the convolutional neural network detection method applied to animals according to an embodiment of the present invention. As shown in the figure, the image 10 captured by the webcam contains objects to be detected and located, such as dogs 1a and pedestrians 1b.

Step S2: Perform a first object detection and positioning process by the processor 200. The image 10 generated by the above-mentioned image capturing device 100 can be input to the processor 200, and the processor 200 can include a first object detection and positioning process. After the first object detection and positioning process is executed, the first object detection and positioning process can be extracted. An object information 21 and a first sub-image 30 corresponding to the first object information 21. The detailed content of the first object detection and location procedure here will be further described below.

Step S3: Perform the first object type identification process by the processor 200. The first sub-image 30 extracted by the above-mentioned first object detection and positioning process can be input to the processor 200 to perform a first object type identification process. After the first object type identification process is executed, a plurality of first object types can be extracted. The probability value of the object type. If the maximum value among the probability values of the plurality of first object types is greater than or equal to a first preset value, the first object information 21 and the maximum value among the probability values of the plurality of first object types are combined to generate the first final Object information 22. Please refer to FIG. 3, which is a diagram after the first object detection and positioning process and the first object type identification process of the convolutional neural network detection method applied to animals according to the embodiment of the present invention. As shown in the figure, the bounding box representing the first final object information 22 is the combined result of the first object information 21 and the maximum value among the probability values of the plurality of first object types. If the maximum value among the probability values of the plurality of first object types is less than a first preset value, step S4 is performed. The detailed content of the first object type identification procedure here will be further described below.

Step S4: Perform a second object detection and positioning process by the processor 200. The first sub-image 30 extracted by the above-mentioned first object detection and positioning process can be input to the processor 200 to perform the second object detection and positioning process. After the second object detection and positioning process is completed, it can be extracted The second object information 31 and the second sub-image 40 corresponding to the second object information 31. Please refer to FIG. 4, which is a schematic diagram after the second object detection and positioning process of the convolutional neural network detection method applied to animals according to an embodiment of the present invention. As shown in the figure, the frame representing the second object information 31 is the area corresponding to the second object information 31 and the second sub-image 40 after executing the second object detection and positioning process. The second object detection and location process here is similar to the above-mentioned first object detection and location process, and the details will be described further below.

Step S5: Perform a second object type identification process by the processor 200. The second sub-image 40 extracted by the above-mentioned second object detection and positioning process can be input to the processor 200 to perform a second object type identification process. After the second object type identification process is executed, a plurality of second object types can be extracted. The probability value of the object type.

Step S6: Perform weighting calculation by the processor 200. Each probability value of the plurality of first object type probability values corresponds to a first weight value, and each probability value of the plurality of second object type probability values also corresponds to a second weight value , The processor 200 performs a weighting operation on the probability value of the above-mentioned object type and the corresponding weight value to generate a plurality of corresponding probability values of the final object type, and the above-mentioned first object information 21, second object information 31, and a plurality of The maximum value among the probability values of the final object type is combined to generate the second final object information 41. Please refer to FIG. 5, which is a schematic diagram of the convolutional neural network detection method applied to animals after weighting operation according to an embodiment of the present invention. As shown in the figure, the two frames representing the second final object information 41 are the combination of the first object information 21, the second object information 31, and the maximum value among the probability values of the plurality of final object types after the weighting operation. result. The detailed content of the weighting calculation here will be further described below.

The first object detection and location process in the above step S2 is the object detection and location process using the convolutional neural network architecture of YOLO (you only look once), the convolution type of YOLO and other object detection Neural network architectures are different. Most other neural network architectures are composed of multiple neural network architectures, while YOLO uses a single neural network architecture. The image is input from the beginning to the place where the object is generated on the same network. In the road architecture. The concept of YOLO is to divide the image into S × S blocks, each block can generate B predicted object boxes, where the information of the object box is a set of five values (x, y, h, w, confidence), ( x, y) represents the center position of the object frame; (h, w) represents the length and width of the object frame, which is the aspect ratio relative to the entire image. Confidence is the predicted trust score of an object when the center point of an object falls within the block. The calculation method is as follows:

(1)

(1)

When the block has no object center, set Pr(Object) = 0, otherwise, Pr(Object) = 1, in addition to predicting the probability of C categories, and the probability of the category should be divided into training and testing phases to represent . During the training phase, the category probability is expressed as follows:

(2)

(2)

In the testing phase, the trust score is multiplied by the category probability, which is expressed as follows:

(3)

(3)

定義如下：Here

It is defined as follows:

(4)

(4)

After the image 10 input from the image capture device 100 is executed by the processor 200 after the first object detection and positioning process, the frame including the object is extracted, for example, the shape corresponding to the animal's appearance and the category of the object ( class), the information of the object frame can correspond to the first object information 21 of step S2 above, and correspond to the position and length and width of the object border (X, Y, H, W) in the first object information 21 That is the first sub-image 30.

The above-mentioned first object information 21 can generate one or more than one in the first object detection and positioning process, that is, identify more than one target object. The trust probability value of the category of the object can be obtained by the product of three parameters. The first parameter is the probability of the object appearing in the discriminated area, and the second parameter is the object appearing in the discriminated area, and the discrimination is It is the probability of a certain category, and the third parameter is the ratio of the size of the area where the object is identified and the size of the object identified in the area, and the area of the intersection and the area of the union. The object information contained in the first object information 21 is determined by these five elements.

The first object type identification program in the above step S3 is an object type identification program generated by using Google's open source Inception V3 architecture. The first sub-image 30 generated after the processor 200 performs the first object detection and positioning process on the image 10 performs the first object type identification process to extract a plurality of first object type probability values (for example, corresponding to the first object type). The position of the object information, the probability value of identifying a certain species of animal), if the maximum value among the plurality of first object type probability values is greater than or equal to a first preset value (for example, 50%), it is combined with the first An object information 21 can generate the first final object information 22 in step S3, that is, in a certain area of the image 10, it is determined that there is one or more objects, and the object is a certain animal, and further The probability of identifying this certain animal as a certain species is above 50%.

The concept and method of the second object detection and location process in the above step S4 are similar to the first object detection and location process. The difference is that the object executed by the first object detection and positioning procedure is the aforementioned image 10, and the object executed by the second object detection and positioning procedure is the aforementioned first sub-image 30. The first object information 21 extracted by the first object detection and positioning process can correspond to the overall appearance shape of the object to be recognized (for example, corresponding to the overall appearance shape of the dog), and the second object detection and positioning process extracts The second object information 31 obtained may correspond to the head shape of the object to be recognized (for example, corresponding to the appearance shape of the dog's head).

The concept and method of the second object type identification process in step S5 are similar to the first object type identification process. The difference is that the object executed by the first object type identification procedure is the aforementioned first sub-image 30, and the object executed by the second object type identification procedure is the aforementioned second sub-image 40. The plurality of first object type probability values extracted by the first object type identification program can correspond to the use of the overall appearance of the object to be identified to identify the type of the object (for example, use the overall appearance of the dog to identify the type of dog), and The plurality of second object type probability values extracted by the second object type recognition program can correspond to the object type recognition using the head appearance of the object to be recognized (for example, the dog type recognition using the head appearance of the dog) .

Regarding a more detailed description of the weighting operation mentioned in step S6, for example, the method is to use the probability values of the plurality of first object types, the probability value of the first five values with the largest value and its corresponding five A weight value, among the probability values of the plurality of second object types, the probability value of the first five values with the largest value and the corresponding five second weight values are weighted. If a certain object type appears in these two sets of probability values Then, the weighted operation result can be obtained. Finally, from the weighted operation result, the one with the largest probability value and its corresponding object type are selected, and combined with the first object information 21 and the second object information 31 to generate the second final Object information 41.

The above-mentioned first object detection and location process and the second object detection and location process, more specifically, use the convolutional neural network generated by the YOLO framework to detect and locate the image 10 Or the first sub-image 30 extracts the appearance and position of the dog or the appearance and position of the dog's head.

The above-mentioned first object type recognition program and the second object type recognition program, more specifically, use the convolutional neural network generated by Google's open source Inception V3 framework to identify the first sub-image of the object type. 30 or the second sub-image 40, to identify the corresponding breeds of various dogs.

The above-mentioned convolutional neural network detection method applied to animals may further include the following steps in sequence: through the processor 200, the first final object information 22 or the second final object information 41 is merged with the image 10 to generate the detection Measurement image 50. The above-mentioned image 10 and the detected image 50 are stored by the storage device 300. And by the image output device 400, the detection image 50 is output.

The above-mentioned use of the YOLO architecture, although the computing performance is relatively good, but the accuracy part needs to be improved, so the new architecture Darknet-19 is used for feature extraction, which includes 19 convolutional layers and 5 maximum pooling layers , As shown in Table 1, and use IamgeNet's 1000 category data set to pre-train the classifier.

Table 1: Darknet-19 architecture Operation type Number of filters Filter size/step size Output size Level 1 Convolutional layer 32 3 x 3 224x 224 Level 2 Maximum pooling layer 2 x 2 /2 112 x 112 Level 3 Convolutional layer 64 3 x 3 112 x 112 Level 4 Maximum pooling layer 2 x 2 /2 56 x 56 Layer 5 Convolutional layer 128 3 x 3 56 x 56 Level 6 Convolutional layer 64 1 x 1 56 x 56 Layer 7 Convolutional layer 128 3 x 3 56 x 56 Layer 8 Maximum pooling layer 2 x 2/2 28x28 Level 9 Convolutional layer 256 3 x 3 28x28 Layer 10 Convolutional layer 128 1 x 1 28x28 Level 11 Convolutional layer 256 3 x 3 28x28 Level 12 Maximum pooling layer 2 x 2/2 14 x14 13th floor Convolutional layer 512 3 x 3 14 x14 14th floor Convolutional layer 256 1 x 1 14 x14 15th floor Convolutional layer 512 3 x 3 14 x14 16th floor Convolutional layer 256 1 x 1 14 x14 17th floor Convolutional layer 512 3 x 3 14 x14 18th floor Maximum pooling layer 2 x 2/2 7 x 7 Level 19 Convolutional layer 1024 3 x 3 7 x 7 20th floor Convolutional layer 512 1 x 1 7 x 7 Level 21 Convolutional layer 1024 3 x 3 7 x 7 Level 22 Convolutional layer 512 1 x 1 7 x 7 Level 23 Convolutional layer 1024 3 x 3 7 x 7 Level 24 Convolutional layer 1000 1 x 1 7 x 7 Level 25 Average pooling layer Global 1000 Level 26 Classifier (Softmax)

In the actual object detection in this embodiment, firstly remove the 24-26 layers in Darknet-19. These three layers are the convolutional layer for classification, the average pooling layer, and the softmax layer of the classifier, as shown in Table 1. As shown, change to 3 convolutional layers, and change the input image resolution to 416 × 416, as shown in Table 2. The two routing (Route) layer and the reorganization (Reorg) layer in the overall architecture are implemented by dividing the 26 × 26 × 256 feature map output from the 16th layer into four 13 × 13 at the first routing layer The feature map of size × 512 is combined into a feature map of 13 × 13 × 2048 size when the layers are reorganized. Finally, the output of the second routing layer and the 24th layer are combined into a feature map of 13 × 13 × 3072 size, which is In order to allow a more detailed classification of the whole. Table 2 shows the YOLO architecture used in the first object detection and location process and the second object detection and location process.

Table 2: YOLO architecture Operation type Number of filters Filter size/step size Output size Level 1 Convolutional layer 32 3 x 3 416 x 416 Level 2 Maximum pooling layer 2 x 2 /2 208 x 208 Level 3 Convolutional layer 64 3 x 3 208 x 208 Level 4 Maximum pooling layer 2 x 2 /2 104 x 104 Layer 5 Convolutional layer 128 3 x 3 104 x 104 Level 6 Convolutional layer 64 1 x 1 104 x 104 Layer 7 Convolutional layer 128 3 x 3 104 x 104 Layer 8 Maximum pooling layer 2 x 2/2 52 x 52 Level 9 Convolutional layer 256 3 x 3 52 x 52 Layer 10 Convolutional layer 128 1 x 1 52 x 52 Level 11 Convolutional layer 256 3 x 3 52 x 52 Level 12 Maximum pooling layer 2 x 2/2 26 x 26 13th floor Convolutional layer 512 3 x 3 26 x 26 14th floor Convolutional layer 256 1 x 1 26 x 26 15th floor Convolutional layer 512 3 x 3 26 x 26 16th floor Convolutional layer 256 1 x 1 26 x 26 17th floor Convolutional layer 512 3 x 3 26 x 26 18th floor Maximum pooling layer 2 x 2/2 13 x 13 Level 19 Convolutional layer 1024 3 x 3 13 x 13 20th floor Convolutional layer 512 1 x 1 13 x 13 Level 21 Convolutional layer 1024 3 x 3 13 x 13 Level 22 Convolutional layer 512 1 x 1 13 x 13 Level 23 Convolutional layer 1024 3 x 3 13 x 13 Level 24 Convolutional layer 1024 3 x 3 13 x 13 Level 25 Routing layer 16 Level 26 Reorganization layer /2 13 x 13 27th floor Routing layer 26 24 28th floor Convolutional layer 1024 3 x 3 13 x 13 Level 29 Convolutional layer 30 1 x 1 13 x 13

,

,

,

)、信任分數(confidence )與類別的預測值。其中(

,

)為中心座標的偏移量，(

,

)為預測框的x, y 方向邊界偏移量。YOLO將

與

經過Sigmoid函數後得到σ(

)與σ(

)，其值代表在網格(grid cell)內x, y 方向的偏移量，因為Sigmoid的關係，所以預測框的中心位置會約束於網格內部，以防止偏移過多。After the last layer of convolution, YOLO outputs a tensor with a size of 13 × 13 × 30. This tensor represents the prediction information of YOLO. The tensor contains a 13 × 13 vector. Each vector has 5 Anchor Boxes, and each Anchor Box has 6 values which are the value of the object prediction box (

,

,

,

), confidence score ( confidence ) and the predicted value of the category. in(

,

) Is the offset of the center coordinate, (

,

) Is the boundary offset of the prediction box in the x and y directions. YOLO will

and

After the Sigmoid function, σ(

) And σ(

), its value represents the offset in the x and y directions within the grid cell. Because of the Sigmoid relationship, the center position of the prediction box will be constrained inside the grid to prevent excessive offset.

The above-mentioned convolutional layer operation refers to the process of sliding on the image 10 by the filter set in the convolutional layer, thereby extracting the features of the image 10 (for example, circles, lines, triangles, etc.). The filter size in Table 1 (for example, 1x1, 2x2, and 3x3) refers to the resolution size of the corresponding input image resolution (416x416), and the step size refers to the corresponding input image resolution, how many slides each time Units (for example, 1 or 2) are used to extract the features of the image, and the convolutional layer operation can be optimized by training. The number of convolutional layers required and the number of filters set in the convolutional layer depends on the type of object to be detected and the complexity or number of features corresponding to the object in the image 10 above. Although the number of filters is intuitively determined The more you can capture the features of the object more accurately, but the complexity of the program and the amount of calculation are also greatly increased, so you also need to choose an appropriate composition.

Then, the above-mentioned maximum pooling layer operation refers to the image after the previous layer operation (convolutional layer operation in Table 1) is executed, and the value with the maximum value in the output filter is then set After the required step size, a compressed and reduced image can be obtained.

When outputting information, each volume base layer and maximum pooling layer will use a leaky linear rectification activation function to multiply some data with a small value by a corresponding small weight value.

Since some of the characteristics of dogs are in the whole body area, and some of the characteristics of dogs are in the head area, it is inevitable that the recognition will be impossible if the recognition is performed from certain parts. Therefore, this disclosure will use the YOLO object detection method to detect the whole body region image and the dog head region image, and then use the convolutional neural network to perform feature extraction respectively, and finally the two feature vectors are merged and then Classification. In order to accurately extract the features in the image, this research uses the InceptionV3 network architecture as the feature extraction architecture. The Inception architecture is Google’s open source convolutional network model. The Inception architecture is mainly composed of convolutional layers in parallel. Most of the convolutional network models are combined in series. This series usually involves There are three problems: 1. There are too many parameters and it is easy to overfit; 2. The calculation is complicated and difficult to apply; 3. The gradient disappears easily. So Google proposed the GoogLeNet architecture (InceptionV1) that uses multiple sizes of convolution kernels to increase the depth and width of the network and improve the performance of deep neural networks. It includes 1 × 1 convolution kernel and 3 × 3 convolution kernel. , 5 × 5 convolution kernel and 1 × 1 pooling layer of two Inception modules combined. Two auxiliary classifiers are added to the architecture. This classifier is only used in the training phase, mainly to avoid the problem of gradient disappearance when the network architecture is too deep.

倍，而利用卷積分解可以在不影響辨識率下，達到減少參數量的結果，其操作方式有：1. 將Inception模組中的5 × 5卷積核替換成連續2個3 × 3卷積核；2. 將n × n尺寸的卷積分解成n × 1尺寸與1 × n尺寸卷積核；3. 3 × 3尺寸的卷積分解成3 × 1尺寸與1 × 3尺寸卷積核。除了運用卷積分解之外，在利用池化層降低維度時，由於特徵圖的大小急劇縮減，在模型訓練時造成特徵表達的瓶頸，如果將池化層與Inception模組的順序顛倒，雖然可以解決此問題，但還是就無法減少參數量。所以， InceptionV3中將池化層融合至Inception的模組。In order to improve the internal covariate shift (Internal covariate shift) problem, the usual way to improve is to adjust the learning rate (learning rate) to be smaller. However, if the learning rate is too small, the convergence speed will be too slow and the training time will be too slow. Elongated. Therefore, the proposed batch normalization (BN) method is added to GoogLeNet. This method is a normalization method that can speed up training and improve the accuracy of classification. Batch normalization is applied at each network layer, and each output data is normalized. In order to further improve the efficiency of calculation, Factorizing Convolutions is used in the InceptionV3 architecture. The convolution solution is to solve the problem of too large calculation amount caused by the convolution kernel with too large size. In the case of 5 × 5 convolution, the parameter amount is 3 × 3 convolution

However, the use of volume integral solution can reduce the number of parameters without affecting the recognition rate. The operation methods are as follows: 1. Replace the 5 × 5 convolution kernel in the Inception module with two consecutive 3 × 3 volumes Convolution kernel; 2. Solve the n × n size convolution kernel into n × 1 size and 1 × n size convolution kernel; 3. 3 × 3 size convolution into 3 × 1 size and 1 × 3 size convolution nuclear. In addition to using the volume integration solution, when the pooling layer is used to reduce the dimensionality, the size of the feature map is sharply reduced, which causes a bottleneck in feature expression during model training. If the order of the pooling layer and the Inception module is reversed, it is possible Solve this problem, but still unable to reduce the amount of parameters. Therefore, the pooling layer is integrated into the Inception module in InceptionV3.

In addition to the aforementioned improvements, three new convolution modules, 8×8, 17×17, and 35×35, have been added to the architecture. The convolution solution is used to decompose the n×n convolution kernel into two n × 1 and 1 × n convolution kernel modules to increase network depth and reduce operations, and change the input image resolution from 224 × 224 to 299 × 299. Table 3 below shows the complete architecture of InceptionV3.

Table 3: Inception V3 architecture Operation type Filter size/step size Input size Level 1 Convolutional layer 3 x 3 /2 299 x 299 x 3 Level 2 Convolutional layer 3 x 3 /1 149 x 149 x 32 Level 3 Convolutional layer pad zero 3 x 3 /1 147 x 147 x 32 Level 4 Pooling layer 3 x 3 /2 147 x 147 x 64 Layer 5 Convolutional layer 3 x 3/ 1 73 x 73 x 64 Level 6 Convolutional layer 3 x 3 /2 71 x 71 x 80 Layer 7 Convolutional layer 3 x 3 /1 35 x 35 x 192 Layer 8 3 x Inception 35 x 35 x 288 Level 9 3 x Inception 17 x 17 x 768 Layer 10 3 x Inception 8 x 8 x 1280 Level 11 Pooling layer 8 x 8 8 x 8 x 2048 Level 12 Linear layer (linear) Auxiliary classification 1 x 1 x 2048 13th floor Classifier (Softmax) Classification 1 x 1 x 1000

(5) 其中Gt_i 表示第i張欲辨識種類的影像的真實區域，Pi為第i張測試影像預測的物件區域的結果，N為測試總張數，Gt_i 與P_i 的交集比例除以P_i 即為對應的召回率。精確率函數可表示成：

(6) 與召回率函數不同的是，精確率函數是利用欲辨識影像的像素Gt_i 中有多少像素被正確偵測的像素P_i ，其交集比例除以P_i 即為對應的精確率。平均交並比函數可以表示成：

(7) 平均交並比函數是利用欲辨識種類的影像的真實區域Gt_i 與預測的物件區域P_i 的交集區域，除以欲辨識種類的影像的真實區域Gt_i 與預測的物件區域P_i 的聯集區域即為對應的平均交並比。判定為辨識成功的方式為若在這三種計算方式中的其中一種得到機率值大於0.5以上，即為辨識成功。以這三種方式來計算辨識的正確率會有較高的可信度。In the above-mentioned first object type identification program and the second object type identification program, the identification methods include a recall function, a precision function, and an average cross-to-comparison function. The recall function can be expressed as:

(5) where _i denotes the i Gt sheet to be real areas of the image recognition type, Pi is the i-th sheet test results prediction object image area, N is the total number of test sheets, the intersection of the _i P _i ratio Gt divided P _{i is} the corresponding recall rate. The precision rate function can be expressed as:

(6) Different from the recall rate function, the accuracy rate function uses the _{number of pixels P i that are correctly detected among the pixels Gt i} of the image to be recognized. The intersection ratio divided by P _{i is} the corresponding accuracy rate. The average intersection ratio function can be expressed as:

(7) The average ratio of the cross and the area of intersection is the use of the function of the image area to be true of Gt _i identifies the type of the predicted object region and the P _i, to be divided by the true type of the image area identification Gt _i and the predicted object region P _i The union area of is the corresponding average intersection ratio. The way to determine that the identification is successful is if the probability value obtained in one of the three calculation methods is greater than 0.5, the identification is successful. Using these three methods to calculate the correct rate of recognition will have a higher credibility.

(8)In the first object detection and location process and the second object detection and location process mentioned above, in order to correctly extract the required information, it is necessary to include the corresponding loss function for training. The corresponding loss function is listed here:

(8)

可預測邊框之X座標、

可預測邊框之Y座標、

可預測邊框之寬度、

可預測邊框之高度、

可判斷第(i,j)個區域(例如，將影像10或第一子影像30分割成多個區域)中是否有物件(若有物件，則

之值為1，若無則為0)、

可判斷第i個物件框中是否有物件(若有物件，則

之值為1，若無則為0)。其中，j之數值與參數B相關，參數B則為第一物件偵測及定位程序或第二物件偵測及定位程序中，每個區域之中可偵測之邊框數量，i之數值與參數s相關，參數s對應影像10及第一子影像30之解析度(例如，416x416或299x299)。The λcoord and λnoobj mentioned in the above loss function represent the weighting factor of the center point coordinates (X, Y) of the frame, the weighting factor of the width and height of the frame (W, H), and the weighting factor of the frame category. , Using parameters such as the center point coordinates of the frame, the width of the frame, and the type of the frame, the Confidence score of the frame can be obtained. The detailed correspondence is defined as

The X coordinate of the predictable frame,

The Y coordinate of the frame can be predicted,

Predictable border width,

The height of the frame can be predicted,

It can be judged whether there is an object in the (i, j)th area (for example, the image 10 or the first sub-image 30 is divided into multiple areas) (if there is an object, then

The value is 1, or 0 if there is no),

Can judge whether there is an object in the ith object box (if there is an object, then

The value is 1, or 0 if there is no). Among them, the value of j is related to parameter B, and parameter B is the number of detectable frames in each area in the first object detection and positioning process or the second object detection and positioning process, the value of i and the parameter s is related, and the parameter s corresponds to the resolution of the image 10 and the first sub-image 30 (for example, 416x416 or 299x299).

XGBoost (eXtreme Gradient Boosting) is a classifier based on an improved extension of the Gradient Boosted Decision Tree (GBDT) algorithm. It has been widely used in the past to solve the problem of supervised learning. Gradient Boosted Decision Tree is a model formed by using a collection of decision tree prediction models. Simply put, it uses multiple classification model results to combine into the final classification result. This model can solve regression and classification problems. The objective function of XGBoost is expressed as follows:

(9)

(9)

在XGBoost中代表的是懲罰項，由於XGBoost的目標在不犧牲精確度的情況下提升效能，而設置懲罰項的目的是為了降低模型複雜度。

XGBoost represents the penalty term. Since XGBoost's goal is to improve performance without sacrificing accuracy, the purpose of setting the penalty term is to reduce the complexity of the model.

Since convolutional neural network training requires a large amount of and various image data to accurately train the neural network model, XGBoost does not require a large amount of data during training, but it must first be manually defined. train. In order to take into account the fact that it is quite difficult to collect a large number of images of dog species in practical applications, and the hardware resources are limited. Therefore, using the InceptionV3 model that has been trained using the data set, it is divided into two processing flows: feature extraction and classification processing. The feature extraction processing is between the input layer and the classification layer; the classification processing is in the classification layer. InceptionV3's feature extraction processing system extracts 2048-dimensional feature vectors from images, and finally combines them with XGBoost for classification.

Because there are some types of dogs, the characteristics are concentrated on the head, and some dogs are distributed on the body part. Therefore, in order to accurately identify the type, the feature information of the head and the body part is fused, and then the type is identified. First, the whole body area image is detected by YOLO. After the head area image is obtained, InceptionV3 is used for feature extraction to obtain the 2048-dimensional head feature vector. Then, the 2048-dimensional whole body feature vector and the 2048-dimensional head feature vector are merged into a 4096-dimensional feature vector, and finally XGBoost is used for classification to obtain the result of category identification.

The above-mentioned image recognition can be used for dog registration and management. Its purpose is to know the relevant information of the dog, such as whether there is a owner, the time and place of vaccination, whether ligation, etc., in addition to the recognition through the chip and the nose pattern. , The image acquisition is relatively easy to achieve, and the resolution does not need to be too high, only the facial features are obvious, for example, the face must have at least two eyes.

Since dogs cannot be fixed when shooting, unlike humans, photos of dogs are often skewed. In this case, the accuracy of identification will be low. Therefore, when performing face recognition, first use YOLO to detect the position of the eyes, make the centers of the two eyes connected in a straight line, and calculate the slope m of this straight line. Knowing that m = tan(θ), you can use the arctangent function Arctan obtains the angle θ with the x-axis (horizontal axis), and then rotates the eyes parallel to the horizontal axis after knowing the angle θ. In addition, identification is to obtain the identity information of the dog. If there is unregistered image data in the database, the classifier cannot be used directly, because the classifier will still classify the image when it encounters an image that has not been seen before. To the most similar category. Therefore, InceptionV3 is used for feature extraction, and after the feature vector is obtained, the similarity calculation is used to determine whether it is similar or not by comparing with the threshold. The similarity is expressed as follows:

(10)

(10)

Refer to FIG. 6, which is a schematic diagram of a convolutional neural network detection system 5 applied to animals according to an embodiment of the present invention. As shown in the figure, it includes an image capture device 100, a processor 200, a storage device 300, and an image output device 400. The animal-applied convolutional neural network detection system 5 can perform the above-mentioned animal-applied convolutional neural network detection method (steps S1 to S6). In other words, the convolutional neural network detection system 5 applied to animals has components corresponding to steps S1 to S6.

The image capturing device 100 of the convolutional neural network detection system 5 applied to animals generates an image 10, which corresponds to step S1 in FIG. 1.

The processor 200 in FIG. 6 may include sub-components such as an object detection and locator 201 and an object type recognizer 202. The processor 200 can correspondingly perform steps S2 to S6 in FIG. 1.

More specifically, the object detection and locator 201 receives the image 10, and after performing the first object detection and positioning process, extracts the first object information 21 and the corresponding first sub-image 30. The object detection and locator 201 receives the first sub-image 30, and after performing the second object detection and positioning process, extracts the second object information 31 and the corresponding second sub-image 40.

More specifically, the object type recognizer 202 receives the first sub-image 30 and extracts the first final object information 22. The object type identifier 202 receives the second sub-image 40 and extracts a plurality of second final object information 41.

More specifically, the processor 200 can merge the first final object information 22 or the second final object information 41 into the image 10 to output the detection image 50.

More specifically, the storage device 300 is connected to the processor 200, and the storage device 300 can store the above-mentioned image 10 and the detected image 50. The image output device 400 is connected to the storage device 300, and the image output device 400 can output the detected image 50.

The above descriptions are merely illustrative and not restrictive. Any equivalent modifications or alterations that do not depart from the spirit and scope of the present invention should be included in the scope of the appended patent application.

1a: Dog 1b: pedestrian 5: Convolutional Neural Network Detection System 10: Image 21: First object information 22: First final object information 30: The first sub-image 31: Second object information 40: Second sub image 41: The second final object information 50: Detect image S1~S6: steps 100: Image capture device 200: processor 201: Object detection and locator 202: Object Type Identifier 300: storage device 400: display device

Figure 1 is a flow chart of the steps of the convolutional neural network detection method applied to animals according to an embodiment of the present invention. FIG. 2 is a schematic diagram of an image to be detected in the convolutional neural network detection method applied to animals according to an embodiment of the present invention. FIG. 3 is a diagram after the first object detection and positioning process and the first object type identification process of the convolutional neural network detection method applied to animals according to the embodiment of the present invention. FIG. 4 is a schematic diagram after the second object detection and positioning process of the convolutional neural network detection method applied to animals according to the embodiment of the present invention. FIG. 5 is a schematic diagram of the convolutional neural network detection method applied to animals after weighting operation according to an embodiment of the present invention. Figure 6 is a schematic diagram of a convolutional neural network detection system applied to animals according to an embodiment of the present invention.

S1~S6: steps

Claims (8)

A convolutional neural network detection method applied to animals, which includes the following steps: Generating an image by an image capturing device; Performing a first object detection and positioning process on the image by a processor, extracting a first object information from the image and a first sub-image corresponding to the first object information; The processor performs a first object type recognition process on the first sub-image to extract a plurality of first object type probability values, if the maximum value of the plurality of first object type probability values is greater than or equal to a first predetermined value If the value is set, the first object information and the maximum value of the probability value of the first object type are combined to generate a first final object information; If the maximum value of the probability values of the plurality of first object types is less than the first preset value, a second object detection and positioning process is performed on the first sub-image by the processor, from the first sub-image Extract a second object information and a second sub-image; Performing a second object type identification process on the second sub-image by the processor to extract a plurality of second object type probability values; and The processor performs a weighted calculation on the plurality of first object type probability values and corresponding plurality of first weight values, and the plurality of second object type probability values and corresponding plurality of second weight values to generate Corresponding probability values of a plurality of final object types, and merge the first object information, the second object information, and the maximum value of the probability values of the plurality of final object types to generate a second final object information; The convolutional neural network detection method applied to animals as described in item 1 of the scope of patent application, wherein the first object detection and positioning procedure and the second object detection and positioning procedure are applied to the detection of dogs and Positioning, and the first object type identification procedure and the second object type identification procedure are applied to dog type identification. The convolutional neural network detection method applied to animals as described in item 1 of the scope of patent application further includes the following steps: By the processor, merge the first final object information or the second final object information into the image to generate a detection image; Storing the above-mentioned image and the detected image by a storage device; and An image output device is used to output the detected image. The convolutional neural network detection method applied to animals as described in item 1 of the scope of patent application, wherein the first object detection and location procedure and the second object detection and location procedure include five-layer maximum pooling Layer operation, 22-layer convolutional layer operation and corresponding one of the linear rectification activation function operation with leakage. The convolutional neural network detection method applied to animals as described in item 1 of the scope of patent application, wherein the first object type identification process and the second object type identification process include a three-layer convolution kernel operation and a six-layer convolution Multi-layer operation and two-layer maximum pooling layer operation. The convolutional neural network detection method applied to animals as described in item 3 of the scope of patent application, wherein the first object type identification process and the second object type identification process include a class identification method, and the type identification method includes A recall function, a precision function, and an average intersection and ratio function. In the convolutional neural network detection method applied to animals as described in claim 1, wherein the first object detection and location process and the second object detection and location process include a loss function. A convolutional neural network detection system applied to animals, which includes: An image capture device that generates an image; A processor is connected to the image capturing device and includes: An object detection and locator receives the image generated by the image capturing device, outputs a first object information and a first sub-image, inputs the first sub-image, and outputs a second object information; An object type identifier that receives the first object information and the second object information, and outputs a first final object information and a second final object information; Wherein, the processor merges the first final object information or the second final object information into the image to output a detection image; A storage device connected to the processor to store the image and the detected image; and An image output device is connected to the storage device to output the detected image.

Images