TW202022632A - Customizable method for detecting specific style objects - Google Patents

Abstract

Basically, a computer program is used to synthesize a specific style object image sample or a specific style object image sample is generated by using a video screenshot or image samples of specific style objects in the image database of various specific style objects prepared in the past are used for learning by a streamlined and dedicated CNN image classifier in order to enable classification and identification of objects of a specific style; further, a CNN image object detector having completed training is generally used to detect the image area containing objects in an image, and the screenshot is sent to the previously trained special CNN image classifier to determine whether it is a specific style object so as to achieve detection of specific style objects. In particular, the specific style training samples of each object in the database can be collected cumulatively and continuously, and more and more suitable training samples can be provided for the future to be selected immediately when required, such that the training of the proprietary CNN image classifier can be completed more quickly, especially for the detection and identification of customized specific style objects.

Description

Customizable methods for detecting objects in specific styles

The present invention relates to a customizable method for detecting objects of specific styles, in particular to training a convolutional neural network image classifier, in conjunction with an existing trained convolutional neural network image object detector, To detect specific style objects.

Generally, there are a single large-scale CNN (Convolutional Neural Networks, convolutional neural network) image object detector such as Faster RCNN (Region-Convolutional Neural Network), Single Shot MultiBox Detector and Yolo (You only look once), etc. It can do good detection and recognition of the general training objects in the image, but if you want to add additional specific style object types for training in addition to the trained general objects, in addition to the need for more training time, redo the training In addition to the possibility of deteriorating the ability to detect previous general objects, it is particularly difficult to manually obtain a large number of images of specific objects for marking training.

In view of the existing single large-scale CNN (Convolutional Neural Network) image object detection method in the past, if the detection of a specific style of the object meets the needs of customization, it will be necessary to re-select a large number of training samples of various objects to make full size Supervise Object detection training for controlling images will affect the previous detection and recognition accuracy of general objects. In addition, it is difficult to obtain a large number of object-specific pattern samples for detection and recognition training in a short period of time.

It can be seen that the existing methods cannot meet the needs of customization. For the specific pattern of the detection object, the problem of further detection and recognition training in a short time is not a good design and needs to be improved.

To achieve the above objective, the present invention provides a customizable method for detecting objects of a specific style, which includes a training step, which includes: building a training sample database, wherein the training sample database includes Image training samples containing objects with specific styles and image training samples without objects with specific styles; providing the training sample database to CNN image classifier training and learning to generate trained classification model parameters; and providing the trained Classify model parameters to a CNN pattern classifier for objects of a specific style; the detection step includes: providing a surveillance image to be detected; capturing the area containing objects in the surveillance image to be detected through an existing CNN image object detector Image; and providing the region image containing the object to the specific style object CNN pattern classifier to identify the specific style object, and then output the detection result of the specific style object; and the optimization step, which includes: interpreting the specific style object CNN The pattern classifier recognizes the specific style object to remake the image of the region containing the object that was incorrectly identified by the CNN pattern classifier of the particular pattern object into a correctly labeled training sample; and input the recreated correctly labeled training sample to The training sample database provides the reconstructed correctly labeled training sample to the training of the CNN image classifier Learning to generate optimized classification model parameters.

In the aforementioned method, the steps of building a training sample database further include: searching for object images and feature images through an object image database or a network image search engine; and using different sizes, positions, angles, and deformations In this way, the object image and the characteristic image are synthesized by a computer into the image training sample containing the specific style object in accordance with the background image, so that the image training sample containing the specific style object is input to the training sample database.

In the aforementioned method, it further includes: synthesizing other images without the characteristic image and the background image into the image training sample without the specific style object, so as to input the image training sample without the specific style object to the Sample database for training.

In the aforementioned method, the step of constructing the training sample database further includes: inputting the monitored video image to the existing CNN image object detector to capture the image of the area containing the object in the video image to be monitored; and interpreting The object-containing area image of the video image to be monitored is used to identify whether the object is the specific style object. If the object is the specific style object, the region image that contains the object is classified as the image training sample containing the specific style object, and if the object is not the specific style object, the region image that contains the object is classified as the non-exclusive Video training samples of specific style objects.

In the aforementioned method, the step of building a training sample database further includes: providing a video image of the specific style object at multiple angles, and if the specific style object occupies the entire video image, classify the video image as the Image training samples containing a specific style object, and if the specific style object Without occupying the whole of the recorded image, input the recorded image to the existing CNN image object detector to capture the image of the area containing the object in the recorded image and classify it as the image training sample containing the object of a specific pattern.

In the aforementioned method, the step of building a training sample database further includes: inputting a surveillance video image containing objects but non-specific style objects to the existing CNN image object detector to capture the surveillance video image containing The area image of the object is classified as the image training sample that does not contain the specific style object.

In the aforementioned method, the CNN pattern classifier for the specific style object is a CNN image classifier with two convolutional layers, two pooling layers, two fully connected layers, and thirty-six classification inputs.

In the aforementioned method, the existing CNN image object detector is a Yolo (You only look once) CNN image object detector.

The customizable method of the present invention for detecting objects of specific styles uses a streamlined and dedicated CNN image classifier to determine whether the object area image detected by a single large CNN image object detector is The specific style of the object is further classified and identified to achieve the detection and identification of the newly added specific style object; a large number of synthetic productions or video captures or various specific style object image databases prepared in the past are designed by computer programs Select to generate training image samples of object-specific styles, quickly complete the training of the streamlined dedicated CNN image classifier, and provide the ability to classify and recognize the required object-specific styles; especially, the specific styles of each object in the training sample database Training samples can be continuously collected and manufactured according to needs, and keep pace with the times to make the training and learning of model parameters of CNN image classifiers more diverse in the future And more accurate, and can be used to customize the detection and recognition of objects with specific styles.

101‧‧‧Object Image Database

102‧‧‧Internet image search engine

103‧‧‧Object image

104‧‧‧Feature image

105‧‧‧Background image

106‧‧‧Computer program image synthesis

107‧‧‧Specific style object image training sample

108‧‧‧Synthetic feature erasure image training sample

109‧‧‧Sample database for training

110‧‧‧CNN image classifier

111‧‧‧Monitoring video

112‧‧‧Video images of specific style objects

114‧‧‧Existing CNN image object detector

117‧‧‧Determine whether it is a specific style object

118‧‧‧General object image training sample

201‧‧‧Monitoring image

210‧‧‧ CNN pattern classifier for specific style objects

214‧‧‧Existing CNN image object detector that completed object detection training

215‧‧‧Correctly label training samples

217‧‧‧Wrong classification of interpretation selection

219‧‧‧Specific style object detection result

Please refer to the detailed description of the present invention and its accompanying drawings to further understand the technical content of the present invention and its objectives and effects.

Figure 1 is a schematic diagram of the method flow of the first embodiment of the present invention.

Figure 2 is a schematic diagram of the method flow of the second embodiment of the present invention.

Figure 3 is a schematic diagram of the method flow of the third embodiment of the present invention.

Figure 4 is a schematic diagram of the method flow of the fourth embodiment of the present invention.

The customizable method of the present invention for detecting objects of a specific style is to synthesize image samples of objects of specific styles through computer program synthesis, or use video captures to generate image samples of objects of specific styles, or use various prepared types The specific style object image samples in the sample database 109 for specific style object image training are provided to the streamlined and dedicated CNN (Convolutional Neural Networks, convolutional neural network) image classifier 110 for learning, so as to generate a classification that can recognize a specific style Model parameters of the object, and then use the existing CNN image object detector 214 (for example: Yolo CNN image object detector) that has completed object detection training to detect the image area containing the object in the image, and input the specific training The style object CNN pattern classifier 210 determines that it is Whether it is a specific style object to achieve the detection of a specific style object.

For the specific style object CNN pattern classifier 210, a simplified convolutional neural network (CNN) pattern classifier can be selected. For example, the network layer contains only two convolutional layers; two pooling layers (Pool layers). ), two fully connected layers and thirty-six classification output and input pixels of 40x40 CNN image classifier, in addition to providing 36 types of classification, for specific style objects that are generally close to the object size Sufficient classification and recognition capabilities, and the existing CNN image object detector 214 that has completed object detection training can choose the current better performance Yolo (You only look once) CNN image object detector.

The specific implementation of the method for detecting objects with a specific style of the present invention can be customized, which can be mainly divided into a training step, a detection step, and an optimization step. The following four embodiments provide an explanation.

The first embodiment of the present invention:

As shown in Figure 1, in the training step, when a certain type of object image contains a certain characteristic in a certain type of object image, the object image database 101 or network image search engine 102 is used to find the type of object Various morphological images of the image 103 and various morphological images of the specific feature image 104, and various appropriate background images 105 are obtained in advance, and the object image 103 of each morphology and the specific feature image 104 are different in size, position, angle, and deformation In the method, computer program image synthesis 106 and various background images 105 are used to automatically synthesize various possible specific style object image training samples 107, and then input the specific style object image training samples 107 into the training sample database 109 to Become a specific style object The image training samples are used for training and learning of the CNN image classifier 110; in addition, in order to avoid the unnaturally synthesized specific features generated by the computer program image synthesis 106, they will be returned together after being trained and learned by the CNN image classifier 110 in the future It is classified as a feature of a specific style object in the training sample 107 of a specific style object image, so the feature image 104 is replaced with any other background image that does not contain the specific feature, and the composite feature is created in the same way as the previous computer program image synthesis 106 Erase the image training sample 108 and put it into the specific style object training sample database 109, as a training sample that does not contain the specific style object, and provide the CNN image classifier 110 for training and learning, so as to eliminate the synthetic feature pair classification The impact of identification; in the detection step, after the CNN image classifier 110 is trained and learned to obtain the best classification model parameters, the classification model parameters are transmitted for use by the CNN pattern classifier 210 of a specific style object. In addition, the existing CNN image object detector 214, which has completed object detection training, is used to detect and extract the area image containing the object of the type from the surveillance image 201, and submit it to the specific style object CNN pattern classifier 210 to perform identification and classification. When a specific style object is recognized by classification, output the specific style object detection result 219 to display the detected image time and image detection frame of the specific style object; in the optimization step, in order to keep pace with the times To improve the accuracy of classification recognition, you can add the interpretation selection error classification 217 under the CNN pattern classifier 210 of the specific style object. When the CNN pattern classifier 210 of the specific style object determines that the classification and recognition is incorrect, the previous existing After completing the object detection training, the CNN image object detector 214 detects and captures the regional image, and then remakes it into the training sample database 109 after correctly labeling the training sample 215. It is then provided to the CNN image classifier 110 for training and learning to train better-calculated classification model parameters, so that the CNN pattern classifier 210 for specific style objects in the future is more accurate.

The second embodiment of the present invention:

As shown in Figure 2, in the training step, when a specific object (such as a person or a car) often appears in the surveillance video and the background is too much to be directly input to the CNN image classifier 110 for learning, use the existing A CNN image object detector 114 detects and captures the area image containing the object (person or car) from the surveillance video 111, and determines whether it is an object of a specific style through step 117, so as to capture the detection Take the regional image classification of the object and mark it as the training sample 107 containing the specific style object image or the general object image training sample 118 without the specific style object, and then input it into the training sample database 109 for the CNN image classifier 110 training and learning. In the training step, the model parameters obtained through training can enable the CNN pattern classifier 210 for a specific object (such as a person or a car) to have the classification and recognition capabilities of the specific object (such as a person or a car). The CNN image object detector 214 that has completed the object detection training detects and extracts the area image containing the object from the surveillance image 201, and sends it to the specific style object CNN pattern classifier 210 for identification and classification. When it is recognized by classification, output the detection result of specific style object 219 to display the image time and image detection frame when the specific style object is detected; in the optimization step, in order to advance with the times, the classification recognition is correct Rate, you can add the interpretation selection error classification 217 under the CNN pattern classifier 210 of the specific style object, when the CNN pattern classifier 210 determines the score for the specific style object When the class recognition is not correct, manually recreate the region image detected and captured by the object detector 214 of the existing CNN image that has completed the object detection training, and then re-create the training sample 215 with the correct label, and then input the training sample The database 109 is then provided to the CNN image classifier 110 for training and learning, so as to train better calculated classification model parameters, so that the CNN pattern classifier 210 for specific style objects in the future is more accurate.

The third embodiment of the present invention:

As shown in Figure 3, in the training step, when there is a specific style object of the entity, the entity specific style object can be directly recorded from multiple angles to obtain the recorded image 112 of the specific style object at each angle. If the style object just fills the entire video image, it can be directly marked and saved as a specific style object image training sample 107 (an image training sample containing a specific style object), and then input it into the training sample database 109. If the specific style object is not full of the recorded image, the existing CNN image object detector 114 is used to capture the area image containing the entity specific style object, and classify and mark it as the specific style object image training sample 107 for input for training In the sample database 109; in order to prevent the general object image similar to the entity specific style object from being misjudged as the specific style object, so as to improve the recognition accuracy of the entity specific style object, the existing CNN image object detector is used 114 From the surveillance video images 111 that are similar to (but not) the specific style object image of the entity (that is, contain the object but not the specific style object), detect and capture the area image containing the object, and classify it as General object image training samples 118 that do not contain objects of specific styles are then put into the sample database 109 for training objects of specific styles, and trained with the CNN image classifier 110 By learning, the model parameters obtained through training can enable the CNN pattern classifier 210 of a specific style object to have the ability to reduce similar misidentifications. With the previously created sample database 109 for training objects of specific styles, select the samples that need to be classified for training and learning of the CNN image classifier 110. In the detection step, the classification model parameters obtained by training can be used for the specific style object CNN The pattern classifier 210 has the ability to classify and recognize the specific object of the entity. As in the first embodiment, the existing CNN image object detector 214 that has completed object detection training is used to detect and extract the object containing the object from the surveillance image 201. The area image of the object is sent to the specific style object CNN pattern classifier 210 to perform identification and classification. When the specific style object is recognized by classification, the specific style object detection result 219 is output to show the detected specific style object Image time and image detection frame; in the optimization step, in order to keep up with the times to improve the accuracy of classification and recognition, you can add interpretation selection error classification 217 under the CNN pattern classifier 210 of specific style objects. When the object CNN pattern classifier 210 determines that the classification recognition is incorrect, it manually reproduces the image of the area detected and captured by the object detector 214, which has previously completed object detection training, into the correct training sample After 215, it is input into the training sample database 109, and then provided to the CNN image classifier 110 for training and learning, so as to train better calculated classification model parameters, so that the future CNN pattern classifier 210 of specific style objects will be more accurate

The fourth embodiment of the present invention:

As shown in Figure 4, in the training step, when the image sample of the object with a specific style has been stored in the training sample database 109 established in the past, the training The training sample database 109 is used to select the required training samples (including training samples of object images of specific styles and training samples of general object images) for training of the CNN image classifier 110. After training, the classification model parameters obtained by training can be input to the specific styles The object CNN pattern classifier 210 classifies and recognizes the specific style objects needed, and can continue to perform continuous and cumulative classification and recognition training for each specific style object when adding classification samples, so as to continuously improve the CNN image classifier 110 Identify the type and accuracy rate; in the training step, as in the first embodiment, the existing CNN image object detector 214 that has completed object detection training is used to detect and extract the area containing the object from the surveillance image 201 The image is handed over to the specific style object CNN pattern classifier 210 to perform identification and classification. When the specific style object is classified and recognized, the specific style object detection result 219 is output to display the image time and image detection of the specific style object detected. Measurement frame; in the optimization step, in order to improve the accuracy of classification and recognition with the times, you can add interpretation selection error classification 217 under the specific style object CNN pattern classifier 210, when the specific style object CNN pattern classifier 210 When it is judged that the classification recognition is incorrect, manually recreate the area image detected and captured by the object detector 214 of the CNN image that has previously completed object detection training, and then re-create the correct training sample 215 and input the training The sample database 109 is then provided to the CNN image classifier 110 for training and learning, so as to train better-calculated classification model parameters, so that the CNN pattern classifier 210 for specific style objects in the future is more accurate.

When compared with other conventional techniques, the customized method for detecting objects with specific styles provided by the present invention has the following advantages:

1. The present invention only further classifies and recognizes the detected object area image, whether the object has a specific style, and can only select a simplified dedicated CNN image classifier to provide the required object specific style classification and identification capability, and the classification training and learning will be improved For speed and simplicity.

2. If you add additional specific style object types for training, it will not degrade the object detection capabilities of the CNN image object detector that has completed the training.

3. The training samples of each specific style object in the training sample database can be continuously collected and manufactured according to the needs, and keep pace with the times to provide more various and more appropriate training samples for immediate selection when needed in the future, and It is better and faster to complete the training of the proprietary CNN image classifier, which is suitable for customized object detection in specific styles.

The above detailed description is a specific description of the possible embodiments of the present invention, but the embodiment is not intended to limit the scope of the present invention. Any equivalent implementation or modification that does not deviate from the technical spirit of the present invention should be included in this case. The scope of patents.

To sum up, this case is not only innovative in terms of technical ideas, but also can enhance the above-mentioned multiple functions compared with conventional articles. It should fully meet the requirements of novel and progressive statutory invention patents. An application should be filed in accordance with the law. Please approve this case. Invention patent applications, to encourage invention, to feel good.

101‧‧‧Object Image Database

102‧‧‧Internet image search engine

103‧‧‧Object image

104‧‧‧Feature image

105‧‧‧Background image

106‧‧‧Computer program image synthesis

107‧‧‧Specific style object image training sample

108‧‧‧Synthetic feature erasure image training sample

109‧‧‧Sample database for training

110‧‧‧CNN image classifier

201‧‧‧Monitoring image

210‧‧‧ CNN pattern classifier for specific style objects

214‧‧‧Existing CNN image object detector that completed object detection training

215‧‧‧Correctly label training samples

217‧‧‧Wrong classification of interpretation selection

219‧‧‧Specific style object detection result

Claims (8)

A customizable method for detecting objects of a specific style, comprising: a training step, including: building a training sample database, wherein the training sample database includes image training samples containing objects of the specific style and Image training samples that do not contain specific style objects; provide the training sample database to the CNN image classifier for training and learning to generate trained classification model parameters; and provide the trained classification model parameters to the specific style object CNN pattern A classifier; a detecting step, which includes: providing a surveillance image to be detected; capturing an image of an area containing an object in the surveillance image to be detected through an existing CNN image object detector; and providing the area containing the object The image is sent to the specific style object CNN pattern classifier to identify the specific style object, and then the detection result of the specific style object is output; and the optimization step includes: recognizing the specific style object recognized by the CNN pattern classifier of the specific style object , To recreate the correctly labeled training sample from the image of the area containing the object that was incorrectly identified by the CNN pattern classifier of the specific style object; and input the recreated correctly labeled training sample to the training sample database to provide the Re-create the correctly labeled training samples to the CNN The image classifier is trained and learned to generate optimized classification model parameters. Such as the method described in item 1 of the scope of patent application, wherein the steps of building a training sample database further include: searching for object images and feature images through an object image database or a network image search engine; and using different The size, position, angle, and deformation method are combined with the background image to synthesize the object image and the characteristic image into the image training sample containing the specific style object through the computer, so as to input the image training sample containing the specific style object to the training Use the sample database. For example, the method described in item 2 of the scope of patent application further includes: synthesizing other images that do not contain the characteristic image and the background image into the image training sample without the specific style object, so that the The image training samples are input to the training sample database. Such as the method described in item 1 of the scope of patent application, wherein the step of constructing a training sample database further includes: inputting a monitoring video image to the existing CNN image object detector to capture the video image to be monitored An image of the area containing the object; and interpreting the image of the area containing the object of the video to be monitored to identify whether the object is the specific style object; wherein, if the object is the specific style object, classify the area containing the object The image is the image training sample that contains the specific style object, and if the object is not the specific style object, classify the containing object The region image of is the training sample of the image without specific style objects. The method described in item 1 of the scope of patent application, wherein the step of building a training sample database further includes: providing a video image of the specific style object at multiple angles, if the specific style object occupies the entire video image , Categorize the recorded image as the image training sample containing the specific style object, and if the specific style object does not occupy the entire recorded image, input the recorded image to the existing CNN image object detector to capture the recorded image The area image containing the object is classified as the image training sample containing the object of the specific style. For example, the method described in item 1 of the scope of patent application, wherein the step of constructing a training sample database further includes: inputting surveillance video images containing objects but non-specific style objects to the existing CNN image object detector. Capture the image of the area containing the object in the surveillance video image and classify it as the image training sample without the object of the specific style. The method described in item 1 of the scope of the patent application, wherein the CNN pattern classifier for the specific style object has two convolutional layers, two pooling layers, two fully connected layers, and thirty-six classification output CNNs Image classifier. The method described in item 1 of the scope of patent application, wherein the existing CNN image object detector is a Yolo (You only look once) CNN image object detector.

Images