TWI695343B - Automatic labeling method for detecting moving objects - Google Patents

Abstract

This invention relates to an automatic labeling method for detecting moving objects, which is suitable for obtaining the object information in the multimedia image file to mark the range and position, and then storing the image, the object feature conversion string and the object tag coordinate in the feature object database in order to manage and use a huge amount of feature object data.

Description

Automatic labeling method for detecting moving objects

The invention relates to a technology for automatically labeling sample information. In detail, it refers to an automatic labeling method for detecting moving objects in multimedia images by using color word string conversion analysis.

The supervised learning of the existing artificial intelligence needs to collect data and mark it for learning. When collecting and marking data, it often takes a lot of human resources to collect and mark the data. It not only consumes time, but also causes problems in how to manage the data. How to directly track the moving objects in the multimedia image and capture and store it in the database is the goal that those skilled in the art should devote themselves to.

The purpose of the present invention is to provide an automatic labeling method for detecting moving objects, which can automatically label the object information in the multimedia image, and solve the problem that it takes a lot of manpower and time to label the object information in the multimedia image in the past.

To achieve the foregoing objective, the present invention provides an automatic label for detecting moving objects Note method, the method includes: reading the image; down-grading the red, green and blue (RGB) colors in the image; processing the down-graded image through a visual background extraction algorithm to distinguish foreground objects and background; extracting the foreground Object; process the foreground object through a simple linear iterative clustering algorithm to produce a foreground object with superpixels; confirm whether the foreground object with superpixels is the same as the foreground object in the next subsequent image; through color mapping characters Convert the background and the foreground object with superpixels to characters; convert each of the foreground objects with superpixels into a feature string through a feature representative string to store the feature string in the feature object database ; And compare the relative positions of the difference foreground objects in the image and the next subsequent image in the feature object database to automatically track and mark the difference foreground objects.

In the aforementioned automatic labeling method, the step of processing the degraded image through the visual background extraction algorithm includes: marking the upper left coordinate and the lower right coordinate of the foreground object.

In the aforementioned automatic labeling method, the step of confirming whether the foreground object with superpixels is the same as the foreground object in the next subsequent image includes: based on the superpixel of the foreground object with superpixels and the next subsequent Similarity clustering between superpixels of foreground objects in the image.

In the aforementioned automatic labeling method, the step of classifying the red, green, and blue colors in the image in reduced order includes: distinguishing the 256 levels of red, blue, and green in the red, green, and blue colors into three segments, so that the Red, blue and green are reduced to three shades.

In the aforementioned automatic labeling method, the step of converting the background and the foreground object into characters through color-mapped characters includes: corresponding to 26 English tones of each of the three tones of the three colors of red, blue, and green Characters and 0 characters.

In the aforementioned automatic labeling method, the character string represented by the feature will have a super The conversion of each of the pixel's foreground objects into a character string includes: combining the background and the same characters in the characters converted by the foreground object with superpixels into a representative character and deleting the 0 character Yuan to convert to the feature string.

In the aforementioned automatic labeling method, the conversion of each of the foreground objects with superpixels into a feature string through the feature representative string further includes: storing the feature string having five or more of the characters In the database of the feature object.

Through the aforementioned invention, it is possible to solve the problem of manually labeling, managing, and classifying image stream data manually, one by one, through the automatic labeling method for detecting moving objects of the present invention, which uses color string conversion analysis Detect moving objects, obtain object information and automatically annotate the range and position, and store their images, positions, length, width, and object types in the database for future management and use of data sets. Users can also automatically Adding, deleting, and modifying labeled information can not only improve past problems, but also make it more convenient and flexible to collect labeled information.

S101‧‧‧Read video stream

S102‧‧‧RGB color reduction

S103‧‧‧ViBe algorithm

S104‧‧‧SLIC algorithm

S105‧‧‧Color mapping characters

S106‧‧‧ character representative character string

S107‧‧‧ Create a database of characteristic objects

S108‧‧‧Automatic annotation of moving object detection

FIG. 1 is a flowchart of the steps of the automatic labeling method for detecting moving objects of the present invention.

Figure 2 is a schematic diagram of the reduced RGB color classification of the present invention.

Figure 3 is a schematic diagram of the tracking of moving objects of the present invention.

Figure 4 is a schematic diagram of the color distribution character string of the present invention.

Figure 5 is a schematic diagram of the feature-to-string conversion and merge of the present invention.

FIG. 6 is a schematic diagram of the feature representative string stored in the feature object database of the present invention.

The following specific examples are provided to illustrate the present invention, and those who are familiar with this field can undoubtedly understand the advantages and effects after reading the invention of this specification.

It should be understood that the structures, proportions, dimensions, etc. described in this specification and the accompanying drawings are only disclosed to match the contents of this specification, so that those familiar with the field can easily understand and read, not intending to The invention is limited to specific circumstances and does not have a technical intent. Any modifications to the structure, changes in the proportional relationship, or adjustments in dimensions should be included in the scope of disclosure of this specification without affecting the manufacturability and achievement of this specification. Changes or adjustments in the relative relationship without substantially changing the technical content should also be deemed to fall within the scope of implementation.

As shown in the flowchart shown in FIG. 1, the steps of an automatic labeling method for detecting moving objects of the present invention include: step S101 of reading an image stream; step S102 of performing RGB color reduction to convert pixels Decrease the color classification; perform step S103 of the ViBe (VIsual Background Extractor) algorithm to segment the foreground model of interest, extract the foreground objects, and mark the upper left and lower right coordinates of the foreground objects; Perform step S104 of the Simple Linear Iterative Clustering (SLIC) algorithm to generate superpixels, to cluster the superpixels with similarities between the colors of the superpixels, to do It is the calculation basis for determining the old and new objects to confirm whether the foreground object in the subsequent image is the same as the previous foreground object; by performing the step S105 of color mapping characters and the step S106 of the character representative string, the color of the pixel group in the foreground object Feature conversion characters and then converted into strings to identify foreground objects; step S107 of creating a feature object database is based on the target of the foreground object extracted in step S103 of the ViBe algorithm Remember, automatically mark the size of the foreground objects of each sample, and store the information about the position of the foreground objects in the feature database; search and analyze the feature database, you can know that the difference object image is in the relative position of the overall image To achieve the step S108 of automatic labeling of moving object detection.

For the aforementioned step S102 of RGB color reduction, further reference can be made to the schematic diagram of the reduced RGB color classification in FIG. 2, which is to red, red, green, and blue of the RGB color, from the original 0~ 255 color scales, divided into three sections of 0~84, 85~169, and 170~255, the red, green and blue colors are divided into three tone classifications (R1 R2 R3; G1 G2 G3; and B1 B2 B3) , As shown in the RGB color descending distribution table of Table 1 below; according to this, the combination of the three tones of red, blue and green will have 3x3x3=27 tones, which can be mapped to 26 English words Element (AB...Z), with an additional 0 character, as shown in Table 2 below, which is reduced from RGB colors and sorted into a character comparison table, for example, when the tone combination is R1G1B1 , Which stands for A character, tone combination is R1G1B2, which stands for B character, tone combination is R1G1B3, which stands for C character, and so on, tone combination is R3G3B2, which stands for Z character, and finally The tone combination R3G3B3, which represents 0 characters.

In addition, as shown in Figure 3, the ViBe algorithm step S103 is used to segment the foreground object and the background to extract the foreground object, and mark the upper left and lower right coordinates of the foreground object, and then make the object background black pixels to achieve Moving object tracking, in which, as shown in Figure 3, the foreground and background are divided to extract the foreground object and generate a tracking image of the moving object.

In step S104 through the SLIC algorithm, it can generate superpixels, and divide the superpixels according to the similarity between the colors of the superpixels, to confirm whether the foreground object in the subsequent image is the same as the previous foreground object. Step S105 of the object image with color mapping characters corresponds to the feature representative character string of the color distribution as shown in FIG. 4, in the step of feature representative character string 106, each row represents an object, and the background pixels All are black, and the foreground objects of the remaining pixels will be reduced to the RGB color of the comparison table 2 and classified into the character comparison table to obtain the corresponding characters. Therefore, the black pixels in Figure 4 are represented as 0, and the rest The foreground objects of color pixels are represented as characters A~Z.

Refer to the schematic diagram of the feature-to-string conversion and merge in Figure 5. When the original color distribution of the feature string has consecutive identical characters, it is merged into one character as a representative, and 0 is a negligible feature, so the The feature string only retains the rest of the characters that are not 0; in addition, when the number of characters represented by the combined feature representative string is less than five, it will be judged that the feature representative string has too few feature amounts. Therefore, only The feature representative string with more than five characters in the foreground object and the mark of the foreground object retrieved in the previous ViBe algorithm are stored in the feature object database. Referring to FIG. 6, the feature representative string stored in the feature object database is the text feature (Text Feature) 2 to the text feature (Text Feature) 4 in conjunction with the upper-left coordinates of the foreground object retrieved by the previous ViBe algorithm and The lower right coordinate is stored in the feature object database, thereby achieving step S107 of creating the feature object database.

According to the order of appearance in the image, search and analyze the comparison in the object feature database, so as to know that the foreground objects with differences in the front and rear images are located in the relative position in the overall image, to automatically track and number the foreground objects, and automatically mark out The size of the range of each foreground object, thereby completing the automatic labeling step S108 of moving object detection, in addition, the user can check and classify the foreground object, or add, delete, or modify according to the content of each image , And then save it into the feature database or export file.

The present invention is mainly to convert image information into a character string, reduce the color level, extract objects, image segmentation, and obtain the character string by corresponding text with color, and then convert the character string, coordinate marks and images of all objects Stored in the database management system in a many-to-many way, this technology can be applied to the automatic labeling of moving objects, automatically tracking out a series of objects from the database according to the order of appearance and numbering, and automatically labeling each object with a range size .

In addition, by using RGB color reduction to classify and use the ViBe algorithm as the preliminary segmentation judgment of the foreground and background, extract the object and mark its upper left and lower right coordinates, and then use the SLIC algorithm to generate superpixels and clusters. As a calculation basis for the determination of new and old objects, to automatically track objects. Furthermore, the method of converting the color features of the pixel group in the object into a string using the color descending classification mapping method represents the continuous image object with the string encoding, and all the automatically tracked objects in the feature object database are String coding and arrangement, according to the mark of the object extracted by the previous ViBe algorithm, automatically mark the size of each sample object, and the user can check and classify the object, and add and delete the content of each image , Modify, and then save the feature object database or export files to automatically mark the object information in the multimedia image.

Accordingly, the present invention has the following advantages:

1. Automatically track and mark the size of the moving objects in the multimedia image to store in the feature object database, which is more efficient than manual frame selection, which can effectively reduce the consumption of labor and time costs.

2. Use color feature blocks to convert into character strings to represent the features of continuous image objects, and find large amounts of image information in the feature object database in a fast and automated manner.

3. Combining ViBe algorithm, SLIC algorithm and color string conversion analysis to reduce the mislabeling of moving objects in complex backgrounds.

The above detailed description is a specific description of possible embodiments of the present invention, but this embodiment is not intended to limit the patent scope of the present invention, and any equivalent implementation or change without departing from the technical spirit of the present invention should be included in this case In the scope of patents.

In summary, this case is not only innovative in terms of technical ideas, but also enhances the above-mentioned multiple effects. It should have fully met the requirements for novelty and progress of the statutory invention patent. You must file an application in accordance with the law and urge your office to approve this application Case, to inspire invention, to feel virtuous.

S101‧‧‧Read video stream

S102‧‧‧RGB color reduction

S103‧‧‧ViBe algorithm

S104‧‧‧SLIC algorithm

S105‧‧‧Color mapping characters

S106‧‧‧ character representative character string

S107‧‧‧ Create a database of characteristic objects

S108‧‧‧Automatic annotation of moving object detection

Claims (7)

An automatic labeling method for detecting moving objects. The method includes: reading an image; down-grading the red, green, and blue (RGB) colors in the image; processing the image through a visual background extraction algorithm to distinguish foreground objects And the background; extract the foreground object; process the foreground object through a simple linear iterative clustering algorithm to produce a foreground object with superpixels; confirm whether the foreground object with superpixels is the same as the foreground object in the next subsequent image ; Converting the background and the foreground object with superpixels to characters through color mapping characters; converting each of the foreground objects with superpixels into a character string through a feature representative string to store the feature The character string is in the feature object database; and the relative positions of the difference foreground objects in the image and the next subsequent image in the feature object database are compared to automatically track and mark the difference foreground object. The automatic labeling method for detecting moving objects as described in item 1 of the patent scope, wherein the step of processing the image through the visual background extraction algorithm includes: marking the upper left and lower right coordinates of the foreground object . The automatic labeling method for detecting moving objects as described in item 1 of the patent scope, wherein the step of confirming whether the foreground object with superpixels is the same as the foreground object in the next subsequent image includes: Similarity clustering between the superpixels of the foreground object with superpixels and the superpixels of the foreground object in the next subsequent image. The automatic labeling method for detecting moving objects as described in item 1 of the patent application scope, wherein the step of classifying the red, blue, and green colors in the image in the reduced order includes: red, green, and blue colors in the red, blue, and green colors Each of the 256 gradations is divided into three sections, so that the red, blue, and green gradations are reduced to three tones. The automatic labeling method for detecting moving objects as described in item 4 of the patent application scope, wherein the step of converting the background and the foreground object into characters through color-mapped characters includes: the red, blue, and green The 27 tone combinations of each of the three shades correspond to 26 English characters and 0 characters respectively. The automatic labeling method for detecting moving objects as described in Item 5 of the patent application scope, wherein the conversion of each of the foreground objects with superpixels into feature character strings through the feature representative character string includes: Combine the background and the foreground object with superpixels with the same consecutive characters in the character into a representative character and delete the 0 character to convert to the character string. The automatic labeling method for detecting moving objects as described in Item 6 of the patent application scope, wherein the conversion of each of the foreground objects with superpixels into feature strings through the feature representative string further includes: The feature string having five or more of the characters is stored in the feature object database.

Images