CN117292173A

CN117292173A - Automatic labeling object label selection method, system and device based on statistical analysis

Info

Publication number: CN117292173A
Application number: CN202311141241.3A
Authority: CN
Inventors: 李保石
Original assignee: Beijing Kaiwang Data Technology Co ltd
Current assignee: Beijing Kaiwang Data Technology Co ltd
Priority date: 2023-09-05
Filing date: 2023-09-05
Publication date: 2023-12-26

Abstract

The invention provides a method, a system and a device for automatically selecting labels of labels based on statistical analysis, which are used for acquiring size characteristics of position frames corresponding to different types of sample labels in a 3D scene by utilizing a statistical analysis division mode based on historical label data, and after performing frame selection on a new target label object, comparing the size of a candidate frame with the size characteristics of the labels of all the types, and marking the target label object by taking the sample label type with the highest similarity as the target label. The automatic labeling method and device for the label machine have the advantages that automation of labeling work is achieved, labeling efficiency is improved, and labeling errors are reduced.

Description

Automatic labeling object label selection method, system and device based on statistical analysis

Technical Field

The invention relates to the technical field of data processing, in particular to a method, a system and a device for automatically selecting a label of a label based on statistical analysis.

Background

Machine learning is a technology in the field of artificial intelligence, and the basic idea is to use computer algorithms and mathematical models to enable a computer to learn and improve its own performance autonomously according to a large amount of data, so as to continuously improve the capability of solving problems. The application range of machine learning is very wide, including the fields of image recognition, speech recognition, natural language processing, recommendation systems, and the like. In performing 3D object detection tasks using machine learning, the computer algorithms and models employed need to be trained first, and in the process, a large amount of annotation data needs to be utilized.

In a general labeling process, firstly, selecting an object in a 3D scene in a frame mode to mark the position of the object, and meanwhile, adding a label to the object selected by the frame according to judgment to serve as a true value for subsequent processing and identification. In the labeling process of some application scenes, the types of labels to be added are different from a few to tens according to the specific content of the task, the more the number of the labels is, the more time is spent by a labeling person when the labels are selected, the labeling efficiency is greatly influenced, and the risk of misoperation is also increased.

Disclosure of Invention

In view of this, the embodiment of the invention provides a method, a system and a device for automatically selecting labels of labels based on statistical analysis, so as to eliminate or improve one or more defects existing in the prior art, and solve the problem of low labeling efficiency and easy error caused by excessive labels in the labeling process of objects in a 3D scene.

One aspect of the present invention provides a method for automatically selecting a label for a label tag based on statistical analysis, the method comprising the steps of:

performing frame selection labeling on a target labeling object in a designated 3D scene by adopting a cuboid frame to obtain a target position frame of the target labeling object, and recording size parameter characteristics of the target position frame, wherein the size parameter characteristics of the target position frame at least comprise length, width and height parameters of the target position frame;

inquiring a preset database, and acquiring label size characteristics of sample labeling objects in a plurality of sample 3D scenes according to a plurality of preset label categories, wherein the labels are manually labeled sample labeling object position frames, and the label size characteristics at least comprise length, width and height standard parameters of the sample labeling object position frames corresponding to each type of labels;

and comparing the similarity between the size parameter characteristics of the target position frame and the size characteristics of the labels corresponding to various labels, determining the size characteristics of the labels closest to the size parameter characteristics of the target position frame, and adding the sample labels corresponding to the size of the labels as target labels to the target labeling objects.

In some embodiments, in the length, width and height standard parameters of the sample labeling object position frame corresponding to each type of tag, the length, width and height standard parameters are a long average value, a wide average value and a high average value of the sample labeling object position frame corresponding to each type of tag.

In some embodiments, the target position frame size parameter characteristics include a length, width, height parameter of the target position frame, and a deflection angle;

and the label size characteristics comprise a long average value, a wide average value, a high average value and a deflection angle average value of the position frames of the labeling objects of each type of labels.

In some embodiments, the similarity comparison is performed on the target position frame size parameter feature and the tag size feature corresponding to each type of tag, one or more algorithms of euclidean distance, manhattan distance and/or cosine similarity are adopted to perform similarity comparison, and the tag size feature with the most similar target position frame size parameter feature is determined according to a majority of comparison results of the algorithms.

In some embodiments, the method further comprises: and adding the target labeling object into a sample labeling object corresponding to the belonging label according to the belonging label, and updating and calculating the label size characteristic of the belonging label by adopting the size parameter characteristic of the target position frame.

On the other hand, the invention also provides an automatic labeling system based on statistical analysis, which comprises the following steps:

the back-end service module comprises a data statistics module and an API interface module;

the data statistics module is used for acquiring sample labels of a plurality of historical sample objects in a target 3D scene, wherein the sample labels are sample position frames marked on the historical sample objects by cuboid frames; classifying the sample tags, and carrying out statistical analysis on sample position frame parameters corresponding to each sample tag in each category to obtain length, width and height standard parameters of each sample position frame corresponding to the sample tag in the corresponding category, wherein the length, width and height standard parameters are used as sample tag size characteristics of the sample tag in the corresponding category; storing various sample tags and corresponding sample tag size characteristics thereof into a database;

the API interface module at least provides a query interface;

the front end marking tool module comprises a drawing frame module, a rear end interaction module and a label matching module;

the frame pulling module is used for carrying out frame selection marking on a target marking object in a designated 3D scene by adopting a cuboid frame to obtain a target position frame of the target marking object, and recording the size parameter characteristics of the target position frame, wherein the size parameter characteristics of the target position frame at least comprise the length, width and height parameters of the target position frame;

the back-end interaction module is used for acquiring a plurality of sample tags and corresponding sample tag size characteristics thereof in the data statistics module through the query interface;

the label matching module is used for comparing the similarity between the size parameter characteristics of the target position frame and the size characteristics of the sample labels, and labeling the sample label with the highest similarity pair as a target label to the target labeling object.

In some embodiments, the API interface module further provides a report tag size interface; the back-end interaction module is further used for storing the target labeling object and the target label corresponding to the target labeling object into the database so as to update and calculate sample label size characteristics of corresponding sample labels.

In some embodiments, the front end annotation tool module further comprises a display module.

In another aspect, the present invention further provides an automatic label selection device based on statistical analysis, including a processor and a memory, where the memory stores computer instructions, and the processor is configured to execute the computer instructions stored in the memory, and when the computer instructions are executed by the processor, the device implements the steps of the method described above.

In another aspect, the present invention also provides a computer readable storage medium having stored thereon a computer program which when executed by a processor performs the steps of the above method.

The invention has the advantages that:

according to the method, the system and the device for automatically selecting the label object labels based on the statistical analysis, the size characteristics of the position frames corresponding to different types of sample labels in the 3D scene are obtained by utilizing the statistical analysis method based on historical label data, after the frame selection of a new target label object is executed, the sizes of the candidate frames are compared with the label size characteristics of each type of label, and the sample label class with the highest similarity is used as the target label to label the target label object. The automatic labeling method and device for the label machine have the advantages that automation of labeling work is achieved, labeling efficiency is improved, and labeling errors are reduced.

Additional advantages, objects, and features of the invention will be set forth in part in the description which follows and in part will become apparent to those having ordinary skill in the art upon examination of the following or may be learned from practice of the invention. The objectives and other advantages of the invention may be realized and attained by the structure particularly pointed out in the written description and drawings.

It will be appreciated by those skilled in the art that the objects and advantages that can be achieved with the present invention are not limited to the above-described specific ones, and that the above and other objects that can be achieved with the present invention will be more clearly understood from the following detailed description.

Drawings

The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this application, illustrate and together with the description serve to explain the invention. In the drawings:

fig. 1 is a flow chart of a method for automatically selecting labels of labels based on statistical analysis according to an embodiment of the invention.

Fig. 2 is a schematic structural diagram of an automatic labeling system based on statistical analysis according to an embodiment of the invention.

Reference numerals illustrate:

100: a back-end service module; 110: a data statistics module; 120: an API interface module;

121: a query interface; 122: reporting a label size interface; 200: a front end marking tool module;

210: a frame pulling module; 220: a back-end interaction module; 230: a tag matching module;

240: and a display module.

Detailed Description

The present invention will be described in further detail with reference to the following embodiments and the accompanying drawings, in order to make the objects, technical solutions and advantages of the present invention more apparent. The exemplary embodiments of the present invention and the descriptions thereof are used herein to explain the present invention, but are not intended to limit the invention.

It should be noted here that, in order to avoid obscuring the present invention due to unnecessary details, only structures and/or processing steps closely related to the solution according to the present invention are shown in the drawings, while other details not greatly related to the present invention are omitted.

It should be emphasized that the term "comprises/comprising" when used herein is taken to specify the presence of stated features, elements, steps or components, but does not preclude the presence or addition of one or more other features, elements, steps or components.

It is also noted herein that the term "coupled" may refer to not only a direct connection, but also an indirect connection in which an intermediate is present, unless otherwise specified.

Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. In the drawings, the same reference numerals represent the same or similar components, or the same or similar steps.

When labeling a 3D object detection task, a labeling person generally needs to draw a frame with a mouse to output an object with a cuboid frame, and then select a proper object from a plurality of labels. The number of labels is quite large, and each task is different, generally from a few to tens of different tasks. When the number of labels is greater, the more time it takes for the annotator to select a label. In such tasks, selecting tags is generally an important factor affecting labeling efficiency. Specifically, labeling personnel manually select labels after frame pulling, so that continuity of actions and thinking continuity of the labeling personnel are broken, and the labeling efficiency is greatly affected. And as the number of labels increases, this effect becomes greater. In the object detection task, the selection of labels is generally an important factor affecting the labeling efficiency, and this effect increases with the number of labels. And manual selection is at risk of false selection.

The invention solves the technical problem of how to automatically select correct labels for labels according to the size characteristics of targets of different labels in a 3D target detection labeling task. The solution is to automatically count the historical tag sizes and provide an API for the front end to obtain the size characteristics of all tags. After the annotator pulls the frame, a closest label is selected from the pull frame size and label size comparison. The main principle of the scheme is that a statistical analysis method is used for analyzing and processing the historical label size data so as to provide accurate label selection for a labeling person and improve the accuracy and efficiency of labeling.

In one aspect, the present invention provides a method for automatically selecting a label of a label based on statistical analysis, referring to fig. 1, the method includes the following steps S101 to S103:

step S101: and carrying out frame selection labeling on the target labeling object in the appointed 3D scene by adopting a cuboid frame to obtain a target position frame of the target labeling object, and recording the size parameter characteristics of the target position frame, wherein the size parameter characteristics of the target position frame at least comprise the length, width and height parameters of the target position frame.

Step S102: inquiring a preset database, and acquiring label size characteristics of sample labeling objects in a plurality of sample 3D scenes according to a plurality of preset label categories, wherein the labels are manually labeled sample labeling object position frames, and the label size characteristics at least comprise length, width and height standard parameters of the sample labeling object position frames corresponding to each type of labels.

Step S103: and comparing the similarity of the size parameter characteristics of the target position frame with the size characteristics of the labels corresponding to various labels, determining the size characteristics of the labels closest to the size parameter characteristics of the target position frame, and adding the sample label corresponding to the size of the label as a target label to the target labeling object.

In step S101, in the 3D detection scene, the form of the marking position box for the target object may be represented using a bounding box (bounding box) or a cube (bounding cube). The bounding box is an axis-aligned cuboid that completely encloses and closely conforms to the boundary of the target object. The bounding box consists of six faces, each parallel to one of the three coordinate axes. The minimum and maximum vertices of the bounding box may be used to define their locations and sizes. A cube is a special bounding box with its six faces square and all sides equal. A cube is suitable for use in a case where the shape of an object is approximately a cube. Instead of bounding boxes and cubes, other forms of marked position boxes may be used, such as a rotating bounding box (oriented bounding box), whose length, width and height may be rotationally adjusted in any direction to better fit the shape of the object.

In this embodiment, for the new sample data added with the tag, the annotator manually frames the target position frame of the target object, and the marked position frame should also adapt to the required recording parameters in the three-dimensional space because the marked position frame is the object in the 3D scene. For one position frame, in order to be able to more accurately frame and select a target object, the embodiment uses a candidate frame of a cube for labeling. In an actual scene, different types of objects have similar morphological characteristics, and position frames generated by the objects in the frame selection process keep consistent in morphology or size. After the annotator finishes the frame selection, the annotator can obtain the corresponding target position frame and automatically output the related parameters of the target position frame by the system. The shape-related parameters, that is, the size parameter characteristics of the target position frame, may include the length, width and height of the target position frame, further, the ratio relationship between the length, width and height may be marked, and further, if a form of rotating bounding box is adopted, the deflection angle in the axial direction may also be recorded.

In some embodiments, in the length, width and height standard parameters of the sample labeling object position frame corresponding to each type of label, the length, width and height standard parameters are the long average value, the wide average value and the high average value of the sample labeling object position frame corresponding to each type of label.

In step S102, a plurality of samples are pre-stored in the database, and each sample marks a sample marking object in the sample 3D scene, including a position frame and a label type. The form of the position frame adopted here is consistent with that in step S101, and the tag type refers to the specific content of the tag, that is, the result of identification and detection required by the sample object in the target detection task. And under each label type, a plurality of samples are included, and statistical analysis is carried out on relevant parameters of a sample labeling object position frame in the same label type sample to obtain label size characteristics corresponding to the label type. Specifically, the length, width and height of a sample labeling object position frame in the same label type sample are subjected to mean value calculation, the deflection angle is subjected to mean value calculation, and the ratio between the length, width and height is subjected to mean value calculation. The foregoing description is not limiting of the tag size features, and it should be appreciated that other forms of tag size features may be employed in the actual application scenario to meet specific application requirements.

In some embodiments, the target position frame size parameter characteristics include a length-width-height parameter of the target position frame, and a deflection angle; the label size characteristics comprise a long average value, a wide average value, a high average value and a deflection angle average value of the position frame of each sample labeling object corresponding to each type of label.

In step S103, the feature of the size parameter of the target position frame of the target labeling object selected by the labeling member is compared with the feature of the label size of each type of label obtained by the statistical analysis in step S102 in similarity, the feature of the label size most similar to the feature of the size parameter of the target position frame is judged, and the label of the corresponding type is labeled to the target labeling object.

In some embodiments, the similarity comparison is performed on the target position frame size parameter feature and the tag size feature corresponding to each type of tag, one or more algorithms of euclidean distance, manhattan distance and/or cosine similarity are adopted for similarity comparison, and the tag size feature with the most similar target position frame size parameter feature is determined according to most of comparison results of the algorithms.

In some embodiments, the method further comprises: and adding the target labeling object into the sample labeling object corresponding to the belonging label according to the belonging label, and updating and calculating the label size characteristic of the belonging label by adopting the size parameter characteristic of the target position frame. The embodiment aims to continuously update parameters of sample labeling objects and label size characteristics in the existing database so as to improve accuracy.

In another aspect, the present invention further provides an automatic labeling system based on statistical analysis, as shown in fig. 2, including:

the back-end service module 100. The back-end service module 100 includes a data statistics module 110 and an API interface module 120.

The data statistics module 110 is configured to obtain sample tags of a plurality of historical sample objects in the target 3D scene, where the sample tags are sample position frames marked with cuboid boxes on the historical sample objects; classifying the sample labels, and carrying out statistical analysis on sample position frame parameters corresponding to each sample label in each class to obtain length, width and height standard parameters of each sample position frame corresponding to the sample label in the corresponding class, wherein the length, width and height standard parameters are used as sample label size characteristics of the sample label in the corresponding class; and storing various sample labels and corresponding sample label size characteristics thereof into a database.

The API interface module 120 provides at least a query interface 121;

the front end labeling tool module 220, the front end labeling tool module 220 comprises a pull frame module 210, a back end interaction module 220 and a label matching module 230;

the frame pulling module 210 is configured to perform frame selection annotation on a target annotation object in a specified 3D scene by using a cuboid frame, obtain a target position frame of the target annotation object, and record a size parameter characteristic of the target position frame, where the size parameter characteristic of the target position frame at least includes a length, width and height parameter of the target position frame.

The back-end interaction module 220 is configured to obtain a plurality of sample tags and corresponding sample tag size features of the sample tags in the data statistics module 110 through the query interface 121.

The tag matching module 230 is configured to compare the similarity between the size parameter feature of the target position frame and the size feature of the sample tag, and label the sample tag with the highest similarity as the target tag to the target labeling object.

In some embodiments, the API interface module 120 also provides a report tag size interface 122; the back-end interaction module 220 is further configured to store the target labeling object and the corresponding target label thereof in a database, so as to update and calculate the sample label size characteristics of the corresponding sample label.

In some embodiments, the front end labeling tool module 220 further includes a display module 240.

The device may include a processor, a memory, and an image acquisition device, where the processor and memory may be connected by a bus or other means.

The processor may be a central processing unit (Central Processing Unit, CPU). The processor may also be any other general purpose processor, digital signal processor (Digital Signal Processor, DSP), application specific integrated circuit (Application Specific Integrated Circuit, ASIC), field programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof.

The memory, as a non-transitory computer readable storage medium, may be used to store a non-transitory software program, a non-transitory computer executable program, and a module, such as a program instruction/module corresponding to a key shielding method of an in-vehicle display device in an embodiment of the present invention. The processor executes various functional applications of the processor and data processing by running non-transitory software programs, instructions, and modules stored in the memory, i.e., to implement the image color correction method in the above-described method embodiments.

The memory may include a memory program area and a memory data area, wherein the memory program area may store an operating system, at least one application program required for a function; the storage data area may store data created by the processor, etc. In addition, the memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one magnetic disk storage device, flash memory device, or other non-transitory solid state storage device. In some embodiments, the memory may optionally include memory located remotely from the processor, the remote memory being connectable to the processor through a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.

The following description is made with reference to specific embodiments:

referring to fig. 1 and 2, the present embodiment provides an automatic labeling system based on statistical analysis, which includes a back-end service module and a front-end labeling tool module. The back-end service module automatically counts the sizes of the historical labels and provides an API interface for the front-end to acquire the size characteristics of all the labels. The front end marking tool module automatically compares the size of the frame with the size of the label according to the size of the frame after the frame is pulled by a marker, and selects a nearest label from the labels. The following is a detailed description of this scheme:

the back-end service module comprises the following modules:

and a data statistics module: the module performs statistical analysis according to the historical tag size data, calculates data such as the mean value, variance, standard deviation and the like of the tag size, and stores the data in a database. To ensure that the data has a certain representativeness and characteristic accuracy, more than 100 pieces of data which are subjected to verification are generally required for each tag.

API interface module: the module provides two interfaces.

And a query interface which allows the front-end labeling tool to query the size data of all the labels so as to match after the labeling personnel pulls the frames.

And the reporting tag size interface is used for reporting the tag and the size of each label to the rear end. In order to continuously optimize the feature values, the front end reports the labels and dimensions of the new audited labels to the back end. And the back end performs statistical analysis and updates the characteristic value in the database so as to obtain more accurate size characteristics of each label.

The front end labeling tool comprises the following modules:

and (5) a frame pulling module: this module allows the annotator to draw a frame to select the target.

And a label matching module: after the annotator pulls the frame, the module automatically compares the size of the pulled frame with the size data of all the tags, and selects a closest tag. The matching algorithm should calculate a similarity score based on the results of the statistical analysis and select the label with the highest score.

And a back-end interaction module: the method is used for pulling all the tag size characteristic information and reporting the tag size and the tag.

The method has the main advantages that the process of labeling tasks is automatically processed, the label size data is automatically calculated according to the historical data, a user-friendly interface and an accurate label matching algorithm are provided, and therefore the labeling accuracy and efficiency are improved.

According to the method, the most suitable label is automatically matched for the target object through the size characteristics of the target object, so that a labeling person is focused on the frame pulling work, the frame pulling and label selecting are not required to be switched, the continuity of actions and the continuity of thought are maintained, the working steps are reduced, and the working efficiency can be greatly improved. Meanwhile, a manual operation is omitted, and unintentional errors caused by manual work are avoided.

In summary, according to the method, system and device for automatically selecting the label object label based on statistical analysis, the size characteristics of the position frames corresponding to different types of sample labels in the 3D scene are obtained by using a statistical analysis division mode based on historical label data, after the frame selection of a new target label object is executed, the similarity comparison is carried out on the size of the candidate frame and the label size characteristics of each type, and the sample label class with the highest similarity is used as the target label to label the target label object. The automatic labeling method and device for the label machine have the advantages that automation of labeling work is achieved, labeling efficiency is improved, and labeling errors are reduced.

Those of ordinary skill in the art will appreciate that the various illustrative components, systems, and methods described in connection with the embodiments disclosed herein can be implemented as hardware, software, or a combination of both. The particular implementation is hardware or software dependent on the specific application of the solution and the design constraints. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention. When implemented in hardware, it may be, for example, an electronic circuit, an Application Specific Integrated Circuit (ASIC), suitable firmware, a plug-in, a function card, or the like. When implemented in software, the elements of the invention are the programs or code segments used to perform the required tasks. The program or code segments may be stored in a machine readable medium or transmitted over transmission media or communication links by a data signal carried in a carrier wave.

It should be understood that the invention is not limited to the particular arrangements and instrumentality described above and shown in the drawings. For the sake of brevity, a detailed description of known methods is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method processes of the present invention are not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between steps, after appreciating the spirit of the present invention.

In this disclosure, features that are described and/or illustrated with respect to one embodiment may be used in the same way or in a similar way in one or more other embodiments and/or in combination with or instead of the features of the other embodiments.

The above description is only of the preferred embodiments of the present invention and is not intended to limit the present invention, and various modifications and variations can be made to the embodiments of the present invention by those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. The automatic label selection method based on the statistical analysis is characterized by comprising the following steps of:

2. The automatic labeling object label selection method based on statistical analysis according to claim 1, wherein in the length, width and height standard parameters of the sample labeling object position frame corresponding to each type of label, the length, width and height standard parameters are a long average value, a wide average value and a high average value of the sample labeling object position frame corresponding to each type of label.

3. The automatic selection method of label tag based on statistical analysis according to claim 1, wherein the size parameter characteristics of the target position frame include length, width and height parameters of the target position frame, and deflection angle;

4. The automatic label selection method based on statistical analysis according to claim 1, wherein the similarity comparison is performed between the size parameter characteristics of the target position frame and the size characteristics of the labels corresponding to various labels, one or more algorithms selected from euclidean distance, manhattan distance and/or cosine similarity are adopted to perform similarity comparison, and the size characteristics of the labels with the most similar size parameter characteristics of the target position frame are determined according to a majority of comparison results of the algorithms.

5. The automated statistical analysis-based label selection method of claim 1, further comprising:

and adding the target labeling object into a sample labeling object corresponding to the belonging label according to the belonging label, and updating and calculating the label size characteristic of the belonging label by adopting the size parameter characteristic of the target position frame.

6. An automatic labeling system based on statistical analysis, comprising:

the API interface module at least provides a query interface;

7. The automated labeling system based on statistical analysis of claim 6, wherein the API interface module further provides a report tag size interface;

the back-end interaction module is further used for storing the target labeling object and the target label corresponding to the target labeling object into the database so as to update and calculate sample label size characteristics of corresponding sample labels.

8. The automated labeling system based on statistical analysis of claim 6, wherein the front-end labeling tool module further comprises a display module.

9. A label tag automatic selection device based on statistical analysis, comprising a processor and a memory, characterized in that the memory has stored therein computer instructions for executing the computer instructions stored in the memory, which device, when executed by the processor, implements the steps of the method according to any one of claims 1 to 5.

10. A computer readable storage medium, on which a computer program is stored, characterized in that the program, when being executed by a processor, implements the steps of the method according to any one of claims 1 to 5.