WO2022247628A1

WO2022247628A1 - Data annotation method and related product

Info

Publication number: WO2022247628A1
Application number: PCT/CN2022/091951
Authority: WO
Inventors: 晋周南; 陈浩; 李在旺
Original assignee: 华为技术有限公司
Priority date: 2021-05-24
Filing date: 2022-05-10
Publication date: 2022-12-01
Also published as: CN115393379A

Abstract

A data annotation method and a related product, which belong to the technical field of data processing. The method comprises: obtaining a first edge line and a second edge line of a target object, which are contained in a target image, wherein the first edge line is different from the second edge line; obtaining a first contour of the target object according to the first edge line; and correcting an edge line of the first contour by using the second edge line, so as to obtain a second contour of the target object. By means of the method, edge detection is performed on a target object in a target image by means of proposing a deep learning method based on a pre-selected box, so as to obtain a first contour of the target object, and then, weak semantic information is introduced to convert the correction of an edge point of the first contour of the target object into the correction of an edge line of the first contour of the target object, so as to obtain a second contour of the target object by means of the correction, such that the accuracy and efficiency of an annotation result can be greatly improved.

Description

A data labeling method and related products

This application claims the priority of the Chinese patent application with the application number 202110564875.4 and the application name "A Data Labeling Method and Related Products" submitted to the China Patent Office on May 24, 2021, the entire contents of which are incorporated in this application by reference middle.

technical field

The present application relates to the technical field of data processing, in particular to a data labeling method and related products.

Background technique

A tag is a data form used to describe the characteristics of a business entity. The tag can be used to describe the business entity and reflect the characteristics of the business entity from multiple perspectives. Labels can be obtained from data labeling, which is the act of labeling data by data labelers with the help of labeling tools. The types of data annotation include: image annotation, voice annotation, text annotation, etc. Among them, image annotation can be applied to application scenarios such as face recognition and automatic driving vehicle recognition. Data labelers need to mark the contours of different target markers with different colors, and then label the corresponding contours, and use the labels to outline the target markers in the contours, so that the model can automatically identify different markers in the image. With the widespread use of data-driven image processing techniques in various fields, more and more application scenarios require the use of labels to automatically identify different markers contained in images. Labels need to be obtained through data labeling. Therefore, how to optimize the accuracy and efficiency of data labeling is crucial.

At present, the method of interactive image segmentation and labeling is usually used for data labeling. Specifically, the initial area is manually given, and the segmentation algorithm is used to segment the target object in the initial area to obtain the edge points of the target object, and then the segmentation result of the target object is corrected by correcting the edge points.

However, the above-mentioned labeling method needs to manually correct the edge points one by one, which is time-consuming and labor-intensive, and the accuracy and efficiency of the labeling results are low.

Contents of the invention

The embodiment of the present application provides a data labeling method and related products. By proposing a deep learning method based on pre-selected boxes, the edge detection of the target object in the target image is obtained, and the first contour of the target object is obtained, and then the weak semantic information is introduced. The correction of the edge points of the first contour of the target object is transformed into the correction of the edge line of the first contour of the target object, and the correction obtains the second contour of the target object, which can greatly improve the accuracy and efficiency of the labeling results.

In the first aspect, the embodiment of the present application provides a data labeling method, which includes:

Obtaining a first edge line and a second edge line of the target object contained in the target image; the first edge line and the second edge line are different;

Obtain a first outline of the target object according to the first edge line;

Using the second edge line to correct the edge line of the first contour to obtain the second contour of the target object.

In the embodiment of the present application, by performing different processing on the target image, different edge lines of the target object included in the target image can be obtained, such as the first edge line and the second edge line. Then the target object is initially marked based on the first edge line to obtain the first outline of the target object. Then, the second edge line is used to correct the edge line of the first contour obtained from the initial calibration, so as to obtain the second contour of the target object.

In the current data labeling method, the segmentation algorithm usually segments the target object to obtain the edge points of the target object, and then corrects the outline of the target object by correcting the edge points. Therefore, it is necessary to manually correct the edge points one by one, which is time-consuming and labor-intensive, and the accuracy and efficiency of the labeling results are low.

The data labeling method provided by the embodiment of the present application, compared with the currently commonly used data labeling method, uses the first edge line to initially mark the target object, and then uses the second edge line to perform initial mark on the edge line of the first contour obtained from the initial mark Correction, so that the correction of the edge point of the first contour of the target object is converted into the correction of the edge line of the first contour of the target object, and the second contour of the target object is obtained by correction, which can greatly improve the accuracy and efficiency of the labeling results .

In a possible implementation manner, the correcting the edge line of the first outline by using the second edge line includes:

A contour formed by the second edge line and the edge line of the first contour is determined as the second contour.

In the embodiment of the present application, a possible implementation manner of using the second edge line to correct the edge line of the first contour is provided. The contour formed by the second edge line and the edge line of the first contour is determined as the second contour of the target object, and the correction method improves the accuracy and efficiency of the labeling result.

In a possible implementation manner, before obtaining the first edge line and the second edge line of the target object contained in the target image, the method further includes:

performing segmentation processing on the target image to obtain a first edge point of the target object; the segmentation processing is used to separate the target object included in the target image;

performing edge detection on the target object to obtain a second edge point of the target object; the edge detection is used to separate the edge of the target object;

The obtaining the first edge line and the second edge line of the target object contained in the target image includes:

The first edge line and the second edge line are determined according to the first edge point and the second edge point.

In the embodiment of the present application, a possible implementation manner of obtaining the first edge line and the second edge line of the target object included in the target image is provided. That is, before determining the first edge line and the second edge line of the target object, the target object in the target image is separated by segmenting the target image to obtain the first edge point of the target object. The segmentation processing here includes image segmentation methods based on non-deep learning (such as segmentation methods based on watershed algorithm, graph theory, etc.) and image segmentation methods based on deep learning. The main difference between the two is that image segmentation methods based on deep learning require Convolutional neural network is used. After the target object in the target image is separated, an edge detection is performed on the target object, and the edge of the target object is separated to obtain a second edge point of the target object. The edge detection here includes edge detection based on deep learning of non-preselected boxes and edge detection based on deep learning of preselected boxes. Whether the pixel is an edge point, but to detect the position and category of the edge point corresponding to each preselection box. Although the above-mentioned first edge point and the second edge point are both edge points of the target object, they are edge points obtained by adopting different methods for the target object. Therefore, the above-mentioned first edge point and the second edge point are not completely the same. According to the first edge point and the second edge point, the first edge line and the second edge line of the above-mentioned target object can be determined. Since the first edge point and the second edge point are not completely the same, correspondingly, the first edge line and the second edge line obtained accordingly are also not completely the same.

Through the implementation of obtaining the first edge line and the second edge line provided by the embodiment of the present application, the accuracy of the acquired edge line of the target object can be improved, and it is beneficial to the subsequent edge points of the first contour of the target object The correction of the target object is transformed into the correction of the edge line of the first contour of the target object, which improves the accuracy and efficiency of the labeling result.

In a possible implementation manner, the determining the first edge line and the second edge line according to the first edge point and the second edge point includes:

According to the first edge point and the second edge point, the first edge line is obtained; the point on the first edge line is an edge point where the first edge point and the second edge point coincide. ;

The second edge line is obtained according to the first edge point, the second edge point, and the first edge line.

In the embodiment of the present application, a possible specific implementation manner of determining the first edge line and the second edge line according to the first edge point and the second edge point is provided. The line formed by the edge points where the first edge point and the second edge point overlap is determined as the first edge line, that is, the first edge line is located on the edge result obtained by the above-mentioned segmentation processing and edge detection, and the above-mentioned segmentation processing and edge The first edge line can be obtained through detection. Then, according to the first edge point, the second edge point and the above-mentioned first edge line, the second edge line is determined. The accuracy of the first edge line and the second edge line obtained in the embodiment of the present application is higher, and thus the accuracy and efficiency of the labeling results are also higher.

In a possible implementation manner, the obtaining the second edge line according to the first edge point, the second edge point, and the first edge line includes:

According to the first edge point and the second edge point, a third edge line is obtained; the point on the third edge line is the first edge point or the second edge point, and is not the an edge point at which the first edge point coincides with the second edge point;

The third edge line having two or more intersection points with the first edge line is determined as the second edge line.

In the embodiment of the present application, a possible specific implementation manner of determining the second edge line according to the first edge point, the second edge point, and the first edge line is provided. The line formed by the union of the first edge point and the second edge point after removing the edge point where the first edge point coincides with the second edge point is determined as the third edge line, that is, the third edge line is located at the position obtained by the above segmentation process. On the edge result of the above edge detection, or on the edge result obtained by the above edge detection, not all on the edge result obtained by the segmentation processing and edge detection. Then, a third edge line having two or more intersection points with the first edge line is determined as the second edge line. The accuracy of the second edge line obtained in the embodiment of the present application is higher, and by using the second edge line, the correction of the edge point of the first contour of the target object can be converted into the correction of the edge line of the first contour of the target object, It is beneficial to improve the accuracy and efficiency of correction.

In a possible implementation manner, the performing edge detection on the target object to obtain the second edge point of the target object includes:

determining the label of the target object according to the position of the edge point on the edge line of the target object;

According to the label of the target object, edge points on the edge line of the target object are adjusted to obtain the second edge point.

In the embodiment of the present application, a possible implementation manner of edge detection is provided. Edge detection is performed on the target object, and the edge of the target object is separated to obtain a second edge point of the target object. Among them, the edge detection based on the deep learning of the preselection box is not to detect whether the pixel point on the target image is an edge point, but to detect the position and category of the edge point corresponding to each preselection box. That is, according to the position of the edge point corresponding to each preselected box on the edge line, determine the category label of the target object, and then adjust the edge point corresponding to each preselected box on the edge line according to the category label, and allocate the edge point corresponding to each preselected box For the edge direction and edge distance, move the edge distance of the edge points corresponding to each pre-selected box according to the edge direction to obtain the above-mentioned second edge point. In the embodiment of the present application, through the edge detection based on the deep learning of the pre-selected frame, the position accuracy of the second edge point obtained can be higher, and the edge line of the target object obtained accordingly is also more accurate, which is conducive to improving the accuracy and efficiency of the labeling results .

In a possible implementation manner, the determining the label of the target object according to the position of the edge point on the edge line of the target object includes:

When the number of edge points on the first label is greater than the number of edge points on the second label, determining the first label as the label of the target object;

Or, when the number of edge points on the first label is less than the number of edge points on the second label, determine the second label as the label of the target object;

Or, when the number of edge points on the first label is equal to the number of edge points on the second label, according to the key degree of the first label and the key degree of the second label , to determine the label of the target object.

In the embodiment of the present application, a possible implementation manner of determining the category label of the target object is provided. Determine the category label of the target object as the category label of the target object, that is, which category the point on the edge line is in, and the category of the preselection box is that category. For example, when the number of edge points on the first label is greater than the number of edge points on the second label, the first label is determined as the category label of the target object. Similarly, on the first label If the number of edge points of is less than the number of edge points located on the second label, the second label is determined as the category label of the target object. When the number of edge points on the first label is equal to the number of edge points on the second label, the category label of the target object is further determined according to the keyness of the label. In the embodiment of the present application, the category label of the target object is determined by assigning categories to the pre-selected boxes, which can improve the accuracy and efficiency of determining the category of the target object, and adjust the edge points on the edge line of the target object accordingly, so that the obtained second The position accuracy of edge points is higher.

In a possible implementation manner, the performing segmentation processing on the target image to obtain the first edge point of the target object includes:

Inputting the target image into a convolutional neural network for the segmentation process to obtain the first edge point.

In the embodiment of this application, the image segmentation method based on deep learning is used to input the target image to the convolutional neural network for segmentation processing. Compared with the currently commonly used interactive image segmentation method, the edge points of the target object obtained by this method are more accurate. more accurate and more efficient.

In a possible implementation manner, the determining the tag of the target object according to the criticality of the first tag and the criticality of the second tag includes:

If the criticality of the first tag is higher than that of the second tag, determining the first tag as the tag of the target object;

If the criticality of the second tag is higher than that of the first tag, the second tag is determined as the tag of the target object.

In the embodiment of the present application, a possible specific implementation manner of determining the category label of the target object is provided. That is, when the number of edge points on the first label is equal to the number of edge points on the second label, the category label of the target object will be further determined according to the keyness of the label. For example, when the criticality of the first label is higher than that of the second label, the first label is determined as the category label of the target object. Similarly, when the criticality of the second label is higher than that of the first label In the case of criticality, the second label is determined as the second label of the target object. In the embodiment of the present application, the category label of the target object is determined by further comparing the label criticality, which can improve the accuracy and efficiency of determining the category of the target object, and adjust the edge points on the edge line of the target object accordingly, so that the obtained second The position accuracy of edge points is higher.

In a possible implementation manner, the method also includes:

According to the target data of the target radar, a first frame and a second frame of the target object are obtained; the first frame is different from the second frame, and the target data includes data obtained by the target radar detecting the target object;

Obtain a first contour of the target object according to the first frame;

Using the second frame to correct the frame of the first contour to obtain the second contour of the target object.

In the embodiment of the present application, by performing different processing on the target data collected by the target radar to detect the target object, different frames of the target object, such as the first frame and the second frame, can be obtained. Then, an initial mark is performed on the target object based on the first bounding box to obtain a first outline of the target object. Then, the second frame is used to correct the frame of the first contour obtained by the initial calibration, so as to obtain the second contour of the target object.

The data labeling method provided in the embodiment of this application is not only applicable to labeling two-dimensional image data, but also suitable for labeling three-dimensional radar data. Converting the correction of the edge points of the first contour of the target object to the correction of the frame of the first contour of the target object, and correcting to obtain the second contour of the target object can greatly improve the accuracy and efficiency of the labeling result.

In a possible implementation manner, the modifying the frame of the first outline by using the second frame includes:

A contour formed by the second frame and the frame of the first contour is determined as the second contour.

In a possible implementation manner, before obtaining the first frame and the second frame of the target object according to the target data of the target radar, the method further includes:

performing segmentation processing on the target object to obtain a first edge surface of the target object; the segmentation processing is used to separate the target object from the detection area of the target radar;

performing edge detection on the target object to obtain a second edge surface of the target object; the edge detection is used to separate the edge of the target object;

According to the target data of the target radar, the first frame and the second frame of the target object are obtained, including:

The first frame and the second frame are determined according to the first edge surface and the second edge surface.

In a possible implementation manner, determining the first frame and the second frame according to the first edge surface and the second edge surface includes:

According to the first edge surface and the second edge surface, the first frame is obtained; the edge surface on the first frame is an edge surface where the first edge surface and the second edge surface overlap;

The second frame is obtained according to the first edge surface, the second edge surface and the first frame.

In a possible implementation manner, obtaining the second frame according to the first edge surface, the second edge surface, and the first frame includes:

According to the first edge surface and the second edge surface, a third frame is obtained; the surface on the third frame is the first edge surface or the second edge surface, and is not the first edge surface an edge face coincident with said second edge face;

The third frame having two or more intersecting surfaces with the first frame is determined as the second frame.

In a possible implementation manner, the performing edge detection on the target object to obtain the second edge surface of the target object includes:

determining the label of the target object according to the position of the edge face on the frame of the target object;

According to the label of the target object, the edge surface on the frame of the target object is adjusted to obtain the second edge surface.

In a possible implementation manner, determining the label of the target object according to the position of the edge face on the frame of the target object includes:

When the number of edge faces on the first label is greater than the number of edge faces on the second label, determining the first label as the label of the target object;

Or, when the number of edge faces on the first label is less than the number of edge faces on the second label, determine the second label as the label of the target object;

Or, when the number of edge faces on the first label is equal to the number of edge faces on the second label, according to the key degree of the first label and the key degree of the second label , to determine the label of the target object.

In the second aspect, the embodiment of the present application provides a data labeling device, which includes:

A determining unit, configured to obtain a first edge line and a second edge line of a target object included in the target image; the first edge line and the second edge line are different;

The determining unit is further configured to obtain a first contour of the target object according to the first edge line;

A correction unit, configured to use the second edge line to correct the edge line of the first contour to obtain the second contour of the target object.

In a possible implementation manner, the correction unit is specifically configured to determine a contour formed by the second edge line and edge lines of the first contour as the second contour.

In a possible implementation manner, the device also includes:

a segmentation unit, configured to perform segmentation processing on the target image to obtain a first edge point of the target object; the segmentation processing is used to separate the target object included in the target image;

An edge detection unit, configured to perform edge detection on the target object to obtain a second edge point of the target object; the edge detection is used to separate the edge of the target object;

The determining unit is specifically configured to determine the first edge line and the second edge line according to the first edge point and the second edge point.

In a possible implementation manner, the determining unit is further configured to obtain the first edge line according to the first edge point and the second edge point; the points on the first edge line An edge point where the first edge point and the second edge point coincide;

The determining unit is specifically further configured to obtain the second edge line according to the first edge point, the second edge point, and the first edge line.

In a possible implementation manner, the determining unit is further configured to obtain a third edge line according to the first edge point and the second edge point; the points on the third edge line are the the first edge point or the second edge point, and is not an edge point where the first edge point and the second edge point coincide;

The determining unit is further configured to determine the third edge line having two or more intersection points with the first edge line as the second edge line.

In a possible implementation manner, the edge detection unit is specifically configured to determine the label of the target object according to the position of the edge point on the edge line of the target object;

The edge detection unit is further configured to adjust edge points on the edge line of the target object according to the label of the target object to obtain the second edge point.

In a possible implementation manner, the edge detection unit is further configured to, when the number of edge points on the first label is greater than the number of edge points on the second label, The label is determined as the label of the target object;

Alternatively, the edge detection unit is further configured to determine the second label as the label of the target object;

Alternatively, the edge detection unit is specifically further configured to: when the number of edge points on the first label is equal to the number of edge points on the second label, according to the key of the first label degree and the key degree of the second label, and determine the label of the target object.

In a possible implementation manner, the segmentation unit is specifically configured to input the target image into a convolutional neural network for the segmentation process to obtain the first edge point.

In a possible implementation manner, the determining unit is further configured to determine the first tag as the the label of the target audience;

The determining unit is specifically further configured to determine the second tag as the tag of the target object when the criticality of the second tag is higher than that of the first tag.

In a possible implementation manner, the determining unit is configured to obtain the first frame and the second frame of the target object according to the target data of the target radar; the first frame and the second frame are different, and the The target data includes data obtained by the target radar detecting the target object;

The determining unit is further configured to obtain a first contour of the target object according to the first frame;

The correction unit is configured to use the second frame to correct the frame of the first contour to obtain the second contour of the target object.

In a possible implementation manner, the correction unit is specifically configured to determine a contour formed by the second frame and a frame of the first contour as the second contour.

In a possible implementation manner, the segmentation unit is configured to perform segmentation processing on the target object to obtain a first edge surface of the target object; the segmentation processing is configured to segment the target object from the separated from the detection zone of the target radar;

The edge detection unit is configured to perform edge detection on the target object to obtain a second edge surface of the target object; the edge detection is used to separate the edge of the target object;

The determining unit is specifically configured to determine the first frame and the second frame according to the first edge surface and the second edge surface.

In a possible implementation manner, the determining unit is further configured to obtain the first frame according to the first edge surface and the second edge surface; the edge surface on the first frame is an edge face where the first edge face and the second edge face coincide;

The determining unit is specifically further configured to obtain the second frame according to the first edge surface, the second edge surface, and the first frame.

In a possible implementation manner, the determining unit is further configured to obtain a third frame according to the first edge surface and the second edge surface; the surface on the third frame is the an edge face or said second edge face, and is not an edge face where said first edge face and said second edge face coincide;

The determining unit is further configured to determine, as the second frame, the third frame that has two or more intersecting surfaces with the first frame.

In a possible implementation manner, the edge detection unit is specifically configured to determine the label of the target object according to the position of the edge face on the frame of the target object;

The edge detection unit is specifically further configured to adjust an edge surface on a border of the target object according to the label of the target object to obtain the second edge surface.

In a possible implementation manner, the edge detection unit is further configured to, when the number of edge faces on the first label is greater than the number of edge faces on the second label, The label is determined as the label of the target object;

Alternatively, the edge detection unit is specifically further configured to: when the number of edge faces on the first label is equal to the number of edge faces on the second label, according to the key of the first label degree and the key degree of the second label, and determine the label of the target object.

Regarding the technical effect brought about by the second aspect and any possible implementation manner, reference may be made to the introduction corresponding to the technical effect of the first aspect and the corresponding implementation manner.

In the third aspect, the embodiment of the present application provides a data labeling device, the data labeling device includes a processor and a memory; the memory is used to store computer programs or instructions; the processor is used to execute the computer stored in the memory A program or an instruction, so that the data tagging device executes the method according to the above first aspect and any possible implementation manner.

Optionally, the data tagging device further includes a transceiver, configured to receive data or send data.

In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, which is used to store a computer program; when the computer program is executed, the first aspect and any possible implementation The method described in the manner is implemented.

In the fifth aspect, the embodiment of the present application provides a computer program product, the computer program product includes a computer program; when the computer program is executed, the method described in the first aspect and any possible implementation manner is executed accomplish.

In the sixth aspect, the embodiment of the present application provides a chip, the chip includes a processor, the processor is used to execute instructions, and when the processor executes the instructions, the chip performs the first aspect and any one of Possible implementations of the methods described.

Optionally, the chip further includes a communication interface for inputting data or outputting data.

In a seventh aspect, an embodiment of the present application provides a terminal, the terminal comprising at least one data tagging device according to the second aspect or the third aspect, or the chip according to the sixth aspect.

In an eighth aspect, the embodiment of the present application provides a server, the server comprising at least one data tagging device according to the second aspect or the third aspect, or the chip according to the sixth aspect.

Optionally, during the process of executing the method described in the above first aspect and any possible implementation manner, the above-mentioned processor may be a processor specially used to execute these methods, or may be executed by a computer in the memory instructions to perform these methods, such as a general-purpose processor. The above-mentioned memory can be a non-transitory (non-transitory) memory, such as a read-only memory (Read Only Memory, ROM), which can be integrated with the processor on the same chip, or can be respectively arranged on different chips. The embodiment does not limit the type of the memory and the arrangement of the memory and the processor.

In a possible implementation manner, the above at least one memory is located outside the device.

In yet another possible implementation manner, the at least one memory is located within the device.

In yet another possible implementation manner, part of the memory of the at least one memory is located inside the device, and another part of the memory is located outside the device.

In this application, the processor and the memory may also be integrated into one device, that is, the processor and the memory may also be integrated together.

In the embodiment of the present application, the edge detection of the target object in the target image is carried out by proposing a deep learning method based on the pre-selected frame, and the first contour of the target object is obtained, and then the weak semantic information is introduced to the edge points of the first contour of the target object The correction of the target object is transformed into the correction of the edge line of the first contour of the target object, and the second contour of the target object is obtained through correction, which can greatly improve the accuracy and efficiency of the labeling result.

Description of drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the accompanying drawings that need to be used in the embodiments of the present application. Obviously, the accompanying drawings described below are only some embodiments of the present application. Those of ordinary skill in the art can also obtain other drawings based on these drawings without making creative efforts.

Fig. 1 is a schematic diagram of the effect of a data annotation provided by the embodiment of the present application;

FIG. 2 is a schematic diagram of the architecture of a data labeling system provided by an embodiment of the present application;

FIG. 3 is a schematic diagram of an application scenario of data labeling provided by an embodiment of the present application;

FIG. 4 is a schematic flow chart of a data labeling method provided by an embodiment of the present application;

Fig. 5a is a schematic diagram of the effect of data labeling provided by the embodiment of the present application;

Fig. 5b is a schematic diagram of the effect of data labeling provided by the embodiment of the present application;

Fig. 5c is a schematic diagram of the effect of data labeling provided by the embodiment of the present application;

FIG. 6a is a schematic diagram of the architecture of a data labeling system provided by an embodiment of the present application;

FIG. 6b is a schematic diagram of the architecture of a data labeling system provided by an embodiment of the present application;

FIG. 6c is a schematic diagram of the architecture of a data labeling system provided by the embodiment of the present application;

FIG. 7 is a schematic flowchart of another data labeling method provided by the embodiment of the present application;

FIG. 8 is a schematic structural diagram of a data labeling device provided by an embodiment of the present application;

FIG. 9 is a schematic structural diagram of an electronic device provided by an embodiment of the present application.

Detailed ways

In order to make the purpose, technical solution and advantages of the present application clearer, the following will describe the embodiments of the present application in conjunction with the accompanying drawings in the embodiments of the present application.

The terms "first" and "second" in the specification, claims and drawings of the present application are used to distinguish different objects, rather than to describe a specific order. Furthermore, the terms "comprising" and "having", as well as any variations thereof, are intended to cover a non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the listed steps or units, but optionally also includes steps or units that are not listed, or optionally It also includes other steps or units inherent to these processes, methods, products, or devices.

Reference herein to "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment may be included in at least one embodiment of the present application. The occurrences of this phrase in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. It is understood explicitly and implicitly by those skilled in the art that the embodiments described herein can be combined with other embodiments.

It should be understood that in this application, "at least one (item)" means one or more, "multiple" means two or more, and "at least two (items)" means two or three And three or more, "and/or", is used to describe the association relationship of associated objects, indicating that there can be three types of relationships, for example, "A and/or B" can mean: only A exists, only B exists, and A exists at the same time and B, where A and B can be singular or plural. The character "/" generally indicates that the contextual objects are an "or" relationship. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one item (piece) of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c ", where a, b, c can be single or multiple.

This application provides a data labeling method. In order to describe the solution of this application more clearly, the following will introduce some knowledge related to data labeling in combination with the drawings in the embodiments of this application, and then describe the embodiments of this application.

Please refer to FIG. 1 . FIG. 1 is a schematic diagram of an effect of data labeling provided by an embodiment of the present application.

Edge surface: (A) in Figure 1 is the three-dimensional model of the target object (target vehicle) obtained by the vehicle-mounted radar according to the collected target data. The three-dimensional model of the target vehicle is composed of several edge surfaces, which can be passed through It is obtained by marking the collected target data, as shown in Figure 1 (B) is the edge surface.

Edge line: if (B) in Fig. 1 is used as the two-dimensional image model of the target object (target vehicle), then the two-dimensional image model of the target vehicle is composed of several edge lines, and the edge line can be collected by The two-dimensional image data is marked, and (C) in Figure 1 is the edge line.

Edge points: Several edge points are obtained by marking the collected two-dimensional image data, and the edge line (C) in Figure 1 can be obtained from these edge points, and the black points marked on the edge line are edge points.

Image annotation: refers to the process of adding textual feature information to the image to reflect its content through the method of machine learning for the visual content of the image. The basic idea is to use the labeled image set or other available information to automatically learn the potential association or mapping relationship between the semantic concept space and the visual feature space, and add text keywords to unknown images. Through the processing of image automatic labeling technology, the problem of image information can be transformed into the problem of text information processing with relatively mature technology.

Image data annotation is widely used in fields such as autonomous driving, face recognition, and medical image recognition. Among them, there are two main methods of labeling image data in the field of autonomous driving, one is frame labeling, and the other is fine segmentation labeling.

The method of drawing frame labeling is to make the edge of the frame close to the edge of the target object when labeling the image, and at the same time indicate the attributes of each frame. For the algorithm, each box is a small image, and each small image corresponds to an object category. Taking vehicles as an example, through the above method of drawing frame labels, you can get cars, vans, minivans, etc. in the image Different classes of cars. When labeling, you need to pay special attention to the tangent between the frame and the edge of the vehicle. If it is not tangent, such as selecting a part that does not belong to the vehicle, then the machine may recognize the part selected by multiple frames as a vehicle during learning. , resulting in inaccurate or even wrong recognition of the machine. In addition, image data labeling requires a large amount of labeling data. During the learning of large amounts of data, machine algorithms will summarize the high-dimensional features of these objects by themselves. When identifying new images, they can use the summarized high-dimensional features to identify new images. Each possible outcome is given a probability.

The segmentation and labeling method uses an interactive image segmentation method. For a given initial area of the original image, the segmentation algorithm is used to segment the target object in the initial area, thereby generating a series of edge points, and then correcting by correcting the edge points. Split results.

The following is a brief description of the segmentation and labeling method as an example. Assuming that there are various types of objects such as people, cars, street trees, and buildings in the original image (a), the segmentation and labeling needs to identify different types of objects in the image and detect their positions. to split.

The first is to perform semantic segmentation on the original image (a), that is, classify each pixel in the original image (a), determine the category of each pixel (such as belonging to the background, person or car, etc.), so that the original image Each object in (a) is marked with a category label, and the objects contained in the image are divided into people, street trees, cars, and building labels, and are divided into regions to obtain image (b).

Then, instance segmentation is performed on the image (b) obtained by semantic segmentation. Instance segmentation is a combination of target detection and semantic segmentation, that is, the target object is detected in the image, and then combined with the category of each pixel determined by the above semantic segmentation, Determine the category label of the target object, resulting in image (c). For example, taking a car as the target object, semantic segmentation does not distinguish different instances belonging to the same car category (for example, all car objects are marked in red), while instance segmentation combines the results of semantic segmentation and target object detection to distinguish different instances of the same type (For example, use different colors to distinguish different car categories).

Finally, panoramic segmentation is performed on the image (c) obtained by instance segmentation. Panoramic segmentation is a combination of semantic segmentation and instance segmentation, that is, all objects in the image are detected, and at the same time, different instances in the same category must be distinguished. Get image (d). Comparing image (c) and image (d), it can be seen that instance segmentation only detects the target object in the image (such as the car in the picture) and segments it by pixel category to distinguish different instances (using different colors), while panoramic segmentation It is to detect and segment all objects in the picture, including the background, to distinguish different instances (using different colors).

Regardless of the above-mentioned frame-drawing labeling method or segmentation labeling method, the edge points of the generated target objects need to be corrected in the end, and it is time-consuming and labor-intensive to correct the edge points one by one, and the accuracy and efficiency of the labeling results are low.

To solve the problem of low accuracy and efficiency of labeling results in the above data labeling methods, the embodiment of the present application provides a data labeling system and a new data labeling method based on the data labeling system. This method detects the edge of the target object in the target image by proposing a deep learning method based on the pre-selected frame to obtain the first contour of the target object, and then introduces weak semantic information to convert the correction of the edge points of the first contour of the target object into The correction of the edge line of the first contour of the target object, and the correction to obtain the second contour of the target object can greatly improve the accuracy and efficiency of the labeling result.

The data labeling system and the data labeling method based on the data labeling system will be described respectively below.

Please refer to FIG. 2 . FIG. 2 is a schematic diagram of an architecture of a data labeling system provided by an embodiment of the present application.

As shown in Figure 2, the system architecture mainly includes three aspects: a data acquisition module, a storage module, and a data labeling module. Among them, the data collection module is used to collect the data to be marked, specifically, the image data can be collected through the camera, or the target data can be collected through the radar. The storage module is used to store the data to be marked collected by the data acquisition module. The storage module can be a cloud or a local server, and realizes data communication with the data acquisition module through a mobile network. The data labeling module can be an independent module, or an intelligent driving platform, such as a mobile data center (Mobile Data Center, MDC), or an intelligent cockpit platform, such as a cockpit domain controller (Cockpit Domain Controller, CDC), or It is a vehicle control platform, such as a vehicle domain controller (Vehicle Domain Controller, VDC), etc., which is used to obtain the data to be marked from the cloud or server for labeling, obtain the labeling results, and store the labeling results in the cloud or server. so that it can be used in subsequent data identification.

It can be understood that the data annotation system architecture in FIG. 2 is only an exemplary implementation in the embodiment of the present application, and the data annotation system architecture in the embodiment of the application includes but not limited to the above data annotation system architecture.

Correspondingly, based on the data labeling system provided in the embodiment of the present application, a new data labeling method is also provided, and the data labeling process of the method is completed by the data labeling module in the above data labeling system. This method can be applied to various scenarios such as automatic driving, portrait recognition, medical image recognition, etc. The following will take the data labeling in the application scenario of automatic driving as an example to illustrate.

Please refer to FIG. 3 . FIG. 3 is a schematic diagram of an application scenario of data labeling provided by an embodiment of the present application.

As shown in FIG. 3 , the application scenario may include a cloud network and multiple vehicle driving control devices (test vehicle, user vehicle a, user vehicle b, and user vehicle c). The above-mentioned multiple vehicle driving control devices can communicate data with the cloud network, thereby realizing data interaction between the test vehicle and the user vehicle, as well as data interaction between different user vehicles (user car a, user b, user car c) . In particular, the above-mentioned vehicle driving control device is a smart car that senses the road environment through a camera or an on-board sensor system, automatically plans a driving route, and controls the vehicle to reach a predetermined target. Smart cars use computers, modern sensing, information fusion, communication, artificial intelligence, and automatic control technologies in a concentrated manner. They are a high-tech complex integrating environmental perception, planning and decision-making, and multi-level assisted driving. Among them, the vehicles in this application (such as test vehicle, user vehicle a, user vehicle b, and user vehicle c) can be vehicles that mainly rely on the computer system-based intelligent driver in the vehicle to achieve the purpose of unmanned driving. It is an intelligent vehicle with an assisted driving system or a fully automatic driving system, and it can also be a wheeled mobile robot.

Based on the above application scenarios, in a possible implementation manner, the data labeling method provided in the embodiment of the present application is completed by a test vehicle. In the driving scene corresponding to the target road section, the test vehicle collects the image data of the target road section during driving through the camera, and relies on the computer system in the car to complete the labeling of the image data, obtain the corresponding data label, and transmit the data label to the cloud network middle. During the actual driving of the user's car, the image data of the driving section is collected through the camera, and the data label marked by the test car is obtained from the cloud network, and the data label is combined with the image recognition algorithm to automatically identify the objects in the collected image, thereby To achieve the purpose of sensing the road environment, automatically planning the driving route and controlling the vehicle to avoid obstacles and objects, and arrive at the predetermined address.

In another possible implementation manner, the data labeling method provided in the embodiment of the present application is completed by the user's car. Due to the complex and changeable road conditions in the actual driving process of the vehicle, it is difficult to complete the recognition of all the collected image data only by relying on the data labels obtained by the test vehicle in the test of the target road section. At this time, the user's car needs to rely on the computer system in the car to complete the labeling of the image data, obtain the corresponding data label, and use the data label combined with the image recognition algorithm to automatically identify the objects in the collected image, so as to achieve road perception. Environment, automatically plan the driving route and control the vehicle to avoid obstacles and objects, and reach the purpose of the predetermined address. In addition, the user's car will also transmit the data label to the cloud network so that it can be used for subsequent image recognition of other user's cars.

It can be understood that the data labeling in the automatic driving application scenario in Figure 3 is only an exemplary implementation in the embodiment of the present application, and the data labeling application scenario in the embodiment of the present application includes but is not limited to the above automatic driving application Scenes.

Please refer to Figure 4, Figure 4 is a schematic flow chart of a data labeling method provided by the embodiment of the present application, which includes but is not limited to the following steps:

Step 401: The electronic device obtains the first edge line and the second edge line of the target object included in the target image.

The electronic device in the embodiment of the present application is a device equipped with a processor capable of executing instructions executed by a computer. The electronic device may be a terminal such as a computer or a controller, or may be a server or the like. Specifically, it can be the vehicle-mounted device equipped with a computer system in the above-mentioned Figure 3, which is used to label the collected image data, obtain the corresponding data label, and use the data label combined with the image recognition algorithm to automatically identify the collected images. Existing objects, in order to realize the purpose of sensing the road environment, automatically planning the driving route and controlling the vehicle to avoid obstacles and objects, and reach the predetermined address.

In the embodiment of the present application, the electronic device obtains the first edge line and the second edge line of the target object included in the target image by performing different processes on the target image, and the first edge line and the second edge line are different.

The electronic device performs different processing on the target image. Specifically, the target image may be segmented to separate the target object in the target image, so as to obtain the first edge point of the target object. The segmentation processing here includes image segmentation methods based on non-deep learning (such as segmentation methods based on watershed algorithm, graph theory, etc.) and image segmentation methods based on deep learning. The main difference between the two is that image segmentation methods based on deep learning require Using the convolutional neural network, the target image is input to the convolutional neural network for segmentation. After the target object in the target image is separated, an edge detection is performed on the target object, and the edge of the target object is separated to obtain a second edge point of the target object. The edge detection here includes edge detection based on deep learning of non-preselected boxes and edge detection based on deep learning of preselected boxes. Whether the pixel is an edge point, but to detect the position and category of the edge point corresponding to each preselection box.

Although the above-mentioned first edge point and the second edge point are both edge points of the target object, they are edge points obtained by adopting different methods for the target object. Therefore, the above-mentioned first edge point and the second edge point are not completely the same. According to the first edge point and the second edge point, the first edge line and the second edge line of the above-mentioned target object can be determined. Since the first edge point and the second edge point are not completely the same, correspondingly, the first edge line and the second edge line obtained accordingly are also not completely the same.

In a possible implementation manner, the above-mentioned determination of the first edge line and the second edge line of the target object according to the first edge point and the second edge point may specifically be that the first edge point and the second edge point coincide The line formed by the edge points is determined as the first edge line, that is, the first edge line is located on the edge result obtained by the above-mentioned segmentation processing and edge detection, and the above-mentioned segmentation processing and edge detection can both obtain the first edge line. Then the line formed by the union of the first edge point and the second edge point after removing the edge points where the first edge point coincides with the second edge point is determined as the third edge line, that is, the third edge line is located in the above-mentioned segmentation process. The edge results obtained, or located on the edge results obtained by the above-mentioned edge detection, are not all located on the edge results obtained by the segmentation process and edge detection. Finally, the third edge line having two or more intersection points with the first edge line is determined as the second edge line.

Through the specific implementation of obtaining the first edge line and the second edge line provided by the embodiment of the present application, the accuracy of the acquired edge line of the target object can be improved, and it is beneficial to the subsequent analysis of the edge of the first contour of the target object The correction of the point is transformed into the correction of the edge line of the first contour of the target object, which improves the accuracy and efficiency of the labeling result.

Exemplarily, the above implementation manner of obtaining the first edge line and the second edge line of the target object included in the target image can also be further described through a schematic diagram of the effect of data annotation.

Please refer to FIG. 5a, which is a schematic diagram of an effect of data labeling provided by the embodiment of the present application.

As shown in FIG. 5 a , by performing segmentation processing on the target image, the target object in the target image is separated to obtain the first edge point of the target object. Then perform edge detection on the target object, separate the edge of the target object, and obtain the second edge point of the target object. The line formed by the coincident edge points of the first edge point and the second edge point (ie, the edge point marked in FIG. 5 a ) is determined as the first edge line of the target object (ie, the edge line a in FIG. 5 a ). It can be seen that the first edge lines are located on the edge results obtained by the above segmentation processing and edge detection.

Then the line formed after the union of the above-mentioned first edge point and the second edge point is removed after the edge point where the first edge point and the second edge point coincide (that is, the edge point marked in Fig. 5a) is determined as the third edge line . It can be seen that the third edge line is located on the edge result obtained by the above-mentioned segmentation processing, or on the edge result obtained by the above-mentioned edge detection, but not all of them are located on the edge results obtained by the segmentation processing and edge detection. Finally, the third edge line having two or more intersection points with the first edge line (ie, edge line a in FIG. 5 a ) is determined as the second edge line (ie, edge line b in FIG. 5 a ).

Exemplarily, the implementation manner of the above-mentioned edge detection based on the deep learning of the pre-selection box can also be further described through the schematic diagram of the effect of the data labeling.

Please refer to FIG. 5b, which is a schematic diagram of an effect of data labeling provided by the embodiment of the present application.

As shown in Figure 5b, a brief area division is made for the content contained in the given target image. The hexagon is the actual outline of the target object, and the closed area within the hexagon is the internal area of the target object, including the hexagon The ellipse area of is the sensitive area of the outline of the target object, and it is necessary to mark the target object in this sensitive area to obtain its precise outline. In the process of marking the target object in the sensitive area, the edge detection of the target object based on the deep learning of the pre-selection box can be separated to obtain the edge point of the target object.

Wherein, the edge detection based on the deep learning of the pre-selection box can refer to FIG. 5c for details, and FIG. 5c is a schematic diagram of an effect of data labeling provided by the embodiment of the present application.

As shown in Figure 5c, in the process of labeling the target image, the target object needs to be divided into several grid units, and the preselected content in the preselection box here is all the horizontal axes of the grid into which the target image is divided segment, such as the line segment marked by the oval area in Figure 5c. The edge detection based on the deep learning of the preselection box is not to detect whether the pixel point on the target image is an edge point, but to detect the position and category of the edge point on the horizontal axis segment corresponding to each preselection box. That is, according to the position of the edge point on the horizontal axis segment corresponding to each preselected box on the edge line, determine the category label of the target object, and label the edge point on the horizontal axis segment corresponding to each preselected box on the edge line of the target object The higher category is determined as the category label of the target object. For example, when the number of edge points on the first label is greater than the number of edge points on the second label, the first label is determined as the category label of the target object. Similarly, on the first label If the number of edge points of is less than the number of edge points located on the second label, the second label is determined as the category label of the target object. When the number of edge points on the first label is equal to the number of edge points on the second label, the category label of the target object is further determined according to the keyness of the label. For example, when the criticality of the first label is higher than that of the second label, the first label is determined as the category label of the target object. Similarly, when the criticality of the second label is higher than that of the first label In the case of criticality, the second label is determined as the second label of the target object. Specifically, the criticality of tags is not a fixed value, and can vary according to different application scenarios. In automatic driving application scenarios, by default, people are more important than cars, and cars are more important than street trees. , therefore, the criticality of people is higher than that of cars, and the criticality of cars is higher than that of street trees. After determining the category label of the target object, adjust the edge points corresponding to each preselection box on the edge line according to the category label, assign the edge direction and edge distance to the edge points corresponding to each preselection box, and move the corresponding edge points of each preselection box according to the edge direction The edge distance of the edge point, so as to obtain the edge point of the target object, that is, the above-mentioned second edge point.

In the embodiment of the present application, through the edge detection based on the deep learning of the pre-selected frame, the pre-selected frame is assigned a category label, and the key degree of the label is further compared to determine the category label of the target object, which can improve the accuracy and efficiency of determining the target object category, and based on this Adjusting the edge points on the edge line of the target object can make the position accuracy of the second edge point obtained higher, and the edge line of the target object obtained accordingly is also more accurate, which is conducive to improving the accuracy and efficiency of the labeling results.

Step 402: Obtain a first contour of the target object according to the first edge line.

After obtaining the first edge line of the target object, the electronic device initially marks the target object based on the first edge line to obtain the first outline of the target object. The first contour can be used as the final labeled contour result of the target object, or as the candidate contour result of the target object label, and on this basis, the accuracy of the labeling result can be improved by correcting the edge of the candidate contour.

Exemplarily, in the above-mentioned FIG. 5a, the obtained edge line a is used to initially mark the target object to obtain the first contour of the target object. It can be seen that the actual contour of the target object is slightly different from the obtained first contour, and the first contour does not completely cover the target object and is tangent to the edge of the target object. It is necessary to correct the edge of the first contour by using the edge line b on the basis of the initial mark to improve the accuracy of the labeling result.

Step 403: Using the second edge line to correct the edge line of the first contour to obtain a second contour of the target object.

The electronic device uses the second edge line to correct the edge line of the first contour obtained from the initial calibration to obtain the second contour of the target object. Specifically, the contour formed by the second edge line and the edge line of the first contour may be determined as the second contour of the target object. This correction method improves the accuracy and efficiency of the labeling result.

Exemplarily, in the above-mentioned FIG. 5a, on the basis of using the edge line a to initially mark the target object to obtain the first contour, and then using the edge line b to correct the edge of the first contour to improve the accuracy of the labeling result, The second contour of the target object obtained after correction completely covers the target object and is tangent to the edge of the target object.

Correspondingly, the data labeling method provided in FIG. 4 above may correspond to the data labeling system architecture shown in FIGS. 6a to 6c. The architecture of the data labeling system in FIGS. 6 a to 6 c will be described below in combination with the data labeling method in FIG. 4 .

Please refer to FIG. 6a. FIG. 6a is a schematic structural diagram of a data labeling system provided by an embodiment of the present application.

As shown in Figure 6a, the image data of the target image is first obtained, and then image segmentation based on non-deep learning, image segmentation based on deep learning, edge detection based on deep learning of preselected boxes, and depth of non-preselected boxes are respectively performed on the image data. The learned edge detection obtains the results of different edge information about the target object, and generates strong semantic information and weak semantic information according to the results obtained by the above different processing.

It can be understood that the strong semantic information is the edge line a in the above-mentioned Figure 5a or the first edge line in the above-mentioned Figure 4, that is, both are located on the result of the edge information obtained by the above-mentioned different processing, in other words, the edge a obtained by the above-mentioned different processing The overlapping part of the information is the strong semantic information. Correspondingly, the weak semantic information is the edge line b in the above-mentioned Figure 5a or the second edge line in the above-mentioned Figure 4, that is, it is only located on the result of the edge information obtained by one of the above-mentioned different processes, in other words, the above-mentioned different The non-overlapping part of the processed edge information is the weak semantic information.

Finally, the target object is initially marked based on the generated strong semantic information, and the result of the initial mark is corrected based on the generated weak semantic information. Combining with the above-mentioned data labeling method in FIG. 4, it can correspond to the above-mentioned step 402: obtain the first outline of the target object according to the first edge line; and step 403: use the second edge line to perform Correction, to obtain the second contour of the target object. Combined with the specific embodiment of the data labeling in Fig. 5a, it can respectively correspond to, use the edge line a to initially mark the target object to obtain the first outline of the target object; and use the edge line b to perform initial marking on the edge line of the first outline Correction to obtain the second contour of the target object.

Further, strong semantic information is generated according to the results obtained from the above-mentioned different processes. The specific process can be referred to FIG. 6b, which is a schematic diagram of the architecture of a data labeling system provided by an embodiment of the present application.

As shown in Figure 6b, image segmentation based on deep learning and image segmentation based on non-deep learning are respectively performed on the image data of the target image, and the results of edge information about the target object obtained by the two are fused to generate edge points of the target object . Then perform edge detection based on deep learning of pre-selected boxes and edge detection based on deep learning of non-pre-selected boxes on the image data of the target image, fuse the edge detection results obtained by the two, and use the edge detection fusion results for the target generated above The correction of the edge points of the object, thereby obtaining strong semantic information, representing the first edge line of the target object.

Further, the weak semantic information is generated according to the results obtained from the above-mentioned different processes. For the specific process, please refer to FIG. 6c, which is a schematic diagram of the architecture of a data labeling system provided by the embodiment of the present application.

As shown in Figure 6c, combined with the strong semantic information generated above, image segmentation based on non-deep learning, image segmentation based on deep learning, edge detection based on deep learning of preselected boxes, and deep learning based on non-preselected boxes are respectively performed on image data. Edge detection, which can generate weak semantic information.

Please refer to Figure 7, Figure 7 is a schematic flowchart of another data labeling method provided by the embodiment of the present application, which includes but is not limited to the following steps:

Step 701: The electronic device obtains the first frame and the second frame of the target object according to the target data of the target radar.

The electronic device in the embodiment of the present application is a device equipped with a processor capable of executing instructions executed by a computer. The electronic device may be a terminal such as a computer or a controller, or may be a server or the like. Specifically, it can be the vehicle-mounted equipment equipped with a computer system in the above-mentioned Figure 3, which is used to mark the target data collected by the vehicle-mounted target radar, obtain the corresponding data label, and use the data label combined with the recognition algorithm to automatically identify the collected data. Objects in the target data to realize the purpose of sensing the road environment, automatically planning the driving route and controlling the vehicle to avoid obstacles and objects, and reach the predetermined address.

In the embodiment of the present application, the electronic device obtains the first frame and the second frame of the target object by performing different processes on the target data collected by the target radar to detect the target object, and the first frame and the second frame are different.

The electronic equipment performs different processing on the target data collected by the target radar to detect the target object. Specifically, the target object can be separated from the detection area of the target radar by performing segmentation processing on the target object, so as to obtain the first an edge face. Then perform edge detection on the target object, separate the edge of the target object, and obtain the second edge surface of the target object.

Although the above-mentioned first edge surface and the second edge surface are both edge surfaces of the target object, they are edge surfaces obtained by adopting different methods for the target object. Therefore, the above-mentioned first edge surface and the second edge surface are not completely the same. According to the first edge surface and the second edge surface, the first frame and the second frame of the target object can be determined. Since the above-mentioned first edge surface and the second edge surface are not completely the same, correspondingly, the first frame and the second frame obtained accordingly are also not completely the same.

In a possible specific implementation manner, the above-mentioned determination of the first frame and the second frame of the target object according to the first edge surface and the second edge surface may specifically be the edge where the first edge surface and the second edge surface overlap The frame formed by the plane is determined as the first frame, that is, the first frame is located on the edge result obtained by the above-mentioned segmentation processing and edge detection, and the above-mentioned segmentation processing and edge detection can both obtain the first frame. Then, the frame formed by the union of the first edge face and the second edge face is determined as the third frame after removing the overlapping edge face of the first edge face and the second edge face, that is, the third frame is located at the position obtained by the above segmentation process. On the edge result, or on the edge result obtained by the above-mentioned edge detection, not all on the edge result obtained by the segmentation process and the edge detection. Finally, the third frame that has two or more intersecting faces with the first frame is determined as the second frame.

In a possible specific implementation manner, the above-mentioned edge detection is performed on the target object to obtain the second edge surface of the target object. Specifically, the label of the target object may be determined according to the position of the edge surface on the border of the target object. For example, when the number of edge faces on the first label is greater than the number of edge faces on the second label, the first label is determined as the label of the target object; similarly, the edge faces on the first label If the number of faces is smaller than the number of edge faces on the second label, the second label is determined as the label of the target object. When the number of edge faces on the first label is equal to the number of edge faces on the second label, further determine the label of the target object according to the keyness of the first label and the keyness of the second label. For example, when the criticality of the first label is higher than that of the second label, the first label is determined as the category label of the target object. Similarly, when the criticality of the second label is higher than that of the first label In the case of criticality, the second label is determined as the second label of the target object. Specifically, the criticality of tags is not a fixed value, and can vary according to different application scenarios. In automatic driving application scenarios, by default, people are more important than cars, and cars are more important than street trees. , therefore, the criticality of people is higher than that of cars, and the criticality of cars is higher than that of street trees. After the category label of the target object is determined, the position of the edge surface on the border of the target object is adjusted according to the category label to obtain a second edge surface.

Step 702: Obtain a first contour of the target object according to the first frame.

After obtaining the first frame of the target object, the electronic device initially marks the target object based on the first frame to obtain the first contour of the target object. The first contour can be used as the final labeled contour result of the target object, or as the candidate contour result of the target object label, and on this basis, the accuracy of the labeling result can be improved by correcting the frame of the candidate contour.

Step 703: Use the second frame to correct the frame of the first contour to obtain a second contour of the target object.

The electronic device uses the second frame to correct the frame of the first contour obtained from the initial calibration to obtain the second contour of the target object. Specifically, the contour formed by the second frame and the frame of the first contour may be determined as the second contour of the target object. This correction method improves the accuracy and efficiency of the labeling result.

In the embodiment of the present application, different frames of the target object, such as the first frame and the second frame, can be obtained by performing different processes on the target data collected by the target radar to detect the target object. Then, an initial mark is performed on the target object based on the first bounding box to obtain a first outline of the target object. Then, the second frame is used to correct the frame of the first contour obtained by the initial calibration, so as to obtain the second contour of the target object.

The data labeling method provided in the embodiment of the present application is not only suitable for labeling two-dimensional image data, but also suitable for labeling three-dimensional radar data. Converting the correction of the edge points of the first contour of the target object to the correction of the frame of the first contour of the target object, and correcting to obtain the second contour of the target object can greatly improve the accuracy and efficiency of the labeling result.

The method of the embodiment of the present application has been described in detail above, and the device of the embodiment of the present application is provided below.

Please refer to FIG. 8. FIG. 8 is a schematic structural diagram of a data labeling device provided by an embodiment of the present application. The data labeling device 80 may include a determination unit 801 and a correction unit 802, where each unit is described as follows:

A determining unit 801, configured to obtain a first edge line and a second edge line of a target object included in the target image; the first edge line and the second edge line are different;

The determining unit 801 is further configured to obtain a first contour of the target object according to the first edge line;

The correction unit 802 is configured to use the second edge line to correct the edge line of the first contour to obtain the second contour of the target object.

In a possible implementation manner, the correction unit 802 is specifically configured to determine a contour formed by the second edge line and edge lines of the first contour as the second contour.

In a possible implementation manner, the device also includes:

A segmentation unit 803, configured to perform segmentation processing on the target image to obtain a first edge point of the target object; the segmentation processing is used to separate the target object contained in the target image;

An edge detection unit 804, configured to perform edge detection on the target object to obtain a second edge point of the target object; the edge detection is used to separate the edge of the target object;

The determining unit 801 is specifically configured to determine the first edge line and the second edge line according to the first edge point and the second edge point.

In a possible implementation manner, the determining unit 801 is further configured to obtain the first edge line according to the first edge point and the second edge point; The point is an edge point where the first edge point and the second edge point coincide;

The determining unit 801 is specifically further configured to obtain the second edge line according to the first edge point, the second edge point, and the first edge line.

In a possible implementation manner, the determining unit 801 is further configured to obtain a third edge line according to the first edge point and the second edge point; the points on the third edge line are The first edge point or the second edge point, which is not an edge point where the first edge point and the second edge point overlap;

The determining unit 801 is further configured to determine the third edge line having two or more intersection points with the first edge line as the second edge line.

In a possible implementation manner, the edge detection unit 804 is specifically configured to determine the label of the target object according to the position of the edge point on the edge line of the target object;

The edge detection unit 804 is further configured to adjust edge points on the edge line of the target object according to the label of the target object to obtain the second edge point.

In a possible implementation manner, the edge detection unit 804 is further configured to: when the number of edge points on the first label is greater than the number of edge points on the second label, a label is determined as the label of the target object;

Alternatively, the edge detection unit 804 is specifically further configured to determine the second label when the number of edge points on the first label is less than the number of edge points on the second label. a label for the target audience;

Alternatively, the edge detection unit 804 is specifically further configured to: when the number of edge points on the first label is equal to the number of edge points on the second label, according to the The key degree and the key degree of the second label determine the label of the target object.

In a possible implementation manner, the segmentation unit 803 is specifically configured to input the target image into a convolutional neural network for the segmentation process to obtain the first edge point.

In a possible implementation manner, the determining unit 801 is further configured to determine the first tag as the label of the target object;

The determining unit 801 is specifically further configured to determine the second tag as the tag of the target object in the case that the criticality of the second tag is higher than that of the first tag.

In a possible implementation manner, the determining unit 801 is configured to obtain the first frame and the second frame of the target object according to the target data of the target radar; the first frame and the second frame are different, so The target data includes data obtained by the target radar detecting the target object;

The determining unit 801 is further configured to obtain a first contour of the target object according to the first frame;

The correction unit 802 is configured to use the second frame to correct the frame of the first contour to obtain the second contour of the target object.

In a possible implementation manner, the correction unit 802 is specifically configured to determine the contour formed by the second frame and the frame of the first contour as the second contour.

In a possible implementation manner, the segmentation unit 803 is configured to perform segmentation processing on the target object to obtain the first edge surface of the target object; the segmentation processing is used to segment the target object from the separated from the detection area of the target radar;

The edge detection unit 804 is configured to perform edge detection on the target object to obtain a second edge surface of the target object; the edge detection is used to separate the edge of the target object;

The determining unit 801 is specifically configured to determine the first frame and the second frame according to the first edge surface and the second edge surface.

In a possible implementation manner, the determining unit 801 is further configured to obtain the first frame according to the first edge face and the second edge face; the edge face on the first frame an edge surface where the first edge surface and the second edge surface coincide;

The determining unit 801 is specifically further configured to obtain the second frame according to the first edge surface, the second edge surface, and the first frame.

In a possible implementation manner, the determining unit 801 is further configured to obtain a third frame according to the first edge surface and the second edge surface; the surface on the third frame is the the first edge face or the second edge face, and is not an edge face where the first edge face and the second edge face coincide;

The determining unit 801 is further configured to determine, as the second frame, the third frame that has two or more intersecting faces with the first frame.

In a possible implementation manner, the edge detection unit 804 is specifically configured to determine the label of the target object according to the position of the edge plane on the frame of the target object;

The edge detection unit 804 is further configured to adjust the edge plane on the frame of the target object according to the label of the target object to obtain the second edge plane.

In a possible implementation manner, the edge detection unit 804 is specifically further configured to, when the number of edge faces on the first label is greater than the number of edge faces on the second label, a label is determined as the label of the target object;

Alternatively, the edge detection unit 804 is further configured to determine the number of edge faces on the first label is less than the number of edge faces on the second label a label for the target audience;

Alternatively, the edge detection unit 804 is specifically further configured to: when the number of edge faces on the first label is equal to the number of edge faces on the second label, according to the The key degree and the key degree of the second label determine the label of the target object.

Regarding the technical effect brought about by any possible implementation in the embodiments of the present application, reference may be made to the introduction corresponding to the first aspect of the summary of the invention and the technical effect of the corresponding implementation.

According to the embodiment of the present application, each unit in the device shown in FIG. 8 can be separately or all combined into one or several other units to form, or one (some) units can be further divided into more functional units. It is composed of multiple small units, which can achieve the same operation without affecting the realization of the technical effects of the embodiments of the present application. The above-mentioned units are divided based on logical functions. In practical applications, the functions of one unit may also be realized by multiple units, or the functions of multiple units may be realized by one unit. In other embodiments of the present application, the network-based device may also include other units. In practical applications, these functions may also be assisted by other units, and may be implemented cooperatively by multiple units.

It should be noted that, for implementation of each unit, reference may also be made to corresponding descriptions of the method embodiments shown in FIG. 4 and FIG. 7 above.

In the data labeling device 80 described in FIG. 8 , the edge detection of the target object in the target image is performed by proposing a deep learning method based on the pre-selected frame, and the first contour of the target object is obtained, and then the weak semantic information is introduced to the target object. The correction of the edge points of the first contour is transformed into the correction of the edge line of the first contour of the target object, and the correction obtains the second contour of the target object, which can greatly improve the accuracy and efficiency of the labeling result.

Please refer to FIG. 9 . FIG. 9 is a schematic structural diagram of an electronic device 90 provided in an embodiment of the present application. The electronic device 90 may include a memory 901 and a processor 902 . Further optionally, a communication interface 903 and a bus 904 may also be included, wherein the memory 901 , the processor 902 and the communication interface 903 are connected to each other through the bus 904 . The communication interface 903 is used for data interaction with the above-mentioned data tagging device 80 .

Wherein, the memory 901 is used to provide a storage space, in which data such as operating systems and computer programs can be stored. Memory 901 includes, but is not limited to, random access memory (random access memory, RAM), read-only memory (read-only memory, ROM), erasable programmable read-only memory (erasable programmable read only memory, EPROM), or Portable read-only memory (compact disc read-only memory, CD-ROM).

The processor 902 is a module for performing arithmetic operations and logic operations, and may be in a processing module such as a central processing unit (central processing unit, CPU), a graphics processing unit (graphics processing unit, GPU) or a microprocessor (microprocessor unit, MPU). one or a combination of more.

A computer program is stored in the memory 901, and the processor 902 calls the computer program stored in the memory 901 to execute the data labeling method shown in FIG. 4 above:

Obtain a first outline of the target object according to the first edge line;

For the specific content of the method executed by the processor 902, reference may be made to the above FIG. 4 , which will not be repeated here.

On the other hand, the processor 902 invokes the computer program stored in the memory 901 to execute the data labeling method shown in FIG. 7 above:

Obtain a first contour of the target object according to the first frame;

For the specific content of the method executed by the processor 902, reference may be made to the foregoing FIG. 7 , which will not be repeated here.

Correspondingly, the processor 902 calls the computer program stored in the memory 901, which can also be used to execute the method steps performed by the various units in the data labeling device 80 shown in FIG. I won't repeat them here.

In the electronic device 90 described in FIG. 9 , the edge detection of the target object in the target image is carried out by proposing a deep learning method based on the pre-selected frame, and the first contour of the target object is obtained, and then the weak semantic information is introduced to the second contour of the target object. The correction of the edge points of the first contour is transformed into the correction of the edge line of the first contour of the target object, and the second contour of the target object is obtained through correction, which can greatly improve the accuracy and efficiency of the labeling results.

The embodiment of the present application also provides a computer-readable storage medium. The above-mentioned computer-readable storage medium stores a computer program. When the above-mentioned computer program is run on one or more processors, the above-mentioned FIG. 4 and FIG. 7 can be implemented. method shown.

An embodiment of the present application further provides a computer program product, where the computer program product includes a computer program, and when the computer program runs on a processor, the methods shown in FIG. 4 and FIG. 7 above can be implemented.

The embodiment of the present application also provides a chip, the chip includes a processor, and the processor is configured to execute instructions, and when the processor executes the instructions, the above methods shown in FIG. 4 and FIG. 7 can be implemented.

Optionally, the chip also includes a communication interface, which is used to input data or output data.

The embodiment of the present application also provides a terminal, which includes at least one of the above-mentioned data labeling apparatus 80, or the electronic device 90, or the above-mentioned chip.

The embodiment of the present application also provides a server, which includes at least one of the above-mentioned data labeling apparatus 80, or the electronic device 90, or the above-mentioned chip.

To sum up, by proposing a deep learning method based on the pre-selected frame to detect the edge of the target object in the target image, the first contour of the target object is obtained, and then the introduction of weak semantic information will correct the edge points of the first contour of the target object It is transformed into the correction of the edge line of the first contour of the target object, and the second contour of the target object is obtained through correction, which can greatly improve the accuracy and efficiency of the labeling result.

Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be realized. The processes can be completed by hardware related to computer programs, and the computer programs can be stored in computer-readable storage media. When the computer programs are executed, , may include the processes of the foregoing method embodiments. The aforementioned storage medium includes: various media capable of storing computer program codes such as read-only memory ROM or random access memory RAM, magnetic disk or optical disk.

Claims

A data labeling method, characterized in that, comprising:

Obtaining a first edge line and a second edge line of the target object contained in the target image; the first edge line and the second edge line are different;

Obtain a first outline of the target object according to the first edge line;

Using the second edge line to correct the edge line of the first contour to obtain the second contour of the target object.
The method according to claim 1, wherein the modifying the edge line of the first outline by using the second edge line comprises:

A contour formed by the second edge line and the edge line of the first contour is determined as the second contour.
The method according to claim 1 or 2, wherein, before obtaining the first edge line and the second edge line of the target object contained in the target image, the method further comprises:

performing segmentation processing on the target image to obtain a first edge point of the target object; the segmentation processing is used to separate the target object included in the target image;

performing edge detection on the target object to obtain a second edge point of the target object; the edge detection is used to separate the edge of the target object;

The obtaining the first edge line and the second edge line of the target object contained in the target image includes:

The first edge line and the second edge line are determined according to the first edge point and the second edge point.
The method according to claim 3, wherein said determining said first edge line and said second edge line according to said first edge point and said second edge point comprises:

According to the first edge point and the second edge point, the first edge line is obtained; the point on the first edge line is an edge point where the first edge point and the second edge point coincide. ;

The second edge line is obtained according to the first edge point, the second edge point, and the first edge line.
The method according to claim 4, wherein said obtaining said second edge line according to said first edge point, said second edge point and said first edge line comprises:

According to the first edge point and the second edge point, a third edge line is obtained; the point on the third edge line is the first edge point or the second edge point, and is not the an edge point at which the first edge point coincides with the second edge point;

The third edge line having two or more intersection points with the first edge line is determined as the second edge line.
The method according to any one of claims 3 to 5, wherein the performing edge detection on the target object to obtain the second edge point of the target object comprises:

determining the label of the target object according to the position of the edge point on the edge line of the target object;

According to the label of the target object, edge points on the edge line of the target object are adjusted to obtain the second edge point.
The method according to claim 6, wherein the determining the label of the target object according to the position of the edge point on the edge line of the target object comprises:

When the number of edge points on the first label is greater than the number of edge points on the second label, determining the first label as the label of the target object;

Or, when the number of edge points on the first label is less than the number of edge points on the second label, determine the second label as the label of the target object;

Or, when the number of edge points on the first label is equal to the number of edge points on the second label, according to the key degree of the first label and the key degree of the second label , to determine the label of the target object.
The method according to any one of claims 3 to 7, wherein the segmenting the target image to obtain the first edge point of the target object comprises:

Inputting the target image into a convolutional neural network for the segmentation process to obtain the first edge point.
A data labeling device is characterized in that it comprises:

A determining unit, configured to obtain a first edge line and a second edge line of a target object included in the target image; the first edge line and the second edge line are different;

The determining unit is further configured to obtain a first contour of the target object according to the first edge line;

A correction unit, configured to use the second edge line to correct the edge line of the first contour to obtain the second contour of the target object.
The device according to claim 9, wherein the correction unit is specifically configured to determine a contour formed by the second edge line and the edge lines of the first contour as the second contour.
The device according to claim 9 or 10, wherein the device further comprises:

a segmentation unit, configured to perform segmentation processing on the target image to obtain a first edge point of the target object; the segmentation processing is used to separate the target object contained in the target image;

An edge detection unit, configured to perform edge detection on the target object to obtain a second edge point of the target object; the edge detection is used to separate the edge of the target object;

The determining unit is specifically configured to determine the first edge line and the second edge line according to the first edge point and the second edge point.
The device according to claim 11, wherein the determining unit is further configured to obtain the first edge line according to the first edge point and the second edge point; the first edge A point on the line is an edge point where the first edge point and the second edge point coincide;

The determining unit is specifically further configured to obtain the second edge line according to the first edge point, the second edge point, and the first edge line.
The device according to claim 12, wherein the determining unit is further configured to obtain a third edge line according to the first edge point and the second edge point; the third edge line The point is the first edge point or the second edge point, and is not an edge point where the first edge point and the second edge point coincide;

The determining unit is further configured to determine the third edge line having two or more intersection points with the first edge line as the second edge line.
The device according to any one of claims 11 to 13, wherein the edge detection unit is specifically configured to determine the label of the target object according to the position of the edge point on the edge line of the target object ;

The edge detection unit is further configured to adjust edge points on the edge line of the target object according to the label of the target object to obtain the second edge point.
The device according to claim 14, wherein the edge detection unit is further configured to: when the number of edge points on the first label is greater than the number of edge points on the second label, The first label is determined as the label of the target object;

Alternatively, the edge detection unit is further configured to determine the second label as the label of the target object;

Alternatively, the edge detection unit is specifically further configured to: when the number of edge points on the first label is equal to the number of edge points on the second label, according to the key of the first label degree and the key degree of the second label, and determine the label of the target object.
The device according to any one of claims 11 to 15, wherein the segmentation unit is specifically configured to input the target image into a convolutional neural network for the segmentation process to obtain the first edge point.
A data tagging device, characterized in that it includes: a processor and a memory;

The memory is used to store computer programs;

The processor is configured to execute the computer program stored in the memory, so that the data labeling device executes the method according to any one of claims 1-8.
A computer-readable storage medium, comprising:

The computer-readable storage medium is used to store a computer program; when the computer program is executed, the method according to any one of claims 1 to 8 is realized.
A computer program product, characterized by comprising: a computer program;

When the computer program is executed, the method according to any one of claims 1 to 8 is realized.
A chip, characterized in that it includes: a processor;

The processor is configured to execute instructions; when the instructions are executed, the method according to any one of claims 1 to 8 is implemented.
A terminal, characterized by comprising the data tagging device according to any one of claims 9 to 16, or the data tagging device according to claim 17, or the chip according to claim 20.
A server, characterized by comprising the data labeling device according to any one of claims 9 to 16, or the data labeling device according to claim 17, or the chip according to claim 20.