CN112308155A - Method and device for determining marking accuracy - Google Patents
Method and device for determining marking accuracy Download PDFInfo
- Publication number
- CN112308155A CN112308155A CN202011218550.2A CN202011218550A CN112308155A CN 112308155 A CN112308155 A CN 112308155A CN 202011218550 A CN202011218550 A CN 202011218550A CN 112308155 A CN112308155 A CN 112308155A
- Authority
- CN
- China
- Prior art keywords
- data
- determining
- lines
- matched
- unmatched
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 50
- 238000007689 inspection Methods 0.000 claims abstract description 97
- 238000002372 labelling Methods 0.000 claims abstract description 93
- 238000004590 computer program Methods 0.000 claims description 18
- 238000013473 artificial intelligence Methods 0.000 abstract description 7
- 230000000694 effects Effects 0.000 abstract description 5
- 238000010586 diagram Methods 0.000 description 14
- 230000008569 process Effects 0.000 description 7
- 230000006870 function Effects 0.000 description 4
- 230000009471 action Effects 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000008859 change Effects 0.000 description 2
- 239000003550 marker Substances 0.000 description 2
- 238000003908 quality control method Methods 0.000 description 2
- 230000004075 alteration Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/22—Matching criteria, e.g. proximity measures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
- G06T7/33—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
- G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30204—Marker
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Artificial Intelligence (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Life Sciences & Earth Sciences (AREA)
- General Engineering & Computer Science (AREA)
- Image Analysis (AREA)
Abstract
The embodiment of the invention provides a method and a device for determining marking accuracy, wherein the method comprises the following steps: acquiring marking data and quality inspection data aiming at a preset picture; the marking data and the quality inspection data comprise marking element data used for marking marked objects; determining pairwise matched data and unmatched data in the labeled element data of the labeled data and the quality inspection data; and determining the labeling accuracy according to the matched data and the unmatched data in the labeling element data of the labeling data and the quality inspection data. The embodiment of the invention can determine the accuracy of image marking, and further can select the image with high marking accuracy to train the artificial intelligence model, thereby improving the model identification effect.
Description
Technical Field
The invention relates to the technical field of data processing, in particular to a method and a device for determining marking accuracy.
Background
With the development of artificial intelligence, artificial intelligence models are applied to more and more business scenarios. Such as automatic driving, trajectory tracking, face recognition, etc.
The artificial intelligence model can be trained by using a manually marked picture, for example, in an automatic driving business scene, the artificial intelligence model for identifying the car position line can be trained by using the manually marked picture. The accuracy of model identification depends on the accuracy of image labeling to a great extent, but a method capable of detecting the accuracy of an image is lacked in the prior art.
Disclosure of Invention
In view of the above problems, embodiments of the present invention are proposed to provide a labeling accuracy determining method and a corresponding labeling accuracy determining apparatus that overcome or at least partially solve the above problems.
The embodiment of the invention discloses a method for determining marking accuracy, which comprises the following steps:
acquiring marking data and quality inspection data aiming at a preset picture; the marking data and the quality inspection data comprise marking element data used for marking marked objects;
determining pairwise matched data and unmatched data in the labeled element data of the labeled data and the quality inspection data;
and determining the labeling accuracy according to the matched data and the unmatched data in the labeling element data of the labeling data and the quality inspection data.
Optionally, the annotation element data comprises point object data, line object data; the point object data includes one or more points; the line object data comprises one or more lines comprising one or more points;
the step of determining pairwise matched data and unmatched data in the labeling element data of the labeling data and the quality inspection data comprises the following steps:
determining centers corresponding to all lines of the marking data and the quality inspection data;
determining pairwise matched lines and unmatched lines according to the centers corresponding to the lines;
determining pairwise matched points and unmatched points for points included by two pairwise matched lines;
the determining the labeling accuracy according to the pairwise matched data and the unmatched data in the labeling element data of the labeling data and the quality inspection data comprises the following steps:
and determining the marking accuracy according to the lines matched in pairs, the lines not matched in pairs, the points matched in pairs and the points not matched in pairs.
Optionally, the annotation element data further comprises group object data, the group object data comprising one or more groups, the groups comprising one or more lines;
the method for determining pairwise matching data and unmatched data in the labeling element data of the labeling data and the quality inspection data further comprises the following steps:
respectively determining corresponding centers according to lines included in each group of the labeling data and the quality inspection data;
determining pairwise matched groups and unmatched groups according to the centers corresponding to the groups;
determining pairwise matched lines and unmatched lines according to centers corresponding to the lines, comprising:
determining pairwise matched lines and unmatched lines according to centers corresponding to the lines for the lines included in the two pairwise matched groups;
determining the marking accuracy according to the lines matched in pairs, the lines not matched in pairs, the points matched in pairs and the points not matched in pairs, and comprising the following steps:
and determining the marking accuracy according to the pairwise matched groups, the unmatched groups, the pairwise matched lines, the unmatched lines, the pairwise matched points and the unmatched points.
Optionally, the determining, according to the center corresponding to the group, a pairwise matched group and an unmatched group includes:
determining the types of the marked objects respectively corresponding to the groups;
determining an unmatched group in the groups corresponding to the same labeled object type;
determining whether two groups with the centers closest and the distance between the centers smaller than a preset first distance threshold exist in the unmatched groups;
if so, determining two groups with the centers closest and the distance between the centers smaller than a preset first distance threshold value as pairwise matched groups;
and if not, determining the unmatched group as the unmatched group.
Optionally, the determining two matched lines and lines that are not matched according to the centers corresponding to the lines includes:
determining the types of the parts of the marked objects respectively corresponding to the lines;
determining unmatched lines among the lines corresponding to the types of the parts of the same marked object;
determining whether two lines with the centers closest to each other and the distance between the centers smaller than a preset second distance threshold exist in the unmatched lines;
if so, determining two lines with the centers closest to each other and the distance between the geometric centers smaller than a preset second distance threshold value as two matched lines;
and if not, determining the unmatched line as the unmatched line.
Optionally, the point has coordinates; the determining of the points included by the two lines matched pairwise and the points not matched pairwise comprises the following steps:
determining unmatched points of the points included by the two pairwise matched lines;
determining whether two points with the closest coordinates and the distance between the coordinates smaller than a preset third distance threshold exist in the unmatched points;
if so, determining two points with the coordinates closest to each other and the distance between the coordinates smaller than a preset third distance threshold value as points which can be matched in pairs;
and if not, determining the unmatched point as the unmatched point.
Optionally, the determining the labeling accuracy according to the pairwise matching group, the unmatched group, the pairwise matching line, the unmatched line, the pairwise matching point, and the unmatched point includes:
determining the number of preset first statistical items according to the pairwise matched groups;
determining the number of preset first difference items according to the unmatched groups;
determining the number of preset second statistical items according to the lines matched in pairs;
determining the number of preset second difference items according to the unmatched lines;
determining the number of preset third statistical items according to the point matched with each other;
determining the number of preset third difference items according to the unmatched points;
and determining the marking accuracy according to the number of the preset first statistical items, the number of the preset first difference items, the number of the preset second statistical items, the number of the preset second difference items, the number of the preset third statistical items and the number of the preset third difference items.
The embodiment of the invention also discloses a device for determining the marking accuracy, which comprises the following components:
the data acquisition module is used for acquiring marking data and quality inspection data aiming at a preset picture; the marking data and the quality inspection data comprise marking element data used for marking marked objects;
the matching module is used for determining pairwise matched data and unmatched data in the labeling element data of the labeling data and the quality inspection data;
and the marking accuracy rate determining module is used for determining the marking accuracy rate according to the data matched with each other and the data not matched in the marking element data of the marking data and the quality inspection data.
Optionally, the annotation element data comprises point object data, line object data; the point object data includes one or more points; the line object data comprises one or more lines comprising one or more points;
the matching module includes:
the line center determining submodule is used for determining centers corresponding to all lines of the marking data and the quality inspection data;
the line matching submodule is used for determining lines which are matched with each other and lines which are not matched with each other according to the corresponding centers of the lines;
the point matching submodule is used for determining points which are matched pairwise and points which are not matched for the points included by the two lines which are matched pairwise;
the labeling accuracy determining module comprises:
and the marking accuracy determining submodule is used for determining the marking accuracy according to the lines matched in pairs, the lines not matched, the points matched in pairs and the points not matched.
Optionally, the annotation element data further comprises group object data, the group object data comprising one or more groups, the groups comprising one or more lines;
the matching module further comprises:
a group center determining submodule for determining corresponding centers according to lines included in each group of the labeling data and the quality inspection data,
the group matching sub-module is used for determining pairwise matched groups and unmatched groups according to the centers corresponding to the groups;
the line matching submodule includes:
the line matching unit is used for determining lines which are matched pairwise and lines which are not matched according to centers corresponding to the lines, wherein the lines are included in the two groups which are matched pairwise;
the labeling accuracy determining submodule comprises:
and the marking accuracy rate determining unit is used for determining the marking accuracy rate according to the pairwise matching group, the unmatched group, the pairwise matching line, the unmatched line, the pairwise matching point and the unmatched point.
Optionally, the group matching sub-module includes:
the group type determining unit is used for determining the types of the marked objects respectively corresponding to the groups;
an unmatched group determination unit, configured to determine an unmatched group in a group corresponding to the same type of the tagged object;
a group distance determination unit for determining whether there are two groups, of which centers are closest and the distance between the centers is smaller than a preset first distance threshold, in the unmatched groups;
the matching group determining unit is used for determining the two groups with the centers closest to each other and the distance between the centers smaller than a preset first distance threshold as pairwise matching groups if the two groups with the centers closest to each other and the distance between the centers smaller than the preset first distance threshold exist;
and the unmatched group determining unit is used for determining the unmatched group as the unmatched group if the two groups with the centers closest to each other and the distance between the centers smaller than a preset first distance threshold value do not exist.
Optionally, the line matching submodule includes:
the line type determining unit is used for determining the types of the parts of the marked objects respectively corresponding to the lines;
an unmatched line determination unit for determining unmatched lines among the lines corresponding to the types of the parts of the same marked object;
the line distance determining unit is used for determining whether two lines with the centers closest to each other and the distance between the centers smaller than a preset second distance threshold exist in the unmatched lines;
the matching line determining unit is used for determining two lines with the centers closest to each other and the distance between the geometric centers smaller than a preset second distance threshold as two lines matched with each other if the two lines with the centers closest to each other and the distance between the centers smaller than the preset second distance threshold exist;
and the unmatched line determining unit is used for determining the unmatched line as the unmatched line if two lines with the centers closest to each other and the distance between the centers smaller than a preset second distance threshold value do not exist.
Optionally, the point has coordinates; the point matching sub-module includes:
an unmatched point determining unit for determining unmatched points for the points included by the two lines matched pairwise;
the point distance determining unit is used for determining whether two points with the closest coordinates and the distance between the coordinates smaller than a preset third distance threshold exist in the unmatched points;
the matching point determining unit is used for determining two points with the coordinates closest to each other and the distance between the coordinates smaller than a preset third distance threshold as points which can be matched pairwise if the two points have the coordinates closest to each other and the distance between the coordinates smaller than the preset third distance threshold;
and the unmatched point determining unit is used for determining the unmatched point as the unmatched point if two points with the closest coordinates and the distance between the coordinates smaller than a preset third distance threshold value do not exist.
Optionally, the annotation accuracy determining unit includes:
the first quantity determining subunit is used for determining the quantity of preset first statistical items according to the pairwise matched groups;
a second number determination subunit for determining the number of preset first difference items according to the group not matched;
the third quantity determining subunit is used for determining the quantity of preset second statistical items according to the lines matched in pairs;
a fourth number determining subunit, configured to determine, according to the unmatched line, the number of preset second difference items;
a fifth quantity determining subunit, configured to determine, according to the pairwise matching points, a quantity of preset third statistical terms;
a sixth quantity determining subunit, configured to determine, according to the unmatched point, a quantity of preset third difference items;
and the labeling accuracy determining subunit is configured to determine the labeling accuracy according to the number of the preset first statistical items, the number of the preset first difference items, the number of the preset second statistical items, the number of the preset second difference items, the number of the preset third statistical items, and the number of the preset third difference items.
The embodiment of the invention also discloses an electronic device, which comprises: a processor, a memory and a computer program stored on the memory and being executable on the processor, the computer program, when executed by the processor, implementing the steps of the annotation accuracy determination method as described above.
The embodiment of the invention also discloses a computer readable storage medium, wherein a computer program is stored on the computer readable storage medium, and when the computer program is executed by a processor, the steps of the method for determining the annotation accuracy are realized.
The embodiment of the invention has the following advantages:
in the embodiment of the invention, the marking data and the quality inspection data aiming at the preset picture can be acquired; determining pairwise matched data and unmatched data in the labeled element data of the labeled data and the quality inspection data; and determining the labeling accuracy according to pairwise matched data and unmatched data in the labeling element data of the labeling data and the quality inspection data.
Drawings
FIG. 1 is a diagram of a annotator in an embodiment of the invention;
FIG. 2 is a diagram illustrating a modification of annotations by a quality inspector according to an embodiment of the present invention;
FIG. 3 is a flowchart illustrating the steps of a method for determining annotation accuracy according to an embodiment of the present invention;
FIG. 4 is a flowchart illustrating steps of another annotation accuracy determination method according to an embodiment of the present invention;
FIG. 5 is a flowchart illustrating steps of another annotation accuracy determination method according to an embodiment of the present invention;
FIG. 6 is a schematic diagram of annotation data and quality control data in accordance with an embodiment of the present invention;
fig. 7 is a block diagram of a device for determining annotation accuracy according to an embodiment of the present invention.
Detailed Description
In order to make the aforementioned objects, features and advantages of the present invention comprehensible, embodiments accompanied with figures are described in further detail below.
In an automatic driving business scene, marked objects such as parking spaces, lanes, drivable areas and the like need to be marked on a picture manually.
In the embodiment of the invention, a marker can use a marking tool to mark the marked object on the picture to obtain marking data. The annotation data may include point object data, line object data, group object data.
The group object data may include one or more groups, a group being data arranged to describe lines for annotating the same annotated object, each group may include one or more lines for annotating the same annotated object in the picture. For example, in an autonomous driving business scenario, the group may include: a group for a parking space, a group for a travelable area, a group for a lane, etc.
The line object data may include one or more lines. For example, in the group for the parking space may include: a line for marking a vehicle location line, a line for marking a stop lever, a line for marking a vehicle location number, a line for marking a ground lock, and the like.
In the marking tool, different types of lines can be selected for marking, and the types of the lines can include: line segment, two-dimensional frame, polygon.
When drawing a line, the currently drawable graphics may be changed according to the setting of the line type. If the type of the currently selected line is a line segment, the drawn graph is the line segment. When drawing a line segment, a user can draw a plurality of points, and the marking tool can draw a line segment containing the points according to the drawn points.
And if the currently selected line type is a two-dimensional frame, the drawn graph is a two-dimensional frame. When the two-dimensional frame is drawn, a user can draw two points, the marking tool can determine a connecting line of the two drawn points, and the connecting line is used as a diagonal line of the two-dimensional frame, so that the two-dimensional frame is drawn.
If the currently selected line type is a polygon, the drawn graph is a polygon. When drawing a multi-edge line, a user can draw a plurality of points, and the marking tool can draw the multi-edge line containing the points according to the drawn points. Adding a point each time is adding a point to the polygon.
When the line type is a line segment and a polygon, if the number of points (pointCount) is set to N, the marking tool may automatically end drawing when drawing N points. If the number of the set groups or the number of a certain line type in a single group exists in the configuration, the annotation tool will make a corresponding prompt according to the configuration when the condition is not met, for example, the prompt indicates that the number of the certain line type in the current group cannot be more than 1.
The point object data may comprise one or more points and the annotation tool may draw a line containing the points in accordance with the points drawn by the user.
And after the annotator marks the picture, the picture is delivered to a quality inspector for quality inspection. The quality inspector can modify the label or add a new label or not change the label data, and the label data after quality inspection by the quality inspector can be used as the quality inspection data. The accuracy of the labeling data can be determined according to the labeling data generated by labeling of the labeling personnel and the quality inspection data after quality inspection of the quality inspection personnel.
Fig. 1 shows a picture labeled by a label maker in an embodiment of the present invention. The quality inspector can draw a two-dimensional box containing the outline of the vehicle for one vehicle in the picture and set the reference numeral 1.
As shown in fig. 2, which is a picture of a quality inspector modifying a label in an embodiment of the present invention, the quality inspector can draw two-dimensional frames containing vehicle outlines for two vehicles in the picture, and set reference numerals 1 and 2, respectively.
Referring to fig. 3, a flowchart illustrating steps of a method for determining an annotation accuracy according to an embodiment of the present invention is shown, where the method specifically includes the following steps:
The preset picture may include a plurality of labeled pictures. The picture can have corresponding label data and quality inspection data, wherein the label data is data labeled by a label, and the quality inspection data is data after quality inspection of a quality inspector. The quality inspector can modify the label or add a new label or not change the label data, and the label data after quality inspection by the quality inspector can be used as the quality inspection data.
Annotation element data can include annotation elements divided by hierarchy. E.g., point object data, line object data, group data object.
Step 302, determining pairwise matched data and unmatched data in the labeling element data of the labeling data and the quality inspection data.
Under the condition that the labeling data are accurate, the labeling element data corresponding to the labeling element data of the labeling data can be found in the quality inspection data. If the annotation data is not accurate enough, and a quality inspector may add a new annotation, the quality inspection data may have additional annotation element data relative to the annotation data. The quality inspector may delete some annotations, and the quality inspection data may have less annotated element data than the annotation data. If the quality inspector can modify some of the annotations, the annotation data may not match the annotation element data of the quality inspection data.
In the embodiment of the invention, the marking data and the quality inspection data aiming at the preset picture can be acquired; determining pairwise matched data and unmatched data in the labeled element data of the labeled data and the quality inspection data; and determining the labeling accuracy according to pairwise matched data and unmatched data in the labeling element data of the labeling data and the quality inspection data.
Referring to fig. 4, a flowchart illustrating steps of another method for determining labeling accuracy according to an embodiment of the present invention is shown, where the method specifically includes the following steps:
The types of lines may include line segments, binary boxes, polygons. If the type of the line segment is a line segment, the center of the line segment may be determined as the center corresponding to the line. If the type of line is a two-bit box, polygon, a polygon formed by the points of the line may be determined; and determining the geometric center of the polygon as the center corresponding to the line.
And step 403, determining pairwise matched lines and unmatched lines according to the corresponding centers of the lines.
Among the plurality of lines, two lines having the closest distance between the centers corresponding to the lines may be determined as lines matched two by two.
A line that is not matched is a line that cannot be matched to other lines. If only one line remains that does not find a matching line, this line can be determined to be a line that is not matched.
Under the condition that the marked data are accurate, lines corresponding to all the lines of the marked data can be found in the quality inspection data. If the annotation data is not accurate enough, the quality inspector may add new annotations, and the quality inspection data may have additional lines relative to the annotation data. The inspector may delete some of the annotations and the quality inspection data may have fewer lines than the annotated data. The quality inspector may also modify some of the annotations and there is a possibility that the lines of annotated data and quality inspection data do not match.
In an embodiment of the present invention, the step of determining pairwise matching lines and unmatched lines according to the corresponding centers of the lines may include the sub-steps of:
and a substep S11 of determining the types of the parts of the marked object respectively corresponding to the lines.
When the marking tool marks, the marker can set one or more attributes for the line object data. Wherein the attribute may set an attribute for indicating a type of the part of the annotated object. For example, the type of the part where the corresponding marked object is provided for one line is a stopper rod.
In sub-step S12, among the lines corresponding to the types of the parts of the same labeled object, an unmatched line is determined.
Unmatched lines refer to lines that have not been matched. For example, the type of the tagged object is a parking space, the type of the part of the tagged object is a parking space line, the lines corresponding to the parking space line in the type have 3 lines in total, and all the 3 lines are unmatched lines.
And a substep S13 of determining whether there are two lines having the centers closest to each other and the distance between the centers smaller than a preset second distance threshold, among the unmatched lines.
In the sub-step S14, if yes, two lines whose centers are closest and whose distance between the geometric centers is smaller than a preset second distance threshold are determined as two matched lines.
If two lines with the centers closest to each other and the distance between the centers smaller than the preset second distance threshold are not matched, the two lines can be considered as lines matched with each other, one line is a line in the label data, and the other line is a line matched with the quality inspection data.
And a substep S15, if not, determining the unmatched line as the unmatched line.
If the number of unmatched lines is greater than one and the distance between the centers corresponding to the lines is not less than the preset second distance threshold, the unmatched lines may be considered as unmatched lines.
If only one unmatched line remains, the unmatched line may be considered as a non-matched group.
And performing data statistics on the two groups found each time, circulating all line types, finding two lines with the same type and the nearest center, and continuously finding out the lines with the same type and the nearest center from the remaining similar lines until no similar lines remain in a certain group.
In the lines corresponding to the types of the parts of the same marked object, the lines matched in pairs are determined, so that the situation that the distance of corresponding centers is the closest to each other possibly can be found between the lines corresponding to the parts of different marked objects, and the accuracy of group matching can be improved.
In step 404, for the points included in the two lines matched in pairs, the points matched in pairs and the points not matched in pairs are determined.
Among a plurality of points included in two lines matched two by two, two points whose distances between the points are the closest may be determined as the points matched two by two.
Points that are not matched are points that cannot be matched with other points. If only one point remains that does not find a matching point, this point can be determined to be a point that is not matched.
Under the condition that the labeled data is accurate, points corresponding to all the labeled data points can be found in the quality inspection data. If the annotation data is not accurate enough, and a quality inspector may add a new annotation, there may be additional points in the quality inspection data relative to the annotation data. The quality inspector may also delete some annotations and the quality inspection data may be a few points less than the annotation data. The quality inspector may also modify some of the annotations, and the points of the annotated data and the quality inspection data may not match.
In an embodiment of the invention, the points have coordinates; for the points included in the two lines matched pairwise, the step of determining pairwise matched points and unmatched points may include the sub-steps of:
in sub-step S21, points included in two lines that match each other are determined as points that are not matched.
Unmatched points refer to points that have not been matched. For example, two lines that match each other include a total of 5 points, and the 5 points are unmatched points.
And a substep S22 of determining whether there are two points having the closest coordinates and the distance between the coordinates being less than a preset third distance threshold, among the unmatched points.
And step S23, if yes, determining two points with the coordinates closest to each other and the distance between the coordinates smaller than a preset third distance threshold as points which can be matched pairwise.
If there are two points with centers closest to each other and the distance between the centers is smaller than the preset third distance threshold value, the two points are considered as pairwise matching points, one point is a point in the labeling data, and the other point is a matching point in the quality inspection data.
And a substep S24, if not, determining the unmatched point as the unmatched point.
If the number of unmatched points is greater than one, and the distance between the centers of the point correspondences is not less than a preset third distance threshold, the unmatched points may be considered as unmatched lines.
If there is only one unmatched point left, the unmatched point can be considered as a non-matched group.
And for each two lines found, circulating all the points of the two lines, finding the two points with the nearest coordinates and carrying out statistics, then finding the two points with the nearest coordinates from the rest points and carrying out statistics, and repeatedly finding the points with the nearest coordinates until the rest points do not exist in a certain line.
The marking accuracy can be determined by the number of lines matched pairwise, the number of lines not matched pairwise, the number of points matched pairwise and the number of points not matched.
In the embodiment of the invention, the marking data and the quality inspection data aiming at the preset picture can be acquired; determining centers corresponding to all lines of the marking data and the quality inspection data; determining pairwise matched lines and unmatched lines according to centers corresponding to the lines; determining pairwise matched points and unmatched points for points included by two pairwise matched lines; and determining the marking accuracy according to the lines matched in pairs, the lines not matched in pairs, the points matched in pairs and the points not matched in pairs.
Referring to fig. 5, a flowchart illustrating steps of another method for determining labeling accuracy according to an embodiment of the present invention is shown, where the method specifically includes the following steps:
Each group may include one or more lines for labeling the same labeled object in the picture. A polygon formed by the points contained by the lines within the group may be determined; and determining the geometric center of the polygon as the center corresponding to each line in the group.
And step 504, determining pairwise matched groups and unmatched groups according to the centers corresponding to the groups.
Among the plurality of groups, two groups whose distances between the corresponding centers of the groups are closest may be determined as pairwise matched groups.
A group that is not matched is a group that cannot be matched with other groups. If only one group remains that does not find a matching group, this group may be determined to be a group that is not matched.
Under the condition that the labeled data is accurate, the groups corresponding to the groups of the labeled data can be found in the quality inspection data. If the annotation data is not accurate enough and the quality inspector may add new annotations, there may be additional sets of quality inspection data relative to the annotation data. The inspector may delete some annotations and the quality inspection data may have fewer sets than the annotation data. The quality inspector may also modify some of the annotations and there is a possibility that the annotated data does not match the set of quality inspection data.
In this embodiment of the present invention, the step of determining pairwise matching groups and unmatched groups according to the centers corresponding to the groups may include the following sub-steps:
and a sub-step S31 of determining the types of the labeled objects respectively corresponding to the groups.
In an automatic driving business scenario, the type of the tagged object may include a parking space, a lane, a drivable area, and the like.
The annotator can set one or more attributes for the group object data when the annotation tool annotates. Wherein the attribute can set an attribute for representing the type of the annotated object. For example, the type of the marked object corresponding to one group is set as a parking space.
Sub-step S32, among the groups corresponding to the type of the same labeled object, determines the group that is not matched.
An unmatched group refers to a group that has not been matched. For example, the type of the tagged object is a parking space, and the number of the groups with the corresponding types of the parking spaces is 5, wherein 2 groups are already determined as pairwise matching groups, and the remaining 3 groups are unmatched groups.
And a substep S33 of determining, among the unmatched groups, whether there are two groups whose centers are closest and whose distance between the centers is less than a preset first distance threshold.
In the sub-step S34, if yes, two groups with the centers closest to each other and the distance between the centers smaller than the preset first distance threshold are determined as pairwise matching groups.
If two groups with the centers closest and the distance between the centers smaller than the preset first distance threshold value are not matched, the two groups can be considered as pairwise matched groups, one group is a group in the annotation data, and the other group is a matched group in the quality inspection data.
And a substep S35, if not, determining the unmatched group as the unmatched group.
If the number of unmatched groups is greater than one and the distance between the centers of the corresponding groups is not less than the preset first distance threshold, the unmatched groups may be considered as unmatched groups.
If there is only one unmatched group left, the unmatched group may be considered as an unmatched group.
And circulating each group type, finding two groups with the nearest centers, continuously finding two groups with the nearest centers from the rest groups with the same type, and repeatedly searching the groups with the nearest centers in the rest groups until no groups with the same type remain in the labeled data or the quality inspection data.
In the groups corresponding to the same type of the labeled object, the groups matched pairwise are determined, so that the situation that the distance of corresponding centers is the closest to each other between the groups corresponding to different labeled objects can be avoided, and the accuracy of group matching can be improved.
And 505, determining pairwise matched lines and unmatched lines according to centers corresponding to the lines for the lines included in the two pairwise matched groups.
Among the plurality of lines included in the two groups matched two by two, the two lines whose distance between the centers corresponding to the lines is the closest may be determined as the lines matched two by two.
Referring to fig. 6, a schematic diagram of comparing the label data with the quality inspection data according to an embodiment of the present invention is shown.
The label data may include 5 label boxes, and the label boxes may be formed by lines of polygons, and the labels are 11, 12, 13, 14, and 15, respectively. The quality control data may include 5 labeled boxes, which may be formed by lines of polygons, numbered 21, 22, 23, 24, 25, respectively. And comparing to obtain that the quality inspector modifies the marking frame on the basis of the marking data.
The distance between the geometric centers of the labeling frame of the label 11 in the labeling data and the labeling frame of the label 21 in the quality inspection data is closest and is smaller than the corresponding threshold value, so that the two labeling frames are matched pairwise.
The distance between the geometric centers of the labeling box labeled with the reference number 15 in the labeling data and the labeling box labeled with the reference number 25 in the quality inspection data is larger than the corresponding threshold value, so that the two labeling boxes cannot be matched.
In step 506, for the points included in the two lines matched in pairs, the points matched in pairs and the points not matched in pairs are determined.
And 507, determining the marking accuracy according to the pairwise matching groups, the unmatched groups, the pairwise matching lines, the unmatched lines, the pairwise matching points and the unmatched points.
In an embodiment of the present invention, the step of determining the labeling accuracy according to the pairwise matching group, the unmatched group, the pairwise matching line, the unmatched line, the pairwise matching point, and the unmatched point may include the following sub-steps:
and a substep S41 of determining the number of preset first statistical terms according to the pairwise matching groups.
The preset first statistic item can be the attribute of the group matched two by two, different types of groups can have different attributes, and the attribute of the group can be set by a annotator. For example, the type of the group is a lane, and the corresponding attribute may include a main road, a branch road, a reverse direction, a travelable, and the like.
And a substep S42 of determining the number of preset first difference items according to the unmatched groups.
The preset first difference item may be an attribute of a group for which statistics are not matched.
And a substep S43 of determining the number of preset second statistical terms according to the pairwise matched lines.
The preset second statistical item may be to count the attributes of two lines matched with each other, different types of lines may have different attributes, and the attributes of the lines may be set by a annotator.
For example, the type of line is a carport line, and the corresponding attributes may include available parking and unavailable parking.
And a substep S44 of determining the number of preset second difference items according to the unmatched line.
The second difference term may be an attribute of a line that is not statistically matched.
And a substep S45, determining the number of preset third statistical items according to the point matched in pairs.
The preset third statistical item may be an attribute of a point which is matched two by two, the attribute of the point may be set by a annotator, and the attribute of the point may include visible, shielded, fuzzy, and the like.
And a substep S46 of determining the number of preset third difference items according to the unmatched point.
The preset third difference item may be an attribute of a point for which statistics are not matched.
And a substep S47, determining the labeling accuracy according to the number of the preset first statistical items, the number of the preset first difference items, the number of the preset second statistical items, the number of the preset second difference items, the number of the preset third statistical items and the number of the preset third difference items.
For example, the accuracy can be calculated according to the following formula:
accuracy 1-total number of error terms/total number of statistical terms
The total number of the error items can be the sum of a preset first difference item, a preset second difference item and a preset third difference item; the total number of the statistical items can be the sum of a preset first statistical item, a preset second statistical item and a preset third statistical item.
Of course, those skilled in the art may also use other formulas to calculate the labeling accuracy, which is not limited in the embodiment of the present invention.
In the embodiment of the invention, the marking data and the quality inspection data aiming at the preset picture can be acquired; respectively determining corresponding centers according to lines included in each group of the labeling data and the quality inspection data, and determining the centers corresponding to the lines of the labeling data and the quality inspection data;
determining pairwise matched groups and unmatched groups according to the centers corresponding to the groups; determining pairwise matched lines and unmatched lines according to centers corresponding to the lines for the lines included in the two pairwise matched groups; determining pairwise matched points and unmatched points for points included by two pairwise matched lines; and determining the marking accuracy according to the pairwise matched groups, the unmatched groups, the pairwise matched lines, the unmatched lines, the pairwise matched points and the unmatched points. The embodiment of the invention can determine the accuracy of image marking, and further can select the image with high marking accuracy to train the artificial intelligence model so as to improve the model identification effect.
It should be noted that, for simplicity of description, the method embodiments are described as a series of acts or combination of acts, but those skilled in the art will recognize that the present invention is not limited by the illustrated order of acts, as some steps may occur in other orders or concurrently in accordance with the embodiments of the present invention. Further, those skilled in the art will appreciate that the embodiments described in the specification are presently preferred and that no particular act is required to implement the invention.
Referring to fig. 7, a block diagram of a structure of a labeling accuracy determining apparatus according to an embodiment of the present invention is shown, which may specifically include the following modules:
a data obtaining module 701, configured to obtain labeling data and quality inspection data for a preset picture; the marking data and the quality inspection data comprise marking element data used for marking marked objects;
a matching module 702, configured to determine pairwise matching data and unmatched data in the labeled element data of the labeled data and the quality inspection data;
and an annotation accuracy determining module 703, configured to determine an annotation accuracy according to the pairwise matched data and the unmatched data in the annotation element data of the annotation data and the quality inspection data.
In an embodiment of the present invention, the annotation element data includes point object data and line object data; the point object data includes one or more points; the line object data comprises one or more lines comprising one or more points;
the matching module 702 may include:
the line center determining submodule is used for determining centers corresponding to all lines of the marking data and the quality inspection data;
the line matching submodule is used for determining lines which are matched with each other and lines which are not matched with each other according to the corresponding centers of the lines;
the point matching submodule is used for determining points which are matched pairwise and points which are not matched for the points included by the two lines which are matched pairwise;
the annotation accuracy determination module 703 may include:
and the marking accuracy determining submodule is used for determining the marking accuracy according to the lines matched in pairs, the lines not matched, the points matched in pairs and the points not matched.
In an embodiment of the present invention, the annotation element data further includes group object data, the group object data includes one or more groups, and the group includes one or more lines;
the matching module 702 may further include:
a group center determining submodule for determining corresponding centers according to lines included in each group of the labeling data and the quality inspection data,
the group matching sub-module is used for determining pairwise matched groups and unmatched groups according to the centers corresponding to the groups;
the line matching submodule includes:
the line matching unit is used for determining lines which are matched pairwise and lines which are not matched according to centers corresponding to the lines, wherein the lines are included in the two groups which are matched pairwise;
the labeling accuracy determining submodule comprises:
and the marking accuracy rate determining unit is used for determining the marking accuracy rate according to the pairwise matching group, the unmatched group, the pairwise matching line, the unmatched line, the pairwise matching point and the unmatched point.
In an embodiment of the present invention, the set of matching sub-modules may include:
the group type determining unit is used for determining the types of the marked objects respectively corresponding to the groups;
an unmatched group determination unit, configured to determine an unmatched group in a group corresponding to the same type of the tagged object;
a group distance determination unit for determining whether there are two groups, of which centers are closest and the distance between the centers is smaller than a preset first distance threshold, in the unmatched groups;
the matching group determining unit is used for determining the two groups with the centers closest to each other and the distance between the centers smaller than a preset first distance threshold as pairwise matching groups if the two groups with the centers closest to each other and the distance between the centers smaller than the preset first distance threshold exist;
and the unmatched group determining unit is used for determining the unmatched group as the unmatched group if the two groups with the centers closest to each other and the distance between the centers smaller than a preset first distance threshold value do not exist.
In an embodiment of the present invention, the line matching sub-module may include:
the line type determining unit is used for determining the types of the parts of the marked objects respectively corresponding to the lines;
an unmatched line determination unit for determining unmatched lines among the lines corresponding to the types of the parts of the same marked object;
the line distance determining unit is used for determining whether two lines with the centers closest to each other and the distance between the centers smaller than a preset second distance threshold exist in the unmatched lines;
the matching line determining unit is used for determining two lines with the centers closest to each other and the distance between the geometric centers smaller than a preset second distance threshold as two lines matched with each other if the two lines with the centers closest to each other and the distance between the centers smaller than the preset second distance threshold exist;
and the unmatched line determining unit is used for determining the unmatched line as the unmatched line if two lines with the centers closest to each other and the distance between the centers smaller than a preset second distance threshold value do not exist.
In an embodiment of the invention, the points have coordinates; the point matching sub-module may include:
an unmatched point determining unit for determining unmatched points for the points included by the two lines matched pairwise;
the point distance determining unit is used for determining whether two points with the closest coordinates and the distance between the coordinates smaller than a preset third distance threshold exist in the unmatched points;
the matching point determining unit is used for determining two points with the coordinates closest to each other and the distance between the coordinates smaller than a preset third distance threshold as points which can be matched pairwise if the two points have the coordinates closest to each other and the distance between the coordinates smaller than the preset third distance threshold;
and the unmatched point determining unit is used for determining the unmatched point as the unmatched point if two points with the closest coordinates and the distance between the coordinates smaller than a preset third distance threshold value do not exist.
In an embodiment of the present invention, the labeling accuracy determining unit may include:
the first quantity determining subunit is used for determining the quantity of preset first statistical items according to the pairwise matched groups;
a second number determination subunit for determining the number of preset first difference items according to the group not matched;
the third quantity determining subunit is used for determining the quantity of preset second statistical items according to the lines matched in pairs;
a fourth number determining subunit, configured to determine, according to the unmatched line, the number of preset second difference items;
a fifth quantity determining subunit, configured to determine, according to the pairwise matching points, a quantity of preset third statistical terms;
a sixth quantity determining subunit, configured to determine, according to the unmatched point, a quantity of preset third difference items;
and the labeling accuracy determining subunit is configured to determine the labeling accuracy according to the number of the preset first statistical items, the number of the preset first difference items, the number of the preset second statistical items, the number of the preset second difference items, the number of the preset third statistical items, and the number of the preset third difference items.
In the embodiment of the invention, the marking data and the quality inspection data aiming at the preset picture can be acquired; respectively determining corresponding centers according to lines included in each group of the labeling data and the quality inspection data, and determining the centers corresponding to the lines of the labeling data and the quality inspection data; determining pairwise matched groups and unmatched groups according to the centers corresponding to the groups; determining pairwise matched lines and unmatched lines according to centers corresponding to the lines for the lines included in the two pairwise matched groups; determining pairwise matched points and unmatched points for points included by two pairwise matched lines; and determining the marking accuracy according to the pairwise matched groups, the unmatched groups, the pairwise matched lines, the unmatched lines, the pairwise matched points and the unmatched points. According to the embodiment of the invention, matching can be carried out from three levels of points, lines and groups, the accuracy rate of image marking is determined according to the matching result, and then the image with high marking accuracy can be selected to train the artificial intelligence model so as to improve the model identification effect.
For the device embodiment, since it is basically similar to the method embodiment, the description is simple, and for the relevant points, refer to the partial description of the method embodiment.
An embodiment of the present invention further provides an electronic device, including:
the method comprises a processor, a memory and a computer program which is stored on the memory and can run on the processor, wherein when the computer program is executed by the processor, each process of the embodiment of the method for determining the marking accuracy is realized, the same technical effect can be achieved, and the process is not repeated to avoid repetition.
The embodiment of the present invention further provides a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when being executed by a processor, the computer program implements each process of the above-mentioned method for determining an annotation accuracy, and can achieve the same technical effect, and in order to avoid repetition, the detailed description is omitted here.
The embodiments in the present specification are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other.
As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, apparatus, or computer program product. Accordingly, embodiments of the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, embodiments of the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
Embodiments of the present invention are described with reference to flowchart illustrations and/or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing terminal to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing terminal to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing terminal to cause a series of operational steps to be performed on the computer or other programmable terminal to produce a computer implemented process such that the instructions which execute on the computer or other programmable terminal provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
While preferred embodiments of the present invention have been described, additional variations and modifications of these embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. Therefore, it is intended that the appended claims be interpreted as including preferred embodiments and all such alterations and modifications as fall within the scope of the embodiments of the invention.
Finally, it should also be noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or terminal that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or terminal. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or terminal that comprises the element.
The method for determining the labeling accuracy and the device for determining the labeling accuracy provided by the invention are described in detail, a specific example is applied in the text to explain the principle and the implementation mode of the invention, and the description of the embodiment is only used for helping to understand the method and the core idea of the invention; meanwhile, for a person skilled in the art, according to the idea of the present invention, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present specification should not be construed as a limitation to the present invention.
Claims (10)
1. A method for determining annotation accuracy rate is characterized by comprising the following steps:
acquiring marking data and quality inspection data aiming at a preset picture; the marking data and the quality inspection data comprise marking element data used for marking marked objects;
determining pairwise matched data and unmatched data in the labeled element data of the labeled data and the quality inspection data;
and determining the labeling accuracy according to the matched data and the unmatched data in the labeling element data of the labeling data and the quality inspection data.
2. The method of claim 1, wherein the annotation element data comprises point object data, line object data; the point object data includes one or more points; the line object data comprises one or more lines comprising one or more points;
the step of determining pairwise matched data and unmatched data in the labeling element data of the labeling data and the quality inspection data comprises the following steps:
determining centers corresponding to all lines of the marking data and the quality inspection data;
determining pairwise matched lines and unmatched lines according to the centers corresponding to the lines;
determining pairwise matched points and unmatched points for points included by two pairwise matched lines;
the determining the labeling accuracy according to the pairwise matched data and the unmatched data in the labeling element data of the labeling data and the quality inspection data comprises the following steps:
and determining the marking accuracy according to the lines matched in pairs, the lines not matched in pairs, the points matched in pairs and the points not matched in pairs.
3. The method of claim 2, wherein the annotation element data further comprises group object data, the group object data comprising one or more groups, the groups comprising one or more lines; the method for determining pairwise matching data and unmatched data in the labeling element data of the labeling data and the quality inspection data further comprises the following steps:
respectively determining corresponding centers according to lines included in each group of the labeling data and the quality inspection data;
determining pairwise matched groups and unmatched groups according to the centers corresponding to the groups;
determining pairwise matched lines and unmatched lines according to centers corresponding to the lines, comprising:
determining pairwise matched lines and unmatched lines according to centers corresponding to the lines for the lines included in the two pairwise matched groups;
determining the marking accuracy according to the lines matched in pairs, the lines not matched in pairs, the points matched in pairs and the points not matched in pairs, and comprising the following steps:
and determining the marking accuracy according to the pairwise matched groups, the unmatched groups, the pairwise matched lines, the unmatched lines, the pairwise matched points and the unmatched points.
4. The method of claim 3, wherein determining pairwise matched and unmatched groups based on the centers to which the groups correspond comprises:
determining the types of the marked objects respectively corresponding to the groups;
determining an unmatched group in the groups corresponding to the same labeled object type;
determining whether two groups with the centers closest and the distance between the centers smaller than a preset first distance threshold exist in the unmatched groups;
if so, determining two groups with the centers closest and the distance between the centers smaller than a preset first distance threshold value as pairwise matched groups;
and if not, determining the unmatched group as the unmatched group.
5. The method of claim 2, wherein determining pairwise matched lines and unmatched lines based on centers corresponding to the lines comprises:
determining the types of the parts of the marked objects respectively corresponding to the lines;
determining unmatched lines among the lines corresponding to the types of the parts of the same marked object;
determining whether two lines with the centers closest to each other and the distance between the centers smaller than a preset second distance threshold exist in the unmatched lines;
if so, determining two lines with the centers closest to each other and the distance between the geometric centers smaller than a preset second distance threshold value as two matched lines;
and if not, determining the unmatched line as the unmatched line.
6. The method of claim 2, wherein the points have coordinates; the determining of the points included by the two lines matched pairwise and the points not matched pairwise comprises the following steps:
determining unmatched points of the points included by the two pairwise matched lines;
determining whether two points with the closest coordinates and the distance between the coordinates smaller than a preset third distance threshold exist in the unmatched points;
if so, determining two points with the coordinates closest to each other and the distance between the coordinates smaller than a preset third distance threshold value as points which can be matched in pairs;
and if not, determining the unmatched point as the unmatched point.
7. The method of claim 3, wherein said determining labeling accuracy from said pairwise matched groups, said unmatched groups, said pairwise matched lines, said unmatched lines, said pairwise matched points, and said unmatched points comprises:
determining the number of preset first statistical items according to the pairwise matched groups;
determining the number of preset first difference items according to the unmatched groups;
determining the number of preset second statistical items according to the lines matched in pairs;
determining the number of preset second difference items according to the unmatched lines;
determining the number of preset third statistical items according to the point matched with each other;
determining the number of preset third difference items according to the unmatched points;
and determining the marking accuracy according to the number of the preset first statistical items, the number of the preset first difference items, the number of the preset second statistical items, the number of the preset second difference items, the number of the preset third statistical items and the number of the preset third difference items.
8. An annotation accuracy determination apparatus, comprising:
the data acquisition module is used for acquiring marking data and quality inspection data aiming at a preset picture; the marking data and the quality inspection data comprise marking element data used for marking marked objects;
the matching module is used for determining pairwise matched data and unmatched data in the labeling element data of the labeling data and the quality inspection data;
and the marking accuracy rate determining module is used for determining the marking accuracy rate according to the data matched with each other and the data not matched in the marking element data of the marking data and the quality inspection data.
9. An electronic device, comprising: a processor, a memory and a computer program stored on the memory and being executable on the processor, the computer program, when executed by the processor, implementing the steps of the annotation accuracy determination method according to any one of claims 1-7.
10. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the steps of the annotation accuracy determination method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011218550.2A CN112308155A (en) | 2020-11-04 | 2020-11-04 | Method and device for determining marking accuracy |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011218550.2A CN112308155A (en) | 2020-11-04 | 2020-11-04 | Method and device for determining marking accuracy |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112308155A true CN112308155A (en) | 2021-02-02 |
Family
ID=74326068
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011218550.2A Pending CN112308155A (en) | 2020-11-04 | 2020-11-04 | Method and device for determining marking accuracy |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112308155A (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110796185A (en) * | 2019-10-17 | 2020-02-14 | 北京爱数智慧科技有限公司 | Method and device for detecting image annotation result |
| CN111310667A (en) * | 2020-02-18 | 2020-06-19 | 北京小马慧行科技有限公司 | Method, device, storage medium and processor for determining whether annotation is accurate |
| CN111368927A (en) * | 2020-03-06 | 2020-07-03 | 广州文远知行科技有限公司 | Method, device and equipment for processing labeling result and storage medium |
| CN111860304A (en) * | 2020-07-17 | 2020-10-30 | 北京百度网讯科技有限公司 | Image labeling method, electronic device, equipment and storage medium |
-
2020
- 2020-11-04 CN CN202011218550.2A patent/CN112308155A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110796185A (en) * | 2019-10-17 | 2020-02-14 | 北京爱数智慧科技有限公司 | Method and device for detecting image annotation result |
| CN111310667A (en) * | 2020-02-18 | 2020-06-19 | 北京小马慧行科技有限公司 | Method, device, storage medium and processor for determining whether annotation is accurate |
| CN111368927A (en) * | 2020-03-06 | 2020-07-03 | 广州文远知行科技有限公司 | Method, device and equipment for processing labeling result and storage medium |
| CN111860304A (en) * | 2020-07-17 | 2020-10-30 | 北京百度网讯科技有限公司 | Image labeling method, electronic device, equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109389275B (en) | Image annotation method and device | |
| CN111695486B (en) | High-precision direction signboard target extraction method based on point cloud | |
| CN110879960B (en) | Method and computing device for generating image data set for convolutional neural network learning | |
| CN111738036B (en) | Image processing method, device, equipment and storage medium | |
| CN113095267B (en) | Data extraction method of statistical chart, electronic device and storage medium | |
| CN111652144A (en) | Topic segmentation method, device, equipment and medium based on target region fusion | |
| CN115687643A (en) | Method for training multi-mode information extraction model and information extraction method | |
| CN107886105A (en) | A kind of annotation equipment of image | |
| CN105677878A (en) | Method and system for vehicle information multi-dimensional display based on BI system | |
| CN112434585A (en) | Method, system, electronic device and storage medium for identifying virtual reality of lane line | |
| CN112308155A (en) | Method and device for determining marking accuracy | |
| CN112200218A (en) | Model training method and device and electronic equipment | |
| CN111414903A (en) | Method, device and equipment for identifying content of indicator | |
| CN113807315B (en) | Method, device, equipment and medium for constructing object recognition model to be recognized | |
| CN113159193B (en) | Model training method, image recognition method, storage medium, and program product | |
| Kavati | Deep learning-based pothole detection for intelligent transportation systems | |
| KR102599196B1 (en) | Method for estimate job cost of work on generating training data, and computer program recorded on record-medium for executing method thereof | |
| KR102546193B1 (en) | Method for learning data classification using color information, and computer program recorded on record-medium for executing method thereof | |
| CN112926371B (en) | Road survey method and system | |
| CN112767512B (en) | Method and device for generating environment linear element, electronic equipment and storage medium | |
| CN115641430B (en) | Method, device, medium and computer equipment for determining interest surface | |
| KR102546198B1 (en) | Method for learning data classification based physical factor, and computer program recorded on record-medium for executing method thereof | |
| KR102531917B1 (en) | Method for annotation using boundary transplant, and computer program recorded on record-medium for executing method thereof | |
| KR102658711B1 (en) | Method for annotation using boundary designation | |
| CN116168201B (en) | Lane line segmentation method and device without accurate data labeling |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |