CN116935134A - Point cloud data labeling method, point cloud data labeling system, terminal and storage medium - Google Patents
Point cloud data labeling method, point cloud data labeling system, terminal and storage medium Download PDFInfo
- Publication number
- CN116935134A CN116935134A CN202310951444.2A CN202310951444A CN116935134A CN 116935134 A CN116935134 A CN 116935134A CN 202310951444 A CN202310951444 A CN 202310951444A CN 116935134 A CN116935134 A CN 116935134A
- Authority
- CN
- China
- Prior art keywords
- data
- labeling
- point cloud
- annotation
- target
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000002372 labelling Methods 0.000 title claims abstract description 298
- 238000003860 storage Methods 0.000 title claims abstract description 16
- 238000000034 method Methods 0.000 claims abstract description 43
- 238000001514 detection method Methods 0.000 claims description 39
- 238000012549 training Methods 0.000 claims description 39
- 238000012216 screening Methods 0.000 claims description 13
- 230000000694 effects Effects 0.000 description 12
- 238000004891 communication Methods 0.000 description 7
- 238000003491 array Methods 0.000 description 5
- 230000009286 beneficial effect Effects 0.000 description 5
- 230000006870 function Effects 0.000 description 5
- 230000007246 mechanism Effects 0.000 description 4
- 238000012545 processing Methods 0.000 description 4
- 238000004590 computer program Methods 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 238000012550 audit Methods 0.000 description 2
- 239000000463 material Substances 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000004140 cleaning Methods 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000009826 distribution Methods 0.000 description 1
- 238000000802 evaporation-induced self-assembly Methods 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 239000012634 fragment Substances 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 238000012827 research and development Methods 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/70—Labelling scene content, e.g. deriving syntactic or semantic representations
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V2201/00—Indexing scheme relating to image or video recognition or understanding
- G06V2201/07—Target detection
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Multimedia (AREA)
- General Physics & Mathematics (AREA)
- Physics & Mathematics (AREA)
- Computing Systems (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Software Systems (AREA)
- Databases & Information Systems (AREA)
- Health & Medical Sciences (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Artificial Intelligence (AREA)
- Computational Linguistics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The application relates to the technical field of point clouds, in particular to a method, a system, a terminal and a storage medium for marking point cloud data. Determining a target labeling model and labeling personnel according to labeling task information by acquiring point cloud data to be labeled and labeling task information; inputting the point cloud data to be marked into a target marking model to obtain predicted marking data; if the number of the labeling personnel is greater than one, splitting the prediction labeling data according to the number of the labeling personnel to obtain sub prediction labeling data of each labeling personnel; and acquiring the manual annotation data determined by each annotation personnel based on the respective sub-prediction annotation data, and determining the target annotation data according to each manual annotation data. The method adopts a mode of combining model labeling and manual labeling, and can give consideration to the efficiency and the precision of point cloud labeling. Meanwhile, the model labeling result can be split into a plurality of labeling personnel to be manually adjusted, and the method is suitable for scenes with the help of a plurality of people.
Description
Technical Field
The application relates to the technical field of point clouds, in particular to a method, a system, a terminal and a storage medium for marking point cloud data.
Background
Under the industry background of new intelligence of new automobiles, most of all automobile enterprises are intelligently transformed: through software definition car, research and development intelligent driving technique guarantees intelligent driving car motorcycle type mass production entirely. The progress of automatic driving is not separated from an AI laser radar sensing algorithm, and a large amount of marking data is often required for training the laser radar sensing algorithm with good effect, high precision and strong generalization. The existing point cloud labeling method is mostly designed based on single person labeling scenes, and is difficult to be suitable for multi-person collaborative labeling scenes.
Accordingly, there is a need for improvement and advancement in the art.
Disclosure of Invention
The application provides a point cloud data labeling method, a point cloud data labeling system, a point cloud data labeling terminal and a point cloud data storage medium, and aims to solve the problem that in the related art, the point cloud labeling method is mostly designed based on single-person labeling scenes and is difficult to be suitable for multi-person collaborative labeling scenes.
In order to achieve the above purpose, the present application adopts the following technical scheme:
a point cloud data labeling method comprises the following steps:
acquiring point cloud data to be marked and marking task information corresponding to the point cloud data to be marked, and determining a target marking model and marking personnel according to the marking task information;
Inputting the point cloud data to be annotated into the target annotation model to obtain prediction annotation data corresponding to the point cloud data to be annotated;
if the number of the labeling personnel is greater than one, splitting the prediction labeling data according to the number of the labeling personnel to obtain sub prediction labeling data corresponding to each labeling personnel respectively;
and acquiring manual annotation data which are respectively determined by the annotators based on the sub-prediction annotation data corresponding to the annotators, and determining target annotation data corresponding to the point cloud data to be annotated according to the manual annotation data.
According to the technical means, the embodiment of the application adopts a mode of combining model labeling and manual labeling, and can give consideration to the efficiency and the precision of point cloud labeling. Meanwhile, the model labeling result can be split into a plurality of labeling personnel to be manually adjusted, and the method is suitable for scenes with the help of a plurality of people.
Optionally, in an embodiment of the present application, the labeling task information includes labeling type information, labeling rule information, and labeling personnel configuration information, and determining the target labeling model and the labeling personnel according to the labeling task information includes:
Determining the target annotation model according to the annotation type information and the annotation rule information;
and determining the labeling personnel according to the configuration information of the labeling personnel.
According to the technical means, the embodiment of the application determines the current suitable target annotation model from the preset plurality of annotation models according to the annotation type and the annotation rule corresponding to the point cloud data to be annotated, so that the reliability and the accuracy of predicting the annotation data can be improved. And determining the required labeling personnel according to the configuration information of the labeling personnel, so as to realize accurate allocation of the rights.
Optionally, in an embodiment of the present application, the target labeling model is trained in advance, and the method for acquiring the training data set of the target labeling model includes:
determining a target scene according to the target annotation model;
acquiring a plurality of scene data according to the target scene, wherein the scene data are point cloud data acquired based on the target scene;
and determining the training data set according to each scene data.
According to the technical means, the training data set constructed by the scene category of the target annotation model better meets the model requirement of the target annotation model, and is beneficial to the target annotation model to obtain better training effect.
Optionally, in one embodiment of the present application, the determining the training data set according to each scene data includes:
inputting each scene data into a target detection model respectively to obtain target detection data corresponding to each scene data respectively;
screening each scene data according to the target detection data of each scene data;
and determining the training data set according to the screened scene data.
According to the technical means, the scene data is screened through the target detection model, so that the data quality of the training data set can be further improved, and the target annotation model achieves a better training effect.
Optionally, in one embodiment of the present application, each of the object detection data includes a plurality of labels, and different types of the labels are respectively used for reflecting different types of objects, and each of the labels corresponds to one of the objects; the screening the scene data according to the target detection data of the scene data includes:
determining label information corresponding to each scene data according to the target detection data of each scene data, wherein for each scene data, a plurality of label categories corresponding to the scene data and the label number corresponding to each label category are determined according to the target detection data corresponding to the scene data; determining the label information corresponding to the scene data according to the label number of each label class;
And screening each scene data according to the label information of each scene data.
According to the technical means, the embodiment of the application screens the scene data through the label information, so that the scene data meeting the model requirement of the target annotation model can be accurately and rapidly screened.
Optionally, in an embodiment of the present application, the obtaining the manual labeling data determined by each labeling person based on the sub-prediction labeling data respectively includes:
for each labeling person, obtaining labeling adjustment data determined by the labeling person based on the corresponding sub-prediction labeling data;
and determining the manual annotation data corresponding to the annotators according to the sub-prediction annotation data and the annotation adjustment data.
According to the technical means, the manual annotation data in the embodiment of the application is obtained by combining the result of the target annotation model and the manually adjusted result, and because the sub-prediction annotation data already contains the annotation result of the model, the annotation can be completed by the annotator only needing a little adjustment on the sub-prediction annotation data, and the labor cost can be saved.
Optionally, in an embodiment of the present application, the determining, according to the sub-prediction annotation data and the annotation adjustment data, the manual annotation data corresponding to the annotator includes:
determining annotation data to be audited according to the sub-prediction annotation data and the annotation adjustment data;
and acquiring auditing information corresponding to the labeling data to be audited, and determining the manual labeling data according to the auditing information and the labeling data to be audited.
According to the technical means, the embodiment of the application can reduce the conditions of non-uniform marking strategy, wrong marking and missing marking when a plurality of persons mark by setting the auditing mechanism so as to obtain a high-quality marking result.
An embodiment of a second aspect of the present application provides a point cloud data labeling system, including:
the data acquisition module is used for acquiring point cloud data to be marked and marking task information corresponding to the point cloud data to be marked, and determining a target marking model and marking personnel according to the marking task information;
the model labeling module is used for inputting the point cloud data to be labeled into the target labeling model to obtain prediction labeling data corresponding to the point cloud data to be labeled;
The data splitting module is used for splitting the prediction annotation data according to the number of the annotators if the number of the annotators is greater than one, so as to obtain sub prediction annotation data corresponding to each annotator respectively;
the manual annotation module is used for acquiring manual annotation data which are determined by each annotation person based on the sub-prediction annotation data respectively corresponding to each annotation person, and determining target annotation data corresponding to the point cloud data to be annotated according to each manual annotation data.
An embodiment of a third aspect of the present application provides a terminal device, where the terminal device includes a memory, a processor, and a labeling program of point cloud data stored in the memory and capable of running on the processor, and when the processor executes the labeling program of point cloud data, the steps of the point cloud data labeling method described in any one of the above are implemented.
An embodiment of a fourth aspect of the present application provides a computer-readable storage medium, where a labeling program of point cloud data is stored, where the labeling program of point cloud data, when executed by a processor, implements the steps of the method for labeling point cloud data according to any one of the above-mentioned aspects.
The application has the beneficial effects that:
the method adopts a mode of combining model labeling and manual labeling, and can give consideration to the efficiency and the precision of point cloud labeling. Meanwhile, the model labeling result can be split into a plurality of labeling personnel to be manually adjusted, and the method is suitable for scenes with the help of a plurality of people. And by setting an auditing mechanism, the conditions of inconsistent labeling strategies, wrong labeling and label missing occurring during labeling of multiple persons can be reduced, so that a high-quality labeling result is obtained.
According to the method and the device for predicting the target annotation model, the current suitable target annotation model is determined from the preset multiple annotation models according to the annotation type and the annotation rule corresponding to the point cloud data to be annotated, and the reliability and the accuracy of predicting the annotation data can be improved. And determining the required labeling personnel according to the configuration information of the labeling personnel, so as to realize accurate allocation of the rights.
The training data set constructed by the scene category of the target annotation model better meets the model requirement of the target annotation model, and is beneficial to the target annotation model to obtain better training effect. And the label information of each scene data is obtained through the target detection model, the scene data meeting the model requirement of the target labeling model can be accurately and rapidly screened according to the label information, the data quality of a training data set is further improved, and the target labeling model achieves a better training effect.
Additional aspects and advantages of the application will be set forth in part in the description which follows and, in part, will be obvious from the description, or may be learned by practice of the application
As will be appreciated from practice of the present application.
Drawings
The foregoing and/or additional aspects and advantages of the application will become apparent and readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings, in which:
fig. 1 is a flow chart of a method for labeling point cloud data according to an embodiment of the present application;
fig. 2 is a schematic structural diagram of a point cloud data labeling system according to an embodiment of the present application;
fig. 3 is a schematic block diagram of an internal structure of a terminal device according to an embodiment of the present application.
Detailed Description
Embodiments of the present application are described in detail below, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to like or similar elements or elements having like or similar functions throughout. The embodiments described below by referring to the drawings are illustrative and intended to explain the present application and should not be construed as limiting the application.
The following describes a method, a system, a terminal and a storage medium for labeling point cloud data according to an embodiment of the present application with reference to the accompanying drawings. Aiming at the problems that the point cloud labeling method mentioned in the background technology center is mostly designed based on single-person labeling scenes and is difficult to be suitable for multi-person collaborative labeling scenes, the application provides a point cloud data labeling method, wherein in the method, target labeling models and labeling personnel are determined according to labeling task information by acquiring point cloud data to be labeled and labeling task information corresponding to the point cloud data to be labeled; inputting the point cloud data to be marked into a target marking model to obtain prediction marking data corresponding to the point cloud data to be marked; if the number of the labeling personnel is greater than one, splitting the prediction labeling data according to the number of the labeling personnel to obtain sub prediction labeling data corresponding to each labeling personnel respectively; and acquiring the manual annotation data determined by each annotation personnel based on the sub-prediction annotation data respectively, and determining the target annotation data corresponding to the point cloud data to be annotated according to each manual annotation data. The method adopts a mode of combining model labeling and manual labeling, and can give consideration to the efficiency and the precision of point cloud labeling. Meanwhile, the model labeling result can be split into a plurality of labeling personnel to be manually adjusted, and the method is suitable for scenes with the help of a plurality of people.
For example, first, point cloud data to be marked, such as a point cloud image of a traffic scene a, is acquired by a laser radar. And creating a point cloud labeling task based on the point cloud image of the traffic scene A, and setting related task information to obtain labeling task information. In the embodiment, a plurality of annotation models for executing different point cloud annotation tasks are trained in advance. And determining a target labeling model suitable for the current situation according to the labeling task information, and determining that the labeling personnel a, b and c finish the current point cloud labeling task together. And inputting the point cloud image of the traffic scene A into a target labeling model to obtain prediction labeling data, wherein lanes, vehicles, pedestrians and obstacles in the traffic scene A are labeled. Because the accuracy of the target labeling model is limited, the accuracy of the predicted labeling data is not high, and therefore labeling personnel a, b and c are required to adjust the predicted labeling data later. Dividing the predicted annotation data into three parts of sub-predicted annotation data, and manually adjusting the annotation personnel a, b and c based on the allocated sub-predicted annotation data to obtain three parts of manual annotation data. And finally, summarizing the three manual annotation data to obtain target annotation data corresponding to the point cloud image of the traffic scene A.
Specifically, fig. 1 is a schematic flow chart of a method for labeling point cloud data according to an embodiment of the present application.
As shown in fig. 1, the method for labeling point cloud data includes the following steps:
and step S100, acquiring point cloud data to be marked and marking task information corresponding to the point cloud data to be marked, and determining a target marking model and marking personnel according to the marking task information.
Specifically, the point cloud data to be marked in the embodiment may be any point cloud data to be marked, which may be acquired by a lidar sensor loaded on a vehicle. In order to label the point cloud data to be labeled, a point cloud labeling task is firstly created based on the point cloud data to be labeled, and relevant task information is set to obtain labeling task information. According to the embodiment, a plurality of annotation models for executing different point cloud annotation tasks are trained in advance, and the annotation models are trained by certain training data and have learned the mapping relation between the input and the output, so that automatic annotation can be performed based on the input point cloud data. And determining a currently suitable target labeling model and related one or more labeling personnel according to the labeling task information. And finishing the current point cloud labeling task through the determined target labeling model and labeling personnel.
For example, the point cloud data to be marked is a point cloud image, firstly, the point cloud image is synchronously processed, the time stamp of the radar and the point cloud image is aligned, the item of the point cloud image and the category of the marking type are determined after the radar and the time stamp of the point cloud image are completed, and a new data set is created in the marking system and used for uploading the point cloud image. And creating a new point cloud labeling task in the labeling system for setting task information related to the point cloud image. The labeling system can automatically determine a proper target labeling model and labeling personnel according to labeling task information.
In one embodiment, the labeling task information includes labeling type information, labeling rule information, and labeling personnel configuration information, and determining the target labeling model and labeling personnel according to the labeling task information includes:
determining a target annotation model according to the annotation type information and the annotation rule information;
and determining the labeling personnel according to the configuration information of the labeling personnel.
The annotation type information is used for reflecting the annotation type corresponding to the point cloud data to be annotated, such as classification annotation, region annotation and semantic annotation. The labeling rule information is used for reflecting labeling rules corresponding to the point cloud data to be labeled, and the labeling rules can comprise label distribution rules or color block configuration rules. Taking the point cloud image and the color block configuration rule of the traffic scene a as an example, the labeling rule can be to set a lane area in the point cloud image of the traffic scene a as a yellow block, a vehicle area as a blue block and a pedestrian area as a green block. The label personnel configuration information is used for reflecting the number of label personnel required for completing the point cloud data to be labeled. In an actual application scene, because of differences between marking types and marking rules applicable to different marking models, in order to improve reliability and accuracy of prediction marking data, the embodiment determines a currently suitable target marking model from a plurality of preset marking models according to marking types and marking rules corresponding to point cloud data to be marked. And determining the required labeling personnel according to the configuration information of the labeling personnel, and distributing relevant rights for the labeling personnel.
And step 200, inputting the point cloud data to be marked into a target marking model to obtain prediction marking data corresponding to the point cloud data to be marked.
Specifically, in order to improve the labeling efficiency of the point cloud data to be labeled, in this embodiment, the point cloud data to be labeled is input into the target labeling model. Because the target annotation model is trained in advance, automatic annotation can be performed based on input point cloud data to be annotated, and prediction annotation data is output.
In one embodiment, the target labeling model is trained in advance, and the method for acquiring the training data set of the target labeling model includes:
determining a target scene according to the target annotation model;
acquiring a plurality of scene data according to a target scene, wherein the scene data are point cloud data acquired based on the target scene;
from each scene data, a training data set is determined.
Specifically, before training the target annotation model, a training dataset of the target annotation model needs to be constructed. The model requirements of different annotation models are different, the model requirements are related to scene categories of point cloud data required by annotation model training, and a training data set is required to be constructed according to the model requirements so as to achieve a better training effect. Therefore, the target scene corresponding to the target annotation model needs to be determined first, and then the point cloud data related to the target scene is acquired, so that the scene data is obtained. A training dataset of the target annotation model is constructed from a sufficient number of scene data. According to the embodiment, the training data set constructed by the scene category of the target annotation model is more in line with the model requirement of the target annotation model, and is beneficial to the target annotation model to obtain a better training effect.
For example, scene categories may be classified into pedestrian streets, highways, gates, tunnels, express ways, intersections, curves, underground parking lots based on traffic characteristics; or on the basis of weather factors, the weather factors are classified into rainy days, sunny days, foggy days and snowy days; or divided into daytime, evening, night based on time. When training data of the target annotation model are collected, a corresponding scene is selected according to the model requirement of the target annotation model.
In one embodiment, the acquiring of the plurality of scene data further comprises:
and aiming at each scene data, segmenting the scene data to obtain target scene data.
Specifically, the conventional data processing link is to analyze and then mine, so that the data efficiency is low and the acquisition feedback cannot be obtained in time. Therefore, after each scene data is obtained, the embodiment segments the scene data, and the segmented effective data is the target scene data. And then storing and analyzing the target scene data.
For example, the collected scene data is stored in a rosbag format, scene tags are marked according to different scene categories, and the scene data in the rosbag format is intercepted into object fragments by using an intercepting tool, so that the target scene data is obtained. And storing the data in the disk arrays of the NAS machines at the vehicle end according to different requirements, taking out the NAS disk arrays at the vehicle end, taking the NAS disk arrays to a machine room terminal server, and copying the data to the divided local object storage space by using the NAS machines with the same model. During the parsing process, the needed topic fields are read from the rosbag package, and the corresponding data are stored in the format of. Pcd and. Jpg.
In one embodiment, the target scene data is automatically copied, and the data can be automatically copied into the storage server by correctly inserting the disk. Thereby reducing the labor cost and meeting the data transmission requirement.
In one embodiment, determining the training data set from the respective scene data includes:
respectively inputting each scene data into a target detection model to obtain target detection data corresponding to each scene data;
screening the scene data according to the target detection data of the scene data;
and determining a training data set according to the screened scene data.
Specifically, in this embodiment, a target detection model is trained in advance, input data of the target detection model is scene data, output data is target detection data, and the target detection data may reflect a plurality of targets identified in the scene data. The annotation model requires scene objects when scene data is used, so that the quality of the scene data is related to the types and the number of the objects contained in the scene data, and the scene data can be cleaned based on the object detection data, namely, the scene data can be screened. And inputting the scene data into a target detection model aiming at each scene data to obtain target detection data corresponding to the scene data, and judging whether to screen the scene data according to the target detection data. And constructing a training data set through all the screened scene data. According to the method, the scene data are screened through the target detection model, so that the data quality of a training data set can be further improved, and a better training effect is achieved by the target annotation model.
In one embodiment, each object detection data includes a plurality of labels, different types of labels are respectively used for reflecting different types of objects, and each label corresponds to one object; screening each scene data according to the target detection data of each scene data comprises the following steps:
determining label information corresponding to each scene data according to the target detection data of each scene data, wherein for each scene data, a plurality of label categories corresponding to the scene data and the label number corresponding to each label category are determined according to the target detection data corresponding to the scene data; determining label information corresponding to the scene data according to the label number of each label class;
and screening the scene data according to the label information of the scene data.
Specifically, when the object detection model performs recognition analysis on each scene data, a corresponding label is marked on the object according to the type of the recognized object, so that each object detection data comprises a plurality of labels. When screening the scene data, firstly determining which label categories are contained in the scene data based on the target detection data of each scene data, and how many labels are corresponding to each label category, and forming label information of the scene data through the information. And then screening the scene data according to the label information, so that the scene data meeting the model requirements of the target annotation model can be accurately and rapidly screened out. For example, the target labeling model is mainly used for labeling pedestrians, and then the scene data containing label categories corresponding to the pedestrians and meeting the requirements in number of the label categories needs to be screened out.
In one embodiment, the filtering of each scene data according to the target detection data of each scene data further includes:
and manually screening the screened scene data.
Specifically, the effective rate of the scene data screened by the target detection model can reach 90%, in order to make the effective rate of the data higher, the embodiment also adds a manual cleaning link, and manually screens the scene data screened by the target detection model again, thereby screening the scene data meeting the model requirements more.
And step 300, if the number of the labeling personnel is greater than one, splitting the prediction labeling data according to the number of the labeling personnel to obtain sub prediction labeling data corresponding to each labeling personnel.
Specifically, when it is determined that the point cloud data to be marked is marked by a plurality of marking personnel, the prediction marking data is split into a plurality of sub prediction marking data equal to the number of the marking personnel, so that each marking personnel is distributed with one sub prediction marking data. According to the embodiment, the point cloud data annotation in the model+multi-person scene can be realized by splitting the prediction annotation data.
For example, assuming that labeling personnel are three persons, the prediction labeling data are split into three sub-prediction labeling data, and each sub-prediction labeling data is respectively sent to the corresponding labeling personnel for display. When the labeling personnel views the sub-prediction labeling data through the labeling system, the angles can be freely adjusted according to the displayed sub-prediction labeling data, and the angles are rotated to adapt to the labeling habit requirements of the labeling personnel.
In one embodiment, each sub-prediction annotation data is continuous frame point cloud data.
Specifically, the point cloud segmentation labeling is characterized by large workload and more details, when a large amount of manpower is used for randomly picking up data, the difference of data labeling results before and after a time stamp is easy to cause, so that the point cloud data can be split based on the time stamp of the point cloud data when the predicted labeling data is split, the point cloud data picked up by each labeling person is continuous, and the consistency of the data labeling results before and after the time stamp is ensured.
In one embodiment, the method for labeling point cloud data further includes:
and acquiring picture parameter adjustment data, and adjusting the display interface according to the picture parameter adjustment data.
Specifically, in order to achieve a better labeling effect, the embodiment can also adjust the brightness, contrast and color block transparency of the interface, so that labeling is convenient.
And step S400, acquiring manual annotation data determined by each annotator based on the sub-prediction annotation data respectively, and determining target annotation data corresponding to the point cloud data to be annotated according to each manual annotation data.
Specifically, each labeling person can adjust sub-prediction labeling data allocated to the labeling person, and because the sub-prediction labeling data already contains labeling results of the model, the labeling person can finish labeling on the sub-prediction labeling data only by a small amount of adjustment, so as to obtain manual labeling data, and finally, summarizing the manual labeling data, so as to obtain labeling results of point cloud data to be labeled. The embodiment adopts a mode of combining model labeling and manual labeling, and can ensure labeling precision while improving labeling efficiency.
In one embodiment, obtaining the manual annotation data determined by each annotator based on the respective corresponding sub-predictive annotation data comprises:
aiming at each labeling person, labeling adjustment data determined by the labeling person based on the corresponding sub-prediction labeling data are obtained;
and determining the manual annotation data corresponding to the annotators according to the sub-prediction annotation data and the annotation adjustment data.
Specifically, because the accuracy of the target annotation model is limited, there may be a case of erroneous annotation or missed annotation in the sub-prediction annotation data. After the labeling personnel obtain the sub-prediction labeling data, the sub-prediction labeling data can be adjusted according to own experience. After the marking system receives marking adjustment data input by marking personnel, the sub-prediction marking data are corrected according to the marking adjustment data, and after correction, manual marking data are generated. In other words, the manual labeling data in this embodiment is actually data obtained by combining the result of the target labeling model and the result of manual adjustment, so that the labeling accuracy can be ensured while the labeling efficiency is improved in this embodiment.
In one embodiment, the method for labeling point cloud data further includes:
And acquiring operation data of the labeling personnel, and adjusting the manual labeling data according to the operation data.
Specifically, because the manual labeling may have errors such as label error, label missing or label multiple, etc. due to carelessness or unclear labeling rules, etc., the labeling personnel in this embodiment may further check and adjust the manual labeling data. Specifically, the labeling system can display the effect graph of the finally generated labeling result to labeling personnel, the labeling personnel can check each region in the labeling result, and when labeling labels of a certain region are selected, the region is highlighted, so that the labeling personnel can check conveniently. The labeling personnel can also delete the label according to the situations such as error labeling or invalid labeling.
In one embodiment, determining the manual annotation data corresponding to the annotators according to the sub-prediction annotation data and the annotation adjustment data comprises:
determining the annotation data to be audited according to the sub-prediction annotation data and the annotation adjustment data;
and acquiring auditing information corresponding to the labeling data to be audited, and determining the manual labeling data according to the auditing information and the labeling data to be audited.
Specifically, when labeling is performed by multiple persons, the labeling strategies are not uniform, label errors and label missing easily occur, so that an auditing mechanism is designed in the embodiment. In the embodiment, sub-prediction annotation data adjusted by the annotators are defined as to-be-checked annotation data. After labeling personnel labeling, each piece of labeling data to be checked is automatically submitted to an auditing link, the auditing personnel sequentially audits each piece of labeling data to be checked, and the data which does not pass the auditing are subjected to rejection operation, and the labeling personnel need to modify and resubmit the labeling according to auditing information. The data passing the audit is used as the manual annotation data generated by the annotator based on the sub-prediction annotation data.
In one embodiment, in order to ensure that the overall labeling quality reaches the standard, the embodiment can perform the spot check after performing the quality check on the target labeling data, and further improve the data quality through multiple quality checks.
In one embodiment, the method for labeling point cloud data further includes:
and storing the target annotation data according to a preset file format.
For example, if the preset file format is json format, the point cloud data to be annotated and the corresponding target annotation data are generated into an integrated json file, the json file name is the same as the corresponding pcd name, for example, the a1234.pcd generates a1234.json, the json file content is a dictionary (direct), the dictionary can be used for subsequent training, the meaning of each field used in file storage is shown in the following table 1, and all the listed fields are necessary fields.
TABLE 1 field meanings associated with File storage
In conclusion, the method adopts a mode of combining model labeling and manual labeling, and can give consideration to the efficiency and the precision of point cloud labeling. Meanwhile, the model labeling result can be split into a plurality of labeling personnel to be manually adjusted, and the method is suitable for scenes with the help of a plurality of people. And by setting an auditing mechanism, the conditions of inconsistent labeling strategies, wrong labeling and label missing occurring during labeling of multiple persons can be reduced, so that a high-quality labeling result is obtained. And secondly, determining the current suitable target annotation model from the preset multiple annotation models according to the annotation type and the annotation rule corresponding to the point cloud data to be annotated, so that the reliability and the accuracy of the predicted annotation data can be improved. And determining the required labeling personnel according to the configuration information of the labeling personnel, so as to realize accurate allocation of the rights. In addition, the training data set constructed by the scene category of the target annotation model better meets the model requirement of the target annotation model, and is beneficial to the target annotation model to obtain better training effect. And the label information of each scene data is obtained through the target detection model, the scene data meeting the model requirement of the target labeling model can be accurately and rapidly screened according to the label information, the data quality of a training data set is further improved, and the target labeling model achieves a better training effect.
Next, a labeling system for point cloud data according to an embodiment of the present application is described with reference to the accompanying drawings.
As shown in fig. 2, the labeling system 10 for point cloud data includes: the system comprises a data acquisition module 100, a model labeling module 200, a data splitting module 300 and a manual labeling module 400.
Specifically, the data acquisition module 100 is configured to acquire point cloud data to be annotated and annotation task information corresponding to the point cloud data to be annotated, and determine a target annotation model and an annotation personnel according to the annotation task information;
the model labeling module 200 is configured to input the point cloud data to be labeled into the target labeling model to obtain predicted labeling data corresponding to the point cloud data to be labeled;
the data splitting module 300 is configured to split the predicted annotation data according to the number of the annotators if the number of the annotators is greater than one, so as to obtain sub-predicted annotation data corresponding to each annotator;
the manual annotation module 400 is configured to obtain manual annotation data determined by each of the annotators based on the sub-prediction annotation data respectively, and determine target annotation data corresponding to the point cloud data to be annotated according to each of the manual annotation data.
It should be noted that the explanation of the embodiment of the method for labeling point cloud data is also applicable to the system for labeling point cloud data in this embodiment, and will not be repeated here.
Fig. 3 is a schematic structural diagram of a terminal device according to an embodiment of the present application. The terminal device may include:
memory 301, processor 302, and a computer program stored on memory 301 and executable on processor 302.
The processor 302 implements the method for labeling point cloud data provided in the above embodiment when executing a program.
Further, the terminal device further includes:
a communication interface 303 for communication between the memory 301 and the processor 302.
A memory 301 for storing a computer program executable on the processor 302.
The memory 301 may comprise a high-speed RAM memory or may further comprise a non-volatile memory (non-volatile memory), such as at least one disk memory.
If the memory 301, the processor 302, and the communication interface 303 are implemented independently, the communication interface 303, the memory 301, and the processor 302 may be connected to each other through a bus and perform communication with each other. The bus may be an industry standard architecture (Industry Standard Architecture, abbreviated ISA) bus, an external device interconnect (Periphera l Component, abbreviated PCI) bus, or an extended industry standard architecture (Extended Industry Standard Architecture, abbreviated EISA) bus, among others. The buses may be divided into address buses, data buses, control buses, etc. For ease of illustration, only one thick line is shown in fig. 3, but not only one bus or one type of bus.
Alternatively, in a specific implementation, if the memory 301, the processor 302, and the communication interface 303 are integrated on a chip, the memory 301, the processor 302, and the communication interface 303 may communicate with each other through internal interfaces.
The processor 302 may be a central processing unit (Central Processing Unit, abbreviated as CPU) or an application specific integrated circuit (Application Specific Integrated Circuit, abbreviated as ASIC) or one or more integrated circuits configured to implement embodiments of the present application.
The present embodiment also provides a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements the point cloud data labeling method as above.
In the description of the present specification, a description referring to terms "one embodiment," "some embodiments," "examples," "particular embodiments," or "some examples," etc., means that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the application. In this specification, schematic representations of the above terms are not necessarily directed to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or N embodiments or examples. Furthermore, the different embodiments or examples described in this specification and the features of the different embodiments or examples may be combined and combined by those skilled in the art without contradiction.
Furthermore, the terms "first," "second," and the like, are used for descriptive purposes only and are not to be construed as indicating or implying a relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defining "a first" or "a second" may explicitly or implicitly include at least one such feature. In the description of the present application, "N" means at least two, for example, two, three, etc., unless specifically defined otherwise.
Any process or method descriptions in flow charts or otherwise described herein may be understood as representing modules, segments, or portions of code which include one or more executable instructions for implementing specific logical functions or steps of the process, and additional implementations are included within the scope of the preferred embodiment of the present application in which functions may be executed out of order from that shown or discussed, including substantially concurrently or in reverse order from that shown or discussed, depending on the functionality involved, as would be understood by those reasonably skilled in the art of the embodiments of the present application.
Logic and/or steps represented in the flowcharts or otherwise described herein, e.g., a ordered listing of executable instructions for implementing logical functions, can be embodied in any computer-readable medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, processor-containing system, or other system that can read instructions from and execute instructions. For the purposes of this description, a "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of the computer-readable medium would include the following: an electrical connection (electronic device) having one or N wires, a portable computer cartridge (magnetic device), a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disc read-only memory (CDROM). Additionally, the computer readable medium may even be paper or other suitable medium upon which the program is printed, as the program may be electronically captured, via optical scanning of the paper or other medium, then compiled, interpreted or otherwise processed in a suitable manner if necessary, and then stored in a computer memory.
It is to be understood that portions of the present application may be implemented in hardware, software, firmware, or a combination thereof. In the above-described embodiments, the N steps or methods may be implemented in software or firmware stored in a memory and executed by a suitable instruction execution system. If implemented in hardware, as in another embodiment, may be implemented using any one or combination of the following techniques, as known in the art: discrete logic circuits having logic gates for implementing logic functions on data signals, application specific integrated circuits having suitable combinational logic gates, programmable Gate Arrays (PGAs), field Programmable Gate Arrays (FPGAs), and the like.
Those of ordinary skill in the art will appreciate that all or part of the steps carried out in the method of the above-described embodiments may be implemented by a program to instruct related hardware, and the program may be stored in a computer readable storage medium, where the program when executed includes one or a combination of the steps of the method embodiments.
In addition, each functional unit in the embodiments of the present application may be integrated in one processing module, or each unit may exist alone physically, or two or more units may be integrated in one module. The integrated modules may be implemented in hardware or in software functional modules. The integrated modules may also be stored in a computer readable storage medium if implemented as software functional modules and sold or used as a stand-alone product.
The above-mentioned storage medium may be a read-only memory, a magnetic disk or an optical disk, or the like. While embodiments of the present application have been shown and described above, it will be understood that the above embodiments are illustrative and not to be construed as limiting the application, and that variations, modifications, alternatives and variations may be made to the above embodiments by one of ordinary skill in the art within the scope of the application.
Claims (10)
1. The method for labeling the point cloud data is characterized by comprising the following steps of:
acquiring point cloud data to be marked and marking task information corresponding to the point cloud data to be marked, and determining a target marking model and marking personnel according to the marking task information;
inputting the point cloud data to be annotated into the target annotation model to obtain prediction annotation data corresponding to the point cloud data to be annotated;
if the number of the labeling personnel is greater than one, splitting the prediction labeling data according to the number of the labeling personnel to obtain sub prediction labeling data corresponding to each labeling personnel respectively;
and acquiring manual annotation data which are respectively determined by the annotators based on the sub-prediction annotation data corresponding to the annotators, and determining target annotation data corresponding to the point cloud data to be annotated according to the manual annotation data.
2. The method of claim 1, wherein the labeling task information includes labeling type information, labeling rule information, and labeling personnel configuration information, and the determining the target labeling model and labeling personnel according to the labeling task information includes:
determining the target annotation model according to the annotation type information and the annotation rule information;
and determining the labeling personnel according to the configuration information of the labeling personnel.
3. The method for labeling point cloud data according to claim 1, wherein the target labeling model is trained in advance, and the method for acquiring the training data set of the target labeling model comprises:
determining a target scene according to the target annotation model;
acquiring a plurality of scene data according to the target scene, wherein the scene data are point cloud data acquired based on the target scene;
and determining the training data set according to each scene data.
4. The method for labeling point cloud data as recited in claim 3, wherein said determining said training data set based on each of said scene data comprises:
inputting each scene data into a target detection model respectively to obtain target detection data corresponding to each scene data respectively;
Screening each scene data according to the target detection data of each scene data;
and determining the training data set according to the screened scene data.
5. The method for labeling point cloud data according to claim 4, wherein each of said object detection data includes a plurality of labels, different types of said labels being respectively used for reflecting different types of objects, each of said labels corresponding to one of said objects; the screening the scene data according to the target detection data of the scene data includes:
determining label information corresponding to each scene data according to the target detection data of each scene data, wherein for each scene data, a plurality of label categories corresponding to the scene data and the label number corresponding to each label category are determined according to the target detection data corresponding to the scene data; determining the label information corresponding to the scene data according to the label number of each label class;
and screening each scene data according to the label information of each scene data.
6. The method of labeling point cloud data according to claim 1, wherein the obtaining manual labeling data determined by each labeling person based on the sub-prediction labeling data respectively corresponding to each labeling person comprises:
for each labeling person, obtaining labeling adjustment data determined by the labeling person based on the corresponding sub-prediction labeling data;
and determining the manual annotation data corresponding to the annotators according to the sub-prediction annotation data and the annotation adjustment data.
7. The method of labeling point cloud data according to claim 6, wherein determining the manual labeling data corresponding to the labeling person according to the sub-prediction labeling data and the labeling adjustment data comprises:
determining annotation data to be audited according to the sub-prediction annotation data and the annotation adjustment data;
and acquiring auditing information corresponding to the labeling data to be audited, and determining the manual labeling data according to the auditing information and the labeling data to be audited.
8. A point cloud data annotation system, comprising:
the data acquisition module is used for acquiring point cloud data to be marked and marking task information corresponding to the point cloud data to be marked, and determining a target marking model and marking personnel according to the marking task information;
The model labeling module is used for inputting the point cloud data to be labeled into the target labeling model to obtain prediction labeling data corresponding to the point cloud data to be labeled;
the data splitting module is used for splitting the prediction annotation data according to the number of the annotators if the number of the annotators is greater than one, so as to obtain sub prediction annotation data corresponding to each annotator respectively;
the manual annotation module is used for acquiring manual annotation data which are determined by each annotation person based on the sub-prediction annotation data respectively corresponding to each annotation person, and determining target annotation data corresponding to the point cloud data to be annotated according to each manual annotation data.
9. A terminal device, characterized in that the terminal device comprises a memory, a processor and a labeling program of point cloud data stored in the memory and operable on the processor, the processor implementing the steps of the point cloud data labeling method according to any one of claims 1-7 when executing the labeling program of point cloud data.
10. A computer-readable storage medium, wherein a labeling program of point cloud data is stored on the computer-readable storage medium, and when the labeling program of point cloud data is executed by a processor, the steps of the point cloud data labeling method according to any one of claims 1 to 7 are implemented.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310951444.2A CN116935134A (en) | 2023-07-31 | 2023-07-31 | Point cloud data labeling method, point cloud data labeling system, terminal and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310951444.2A CN116935134A (en) | 2023-07-31 | 2023-07-31 | Point cloud data labeling method, point cloud data labeling system, terminal and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116935134A true CN116935134A (en) | 2023-10-24 |
Family
ID=88392317
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310951444.2A Pending CN116935134A (en) | 2023-07-31 | 2023-07-31 | Point cloud data labeling method, point cloud data labeling system, terminal and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116935134A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117933976A (en) * | 2024-03-25 | 2024-04-26 | 北京三五通联科技发展有限公司 | Data labeling business process management method and system |
-
2023
- 2023-07-31 CN CN202310951444.2A patent/CN116935134A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117933976A (en) * | 2024-03-25 | 2024-04-26 | 北京三五通联科技发展有限公司 | Data labeling business process management method and system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109087510B (en) | Traffic monitoring method and device | |
| KR101589711B1 (en) | Methods and systems for processing of video data | |
| US11455805B2 (en) | Method and apparatus for detecting parking space usage condition, electronic device, and storage medium | |
| KR102177880B1 (en) | Class labeling apparatus for autonomous driving | |
| US11380104B2 (en) | Method and device for detecting illegal parking, and electronic device | |
| JP2022506920A (en) | Compartment line detection methods, devices, electronic devices and readable storage media | |
| WO2021036243A1 (en) | Method and apparatus for recognizing lane, and computing device | |
| CN116935134A (en) | Point cloud data labeling method, point cloud data labeling system, terminal and storage medium | |
| WO2021185121A1 (en) | Model generation method and apparatus, object detection method and apparatus, device, and storage medium | |
| CN113449632B (en) | Vision and radar perception algorithm optimization method and system based on fusion perception and automobile | |
| CN112447060A (en) | Method and device for recognizing lane and computing equipment | |
| CN113902740A (en) | Construction method of image blurring degree evaluation model | |
| CN115705685A (en) | Image data set labeling method and device and electronic equipment | |
| CN111325811B (en) | Lane line data processing method and processing device | |
| CN116794650A (en) | Millimeter wave radar and camera data fusion target detection method and device | |
| CN113345101B (en) | Three-dimensional point cloud labeling method, device, equipment and storage medium | |
| CN115909241A (en) | Lane line detection method, system, electronic device and storage medium | |
| CN111143423B (en) | Dynamic scene labeling data mining method and device and terminal | |
| CN113435350A (en) | Traffic marking detection method, device, equipment and medium | |
| US20240135679A1 (en) | Method for classifying images and electronic device | |
| CN116935168B (en) | Method, device, computer equipment and storage medium for target detection | |
| WO2024066798A1 (en) | Vehicle control method and apparatus, and device and storage medium | |
| CN117739950B (en) | Map generation method, device and equipment | |
| WO2023178510A1 (en) | Image processing method, device, and system and movable platform | |
| WO2023221848A1 (en) | Vehicle starting behavior prediction method and apparatus, storage medium, and program product |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |