WO2022068225A1 - Point cloud annotating method and apparatus, electronic device, storage medium, and program product - Google Patents
Point cloud annotating method and apparatus, electronic device, storage medium, and program product Download PDFInfo
- Publication number
- WO2022068225A1 WO2022068225A1 PCT/CN2021/095444 CN2021095444W WO2022068225A1 WO 2022068225 A1 WO2022068225 A1 WO 2022068225A1 CN 2021095444 W CN2021095444 W CN 2021095444W WO 2022068225 A1 WO2022068225 A1 WO 2022068225A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- dimensional
- information
- marked
- view
- point cloud
- Prior art date
Links
- 238000000034 method Methods 0.000 title claims abstract description 85
- 235000004522 Pentaglottis sempervirens Nutrition 0.000 claims abstract description 94
- 230000004044 response Effects 0.000 claims abstract description 51
- 238000002372 labelling Methods 0.000 claims description 190
- 240000004050 Pentaglottis sempervirens Species 0.000 claims description 78
- 238000012795 verification Methods 0.000 claims description 56
- 238000013507 mapping Methods 0.000 claims description 31
- 238000004590 computer program Methods 0.000 claims description 15
- 230000007246 mechanism Effects 0.000 description 17
- 238000012546 transfer Methods 0.000 description 17
- 230000008569 process Effects 0.000 description 16
- 238000001514 detection method Methods 0.000 description 6
- 238000012545 processing Methods 0.000 description 6
- 238000010586 diagram Methods 0.000 description 5
- 238000012549 training Methods 0.000 description 5
- 230000003993 interaction Effects 0.000 description 4
- 238000011160 research Methods 0.000 description 4
- 238000004891 communication Methods 0.000 description 3
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 230000002452 interceptive effect Effects 0.000 description 3
- 238000006243 chemical reaction Methods 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000001413 cellular effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 239000012141 concentrate Substances 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
- 230000009897 systematic effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/70—Labelling scene content, e.g. deriving syntactic or semantic representations
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
- G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
- G06T7/75—Determining position or orientation of objects or cameras using feature-based methods involving models
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
- G01S17/88—Lidar systems specially adapted for specific applications
- G01S17/89—Lidar systems specially adapted for specific applications for mapping or imaging
- G01S17/894—3D imaging with simultaneous measurement of time-of-flight at a 2D array of receiver pixels, e.g. time-of-flight cameras or flash lidar
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
- G01S17/88—Lidar systems specially adapted for specific applications
- G01S17/93—Lidar systems specially adapted for specific applications for anti-collision purposes
- G01S17/931—Lidar systems specially adapted for specific applications for anti-collision purposes of land vehicles
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T19/00—Manipulating 3D models or images for computer graphics
- G06T19/20—Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
- G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/97—Determining parameters from multiple pictures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
- G06V20/58—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10028—Range image; Depth image; 3D point clouds
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10032—Satellite or aerial image; Remote sensing
- G06T2207/10044—Radar image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30248—Vehicle exterior or interior
- G06T2207/30252—Vehicle exterior; Vicinity of vehicle
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2219/00—Indexing scheme for manipulating 3D models or images for computer graphics
- G06T2219/20—Indexing scheme for editing of 3D models
- G06T2219/2004—Aligning objects, relative positioning of parts
Definitions
- the present disclosure relates to the field of computer technology, and in particular, to a method, device, electronic device, storage medium and program product for point cloud labeling.
- Lidar is widely used in various technical fields. Taking the field of autonomous driving as an example, it is particularly important to use the point cloud data collected by lidar to accurately detect obstacles around the vehicle, such as pedestrians and other vehicles.
- the detection model In order to accurately detect obstacles, the detection model needs to be trained based on the collected point cloud data as training samples. Before training, it is necessary to accurately label the collected point cloud data to improve the accuracy of the detection results.
- Embodiments of the present disclosure provide a method, apparatus, electronic device, storage medium, and program product for point cloud labeling.
- an embodiment of the present disclosure provides a point cloud labeling method, the method comprising:
- the three-dimensional annotation frame information corresponding to the object to be annotated is determined.
- the above point cloud labeling method when the 3D point cloud data to be labelled is obtained, the corresponding bird's-eye view can be generated first. In this way, after responding to the frame selection operation for the objects to be labelled in the bird's eye view, the first One-dimensional labeling frame information, and at the same time, the initial parameter information of the to-be-labeled object can also be determined in combination with the 3D point cloud data of the frame-selected to-be-labeled object. Information about the 3D callout box corresponding to the callout object. It can be seen that the above point cloud labeling method is a frame selection operation performed in a two-dimensional view (corresponding to a bird's-eye view).
- the complexity of performing the frame selection operation under the view is low, and the magnified view of the local point cloud facilitates the user to accurately identify the boundary of the labeling box, which can ensure the accuracy of the framed object to be labelled to a certain extent.
- the obtained first two-dimensional labeling frame information and initial parameter information can realize automatic labeling of the three-dimensional labeling frame for the object to be labelled, and the labeling efficiency and accuracy are high.
- the method further includes:
- a view to be adjusted corresponding to the three-dimensional point cloud data to be marked is generated;
- the to-be-adjusted view includes at least one of a front view and a side view;
- second two-dimensional labeling frame information of the to-be-labeled object in the to-be-adjusted view is determined.
- the determining, based on the first 2D annotation frame information and the initial parameter information, the 3D annotation frame information corresponding to the object to be annotated includes:
- the 3D annotation frame information corresponding to the object to be annotated is determined.
- the view to be adjusted and the bird's-eye view are two different 2D views that belong to the same 3D view where the 3D point cloud data is located, there is a correlation of parameter information between the two different 2D views.
- the second two-dimensional annotation frame information determined by the view and the parameter information determined by the first two-dimensional annotation frame information determined by the bird's-eye view more accurate three-dimensional annotation frame information can be determined.
- the 3D annotation frame information corresponding to the object to be annotated is determined based on the first 2D annotation frame information, the second 2D annotation frame information and the initial parameter information ,include:
- the 3D annotation frame information corresponding to the object to be annotated is determined.
- the initial parameter information can be adjusted based on the adjustment operation for the second two-dimensional annotation frame information of the object to be labeled in the view to be adjusted, so that the adjusted parameter information and the first parameter corresponding to the bird's-eye view can be adjusted.
- One 2D annotation frame information and second 2D annotation frame information corresponding to the view to be adjusted determine the 3D annotation frame information corresponding to the to-be-annotated object. This may be due to the fact that different views focus on different information about the object to be marked.
- the corresponding display may be the length and width of the object to be marked, and for the front view, the corresponding display The display can be the width and height of the object to be marked. Therefore, on the basis of the initial parameter information, parameter adjustment is performed in combination with the view to be adjusted, which will further improve the accuracy of the determined 3D annotation frame information.
- determining the adjusted parameter information of the object to be annotated including:
- the initial parameter information is adjusted based on the parameter adjustment information to obtain the adjusted parameter information.
- the 3D annotation corresponding to the object to be annotated is determined based on the first 2D annotation frame information, the second 2D annotation frame information and the adjusted parameter information Box information, including:
- the corresponding relationship between the three-dimensional annotation frames is determined on multiple planes, and the information of the three-dimensional annotation frames is determined by combining the information.
- the method further includes:
- the to-be-adjusted view marked with a second two-dimensional annotation frame corresponding to the second two-dimensional annotation frame information is displayed.
- the method further includes:
- mapping the coordinate system corresponding to the three-dimensional point cloud data to be marked and the coordinate system corresponding to the verification image Based on the mapping relationship between the coordinate system corresponding to the three-dimensional point cloud data to be marked and the coordinate system corresponding to the verification image, map the three-dimensional annotation frame information corresponding to the object to be marked into the verification image , determine the coordinate range of the image after mapping;
- the accuracy of the three-dimensional annotation frame information is verified according to the mapped image coordinate range and the image coordinate range in the verification image where the object to be marked is located.
- the accuracy of the 3D annotation frame information can be verified based on the obtained verification image, which may depend on the coordinate system corresponding to the 3D point cloud data to be annotated.
- the mapping relationship between the coordinate systems corresponding to the verification image is realized by mapping the image coordinate range, so that the specific object to be marked can be verified more accurately.
- the accuracy of the three-dimensional annotation frame information is checked. After verification, it also includes:
- the adjustment operation of the two-dimensional annotation frame is to adjust the adjusted parameter information until the image coordinate range after the mapping is consistent with the image coordinate range where the object to be annotated in the verification image is located; the adjusted image coordinate range is The parameter information is obtained by adjusting the initial parameter information.
- the method further includes:
- the 3D annotation frame information of the marked object determine the 3D annotation frame information of other objects to be marked in the 3D view; the other objects to be marked and the marked object belong to the same object type;
- the 3D annotation frame information of the other objects to be annotated is displayed in the 3D view.
- the determining, according to the 3D annotation frame information of the marked object, the 3D annotation frame information of other objects to be annotated in the 3D view includes:
- the 3D annotation frame information of the marked object can be pasted to the On other to-be-labeled objects belonging to the same object type, three-dimensional labeling of other to-be-labeled objects can be quickly realized, and the labeling efficiency is further improved.
- the method further includes:
- determining the first two-dimensional frame information in response to a frame selection operation on the object to be marked in the bird's-eye view, includes:
- the reference image and the to-be-labeled 3D point cloud data originate from the same scene;
- the first two-dimensional frame information is determined based on the reference position information.
- the reference position information of the object to be marked in the bird's-eye view can be determined based on the obtained reference image, and based on this reference position information, the user can be instructed to perform a frame selection operation for the object to be marked in the bird's-eye view.
- the resulting first two-dimensional labeling frame will cover the point cloud data corresponding to the object to be labelled to a greater extent, thereby improving the labeling accuracy.
- the initial parameter information includes at least one of initial height information and initial pose information.
- an embodiment of the present disclosure further provides a point cloud labeling device, the device comprising:
- an acquisition module configured to acquire the 3D point cloud data to be marked
- a generating module configured to generate a bird's-eye view corresponding to the three-dimensional point cloud data to be marked based on the three-dimensional point cloud data to be marked;
- a first determination module configured to determine first two-dimensional labeling frame information in response to a frame selection operation for objects to be labelled in the bird's-eye view; The initial parameter information of the annotation object;
- the labeling module is configured to determine the three-dimensional labeling frame information corresponding to the object to be labelled based on the first two-dimensional labeling frame information and the initial parameter information.
- embodiments of the present disclosure further provide an electronic device, including: a processor, a memory, and a bus, where the memory stores machine-readable instructions executable by the processor, and the processor is configured to execute all The machine-readable instructions stored in the memory, when the electronic device is running, the processor communicates with the memory through a bus, and the machine-readable instructions are executed by the processor.
- an electronic device including: a processor, a memory, and a bus, where the memory stores machine-readable instructions executable by the processor, and the processor is configured to execute all The machine-readable instructions stored in the memory, when the electronic device is running, the processor communicates with the memory through a bus, and the machine-readable instructions are executed by the processor.
- an embodiment of the present disclosure further provides a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is run by an electronic device, the electronic device executes the first Aspects and steps of the method for point cloud labeling described in any one of its various embodiments.
- embodiments of the present disclosure further provide a computer program product, including a computer-readable storage medium storing program codes, where the program codes include instructions that can be configured to execute any of the first aspect and its various implementations. 1. The steps of the method for point cloud labeling.
- FIG. 1 shows a schematic flowchart of a method for labeling a point cloud provided by an embodiment of the present disclosure
- FIG. 2 shows a schematic flowchart of a point cloud labeling provided by an embodiment of the present disclosure
- FIG. 3 shows a schematic structural diagram of a device for point cloud labeling provided by an embodiment of the present disclosure
- FIG. 4 shows a schematic structural diagram of an electronic device provided by an embodiment of the present disclosure.
- the present disclosure provides at least one point cloud labeling solution, which realizes the determination of 3D labeling frame information based on the frame selection operation of the 2D bird's-eye view, which is simple to operate, and has high labeling efficiency and accuracy.
- Lidar is widely used in today's autonomous driving systems. It can provide accurate three-dimensional environmental spatial information to assist the system's scene understanding and decision-making process.
- many deep learning-based perceptual algorithms have emerged to process such data, and these algorithms are significantly superior to monocular and stereo vision methods in application.
- the rapid development of these algorithms is supported by several challenging benchmarks built on multiple open datasets.
- actual product deployment still requires more accurately labeled data to feed algorithms.
- publicly available annotation tools are crude, which limits research progress in this field.
- Equipment the electronic equipment for example includes: terminal equipment or server or other processing equipment
- the terminal equipment can be user equipment (User Equipment, UE), mobile equipment, user terminal, terminal, cellular phone, cordless phone, Personal Digital Processing (Personal Digital Assistant, PDA), handheld devices, computing devices, in-vehicle devices, wearable devices, etc.
- the point cloud labeling method can be implemented by the processor calling computer-readable instructions stored in the memory.
- the following describes the point cloud labeling method provided by the embodiment of the present disclosure by taking the execution subject as a terminal device as an example.
- Fig. 1 shows a schematic flowchart of a point cloud labeling method provided by an embodiment of the present disclosure.
- the method includes steps S101 to S105, and the method can be applied to a point cloud labeling device or a point cloud labeling device , the point cloud labeling device may be a processor or a chip, and the processor or chip may be used in the point cloud labeling device, wherein the method includes:
- the point cloud labeling method provided by the embodiments of the present disclosure can be applied to any scene that requires 3D labeling of the object to be labelled. Based on the 3D labeling frame information after 3D labeling, training related to target detection and following models can be implemented, and can also be applied In other related application scenarios that require 3D annotation frame information, for example, it can be applied to automatic driving vehicles to identify the information of vehicles ahead to avoid traffic hazards such as vehicle collisions. In addition, it can also be applied to other application scenarios. , there is no restriction here.
- the related technology can be manually operated in three-dimensional space for labeling, and because the three-dimensional point cloud data has more labeling degrees of freedom, for example, the point cloud of an object can be marked with its corresponding height and rotation. Angle and other information, the operation complexity is relatively high. Considering that for sparse and irregularly distributed point cloud data, it is difficult to correctly identify the object of interest, which increases the difficulty of labeling to a certain extent and reduces the efficiency of labeling.
- the embodiments of the present disclosure provide a point cloud labeling method, which realizes the determination of 3D labeling frame information based on the frame selection operation of the 2D bird's-eye view.
- the operation is simple, and the labeling efficiency and accuracy are high. .
- the three-dimensional point cloud data to be labeled in the embodiment of the present disclosure may be point cloud data collected by a radar device.
- the radar device here may use a rotary scanning laser radar, or other radar devices, which are not specifically limited.
- the lidar can obtain the relevant three-dimensional point cloud data in the surrounding environment when it rotates and scans in the horizontal direction.
- the lidar in the process of rotating scanning, can use a multi-line scanning method, that is, the emission uses multiple laser tubes to emit sequentially, and the structure is that multiple laser tubes are arranged longitudinally, that is, in the process of rotating and scanning in the horizontal direction, vertical scanning is performed. Orientation of multi-layer scanning.
- each laser tube There is a certain angle between each laser tube, and the vertical emission field can be 30° to 40°. In this way, a data packet returned by multiple laser tubes can be obtained for each rotation of a scanning angle of the lidar device.
- One frame of point cloud data can be obtained by splicing the data packets obtained from each scanning angle (corresponding to a 360° scan of one rotation), and one frame of point cloud data can be collected after one scan of the lidar.
- 3D point cloud data to be marked in the embodiments of the present disclosure may refer to a frame of point cloud data, a half-frame point cloud data, or point cloud data determined by other scanning ranges. This does not make specific restrictions.
- the determination of the three-dimensional labeling frame information corresponding to the to-be-labeled object may be implemented based on the frame selection operation of the to-be-labeled object in the two-dimensional bird's-eye view.
- the three-dimensional point cloud data to be marked can be converted to the bird's-eye view, and the bird's-eye view can be is the mapping result of the three-dimensional point cloud data on the top view, which can correspond to the first two-dimensional point cloud data.
- frame selection can be performed for the object to be marked in the bird's eye view. operation, so as to determine the corresponding first two-dimensional callout frame information.
- Initial parameter information where the initial parameter information may be initial height information, or may be initial pose information, or may be a combination of initial height information and initial pose information, which is not specifically limited in this embodiment of the present disclosure .
- the relevant initial height information may be determined by the difference between the coordinate information of the highest point cloud point and the coordinate information of the lowest point cloud point of the 3D point cloud data corresponding to the frame selection, and the relevant initial pose information may include the initial Position information and initial attitude information.
- the initial position information can be determined based on the relative position of the 3D point cloud data selected by the frame in the 3D point cloud data to be marked, and the initial attitude information can be based on the two sides corresponding to the frame selection. The angle between the point cloud point set and the reference direction is determined.
- the 3D annotation frame information of the to-be-annotated object can be determined.
- the 3D annotation frame information here can be the initial 3D annotation frame information. Adjust the frame to synchronize the adjustment.
- a corresponding bird's-eye view can be generated first, so that after responding to the frame selection operation for the objects to be labeled in the bird's-eye view, the first
- the two-dimensional labeling frame information and at the same time, the initial parameter information of the to-be-labeled object can also be determined in combination with the 3D point cloud data of the frame-selected object to be labeled.
- Information about the 3D callout box corresponding to the object is a frame selection operation performed in a two-dimensional view (corresponding to a bird's-eye view).
- the complexity of performing the frame selection operation under the view is low, and the magnified view of the local point cloud facilitates the user to accurately identify the boundary of the labeling box, which can ensure the accuracy of the framed object to be labelled to a certain extent.
- the obtained first two-dimensional labeling frame information and initial parameter information can realize automatic labeling of the three-dimensional labeling frame for the object to be labelled, and the labeling efficiency and accuracy are high.
- the 3D annotation frame indicated by the 3D annotation frame information has different information on the object to be annotated observed in different views, for example, for the top view, what is observed may be the length and width of the object to be annotated, and for the front view. In other words, what is observed can be the width and height of the object to be annotated. Therefore, in order to further improve the accuracy of the annotation, the 2D annotation frame information in other views can be combined to determine the 3D annotation frame information, for example, the following steps can be included:
- Step 1 Generate a view to be adjusted corresponding to the three-dimensional point cloud data to be labeled based on the three-dimensional point cloud data of the object to be labeled; the view to be adjusted includes at least one of a front view and a side view;
- Step 2 According to the generated view to be adjusted, determine the second two-dimensional labeling frame information of the object to be labeled in the view to be adjusted.
- the view to be adjusted may be generated before the second two-dimensional frame information in the view to be adjusted is determined.
- the second two-dimensional annotation frame information here may be a mapping result of the three-dimensional point cloud data corresponding to the first two-dimensional annotation frame information on the view to be adjusted.
- the view to be adjusted in the embodiment of the present disclosure may be a front view, a side view, or a combination of the above two types of views.
- the generation process of the front view and the side view of the views to be adjusted may refer to the above-mentioned generation process of the two-dimensional bird's-eye view, that is, by using the conversion relationship between the corresponding coordinate systems.
- the front view and the side view waiting to be adjusted can be understood as different perspectives for observing the three-dimensional view, and the three-dimensional view can be more accurately analyzed under different perspectives to a certain extent.
- Download the 3D annotation frame information corresponding to the object to be annotated may be determined based on the first 2D annotation frame information, the second 2D annotation frame information, and the initial parameter information.
- the view to be adjusted and the bird's-eye view are two different 2D views that belong to the same 3D view where the 3D point cloud data is located, there is a correlation of parameter information between the two different 2D views.
- more accurate three-dimensional annotation frame information can be determined to a certain extent by combining the second two-dimensional annotation frame information determined by the view to be adjusted and the parameter information determined by the first two-dimensional annotation frame information determined by the bird's-eye view.
- determining the 3D annotation frame information corresponding to the object to be annotated based on the first 2D annotation frame information, the second 2D annotation frame information and the initial parameter information can be achieved by the following steps:
- Step 1 In response to the adjustment operation for the second two-dimensional labeling frame information, based on the initial parameter information, determine the adjusted parameter information of the object to be labelled;
- Step 2 Based on the first two-dimensional annotation frame information, the second two-dimensional annotation frame information, and the adjusted parameter information, determine the three-dimensional annotation frame information corresponding to the object to be annotated.
- the initial parameter information can be adjusted based on the adjustment operation for the second two-dimensional annotation frame information of the object to be labeled in the view to be adjusted.
- the corresponding first 2D annotation frame information and the second 2D annotation frame information corresponding to the view to be adjusted are used to determine the 3D annotation frame information corresponding to the object to be annotated. This may be due to the fact that different views focus on different information about the object to be marked. For example, for the bird's-eye view, the corresponding display may be the length and width of the object to be marked, and for the front view, the corresponding display The display can be the width and height of the object to be marked. Therefore, on the basis of the initial parameter information, parameter adjustment is performed in combination with the view to be adjusted, which will further improve the accuracy of the determined 3D annotation frame information.
- determining the adjusted parameter information of the object to be annotated based on the initial parameter information in response to the adjustment operation for the second two-dimensional annotation frame information includes: responding to the adjustment operation for the to-be-annotated object. Adjusting the adjustment operation of the second two-dimensional callout frame in the view to determine parameter adjustment information corresponding to the adjustment operation; and adjusting the initial parameter information based on the parameter adjustment information to obtain the adjusted parameter information.
- an adjustment operation can be performed on the information of the second two-dimensional annotation frame, that is, the initial parameter information can be adjusted on the basis of the initial parameter information to obtain the adjusted parameter information of the object to be annotated.
- the updated 3D annotation frame information can be obtained from the 2D annotation frame information and the second 2D annotation frame information.
- the first two-dimensional annotation frame information may also be adjusted correspondingly.
- the parameter of height information is adjusted for the second 2D annotation frame information
- the corresponding The height information of the first 2D annotation frame information can also change accordingly, and the adjustment in the 2D view can be adaptively mapped to the 3D annotation frame indicated by the 3D annotation frame information.
- the view to be adjusted may be displayed first.
- the second two-dimensional frame marked on the view can be adjusted.
- parameter adjustment information corresponding to the adjustment operation may be determined. For example, after dragging the second two-dimensional callout box to stretch outward, the parameter adjustment information can be determined according to the stretching degree. The greater the stretching degree, the greater the value of the parameter adjustment information. The parameter information can determine the adjusted parameter information.
- determine the At least one of the parameter information of the side and front of the 3D annotation frame corresponding to the annotation object so that the determined parameter information of the bottom surface of the 3D annotation frame and at least one of the parameter information of the side and front of the 3D annotation frame Combining them, you can get the 3D annotation frame information.
- the 2D point cloud data corresponding to the first 2D annotation frame information, and the 2D points corresponding to the second 2D annotation frame information determines the parameter information of the 3D annotation frame on multiple planes, and then determines the information of the 3D annotation frame through the combination of information.
- the following step may be further performed: based on the determined second two-dimensional annotation of the object to be annotated Dimensional annotation frame information, displaying the to-be-adjusted view marked with a second 2D annotation frame corresponding to the second 2D annotation frame information.
- the method for point cloud labeling provided by the embodiment of the present disclosure can be combined with a check image to check the labelled 3D label frame information, and can be implemented by the following steps:
- Step 1 Obtain a verification image; the verification image and the 3D point cloud data to be marked originate from the same scene;
- Step 2 Based on the mapping relationship between the coordinate system corresponding to the three-dimensional point cloud data to be marked and the coordinate system corresponding to the verification image, map the three-dimensional marked frame information corresponding to the object to be marked into the verification image, and determine the mapped image. image coordinate range;
- Step 3 Verify the accuracy of the three-dimensional annotation frame information according to the mapped image coordinate range and the image coordinate range where the object to be marked in the verification image is located.
- the accuracy of the 3D annotation frame information can be verified based on the obtained verification image, which may depend on the corresponding 3D point cloud data to be marked.
- the mapping relationship between the coordinate system of the image and the coordinate system corresponding to the verification image is realized by mapping the image coordinate range, so that the specific object to be marked can be verified more accurately.
- the verification image here may be a related image from the same scene as the 3D point cloud data to be marked, for example, an RGB image.
- the verification image here may be collected by a camera device installed on the same vehicle as the radar device, and the camera range of the camera device may be consistent with or overlap with the scan range of the radar device.
- the 3D labeling frame information corresponding to the object to be labelled in the world coordinate system can be mapped to the image coordinate system to obtain
- the mapped image coordinate range in the case of determining the image coordinate range where the object to be marked in the verification image is located, can be based on the matching result between the image coordinate range and the mapped image coordinate range. Check for accuracy.
- the point cloud labeling method can respond to the adjustment operation of the second two-dimensional labeling frame in the view to be adjusted corresponding to the to-be-labeled three-dimensional point cloud data when the two coordinate ranges are inconsistent.
- the adjusted parameter information is adjusted until the image coordinate range after mapping is consistent with the image coordinate range where the object to be marked in the verification image is located. Adjusted to further improve the accuracy of the marked 3D annotation frame information.
- the point cloud labeling method provided by the embodiment of the present disclosure can not only realize the calibration of the three-dimensional labeling frame information based on the verification image, but also provide a reference for the frame selection operation of the object to be labelled in the bird's-eye view based on the reference image, so as to determine the first and second Dimensional callout box information.
- the reference image and the verification image here may be the same image, and the above description may be referred to for the process of collecting the reference image and the like.
- the image position information of the object to be labeled in the reference image can be converted into the reference position information of the object to be labeled in the bird's-eye view, depending on Due to the reference position information, users can perform the frame selection operation of the objects to be marked in the bird's-eye view, especially for sparse 3D point clouds, the reference position information provided by the reference image can greatly improve the quality of the objects to be marked in the bird's-eye view. accuracy.
- the embodiments of the present disclosure provide an inter-object transfer mechanism to realize the 3D annotation of unlabeled objects based on the 3D annotation frame information of the labeled objects, which can be implemented according to the following steps:
- Step 1 Display the 3D labeling frame information corresponding to the object to be labelled in the 3D view corresponding to the 3D point cloud data to be labelled, and use the object to be labelled as the labelled object;
- Step 2 According to the 3D annotation frame information of the marked object, determine the 3D annotation frame information of other objects to be marked in the 3D view; the other objects to be marked belong to the same object type as the marked object;
- Step 3 Display the 3D labeling frame information of other objects to be labelled in the 3D view.
- the object to be annotated can be regarded as a marked object.
- the annotated object belongs to the 3D annotation frame information of other objects to be annotated of the same object type, and the 3D annotation frame information of other objects to be annotated can be displayed in the 3D view.
- Step 1 in response to the copy instruction for the 3D annotation frame of the marked object in the 3D view, copy the 3D annotation frame information of the marked object;
- Step 2 In response to the pasting instructions for other objects to be marked in the 3D view, according to the 3D annotation frame information of the marked objects in the 3D view, perform 3D annotation on the other objects to be annotated, and obtain the 3D annotation frame information of the other objects to be annotated. .
- the 3D annotation frame of the marked object can be copied based on the copy operation of the 3D annotation frame of the marked object.
- the information is pasted to other objects to be labeled that belong to the same object type, so that 3D labeling of other objects to be labeled can be quickly realized, and the labeling efficiency is further improved.
- the 3D annotation frame information of the annotated object is copied by copying the 3D annotation frame of the annotated object.
- the 3D annotation corresponding to the annotated object can be copied according to the
- the frame information performs 3D labeling on other objects to be labelled to obtain corresponding 3D labeling frame information.
- the three-dimensional labeling of each vehicle on the road can be implemented based on the above-mentioned inter-object transfer mechanism.
- the farther the object to be labeled, the sparser the corresponding 3D point cloud data, and the lower the corresponding data integrity, and the object to be labeled with higher data integrity will be more accurate to a certain extent than the object to be labeled with lower data integrity. Easy to label.
- the embodiment of the present disclosure can perform 3D annotation of the same object type according to the annotation method of distance, which can significantly reduce the adjustment of pose and height and other information during annotation, so as to ensure that one frame of 3D point cloud data to be annotated On the premise of the accuracy of labeling, the labeling efficiency is improved.
- the embodiments of the present disclosure provide an inter-frame transfer mechanism to realize the 3D labeling of unlabeled objects based on the 3D labeling frame information of the labeled objects, which can be implemented according to the following steps:
- Step 1 Display the 3D labeling frame information corresponding to the object to be labelled in the first 3D view corresponding to the 3D point cloud data to be labelled in the current frame, and use the object to be labelled as the labelled object; wherein, the 3D points to be labelled in different frames Cloud data corresponds to different 3D views;
- Step 2 in response to the copy instruction for the 3D annotation frame of the marked object in the first 3D view, copy the 3D annotation frame information of the marked object;
- Step 3 In the case of switching to the second 3D view corresponding to the 3D point cloud data to be labeled in the next frame, in response to the paste instruction for other objects to be labeled in the second 3D view, according to the labeled objects in the first 3D view.
- the 3D labeling frame information in the second 3D view is 3D labelled for other objects to be labelled in the second 3D view to obtain the 3D labelling frame information of the other objects to be labelled.
- the object to be annotated can be regarded as the marked object.
- the marked object can be copied through the operation of copying the 3D annotation frame of the marked object. Copy the 3D annotation frame information of the object.
- the 3D annotation frame corresponding to the marked object can be copied.
- the information is three-dimensional marked on other objects to be marked to obtain the corresponding three-dimensional marked frame information.
- the pairing operation based on the above copy-paste can be used to realize the next frame of the 3D point cloud data to be labeled. Annotation of the object to be annotated.
- the to-be-labeled objects of the labeled adjacent frames may be the same object or may not be the same object.
- it can also be combined with the target tracking or target detection mechanism in the case of switching to the 3D view where the 3D point cloud data to be marked in the next frame is located. Automatically label the objects to be labeled that belong to the same object as the labeled object in the previous frame in the 3D point cloud data to be labeled in the next frame.
- an annotation transfer mechanism includes an inter-object transfer mechanism. mechanism and inter-frame transfer mechanism.
- the inter-object transfer mechanism can be: in a single frame, annotations can be transferred between objects from the same class, especially on the same lane.
- the inter-frame transfer mechanism can be that, in successive frames, annotations from previously marked frames can be transferred. Both transfer mechanisms can significantly reduce the adjustment of rotation angle and height during labeling, avoiding missing or inaccurate labels in sparse regions of the point cloud.
- the inter-frame transfer mechanism can also realize one-to-one correspondence between the labels of consecutive frames, so that labels can be used for both 3D object detection and 3D object tracking.
- the inter-object transfer mechanism and the inter-frame transfer mechanism through the inter-object transfer mechanism and the inter-frame transfer mechanism, most 3D labeling tasks can be simplified into labeling adjustments for bird's-eye views, which greatly reduces unnecessary operations and has little impact on the load of electronic devices. The labeling efficiency and accuracy are greatly improved.
- determining the first two-dimensional annotation frame information in response to the frame selection operation of the object to be annotated in the bird's-eye view in S103 may be implemented by: acquiring a reference image; The three-dimensional point cloud data to be marked comes from the same scene; based on the image position information of the object to be marked in the reference image and the relationship between the coordinate system corresponding to the reference image and the coordinate system corresponding to the bird's-eye view mapping relationship, determine the reference position information of the object to be marked in the bird's-eye view; in response to the frame selection operation for the object to be marked in the bird's-eye view, determine the first and second based on the reference position information Dimensional callout box information.
- the reference position information of the object to be marked in the bird's-eye view can be determined based on the obtained reference image, and based on the reference position information, the user is guided to perform a frame selection operation for the object to be marked in the bird's-eye view.
- the first two-dimensional labeling frame selected by the frame will cover the point cloud data corresponding to the object to be labelled to a greater extent, thereby improving the labeling accuracy.
- the point cloud labeling method provided by the embodiments of the present disclosure can realize view display in combination with a user interface.
- the corresponding view switching button can be set on the user interaction interface to realize the switching between the bird's-eye view and the three-dimensional view mode.
- the first 2D annotation frame information is determined, and in the view of switching to the 3D view, the marked 3D annotation frame information can be displayed.
- the embodiment of the present disclosure can also display the corresponding view to be adjusted on the user interface, and in the view to be adjusted, the second two-dimensional annotation frame can be adjusted, the information of the second two-dimensional annotation frame can be determined, and The initial parameter information of the object to be annotated is adjusted.
- This adjustment operation can be synchronized under the bird's-eye view and the three-dimensional view, which provides a guarantee for fast annotation.
- the local point cloud that needs to be operated can be highlighted.
- the point cloud corresponding to the frame selection can be highlighted on the bird's-eye view displayed on the user interface, and the point cloud corresponding to the view to be adjusted can also be highlighted simultaneously, so as to facilitate the adjustment operation. .
- the embodiments of the present disclosure may also mark relevant length, width and other information for each markup frame.
- a lightweight WAN-based labeling tool can be used.
- the labeling tool can be less dependent on the performance of the computer, so that it can run smoothly on ordinary computers. .
- the method for point cloud labeling involves complex interactive operations, in order to effectively realize the two technical purposes of correctly identifying objects and accurately labeling objects, in some applications, it can be divided into four steps, namely, Finding, locating, adjusting and verifying, the first two steps can be to identify and locate objects in the global view, the third step is to label accurately, and the last step is to ensure that all labels are accurate enough.
- Finding, locating, adjusting and verifying the first two steps can be to identify and locate objects in the global view
- the third step is to label accurately
- the last step is to ensure that all labels are accurate enough.
- Each stage focuses on the most important tasks, further ensuring the accuracy of the annotation results.
- the shape information reflected by the 3D point cloud data and the semantic information in the RGB image can be combined to find the target object. Then, the bounding box is drawn in the bird's-eye view corresponding to the 3D point cloud data, and the height information is generated according to the local point cloud for preliminary positioning. Among them, using the mapping relationship between the coordinate system corresponding to the 3D point cloud data to be marked and the coordinate system corresponding to the RGB image, the 3D pyramid obtained from the 2D image can be used to estimate which points to search for belong to the target object.
- FIG. 2 shows a schematic flowchart of a point cloud labeling provided by an embodiment of the present disclosure.
- the three-dimensional point cloud data corresponds to the RGB image
- the four steps (a), (b), (c) and (d) can be processed in turn for the RGB image and the bird's-eye view. These four steps correspond to search, Locate, adjust and verify.
- the position of the closer object (the closer object, the closer object occupies a larger size in the RGB image) can be directly determined from the bird's eye view in the bird's eye view, and the farther
- the position of the object (the farther object, the farther object occupies a smaller size in the RGB image) is not easy to determine in the bird's eye view, so the position of the farther object in the RGB image can be determined, based on the RGB image.
- the position of distant objects which determines the position of distant objects in the bird's eye view.
- the positions of the two objects when the positions of the two objects are easy to be determined in the bird's eye view, for example, the two objects are close, the positions of the two objects can be directly determined from the bird's eye view, or, in the case of the two objects
- the positions of the two objects in the bird's eye view can be determined based on the positions of the two objects in the RGB image.
- the position of the target After obtaining the position of the target in the bird's-eye view, the position of the target can be located in (b) to obtain the above-mentioned first two-dimensional labeling frame information.
- the bird's-eye view shown in (a) and the bird's-eye view shown in (b) are bird's-eye views from different viewing angles. After that, perform the conversion between (b) and (c), obtain the position of the target in the bird's-eye view through (b), and adjust the size of the two sides of the target.
- the two sides can be the front and the side respectively. , any two of the top-view planes, so as to obtain two side dimensions corresponding to each target, as shown in (c). In other embodiments, the dimensions of the three sides may be adjusted.
- the 3D annotation frame information of a certain object can be verified. For example, in (d), the 3D annotation frame information of the closer object in the RGB image can be determined according to the two side dimensions of the closer object.
- light-weight web-based annotation tools can be used to realize annotation, and the above annotation tools can run smoothly on ordinary computers. This design is very user-friendly and provides the possibility to perform in the order of each stage.
- the writing order of each step does not mean a strict execution order and constitutes any limitation on the implementation process, and the execution order of each step should be based on its function and possible inherent Logical OK.
- the embodiment of the present disclosure also provides a point cloud labeling device corresponding to the point cloud labeling method, because the principle of solving the problem by the device in the embodiment of the present disclosure is the same as the above-mentioned point cloud labeling method in the embodiment of the present disclosure. Similar, so the implementation of the device can refer to the implementation of the method, and the repeated places will not be described in detail.
- a systematic labeling process is proposed to solve two key problems in 3D labeling tasks: correctly identifying objects and accurately labeling objects.
- it provides a clear user interface and annotation delivery mechanism, which makes it easier for annotators to concentrate on simpler tasks at each stage of annotation and complete them with fewer operations.
- FIG. 3 shows a schematic structural diagram of a point cloud labeling device provided by an embodiment of the present disclosure.
- the device includes: an acquisition module 201 , a generation module 202 , a first determination module 203 , and a labeling module 204 ; in,
- the obtaining module 201 is configured to obtain the three-dimensional point cloud data to be marked
- the generating module 202 is configured to generate a bird's-eye view corresponding to the three-dimensional point cloud data to be marked based on the three-dimensional point cloud data to be marked;
- the first determination module 203 is configured to determine the first two-dimensional annotation frame information in response to the frame selection operation for the object to be marked in the bird's-eye view; initial parameter information;
- the labeling module 204 is configured to determine the three-dimensional labeling frame information corresponding to the object to be labelled based on the first two-dimensional labeling frame information and the initial parameter information.
- the frame selection operation performed in the embodiment of the present disclosure in a two-dimensional view compared with the problem of high operational complexity in directly annotating 3D point cloud data in the related art, performing frame selection in a two-dimensional view
- the complexity of the operation is low, and to a certain extent, the accuracy of the selected objects to be marked can be ensured.
- the first two-dimensional marked frame information determined based on the frame selection can be combined with the determined initial parameter information.
- the object is automatically marked with a 3D labeling frame, and the labeling efficiency and accuracy are high.
- the above device further includes:
- the second determination module 205 is configured to generate, based on the three-dimensional point cloud data of the object to be marked, a view to be adjusted corresponding to the three-dimensional point cloud data to be marked; the view to be adjusted includes at least one of a front view and a side view; Adjusting the view, and determining the second two-dimensional labeling frame information of the object to be labelled in the view to be adjusted.
- the labeling module 204 is configured to determine the three-dimensional labeling frame information corresponding to the object to be labelled based on the first two-dimensional labeling frame information and the initial parameter information according to the following steps:
- the 3D annotation frame information corresponding to the object to be annotated is determined.
- the labeling module 204 is configured to determine the three-dimensional labeling frame information corresponding to the object to be labelled based on the first two-dimensional labeling frame information, the second two-dimensional labeling frame information and the initial parameter information, including:
- the 3D annotation frame information corresponding to the object to be annotated is determined.
- the labeling module 204 is configured to determine the adjusted parameter information of the object to be labelled according to the following steps:
- the initial parameter information is adjusted based on the parameter adjustment information to obtain adjusted parameter information.
- the labeling module 204 is configured to determine the 3D labeling frame corresponding to the object to be labelled based on the first 2D labeling frame information, the second 2D labeling frame information and the adjusted parameter information according to the following steps information:
- the second determining module 205 is further configured to:
- the above device further includes:
- the verification module 206 is configured to obtain a verification image after determining the three-dimensional annotation frame information corresponding to the object to be marked; the verification image and the three-dimensional point cloud data to be marked originate from the same scene; based on the coordinates corresponding to the three-dimensional point cloud data to be marked system, the mapping relationship between the coordinate system corresponding to the verification image, map the three-dimensional annotation frame information corresponding to the object to be marked into the verification image, and determine the coordinate range of the image after mapping; The image coordinate range where the object to be annotated in the verification image is located, and the accuracy of the three-dimensional annotation frame information is verified.
- the verification module 206 is further configured to:
- the coordinate range of the image after mapping and the object to be marked in the verification image are verified.
- the image coordinate ranges are inconsistent
- in response to the adjustment operation for the second two-dimensional annotation frame in the to-be-adjusted view corresponding to the three-dimensional point cloud data to be marked adjust the adjusted parameter information until the mapped image coordinate range It is consistent with the image coordinate range where the object to be marked in the verification image is located; the adjusted parameter information is obtained by adjusting the initial parameter information.
- the labeling module 204 is further configured to:
- the 3D annotation frame information corresponding to the object to be annotated After determining the 3D annotation frame information corresponding to the object to be annotated, display the 3D annotation frame information corresponding to the object to be annotated in the 3D view corresponding to the 3D point cloud data to be annotated, and use the object to be annotated as an annotated object;
- the 3D annotation frame information of the object determines the 3D annotation frame information of other objects to be annotated in the 3D view; other objects to be annotated and the marked object belong to the same object type; the 3D annotation frame information of other objects to be annotated is displayed in the 3D view .
- the labeling module 204 is configured to determine the 3D labeling frame information of other objects to be labelled in the 3D view according to the 3D labeling frame information of the labelled object:
- copy the 3D annotation frame information of the marked object In response to the copy instruction for the 3D annotation frame of the marked object in the 3D view, copy the 3D annotation frame information of the marked object; in response to the paste instruction for other objects to be marked in the 3D view, according to the marked object in the 3D view.
- 3D labeling frame information perform 3D labeling on other objects to be labelled, and obtain 3D labeling frame information of other objects to be labelled.
- the labeling module 204 is further configured to:
- the 3D annotation frame information corresponding to the object to be annotated After determining the 3D annotation frame information corresponding to the object to be annotated, display the 3D annotation frame information corresponding to the object to be annotated in the first 3D view corresponding to the 3D point cloud data to be annotated in the current frame, and use the object to be annotated as the marked object ; wherein, the three-dimensional point cloud data to be marked in different frames corresponds to different three-dimensional views; in response to the copy instruction for the three-dimensional marked frame of the marked object in the first three-dimensional view, copy the three-dimensional marked frame information of the marked object; after switching to the next In the case of a second 3D view corresponding to one frame of point cloud data to be annotated, in response to the pasting instruction for other objects to be annotated in the second 3D view, according to the 3D annotation frame information of the annotated object in the first 3D view, The other objects to be marked in the second three-dimensional view are three-dimensionally marked, and the three-dimensional
- the first determining module 203 is configured to respond to a frame selection operation for the object to be marked in the bird's-eye view according to the following steps, and determine the first two-dimensional frame information:
- the first two-dimensional frame information is determined based on the reference position information.
- the initial parameter information includes at least one of initial height information and initial pose information.
- FIG. 4 shows a schematic structural diagram of an electronic device provided by an embodiment of the present disclosure.
- the electronic device provided by an embodiment of the present disclosure is a schematic structural diagram, including: a processor 301 , a memory 302 , and a bus 303 .
- the memory 302 stores machine-readable instructions executable by the processor 301 (for example, the execution instructions corresponding to the acquisition module 201, the generation module 202, the first determination module 203, and the labeling module 204 in the device for point cloud labeling in FIG. 3, etc.) , when the electronic device is running, the processor 301 communicates with the memory 302 through the bus 303, and the machine-readable instructions are executed by the processor 301 to perform the following processing:
- the three-dimensional labeling frame information corresponding to the object to be labelled is determined.
- Embodiments of the present disclosure further provide a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is run by a processor, the steps of the point cloud labeling method described in the foregoing method embodiments are executed.
- the storage medium may be a volatile or non-volatile computer-readable storage medium.
- An embodiment of the present disclosure further provides a computer program product, including a computer-readable storage medium storing program codes, and the instructions included in the program codes can be configured to execute the steps of the point cloud labeling method described in the above method embodiments , for details, refer to the above method embodiments.
- Embodiments of the present disclosure further provide a computer program, which implements any one of the methods in the foregoing embodiments when the computer program is executed by a processor.
- the computer program product can be specifically implemented by hardware, software or a combination thereof.
- the computer program product is embodied as a computer storage medium, and in another optional embodiment, the computer program product is embodied as a software product, such as a software development kit (Software Development Kit, SDK), etc. Wait.
- the units described as separate components may or may not be physically separated, and components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution in this embodiment.
- each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, or each unit may exist physically alone, or two or more units may be integrated into one unit.
- the functions, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a processor-executable non-volatile computer-readable storage medium.
- the computer software products are stored in a storage medium, including Several instructions are used to cause an electronic device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present disclosure.
- the aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disk or optical disk and other media that can store program codes .
- the first two-dimensional labeling frame information is determined by responding to a frame selection operation for the object to be labelled in the bird's-eye view corresponding to the 3D point cloud data to be labelled; Based on the cloud data, the initial parameter information of the object to be marked is determined; based on the first two-dimensional marked frame information and the initial parameter information, the three-dimensional marked frame information corresponding to the to-be-marked object is determined, so that the frame selection can be performed.
- the obtained first two-dimensional labeling frame information and initial parameter information realize automatic labeling of the three-dimensional labeling frame for the object to be labelled, and the labeling efficiency and accuracy are high.
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Multimedia (AREA)
- Remote Sensing (AREA)
- Radar, Positioning & Navigation (AREA)
- Electromagnetism (AREA)
- Computer Networks & Wireless Communication (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Computational Linguistics (AREA)
- Medical Informatics (AREA)
- Evolutionary Computation (AREA)
- Databases & Information Systems (AREA)
- Computing Systems (AREA)
- Artificial Intelligence (AREA)
- General Health & Medical Sciences (AREA)
- Architecture (AREA)
- Computer Graphics (AREA)
- Computer Hardware Design (AREA)
- General Engineering & Computer Science (AREA)
- Processing Or Creating Images (AREA)
- Length Measuring Devices By Optical Means (AREA)
Abstract
Description
Claims (31)
- 一种点云标注的方法,所述方法包括:A method for point cloud labeling, the method comprising:获取待标注三维点云数据;Obtain the 3D point cloud data to be labeled;基于所述待标注三维点云数据,生成所述待标注三维点云数据对应的鸟瞰图;generating a bird's-eye view corresponding to the three-dimensional point cloud data to be marked based on the three-dimensional point cloud data to be marked;响应针对所述鸟瞰图中的待标注对象的框选操作,确定第一二维标注框信息;In response to the frame selection operation for the object to be marked in the bird's-eye view, determine the first two-dimensional frame information;基于被框选的待标注对象的三维点云数据,确定所述待标注对象的初始参数信息;Determine the initial parameter information of the object to be marked based on the three-dimensional point cloud data of the object to be marked selected by the frame;基于所述第一二维标注框信息,以及所述初始参数信息,确定所述待标注对象对应的三维标注框信息。Based on the first two-dimensional annotation frame information and the initial parameter information, the three-dimensional annotation frame information corresponding to the object to be annotated is determined.
- 根据权利要求1所述的方法,其中,所述方法还包括:The method of claim 1, wherein the method further comprises:基于所述待标注对象的三维点云数据,生成所述待标注三维点云数据对应的待调整视图;所述待调整视图至少包括正视图和侧视图中的一种;Based on the three-dimensional point cloud data of the object to be marked, a view to be adjusted corresponding to the three-dimensional point cloud data to be marked is generated; the to-be-adjusted view includes at least one of a front view and a side view;根据生成的所述待调整视图,确定所述待标注对象在所述待调整视图中的第二二维标注框信息。According to the generated view to be adjusted, second two-dimensional labeling frame information of the to-be-labeled object in the to-be-adjusted view is determined.
- 根据权利要求2所述的方法,其中,所述基于所述第一二维标注框信息,以及所述初始参数信息,确定所述待标注对象对应的三维标注框信息,包括:The method according to claim 2, wherein the determining, based on the first 2D annotation frame information and the initial parameter information, the 3D annotation frame information corresponding to the object to be annotated comprises:基于所述第一二维标注框信息、所述第二二维标注框信息以及所述初始参数信息,确定所述待标注对象对应的三维标注框信息。Based on the first 2D annotation frame information, the second 2D annotation frame information and the initial parameter information, the 3D annotation frame information corresponding to the object to be annotated is determined.
- 根据权利要求3所述的方法,其中,所述基于所述第一二维标注框信息、所述第二二维标注框信息以及所述初始参数信息,确定所述待标注对象对应的三维标注框信息,包括:The method according to claim 3, wherein the 3D annotation corresponding to the object to be annotated is determined based on the first 2D annotation frame information, the second 2D annotation frame information and the initial parameter information Box information, including:响应针对所述第二二维标注框信息的调整操作,基于所述初始参数信息,确定所述待标注对象的调整后的参数信息;In response to the adjustment operation for the second two-dimensional annotation frame information, based on the initial parameter information, determine the adjusted parameter information of the object to be annotated;基于所述第一二维标注框信息、所述第二二维标注框信息以及所述调整后的参数信息,确定所述待标注对象对应的三维标注框信息。Based on the first 2D annotation frame information, the second 2D annotation frame information and the adjusted parameter information, the 3D annotation frame information corresponding to the object to be annotated is determined.
- 根据权利要求4所述的方法,其中,所述响应针对所述第二二维标注框信息的调整操作,基于所述初始参数信息,确定所述待标注对象的调整后的参数信息,包括:The method according to claim 4, wherein, in response to an adjustment operation for the second two-dimensional annotation frame information, determining the adjusted parameter information of the object to be annotated based on the initial parameter information, comprising:响应针对所述待调整视图中的第二二维标注框的调整操作,确定与所述调整操作对应的参数调整信息;determining parameter adjustment information corresponding to the adjustment operation in response to an adjustment operation for the second two-dimensional frame in the to-be-adjusted view;基于所述参数调整信息对所述初始参数信息进行调整,得到所述调整后的参数信息。The initial parameter information is adjusted based on the parameter adjustment information to obtain the adjusted parameter information.
- 根据权利要求4或5所述的方法,其中,所述基于所述第一二维标注框信息、所述第二二维标注框信息以及所述调整后的参数信息,确定所述待标注对象对应的三维标注框信息,包括:The method according to claim 4 or 5, wherein the object to be annotated is determined based on the first two-dimensional annotation frame information, the second two-dimensional annotation frame information and the adjusted parameter information Corresponding 3D callout box information, including:基于所述调整后的参数信息所对应的三维点云数据,与所述第一二维标注框信息所对应的二维点云数据之间的第一对应关系,确定所述待标注对象对应的三维标注框的底面的参数信息;Based on the first correspondence between the three-dimensional point cloud data corresponding to the adjusted parameter information and the two-dimensional point cloud data corresponding to the first two-dimensional labeling frame information, determine the corresponding to-be-labeled object. The parameter information of the bottom surface of the 3D callout box;基于所述调整后的参数信息所对应的三维点云数据,与所述第二二维标注框信息所对应的二维点云数据之间的第二对应关系,确定所述待标注对象对应的三维标注框的侧面和正面的参数信息中的至少之一;Based on the second correspondence between the three-dimensional point cloud data corresponding to the adjusted parameter information and the two-dimensional point cloud data corresponding to the second two-dimensional labeling frame information, determine the corresponding to-be-labeled object. At least one of the parameter information of the side and the front of the 3D callout frame;将确定的所述三维标注框的底面的参数信息、以及所述三维标注框的侧面和正 面的参数信息中的至少之一进行组合,得到所述三维标注框信息。Combining at least one of the determined parameter information of the bottom surface of the 3D annotation frame and the parameter information of the side surface and the front surface of the 3D annotation frame to obtain the 3D annotation frame information.
- 根据权利要求2至6任一所述的方法,其中,在所述确定所述待标注对象在所述待调整视图中的第二二维标注框信息之后,所述方法还包括:The method according to any one of claims 2 to 6, wherein after the determining the second two-dimensional annotation frame information of the object to be annotated in the view to be adjusted, the method further comprises:基于确定的所述待标注对象的第二二维标注框信息,展示标注有与所述第二二维标注框信息对应的第二二维标注框的所述待调整视图。Based on the determined second two-dimensional annotation frame information of the object to be annotated, the to-be-adjusted view marked with a second two-dimensional annotation frame corresponding to the second two-dimensional annotation frame information is displayed.
- 根据权利要求1至7任一所述的方法,其中,在所述确定所述待标注对象对应的三维标注框信息之后,所述方法还包括:The method according to any one of claims 1 to 7, wherein after the determining the three-dimensional annotation frame information corresponding to the object to be annotated, the method further comprises:获取校验图像;所述校验图像与所述待标注三维点云数据来源于同一场景;Obtaining a verification image; the verification image and the to-be-labeled 3D point cloud data originate from the same scene;基于所述待标注三维点云数据对应的坐标系,与所述校验图像对应的坐标系之间的映射关系,将所述待标注对象对应的三维标注框信息映射至所述校验图像中,确定映射后的图像坐标范围;Based on the mapping relationship between the coordinate system corresponding to the three-dimensional point cloud data to be marked and the coordinate system corresponding to the verification image, map the three-dimensional annotation frame information corresponding to the object to be marked into the verification image , determine the coordinate range of the image after mapping;根据所述映射后的图像坐标范围与所述校验图像中的所述待标注对象所在图像坐标范围,对所述三维标注框信息的准确性进行校验。The accuracy of the three-dimensional annotation frame information is verified according to the mapped image coordinate range and the image coordinate range in the verification image where the object to be marked is located.
- 根据权利要求8所述的方法,其中,在所述根据所述映射后的图像坐标范围与所述校验图像中的所述待标注对象所在图像坐标范围,对所述三维标注框信息的准确性进行校验之后,还包括:The method according to claim 8, wherein, according to the image coordinate range after the mapping and the image coordinate range where the object to be annotated in the verification image is located, the accuracy of the three-dimensional annotation frame information is accurate. After verification, it also includes:在所述映射后的图像坐标范围与所述校验图像中的所述待标注对象所在图像坐标范围不一致的情况下,响应针对所述待标注三维点云数据对应的待调整视图中的第二二维标注框的调整操作,对调整后的参数信息进行调整,直至所述映射后的图像坐标范围与所述校验图像中的所述待标注对象所在图像坐标范围一致;所述调整后的参数信息是通过对所述初始参数信息进行调整得到的。In the case where the mapped image coordinate range is inconsistent with the image coordinate range where the object to be marked in the verification image is located, respond to the second image coordinate range in the to-be-adjusted view corresponding to the three-dimensional point cloud data to be marked. The adjustment operation of the two-dimensional annotation frame is to adjust the adjusted parameter information until the image coordinate range after the mapping is consistent with the image coordinate range where the object to be annotated in the verification image is located; the adjusted image coordinate range is The parameter information is obtained by adjusting the initial parameter information.
- 根据权利要求1至9任一所述的方法,其中,所述确定所述待标注对象对应的三维标注框信息之后,还包括:The method according to any one of claims 1 to 9, wherein after the determining the three-dimensional labeling frame information corresponding to the object to be labelled, the method further comprises:将所述待标注对象对应的三维标注框信息,展示在所述待标注三维点云数据对应的三维视图中,并将所述待标注对象作为已标注对象;Displaying the 3D labeling frame information corresponding to the object to be labelled in the 3D view corresponding to the 3D point cloud data to be labelled, and using the object to be labelled as the labelled object;根据所述已标注对象的三维标注框信息,确定所述三维视图中的其它待标注对象的三维标注框信息;所述其它待标注对象与所述已标注对象属于同一对象类型;According to the 3D annotation frame information of the marked object, determine the 3D annotation frame information of other objects to be marked in the 3D view; the other objects to be marked and the marked object belong to the same object type;将所述其它待标注对象的三维标注框信息展示在所述三维视图中。The 3D annotation frame information of the other objects to be annotated is displayed in the 3D view.
- 根据权利要求10所述的方法,其中,所述根据所述已标注对象的三维标注框信息,确定所述三维视图中的其它待标注对象的三维标注框信息,包括:The method according to claim 10, wherein the determining, according to the 3D annotation frame information of the marked object, the 3D annotation frame information of other objects to be annotated in the 3D view comprises:响应于针对所述三维视图中已标注对象的三维标注框的复制指令,复制所述已标注对象的三维标注框信息;In response to a copy instruction for the three-dimensional call-out frame of the marked object in the three-dimensional view, copying the three-dimensional call-out frame information of the marked object;响应于针对所述三维视图中其它待标注对象的粘贴指令,根据所述已标注对象在所述三维视图中的三维标注框信息,对所述其它待标注对象进行三维标注,得到所述其它待标注对象的三维标注框信息。In response to the paste instruction for other objects to be marked in the three-dimensional view, perform three-dimensional marking on the other objects to be marked according to the three-dimensional marking frame information of the marked objects in the three-dimensional view, and obtain the other objects to be marked. The 3D callout box information of the callout object.
- 根据权利要求1至11任一所述的方法,其中,所述确定所述待标注对象对应的三维标注框信息之后,还包括:The method according to any one of claims 1 to 11, wherein after the determining the three-dimensional labeling frame information corresponding to the object to be labelled, the method further comprises:将所述待标注对象对应的三维标注框信息,展示在当前帧待标注三维点云数据对应的第一三维视图中,并将所述待标注对象作为已标注对象;其中,不同帧待标注三维点云数据对应不同的三维视图;Display the 3D labeling frame information corresponding to the object to be labelled in the first 3D view corresponding to the 3D point cloud data to be labelled in the current frame, and use the object to be labelled as the labelled object; wherein, different frames to be labelled 3D view Point cloud data corresponds to different 3D views;响应于针对所述第一三维视图中所述已标注对象的三维标注框的复制指令,复制所述已标注对象的三维标注框信息;In response to a copy instruction for the 3D annotation frame of the annotated object in the first 3D view, copying the 3D annotation frame information of the annotated object;在切换到下一帧待标注三维点云数据对应的第二三维视图的情况下,响应于针对所述第二三维视图中其它待标注对象的粘贴指令,根据所述已标注对象在所述第一 三维视图中的三维标注框信息,对所述第二三维视图中的其它待标注对象进行三维标注,得到其它待标注对象的三维标注框信息。In the case of switching to the second 3D view corresponding to the 3D point cloud data to be labeled in the next frame, in response to the paste instruction for other objects to be labeled in the second 3D view, according to the labeled objects in the second 3D view 3D labeling frame information in a 3D view, perform 3D labeling on other objects to be labelled in the second 3D view, and obtain 3D labeling frame information of other objects to be labelled.
- 根据权利要求1至12任一所述的方法,其中,所述响应针对所述鸟瞰图中的待标注对象的框选操作,确定第一二维标注框信息,包括:The method according to any one of claims 1 to 12, wherein, in response to a frame selection operation on an object to be marked in the bird's-eye view, determining the first two-dimensional frame information includes:获取参考图像;所述参考图像与所述待标注三维点云数据来源于同一场景;obtaining a reference image; the reference image and the to-be-labeled 3D point cloud data originate from the same scene;基于所述待标注对象在所述参考图像中的图像位置信息、以及所述参考图像对应的坐标系与所述鸟瞰图对应的坐标系之间的映射关系,确定所述待标注对象在所述鸟瞰图中的参考位置信息;Based on the image position information of the object to be labeled in the reference image and the mapping relationship between the coordinate system corresponding to the reference image and the coordinate system corresponding to the bird's-eye view, it is determined that the object to be labeled is located in the reference image. Reference location information in the bird's eye view;响应针对所述鸟瞰图中的所述待标注对象的框选操作,基于所述参考位置信息确定所述第一二维标注框信息。In response to a frame selection operation on the object to be marked in the bird's-eye view, the first two-dimensional frame information is determined based on the reference position information.
- 根据权利要求1至13任一所述的方法,其中,所述初始参数信息包括初始高度信息和初始位姿信息中的至少之一。The method according to any one of claims 1 to 13, wherein the initial parameter information includes at least one of initial height information and initial pose information.
- 一种点云标注的装置,所述装置包括:A device for point cloud labeling, the device comprising:获取模块,配置为获取待标注三维点云数据;an acquisition module, configured to acquire the 3D point cloud data to be marked;生成模块,配置为基于所述待标注三维点云数据,生成所述待标注三维点云数据对应的鸟瞰图;a generating module, configured to generate a bird's-eye view corresponding to the three-dimensional point cloud data to be marked based on the three-dimensional point cloud data to be marked;第一确定模块,配置为响应针对所述鸟瞰图中的待标注对象的框选操作,确定第一二维标注框信息;基于被框选的待标注对象的三维点云数据,确定所述待标注对象的初始参数信息;A first determination module, configured to determine first two-dimensional labeling frame information in response to a frame selection operation for objects to be labelled in the bird's-eye view; The initial parameter information of the annotation object;标注模块,配置为基于所述第一二维标注框信息,以及所述初始参数信息,确定所述待标注对象对应的三维标注框信息。The labeling module is configured to determine the three-dimensional labeling frame information corresponding to the object to be labelled based on the first two-dimensional labeling frame information and the initial parameter information.
- 根据权利要求15所述的装置,其中,所述装置还包括:The apparatus of claim 15, wherein the apparatus further comprises:第二确定模块,配置为:基于所述待标注对象的三维点云数据,生成所述待标注三维点云数据对应的待调整视图;所述待调整视图至少包括正视图和侧视图中的一种;根据生成的所述待调整视图,确定所述待标注对象在所述待调整视图中的第二二维标注框信息。The second determination module is configured to: generate a view to be adjusted corresponding to the three-dimensional point cloud data to be labeled based on the three-dimensional point cloud data of the object to be labeled; the view to be adjusted includes at least one of a front view and a side view. Type; according to the generated view to be adjusted, determine the second two-dimensional annotation frame information of the to-be-annotated object in the to-be-adjusted view.
- 根据权利要求16所述的装置,其中,所述标注模块,还配置为:基于所述第一二维标注框信息、所述第二二维标注框信息以及所述初始参数信息,确定所述待标注对象对应的三维标注框信息。The apparatus according to claim 16, wherein the labeling module is further configured to: determine the Information about the 3D annotation frame corresponding to the object to be annotated.
- 根据权利要求17所述的装置,其中,所述标注模块,还配置为:响应针对所述第二二维标注框信息的调整操作,基于所述初始参数信息,确定所述待标注对象的调整后的参数信息;基于所述第一二维标注框信息、所述第二二维标注框信息以及所述调整后的参数信息,确定所述待标注对象对应的三维标注框信息。The apparatus according to claim 17, wherein the labeling module is further configured to: in response to an adjustment operation for the second two-dimensional labeling frame information, determine the adjustment of the object to be labelled based on the initial parameter information and based on the first 2D annotation frame information, the second 2D annotation frame information and the adjusted parameter information, determine the 3D annotation frame information corresponding to the object to be annotated.
- 根据权利要求18所述的装置,其中,所述标注模块,还配置为:响应针对所述待调整视图中的第二二维标注框的调整操作,确定与所述调整操作对应的参数调整信息;基于所述参数调整信息对所述初始参数信息进行调整,得到所述调整后的参数信息。The apparatus according to claim 18, wherein the annotation module is further configured to: in response to an adjustment operation for the second two-dimensional annotation frame in the to-be-adjusted view, determine parameter adjustment information corresponding to the adjustment operation ; Adjust the initial parameter information based on the parameter adjustment information to obtain the adjusted parameter information.
- 根据权利要求18或19所述的装置,其中,所述标注模块,还配置为:基于所述调整后的参数信息所对应的三维点云数据,与所述第一二维标注框信息所对应的二维点云数据之间的第一对应关系,确定所述待标注对象对应的三维标注框的底面的参数信息;基于所述调整后的参数信息所对应的三维点云数据,与所述第二二维标注框信息所对应的二维点云数据之间的第二对应关系,确定所述待标注对象对应的三维标注框的侧面和正面的参数信息中的至少之一;将确定的所述三维标注框的底面的参数信息、以及所述三维标注框的侧面和正面的参数信息中的至少之一进行组合,得 到所述三维标注框信息。The device according to claim 18 or 19, wherein the labeling module is further configured to: based on the three-dimensional point cloud data corresponding to the adjusted parameter information, corresponding to the first two-dimensional labeling frame information The first corresponding relationship between the two-dimensional point cloud data, determine the parameter information of the bottom surface of the three-dimensional mark frame corresponding to the object to be marked; based on the three-dimensional point cloud data corresponding to the adjusted parameter information, and the The second correspondence between the two-dimensional point cloud data corresponding to the second two-dimensional labeling frame information is to determine at least one of the side and front parameter information of the three-dimensional labeling frame corresponding to the object to be labelled; At least one of the parameter information of the bottom surface of the 3D annotation frame and the parameter information of the side surface and the front surface of the 3D annotation frame are combined to obtain the 3D annotation frame information.
- 根据权利要求16至20任一所述的装置,其中,所述第二确定模块,还配置为:基于确定的所述待标注对象的第二二维标注框信息,展示标注有与所述第二二维标注框信息对应的第二二维标注框的所述待调整视图。The device according to any one of claims 16 to 20, wherein the second determining module is further configured to: display a label with the The to-be-adjusted view of the second two-dimensional callout frame corresponding to the two-dimensional callout frame information.
- 根据权利要求15至22任一所述的装置,其中,所述装置还包括:The apparatus of any one of claims 15 to 22, wherein the apparatus further comprises:校验模块,配置为:获取校验图像;所述校验图像与所述待标注三维点云数据来源于同一场景;基于所述待标注三维点云数据对应的坐标系,与所述校验图像对应的坐标系之间的映射关系,将所述待标注对象对应的三维标注框信息映射至所述校验图像中,确定映射后的图像坐标范围;根据所述映射后的图像坐标范围与所述校验图像中的所述待标注对象所在图像坐标范围,对所述三维标注框信息的准确性进行校验。a verification module, configured to: obtain a verification image; the verification image and the three-dimensional point cloud data to be marked originate from the same scene; based on the coordinate system corresponding to the three-dimensional point cloud data to be marked, the verification image and the verification The mapping relationship between the coordinate systems corresponding to the images, the three-dimensional annotation frame information corresponding to the object to be marked is mapped to the verification image, and the mapped image coordinate range is determined; according to the mapped image coordinate range and In the verification image, the image coordinate range where the object to be marked is located, and the accuracy of the three-dimensional marked frame information is verified.
- 根据权利要求22所述的装置,其中,所述校验模块,还配置为:在所述映射后的图像坐标范围与所述校验图像中的所述待标注对象所在图像坐标范围不一致的情况下,响应针对所述待标注三维点云数据对应的待调整视图中的第二二维标注框的调整操作,对调整后的参数信息进行调整,直至所述映射后的图像坐标范围与所述校验图像中的所述待标注对象所在图像坐标范围一致;所述调整后的参数信息是通过对所述初始参数信息进行调整得到的。The apparatus according to claim 22, wherein the verification module is further configured to: in the case where the coordinate range of the image after the mapping is inconsistent with the coordinate range of the image where the object to be marked in the verification image is located Next, in response to the adjustment operation for the second two-dimensional annotation frame in the view to be adjusted corresponding to the three-dimensional point cloud data to be marked, the adjusted parameter information is adjusted until the mapped image coordinate range is the same as the described image coordinate range. Verify that the image coordinate ranges of the objects to be marked in the verification image are consistent; the adjusted parameter information is obtained by adjusting the initial parameter information.
- 根据权利要求15至23任一所述的装置,其中,所述标注模块,还配置为:将所述待标注对象对应的三维标注框信息,展示在所述待标注三维点云数据对应的三维视图中,并将所述待标注对象作为已标注对象;根据所述已标注对象的三维标注框信息,确定所述三维视图中的其它待标注对象的三维标注框信息;所述其它待标注对象与所述已标注对象属于同一对象类型;将所述其它待标注对象的三维标注框信息展示在所述三维视图中。The device according to any one of claims 15 to 23, wherein the labeling module is further configured to: display the 3D labeling frame information corresponding to the to-be-labeled object in the 3D corresponding to the to-be-labeled 3D point cloud data In the view, the object to be marked is regarded as the marked object; according to the three-dimensional markup frame information of the marked object, the three-dimensional markup frame information of other objects to be marked in the three-dimensional view is determined; the other objects to be marked belonging to the same object type as the marked object; displaying the three-dimensional marked frame information of the other objects to be marked in the three-dimensional view.
- 根据权利要求24所述的装置,其中,所述标注模块,还配置为:响应于针对所述三维视图中已标注对象的三维标注框的复制指令,复制所述已标注对象的三维标注框信息;响应于针对所述三维视图中其它待标注对象的粘贴指令,根据所述已标注对象在所述三维视图中的三维标注框信息,对所述其它待标注对象进行三维标注,得到所述其它待标注对象的三维标注框信息。The apparatus according to claim 24, wherein the annotation module is further configured to: in response to a copy instruction for the 3D annotation frame of the annotated object in the 3D view, copy the 3D annotation frame information of the annotated object ; In response to the paste instruction for other objects to be marked in the three-dimensional view, according to the three-dimensional markup frame information of the marked objects in the three-dimensional view, perform three-dimensional marking on the other objects to be marked, and obtain the other objects to be marked. Information about the 3D annotation frame of the object to be annotated.
- 根据权利要求15至25任一所述的装置,其中,所述标注模块,还配置为:将所述待标注对象对应的三维标注框信,息展示在当前帧待标注三维点云数据对应的第一三维视图中,并将所述待标注对象作为已标注对象;其中,不同帧待标注三维点云数据对应不同的三维视图;响应于针对所述第一三维视图中所述已标注对象的三维标注框的复制指令,复制所述已标注对象的三维标注框信息;在切换到下一帧待标注三维点云数据对应的第二三维视图的情况下,响应于针对所述第二三维视图中其它待标注对象的粘贴指令,根据所述已标注对象在所述第一三维视图中的三维标注框信息,对所述第二三维视图中的其它待标注对象进行三维标注,得到其它待标注对象的三维标注框信息。The device according to any one of claims 15 to 25, wherein the labeling module is further configured to: display the information of the 3D labeling frame corresponding to the object to be labelled in the current frame corresponding to the 3D point cloud data to be labelled In the first three-dimensional view, the object to be marked is regarded as the marked object; wherein, different frames of three-dimensional point cloud data to be marked correspond to different three-dimensional views; in response to the marked object in the first three-dimensional view The instruction for copying the 3D annotation frame, copying the 3D annotation frame information of the marked object; in the case of switching to the second 3D view corresponding to the 3D point cloud data to be marked in the next frame, in response to the second 3D view The pasting instruction of other objects to be annotated in the second 3D view, according to the 3D annotation frame information of the marked object in the first 3D view, perform 3D annotation on the other objects to be annotated in the second 3D view, and obtain other objects to be annotated. The 3D callout box information of the object.
- 根据权利要求15至25任一所述的装置,其中,所述第一确定模块,还配置为获取参考图像;所述参考图像与所述待标注三维点云数据来源于同一场景;基于所述待标注对象在所述参考图像中的图像位置信息、以及所述参考图像对应的坐标系与所述鸟瞰图对应的坐标系之间的映射关系,确定所述待标注对象在所述鸟瞰图中的参考位置信息;响应针对所述鸟瞰图中的所述待标注对象的框选操作,基于所述参考位置信息确定所述第一二维标注框信息。The apparatus according to any one of claims 15 to 25, wherein the first determining module is further configured to acquire a reference image; the reference image and the to-be-labeled 3D point cloud data originate from the same scene; based on the The image position information of the object to be labeled in the reference image, and the mapping relationship between the coordinate system corresponding to the reference image and the coordinate system corresponding to the bird's-eye view, determine that the object to be labeled is in the bird's-eye view The reference position information; in response to the frame selection operation on the object to be marked in the bird's-eye view, the first two-dimensional frame information is determined based on the reference position information.
- 根据权利要求22至26任一所述的装置,其中,所述初始参数信息包括初始高度信息和初始位姿信息中的至少之一。The apparatus according to any one of claims 22 to 26, wherein the initial parameter information includes at least one of initial height information and initial pose information.
- 一种电子设备,包括:处理器、存储器和总线,所述存储器存储有所述处理器可执行的机器可读指令,所述处理器配置为执行所述存储器中存储的机器可读指令,在所述电子设备运行的情况下,所述处理器与所述存储器之间通过总线通信,所述机器可读指令被所述处理器执行时执行如权利要求1至14任一所述的点云标注的方法的步骤。An electronic device comprising: a processor, a memory and a bus, the memory stores machine-readable instructions executable by the processor, the processor is configured to execute the machine-readable instructions stored in the memory, When the electronic device is running, the processor and the memory communicate through a bus, and the machine-readable instructions are executed by the processor to execute the point cloud according to any one of claims 1 to 14 Annotated method steps.
- 一种计算机可读存储介质,所述计算机可读存储介质上存储有计算机程序,所述计算机程序被电子设备运行时,所述电子设备执行如权利要求1至14任一所述的点云标注的方法的步骤。A computer-readable storage medium on which a computer program is stored, and when the computer program is run by an electronic device, the electronic device executes the point cloud labeling according to any one of claims 1 to 14 steps of the method.
- 一种计算机程序产品,包括存储程序代码的计算机可读存储介质,所述程序代码包括的指令可配置为执行如权利要求1至14任一所述的点云标注的方法的步骤。A computer program product comprising a computer-readable storage medium storing program code, the program code comprising instructions configurable to perform the steps of the method for point cloud labeling as claimed in any one of claims 1 to 14.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
KR1020227018072A KR20220092928A (en) | 2020-09-30 | 2021-05-24 | Point cloud labeling methods, devices, electronic devices, storage media and program products |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202011061734.2 | 2020-09-30 | ||
CN202011061734.2A CN112132901A (en) | 2020-09-30 | 2020-09-30 | Point cloud labeling method and device, electronic equipment and storage medium |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2022068225A1 true WO2022068225A1 (en) | 2022-04-07 |
Family
ID=73843536
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/CN2021/095444 WO2022068225A1 (en) | 2020-09-30 | 2021-05-24 | Point cloud annotating method and apparatus, electronic device, storage medium, and program product |
Country Status (3)
Country | Link |
---|---|
KR (1) | KR20220092928A (en) |
CN (1) | CN112132901A (en) |
WO (1) | WO2022068225A1 (en) |
Cited By (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN115423934A (en) * | 2022-08-12 | 2022-12-02 | 北京城市网邻信息技术有限公司 | House type graph generation method and device, electronic equipment and storage medium |
CN115542338A (en) * | 2022-11-30 | 2022-12-30 | 湖南大学 | Laser radar data learning method based on point cloud space distribution mapping |
CN115880536A (en) * | 2023-02-15 | 2023-03-31 | 北京百度网讯科技有限公司 | Data processing method, training method, target object detection method and device |
CN116152783A (en) * | 2023-04-18 | 2023-05-23 | 安徽蔚来智驾科技有限公司 | Target element labeling data acquisition method, computer equipment and storage medium |
Families Citing this family (9)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN111259980B (en) * | 2020-02-10 | 2023-10-03 | 北京小马慧行科技有限公司 | Method and device for processing annotation data |
CN112132901A (en) * | 2020-09-30 | 2020-12-25 | 上海商汤临港智能科技有限公司 | Point cloud labeling method and device, electronic equipment and storage medium |
CN113689508B (en) * | 2021-09-09 | 2024-02-02 | 北京地平线机器人技术研发有限公司 | Point cloud labeling method and device, storage medium and electronic equipment |
CN113763572B (en) * | 2021-09-17 | 2023-06-27 | 北京京航计算通讯研究所 | 3D entity labeling method based on AI intelligent recognition and storage medium |
CN113763573B (en) * | 2021-09-17 | 2023-07-11 | 北京京航计算通讯研究所 | Digital labeling method and device for three-dimensional object |
CN115423933B (en) * | 2022-08-12 | 2023-09-29 | 北京城市网邻信息技术有限公司 | House type diagram generation method and device, electronic equipment and storage medium |
CN115620199B (en) * | 2022-10-24 | 2023-06-13 | 四川警察学院 | Traffic safety risk diagnosis method and device |
CN116309962B (en) * | 2023-05-10 | 2023-09-26 | 倍基智能科技(四川)有限公司 | Laser radar point cloud data labeling method, system and application |
CN117557999A (en) * | 2023-11-20 | 2024-02-13 | 镁佳(北京)科技有限公司 | Image joint labeling method, computer equipment and medium |
Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN109727312A (en) * | 2018-12-10 | 2019-05-07 | 广州景骐科技有限公司 | Point cloud mask method, device, computer equipment and storage medium |
CN110136273A (en) * | 2019-03-29 | 2019-08-16 | 初速度(苏州)科技有限公司 | A kind of sample data mask method and device in machine learning |
CN110135453A (en) * | 2019-03-29 | 2019-08-16 | 初速度(苏州)科技有限公司 | A kind of laser point cloud data mask method and device |
CN111310667A (en) * | 2020-02-18 | 2020-06-19 | 北京小马慧行科技有限公司 | Method, device, storage medium and processor for determining whether annotation is accurate |
US20200200907A1 (en) * | 2018-12-19 | 2020-06-25 | Dalong Li | Semi-automatic lidar annotation system for autonomous driving |
CN112132901A (en) * | 2020-09-30 | 2020-12-25 | 上海商汤临港智能科技有限公司 | Point cloud labeling method and device, electronic equipment and storage medium |
-
2020
- 2020-09-30 CN CN202011061734.2A patent/CN112132901A/en active Pending
-
2021
- 2021-05-24 WO PCT/CN2021/095444 patent/WO2022068225A1/en active Application Filing
- 2021-05-24 KR KR1020227018072A patent/KR20220092928A/en not_active Application Discontinuation
Patent Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN109727312A (en) * | 2018-12-10 | 2019-05-07 | 广州景骐科技有限公司 | Point cloud mask method, device, computer equipment and storage medium |
US20200200907A1 (en) * | 2018-12-19 | 2020-06-25 | Dalong Li | Semi-automatic lidar annotation system for autonomous driving |
CN110136273A (en) * | 2019-03-29 | 2019-08-16 | 初速度(苏州)科技有限公司 | A kind of sample data mask method and device in machine learning |
CN110135453A (en) * | 2019-03-29 | 2019-08-16 | 初速度(苏州)科技有限公司 | A kind of laser point cloud data mask method and device |
CN111310667A (en) * | 2020-02-18 | 2020-06-19 | 北京小马慧行科技有限公司 | Method, device, storage medium and processor for determining whether annotation is accurate |
CN112132901A (en) * | 2020-09-30 | 2020-12-25 | 上海商汤临港智能科技有限公司 | Point cloud labeling method and device, electronic equipment and storage medium |
Cited By (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN115423934A (en) * | 2022-08-12 | 2022-12-02 | 北京城市网邻信息技术有限公司 | House type graph generation method and device, electronic equipment and storage medium |
CN115423934B (en) * | 2022-08-12 | 2024-03-08 | 北京城市网邻信息技术有限公司 | House type diagram generation method and device, electronic equipment and storage medium |
CN115542338A (en) * | 2022-11-30 | 2022-12-30 | 湖南大学 | Laser radar data learning method based on point cloud space distribution mapping |
CN115880536A (en) * | 2023-02-15 | 2023-03-31 | 北京百度网讯科技有限公司 | Data processing method, training method, target object detection method and device |
CN115880536B (en) * | 2023-02-15 | 2023-09-01 | 北京百度网讯科技有限公司 | Data processing method, training method, target object detection method and device |
CN116152783A (en) * | 2023-04-18 | 2023-05-23 | 安徽蔚来智驾科技有限公司 | Target element labeling data acquisition method, computer equipment and storage medium |
CN116152783B (en) * | 2023-04-18 | 2023-08-04 | 安徽蔚来智驾科技有限公司 | Target element labeling data acquisition method, computer equipment and storage medium |
Also Published As
Publication number | Publication date |
---|---|
CN112132901A (en) | 2020-12-25 |
KR20220092928A (en) | 2022-07-04 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
WO2022068225A1 (en) | Point cloud annotating method and apparatus, electronic device, storage medium, and program product | |
EP3690815B1 (en) | Method, medium and apparatus for automatically labeling target object within image | |
US20210279503A1 (en) | Image processing method, apparatus, and device, and storage medium | |
CN112861653B (en) | Method, system, equipment and storage medium for detecting fused image and point cloud information | |
US10824862B2 (en) | Three-dimensional object detection for autonomous robotic systems using image proposals | |
WO2020102944A1 (en) | Point cloud processing method and device and storage medium | |
US10977818B2 (en) | Machine learning based model localization system | |
Li et al. | Sustech points: A portable 3d point cloud interactive annotation platform system | |
US11978243B2 (en) | System and method using augmented reality for efficient collection of training data for machine learning | |
WO2021114884A1 (en) | Point cloud labeling method, apparatus, and system, device, and storage medium | |
WO2019042426A1 (en) | Augmented reality scene processing method and apparatus, and computer storage medium | |
CN110986969B (en) | Map fusion method and device, equipment and storage medium | |
US20110248995A1 (en) | System and methods for creating interactive virtual content based on machine analysis of freeform physical markup | |
EP2560145A2 (en) | Methods and systems for enabling the creation of augmented reality content | |
US20150138193A1 (en) | Method and device for panorama-based inter-viewpoint walkthrough, and machine readable medium | |
CN108876706A (en) | It is generated according to the thumbnail of panoramic picture | |
CN110070578B (en) | Loop detection method | |
WO2022021782A1 (en) | Method and system for automatically generating six-dimensional posture data set, and terminal and storage medium | |
KR20150082195A (en) | Annotation method and apparatus | |
CN108597034B (en) | Method and apparatus for generating information | |
CN116978010A (en) | Image labeling method and device, storage medium and electronic equipment | |
CN112948605B (en) | Point cloud data labeling method, device, equipment and readable storage medium | |
JP6304815B2 (en) | Image processing apparatus and image feature detection method, program and apparatus thereof | |
CN114089836B (en) | Labeling method, terminal, server and storage medium | |
CN113920191B (en) | 6D data set construction method based on depth camera |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
ENP | Entry into the national phase |
Ref document number: 2022514231 Country of ref document: JP Kind code of ref document: A |
|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 21873887 Country of ref document: EP Kind code of ref document: A1 |
|
ENP | Entry into the national phase |
Ref document number: 20227018072 Country of ref document: KR Kind code of ref document: A |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
32PN | Ep: public notification in the ep bulletin as address of the adressee cannot be established |
Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 28.08.2023) |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 21873887 Country of ref document: EP Kind code of ref document: A1 |