WO2022001325A1 - 点云数据的融合方法、装置、电子设备、存储介质和计算机程序 - Google Patents
点云数据的融合方法、装置、电子设备、存储介质和计算机程序 Download PDFInfo
- Publication number
- WO2022001325A1 WO2022001325A1 PCT/CN2021/089444 CN2021089444W WO2022001325A1 WO 2022001325 A1 WO2022001325 A1 WO 2022001325A1 CN 2021089444 W CN2021089444 W CN 2021089444W WO 2022001325 A1 WO2022001325 A1 WO 2022001325A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- point cloud
- cloud data
- target
- reflectivity
- sample point
- Prior art date
Links
- 238000004590 computer program Methods 0.000 title claims abstract description 15
- 238000007500 overflow downdraw method Methods 0.000 title claims abstract description 13
- 238000002310 reflectometry Methods 0.000 claims abstract description 312
- 238000000034 method Methods 0.000 claims abstract description 53
- 230000004927 fusion Effects 0.000 claims description 31
- 238000012545 processing Methods 0.000 claims description 19
- 230000015654 memory Effects 0.000 claims description 18
- 230000009466 transformation Effects 0.000 claims description 10
- 230000005540 biological transmission Effects 0.000 claims 1
- 101150007742 RING1 gene Proteins 0.000 description 12
- 208000035217 Ring chromosome 1 syndrome Diseases 0.000 description 12
- 238000001514 detection method Methods 0.000 description 10
- 230000008569 process Effects 0.000 description 8
- 238000010586 diagram Methods 0.000 description 5
- 238000005259 measurement Methods 0.000 description 5
- 230000006870 function Effects 0.000 description 4
- 238000004891 communication Methods 0.000 description 3
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 208000032825 Ring chromosome 2 syndrome Diseases 0.000 description 2
- 238000012935 Averaging Methods 0.000 description 1
- 208000032836 Ring chromosome 15 syndrome Diseases 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S13/00—Systems using the reflection or reradiation of radio waves, e.g. radar systems; Analogous systems using reflection or reradiation of waves whose nature or wavelength is irrelevant or unspecified
- G01S13/88—Radar or analogous systems specially adapted for specific applications
- G01S13/89—Radar or analogous systems specially adapted for specific applications for mapping or imaging
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
- G01S17/88—Lidar systems specially adapted for specific applications
- G01S17/89—Lidar systems specially adapted for specific applications for mapping or imaging
- G01S17/894—3D imaging with simultaneous measurement of time-of-flight at a 2D array of receiver pixels, e.g. time-of-flight cameras or flash lidar
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
- G01S17/02—Systems using the reflection of electromagnetic waves other than radio waves
- G01S17/06—Systems determining position data of a target
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S13/00—Systems using the reflection or reradiation of radio waves, e.g. radar systems; Analogous systems using reflection or reradiation of waves whose nature or wavelength is irrelevant or unspecified
- G01S13/87—Combinations of radar systems, e.g. primary radar and secondary radar
- G01S13/872—Combinations of primary radar and secondary radar
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S13/00—Systems using the reflection or reradiation of radio waves, e.g. radar systems; Analogous systems using reflection or reradiation of waves whose nature or wavelength is irrelevant or unspecified
- G01S13/87—Combinations of radar systems, e.g. primary radar and secondary radar
- G01S13/878—Combination of several spaced transmitters or receivers of known location for determining the position of a transponder or a reflector
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S13/00—Systems using the reflection or reradiation of radio waves, e.g. radar systems; Analogous systems using reflection or reradiation of waves whose nature or wavelength is irrelevant or unspecified
- G01S13/88—Radar or analogous systems specially adapted for specific applications
- G01S13/93—Radar or analogous systems specially adapted for specific applications for anti-collision purposes
- G01S13/931—Radar or analogous systems specially adapted for specific applications for anti-collision purposes of land vehicles
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
- G01S17/87—Combinations of systems using electromagnetic waves other than radio waves
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
- G01S17/88—Lidar systems specially adapted for specific applications
- G01S17/89—Lidar systems specially adapted for specific applications for mapping or imaging
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
- G01S17/88—Lidar systems specially adapted for specific applications
- G01S17/93—Lidar systems specially adapted for specific applications for anti-collision purposes
- G01S17/931—Lidar systems specially adapted for specific applications for anti-collision purposes of land vehicles
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S7/00—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00
- G01S7/02—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S13/00
- G01S7/40—Means for monitoring or calibrating
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S7/00—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00
- G01S7/02—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S13/00
- G01S7/40—Means for monitoring or calibrating
- G01S7/4004—Means for monitoring or calibrating of parts of a radar system
- G01S7/4021—Means for monitoring or calibrating of parts of a radar system of receivers
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S7/00—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00
- G01S7/48—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S17/00
- G01S7/4802—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S17/00 using analysis of echo signal for target characterisation; Target signature; Target cross-section
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S7/00—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00
- G01S7/48—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S17/00
- G01S7/497—Means for monitoring or calibrating
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T15/00—3D [Three Dimensional] image rendering
- G06T15/08—Volume rendering
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
- G06T17/20—Finite element generation, e.g. wire-frame surface description, tesselation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10032—Satellite or aerial image; Remote sensing
- G06T2207/10044—Radar image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2210/00—Indexing scheme for image generation or computer graphics
- G06T2210/56—Particle system, point based geometry or rendering
Definitions
- the present disclosure relates to the technical field of computer vision, and relates to, but is not limited to, a method, apparatus, electronic device, computer-readable storage medium and computer program for fusion of point cloud data.
- Lidar detects the position of a target by reflecting a laser beam. It has the characteristics of long detection distance and high measurement accuracy, so it can be widely used in the field of automatic driving.
- multiple lidars can be installed on the vehicle.
- the manufacturers of the installed multiple lidars may be different, or the models corresponding to the multiple lidars may be different, resulting in inconsistent standards for measuring the reflectivity of the multiple lidars, making the reflectivity corresponding to the different point cloud data after fusion different.
- Inconsistent measurement standards lead to distortion of the target represented by the fused point cloud data.
- the embodiments of the present disclosure provide at least a method, an apparatus, an electronic device, a computer-readable storage medium, and a computer program for fusion of point cloud data.
- Embodiments of the present disclosure provide a method for fusion of point cloud data, including:
- the main radar is one of the radars on the target vehicle
- the sub-radar is the other than the radar on the target vehicle. radars other than the main radar;
- the reflectivity in the point cloud data collected by the secondary radar is adjusted to obtain the adjusted point cloud data of the secondary radar; wherein, the reflectivity
- the calibration table represents the target reflectivity information of the main radar matched with each reflectivity corresponding to each scan line of the secondary radar;
- the point cloud data collected by the main radar and the adjusted point cloud data corresponding to the sub-radar are fused to obtain the fused point cloud data.
- Embodiments of the present disclosure also provide a point cloud data fusion device, including:
- the acquisition part is configured to acquire the point cloud data collected by the main radar and the sub-radar arranged on the target vehicle respectively;
- the main radar is one of the radars on the target vehicle, and the sub-radar is the target vehicle radars other than the main radar on the radar;
- the adjustment part is configured to adjust the reflectivity in the point cloud data collected by the sub-radar based on the predetermined reflectivity calibration table of the sub-radar to obtain the adjusted point cloud data of the sub-radar; wherein , the reflectivity calibration table represents the target reflectivity information of the primary radar matched with each reflectivity corresponding to each scan line of the secondary radar;
- the fusion part is configured to fuse the point cloud data collected by the main radar with the adjusted point cloud data of the secondary radar to obtain the fused point cloud data.
- Embodiments of the present disclosure further provide an electronic device, including: a processor, a memory, and a bus, where the memory stores machine-readable instructions executable by the processor, and when the electronic device runs, the processor and the The memories communicate with each other through a bus, and when the machine-readable instructions are executed by the processor, any one of the above-mentioned fusion methods for point cloud data is performed.
- Embodiments of the present disclosure further provide a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is run by a processor, any of the above-mentioned methods for fusion of point cloud data is executed.
- Embodiments of the present disclosure further provide a computer program, including computer-readable codes, when the computer-readable codes are executed in an electronic device, a processor in the electronic device executes the fusion of any of the above-mentioned point cloud data. method.
- Embodiments of the present disclosure provide a method, device, electronic device, storage medium, and computer program for fusion of point cloud data. Since a reflectivity calibration table is pre-generated, the reflectivity calibration table represents the corresponding value of each scan line of the secondary radar.
- the target reflectivity information of the main radar matched by each reflectivity, so that after obtaining the point cloud data collected by the sub-radar, the reflectivity in the point cloud data collected by the sub-radar can be adjusted according to the reflectivity calibration table, so that the main radar can adjust the reflectivity.
- the point cloud data collected by the radar is consistent with the measurement standard corresponding to the reflectivity in the adjusted point cloud data collected by the sub-radar, which can alleviate the distortion problem of the fused point cloud data and improve the accuracy of target detection.
- FIG. 1A is a schematic flowchart of a method for fusing point cloud data according to an embodiment of the present disclosure
- FIG. 1B is a schematic diagram of an application scenario provided by an embodiment of the present disclosure.
- FIG. 2 is a schematic flowchart of a method for determining a reflectivity calibration table in a point cloud data fusion method provided by an embodiment of the present disclosure
- FIG. 3 is a schematic structural diagram of a device for fusion of point cloud data according to an embodiment of the present disclosure
- FIG. 4 is a schematic structural diagram of an electronic device according to an embodiment of the present disclosure.
- multiple radars can be set on the target vehicle, each radar collects point cloud data separately, and the point cloud data collected by multiple radars are fused to obtain richer fused point cloud data, and then Object detection, or object tracking, can be performed based on the fused point cloud data.
- Object detection, or object tracking can be performed based on the fused point cloud data.
- the corresponding reflectivity between different radars may be inconsistent, the reflectivity of point cloud data from different sources is not uniform when fused, and the obtained fused point cloud data has the problem of distortion, which reduces the accuracy of the execution result.
- the radars in the embodiments of the present disclosure include lidars, millimeter-wave radars, and ultrasonic radars.
- the radars that perform point cloud data fusion may be the same type of radar, or may be different types of radars. In the embodiments of the present disclosure, only the radars that perform point cloud data fusion are laser radars as an example for description.
- the lidar can be manually calibrated or automatically calibrated.
- the manual calibration has high calibration accuracy, and the manual calibration result can be used as the true value.
- the lidar manufacturer performs manual calibration when the equipment leaves the factory, but manual calibration is performed manually. Calibration requires a special darkroom and calibration equipment; the automatic calibration method generally requires the lidar to perform a certain known motion while collecting point cloud data; however, there is no reflectivity calibration for multiple lidars in related technologies.
- embodiments of the present disclosure provide a fusion method for point cloud data.
- FIG. 1A is a schematic flowchart of a point cloud data fusion method provided by an embodiment of the present disclosure, the method includes steps S101-S103, wherein:
- Step S101 obtain the point cloud data collected by the main radar and the auxiliary radar set on the target vehicle respectively; the main radar is one of the radars on the target vehicle, and the auxiliary radar is the radar on the target vehicle except the main radar. .
- Step S102 based on the predetermined reflectivity calibration table corresponding to the secondary radar, adjust the reflectivity in the point cloud data collected by the secondary radar to obtain the adjusted point cloud data of the secondary radar; wherein, the reflectivity calibration table represents the secondary radar.
- the target reflectivity information of the main radar matched with each reflectivity corresponding to each scan line of the radar.
- Step S103 fuse the point cloud data collected by the primary radar with the adjusted point cloud data corresponding to the secondary radar to obtain the fused point cloud data.
- the target vehicle may be controlled according to the fused point cloud data.
- target detection and target tracking can be performed according to the fused point cloud data, and the target vehicle can be controlled according to the detection and tracking results.
- a reflectivity calibration table is pre-generated, and the reflectivity calibration table represents the target reflectivity information of the main radar matched by each reflectivity corresponding to each scan line of the sub-radar, so as to obtain the point collected by the sub-radar.
- the reflectivity in the point cloud data collected by the sub-radar can be adjusted according to the reflectivity calibration table, so that the point cloud data collected by the main radar corresponds to the reflectivity in the adjusted point cloud data collected by the sub-radar
- the measurement standard is consistent, which can alleviate the distortion problem of the fused point cloud data and improve the accuracy of target detection.
- Steps S101 to S103 will be described in detail below.
- the primary radar and the secondary radar may be radars disposed at different positions on the target vehicle, and the primary radar and the secondary radar may be multi-line radars.
- the types and setting positions of the primary radar and the secondary radar can be set according to actual needs, and the number of secondary radars can be multiple.
- the main radar can be a lidar set at the center of the target vehicle, that is, the primary lidar, and the two secondary radars can be lidars set at two sides of the target vehicle, ie, secondary lidars .
- the four radars are a first radar 11 , a second radar 12 , a third radar 13 and a fourth radar 14 , the first radar 11 , Any one of the second radar 12 , the third radar 13 and the fourth radar 14 is the main radar, and the three radars of the four radars other than the main radar are the sub-radars.
- the main radar can be a 16-line, 32-line, 64-line or 128-line lidar
- the secondary radar can be a 16-line, 32-line, 64-line or 128-line lidar
- the point cloud data respectively collected by the main radar and the sub-radar can be obtained.
- the point cloud data collected by the main radar includes data corresponding to a plurality of scanning points.
- the data corresponding to each scanning point includes the scanning point corresponding to the main radar.
- Position information and reflectivity in the Cartesian coordinate system; the point cloud data collected by the sub-radar may include data corresponding to multiple scanning points respectively.
- the data corresponding to each scanning point includes The position information and reflectivity of the scanning point in the Cartesian coordinate system corresponding to the secondary radar.
- coordinate transformation is performed on the point cloud data corresponding to the sub-radar, so that the point cloud data after the coordinate transformation is the same as the point cloud data collected by the main radar. It is located in the same coordinate system, that is, the point cloud data after coordinate transformation is located in the rectangular coordinate system corresponding to the main radar. Then, the reflectivity in the point cloud data collected by the sub-radar can be adjusted by using the predetermined reflectivity calibration table of the sub-radar to obtain the adjusted point cloud data corresponding to the sub-radar. Then, the point cloud data collected by the main radar and the adjusted point cloud data corresponding to the sub-radar are fused to obtain the fused point cloud data.
- a corresponding reflectivity calibration table can be generated for each sub-radar, and the point cloud data collected by the corresponding sub-radar can be analyzed by using the reflectivity calibration table corresponding to each sub-radar. Adjust to get the adjusted point cloud data corresponding to each sub-radar.
- a reflectivity calibration table with m rows and n columns can be obtained, where m represents the number of scan lines of the secondary radar, and n represents the reflectivity value range corresponding to each scan line; it can be seen that in When the number of sub-radars is a, a reflectivity calibration table with m rows and n columns can be obtained, where a is an integer greater than or equal to 1.
- the reflectivity calibration table may be as shown in Table 1 below, and the reflectivity calibration table may be a reflectivity calibration table corresponding to a 16-line sub-lidar.
- Table 1 includes the target reflectivity information of the main lidar whose reflectivity matches each scan line in the secondary lidar, and the 256 reflectivities corresponding to each scan line (the 256 reflectivities can be A reflectance of 0, a reflectance of 1, ..., a reflectance of 255), that is, the reflectance calibration table includes each reflection of each scan line in the 16 lines rate-matched target reflectance information.
- the target reflectivity information may include the target reflectivity average value, target reflectivity variance, target reflectivity maximum value, target reflectivity minimum value, etc., wherein the target reflectivity average value can be a positive integer, and the target reflectivity variance can be a positive real number .
- the target emissivity information of the main lidar whose scan line Ring0 and reflectivity matches 0 may be information X00; the target emissivity information of the main lidar whose scan line Ring15 and reflectivity matches 255 may be information X15255.
- the reflectivity calibration table is determined according to the following steps:
- Step S201 acquiring first sample point cloud data collected by a main radar installed on the sample vehicle, and second sample point cloud data collected by a secondary radar installed on the sample vehicle.
- Step S202 based on the first sample point cloud data, generate voxel map data, wherein the voxel map data includes data of a plurality of three-dimensional voxel grids, and the data of each three-dimensional voxel grid includes data based on the three-dimensional voxel grid. Reflectivity information determined from point cloud data for multiple scan points within the grid.
- Step S203 based on the second sample point cloud data and the data of a plurality of three-dimensional voxel grids, generate a reflectivity calibration table.
- the sample vehicle may be the same vehicle as the target vehicle, or may be a different vehicle.
- a sample vehicle equipped with a main radar and a sub-radar can be controlled to drive a preset distance on a preset road to obtain the first sample point cloud data and the second sample point cloud data. If there are multiple sub-radars, the second sample point cloud data corresponding to each sub-radar can be obtained.
- voxel map data may be generated based on the first sample point cloud data.
- the range of the voxel map data can be determined according to the first sample point cloud data. For example, if the first sample point cloud data is sample point cloud data within the first distance , the second distance range corresponding to the voxel map data is determined, wherein the second distance range corresponding to the voxel map data is located within the first distance range. Then divide the voxel map data of the second distance range to obtain a plurality of three-dimensional voxel grids within the second distance range, and determine the initial data of each three-dimensional voxel grid, that is, the initial data of each three-dimensional voxel grid.
- the initial data is set to a preset initial value.
- the initial data of each three-dimensional voxel grid can be the average of the reflectance and the variance of the reflectance of 0. , the number of scan points is 0.
- the initial data of each three-dimensional voxel grid is updated, and the updated data of each three-dimensional voxel grid is obtained.
- the above embodiment provides a method for generating a reflectivity calibration table, by generating voxel map data based on the first sample point cloud data, and obtaining the reflection of the first sample point cloud data on each three-dimensional voxel grid. Then, based on the second sample point cloud data and voxel map data, a reflectivity calibration table is generated.
- the reflectivity calibration table can more accurately reflect the main radar matching each reflectivity of each scan line of the secondary radar.
- the target reflectivity information, that is, the generated reflectivity calibration table has high accuracy.
- the reflectivity calibration table in order to generate the reflectivity calibration table, only the second sample point cloud data and the data of multiple 3D voxel grids need to be acquired, without the need for a harsh calibration environment and complex professional calibration equipment; in addition, the reflectivity calibration is generated.
- the process of the table can be automatically realized based on the second sample point cloud data and the data of multiple three-dimensional voxel grids, without requiring a lot of manual intervention to generate a reflectivity calibration table. Calibration is performed.
- generating voxel map data based on the first sample point cloud data including:
- De-distortion processing is performed on the first sample point cloud data based on the plurality of pose data to obtain processed first sample point cloud data.
- voxel map data is generated.
- a positioning device such as a Global Navigation Satellite System-Inertial Navigation System (GNSS-INS) may be set on the sample vehicle, and the sample vehicle is positioned by the positioning device to obtain that the sample vehicle is moving.
- the positioning accuracy of the positioning equipment can reach centimeter-level accuracy through the multiple pose data collected in sequence during the process.
- the sample vehicle can also be controlled to drive at a constant speed, and multiple pose data can be calculated according to the time when the primary radar or the secondary radar transmits and receives radio beams.
- the first sample point cloud data may be de-distorted by using a plurality of pose data to obtain processed first sample point cloud data. Since the radar obtains point cloud data by scanning the environment in a scanning cycle, when the radar is in motion, the generated point cloud data will be distorted, and the method of de-distortion is to transform the obtained point cloud data to the same moment, that is The dedistorted point cloud data can be considered as the point cloud data obtained at the same time. Therefore, the processed first sample point cloud data can be understood as the first sample point cloud data obtained at the same time. Further, voxel map data may be generated based on the processed first sample point cloud data.
- the de-distortion processing process can eliminate the first sample point cloud data of different frames and the first sample point cloud data of different batches in the first sample point cloud data of each frame in the first sample point cloud data.
- the deviation caused by the different radar positions corresponding to the data makes the processed first sample point cloud data can be understood as the first sample point cloud data measured at the same radar position, so that the first sample obtained based on the de-distortion processing
- the voxel map data is generated from the point cloud data, the accuracy of the generated voxel map data can be improved, thereby making the generated reflectance calibration table more accurate.
- the reflectance information includes an average reflectance
- the data for each three-dimensional voxel grid included in the voxel map data is determined according to the following steps:
- the average reflectance value corresponding to the three-dimensional voxel grid is determined.
- the three-dimensional voxel grid where each scan point is located can be determined according to the position information corresponding to each scan point in the first sample point cloud data, and then each three-dimensional voxel grid can be obtained by including: of each scan point.
- the reflectance of each scanning point in the three-dimensional voxel grid is averaged to obtain an average reflectance value corresponding to the three-dimensional voxel grid.
- generating a reflectance calibration table based on the second sample point cloud data and data of a plurality of three-dimensional voxel grids may include:
- each reflectivity of each scan line of the secondary radar from the second sample point cloud data, determine the position information of multiple target scan points corresponding to the reflectivity, and the multiple target scan points are scanned by the line Scan points obtained by line scanning; based on the position information of the multiple target scan points, determine at least one three-dimensional voxel grid corresponding to the multiple target scan points; The target reflectivity information of the main radar to which the reflectivity of the scan line matches;
- the reflectivity calibration table is generated based on the target reflectivity information of the primary radar matched with the determined reflectivity of each scan line of each scan line of the secondary radar.
- the scan points scanned by the scan line Ring1 are determined from the second sample point cloud data, and among the scan points that can be scanned from Ring1, it is determined that the reflectivity is 1.
- target scanning points; at least one three-dimensional voxel grid corresponding to the multiple target scanning points is determined according to the position information of the multiple target scanning points; it can be calculated based on the average reflectance values corresponding to the at least one three-dimensional voxel grid respectively
- the average target reflectance and target reflectance variance of the main radar whose scan line Ring1 and reflectance match 1 (the average target reflectance and target reflectance variance are the target reflectance information).
- a reflectivity calibration table can be generated based on the target reflectivity information of the primary radar matched with each reflectivity of each scan line of the secondary radar.
- multiple target scan points corresponding to each reflectivity of each scan line are determined; and then each scan line is determined based on the position information of the multiple target scan points At least one three-dimensional voxel grid corresponding to each reflectivity of each scan line; and then each scan line can be determined based on the average reflectance value of at least one three-dimensional voxel grid corresponding to each reflectivity of each scan line.
- the target reflectivity information of the main radar matched with each reflectivity of each scan line; finally, the reflectivity calibration table was generated based on the target reflectivity information of the main radar matched with each reflectivity of each scan line.
- multiple target scan points corresponding to each reflectivity of each scan line are determined; then based on the position information of the multiple target scan points, the corresponding reflectivity of each scan line is determined At least one three-dimensional voxel grid corresponding to each grid in the reflection calibration table is determined; and then the reflectance average corresponding to the at least one three-dimensional voxel grid corresponding to each grid can be determined. value, the target reflectance information of each grid is determined, and the reflectance calibration table is generated.
- the corresponding reflectivity should be consistent, that is, it can be considered that in the same three-dimensional voxel grid, the reflectivity of the scanning point obtained by the main radar scan is the same as that of the secondary radar.
- the reflectivity of the scan points obtained by scanning is the same. Therefore, at least one three-dimensional voxel grid corresponding to each reflectivity of each scan line of the sub-radar can be determined, and according to the average reflectivity corresponding to at least one three-dimensional voxel grid, the scan line can be more accurately determined.
- the target reflectivity information of the main radar with the reflectivity matching can then generate a more accurate reflectivity calibration table.
- the data of the three-dimensional voxel grid includes an average reflectance value and a weight influencing factor, and the weight influencing factor includes at least one of reflectance variance and the number of scanning points.
- the main radar matching the reflectivity of the scan line based on the reflectance average values corresponding to the at least one 3D voxel grid respectively target reflectivity information, including:
- the target reflectivity information of the main radar matched by the reflectivity of the scan line is determined.
- each three-dimensional voxel grid in the at least one three-dimensional voxel grid may be determined according to the weight influence factor corresponding weight.
- the weight influencing factor is the reflectivity variance value
- the weight of the three-dimensional voxel grid with large reflectivity variance can be set smaller, and the weight of the three-dimensional voxel grid with small reflectivity variance can be set relatively small.
- the weight influencing factor is the number of scan points
- the weight of the three-dimensional voxel grid with a large number of scan points can be set larger, and the weight of the three-dimensional voxel grid with a small number of scan points can be set smaller.
- the weight influencing factors include the reflectance variance and the number of scanning points
- the weight of the voxel grid is set smaller, etc.
- the average value of target reflectance can be obtained by weighted averaging, and the variance of target reflectance can be obtained by weighted variance, that is, the reflectivity of each scan line can be obtained.
- Matching target reflectivity information of the primary radar based on the weight and reflectance average value corresponding to each three-dimensional voxel grid, the average value of target reflectance can be obtained by weighted averaging, and the variance of target reflectance can be obtained by weighted variance, that is, the reflectivity of each scan line can be obtained. Matching target reflectivity information of the primary radar.
- a weight may be determined for each 3D voxel grid, and the weight of the 3D voxel grid with higher reliability is set to be larger (for example, the reflectance variance is smaller and the number of scanning points is larger). More 3D voxel grids have higher reliability), set the weights of 3D voxel grids with lower reliability smaller, so that the corresponding weights and reflectances based on each 3D voxel grid are averaged. value, the target reflectivity information of the main radar whose reflectivity matches the reflectivity of the scan line can be more accurately determined, thereby making the obtained reflectivity calibration table more accurate.
- a reflectivity calibration table is generated, including:
- a plurality of pose data collected in sequence during the movement of the sample vehicle is acquired, and the second sample point cloud data is subjected to de-distortion processing based on the plurality of pose data to obtain processed second sample point cloud data.
- the relative position information between the first sample point cloud data and the second sample point cloud data is determined.
- Coordinate transformation is performed on the processed second sample point cloud data by using the relative position information to obtain the second sample point cloud data in the target coordinate system; wherein, the target coordinate system is the coordinate system corresponding to the first sample point cloud data.
- a reflectivity calibration table is generated.
- the second sample point cloud data after processing can be obtained by performing de-distortion processing on the second sample point cloud data based on the plurality of pose data corresponding to the obtained sample vehicle and based on the plurality of pose data. And use the determined relative position information to perform coordinate transformation on the second sample point cloud data to obtain the second sample point cloud data in the target coordinate system, so that the second sample point cloud data obtained after the coordinate transformation is the same as the first sample point.
- the cloud data is located in the same coordinate system; finally, the reflectivity calibration table is generated by using the second sample point cloud data in the target coordinate system and the data of multiple three-dimensional voxel grids.
- the second sample point cloud data may be first de-distorted to eliminate the difference in radar positions corresponding to each batch of sample point cloud data and each frame of sample point cloud data in the second sample point cloud data Then convert the second sample point cloud data to the target coordinate system corresponding to the first sample point cloud data, and eliminate the difference between the radar positions of the second sample point cloud data and the first sample point cloud data. deviation, so that when the reflectivity calibration table is generated based on the second sample point cloud data obtained after de-distortion processing and coordinate transformation, the accuracy of the generated reflectivity calibration table can be improved.
- the first sample point cloud data and the second sample point cloud data may be respectively used as the target sample point cloud data, and when the target sample point cloud data is the first sample point cloud data, the main radar is used as the target sample point cloud data.
- the target radar when the target sample point cloud data is the second sample point cloud data, the sub-lidar is used as the target radar; there are multiple frames in the target sample point cloud data, and each frame of the target sample point cloud data includes multiple scans emitted by the target radar.
- the target sample point cloud data can be de-distorted according to the following steps:
- the batch of scan lines is collected.
- the coordinates of the obtained target sample point cloud data are converted to the coordinate system of the target radar corresponding to the target sample point cloud data collected by transmitting the first scan lines in the target sample point cloud data of this frame, and the target sample of this frame is obtained.
- the target sample point cloud data of the frame is obtained.
- the coordinates of the sample point cloud data are converted to the coordinate system of the target radar corresponding to the target sample point cloud data of the first frame, and the target sample point cloud data corresponding to the target sample point cloud data of this frame after the second de-distortion is obtained.
- the first sample point cloud data may include multiple frames of the first sample point cloud data, and each frame of the first sample point cloud data includes multiple batches of The first sample point cloud data of the times.
- the non-first sample point cloud data in the frame of the first sample point cloud data The first sample point cloud data collected by the batch emission scan lines are converted to the coordinate system of the main radar corresponding to the first scan line emission time in the first sample point cloud data of the frame, and the first de-distortion is completed. deal with.
- the coordinates of the first sample point cloud data of the frame can be converted to the first sample point cloud data of the first frame.
- the second de-distortion processing is completed.
- each frame of the first sample point cloud data includes 10 batches of the first sample point cloud data, namely the first batch of the first sample point cloud data, the second batch of the first sample point cloud data One sample point cloud data, ..., the tenth batch of the first sample point cloud data.
- the second batch of the first sample point cloud data For each frame of the first sample point cloud data, the second batch of the first sample point cloud data to the tenth batch of the first sample point cloud data, each batch of the first sample point cloud data, through interpolation
- the method determines the pose information when the main radar transmits the batch of scan lines, and converts the coordinates of the first sample point cloud data of the batch (that is, the first sample point cloud data collected by the batch of scan lines) to the In the coordinate system of the main radar corresponding to the emission moment of the first batch of scan lines in the first sample point cloud data of the frame, it is converted to the first batch of the first sample point cloud data in the first sample point cloud data of the frame In the corresponding coordinate system of the main radar, the first dedistorted first sample point cloud data corresponding to the first sample point cloud data of each frame can be obtained.
- the main radar scans to obtain the first sample point cloud data of the frame
- the pose information of the first sample point cloud data of the frame is converted into the coordinate system of the main radar corresponding to the first sample point cloud data of the first frame, and then the corresponding coordinates of the first sample point cloud data can be obtained.
- the first sample point cloud data after the second dewarping.
- the target sample point cloud data of each frame of target sample point cloud data that is not collected by the first scan line and the target sample point cloud data of different frames of target sample point cloud data that are not the first frame of target sample point cloud data are uniformly transformed to the first frame target.
- the first batch of target sample point cloud data in the sample point cloud data corresponds to the coordinate system of the target radar, thereby improving the accuracy of the generated reflectivity calibration table.
- the reflectivity of the scan line without matching target reflectivity information may also be determined in the reflectivity calibration table. Based on the target reflectivity information of the main radar in the reflectivity calibration table, determine the target reflectivity information of the main radar corresponding to the reflectivity of the scan line for which no matching target reflectivity information exists. The reflectivity calibration table is updated based on the target reflectivity information of the main radar corresponding to the reflectivity of the scan line for which no matching target reflectivity information exists determined.
- the reflectivity calibration table when there is matching target reflectivity information for each reflectivity of each scan line, that is, there is a corresponding target reflectivity in each grid in the generated reflectivity calibration table information, the reflectivity calibration table does not need to be updated.
- At least one target reflectivity information with reflectivity matching can be obtained through a linear interpolation method.
- the grid corresponding to Ring1 and reflectivity 4 has target reflectivity information, and the grid corresponding to Ring1 and reflectivity 6 exists.
- the target reflectivity information can be obtained by linear interpolation according to the target reflectivity information in the grid corresponding to Ring1 and reflectivity 4 and the target reflectivity information in the grid corresponding to Ring1 and reflectivity 6 in the reflectivity calibration table. method to obtain the target reflectance information in the grid corresponding to Ring1 and reflectivity 5.
- the target reflectivity information in the grid corresponding to Ring0 and reflectivity 5, and the target reflectivity information in the grid corresponding to Ring2 and reflectivity 5 can be obtained by The linear interpolation method is used to obtain the target reflectivity information in the grid corresponding to Ring1 and reflectivity of 5.
- the reflectivity calibration table may be updated based on the target reflectivity information of the main radar corresponding to the determined at least one reflectivity to generate an updated reflectivity calibration table, wherein, in the updated reflectivity calibration table, the target reflectance
- the average target reflectance in the rate information can be a positive integer, that is, the average target reflectance corresponding to each grid in the reflectance calibration table can be adjusted to a positive integer by rounding to generate an updated reflectance calibration table.
- the generated reflectance calibration table may be incomplete. In order to ensure the integrity of the reflectance calibration table , you can determine the missing target reflectivity information in the reflectivity calibration table based on the target reflectivity information of the main radar existing in the reflectivity calibration table, complete the reflection calibration table, and generate an updated reflectivity calibration table, that is, get A complete reflectance calibration table is available.
- the writing order of each step does not mean a strict execution order but constitutes any limitation on the implementation process, and the specific execution order of each step should be based on its function and possible Internal logic is determined.
- an embodiment of the present disclosure further provides a point cloud data fusion device.
- a schematic diagram of the architecture of a point cloud data fusion device provided by an embodiment of the present disclosure includes an acquisition part 301 . , adjustment part 302, fusion part 303, reflectivity calibration determination part 304 and update part 305, wherein,
- the acquisition part 301 is configured to acquire point cloud data respectively collected by the main radar and the sub-radar arranged on the target vehicle;
- the main radar is one of the radars on the target vehicle, and the sub-radar is the target Radars other than the main radar among the radars on the vehicle;
- the adjustment part 302 is configured to adjust the reflectivity in the point cloud data collected by the sub-radar based on the predetermined reflectivity calibration table of the sub-radar to obtain the adjusted point cloud data of the sub-radar;
- the reflectivity calibration table represents the target reflectivity information of the primary radar matched with each reflectivity corresponding to each scan line of the secondary radar;
- the fusion part 303 is configured to fuse the point cloud data collected by the main radar with the adjusted point cloud data corresponding to the sub-radar to obtain the fused point cloud data, and align the target according to the fused point cloud data. Control the vehicle.
- the fusion apparatus further includes: a reflectivity calibration determination part 304;
- the reflectance calibration determination part 304 is configured to determine the reflectance calibration table according to the following steps:
- voxel map data is generated, wherein the voxel map data includes data of a plurality of three-dimensional voxel grids, and the data of each three-dimensional voxel grid includes Reflectivity information determined by point cloud data of multiple scanning points within each 3D voxel grid;
- the reflectivity calibration table is generated based on the second sample point cloud data and the data of the plurality of three-dimensional voxel grids.
- the reflectivity calibration determination part 304 when generating voxel map data based on the first sample point cloud data, is configured to:
- voxel map data is generated.
- the reflectance information includes a reflectance average value
- the reflectance calibration determination section 304 is configured to determine data for each three-dimensional voxel grid included in the voxel map data according to the following steps :
- each three-dimensional voxel grid For each three-dimensional voxel grid, based on the reflectivity in the point cloud data of each scanning point in the each three-dimensional voxel grid, determine the average reflectance value corresponding to each three-dimensional voxel grid ;
- the reflectance calibration determination part 304 when generating the reflectance calibration table based on the second sample point cloud data and the data of the plurality of three-dimensional voxel grids, is configured as follows:
- the position information of multiple target scanning points corresponding to each reflectivity is determined from the second sample point cloud data, and the multiple targets scan
- the point is a scan point obtained by scanning each scan line; based on the position information of the multiple target scan points, at least one three-dimensional voxel grid corresponding to the multiple target scan points is determined; based on the at least one The average reflectance values corresponding to the three-dimensional voxel grids, respectively, to determine the target reflectivity information of the main radar matched by the reflectivity of each scan line;
- the reflectivity calibration table is generated based on the target reflectivity information of the primary radar matched with each reflectivity of each scan line of the sub-radar determined.
- the data of the three-dimensional voxel grid includes the reflectance average value and a weight influence factor, and the weight influence factor includes at least one of reflectance variance and the number of scanning points;
- the reflectance calibration determination part is based on the reflectance average values corresponding to the at least one three-dimensional voxel grid respectively , when determining the target reflectivity information of the main radar that matches the reflectivity of each scan line, the configuration is:
- the reflectance calibration determination part 304 when generating the reflectance calibration table based on the second sample point cloud data and the data of the plurality of three-dimensional voxel grids, is configured to:
- the reflectivity calibration table is generated based on the second sample point cloud data in the target coordinate system and the data of the plurality of three-dimensional voxel grids.
- the first sample point cloud data and the second sample point cloud data are respectively used as target sample point cloud data, when the target sample point cloud data is the first sample point cloud data , taking the main radar as the target radar, and taking the secondary laser radar as the target radar when the target sample point cloud data is the second sample point cloud data;
- the target sample point cloud data has multiple frames, and each The frame target sample point cloud data includes sample point cloud data collected by the target radar transmitting multiple scan lines; wherein the target radar transmits scan lines in batches according to a preset frequency, and each batch transmits multiple scan lines.
- the reflectivity calibration determination part 304 is configured to perform de-distortion processing on the target sample point cloud data according to the following steps:
- the target sample point cloud data collected in the target sample point cloud data of each frame that is not the first scan line based on the pose information of the target radar when the non-first scan line is transmitted, the The coordinates of the target sample point cloud data collected by the non-first batch of scan lines are transmitted, and converted to the target sample point cloud data corresponding to the target sample point cloud data collected by the first batch of scan lines in the target sample point cloud data of each frame. Under the coordinate system of the target radar, obtain the target sample point cloud data after the first de-distortion of the target sample point cloud data of each frame;
- any non-first frame target sample point cloud data in the multi-frame target sample point cloud data after the first de-distortion based on the position of the target radar when scanning to obtain the any non-first frame target sample point cloud data Attitude information, convert the coordinates of any non-first frame target sample point cloud data to the coordinate system of the target radar corresponding to the first frame target sample point cloud data, and obtain any non-first frame target sample point cloud data corresponding to The target sample point cloud data after the second dewarping.
- the fusion apparatus further includes an update part 305, and the update part 305 is configured to:
- the reflectivity calibration table is updated based on the target reflectivity information of the main radar corresponding to the determined reflectivity of the scan line without matching target reflectivity information.
- the functions or templates included in the apparatus provided by the embodiments of the present disclosure may be used to execute the methods described in the above method embodiments.
- the functions or templates included in the apparatus provided by the embodiments of the present disclosure may be used to execute the methods described in the above method embodiments.
- an embodiment of the present disclosure also provides an electronic device.
- a schematic structural diagram of an electronic device 400 provided by an embodiment of the present disclosure includes a processor 401 , a memory 402 , and a bus 403 .
- the memory 402 is configured to store execution instructions, including the memory 4021 and the external memory 4022; the memory 4021 here is also called the internal memory, and is configured to temporarily store the operation data in the processor 401 and the data exchanged with the external memory 4022 such as the hard disk,
- the processor 401 exchanges data with the external memory 4022 through the memory 4021.
- the processor 401 communicates with the memory 402 through the bus 403, so that the processor 401 is executing any of the above methods for integrating cloud data.
- Embodiments of the present disclosure further provide a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is run by a processor, the fusion of point cloud data described in any of the foregoing method embodiments is performed. method.
- Embodiments of the present disclosure further provide a computer program, including computer-readable codes, when the computer-readable codes are executed in an electronic device, a processor in the electronic device executes the fusion of any of the above-mentioned point cloud data. method. For details, reference may be made to the foregoing method embodiments, which will not be repeated here.
- the units described as separate components may or may not be physically separated, and components shown as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution in this embodiment.
- each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, or each unit may exist physically alone, or two or more units may be integrated into one unit.
- the functions, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a processor-executable non-volatile computer-readable storage medium.
- the computer software products are stored in a storage medium, including Several instructions are used to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present disclosure.
- the aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disk or optical disk and other media that can store program codes .
- the embodiments of the present disclosure provide a method, device, electronic device, storage medium and computer program for fusion of point cloud data.
- the method includes: acquiring point cloud data collected by a primary radar and a secondary radar set on a target vehicle, respectively;
- the main radar is one of the radars on the target vehicle, and the sub-radar is the radar other than the main radar among the radars on the target vehicle;
- the collected point cloud data is fused with the adjusted point cloud data corresponding to the sub-radar to obtain the fused point cloud data.
- the reflectivity calibration table is pre-generated, and the reflectivity calibration table represents the target reflectivity information of the main radar matched with each reflectivity corresponding to each scan line of the sub-radar, so as to obtain the point collected by the sub-radar.
- the reflectivity in the point cloud data collected by the sub-radar can be adjusted according to the reflectivity calibration table, so that the point cloud data collected by the main radar corresponds to the reflectivity in the adjusted point cloud data collected by the sub-radar
- the measurement standard is consistent, which can alleviate the distortion problem of the fused point cloud data and improve the accuracy of target detection.
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Remote Sensing (AREA)
- Radar, Positioning & Navigation (AREA)
- General Physics & Mathematics (AREA)
- Computer Networks & Wireless Communication (AREA)
- Electromagnetism (AREA)
- Theoretical Computer Science (AREA)
- Computer Graphics (AREA)
- Geometry (AREA)
- Software Systems (AREA)
- Radar Systems Or Details Thereof (AREA)
- Optical Radar Systems And Details Thereof (AREA)
Abstract
Description
Claims (19)
- 一种点云数据的融合方法,应用于电子设备中,所述方法包括:获取设置在目标车辆上的主雷达和副雷达分别采集得到的点云数据;所述主雷达为所述目标车辆上的雷达中的一个,所述副雷达为所述目标车辆上的雷达中除主雷达以外的雷达;基于预先确定的所述副雷达的反射率标定表,对所述副雷达采集的点云数据中的反射率进行调整,得到所述副雷达的调整后的点云数据;其中,所述反射率标定表表征所述副雷达的每条扫描线对应的每种反射率所匹配的主雷达的目标反射率信息;将所述主雷达采集到的点云数据与所述副雷达的调整后的点云数据进行融合,得到融合后的点云数据。
- 根据权利要求1所述的点云数据的融合方法,其中,根据下述步骤确定所述反射率标定表:获取设置在样本车辆上的所述主雷达采集得到的第一样本点云数据,以及设置在所述样本车辆上的所述副雷达采集得到的第二样本点云数据;基于所述第一样本点云数据,生成体素地图数据,其中,所述体素地图数据包括多个三维体素网格的数据,每个三维体素网格的数据包括基于所述每个三维体素网格内的多个扫描点的点云数据确定的反射率信息;基于所述第二样本点云数据以及所述多个三维体素网格的数据,生成所述反射率标定表。
- 根据权利要求2所述的点云数据的融合方法,其中,所述基于所述第一样本点云数据,生成体素地图数据,包括:获取所述样本车辆在移动过程中依次采集的多个位姿数据;基于所述多个位姿数据对所述第一样本点云数据进行去畸变处理,得到处理后的第一样本点云数据;基于处理后的第一样本点云数据,生成所述体素地图数据。
- 根据权利要求2或3所述的点云数据的融合方法,其中,所述反射率信息包括反射率平均值,根据以下步骤确定所述体素地图数据中包括的每个三维体素网格的数据:针对所述每个三维体素网格,基于该三维体素网格内各个扫描点的点云数据中的反射率,确定该三维体素网格对应的反射率平均值;所述基于所述第二样本点云数据以及所述多个三维体素网格的数据,生成所述反射率标定表,包括:针对所述副雷达的每条扫描线的每种反射率,从所述第二样本点云数据中,确定与该种反射率对应的多个目标扫描点的位置信息,所述多个目标扫描点是由该条扫描线扫描得到的扫描点;基于所述多个目标扫描点的位置信息,确定所述多个目标扫描点对应的至少一个三维体素网格;基于所述至少一个三维体素网格分别对应的所述反射率平均值,确定该条扫描线的该种反射率所匹配的所述主雷达的目标反射率信息;基于确定的所述副雷达的每条扫描线的每种反射率所匹配的所述主雷达的所述目标反射率信息,生成所述反射率标定表。
- 根据权利要求4所述的点云数据的融合方法,其中,所述三维体素网格的数据中包括所述反射率平均值和权重影响因子,所述权重影响因子包括反射率方差和扫描点数量中的至少一项;在所述至少一个三维体素网格为多个三维体素网格的情况下,所述基于所述至少一 个三维体素网格分别对应的所述反射率平均值,确定该条扫描线的该种反射率匹配的所述主雷达的目标反射率信息,包括:基于所述权重影响因子,确定所述至少一个三维体素网格中每个所述三维体素网格对应的权重;基于所述每个三维体素网格对应的权重及其对应的反射率平均值,确定该条扫描线的该种反射率所匹配的所述主雷达的所述目标反射率信息。
- 根据权利要求2-5任一所述的点云数据的融合方法,其中,所述基于所述第二样本点云数据以及所述多个三维体素网格的数据,生成所述反射率标定表,包括:获取所述样本车辆在移动过程中依次采集的多个位姿数据,并基于所述多个位姿数据对所述第二样本点云数据进行去畸变处理,得到处理后的第二样本点云数据;基于所述主雷达在所述样本车辆上的位置信息以及所述副雷达在所述样本车辆上的位置信息,确定所述第一样本点云数据与所述第二样本点云数据之间的相对位置信息;利用所述相对位置信息对处理后的第二样本点云数据进行坐标转换,得到目标坐标系下的第二样本点云数据;其中,所述目标坐标系为所述第一样本点云数据对应的坐标系;基于所述目标坐标系下的第二样本点云数据以及所述多个三维体素网格的数据,生成所述反射率标定表。
- 根据权利要求3或6所述的点云数据的融合方法,其中,将所述第一样本点云数据和所述第二样本点云数据分别作为目标样本点云数据,在所述目标样本点云数据为第一样本点云数据时,将所述主雷达作为目标雷达,在所述目标样本点云数据为第二样本点云数据时,将所述副雷达作为目标雷达;所述目标样本点云数据中有多帧,每帧目标样本点云数据包括目标雷达发射多条扫描线采集到的目标样本点云数据;其中,所述目标雷达按照预设频率分批次发射扫描线,每批次发射多条扫描线;根据以下步骤对所述目标样本点云数据进行去畸变处理:基于所述多个位姿数据,确定所述目标雷达在发射每批次扫描线时的位姿信息;针对每帧目标样本点云数据中非首批次发射扫描线所采集到的目标样本点云数据,基于所述目标雷达在发射该批次扫描线时的位姿信息,将发射该批次扫描线所采集到的目标样本点云数据的坐标,转换到该帧目标样本点云数据中发射首批次扫描线所采集到的目标样本点云数据所对应的目标雷达的坐标系下,得到该帧目标样本点云数据的第一次去畸变后的目标样本点云数据;针对第一次去畸变后的多帧目标样本点云数据中任一非首帧目标样本点云数据,基于所述目标雷达在扫描得到该帧目标样本点云数据时的位姿信息,将该帧目标样本点云数据的坐标转换到首帧目标样本点云数据对应的目标雷达的坐标系下,得到该帧目标样本点云数据对应的第二次去畸变后的目标样本点云数据。
- 根据权利要求1-7任一所述的点云数据的融合方法,其中,所述方法还包括:在所述反射率标定表中,确定不存在匹配的目标反射率信息的扫描线的反射率;基于所述反射率标定表中所述主雷达的所述目标反射率信息,确定所述不存在匹配的目标反射率信息的扫描线的反射率所对应的主雷达的目标反射率信息;基于确定的所述不存在匹配的目标反射率信息的扫描线的反射率所对应的主雷达的目标反射率信息,更新所述反射率标定表。
- 一种点云数据的融合装置,包括:获取部分,配置为获取设置在目标车辆上的主雷达和副雷达分别采集得到的点云数据;所述主雷达为所述目标车辆上的雷达中的一个,所述副雷达为所述目标车辆上的雷达中除主雷达以外的雷达;调整部分,配置为基于预先确定的所述副雷达的反射率标定表,对所述副雷达采集 的点云数据中的反射率进行调整,得到所述副雷达的调整后的点云数据;其中,所述反射率标定表表征所述副雷达的每条扫描线对应的每种反射率所匹配的主雷达的目标反射率信息;融合部分,配置为将所述主雷达采集到的点云数据与所述副雷达的调整后的点云数据进行融合,得到融合后的点云数据。
- 根据权利要求9所述的点云数据的融合装置,其中,所述融合装置还包括:反射率标定确定部分;所述反射率标定确定部分,配置为根据下述步骤确定所述反射率标定表:获取设置在样本车辆上的所述主雷达采集得到的第一样本点云数据,以及设置在所述样本车辆上的所述副雷达采集得到的第二样本点云数据;基于所述第一样本点云数据,生成体素地图数据,其中,所述体素地图数据包括多个三维体素网格的数据,每个三维体素网格的数据包括基于所述每个三维体素网格内的多个扫描点的点云数据确定的反射率信息;基于所述第二样本点云数据以及所述多个三维体素网格的数据,生成所述反射率标定表。
- 根据权利要求10所述的点云数据的融合装置,其中,所述反射率标定确定部分,在基于所述第一样本点云数据,生成体素地图数据时,配置为:获取所述样本车辆在移动过程中依次采集的多个位姿数据;基于所述多个位姿数据对所述第一样本点云数据进行去畸变处理,得到处理后的第一样本点云数据;基于处理后的第一样本点云数据,生成体素地图数据。
- 根据权利要求10或11所述的点云数据的融合装置,其中,所述反射率信息包括反射率平均值,所述反射率标定确定部分,配置为根据以下步骤确定所述体素地图数据中包括的每个三维体素网格的数据:针对所述每个三维体素网格,基于所述每个三维体素网格内各个扫描点的点云数据中的反射率,确定所述每个三维体素网格对应的反射率平均值;所述反射率标定确定部分,在基于所述第二样本点云数据以及所述多个三维体素网格的数据,生成所述反射率标定表时,配置为:针对所述副雷达的每条扫描线的每种反射率,从所述第二样本点云数据中确定所述每种反射率对应的多个目标扫描点的位置信息,所述多个目标扫描点是由所述每条扫描线扫描得到的扫描点;基于所述多个目标扫描点的位置信息,确定所述多个目标扫描点对应的至少一个三维体素网格;基于所述至少一个三维体素网格分别对应的所述反射率平均值,确定所述每条扫描线的所述每种反射率所匹配的所述主雷达的目标反射率信息;基于确定的所述副雷达的每条扫描线的每种反射率所匹配的所述主雷达的所述目标反射率信息,生成所述反射率标定表。
- 根据权利要求12所述的点云数据的融合装置,其中,所述三维体素网格的数据中包括所述反射率平均值和权重影响因子,所述权重影响因子包括反射率方差和扫描点数量中的至少一项;在所述至少一个三维体素网格为多个三维体素网格的情况下,所述反射率标定确定部分,在基于所述至少一个三维体素网格分别对应的所述反射率平均值,确定所述每条扫描线的所述每种反射率匹配的所述主雷达的目标反射率信息时,配置为:基于所述权重影响因子,确定所述至少一个三维体素网格中每个所述三维体素网格对应的权重;基于所述每个三维体素网格对应的权重及其对应的所述反射率平均值,确定所述每条扫描线的所述每种反射率所匹配的所述主雷达的所述目标反射率信息。
- 根据权利要求10至13任一所述的点云数据的融合装置,其中,所述反射率标定确定部分,在基于所述第二样本点云数据以及所述多个三维体素网格的数据,生成所述反射率标定表时,配置为:获取所述样本车辆在移动过程中依次采集的多个位姿数据,并基于所述多个位姿数据对所述第二样本点云数据进行去畸变处理,得到处理后的第二样本点云数据;基于所述主雷达在所述样本车辆上的位置信息以及所述副雷达在所述样本车辆上的位置信息,确定所述第一样本点云数据与所述第二样本点云数据之间的相对位置信息;利用所述相对位置信息对处理后的第二样本点云数据进行坐标转换,得到目标坐标系下的第二样本点云数据;其中,所述目标坐标系为所述第一样本点云数据对应的坐标系;基于所述目标坐标系下的第二样本点云数据以及所述多个三维体素网格的数据,生成所述反射率标定表。
- 根据权利要求11或14所述的点云数据的融合装置,其中,将所述第一样本点云数据和所述第二样本点云数据分别作为目标样本点云数据,在所述目标样本点云数据为第一样本点云数据时,将所述主雷达作为目标雷达,在所述目标样本点云数据为第二样本点云数据时,将所述副激光雷达作为目标雷达;所述目标样本点云数据中有多帧,每帧目标样本点云数据包括目标雷达发射多条扫描线采集到的样本点云数据;其中,所述目标雷达按照预设频率分批次发射扫描线,每批次发射多条扫描线;所述反射率标定确定部分,配置为根据以下步骤对所述目标样本点云数据进行去畸变处理:基于所述多个位姿数据,确定所述目标雷达在发射每批次扫描线时的位姿信息;针对所述每帧目标样本点云数据中非首批次发射扫描线所采集到的目标样本点云数据,基于所述目标雷达在发射所述非首批次扫描线时的位姿信息,将发射所述非首批次扫描线所采集到的目标样本点云数据的坐标,转换到所述每帧目标样本点云数据中发射首批次扫描线所采集到的目标样本点云数据所对应的目标雷达的坐标系下,得到所述每帧目标样本点云数据的第一次去畸变后的目标样本点云数据;针对第一次去畸变后的多帧目标样本点云数据中任一非首帧目标样本点云数据,基于所述目标雷达在扫描得到所述任一非首帧目标样本点云数据时的位姿信息,将所述任一非首帧目标样本点云数据的坐标转换到首帧目标样本点云数据对应的目标雷达的坐标系下,得到所述任一非首帧目标样本点云数据对应的第二次去畸变后的目标样本点云数据。
- 根据权利要求9至15任一所述的点云数据的融合装置,其中,所述融合装置还包括更新部分,所述更新部分配置为:在所述反射率标定表中,确定不存在匹配的目标反射率信息的扫描线的反射率;基于所述反射率标定表中所述主雷达的所述目标反射率信息,确定所述不存在匹配的目标反射率信息的扫描线的反射率所对应的主雷达的目标反射率信息;基于确定的所述不存在匹配的目标反射率信息的扫描线的反射率所对应的主雷达的目标反射率信息,更新所述反射率标定表。
- 一种电子设备,包括:处理器、存储器和总线,所述存储器存储有所述处理器可执行的机器可读指令,当电子设备运行时,所述处理器与所述存储器之间通过总线通信,所述机器可读指令被所述处理器执行时执行如权利要求1至8任一所述的点云数据的融合方法。
- 一种计算机可读存储介质,该计算机可读存储介质上存储有计算机程序,该计算机程序被处理器运行时执行如权利要求1至8任一所述的点云数据的融合方法。
- 一种计算机程序,包括计算机可读代码,当所述计算机可读代码在电子设备中 运行时,所述电子设备中的处理器执行用于实现权利要求1至8任一所述的点云数据的融合方法。
Priority Applications (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2021564866A JP2022541976A (ja) | 2020-06-30 | 2021-04-23 | 点群データの融合方法、装置、電子機器、記憶媒体及びコンピュータプログラム |
KR1020217037652A KR102359063B1 (ko) | 2020-06-30 | 2021-04-23 | 포인트 클라우드 데이터의 융합 방법, 장치, 전자 기기, 저장 매체 및 컴퓨터 프로그램 |
US17/653,275 US20220214448A1 (en) | 2020-06-30 | 2022-03-02 | Point cloud data fusion method and apparatus, electronic device, storage medium and computer program |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202010618348.2 | 2020-06-30 | ||
CN202010618348.2A CN113866779A (zh) | 2020-06-30 | 2020-06-30 | 点云数据的融合方法、装置、电子设备及存储介质 |
Related Child Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US17/653,275 Continuation US20220214448A1 (en) | 2020-06-30 | 2022-03-02 | Point cloud data fusion method and apparatus, electronic device, storage medium and computer program |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2022001325A1 true WO2022001325A1 (zh) | 2022-01-06 |
Family
ID=78981860
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/CN2021/089444 WO2022001325A1 (zh) | 2020-06-30 | 2021-04-23 | 点云数据的融合方法、装置、电子设备、存储介质和计算机程序 |
Country Status (5)
Country | Link |
---|---|
US (1) | US20220214448A1 (zh) |
JP (1) | JP2022541976A (zh) |
KR (1) | KR102359063B1 (zh) |
CN (1) | CN113866779A (zh) |
WO (1) | WO2022001325A1 (zh) |
Families Citing this family (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
GB2596867A (en) * | 2020-07-10 | 2022-01-12 | Scout Drone Inspection As | Inspection device for inspecting a building or structure |
CN114763997A (zh) * | 2022-04-14 | 2022-07-19 | 中国第一汽车股份有限公司 | 车辆采集到的雷达点云数据的处理方法、装置及电子设备 |
CN114842075B (zh) * | 2022-06-30 | 2023-02-28 | 小米汽车科技有限公司 | 数据标注方法、装置、存储介质及车辆 |
CN115966095A (zh) * | 2022-12-02 | 2023-04-14 | 云控智行科技有限公司 | 基于车辆的交通数据融合处理方法、装置、设备及介质 |
CN116184342B (zh) * | 2023-04-27 | 2023-07-21 | 无锡智鸿达电子科技有限公司 | 一种基于多雷达组网的测云雷达数据校准方法及系统 |
Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20150139532A1 (en) * | 2013-11-20 | 2015-05-21 | Electronics And Telecommunications Research Institute | Camara tracking apparatus and method using reconstruction segments and volumetric surface |
CN109839624A (zh) * | 2017-11-27 | 2019-06-04 | 北京万集科技股份有限公司 | 一种多激光雷达位置标定方法及装置 |
CN109991984A (zh) * | 2019-04-22 | 2019-07-09 | 上海蔚来汽车有限公司 | 用于生成高精细地图的方法、装置和计算机存储介质 |
CN110007300A (zh) * | 2019-03-28 | 2019-07-12 | 东软睿驰汽车技术(沈阳)有限公司 | 一种得到点云数据的方法及装置 |
CN110658530A (zh) * | 2019-08-01 | 2020-01-07 | 北京联合大学 | 一种基于双激光雷达数据融合的地图构建方法、系统及地图 |
CN110850394A (zh) * | 2019-12-02 | 2020-02-28 | 苏州智加科技有限公司 | 一种自动驾驶激光雷达强度标定方法 |
Family Cites Families (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN105184852B (zh) * | 2015-08-04 | 2018-01-30 | 百度在线网络技术(北京)有限公司 | 一种基于激光点云的城市道路识别方法及装置 |
KR102373926B1 (ko) * | 2016-02-05 | 2022-03-14 | 삼성전자주식회사 | 이동체 및 이동체의 위치 인식 방법 |
JP6938846B2 (ja) * | 2016-03-14 | 2021-09-22 | 株式会社デンソー | 物体認識装置 |
CN109425365B (zh) * | 2017-08-23 | 2022-03-11 | 腾讯科技(深圳)有限公司 | 激光扫描设备标定的方法、装置、设备及存储介质 |
CN110221275B (zh) * | 2019-05-21 | 2023-06-23 | 菜鸟智能物流控股有限公司 | 一种激光雷达与相机之间的标定方法和装置 |
CN113748359B (zh) * | 2020-03-27 | 2024-04-09 | 深圳市速腾聚创科技有限公司 | 基于连续波测距的补偿方法、装置和激光雷达 |
-
2020
- 2020-06-30 CN CN202010618348.2A patent/CN113866779A/zh active Pending
-
2021
- 2021-04-23 WO PCT/CN2021/089444 patent/WO2022001325A1/zh active Application Filing
- 2021-04-23 KR KR1020217037652A patent/KR102359063B1/ko active IP Right Grant
- 2021-04-23 JP JP2021564866A patent/JP2022541976A/ja active Pending
-
2022
- 2022-03-02 US US17/653,275 patent/US20220214448A1/en not_active Abandoned
Patent Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20150139532A1 (en) * | 2013-11-20 | 2015-05-21 | Electronics And Telecommunications Research Institute | Camara tracking apparatus and method using reconstruction segments and volumetric surface |
CN109839624A (zh) * | 2017-11-27 | 2019-06-04 | 北京万集科技股份有限公司 | 一种多激光雷达位置标定方法及装置 |
CN110007300A (zh) * | 2019-03-28 | 2019-07-12 | 东软睿驰汽车技术(沈阳)有限公司 | 一种得到点云数据的方法及装置 |
CN109991984A (zh) * | 2019-04-22 | 2019-07-09 | 上海蔚来汽车有限公司 | 用于生成高精细地图的方法、装置和计算机存储介质 |
CN110658530A (zh) * | 2019-08-01 | 2020-01-07 | 北京联合大学 | 一种基于双激光雷达数据融合的地图构建方法、系统及地图 |
CN110850394A (zh) * | 2019-12-02 | 2020-02-28 | 苏州智加科技有限公司 | 一种自动驾驶激光雷达强度标定方法 |
Also Published As
Publication number | Publication date |
---|---|
JP2022541976A (ja) | 2022-09-29 |
US20220214448A1 (en) | 2022-07-07 |
KR102359063B1 (ko) | 2022-02-08 |
CN113866779A (zh) | 2021-12-31 |
KR20220004099A (ko) | 2022-01-11 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
WO2022001325A1 (zh) | 点云数据的融合方法、装置、电子设备、存储介质和计算机程序 | |
CN107340522B (zh) | 一种激光雷达定位的方法、装置及系统 | |
EP4283328A1 (en) | Multi-radar and camera joint calibration method, system and device, and storage medium | |
CN108921947B (zh) | 生成电子地图的方法、装置、设备、存储介质以及采集实体 | |
US10764487B2 (en) | Distance image acquisition apparatus and application thereof | |
CN110501712B (zh) | 无人驾驶中用于确定位置姿态数据的方法、装置和设备 | |
EP3540464A2 (en) | Ranging method based on laser radar system, device and readable storage medium | |
CN111913169B (zh) | 激光雷达内参、点云数据的修正方法、设备及存储介质 | |
WO2021097983A1 (zh) | 定位的方法、装置、设备及存储介质 | |
CN116008964A (zh) | 激光雷达及其反射率标定方法和反射率测量方法 | |
CN113777592B (zh) | 方位角标定方法和装置 | |
CN113534110B (zh) | 一种多激光雷达系统静态标定方法 | |
EP4198901A1 (en) | Camera extrinsic parameter calibration method and apparatus | |
CN113702927A (zh) | 一种车辆传感器标定方法、装置及存储介质 | |
CN116299319B (zh) | 多激光雷达的同步扫描及点云数据处理方法和雷达系统 | |
CN113240745A (zh) | 点云数据标定方法、装置、计算机设备和存储介质 | |
CN110515060B (zh) | 多线激光雷达反射率标定的方法 | |
CN116203544A (zh) | 一种移动测量系统往返测回无控自检校方法、装置及介质 | |
WO2022256976A1 (zh) | 稠密点云真值数据的构建方法、系统和电子设备 | |
US11978158B2 (en) | Determining minimum region for finding planar surfaces | |
CN115100287A (zh) | 外参标定方法及机器人 | |
EP4235093A2 (en) | Oct system, method of generating oct image and storage medium | |
CN117129979B (zh) | 一种基于机器学习模型的激光雷达标定方法及系统 | |
JP6823690B2 (ja) | 位置調整方法、位置調整装置、及び、位置調整プログラム | |
CN117668574B (zh) | 光影秀的数据模型优化方法、装置、设备及存储介质 |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
ENP | Entry into the national phase |
Ref document number: 2021564866 Country of ref document: JP Kind code of ref document: A |
|
ENP | Entry into the national phase |
Ref document number: 20217037652 Country of ref document: KR Kind code of ref document: A |
|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 21831602 Country of ref document: EP Kind code of ref document: A1 |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 21831602 Country of ref document: EP Kind code of ref document: A1 |