JP2023503767A - Data processing method and device - Google Patents

Abstract

The present invention provides a data processing method and device, wherein the method is to acquire positioning data collected by a positioning device and point cloud data collected by a radar device, wherein the positioning device and the radar the device is deployed in the same vehicle; based on the positioning data and the point cloud data, determines position data of the radar device when collecting the point cloud data; Determining ground parameter information representing the ground based on the position data and determining initial pose data of the radar device based on the position data; and determining the positioning based on the ground parameter information and the initial pose data. and adjusting extrinsic parameter data representing the relative positional relationship between the device and the radar device. [Selection drawing] Fig. 1

Description

CROSS REFERENCE TO RELATED APPLICATIONS This application is a Chinese patent application entitled "Method and Apparatus for Processing Data" filed with the Patent Office of China on October 30, 2020 with application number 202011190058.9 and the entire content of the Chinese patent application is incorporated herein by reference.

The present invention relates to the technical field of information processing, and more particularly to a data processing method and apparatus.

With the development of information technology, automated driving technology has gradually become widely applied. Currently, autonomous driving technology mainly collects radar point cloud data through lidar (Light Detection And Ranging, LiDAR), and then through positioning devices (Global Positioning System (GPS), integrated inertial navigation system etc.), and perform information fusion on point cloud data and position information to determine the positional relationship between the autonomous driving device and obstacles, thereby avoiding obstacles. Realize Here, when determining the position information of the rider based on the positioning device, generally, the position information of the positioning device is determined first, and then the position information of the positioning device is converted to the position of the rider based on the external parameter data. Convert to information. Here, the external parameter data are used to express the relative positional relationship between the positioning device and the lidar.

In the related art, when determining extrinsic parameter data, it is necessary to perform calibrations to the position of the positioning device and lidar using a series of instruments. However, since the positions of the positioning device and the rider may change during the use of the automatic driving device, it is necessary to update the external parameter data every preset time. Each update requires manual recalibration of positioning device and lidar positions using measurement equipment and redetermination of extrinsic parameter data, but such methods are inefficient.

Embodiments of the present invention provide at least a method and apparatus for processing data.

In a first aspect, an embodiment of the present invention provides a data processing method, the method is to acquire positioning data collected by a positioning device and point cloud data collected by a radar device, comprising: The positioning device and the radar device are installed in the same vehicle, and based on the positioning data and the point cloud data, position data of the radar device when collecting the point cloud data is determined. determining ground parameter information representing the ground based on the point cloud data, and determining initial pose data of the radar device based on the position data; the ground parameter information and the initial pose data; and adjusting external parameter data representing the relative positional relationship between the positioning device and the radar device based on.

According to the above embodiment, according to the positioning data collected by the positioning device and the radar data collected by the radar device, the ground parameter information representing the ground and the initial pose data of the radar device are determined, and then the ground parameter information and the radar device are determined. The ground parameter information and the initial pose data of the radar device can both be obtained by data collection or calculation, so that other equipment can be adjusted while adjusting the extrinsic parameter data. does not need to be used to manually calibrate the position of the positioning device and the radar device, improving the efficiency of adjusting external parameter data and saving labor costs.

In one possible embodiment, the point cloud data includes coordinate information of a plurality of radar scanning points, and based on the point cloud data, determining ground parameter information representing the ground surface comprises: performing plane fitting to obtain fitting plane information; and based on the coordinate information of the radar scanning points in the point cloud data, each radar scanning point and the plane indicated by the fitting plane information. and performing point cloud data screening based on the distance between each determined radar scan point and the plane indicated by the fitting plane information, and based on the screened point cloud data and returning to performing plane fitting based on the point cloud data until a preset iteration condition is reached; and determining the ground parameter information based on the final filtered point cloud data. and including.

In the above embodiment, the point cloud data screening is performed multiple times through the method of iterative fitting, and then, based on the finally screened point cloud data, the ground parameter information is determined, and the accuracy of the ground parameter information is improve.

In one possible embodiment, prior to performing plane fitting based on said point cloud data, a downsampling operation is performed on said point cloud data to determine the distribution density of radar scan points within said point cloud data. satisfies a preset condition, and screening the radar scan points whose corresponding coordinate information is located within the target coordinate range, wherein the target The coordinate range is determined according to the preset mounting height of the radar device.

Before performing plane fitting based on the point cloud data, we first perform data screening on the point cloud data, on the one hand, to improve the calculation accuracy of the ground parameter information, and on the other hand, to improve the accuracy of the plane fitting process in the plane fitting process. The amount of calculation can be reduced and the calculation efficiency can be improved.

In one possible embodiment, the position data includes height data, and the method of processing the data is based on the height data in the position data and the correspondence between the position data and the point cloud data. It involves dividing the point cloud data into multiple acquisition intervals.

In one possible embodiment, dividing the point cloud data into a plurality of acquisition intervals based on height data within the position data and correspondence between the position data and the point cloud data includes: filtering the data to determine at least one extreme point of the height data retained after filtering; into multiple collection intervals.

In one possible embodiment, determining ground parameter information representative of the ground based on said point cloud data and determining initial pose data for said radar device based on said position data are performed at each acquisition interval. , determining the ground parameter information corresponding to the collection interval based on the point cloud data within the collection interval, and determining the position data of the radar device when collecting the point cloud data within the collection interval and determining the initial pose data of the radar device corresponding to the acquisition interval based on the relative position of the positioning device and the radar device based on the ground parameter information and the initial pose data. Adjusting the extrinsic parameter data representing the relationship comprises adjusting the extrinsic parameter data representing the relative positional relationship between the positioning device and the radar device based on the ground parameter information and the initial pose data corresponding to each acquisition interval. including doing

In the above embodiment, additional consideration is given to the ground height information, and the point cloud data is divided into multiple collection intervals to determine ground parameter information corresponding to different collection intervals, respectively, and and the initial pose data within the collection interval, the adjusted extrinsic parameters are more accurate.

In one possible embodiment, adjusting extrinsic parameter data representing a relative positional relationship between the positioning device and the radar device based on the ground parameter information and the initial pose data corresponding to each acquisition interval includes: , determining optimized pose data corresponding to each acquisition interval based on the ground parameter information and the initial pose data corresponding to each acquisition interval; the initial pose data corresponding to each acquisition interval; and adjusting the extrinsic parameter data based on optimized pose data.

In one possible embodiment, determining optimized pose data corresponding to each acquisition interval, based on the ground parameter information and the initial pose data corresponding to each acquisition interval, comprises: In this case, the pose data that minimizes the value of the target function is determined based on the ground parameter information corresponding to the collection interval and the initial pose data, and the pose data that minimizes the value of the target function is set to the collection interval wherein the target function is the absolute value of the difference between the ground parameter information before and after the pose data optimization and the difference between the pose data before and after the pose data optimization. It is the sum of absolute values.

In one possible embodiment, adjusting the extrinsic parameter data based on the initial pose data corresponding to each acquisition interval and optimized pose data comprises: , and based on the optimized pose data, determining a first average pose variation within the acquisition interval; and based on the first average pose variation within each acquisition interval, the plurality of acquisition intervals. and adjusting the extrinsic parameter data based on the second average pose variation.

Since the ground parameter information in different collection intervals is different, the optimized pose data corresponding to the initial pose data determined by the ground parameter information in different collection intervals is more accurate, and the first average pose variation is and calculating a second average pose variation, and adjusting the extrinsic parameter data based on the second average pose variation, the adjusted extrinsic parameter data will be more accurate.

In one possible embodiment, adjusting the extrinsic parameter data based on the second average pose variation comprises adjusting the product of the second average pose variation and the extrinsic parameter data before adjustment. including use as external parameter data.

In one possible embodiment, the positioning data comprises position data each collected at a plurality of first collection time points, and the point cloud data comprises point cloud data each collected at a plurality of second collection time points. determining, based on the positioning data and the point cloud data, position data of the radar device when collecting the point cloud data, for each of the second collection time points, the plurality of first collection If there is a target first collection time point that overlaps with the second collection time point among the time points, the position of the radar device at the second collection time point is calculated based on the position data corresponding to the target first collection time point. Determining data, and if the target first collection time point does not exist among the plurality of first collection time points, the time interval from the second collection time point of the plurality of first collection time points is the shortest. Determining position data of the radar device at a second collection time point based on position data corresponding to two adjacent first collection time points, wherein the second collection time points correspond to the two adjacent first collection time points. and being located between the first collection time points.

According to the above embodiment, the position data of the radar device at each second collection time can be determined, and interference due to different data collection frequencies of the radar device and the positioning device can be avoided.

In a second aspect, embodiments of the present invention further provide an apparatus for processing data, configured to obtain positioning data collected by the positioning device and point cloud data collected by the radar device, wherein: The positioning device and the radar device, based on an acquisition module deployed in the same vehicle, and the positioning data and the point cloud data, determine position data of the radar device when collecting the point cloud data. a first determining module, for determining ground parameter information representing the ground based on said point cloud data, and determining initial pose data for said radar device based on said position data; a second determining module configured to adjust external parameter data representing a relative positional relationship between the positioning device and the radar device based on the ground parameter information and the initial pose data; an adjustment module.

In a third aspect, embodiments of the present invention further provide a computing device comprising a processor, a memory and a bus, wherein said memory stores said processor-executable machine-readable instructions for execution by said computing device. Sometimes, the processor and the memory communicate via the bus, and when the processor executes the machine-readable instructions, the steps of the first aspect or method of processing data according to the first aspect. to run.

In a fourth aspect, embodiments of the invention provide a computer storage medium having stored thereon a computer program, the computer program, when executed by a processor, performing the steps described in the first aspect or the first aspect. data processing method steps.

For the effects related to the data processing device, computer equipment, computer-readable storage medium and computer program, reference can be made to the above description of the data processing method, which will not be repeated here. In order to make the above objects, features and advantages of the present invention more comprehensible, preferred embodiments are described in detail below with reference to the accompanying drawings.

To describe the technical solutions in the embodiments of the present invention more clearly, the following briefly describes the drawings required for the description of the embodiments. These drawings show embodiments consistent with the present invention and are used to describe the technical solutions of the present invention together with the description. The following drawings only show a part of the embodiments of the present invention, and should not be regarded as limiting the protection scope of the embodiments. It should be understood that other related drawings may be derived from the drawings.
4 is a flow chart of a method for processing data according to an embodiment of the present invention; FIG. 4 is a comparison diagram of a first collection time and a second collection time according to an embodiment of the present invention; FIG. 4 is another comparison diagram of the first collection time and the second collection time according to the embodiment of the present invention; 4 is a flowchart of a method for determining ground parameter information representing the ground based on point cloud data according to an embodiment of the present invention; FIG. 4 is a schematic diagram of a method for determining extreme points of height data according to an embodiment of the present invention; 4 is a flowchart of a method for adjusting extrinsic parameter data according to an embodiment of the invention; 1 is an architecture diagram of a data processing device according to an embodiment of the present invention; FIG. 1 is an exemplary structural diagram of a computer device according to an embodiment of the present invention; FIG.

In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the following clearly describes the technical solutions of the embodiments of the present invention with reference to the drawings of the embodiments of the present invention. Explain fully. Apparently, the described embodiments are some but not all embodiments of the present invention. In general, the components of the embodiments of the invention described and illustrated in the drawings herein can be arranged and designed in a variety of different configurations. Therefore, the following detailed description of embodiments of the invention with accompanying drawings is not intended to limit the scope of protection of the claimed invention, but merely shows selected embodiments of the invention. is. All other embodiments obtained by persons skilled in the art based on the embodiments of the present invention without creative work fall within the protection scope of the present invention.

Due to the more or less unique limitations of a single sensor, current mapping and positioning algorithms mainly focus on multi-sensor fusion schemes, i.e., combining observations of multiple sensors in their own coordinate system into the same coordinate system. The bridge that integrates and integrates the observations of multiple sensors in their respective coordinate systems into the same coordinate system is the extrinsic parameter data between the sensors. Therefore, the quality of extrinsic parameter calibration has a significant impact on the performance of multi-sensor fusion algorithms.

For example, common in the field of autonomous driving are mapping and positioning schemes that integrate positioning devices (such as integrated inertial navigation systems) and radar devices. Generally, before an autonomous vehicle leaves the factory, a set of equipment is used to calibrate the position of the positioning device and radar of the autonomous vehicle, so that the relative position of the positioning device and the radar device Determine the external parameter data representing the positional relationship. However, since the relative position between the positioning device and the radar device changes while the autonomous vehicle is in use, it is necessary to update the external parameter data periodically.

In the related art, even when updating the external parameter data, the method of determining the external parameter data at the factory shipment of the automatic driving vehicle is adopted, so the procedure is complicated and the efficiency is low.

Alternatively, the extrinsic parameter data can be estimated by a hand eye calibration algorithm. Specifically, the autonomous vehicle is controlled to run on a certain track, and while the autonomous vehicle is running, the positioning position data collected by the positioning device and the point cloud data collected by the radar device are recorded. , then the pose trajectory of the positioning device and the radar device can be calculated respectively, and the extrinsic parameter data can be solved based on the hand-eye calibration algorithm.

However, this method places high demands on the measurement accuracy of the positioning device and the radar device, and can only provide variable parameters with three degrees of freedom. The extrinsic parameter data consists of six degree-of-freedom parameters, i.e. the distance to translate along the X-, Y- and Z-axes of a Cartesian coordinate system, and the angle to rotate along the X-, Y- and Z-axes respectively. can be represented by (X, Y, Z, Roll, Pitch, Yaw), the error of the extrinsic parameter data determined by this method is large.

In view of the deficiencies of the above technical solution, the present invention provides a data processing method, according to the positioning data collected by the positioning device and the radar data collected by the radar device, the ground parameter information representing the ground and the radar Initial pose data for the device can be determined, and then the extrinsic parameter data can be adjusted based on the ground parameter information and the initial pose data for the radar device. Since the ground parameter information and the initial pose data of the radar device can be obtained by data collection or calculation, the process of adjusting the external parameter data is manually performed with respect to the relative position of the positioning device and the radar device using other equipment. No need to perform calibration, improve the efficiency of adjusting the external parameter data, save labor costs is a parameter with six degrees of freedom, so when adjusting the extrinsic parameter data, the adjustment can be performed on all six extrinsic parameter data, thereby improving the adjustment accuracy of the extrinsic parameter data.

All the deficiencies of the above technical solutions are the results obtained by the inventors after implementation and careful research, therefore, the process of finding the above problems and the solutions proposed by the following examples of the present invention to the above problems All measures should be the inventor's contribution to the embodiment of the invention.

The following clearly and completely describes the technical solutions of the present invention with reference to the drawings of the present invention. Apparently, the described embodiments are some but not all embodiments of the present invention. In general, the components of the invention described and illustrated in the drawings herein can be arranged and designed in a variety of different configurations. Therefore, the following detailed description of embodiments of the invention with accompanying drawings is not intended to limit the scope of protection of the claimed invention, but merely shows selected embodiments of the invention. is. All other embodiments obtained by persons skilled in the art based on the embodiments of the present invention without creative work fall within the protection scope of the present invention.

It is noted that like reference numerals and letters indicate like items in the following drawings, and thus, once an item is defined in one drawing, it need not be defined and interpreted in subsequent drawings. sea bream.

To facilitate understanding of the embodiments of the present invention, first, the data processing method disclosed in the embodiments of the present invention will be described in detail. The execution subject of the data processing method provided by the embodiments of the present invention is generally a computer device with specific computing capabilities, such as a terminal device or a server or other processing device. Inclusive, the terminal equipment can be User Equipment (UE), mobile equipment, user terminal, terminal, mobile phone, cordless phone, Personal Digital Assistant (PDA), and the like. In some possible embodiments, the data processing method can be performed by a processor on board the autonomous vehicle.

Hereinafter, the data processing method provided by the embodiment of the present invention will be described, taking as an example that the execution subject is a processor installed in an automatic driving device. Here, it should be noted that the positioning device and the radar device are respectively connected to the processor, and the connection method includes, but is not limited to, wired connection and wireless connection such as Bluetooth connection and wireless local area network connection. .

Referring to FIG. 1, an embodiment of the present invention provides a data processing method, which includes the following steps.

In step 101, obtaining positioning data collected by a positioning device and point cloud data collected by a radar device, wherein the positioning device and the radar device are deployed in the same vehicle.

In step 102, based on the positioning data and the point cloud data, determine position data of the radar device when collecting the point cloud data.

In step 103, ground parameter information representing the ground is determined based on the point cloud data, and initial pose data of the radar device is determined based on the position data.

In step 104, external parameter data representing the relative positional relationship between the positioning device and the radar device is adjusted based on the ground parameter information and the initial pose data.

According to the above implementation method, the ground parameter information representing the ground and the initial pose data of the radar device are determined according to the positioning data collected by the positioning device and the radar data collected by the radar device, and then the ground parameter information and the radar device The extrinsic parameter data can be adjusted based on the initial pose data of . Since the ground parameter information and the initial pose data of the radar device can be obtained by data collection or calculation, the process of adjusting the external parameter data is manually performed with respect to the relative position of the positioning device and the radar device using other equipment. There is no need to perform calibration, improving the efficiency of adjusting external parameter data and saving labor costs.

The following is a detailed description of steps 101-104.

For step 101,
Both the positioning device and the radar device can be mounted on the same vehicle, and the mounting positions of the positioning device and the radar device are different. The positioning device can be, for example, the Global Positioning System (GPS) or an integrated inertial navigation system, which is a system that integrates GPS and inertial sensors.

For step 102,
In practical applications, the data acquisition frequencies of the radar device and the positioning device are different within the same period. The positioning device can collect the position data according to the collection frequency set by the GPS satellite within the data collection time when collecting the position data, and the radar device can according to the set collection frequency when collecting the point cloud data A radio beam can be emitted and point cloud data can be collected at the set frequency. In practical applications, the collection frequencies set by GPS satellites are higher than the set collection frequencies corresponding to radar equipment.

The positioning data collected by the positioning device includes a plurality of first collection time points and position data collected at each first collection time point, wherein the position data collected at each first collection time point is latitude and longitude data , and height relative to sea level, the point cloud data comprising a plurality of second collection time points and point cloud data collected at each second collection time point, wherein at each second collection time point The collected point cloud data includes coordinate information of multiple radar scan points.

The positioning data output by the positioning device is the positioning data after performing adjustment based on the external parameter data representing the relative positional relationship between the positioning device and the radar device, that is, the positioning data output by the positioning device is , is the positioning data of the radar device.

Since the frequency at which the radar device collects the radar data is different from the frequency at which the positioning device collects the positioning data, the position data of the radar device when the positioning device collects the positioning data is obtained based on the positioning data and the radar data. It is necessary to determine, ie determine the position data of the corresponding radar device when collecting the point cloud data at each second collection time. Here, the position data of the corresponding radar device when collecting the point cloud data at each second collection time is hereinafter referred to as "corresponding to the radar device when collecting the point cloud data at the second collection time" position data" or, more simply, "position data of the radar device at the time of the second acquisition".

Specifically, for each second collection time point, first, it is detected whether or not there is a first collection time point corresponding to the second collection time point, and the first collection time point corresponding to the second collection time point is detected. If it exists, the position data at the first collection time corresponding to the second collection time is determined as the position data of the radar device at the second collection time, and the first collection time corresponding to the second collection time is determined If not, the position data of the radar device at the second collection time can be calculated based on the time interval between the first collection time and the second collection time.

In one possible embodiment, when calculating the position data of the radar device at the second collection instants, for each second collection instant first the two first determining each collection time point, the two first collection time points including one first collection time point before the second collection time point and one first collection time point after the second collection time point; , based on the position data respectively corresponding to the two first collection times, the position data of the radar device at the second collection time can be calculated.

Specifically, when the position data of the radar device at the second collection time is determined based on the position data respectively corresponding to the two first collection time points, the first collection time and the second collection time are evenly distributed. When distributed, the average value of the position data at the two first collection points of time with the shortest time interval from the second collection point of time can be determined as the position data of the radar device at the second collection point of time.

Illustratively, as shown in FIG. 2, points a1, b1, c1, d1, e1, f1, g1 above the horizontal line in FIG. , there is corresponding position data, and a2, b2, c2, d2, e2 below the horizontal line all represent second collection time points, and each second collection time point has corresponding point cloud data. Here, the first collection time points a1, d1, and g1 all have corresponding second collection time points. The position data corresponding to a1 is determined as the position data of the radar device at the second collection time point a2, and the position data corresponding to the first collection time point d1 is determined as the position data of the radar device at the second collection time point c2. Then, the position data corresponding to the first collection time point g1 is determined as the position data of the radar device at the second collection time point e2. For a second collection time without a corresponding first collection time, e.g. b2, select the first collection time b1 and c1 closest to the second collection time b2, then select the first collection time b1 and c1 The average value of the corresponding position data is determined as the position data of the radar device at the second collection point b2. In this way, position data of the radar device at the second collection time d2 can be calculated.

Here, when calculating the position data of the radar device at the two first collection points, the average value of the data of each item included in the position data can be obtained. For example, if the location data includes longitude, latitude, and height data, calculate the mean longitude, mean latitude, and mean height, respectively, then calculate the mean longitude, mean latitude, and mean latitude values, and height averages can be used as position data after averaging.

When the position data of the radar device at the second collection time points are determined based on the position data corresponding to the two first collection time points, the first collection time points and the second collection time points are not evenly distributed. , when calculating the position data of the radar device at the second collection time when there is no corresponding first collection time, it can be determined based on the interval between the two first collection time and the second collection time.

Exemplarily, as shown in FIG. 3, if points a and b are the first collection time and point c is the second collection time, the position data of the radar device at the second collection time c is determined. Then, according to the distance between the first collection time a and the second collection time and the distance between the first collection time b and the second collection time c, the weight corresponding to the first collection time a and the position data at the second collection time are The corresponding weights are respectively determined, then the position data at the first collection time a and the position data at the first collection time b are weighted and added according to their corresponding weights to obtain the position of the radar device at the second collection time c data can be obtained.

According to the above embodiment, the position data of the radar device at each second collection time can be determined, and interference due to different data collection frequencies of the radar device and the positioning device can be avoided.

For step 103,
Ground parameter information representing the ground includes a normal vector and an intercept. Specifically, when determining the ground parameter information representing the ground based on the point cloud data, the method shown in FIG. 4 can be referred to, which includes steps 401 to 405 below.

In step 401, plane fitting is performed based on the point cloud data to obtain fitting plane information.

Here, when the plane fitting is performed based on the point cloud data, the plane fitting can be performed based on the Random sample consensus (RANSAC) algorithm. Specifically, point cloud data with a preset ratio can be randomly selected from the point cloud data, and plane fitting can be performed based on the randomly selected point cloud data.

At step 402, the distance between each radar scan point and the plane indicated by the fitting plane information is determined based on the coordinate information of the radar scan points in the point cloud data.

In step 403, point cloud data filtering is performed based on the distance between each determined radar scan point and the plane indicated by the fitting plane information.

When determining the ground parameter information, it is necessary to fit the ground according to the point cloud data, so when performing the point cloud data screening, the radar scanning points that are not on the ground are filtered. Specifically, when performing point cloud data selection based on the distance between each determined radar scanning point and the plane indicated by the fitting plane information, the distance to the plane indicated by the fitting plane information is Radar scan points beyond a preset distance range can be filtered.

At step 404, it is determined whether a preset iteration condition is met.

If the preset iteration condition is satisfied, execute step 405;
If the iteration condition is not met, the process returns to step 401 to continue sorting the point cloud data based on the sorted point cloud data.

Here, the preset iteration condition is reaching a preset number of iterations or, when performing point cloud data screening in step 403, all the radar scan points and the plane indicated by the fitting plane information. is within a preset distance range.

At step 405, ground parameter information is determined based on the final screened point cloud data.

In the above method, the point cloud data is selected multiple times by iterative fitting method, and then the ground parameter information is determined based on the finally selected point cloud data to improve the accuracy of the ground parameter information. Let

When the ground parameter information is determined based on the final screened point cloud data, the plane fitting is performed again based on the final screened point cloud data, and then the ground parameters of the fitted plane determining information, where the ground parameter information includes normal vectors and intercepts.

In one possible embodiment, to improve the efficiency of data culling, perform at least one data culling process of processes 1-2 below before performing plane fitting based on the point cloud data. can also

In process 1, a downsampling process is performed on the point cloud data so that the distribution density of radar scanning points in the point cloud data satisfies preset conditions.

Here, since the density of point cloud data affects the efficiency of plane fitting, downsampling process is performed on the point cloud data, so as to improve the efficiency of point cloud data selection, and furthermore, the ground parameter information Improve decision efficiency. Illustratively, a downsampling process can be performed on the point cloud data via a voxel filtering method.

In process 2, the target coordinate range is determined according to the preset mounting height of the radar device, and the radar scanning points whose corresponding coordinate information is located within the target coordinate range are screened.

The preset mounting height of the radar system is the height of the radar system from the ground, and when establishing the world coordinate system, the location point of the radar system is used as the starting point of the Z-axis, and then A target coordinate range is determined according to the preset mounting height of the radar device, and the target coordinate range can be used to represent the height range of the radar scanning point located on the ground in the Z-axis, and the corresponding By selecting the radar scanning points whose coordinate information is located within the target coordinate range, it is possible to roughly select the point cloud data, further improve the efficiency of the point cloud data selection, and improve the determination efficiency of the ground parameter information. can.

The pose data of the radar system includes coordinates of the radar system in an established world coordinate system, as well as at least one of pitch, yaw, and roll angles. . When determining the initial pose data of the radar device based on the position data of the radar device at each second acquisition time, the position data includes latitude, longitude and height, so that the position of the radar device in three-dimensional space is determined. Therefore, after establishing the world coordinate system in three-dimensional space, the pose data of the radar device in the world coordinate system are also determined.

Considering that in the data collection process, the autonomous driving device may not run on the same ground plane, the ground parameter information between different ground planes may be different, so in another embodiment of the present application, The point cloud data can also be divided into a plurality of collection intervals according to the height data in the position data and the correspondence between the position data and the point cloud data, each collection interval including a plurality of second collection time points, each Within the collection interval, it can be assumed that the vehicles run on the same ground plane.

In one possible embodiment, when dividing the point cloud data into a plurality of collection intervals according to the height data in the position data, the height data is collected based on the height value of the height data in each collection interval. and then determining at least one extremum point of the height data retained after filtering, using the acquisition time point corresponding to each extremum point as a dividing point of the acquisition interval, and dividing the point cloud data into a plurality of of collection intervals.

Considering that the height data collected by the autonomous driving device during driving may be affected by obstacles on the ground, such as vibrations caused by obstacles, first, the height data collected by the autonomous driving device Data can be filtered. Specifically, height values whose fluctuation frequency is higher than the set frequency or height values whose fluctuation amplitude is lower than the set amplitude can be filtered.

Exemplarily, as shown in FIG. 5, the abscissa is the second collection time point, the ordinate is the height data of the position data corresponding to each second collection time point, and the line graph points are extreme points. and the second collection time points between the two extrema points is one collection interval, and each collection interval includes a plurality of second collection time points, as well as a plurality of frames of point cloud images, such as Contains the corresponding point cloud data.

After dividing the point cloud data into a plurality of collection intervals, for each collection interval, based on the point cloud data within the collection interval, determine ground parameter information corresponding to the collection interval; initial pose data of the radar device corresponding to the collection interval can be determined based on the position data of the radar device when collecting the point cloud data of .

After determining the ground parameters for each acquisition interval and the initial pose data of the radar device corresponding to the acquisition interval, the positioning device and the External parameter data representing the relative positional relationship with the radar device can be adjusted.

In one possible embodiment, when adjusting extrinsic parameter data based on the ground parameter information and the initial pose data corresponding to each said acquisition interval, the method for adjusting extrinsic parameter data shown in FIG. , the method includes steps 601-602 below.

In step 601, optimized pose data corresponding to each acquisition interval is determined based on the ground parameter information and the initial pose data corresponding to each acquisition interval.

Specifically, when determining the optimized pose data corresponding to each collection interval based on the ground parameter information and the initial pose data corresponding to each collection interval, for any collection interval, the collection Determine optimized pose data that minimizes the value of the objective function based on the ground parameter information corresponding to the interval and the initial pose data, where the objective function is the absolute value of the difference between the ground parameter information before and after the pose data optimization and the absolute value of the difference between the pose data before and after the pose data optimization is the sum of the values.

Exemplarily, the objective function can be represented by the following formula.

は、点群画像のｉ番目のフレームの初期ポーズデータを表し、

は、点群画像のｉ番目のフレームの初期ポーズデータに対応する、計算される最適化されたポーズデータを表し、

は、最適化前の地上パラメータ情報、即ち、当該収集間隔内の点群データに基づいて決定された地面パラメータ情報を表し、

は、最適化された地面パラメータ情報を表し、Ｎは、当該収集間隔内の点群データに含まれる点群画像の数、即ち、当該収集間隔内の点群データに含まれる第２収集時点の数を表す。 here,

represents the initial pose data of the i-th frame of the point cloud image,

represents the calculated optimized pose data corresponding to the initial pose data of the i-th frame of the point cloud image,

represents the ground parameter information before optimization, that is, the ground parameter information determined based on the point cloud data within the collection interval,

represents the optimized ground parameter information, and N is the number of point cloud images included in the point cloud data within the collection interval, i.e., the number of second collection time points included in the point cloud data within the collection interval. represents a number.

For any acquisition interval, at each second acquisition time point of the acquisition interval, a set of point cloud data is acquired, the set of point cloud data can constitute one frame of point cloud image, and the data acquisition process contains a plurality of second acquisition time points, corresponding to a plurality of sets of point cloud data, each set of point cloud data corresponding to a frame of point cloud images, and for each set of point cloud data, By computing the initial pose data corresponding to the point cloud data of the set, working with the above equation, and solving this equation with the least squares method, the optimized pose corresponding to the initial pose data of the point cloud image for each frame data can be obtained.

Furthermore, optimized ground parameter information can also be obtained by solving the above equations, but the optimized ground parameter information is independent of the adjustment of the external parameter data.

Illustratively, for N=3, the above equation can be expanded as follows.

、第２フレームの点群画像に対応する最適化されたポーズデータである

、及び第３フレームフレームの点群画像に対応する、最適化されたポーズデータである

、及び最適化された地面パラメータ情報である

をそれぞれ計算できる。 is the optimized pose data corresponding to the point cloud image of the first frame via the least squares method

, is the optimized pose data corresponding to the point cloud image of the second frame

, and the optimized pose data corresponding to the point cloud image of the third frame frame

, and the optimized ground parameter information is

can be calculated respectively.

及び

は、六次元ベクトルであり得、ベクトル内の各要素は、世界座標系でのレーダ装置の座標ｘ、ｙ、ｚ、ピッチ角、ヨー角、及びロール角をそれぞれ表し、

及び

は、四次元ベクトルであり得、ベクトルの各要素は、法線ベクトルの値及び切片を表し、法線ベクトルは、三次元ベクトルであることに留意されたい。

as well as

can be a six-dimensional vector, each element in the vector representing the coordinates x, y, z, pitch angle, yaw angle, and roll angle of the radar device in the world coordinate system, respectively;

as well as

can be a four-dimensional vector, each element of the vector representing the value and intercept of the normal vector, which is a three-dimensional vector.

At step 602, the extrinsic parameter data are adjusted based on the initial pose data corresponding to each acquisition interval and the optimized pose data.

Specifically, when adjusting the extrinsic parameter data based on the initial pose data corresponding to each acquisition interval and the optimized pose data, first, the initial pose data corresponding to each acquisition interval and the optimized pose data determining a first average pose variation within the acquisition interval based on the collected pose data; and then determining a second average pose variation corresponding to a plurality of acquisition intervals based on the first average pose variation within each acquisition interval. A variation can be determined and the extrinsic parameter data can be adjusted based on the second average pose variation.

In one possible embodiment, when determining the first average pose variation for a given acquisition interval, for each frame point cloud image corresponding to point cloud data within that acquisition interval, for that frame's point cloud image: determining a pose variation between the corresponding initial pose data and the optimized pose variation pose data; Average pose variation can be calculated.

In the case of a point cloud image of an arbitrary frame, when calculating the pose variation between the initial pose data corresponding to the point cloud image of the frame and the optimized pose variation pose data, the latitude and longitude of the initial pose data and The height, the latitude and longitude of the optimized initial pose data, and the amount of change corresponding to the height can be calculated respectively.

In one possible embodiment, when adjusting the extrinsic parameter data based on the second average pose variation, the product of the second average pose variation and the extrinsic parameter data before adjustment is used as the adjusted extrinsic parameter data. Available.

According to the above method, the ground parameter information and the initial pose data of the radar device are determined according to the positioning data collected by the positioning device and the radar data collected by the radar device; Based on the pose data, the extrinsic parameter data can be adjusted, the process of adjusting the extrinsic parameter data does not require using other equipment to perform calibration with respect to the relative position of the positioning device and the radar device, and the extrinsic parameter data The data accuracy and adjustment efficiency are improved, and the efficiency and accuracy in determining the pose data of the radar system are higher.

It is obvious to those skilled in the art that in the method in the above specific embodiments, the described order of each step does not restrict the implementation process by a strict order of execution, and the specific order of execution of each step is its function. and possible internal logic.

Based on the same technical concept, the embodiments of the present invention further provide a data processing apparatus corresponding to the data processing method, and the principle of the apparatus in the embodiments of the present invention for solving the problem is , the implementation of the device can refer to the above method embodiments, and will not be repeated here.

Referring to FIG. 7, an embodiment of the present invention provides a data processing device, which comprises an acquisition module 701, a first determination module 702, a second determination module 703 and an adjustment module 704,
the acquisition module 701 is configured to acquire positioning data collected by a positioning device and point cloud data collected by a radar device, wherein the positioning device and the radar device are deployed in the same vehicle;
the first determination module 702 is configured to determine position data of the radar device when collecting the point cloud data based on the positioning data and the point cloud data;
a second determination module 703 is configured to determine ground parameter information representing the ground based on the point cloud data and to determine initial pose data of the radar device based on the position data;
The adjustment module 704 is configured to adjust extrinsic parameter data representing a relative positional relationship between the positioning device and the radar device based on the ground parameter information and the initial pose data.

In one possible embodiment, said point cloud data comprises coordinate information of a plurality of radar scan points. Correspondingly, when the second determination module 703 determines ground parameter information representing the ground surface based on the point cloud data, the second determining module 703 performs plane fitting based on the point cloud data to obtain fitting plane information is obtained, based on the coordinate information of the radar scanning points in the point cloud data, the distance between each radar scanning point and the plane indicated by the fitting plane information is determined, and each determined radar scanning point and , based on the distance to the plane indicated by said fitting plane information, perform point cloud data screening, and based on the screened point cloud data, until a preset iteration condition is reached, returning to the step of performing plane fitting, and finally determining the ground parameter information based on the filtered point cloud data.

In one possible embodiment, the second determination module 703 further performs a downsampling operation on the point cloud data to obtain the point cloud data before performing plane fitting based on the point cloud data. At least one of: ensuring that the distribution density of radar scanning points in the data satisfies a preset condition; and selecting radar scanning points whose corresponding coordinate information is located within the target coordinate range. A screening process is performed, wherein the target coordinate range is configured to be determined according to a preset mounting height of the radar device.

In one possible embodiment, said position data includes height data. Correspondingly, the second determining module 703 further divides the point cloud data into a plurality of collection intervals based on the height data in the position data and the corresponding relationship between the position data and the point cloud data. configured to

In one possible embodiment, the second determination module 703 divides the point cloud data into a plurality of collection intervals based on height data in the position data and correspondence between the position data and the point cloud data. When splitting, filtering the height data to determine at least one extreme point of the height data retained after filtering, and using the collection time point corresponding to each extreme point as a division point for the collection interval. , configured to divide the point cloud data into a plurality of acquisition intervals.

In one possible embodiment, the second determination module 703 determines ground parameter information representing the ground based on the point cloud data, and determines initial pose data of the radar device based on the position data. When determining, for each collection interval, determining ground parameter information corresponding to the collection interval based on the point cloud data within the collection interval; It is configured to determine initial pose data of the radar device corresponding to the acquisition interval based on position data of the radar device. Correspondingly, the adjustment module 704 adjusts the external parameter data representing the relative positional relationship between the positioning device and the radar device based on the ground parameter information and the initial pose data, each acquisition It is configured to adjust external parameter data representing a relative positional relationship between the positioning device and the radar device based on the ground parameter information corresponding to the distance and the initial pose data.

In one possible embodiment, the adjustment module 704 generates extrinsic parameter data representing the relative positional relationship between the positioning device and the radar device based on the ground parameter information and the initial pose data corresponding to each acquisition interval. , determining optimized pose data corresponding to each acquisition interval based on the ground parameter information and the initial pose data corresponding to each acquisition interval, and adjusting the initial pose data corresponding to each acquisition interval and adjusting the extrinsic parameter data based on the optimized pose data.

In one possible embodiment, when the adjustment module 704 determines optimized pose data corresponding to each acquisition interval based on the ground parameter information and the initial pose data corresponding to each acquisition interval: In the case of an arbitrary collection interval, the pose data that minimizes the value of the objective function is determined based on the ground parameter information corresponding to the collection interval and the initial pose data, and the pose that minimizes the value of the objective function is determined. data is used as the optimized pose data corresponding to the collection interval, where the objective function is the absolute value of the difference of the ground parameter information before and after the pose data optimization and the It is the sum of the absolute values of the pose data differences.

In one possible embodiment, the adjustment module 704 adjusts the extrinsic parameter data based on the initial pose data and optimized pose data corresponding to each acquisition interval. determining a first average pose variation within the acquisition interval based on the initial pose data and the optimized pose data; and based on the first average pose variation within each acquisition interval, the plurality of acquisitions It is configured to determine a second average pose variation corresponding to an interval and adjust the extrinsic parameter data based on the second average pose variation.

In one possible embodiment, when the adjustment module 704 adjusts the extrinsic parameter data based on the second average pose variation, the second average pose variation and the extrinsic parameter data before adjustment are: product as adjusted extrinsic parameter data.

In one possible embodiment, the positioning data comprises position data each collected at a plurality of first collection time points, and the point cloud data comprises point cloud data each collected at a plurality of second collection time points. and determining position data of the radar device when collecting the point cloud data based on the positioning data and the point cloud data. Correspondingly, the first determining module 702, based on the positioning data and the point cloud data, determines the position data of the radar device when collecting the point cloud data, each of the second In the case of collection time points, if there is a target first collection time point that overlaps with the second collection time point among the plurality of first collection time points, the target first collection time point is determined based on the position data corresponding to the target first collection time point. Position data of the radar device at a second collection time is determined, and if the target first collection time does not exist among the plurality of first collection time points, the second of the plurality of first collection time points is determined. The position data of the radar device at the second collection time is determined based on the position data respectively corresponding to two adjacent first collection time points with the shortest time intervals from the collection time, Two collection time points are located between said two adjacent first collection time points.

For the description of the processing flow of each module in the device and the interaction flow between each module, please refer to the relevant descriptions in the method embodiments and will not be repeated here.

In some embodiments, the functions or modules included in the apparatus provided by the embodiments of the present invention can be used to perform the methods described in the above method embodiments, and the specific implementation thereof can refer to the description of the above method embodiment, which is not repeated here for the sake of brevity.

Based on the same technical concept, embodiments of the present invention further provide a computer device. Referring to FIG. 8, the computer device comprises a processor 801, a memory 802 and a bus 803. Here, the memory 802 comprises a memory 8021 and an external memory 8022 and is configured to store execution instructions, and the memory 8021 is also referred to as internal memory to temporarily store computational data in the processor 801 and store the data. Configured to exchange with an external memory 8022 such as a hard disk, the processor 801 is configured to exchange data with the external memory 8022 via the memory 8021 .

When the computer device 800 is running, the processor 801 and the memory 802 communicate via the bus 803 to provide the processor 801 with positioning data collected by the positioning device and point cloud data collected by the radar device. The positioning device and the radar device are mounted on the same vehicle, and based on the positioning data and the point cloud data, determine position data when the radar device collects the point cloud data, Determining ground parameter information representing the ground based on the group data; determining initial pose data of the radar device based on the position data; determining the initial pose data of the radar device based on the ground parameter information and the initial pose data; An instruction such as adjusting external parameter data representing the relative positional relationship between the positioning device and the radar device is executed.

Here, the specific processing process performed by the processor 801 can refer to the description in the above method embodiments and will not be repeated here.

An embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, performs the method of processing data as described in the above method embodiment. Here, the storage medium may be a volatile or non-volatile computer-readable storage medium.

A computer program product of a data processing method provided by an embodiment of the present invention includes computer readable code, and when said computer readable code executes in an electronic device, a processor in said computer device performs the method described above. Execute the data processing method described in the example. The computer program product may be specifically implemented in the form of hardware, software, or a combination thereof. In one alternative embodiment, the computer program product is embodied as a computer storage medium, and in another alternative embodiment, the computer program product is embodied as a software product, such as a Software Development Kit (SDK). be.

As can be clearly understood by those skilled in the art, for convenience and brevity of explanation, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the foregoing method embodiments, It will not be repeated here. It should be appreciated that in some of the embodiments provided by the present invention, the disclosed systems, devices and methods can be implemented in other manners. The embodiments of the apparatus described above are only exemplary, for example, the division of the units is only the division of logical functions, and other division methods can be adopted in actual implementation. , for example, multiple units or components may be combined, integrated into another system, or some features may be ignored or not performed. It should be noted that the mutual or direct couplings or communication connections shown or discussed may be implemented using any number of communication interfaces, and the indirect couplings or communication connections between devices or units may be electrical, mechanical or mechanical. or other forms.

Units described as separate parts above may or may not be physically separate, and parts shown as units may or may not be physical units and may or may not be in one place. It may be located or distributed among multiple network units. According to actual needs, some or all of the units therein can be selected to achieve the purpose of the technical solution in this embodiment.

Furthermore, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, and two or more units may be combined into one processing unit. may be integrated into one unit.

When the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the essential part of the technical solution of the present invention, that is, the part that contributes to the prior art, or the whole or part of the technical solution is embodied in the form of a software product. The computer software product described in each embodiment of the present invention can be stored on a single storage medium and stored on a single computer device (which can be a personal computer, a server, or a network device). It contains some instructions for performing all or part of the steps of the method. The aforementioned storage media include various media capable of storing program code, such as U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), magnetic disk or optical disk. include.

Finally, the above examples are only implementations of the embodiments of the present invention, and are used to describe the technical solutions in the examples of the present invention, and the protection scope of the examples of the present invention is limited thereto. not. Although embodiments of the present invention have been described in detail with reference to the above embodiments, those skilled in the art will appreciate the techniques described in the above embodiments within the technical scope disclosed in the embodiments of the present invention. The technical solution may be modified or easily conceivable changes, or part of its technical features may be equivalently replaced. All shall fall within the protection scope of the embodiments of the present invention without departing from the spirit and scope of the technical solutions of the embodiments of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims (14)

A method of processing data, comprising:
obtaining positioning data collected by a positioning device and point cloud data collected by a radar device, wherein the positioning device and the radar device are deployed in the same vehicle;
Determining position data of the radar device when collecting the point cloud data based on the positioning data and the point cloud data;
determining ground parameter information representing the ground based on the point cloud data and determining initial pose data of the radar device based on the position data;
and adjusting external parameter data representing a relative positional relationship between the positioning device and the radar device based on the ground parameter information and the initial pose data. The point cloud data includes coordinate information of a plurality of radar scanning points, and determining ground parameter information representing the ground based on the point cloud data includes:
performing plane fitting based on the point cloud data to obtain fitting plane information;
determining a distance between each radar scanning point and a plane indicated by the fitting plane information based on the coordinate information of the radar scanning points in the point cloud data;
performing point cloud data culling based on the distance between each determined radar scan point and a plane indicated by the fitting plane information, and preset iterations based on the culled point cloud data; returning to performing plane fitting based on the point cloud data until a condition is reached;
determining the ground parameter information based on the final screened point cloud data;
A data processing method according to claim 1 . Before performing plane fitting based on the point cloud data,
performing a downsampling process on the point cloud data such that a distribution density of radar scan points in the point cloud data satisfies a preset condition;
screening radar scan points whose corresponding coordinate information is located within a target coordinate range, wherein the target coordinate range is determined by a preset mounting of the radar device. determine according to the height,
3. The data processing method according to claim 2. The position data includes height data, and the method for processing the data includes:
dividing the point cloud data into a plurality of collection intervals based on height data of the position data and a correspondence relationship between the position data and the point cloud data;
A data processing method according to claim 1 . Dividing the point cloud data into a plurality of collection intervals based on the height data of the position data and the correspondence relationship between the position data and the point cloud data includes:
filtering the height data to determine at least one extreme point of the height data retained after filtering;
dividing the point cloud data into a plurality of collection intervals using collection time points corresponding to each extreme point as collection interval division points;
5. The data processing method according to claim 4. Determining ground parameter information representing the ground based on the point cloud data and determining initial pose data of the radar device based on the position data includes:
For each said collection interval,
determining ground parameter information corresponding to the collection interval based on the point cloud data within the collection interval; and
determining initial pose data of the radar device corresponding to the collection interval based on position data of the radar device when collecting point cloud data within the collection interval;
Adjusting external parameter data representing a relative positional relationship between the positioning device and the radar device based on the ground parameter information and the initial pose data,
adjusting external parameter data representing a relative positional relationship between the positioning device and the radar device based on the ground parameter information and the initial pose data corresponding to each of the acquisition intervals;
5. The data processing method according to claim 4. Adjusting external parameter data representing a relative positional relationship between the positioning device and the radar device based on the ground parameter information and the initial pose data corresponding to each of the acquisition intervals,
determining optimized pose data corresponding to each acquisition interval based on the ground parameter information and the initial pose data corresponding to each acquisition interval;
adjusting the extrinsic parameter data based on the initial pose data corresponding to each acquisition interval and the optimized pose data;
7. A data processing method according to claim 6. Determining optimized pose data corresponding to each acquisition interval based on the ground parameter information and the initial pose data corresponding to each acquisition interval comprises:
For any said collection interval,
determining pose data that minimizes a value of an objective function based on the ground parameter information corresponding to the acquisition interval and the initial pose data;
using pose data that minimizes the value of the objective function as optimized pose data corresponding to the acquisition interval;
The objective function is the sum of the absolute value of the difference between the ground parameter information before and after the pose data optimization and the absolute value of the difference between the pose data before and after the pose data optimization.
The data processing method according to claim 7. adjusting the extrinsic parameter data based on the initial pose data and the optimized pose data corresponding to each of the acquisition intervals;
based on the initial pose data and the optimized pose data corresponding to each acquisition interval, determining a first average pose variation within that acquisition interval;
determining a second average pose variation corresponding to the plurality of acquisition intervals based on the first average pose variation within each of the acquisition intervals;
adjusting the extrinsic parameter data based on the second average pose variation;
The data processing method according to claim 7. Adjusting the extrinsic parameter data based on the second average pose variation includes:
using the product of the second average pose variation and the extrinsic parameter data before adjustment as adjusted extrinsic parameter data;
A data processing method according to claim 9 . The positioning data includes position data each collected at a plurality of first collection time points, the point cloud data includes point cloud data each collected at a plurality of second collection time points, and the positioning data and the points Determining position data of the radar device when collecting the point cloud data based on the cloud data includes:
For each said second collection point,
If there is a target first collection time point that overlaps with the second collection time point among the plurality of first collection time points, based on the position data corresponding to the target first collection time point, at the second collection time point determining position data of the radar device of
When the target first collection time point does not exist among the plurality of first collection time points, two adjacent first collection times having the shortest time interval from the second collection time point among the plurality of first collection time points Determining the position data of the radar device at the second collection time points based on the position data respectively corresponding to the time points, wherein the second collection time points are between the two adjacent first collection time points. located in and including
A data processing method according to claim 1 . A data processing device,
an acquisition module configured to acquire positioning data collected by a positioning device and point cloud data collected by a radar device, wherein the positioning device and the radar device are deployed in the same vehicle; a module;
a first determination module configured to determine position data of the radar device when collecting the point cloud data based on the positioning data and the point cloud data;
a second determination module configured to determine ground parameter information representing the ground based on the point cloud data and to determine initial pose data of the radar device based on the position data;
an adjustment module configured to adjust external parameter data representing a relative positional relationship between the positioning device and the radar device based on the ground parameter information and the initial pose data. processing equipment. A computing device comprising a processor, a memory and a bus, wherein the memory stores the processor-executable machine-readable instructions, and when the computing device executes, the processor and the memory are coupled to the bus. and performs the method of processing data according to any one of claims 1 to 11 when said processor executes said machine-readable instructions. A computer-readable storage medium having a computer program stored thereon, said computer performing the method of processing data according to any one of claims 1 to 11 when said computer program is executed by a processor. readable storage medium.

Images