WO2022061850A1

WO2022061850A1 - Point cloud motion distortion correction method and device

Info

Publication number: WO2022061850A1
Application number: PCT/CN2020/118270
Authority: WO
Inventors: 宫正; 李延召
Original assignee: 深圳市大疆创新科技有限公司
Priority date: 2020-09-28
Filing date: 2020-09-28
Publication date: 2022-03-31

Abstract

A point cloud motion distortion correction method and device, a radar, and a computer readable storage medium. The method comprises: obtaining a first point cloud frame and a second point cloud frame, the first point cloud frame and the second point cloud frame being point cloud frames at different moments of the same target scenario (110); extracting the same target object from the first point cloud frame and the second point cloud frame (120); estimating the motion speed of the target object according to a target object in the first point cloud frame and a target object in the second point cloud frame (130); and performing distortion correction on the target object in the first point cloud frame and/or the second point cloud frame according to the motion speed of the target object (140). According to the method, the motion speed of the target object is estimated according to a target object in the first point cloud frame and a target object in the second point cloud frame, and distortion correction is performed on the target object on the basis of the estimated motion speed, so that the distortion of a moving target can be eliminated to the greatest extent, and the measurement result is corrected.

Description

Point cloud motion distortion correction method and device

Copyright notice

The disclosure of this patent document contains material that is subject to copyright protection. This copyright belongs to the copyright owner. The copyright owner has no objection to the reproduction by anyone of the patent document or the patent disclosure as it exists in the official records and archives of the Patent and Trademark Office.

technical field

The present application relates to the technical field of photoelectric measuring instruments, and more particularly, to a method and device for correcting point cloud motion distortion.

Background technique

With the development of radar technology, more and more attention is paid to the detection of moving targets. Since each point cloud in a frame of point cloud output by the scanning lidar is not obtained by scanning the moving target at the same time, ghosting occurs in the point cloud frame, resulting in motion distortion.

Therefore, how to eliminate motion distortion is a problem that needs to be solved.

SUMMARY OF THE INVENTION

The present application proposes a point cloud motion distortion correction method and device, which can eliminate the distortion of the moving target to the greatest extent and correct the measurement result.

A first aspect provides a point cloud motion distortion correction method, comprising: acquiring a first point cloud frame and a second point cloud frame, the first point cloud frame and the second point cloud frame are for the same target scene point cloud frames at different times; extract the same target object from the first point cloud frame and the second point cloud frame; according to the target object in the first point cloud frame and the second point cloud frame Estimate the movement speed of the target object; perform distortion correction on the target object in the first point cloud frame and/or the second point cloud frame according to the movement speed of the target object.

In a second aspect, a point cloud motion distortion correction device is provided, including: a memory and a processor; the memory is used for storing program codes; the processor calls the program codes, and when the program codes are executed, is used for Perform the following operations: obtain a first point cloud frame and a second point cloud frame, the first point cloud frame and the second point cloud frame are point cloud frames at different times for the same target scene; Extracting the same target object from the point cloud frame and the second point cloud frame; estimating the movement speed of the target object according to the target object in the first point cloud frame and the target object in the second point cloud frame; Distortion correction is performed on the target object in the first point cloud frame and/or the second point cloud frame according to the moving speed of the target object.

In a third aspect, a radar is provided, including a processor and a memory. The memory is used for storing a computer program, and the processor is used for calling and running the computer program stored in the memory to execute the method in the above-mentioned first aspect or each implementation manner thereof.

In a fourth aspect, a chip is provided for implementing the method in the above-mentioned first aspect or each of its implementation manners.

Specifically, the chip includes: a processor for invoking and running a computer program from a memory, so that a device installed with the chip executes the method in the first aspect or each of its implementations.

In a fifth aspect, a computer-readable storage medium is provided for storing a computer program, the computer program comprising instructions for performing the method in the first aspect or any possible implementation of the first aspect.

In a sixth aspect, a computer program product is provided, comprising computer program instructions, the computer program instructions causing a computer to execute the method in the first aspect or each implementation manner of the first aspect.

In a seventh aspect, a computer program is provided, which, when run on a computer, causes the computer to execute the method in the first aspect or any possible implementation manner of the first aspect.

In the solution provided by the present application, by estimating the movement speed of the target object according to the target object in the first point cloud frame and the target object in the second point cloud frame, and performing distortion correction on the target object based on the estimated movement speed, it is possible to Eliminate the distortion of moving objects to the greatest extent and correct the measurement results.

Description of drawings

The accompanying drawings used in the embodiments will be briefly introduced below.

FIG. 1 is a point cloud motion distortion correction method provided by an embodiment of the present application.

FIG. 2 is a schematic diagram of a target scene provided by an embodiment of the present application.

FIG. 3 is a schematic diagram of a target scene provided by another embodiment of the present application.

FIG. 4 is a schematic diagram of a point cloud frame before and after motion distortion correction provided by an embodiment of the present application.

FIG. 5 is a schematic diagram of a point cloud frame before and after motion distortion correction provided by another embodiment of the present application.

FIG. 6 is a schematic diagram of a scene flow provided by an embodiment of the present application.

FIG. 7 is a schematic diagram of motion estimation based on a scene flow provided by an embodiment of the present application.

FIG. 8 is a schematic diagram for the target motion estimation shown in FIG. 7 .

FIG. 9 is a schematic diagram of motion estimation based on a scene flow provided by another embodiment of the present application.

FIG. 10 is a schematic diagram of motion estimation for the target shown in FIG. 9 .

FIG. 11 is a schematic diagram of motion estimation based on a scene flow provided by another embodiment of the present application.

FIG. 12 is a schematic diagram of motion estimation for the target shown in FIG. 11 .

FIG. 13 is a schematic diagram of a target scene provided by another embodiment of the present application.

FIG. 14 is a point cloud motion distortion correction method provided by another embodiment of the present application.

FIG. 15 is a point cloud motion distortion correction device provided by an embodiment of the present application.

FIG. 16 is a schematic structural diagram of a point cloud motion distortion correction device provided by another embodiment of the present application.

FIG. 17 is a schematic structural diagram of a chip according to an embodiment of the present application.

detailed description

The technical solutions in the embodiments of the present application are described below.

Unless otherwise specified, all technical and scientific terms used in the embodiments of the present application have the same meaning as commonly understood by those skilled in the technical field of the present application. The terminology used in this application is for the purpose of describing specific embodiments only and is not intended to limit the scope of the application.

This application is mainly applied to the scene of scanning lidar. Since each point cloud in a frame of point cloud output by scanning lidar is not obtained by scanning the moving target at the same time, ghosting occurs in the point cloud frame, which causes motion distortion.

It should be understood that the above scenarios are only illustrative, and the present application may also refer to other scenarios, which should not be particularly limited to the embodiments of the present application.

In essence, the source of point cloud motion distortion is caused by the fact that the scanning of the point cloud included in the target object is not at the same time. Regarding the scanning of moving targets by scanning lidar, there is currently no motion distortion correction algorithm for scanning radar. .

Therefore, the present application proposes a point cloud motion distortion correction method and device, which can eliminate the distortion of the moving target to the greatest extent and correct the measurement result.

The embodiments of the present application can be applied to 3S assumption scenarios, that is, Structural Consistency Assumption, Speed Consistency Assumption, and Small Motion Assumption.

Structural consistency assumption: the same object in adjacent frames has consistent features in structure;

Speed consistency assumption: the same object in adjacent frames has the same moving speed and direction;

Slight motion assumption: The target is moving at a small speed (eg, within 60km/h).

With the structural consistency assumption, we can estimate the speed of motion. Through the velocity consistency assumption, we can give velocity consistency constraints for the same target. Through the small motion assumption, we can perform data association and finally estimate the target motion state.

The following describes the point cloud motion distortion correction method 100 provided by the embodiment of the present application in detail with reference to FIG. 1 . The method can be applied to the scanning radar, and can also be applied to the server connected to the scanning radar. In some embodiments, some of the steps may be performed by a scanning radar and some by a server.

As shown in FIG. 1 , a point cloud motion distortion correction method 100 provided by an embodiment of the present application may include steps 110 - 140 .

110. Acquire a first point cloud frame and a second point cloud frame, where the first point cloud frame and the second point cloud frame are point cloud frames at different times for the same target scene.

The first point cloud frame and the second point cloud frame in this embodiment of the present application may be point cloud frames at different times for the same target scene, and the points included in the point cloud frames at different times for the same target scene here The number of clouds may be basically the same, or the number of point clouds included in point cloud frames at different times of the same target scene may vary greatly.

Exemplarily, as shown in FIG. 2 , it is a schematic diagram of a target scene provided by an embodiment of the present application.

Assuming that (a) in FIG. 2 and (b) in FIG. 2 are respectively the first point cloud frame and the second point cloud frame in the embodiment of the present application, it can be seen from the figure that the first point cloud frame and the second point cloud frame The number of point clouds included in the two point cloud frames is basically the same.

As shown in FIG. 3 , it is a schematic diagram of a target scene provided by another embodiment of the present application.

Assuming that (a) in FIG. 3 and (b) in FIG. 3 are respectively the first point cloud frame and the second point cloud frame in the embodiment of the present application, it can be seen from the figure that the first point cloud frame and the second point cloud frame The number of point clouds included in the two point cloud frames varies greatly, mainly due to the reduction in the number of the leftmost point cloud in (b) in Figure 3.

Optionally, in some embodiments, the first point cloud frame and the second point cloud frame are point cloud frames at adjacent moments of the same target scene.

The adjacent moments in this embodiment of the present application may be determined based on time units. For example, the first point cloud frame and the second point cloud frame may be point cloud frames of adjacent seconds, or point cloud frames of adjacent milliseconds, etc., No restrictions.

Exemplarily, the first point cloud frame may be the point cloud frame at the current moment (unit is seconds), and the second point cloud frame may be the point cloud frame at the next second at the current moment; or, the first point cloud frame may be the current point cloud frame. The point cloud frame of the moment (in milliseconds), and the second point cloud frame may be the point cloud frame of the next millisecond of the current moment.

120. Extract the same target object from the first point cloud frame and the second point cloud frame.

As is known to all, the target object may be composed of point clouds, and after the first point cloud frame and the second point cloud frame are acquired, the target object may be extracted from the first point cloud frame and the second point cloud frame, respectively.

It can be understood that, the target object in this embodiment of the present application may include one object, or may include multiple objects, which is not limited.

130. Estimate the movement speed of the target object according to the target object in the first point cloud frame and the target object in the second point cloud frame.

It can be understood that, in this embodiment of the present application, for a certain target object, the movement speed of the target object is estimated according to the target object in the first point cloud frame and the target object in the second point cloud frame.

In other words, when multiple target objects are extracted from the first point cloud frame and the second point cloud frame, when calculating the speed of the target object, the same target object is aimed.

140. Perform distortion correction on the target object in the first point cloud frame and/or the second point cloud frame according to the moving speed of the target object.

In the embodiment of the present application, performing distortion correction on the target object in the first point cloud frame and/or the second point cloud frame can be understood as correcting the coordinate position of the point cloud included in the target object.

Optionally, in some embodiments, performing distortion correction on the target object in the first point cloud frame and/or the second point cloud frame according to the moving speed of the target object includes: Determine the distortion coefficient of the target object according to the movement speed of the target object; perform distortion correction on the target object in the first point cloud frame and/or the second point cloud frame according to the distortion coefficient.

Optionally, in some embodiments, the distortion coefficients include rotational interpolation distortion coefficients and linear motion interpolation distortion coefficients.

In the embodiment of the present application, after estimating the movement speed of the target object according to the target object in the first point cloud frame and the target object in the second point cloud frame, the distortion of the target object may be further determined according to the estimated movement speed coefficient, so as to perform distortion correction on the target object based on the distortion coefficient.

The distortion coefficients in the embodiments of the present application may include rotational interpolation distortion coefficients and linear motion interpolation distortion coefficients.

Among them, the rotation motion interpolation coefficient can use the slerp algorithm, that is, the spherical linear interpolation algorithm, which is a linear interpolation operation of the quaternion, which is mainly used to smooth the difference between two quaternions representing rotation. value.

The translational motion interpolation coefficient, that is, the motion is directly divided proportionally by the distortion coefficient (short-term uniform motion assumption).

It is pointed out above that the distortion correction can be performed on the target object in the first point cloud frame and/or the second point cloud frame according to the distortion coefficient, and the specific implementation process will be described below.

Optionally, in some embodiments, performing distortion correction on the target object in the first point cloud frame and/or the second point cloud frame according to the distortion coefficient includes: for the first point cloud frame and/or the point cloud of the target object in the second point cloud frame, according to the product of the rotation interpolation distortion coefficient of the point cloud and the coordinate position of the point cloud, and the linear motion interpolation distortion of the point cloud The sum of the coefficients determines the corrected coordinate position of the point cloud.

Optionally, in some embodiments, the determining the distortion coefficient of the target object according to the movement speed of the target object includes: determining the distortion coefficient according to the movement speed of the target object and an interpolation coefficient, the The interpolation coefficient is calculated based on the timestamps of the point clouds in the first point cloud frame and the second point cloud frame.

As described above, the motion distortion correction for the target object in the first point cloud frame and the second point cloud frame is essentially the point cloud included in the target object in the first point cloud frame and the second point cloud frame. The coordinate position is corrected.

In the specific implementation process, the motion distortion correction of the target object can be realized through the following steps.

The first step: Initialize the read-in point cloud and the corresponding timestamp, as well as the motion direction delta_t and the speed size delta_q.

Among them, the process of initializing the read-in point cloud is essentially the coordinate position of the read-in point cloud, the timestamp of the corresponding point cloud, and the speed of the target object.

Step 2: Calculate the interpolation coefficient _si of each point cloud according to the first and last points and the timestamp of each point, as shown in formula (1).

Among them, s _i represents the interpolation coefficient of the ith point cloud, s _i ∈ [0,1], t_i represents the timestamp of the ith point cloud, t_n represents the timestamp of the nth point cloud, and t_0 represents the 0th point cloud Timestamp of the point cloud.

Assuming that the first point cloud frame and the second point cloud frame include 1 target object, and the 1 target object consists of 10 point clouds, for any point cloud, the corresponding equation (1) can be used to calculate its corresponding Interpolation coefficients.

Assuming that the 0th point cloud is obtained at the 10th ms, and the nth point cloud is obtained at the 20th ms, for the 0th point cloud, then:

If the first point cloud is obtained at 11ms, then:

If the second point cloud is obtained at 12ms, then:

...

By analogy, if the 9th point cloud is obtained in the 20th ms, then:

Step 3: Calculate the corresponding rotational motion interpolation coefficient and translational motion interpolation coefficient according to the interpolation coefficient s _i and the forward and backward motion estimation t_0 and t_n of the point cloud cluster.

This paper calls the slerp function in the Eigen library to complete the corresponding calculation, as shown in formula (2).

in,

represents the rotational motion interpolation coefficient of the i-th point cloud,

are the head and tail vectors of the interpolation, respectively, and θ is

the angle between.

The translational motion interpolation coefficient can be calculated by formula (3), that is, the motion is directly divided proportionally by the distortion coefficient (short-term uniform motion assumption).

t_point_last _i = s _i *delta_t (3)

Among them, t_point_last _i represents the translational motion interpolation coefficient of the i-th point cloud.

①Calculation of the rotation motion interpolation coefficient of the point cloud

Still take the above first point cloud frame and the second point cloud frame including one target object, and the one target object is composed of 10 point clouds as an example, if

The lengths of are 3 and 2, respectively, and the angle between them is 30°. For the 0th point cloud, then:

For the first point cloud, then:

...

And so on, for the ninth point cloud, then:

②Calculation of the translational motion interpolation coefficient of the point cloud

Still take the above first point cloud frame and the second point cloud frame including 1 target object, and the 1 target object is composed of 10 point clouds as an example, and the speed of the target object composed of the 10 point clouds is 10m/s, for the 0th point cloud, then:

t_point_last ₀ =s ₀ *delta_t=1*10=10.00

For the first point cloud, then:

t_point_last ₁ =s ₁ *delta_t=0.9*10=9.00

...

And so on, for the ninth point cloud, then:

t_point_last ₉ =s ₉ *delta_t=0*10=0

Step 4: According to the rotation motion interpolation coefficient and the translation motion interpolation coefficient, the corresponding point is restored and aligned to the coordinate system of the last point of the frame according to this motion interpolation, and the calculation is completed, as shown in formula (4).

p_i_update=q_point_last*p_i+t_point_last (4)

Among them, p_i_update represents the coordinate position of the ith point cloud after updating, p_i represents the coordinate position of the ith point cloud before updating, and q_point_last is the above formula (2)

As described above, the rotational motion interpolation coefficient q_point_last and the translational motion interpolation coefficient t_point_last of the 0th point cloud obtained according to the above equations (2) and (3) are 2.00 and 10.00, respectively. For the 0th point cloud, assuming that the coordinate position before correction is (1, 4, 3), the corrected coordinate position can be obtained by the above formula (4), that is, the corrected coordinate position is (12, 18, 16 ).

The rotational motion interpolation coefficient q_point_last and translation motion interpolation coefficient t_point_last of the first point cloud obtained according to the above equations (2) and (3) are 2.13 and 9.00, respectively. For the first point cloud, assuming that the coordinate position before correction is (1, 6, 3), the corrected coordinate position can be obtained by the above formula (4), that is, the corrected coordinate position is (11.13, 21.78, 15.39 ).

Similarly, for other point clouds, the above method can also be used for correction, which is not repeated here for brevity.

It should be understood that the above examples are described with 10 point clouds as an example. In the actual process, the number of acquired point clouds may far exceed this value, but the specific implementation process is similar to the above examples. .

As shown in FIG. 4 , it is a schematic diagram of a point cloud frame before and after motion distortion correction according to an embodiment of the present application.

(a) in FIG. 4 is a schematic diagram of a point cloud frame before motion distortion correction, and (b) in FIG. 4 is a schematic diagram of a point cloud frame after motion distortion correction. Referring to Fig. 4(a) and Fig. 4(b), it can be seen that the object formed by the point cloud in this figure may be a car.

Among them, for the corrected point cloud, please refer to the point cloud in the area enclosed by the thicker dashed circle in Fig. 4(a) and Fig. 4(b), that is, the rear of the car. It can be clearly seen from the figure that the rear of the car in (a) in Figure 4 has obvious point cloud stacking phenomenon, and three distinct lines are formed at the rear of the car due to the stacking of point clouds. After the point cloud distortion correction, the coordinate position of the point cloud at the rear of the car in Figure 4(b) has obvious changes, and an obvious line is formed at the rear of the car at this time.

As shown in FIG. 5 , it is a schematic diagram of a point cloud frame before and after motion distortion correction according to another embodiment of the present application. (a) in FIG. 5 is a schematic diagram of a point cloud frame before motion distortion correction, and (b) in FIG. 5 is a schematic diagram of a point cloud frame after motion distortion correction.

Among them, for the corrected point cloud, please refer to the point cloud in (a) and (b) of FIG. 5 in the area enclosed by the thicker dashed circle, that is, the right part of the target object. It can be clearly seen from the figure that the right part of the target object in (a) in Figure 5 has obvious point cloud stacking phenomenon, and due to the stacking of point clouds, three lines are formed on the right part of the target object. Obvious lines. After the point cloud distortion correction, the coordinate position of the point cloud on the right part of the target object in (b) in Figure 5 has obvious changes. At this time, an obvious line is formed on the right part of the target object.

The solution provided by this application can further eliminate the distortion of the moving target by determining the distortion coefficient according to the movement speed of the target object and the interpolation coefficient of the point cloud, and performing distortion correction on the position of the point cloud included in the target object based on the distortion coefficient. Correct the measurement results.

Based on this, the process of performing distortion correction on the target object in the first point cloud frame and the second point cloud frame has been described above. Regarding the content of extracting the target object from the first point cloud frame and the second point cloud frame, See below.

Optionally, in some embodiments, the extracting the same target object from the first point cloud frame and the second point cloud frame includes: using a clustering algorithm to separate the first point cloud frame The point cloud in the point cloud frame and the point cloud in the second point cloud frame are clustered into at least one object; and the object in the first point cloud frame and the object in the second point cloud frame are associated.

The clustering algorithm in the embodiment of the present application may include an Euclidean clustering algorithm or a generalized Euclidean clustering algorithm, etc., which is not limited.

In addition, this application can also use other methods to extract at least one object from the first point cloud frame and the second point cloud frame, such as target detection method, etc. This application does not specifically limit this, as long as it can be extracted from the point cloud frame The methods of the object can all be applied to the embodiments of the present application.

In this embodiment of the present application, after at least one object is extracted from the first point cloud frame and the second point cloud frame by using a clustering algorithm, the extracted at least one object may be associated, that is, the first point cloud frame is associated with the second point cloud frame. The same objects in the point cloud frame are mapped.

In other words, for each object extracted from the first point cloud frame, the corresponding object can be found from at least one object extracted from the second point cloud frame.

Optionally, in some embodiments, the associating the object in the first point cloud frame with the object in the second point cloud frame includes: according to the object in the first point cloud frame The target point of , and the target point of the object in the second point cloud frame associate the object in the first point cloud frame with the object in the second point cloud frame, wherein the first point cloud frame The distance between the target point of the target object in the frame and the target point of the target object in the second point cloud frame is smaller than the distance to the target points of other objects in the second point cloud frame.

Optionally, in some embodiments, the target point is a center point or a center of gravity point.

It should be noted that, in some implementations, the target point may also be any point on the extracted target object, which is not limited.

It is pointed out above that the point cloud in the first point cloud frame and the point cloud in the second point cloud frame are respectively clustered into at least one object by using the clustering algorithm, that is, from the first point cloud frame and the second point cloud frame One or more objects may be included in the objects extracted in the cloud frame.

If the point clouds in the first point cloud frame and the second point cloud frame are clustered to form one object, since this application is carried out under the scenario of micro-motion assumption, the one of the two frames can be The objects are associated; if the point clouds in the first point cloud frame and the second point cloud frame are clustered to form multiple objects, it is necessary to associate the multiple objects in the two frames respectively.

In terms of specific implementation, the following process can be referred to to associate the target objects in the first point cloud frame and the second point cloud frame.

Taking the target point as the center point as an example, it is assumed that in a certain point cloud scene, there are n moving objects K _t0 at time t ₀ (which can be considered as the time corresponding to the first point cloud frame in this application),

At time _t1 (which can be considered as the time corresponding to the second point cloud frame in this application), there are m moving objects K _t1 ,

Since K _t0 and K _t1 may not be the same moving target, K _t0 and K _t1 can be associated with the target first:

After the target association, the movement speed of the target can be calculated:

Since the scene flow consists of different moving objects, the final scene flow can be obtained by traversing and calculating all moving objects:

In order to facilitate understanding of the solution of the present application, a brief introduction to the scene flow is given below.

Scene flow, the 3D version of optical flow, describes the changes of each point cloud in the point cloud two frames before and after. Current algorithms for scene flow estimation using point cloud data include flownet3D and HPLFlownet.

Among them, the core idea of flownet3D is: using pointnet++ as the basic module, extracting the point cloud features of the two frames before and after, merging and upsampling, and directly fitting the scene flow. The core idea of HPLFlownet is: using Bilateral Convolutional Layers as the basic module, extracting the point cloud features of the two frames before and after, merging and upsampling, and directly fitting the scene flow.

As shown in FIG. 6 , it is a schematic diagram of a scene flow according to an embodiment of the present application.

Referring to (a) in FIG. 6 , the small white circles represent point cloud 1 , and the small black circles represent point cloud 2 . Among them, the point cloud 1 and the point cloud 2 can be respectively the point clouds of the frame before and after the same target object, and the scene flow of the point cloud can be obtained by the motion estimation of the point cloud 1 and the point cloud 2, as shown in Figure 6 (b).

Wherein, (a) in FIG. 7 is the point cloud frame shot at time t0 (that is, the first point cloud frame in this application), and (b) in FIG. 7 is the point cloud frame shot at time t1 (that is, the first point cloud frame in this application) second point cloud frame in this application).

First, the target object can be obtained from the first point cloud frame and the second point cloud frame respectively. This application takes the Euclidean clustering method as an example. Referring to (a) in FIG. 7 , the center points O1 and Q1 are used as clusters respectively. Points are clustered. For the center point O1, the points whose distance O1 is smaller than the preset threshold may be clustered into one category; for the center point Q1, the points whose distance Q1 is smaller than the preset threshold may be clustered into one category. Through the above clustering algorithm, the target object A1 and the target object B1 shown in the figure can be obtained by clustering.

Similarly, referring to (b) in FIG. 7 , clustering is performed with center points O2 and Q2 as clustering points, respectively. For the center point O2, the points whose distance O2 is smaller than the preset threshold may be clustered into one category; for the center point Q2, the points whose distance Q2 is smaller than the preset threshold may be clustered into one category. Through the above clustering algorithm, the target object A2 and the target object B2 shown in the figure can be obtained by clustering.

Secondly, target association can be performed on the obtained target object. That is, for the cluster center at time t0, the nearest neighbor search is performed in the cluster target center set at time t1, and the point with the smallest distance is taken as the target correlation point.

Exemplarily, for the cluster center O1 (ie the center point O1 above), the clustering target center set at time t1 (ie the cluster center O2 (ie the center point O2 above), the cluster center Q2 (ie the center point O2 above) The nearest neighbor search is performed in the center point Q2)) above. It can be seen from the figure that the distance between the cluster center O2 and the cluster center O1 is smaller than the distance between the cluster center Q2 and the cluster center O1. Therefore, The cluster center O1 and the cluster center O2 can be associated, that is, the target object A1 at time t0 and the target object A2 at time t1 are the same target object.

For the cluster center Q1 (that is, the center point Q1 above), the nearest neighbor search is performed in the clustering target center set (that is, the cluster center O2, the cluster center Q2) at the time of t1. It can be seen from the figure that the clustering The distance between the center Q2 and the cluster center Q1 is smaller than the distance between the cluster center O2 and the cluster center Q1. Therefore, the cluster center Q1 and the cluster center Q2 can be associated, that is, the target object B1 at the time of t0 is different from the cluster center Q1. The target object B2 at time t1 is the same target object.

After completing the target association for the targets of different point cloud frames, further, the motion of the target object can be calculated. Since FIG. 7 is a schematic diagram of a scene of point cloud frames captured at different times, the target object in FIG. 7 can be processed first. Exemplarily, the same target object in FIG. 7 can be placed on the same horizontal line, that is, the target object A1 in (a) in FIG. 7 and the target object A2 in (b) in FIG. 7 are extracted and placed on the same horizontal line. The horizontal line, as shown in (a) in Figure 8; the target object B1 in (a) in Figure 7 and the target object B2 in (b) in Figure 7 are extracted and placed on the same horizontal line, as shown in Figure 8 shown in (b).

Referring to (a) in Figure 8, it can be seen that from time t0 to time t1, the target object A1 moves from O1 to the position of O2 (that is, the target object A2), and the displacement between O1 and O2 is denoted as

Then the movement speed of target A1 can be obtained by the ratio of displacement and time

(including magnitude and direction of velocity), i.e.

Among them, t is the duration from time t0 to time t1.

Referring to (b) in Figure 8, it can be seen that from time t0 to time t1, the target object B1 moves from Q1 to the position of Q2 (that is, the target object B2), and the displacement between Q1 and Q2 is denoted as

Then the movement speed of the target B1 can be obtained by the ratio of displacement and time

(including magnitude and direction of velocity), i.e.

Exemplarily, assuming that t in the above formula is 1s, obtained through measurement

Then the speed of the target object A1

Speed of target object B1

The direction of speed is westward.

It should be understood that the preset threshold in this embodiment of the present application may be a fixed value, or may be a continuously adjusted value, which is not specifically limited in the present application.

As shown in FIG. 9 , it is a schematic diagram of motion estimation based on a scene flow provided by another embodiment of the present application.

Wherein, (a) in FIG. 9 is the point cloud frame shot at time t0 (that is, the first point cloud frame in this application), and (b) in FIG. 9 is the point cloud frame shot at time t1 (that is, the first point cloud frame in this application) second point cloud frame in this application).

The present application takes the Euclidean clustering method as an example, and with reference to (a) in FIG. 9 , the center point O1 , the center point Q1 , and the center point P1 are respectively used for clustering to perform clustering. For the center point O1, the points whose distance O1 is less than the preset threshold can be clustered into one category; for the center point Q1, the points whose distance Q1 is less than the preset threshold can be clustered into one category; for the center point P1, the distance Points whose P1 is less than the preset threshold are clustered into one class. Through the above clustering algorithm, the target object A1, the target object B1 and the target object C1 in the figure can be obtained.

Similarly, referring to (b) in FIG. 9 , the center point O2 , the center point Q2 , and the center point P2 are respectively used as cluster points to perform clustering. For the center point O2, the points whose distance O2 is less than the preset threshold can be clustered into one category; for the center point Q2, the points whose distance Q2 is less than the preset threshold can be clustered into one category; for the center point P2, the distance Points whose P2 is less than the preset threshold are clustered into one class. Through the above clustering algorithm, the target object A2, the target object B2 and the target object C2 in the figure can be obtained. As can be seen from the figure, the target object A2 may not be a complete object.

It should be noted that, in this embodiment, it is assumed that all target objects move in the same direction, and hereinafter, it is assumed that all target objects move westward.

Exemplarily, for the cluster center O1 (ie the center point O1 above), the clustering target center set at time t1 (ie the cluster center O2 (ie the center point O2 above), the cluster center Q2 (ie the center point O2 above) The nearest neighbor search is performed in the center point Q2) and the cluster center P2 (that is, the center point P2 above)). It can be seen from the figure that the distance between the cluster center Q2 and the cluster center O1 is the smallest, but Since the target object moves westward in this scene, for the target object A1, the position of the target object A1 at time t1 should be located on the west side of the position at time t0, while the target object B2 is located at the east side of the target object A1 at time t0. , therefore, the cluster center O1 and the cluster center Q2 cannot be associated.

Considering the consistency of the movement direction of the target object, at this time, since the distance between the cluster center O2 and the cluster center O1 is second, the cluster center O2 can be associated with the cluster center O1, that is, the target at time t0 The object A1 and the target object A2 at time t1 are the same target object.

For the cluster center Q1, the nearest neighbor search is performed in the cluster target center set at time t1 (ie, the cluster center O2, the cluster center Q2, and the cluster center P2). It can be seen from the figure that combined with the movement direction of the target object and distance, the cluster center Q2 and the cluster center Q1 can be associated, that is, the target object B1 at time t0 and the target object B2 at time t1 are the same target object.

Similarly, for the cluster center P1, the nearest neighbor search is performed in the cluster target center set at time t1 (ie, the cluster center O2, the cluster center Q2, and the cluster center P2). It can be seen from the figure that combined with the target object The movement direction and distance of , the cluster center P2 and the cluster center P1 can be associated, that is, the target object C1 at time t0 and the target object C2 at time t1 are the same target object.

After completing the target association for the target objects in different point cloud frames, further, the motion of the target object can be calculated. Since FIG. 9 is a schematic diagram of the point cloud frame scene shot at different times, the target object in FIG. 9 can be processed first, for example, the same target object in FIG. 9 can be placed on the same horizontal line, that is, the ( The target object A1 in a) and the target object A2 in (b) in Figure 9 are extracted and placed on the same horizontal line, as shown in (a) in Figure 10; the target object in (a) in Figure 9 B1 and the target object B2 in (b) in FIG. 9 are extracted and placed on the same horizontal line, as shown in (b) in FIG. 10 ; the target object C1 in (a) in FIG. The target object C2 in (b) is extracted and placed on the same horizontal line, as shown in (c) in FIG. 10 .

Referring to (a) in Figure 10, it can be seen that from time t0 to time t1, the target object A1 moves from O1 to the position of O2 (that is, the target object A2), and the displacement between O1 and O2 is denoted as

Then the movement speed of the target object A1 can be obtained by the ratio of displacement and time

(including magnitude and direction of velocity), i.e.

Among them, t is the duration from time t0 to time t1.

Referring to (b) in Figure 10, it can be seen that from time t0 to time t1, the target object B1 moves from Q1 to the position of Q2 (that is, the target object B2), and the displacement between Q1 and Q2 is denoted as

Then the movement speed of the target object B1 can be obtained by the ratio of displacement to time

(including magnitude and direction of velocity), i.e.

Referring to (c) in Figure 10, it can be seen that from time t0 to time t1, the target object C1 moves from P1 to the position of P2 (ie, the target object C2), and the displacement between P1 and P2 is denoted as

Then the movement speed of the target object C1 can be obtained by the ratio of displacement and time

(including magnitude and direction of velocity), i.e.

Then the speed of the target object A1

Speed of target object B1

The velocity of the target object C1 is

The direction of speed is westward.

It can be understood that, for the target object A1, although the cluster center at time t0 and the cluster center at time t1 are not the same center, it can also reflect the movement speed of the target object A1 to a certain extent.

Wherein, (a) of FIG. 11 is the point cloud frame shot at time t0 (that is, the first point cloud frame in this application), and (b) of FIG. 11 is the point cloud frame shot at time t1 (that is, the first point cloud frame in this application) second point cloud frame in this application).

In this application, the Euclidean clustering method is used as an example, referring to (a) in FIG. 11 , the center point O1 and the center point Q1 are respectively used as clustering points to perform clustering. For the center point O1, the points whose distance O1 is less than the preset threshold can be clustered into one category; for the center point Q1, the points whose distance Q1 is less than the preset threshold can be clustered into one category. Through the above clustering algorithm, the target object A1 and the target object B1 in the figure can be obtained by clustering.

Similarly, referring to (b) in FIG. 11 , clustering is performed with O2 and Q2 as clustering points, respectively. For the center point O2, the points whose distance O2 is smaller than the preset threshold may be clustered into one category; for the center point Q2, the points whose distance Q2 is smaller than the preset threshold may be clustered into one category. Through the above clustering algorithm, the target object A2 and the target object B2 in the figure can be obtained by clustering.

It should be noted that, in this embodiment, it is assumed that the moving directions of different target objects are inconsistent. In the following, it is assumed that the target object A1 moves westward and the target object A2 moves eastward.

Exemplarily, for the cluster center O1, the nearest neighbor search is performed in the cluster target center set (ie, the cluster center O2, the cluster center Q2) at the time t1. It can be seen from the figure that the cluster center O2 and the cluster The distance between the centers O1 is smaller than the distance between the cluster center Q2 and the cluster center O1. Therefore, the cluster center O1 and the cluster center O2 can be associated, that is, the target object A1 at time t0 and the target object at time t1 A2 is the same target object.

For the cluster center Q1, the nearest neighbor search is performed in the cluster target center set at time t1 (ie, the cluster center O2, the cluster center Q2). It can be seen from the figure that the distance between the cluster center Q2 and the cluster center Q1 The distance between the cluster center O2 and the cluster center Q1 is smaller than the distance between the cluster center O2 and the cluster center Q1. Therefore, the cluster center Q1 and the cluster center Q2 can be associated, that is, the target object B1 at time t0 and the target object B2 at time t1 are the same target. object.

After completing the target association for the target objects in different point cloud frames, further, the motion of the target object can be calculated. Since FIG. 11 is a schematic diagram of a point cloud frame scene captured at different times, the target object in FIG. 11 may be processed first, for example, the same target object in FIG. 11 may be placed on the same horizontal line. That is, the target object A1 in (a) in FIG. 11 and the target object A2 in (b) in FIG. 11 are extracted and placed on the same horizontal line, as shown in (a) in FIG. 12 ; The target object B1 in a) and the target object B2 in (b) in FIG. 11 are extracted and placed on the same horizontal line, as shown in (b) in FIG. 12 .

Referring to (a) in Figure 12, it can be seen that from time t0 to time t1, the target object A1 moves from O1 to the position of O2 (that is, the target object A2), and the displacement between O1 and O2 is denoted as

(including magnitude and direction of velocity), i.e.

Among them, t is the duration from time t0 to time t1.

Referring to (b) in Figure 12, it can be seen that from time t0 to time t1, the target object B1 moves from Q1 to the position of Q2 (that is, the target object B2), and the displacement between Q1 and Q2 is denoted as

(including magnitude and direction of velocity), i.e.

Then the speed of the target object A1

The speed direction is west; the speed of the target object B1

The speed direction is east.

It should be understood that the above numerical values are only for illustration, and in the specific implementation process, other numerical values may also be used, which should not be particularly limited to the present application.

It should be noted that, in the above embodiments, the point cloud frame before point cloud distortion correction is used as an example to illustrate the process of extracting target objects and target associations. For the point cloud frame after point cloud distortion correction, the process is similar to the above. It is concise and will not be repeated here.

In the solution provided by the embodiments of the present application, the target objects extracted from the first point cloud frame and the second point cloud frame are associated, and motion estimation is performed based on the associated target objects, that is, the target objects are moved through the scene flow. estimation, the motion estimation of the target object can be realized efficiently and quickly, and the data dependence and empirical risk can be reduced.

Optionally, in some embodiments, before the extracting the same target object from the first point cloud frame and the second point cloud frame, the method further includes: frame and the second point cloud frame to perform a preprocessing operation; the extracting the same target object from the first point cloud frame and the second point cloud frame includes: extracting the same target object from the preprocessing operation The target object is extracted from the first point cloud frame and the second point cloud frame.

Optionally, in some embodiments, the preprocessing operations include ground filtering operations and/or downsampling operations.

In this embodiment of the present application, after the first point cloud frame and the second point cloud frame are acquired, preprocessing operations may be performed on the acquired point cloud frames, such as ground filtering operations and/or downsampling operations, and then based on the preprocessing operations The processed point cloud frame clusters at least one object.

It can be understood that the ground filtering operation is essentially filtering out the acquired point cloud of the ground, as shown in FIG. 13 , which is a schematic diagram of a target scene provided by another embodiment of the present application.

Referring to (a) in Figure 13, the point cloud in the dotted line can be filtered out to obtain the scene point cloud shown in (b) in Figure 13. When the target object is subsequently extracted from the point cloud frame, it can be improved. efficiency and target extraction accuracy.

The downsampling operation, also known as the downsampling operation, is a multi-rate digital signal processing technique or a process of reducing the signal sampling rate, which can be used to increase the data transmission rate or reduce the data transmission size.

In the solution provided by the embodiments of the present application, by performing a preprocessing operation on the acquired point cloud frame, and extracting the target object based on the preprocessed point cloud frame, the efficiency and the extraction accuracy of the target object can be improved.

As shown in FIG. 14 , it is a schematic diagram of a point cloud motion distortion correction method 1400 provided by another embodiment of the present application. The method 1400 may include steps 1410-1490.

1410. Acquire a point cloud of a scene at time t0.

1420. Obtain the point cloud of the scene at time t1.

1430, preprocessing the acquired point cloud.

1440, clustering the preprocessed point cloud.

1450, perform center calculation on the clustered point cloud.

1460, perform target association according to the center.

1470. Perform motion estimation on the target according to the associated target.

1480, motion distortion correction.

1490, the scene cloud output.

For the specific content of the method 1400, reference may be made to the descriptions of FIG. 1 to FIG. 13 above, which are not repeated here for brevity.

The method embodiments of the present application are described in detail above with reference to FIGS. 1 to 14 , and the apparatus embodiments of the present application are described below with reference to FIGS. 15 to 17 . The apparatus embodiments and method embodiments correspond to each other, and therefore are not described in detail. In part, please refer to the method embodiments of the previous parts.

FIG. 15 is a point cloud motion distortion correction apparatus 1500 provided by an embodiment of the present application. The apparatus 1500 may include a memory 1510 and a processor 1520 .

The memory 1510 is used to store program codes;

The processor 1520 calls the program code, and when the program code is executed, is configured to perform the following operations:

Obtain the first point cloud frame and the second point cloud frame, and the first point cloud frame and the second point cloud frame are point cloud frames at different moments for the same target scene;

extracting the same target object from the first point cloud frame and the second point cloud frame;

Estimate the movement speed of the target object according to the target object in the first point cloud frame and the target object in the second point cloud frame;

Distortion correction is performed on the target object in the first point cloud frame and/or the second point cloud frame according to the moving speed of the target object.

Optionally, in some embodiments, the processor 1520 is further configured to: determine a distortion coefficient of the target object according to the movement speed of the target object; /Or perform distortion correction on the target object in the second point cloud frame.

Optionally, in some embodiments, the processor 1520 is further configured to: for the point cloud of the target object in the first point cloud frame and/or the second point cloud frame, according to the point cloud The product of the rotation interpolation distortion coefficient and the coordinate position of the point cloud, and the sum of the linear motion interpolation distortion coefficient of the point cloud determine the corrected coordinate position of the point cloud.

Optionally, in some embodiments, the processor 1520 is further configured to: determine the distortion coefficient according to the motion speed of the target object and an interpolation coefficient, where the interpolation coefficient is based on the first point cloud frame and all The timestamp of the point cloud in the second point cloud frame is calculated.

Optionally, in some embodiments, the processor 1520 is further configured to: cluster the point clouds in the first point cloud frame and the point clouds in the second point cloud frame respectively by using a clustering algorithm Classify into at least one object; associate the object in the first point cloud frame with the object in the second point cloud frame.

Optionally, in some embodiments, the processor 1520 is further configured to: pair the target point of the object in the first point cloud frame with the target point of the object in the second point cloud frame The object in the first point cloud frame is associated with the object in the second point cloud frame, wherein the target point of the target object in the first point cloud frame is related to the target point of the target object in the second point cloud frame The distance of the target point is smaller than the distance to the target points of other objects in the second point cloud frame.

Optionally, in some embodiments, before extracting the same target object from the first point cloud frame and the second point cloud frame, the processor 1520 is further configured to: The point cloud frame and the second point cloud frame are subjected to a preprocessing operation; the processor is further configured to: extract the first point cloud frame and the second point cloud frame from the first point cloud frame and the second point cloud frame after the preprocessing operation. Extract the target object.

Embodiments of the present application further provide a computer-readable storage medium for storing a computer program.

Optionally, the computer-readable storage medium can be applied to the device for correcting point cloud motion distortion in the embodiments of the present application, and the computer program enables the computer to execute the methods implemented by the device for correcting point cloud motion distortion in the embodiments of the present application. For the sake of brevity, the corresponding process is not repeated here.

Embodiments of the present application also provide a computer program product, including computer program instructions.

Optionally, the computer program product can be applied to the point cloud motion distortion correction device in the embodiments of the present application, and the computer program instructions cause the computer to execute the corresponding methods implemented by the point cloud motion distortion correction device in each method of the embodiments of the present application. The process, for the sake of brevity, will not be repeated here.

The embodiments of the present application also provide a computer program.

Optionally, the computer program can be applied to the device for correcting point cloud motion distortion in the embodiments of the present application. When the computer program runs on the computer, the computer executes the correction by point cloud motion distortion in each method of the embodiments of the present application. For the sake of brevity, the corresponding process implemented by the device will not be repeated here.

The embodiments of the present application also provide a radar, the radar includes a memory and a processor, and the processor can call and run a computer program from the memory to implement the methods described in the embodiments of the present application.

FIG. 16 is a schematic structural diagram of a point cloud motion distortion correction device provided by another embodiment of the present application. The point cloud motion distortion correction device 1600 shown in FIG. 16 includes a processor 1610, and the processor 1610 can call and run a computer program from a memory to implement the methods described in the embodiments of the present application.

Optionally, as shown in FIG. 16 , the point cloud motion distortion correction apparatus 1600 may further include a memory 1620 . The processor 1610 may call and run a computer program from the memory 1620 to implement the methods in the embodiments of the present application.

The memory 1620 may be a separate device independent of the processor 1610, or may be integrated in the processor 1610.

Optionally, as shown in FIG. 16 , the point cloud motion distortion correction device 1600 may further include a transceiver 1630, and the processor 1610 may control the transceiver 1630 to communicate with other devices, specifically, may send information or data to other devices , or receive information or data sent by other devices.

Optionally, the point cloud motion distortion correction device may be, for example, a radar, etc., and the point cloud motion distortion correction device 1600 may implement the corresponding processes in each method of the embodiments of the present application, which will not be repeated here for brevity.

FIG. 17 is a schematic structural diagram of a chip according to an embodiment of the present application. The chip 1700 shown in FIG. 17 includes a processor 1710, and the processor 1710 can call and run a computer program from a memory, so as to implement the methods in the embodiments of the present application.

Optionally, as shown in FIG. 17 , the chip 1700 may further include a memory 1720 . The processor 1710 may call and run a computer program from the memory 1720 to implement the methods in the embodiments of the present application.

The memory 1720 may be a separate device independent of the processor 1710, or may be integrated in the processor 1710.

Optionally, the chip 1700 may further include an input interface 1730 . The processor 1710 can control the input interface 1730 to communicate with other devices or chips, and specifically, can obtain information or data sent by other devices or chips.

Optionally, the chip 1700 may further include an output interface 1740 . The processor 1710 can control the output interface 1740 to communicate with other devices or chips, and specifically, can output information or data to other devices or chips.

It should be understood that the chip mentioned in the embodiments of the present application may also be referred to as a system-on-chip, a system-on-chip, a system-on-chip, or a system-on-a-chip, or the like.

It should be understood that the processor in this embodiment of the present application may be an integrated circuit image processing system, which has signal processing capability. In the implementation process, each step of the above method embodiments may be completed by a hardware integrated logic circuit in a processor or an instruction in the form of software. The above-mentioned processor can be a general-purpose processor, a digital signal processor (Digital Signal Processor, DSP), an application specific integrated circuit (Application Specific Integrated Circuit, ASIC), an off-the-shelf programmable gate array (Field Programmable Gate Array, FPGA) or other available Programming logic devices, discrete gate or transistor logic devices, discrete hardware components. The methods, steps, and logic block diagrams disclosed in the embodiments of this application can be implemented or executed. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of the method disclosed in conjunction with the embodiments of the present application may be directly embodied as executed by a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules may be located in random access memory, flash memory, read-only memory, programmable read-only memory or electrically erasable programmable memory, registers and other storage media mature in the art. The storage medium is located in the memory, and the processor reads the information in the memory, and completes the steps of the above method in combination with its hardware.

It can be understood that the memory in this embodiment of the present application may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memory. Wherein, the non-volatile memory may be a read-only memory (Read-Only Memory, ROM), a programmable read-only memory (Programmable ROM, PROM), an erasable programmable read-only memory (Erasable PROM, EPROM), an electrically programmable read-only memory (Erasable PROM, EPROM). Erase programmable read-only memory (Electrically EPROM, EEPROM) or flash memory. Volatile memory may be Random Access Memory (RAM), which acts as an external cache. By way of example and not limitation, many forms of RAM are available, such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM, SDRAM), double data rate synchronous dynamic random access memory (Double Data Rate SDRAM, DDR SDRAM), enhanced synchronous dynamic random access memory (Enhanced SDRAM, ESDRAM), synchronous link dynamic random access memory (Synchlink DRAM, SLDRAM) ) and direct memory bus random access memory (Direct Rambus RAM, DR RAM). It should be noted that the memory of the systems and methods described herein is intended to include, but not be limited to, these and any other suitable types of memory.

It should be understood that the above-mentioned memory is an exemplary but non-limiting description, for example, the memory in the embodiment of the present application may also be a static random access memory (Static RAM, SRAM), a dynamic random access memory (Dynamic RAM, DRAM), Synchronous dynamic random access memory (Synchronous DRAM, SDRAM), double data rate synchronous dynamic random access memory (Double Data Rate SDRAM, DDR SDRAM), enhanced synchronous dynamic random access memory (Enhanced SDRAM, ESDRAM), synchronous connection Dynamic random access memory (Synch Link DRAM, SLDRAM) and direct memory bus random access memory (Direct Rambus RAM, DR RAM) and so on. That is, the memory in the embodiments of the present application is intended to include but not limited to these and any other suitable types of memory.

The memory in the embodiments of the present application may provide instructions and data to the processor. A portion of the memory may also include non-volatile random access memory. For example, the memory may also store device type information. The processor may be configured to execute the instruction stored in the memory, and when the processor executes the instruction, the processor may execute each step corresponding to the terminal device in the foregoing method embodiments.

In the implementation process, each step of the above-mentioned method can be completed by a hardware integrated logic circuit in a processor or an instruction in the form of software. The steps of the methods disclosed in conjunction with the embodiments of the present application may be directly embodied as executed by a hardware processor, or executed by a combination of hardware and software modules in the processor. The software modules may be located in random access memory, flash memory, read-only memory, programmable read-only memory or electrically erasable programmable memory, registers and other storage media mature in the art. The storage medium is located in the memory, and the processor executes the instructions in the memory, and completes the steps of the above method in combination with its hardware. To avoid repetition, detailed description is omitted here.

It should also be understood that the above description of the embodiments of the present application focuses on emphasizing the differences between the various embodiments, and the unmentioned same or similar points can be referred to each other, and are not repeated here for brevity.

It should be understood that, in this embodiment of the present application, the term "and/or" is only an association relationship for describing associated objects, indicating that there may be three kinds of relationships. For example, A and/or B can mean that A exists alone, A and B exist at the same time, and B exists alone. In addition, the character "/" in this document generally indicates that the related objects are an "or" relationship.

Those of ordinary skill in the art can realize that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. Interchangeability, the above description has generally described the components and steps of each example in terms of function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Skilled artisans may implement the described functionality using different methods for each particular application, but such implementations should not be considered beyond the scope of this application.

Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the system, device and unit described above may refer to the corresponding process in the foregoing method embodiments, which will not be repeated here.

In the several embodiments provided in this application, it should be understood that the disclosed system, apparatus and method may be implemented in other manners. For example, the apparatus embodiments described above are only illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components may be combined or Can be integrated into another system, or some features can be ignored, or not implemented. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be indirect coupling or communication connection through some interfaces, devices or units, and may also be electrical, mechanical or other forms of connection.

The units described as separate components may or may not be physically separated, and components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solutions of the embodiments of the present application.

In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically alone, or two or more units may be integrated into one unit. The above-mentioned integrated units may be implemented in the form of hardware, or may be implemented in the form of software functional units.

The integrated unit, if implemented in the form of a software functional unit and sold or used as an independent product, may be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the present application are essentially or part of contributions to the prior art, or all or part of the technical solutions can be embodied in the form of software products, and the computer software products are stored in a storage medium , including several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium includes: a U disk, a removable hard disk, a ROM, a RAM, a magnetic disk, or an optical disk and other mediums that can store program codes.

The above are only specific embodiments of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of various equivalents within the technical scope disclosed in the present application. Modifications or substitutions shall be covered by the protection scope of this application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

A point cloud motion distortion correction method, comprising:

acquiring a first point cloud frame and a second point cloud frame, where the first point cloud frame and the second point cloud frame are point cloud frames at different times for the same target scene;

extracting the same target object from the first point cloud frame and the second point cloud frame;

Estimate the movement speed of the target object according to the target object in the first point cloud frame and the target object in the second point cloud frame;

Distortion correction is performed on the target object in the first point cloud frame and/or the second point cloud frame according to the moving speed of the target object.
The method according to claim 1, wherein the performing distortion correction on the target object in the first point cloud frame and/or the second point cloud frame according to the moving speed of the target object, comprising: :

Determine the distortion coefficient of the target object according to the moving speed of the target object;

Perform distortion correction on the target object in the first point cloud frame and/or the second point cloud frame according to the distortion coefficient.
The method according to claim 2, wherein the distortion coefficients include rotational interpolation distortion coefficients and linear motion interpolation distortion coefficients.
The method according to claim 3, wherein the performing distortion correction on the target object in the first point cloud frame and/or the second point cloud frame according to the distortion coefficient comprises:

For the point cloud of the target object in the first point cloud frame and/or the second point cloud frame, according to the product of the rotation interpolation distortion coefficient of the point cloud and the coordinate position of the point cloud, and the The sum of the linear motion interpolation distortion coefficients of the point cloud determines the corrected coordinate position of the point cloud.
The method according to any one of claims 2 to 4, wherein the determining the distortion coefficient of the target object according to the movement speed of the target object comprises:

The distortion coefficient is determined according to the moving speed of the target object and an interpolation coefficient, where the interpolation coefficient is calculated based on the timestamps of the point clouds in the first point cloud frame and the second point cloud frame.
The method according to any one of claims 1 to 5, wherein the extracting the same target object from the first point cloud frame and the second point cloud frame comprises:

Using a clustering algorithm, the point cloud in the first point cloud frame and the point cloud in the second point cloud frame are respectively clustered into at least one object;

Associating objects in the first point cloud frame with objects in the second point cloud frame.
The method according to claim 6, wherein the associating the objects in the first point cloud frame with the objects in the second point cloud frame comprises:

According to the target point of the object in the first point cloud frame and the target point of the object in the second point cloud frame, the object in the first point cloud frame and the object in the second point cloud frame are compared Perform association, wherein the distance between the target point of the target object in the first point cloud frame and the target point of the target object in the second point cloud frame is smaller than the target point of other objects in the second point cloud frame the distance.
The method according to claim 7, wherein the target point is a center point or a center of gravity point.
The method according to any one of claims 1 to 8, wherein before extracting the same target object from the first point cloud frame and the second point cloud frame, the method further comprises: :

Perform preprocessing operations on the first point cloud frame and the second point cloud frame respectively;

The extracting the same target object from the first point cloud frame and the second point cloud frame includes:

The target object is extracted from the first point cloud frame and the second point cloud frame after the preprocessing operation.
The method according to claim 9, wherein the preprocessing operation comprises a ground filtering operation and/or a downsampling operation.
The method according to any one of claims 1 to 10, wherein the first point cloud frame and the second point cloud frame are point cloud frames at adjacent moments of the same target scene.
A point cloud motion distortion correction device, characterized in that the device includes a memory and a processor;

the memory is used to store program codes;

The processor calls the program code, and when the program code is executed, is configured to perform the following operations:

acquiring a first point cloud frame and a second point cloud frame, where the first point cloud frame and the second point cloud frame are point cloud frames at different times for the same target scene;

extracting the same target object from the first point cloud frame and the second point cloud frame;

Estimate the movement speed of the target object according to the target object in the first point cloud frame and the target object in the second point cloud frame;

Distortion correction is performed on the target object in the first point cloud frame and/or the second point cloud frame according to the moving speed of the target object.
The apparatus of claim 12, wherein the processor is further configured to:

Determine the distortion coefficient of the target object according to the moving speed of the target object;

Perform distortion correction on the target object in the first point cloud frame and/or the second point cloud frame according to the distortion coefficient.
The apparatus according to claim 13, wherein the distortion coefficients include rotational interpolation distortion coefficients and linear motion interpolation distortion coefficients.
The apparatus of claim 14, wherein the processor is further configured to:

For the point cloud of the target object in the first point cloud frame and/or the second point cloud frame, according to the product of the rotation interpolation distortion coefficient of the point cloud and the coordinate position of the point cloud, and the The sum of the linear motion interpolation distortion coefficients of the point cloud determines the corrected coordinate position of the point cloud.
The apparatus according to any one of claims 13 to 15, wherein the processor is further configured to:

The distortion coefficient is determined according to the moving speed of the target object and an interpolation coefficient, and the interpolation coefficient is calculated based on the timestamps of the point clouds in the first point cloud frame and the second point cloud frame.
The apparatus according to any one of claims 12 to 16, wherein the processor is further configured to:

Using a clustering algorithm, the point cloud in the first point cloud frame and the point cloud in the second point cloud frame are respectively clustered into at least one object;

Associating objects in the first point cloud frame with objects in the second point cloud frame.
The apparatus of claim 17, wherein the processor is further configured to:

According to the target point of the object in the first point cloud frame and the target point of the object in the second point cloud frame, the object in the first point cloud frame and the object in the second point cloud frame are compared Perform association, wherein the distance between the target point of the target object in the first point cloud frame and the target point of the target object in the second point cloud frame is smaller than the target point of other objects in the second point cloud frame the distance.
The device according to claim 18, wherein the target point is a center point or a center of gravity point.
The apparatus according to any one of claims 12 to 19, wherein before extracting the same target object from the first point cloud frame and the second point cloud frame, the processor further Used for:

Perform preprocessing operations on the first point cloud frame and the second point cloud frame respectively;

The processor is further configured to: extract the target object from the first point cloud frame and the second point cloud frame after the preprocessing operation.
The apparatus of claim 20, wherein the preprocessing operation includes a ground filtering operation and/or a downsampling operation.
The apparatus according to any one of claims 12 to 21, wherein the first point cloud frame and the second point cloud frame are point cloud frames at adjacent moments of the same target scene.
A radar, characterized in that the radar includes a memory and a processor;

the memory is used to store program codes;

The processor calls the program code, and when the program code is executed, is used to execute the point cloud motion distortion correction method according to any one of claims 1 to 11.
A computer-readable storage medium, characterized by comprising instructions for executing the point cloud motion distortion correction method according to any one of claims 1 to 11.