CN114631124A

CN114631124A - Three-dimensional point cloud segmentation method and device and movable platform

Info

Publication number: CN114631124A
Application number: CN202080071116.8A
Authority: CN
Inventors: 李星河; 韩路新; 于亦奇
Original assignee: SZ DJI Technology Co Ltd
Current assignee: SZ DJI Technology Co Ltd
Priority date: 2020-11-13
Filing date: 2020-11-13
Publication date: 2022-06-14
Also published as: WO2022099620A1

Abstract

A three-dimensional point cloud segmentation method and device and a movable platform are used for carrying out point cloud segmentation on three-dimensional point cloud collected by the movable platform, and the method comprises the following steps: acquiring a plurality of candidate points (301) in the three-dimensional point cloud; searching the candidate points on a v-disparity plane, and determining a target candidate point (302) on the traveling road surface of the movable platform in the candidate points; and fitting a model of the driving road surface based on the target candidate points, and performing second point cloud segmentation on the three-dimensional point cloud on a u-disparity plane based on the model of the driving road surface to obtain a point cloud segmentation result (303).

Description

Three-dimensional point cloud segmentation method and device and movable platform

Technical Field

The disclosure relates to the technical field of computer vision, in particular to a three-dimensional point cloud segmentation method and device and a movable platform.

Background

During the driving process of the movable platform, the driving state (for example, pose and speed) of the movable platform can be decision-making planned through a path planning (planning) module on the movable platform. In order to enable the planning module to complete decision planning, a point cloud acquisition device on a movable platform is required to acquire a three-dimensional point cloud of a surrounding environment, perform point cloud segmentation to distinguish the ground and obstacles in the three-dimensional point cloud, and further distinguish a dynamic object and a static object from the obstacles. Therefore, point cloud segmentation is an important link for decision planning of the driving state of the movable platform.

The traditional point cloud segmentation method generally segments ground points and non-ground points based on local features of three-dimensional point cloud. However, the above method can cause significant degradation when processing three-dimensional point clouds with a long distance, and the segmentation accuracy is low.

Disclosure of Invention

In view of this, embodiments of the present disclosure provide a three-dimensional point cloud segmentation method and apparatus, and a movable platform, so as to accurately perform point cloud segmentation on a three-dimensional point cloud acquired by the movable platform.

According to a first aspect of the embodiments of the present disclosure, there is provided a three-dimensional point cloud segmentation method for performing point cloud segmentation on a three-dimensional point cloud acquired by a movable platform, the method including: acquiring a plurality of candidate points in the three-dimensional point cloud; searching the candidate points on a v-disparity plane, and determining a target candidate point on the traveling road surface of the movable platform in the candidate points; and fitting a model of the driving road surface based on the target candidate points, and performing second point cloud segmentation on the three-dimensional point cloud on a u-disparity plane based on the model of the driving road surface to obtain a point cloud segmentation result.

According to a second aspect of the embodiments of the present disclosure, there is provided a three-dimensional point cloud segmentation apparatus, including a processor, the three-dimensional point cloud segmentation apparatus being configured to perform point cloud segmentation on a three-dimensional point cloud acquired by a movable platform, the processor being configured to perform the following steps: acquiring a plurality of candidate points in the three-dimensional point cloud; searching the candidate points on a v-disparity plane, and determining target candidate points on the traveling road surface of the movable platform in the candidate points; and fitting a model of the driving road surface based on the target candidate points, and performing second point cloud segmentation on the three-dimensional point cloud on a u-disparity plane based on the model of the driving road surface.

According to a third aspect of the embodiments of the present disclosure, there is provided a movable platform, comprising: a housing; the point cloud acquisition device is arranged on the shell and used for acquiring three-dimensional point cloud; and a three-dimensional point cloud segmentation device arranged in the shell and used for executing the method of any embodiment of the disclosure.

According to a fourth aspect of the embodiments of the present disclosure, there is provided a computer-readable storage medium, on which a computer program is stored, which when executed by a processor implements the method according to any of the embodiments of the present disclosure.

By applying the scheme of the embodiment of the disclosure, candidate points are searched on a v-disparity plane, target candidate points on a driving road surface of the movable platform in the candidate points are determined, then a model of the driving road surface is fitted based on the target candidate points, the model is used as a reference for point cloud segmentation, and second point cloud segmentation is performed on the three-dimensional point cloud based on the model of the driving road surface on a u-disparity plane, so that the accuracy of point cloud segmentation is improved, and regions with high segmentation difficulty, such as a slope surface, a distance and the like in the three-dimensional point cloud, can be accurately segmented.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present disclosure, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without inventive labor.

FIG. 1 is a schematic diagram of a point cloud segmentation process of some embodiments.

Fig. 2 is a schematic diagram of a decision planning process during mobile platform travel according to some embodiments.

Fig. 3 is a flowchart of a point cloud segmentation method according to an embodiment of the present disclosure.

Fig. 4A and 4B are schematic diagrams of an uvd coordinate system, respectively, of an embodiment of the disclosure.

FIG. 5 is a schematic view of a projection process of a u-disparity plane according to an embodiment of the disclosure.

Fig. 6 is a schematic diagram illustrating a relationship between parallax and depth according to an embodiment of the disclosure.

Fig. 7 is a schematic diagram of a point cloud segmentation apparatus of an embodiment of the present disclosure.

Fig. 8 is a schematic view of a movable platform of an embodiment of the disclosure.

Detailed Description

Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, like numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the exemplary embodiments below do not represent all implementations consistent with the present disclosure. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present disclosure, as detailed in the appended claims.

The terminology used in the present disclosure is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure. As used in this disclosure and the appended claims, the singular forms "a", "an", and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the term "and/or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.

It is to be understood that although the terms first, second, third, etc. may be used herein to describe various information, such information should not be limited to these terms. These terms are only used to distinguish one type of information from another. For example, first information may also be referred to as second information, and similarly, second information may also be referred to as first information, without departing from the scope of the present disclosure. The word "if" as used herein may be interpreted as "at … …" or "when … …" or "in response to a determination", depending on the context.

In the driving process of the movable platform, decision planning can be carried out on the driving state of the movable platform through a path planning (planning) module on the movable platform. The point cloud segmentation is an important link for decision planning of the driving state of the movable platform. Fig. 1 is a schematic diagram of a point cloud segmentation process according to some embodiments. In step 101, a three-dimensional point cloud may be collected by a point cloud collection device on a movable platform, and then, in step 102, for the movable platform (e.g., an unmanned vehicle) traveling on the ground, ground segmentation may be performed on the collected three-dimensional point cloud, i.e., three-dimensional points in the three-dimensional point cloud are segmented into ground points and non-ground points. For other types of movable platforms (e.g., movable robots), the acquired three-dimensional point cloud may be segmented to segment three-dimensional points in the three-dimensional point cloud into points on the movable platform travel surface and points not on the movable platform travel surface. For convenience of description, the following description will be made with reference to a traveling road surface as a ground surface. In step 103, if a three-dimensional point is a ground point, step 104 is executed to add a ground point label to the three-dimensional point, otherwise step 105 is executed to perform dynamic and static segmentation on the three-dimensional point, that is, the three-dimensional point is segmented into a static point which is static and a dynamic point which moves. In step 106, if a three-dimensional point is a static point, step 107 is performed, a static point tag is added to the three-dimensional point, otherwise step 108 is performed, a dynamic point tag is added to the three-dimensional point, and the three-dimensional point cloud with the tag is output to a downstream module in step 109. All or part of the three-dimensional points in the three-dimensional point cloud can be labeled. The label may include at least one of a first label for characterizing whether the three-dimensional point is a ground point and a second label for characterizing whether the three-dimensional point is a static point, and may further include a label for characterizing other information of the three-dimensional point.

The downstream module may be a planning module on a movable platform, such as an Electronic Control Unit (ECU), a Central Processing Unit (CPU), and the like. The Planning module can perform decision Planning on the driving state of the movable platform based on the label of the three-dimensional point after receiving the labeled three-dimensional point cloud. The driving state may include at least one of a pose and a speed of the movable platform. Fig. 2 is a schematic diagram of a decision planning process according to some embodiments. In step 201 and step 202, the planning module may receive the three-dimensional point cloud and read a tag carried in the three-dimensional point cloud. In step 203, it may be determined whether a three-dimensional point in the three-dimensional point cloud is a point on a moving platform travel surface (e.g., ground) based on the tag. Taking the ground points as an example, if yes, step 204 is executed to identify three-dimensional points belonging to the lane line from the ground points, and determine the attitude of the movable platform according to the direction of the lane line, so that the movable platform drives along the direction of the lane line. If the non-ground point is a non-ground point, step 205 is executed to determine whether the non-ground point is a static point. If so, step 206 is performed to determine the pose of the movable platform based on the positions of the static points. For example, it is determined whether the static point is on a pre-planned driving path, and if so, the path is re-planned to avoid the movable platform colliding with the static point. If the non-ground point is a dynamic point, step 207 is performed to determine at least one of the attitude and the velocity of the movable platform based on the orientation and the velocity of the static point. For example, if the dynamic point is on a driving path planned in advance by the movable platform and the moving speed of the dynamic point is less than or equal to the moving speed of the movable platform, the movable platform is controlled to decelerate, or the attitude of the movable platform is adjusted, so that the movable platform bypasses the dynamic point. Also for example, the movable platform may be controlled to travel at the same speed as the dynamic point.

Therefore, point cloud segmentation is an important link for decision planning of the driving state of the movable platform, and accurate point cloud segmentation is beneficial to accurate decision planning of the driving state of the movable platform. The current point cloud segmentation method is mainly realized based on local features, specifically, three-dimensional point cloud is transformed into xyz space, and rasterization or proximity search is performed in the xyz space. Finding out adjacent points around the candidate points, and judging the probability that the candidate points belong to the ground points according to the characteristics of the adjacent point cloud, such as thickness, height, normal vector and the like. The method has the advantages of obvious degradation when processing three-dimensional point clouds with long distance, low segmentation accuracy, difficulty in establishing a global ground model and incapability of making correct judgment on a plane other than the ground.

Based on this, the present disclosure provides a three-dimensional point cloud segmentation method for performing point cloud segmentation on a three-dimensional point cloud acquired by a movable platform, as shown in fig. 3, the method includes:

step 301: acquiring a plurality of candidate points in the three-dimensional point cloud;

step 302: searching the candidate points on a v-disparity plane, and determining a target candidate point on the traveling road surface of the movable platform in the candidate points;

step 303: and fitting a model of the driving road surface based on the target candidate points, and performing second point cloud segmentation on the three-dimensional point cloud on a u-disparity plane based on the model of the driving road surface to obtain a point cloud segmentation result.

In step 301, a three-dimensional point cloud may be acquired by a point cloud acquisition device (e.g., a laser radar, a vision sensor, etc.) on a movable platform. The movable platform can be an unmanned vehicle, an unmanned aerial vehicle, an unmanned ship, a movable robot and the like.

The candidate points may be some or all of the points in the three-dimensional point cloud. Optionally, semantic segmentation may be performed on the three-dimensional point cloud, and a plurality of candidate points in the three-dimensional point cloud are obtained based on a semantic segmentation result. Through semantic segmentation, the categories of a plurality of three-dimensional points in the three-dimensional point cloud can be obtained, such as vehicle categories, traffic light categories, pedestrian categories, lane line categories and the like. Candidate points may then be determined based on the categories of the three-dimensional points. For example, a three-dimensional point of the lane line category is determined as a candidate point.

Alternatively, the three-dimensional point cloud may be pre-segmented on the u-disparity plane, and candidate points may be determined based on the result of the pre-segmentation. Specifically, a plurality of candidate points in the three-dimensional point cloud may be acquired based on a projection density of the three-dimensional point cloud on a u-disparity plane and a first reference projection density of a plane model of the driving road surface on the u-disparity plane. In the embodiment, the driving road surface of the movable platform is assumed as a plane, a first reference projection density is determined based on the plane, and then pre-segmentation is performed based on the first reference projection density to determine candidate points on the driving road surface of the movable platform. The pre-segmentation on the u-disparity plane can improve the signal-to-noise ratio of the candidate points, so that the candidate points can be selected in a long-distance region (with less signal quantity), and the distance and the accuracy of searching on the v-disparity plane are improved.

Wherein the various coordinate axes in uvd space may be determined based on the direction of the travel surface of the movable platform. For example, in fig. 4A, the movable platform 401 travels on a horizontal ground 402, and the u-axis, the v-axis, and the d-axis (i.e., the disparity axis) may be a coordinate axis perpendicular to the traveling direction of the movable platform on the ground, a coordinate axis parallel to the traveling direction of the movable platform on the ground, and a coordinate axis in the height direction of the movable platform (i.e., vertically upward), respectively. In fig. 4B, the movable platform 404 is a glass wiping robot traveling on the vertical glass plane 403, and the u-axis, the v-axis, and the d-axis may be a coordinate axis perpendicular to the traveling direction of the movable platform on the glass plane, a coordinate axis parallel to the traveling direction of the movable platform on the glass plane, and a coordinate axis in the height direction (i.e., horizontal direction) of the movable platform, respectively. Besides, the uvd space coordinate axes may point in other directions, and the specific directions may be set according to actual needs, which is not limited by the present disclosure.

In some embodiments, the u-disparity plane may be pre-divided into a plurality of grids. And in a first grid of the grids, if the ratio of the projection density to the first reference projection density is greater than or equal to a first preset ratio, determining a point projected into the first pixel grid from the three-dimensional point cloud as a candidate point. Wherein the first preset ratio is greater than 1.

As shown in fig. 5, the u-disparity plane may be pre-divided into a plurality of grids, and the size of each grid may be the same for the convenience of comparing the projection densities. Each black point represents a projection point of the three-dimensional point in the three-dimensional point cloud on the u-disparity plane, and the number of projection points in a grid is equal to the number of three-dimensional points projected to the grid in the three-dimensional point cloud. The density of projections within a grid may be determined as the ratio of the number of projection points within the grid to the area of the grid. Taking a vehicle traveling on a road surface as an example, when the vehicle is traveling, no obstacle exists in a region (referred to as a first region) which is d1 to d2 away from the vehicle, each three-dimensional point in the first region is a three-dimensional point on the road surface, a plane of the three-dimensional point is parallel to a traveling direction (i.e., a direction of a disparity coordinate axis) of the vehicle or forms a small angle with the traveling direction, the three-dimensional points extend along the disparity coordinate axis, a parallax variation range is large, and the density of the three-dimensional points in the first region is small. In the area d3 to d4 away from the vehicle (referred to as the second area), an obstacle is present in a plane generally perpendicular to the traveling direction of the vehicle or at a large angle, which hinders the vehicle from traveling. That is, the range of parallax change of the three-dimensional points in the second region is small, and the density of the three-dimensional points in the second region is large. Thus, knowing the first reference density of the movable platform's travel plane, it can be roughly inferred whether a three-dimensional point is a point on the travel plane or a point outside the travel plane (e.g., an obstacle).

Further, since the actual driving surface may not be a plane, the preset ratio λ (i.e., the redundancy of the segmentation) is set to a value greater than 1, so as to provide a certain redundancy and reduce the selection error of the candidate point. The lambda can be fixedly set according to the model of the visual sensor, or can be dynamically set according to the actual application scene. When the reliability of the visual sensor is low, λ may be set to a large value, whereas λ may be set to a small value. For example, in the case where the focal length of the visual sensor is long, or the ambient light is dark, or the like, λ may be set to a large value.

In some embodiments, the first reference projected density of the planar model on the u-disparity plane is proportional to a disparity value of a point on the planar model. Specifically, the first reference projection density may be determined according to a base length of a vision sensor, an ending of a first coordinate axis of the plane model in a coordinate system of the vision sensor, and a parallax value of a point on the plane model. Assuming a planar model of the driving surface as:

y＝αz+h

wherein α is a slope of the traveling road surface. A ratio of the intercept to a baseline length of the vision sensor may be calculated, and a product of the ratio and a parallax value of a point on the planar model may be determined as the first reference projection density. The first reference projection density of a point on the planar model on the u-disparity plane can be written as:

wherein h is the intercept, b is the base length, Δ d is the density of projection points in a grid on the u-disparity plane, Δ v_gA first reference projection density within the corresponding grid. The first coordinate axis is a coordinate axis in the height direction of the movable platform, for example, when the movable platform travels on the ground shown in fig. 4A, the first coordinate axis may be a vertically upward coordinate axis; for another example, when the movable platform travels on the glass plane shown in fig. 4B, the first coordinate axis may be a coordinate axis in the horizontal direction.

By using the above principle, it is possible to obtain which points are points on the traveling road surface when the model of the traveling road surface is a plane model, and to extract the points on the traveling road surface as candidate points. It can be seen that the first reference projected density is related only to the intercept, the baseline length, and the disparity value, and not to the distance to the ground (z-value). Therefore, the driving road surface of the movable platform is assumed to be a plane, and the first reference projection density is determined based on the plane model so as to be divided on the u-disparity plane to determine the candidate points, so that on one hand, the calculation amount is reduced, on the other hand, the signal-to-noise ratio of the candidate points is improved, the candidate points can be selected and taken out under the condition of long distance (small signal amount), the subsequent searching distance and accuracy on the v-disparity plane are improved, and the regions with high dividing difficulty, such as the slope surface and the far distance, can be accurately cut.

In some embodiments, the point cloud acquisition device does not acquire a complete point cloud frame due to clock resetting or the like, and therefore, the acquired three-dimensional point cloud may include both valid points and invalid points. In order to improve the reliability of obtaining candidate points, a plurality of candidate points may be obtained from only valid points in the three-dimensional point cloud. Wherein, the invalid point can be set as invalid (invalid) to avoid the invalid point being selected as a candidate point.

In some embodiments, outlier points may be further filtered from the three-dimensional points, and a plurality of candidate points in the filtered three-dimensional point cloud may be obtained. Outlier points are points whose value range is outside the valid range. Outlier points can be filtered out of the three-dimensional points by means of filtering.

In some embodiments, before obtaining a plurality of candidate points in the three-dimensional point cloud based on a first projection density of the three-dimensional point cloud on a u-disparity plane and a first reference projection density of a plane model of the driving road surface on the u-disparity plane, a preset scale transformation parameter may be obtained, a scale transformation is performed on u coordinate values of each three-dimensional point in the three-dimensional point cloud based on the scale transformation parameter, and the three-dimensional point cloud after the scale transformation is projected on the u-disparity plane.

And the scale transformation parameter scale of one three-dimensional point is used for amplifying or reducing the u coordinate value of the three-dimensional point. And under the condition that the scale transformation parameter scale is larger than 1, the u coordinate value of the three-dimensional point is amplified, namely 1 line of projection points on the u-disparity plane are mapped into a transformed scale line. And when the scale transformation parameter scale is less than 1, reducing the u coordinate value of the three-dimensional point, namely mapping the scale line projection point on the u-disparity plane into the transformed line 1.

And the value of the scale transformation parameter of one three-dimensional point corresponds to the u coordinate value of the three-dimensional point. For example, if the u coordinate value of the first three-dimensional point in the three-dimensional point cloud is smaller than a first preset coordinate value, the scale transformation parameter of the first three-dimensional point is greater than 1. For another example, if the u coordinate value of the first three-dimensional point in the three-dimensional point cloud is greater than or equal to a second preset coordinate value, the scale conversion parameter of the first three-dimensional point is less than 1. The first preset coordinate value may be smaller than or equal to the second preset coordinate value. Further, if the first preset coordinate value is smaller than the second preset coordinate value, when the u coordinate value of the first three-dimensional point in the three-dimensional point cloud is greater than or equal to the first preset coordinate value and smaller than the second preset coordinate value, the scale transformation parameter of the first three-dimensional point is equal to 1.

Further, a plurality of scaling parameters larger than 1 and/or a plurality of scaling parameters smaller than 1 may also be set. For example, if the u coordinate value of the first three-dimensional point is smaller than a third preset coordinate value, the scale transformation parameter of the first three-dimensional point is set as a first parameter value; and if the u coordinate value of the first three-dimensional point is greater than or equal to the third preset coordinate value and smaller than the first preset coordinate value, setting the scale transformation parameter of the first three-dimensional point as a second parameter value. The first parameter value and the second parameter value are both larger than 1, the first parameter value is larger than the second parameter value, and the third preset coordinate value is smaller than the first preset coordinate value.

For another example, if the u-coordinate value of the first three-dimensional point is greater than or equal to a fourth preset coordinate value, the scaling parameter of the first three-dimensional point is set as a third parameter value, and if the u-coordinate value of the first three-dimensional point is less than the fourth preset coordinate value and greater than or equal to the second preset coordinate value, the scaling parameter of the first three-dimensional point is set as a fourth parameter value. And the fourth preset coordinate value is greater than the second preset coordinate value, and the fourth parameter value is less than the third parameter value.

Table 1 shows the scale parameter and the corresponding relation between the number of rows before and after transformation in some embodiments, and those skilled in the art will understand that the table 1 is only an illustration and is not used to limit the disclosure. For example, in the case where the scaling parameter scale is 4.0, the projection points of 1 line before transformation are mapped to the projection points of 4 lines after transformation. For example, the 3 rd row of proxels before transformation is mapped to the 0 th to 3 rd row of proxels after transformation, the 4 th row of proxels before transformation is mapped to the 4 th to 7 th row of proxels after transformation, and so on.

Note that the reason why the starting line number of the projection point before conversion is 3 is that the error of the point correspondence is large because the parallax value is too small. Therefore, only the points whose disparity values are greater than or equal to 3 are taken for processing here. Those skilled in the art will appreciate that the points before line 3 may also be used without taking into account errors; in other cases, the starting line number of the proxels before transformation may also be set to a value greater than 3.

TABLE 1 relationship of scale parameters to number of rows before and after transformation

scale	4.0	2.0	1.0	0.5	0.25
Transformed line range	[0,108)	[108,128)	[128,148)	[148,162)	[162,172)
Line range before transformation	[3,30)	[30,40)	[40,60)	[60,88)	[88,128)

Since disparity is related to depth by:

z＝b*f/d

wherein f is the focal length of a visual sensor used for collecting three-dimensional point cloud, b is the focal length of the visual sensor, d is a parallax value, and z is depth. It can be seen that z is inversely proportional to d, as shown in FIG. 6. The u coordinate values are subjected to scale transformation by using the scale transformation parameters, so that the characteristic that the disparity is corresponding to the depth nonlinear change is resisted, a near high-resolution area is compressed, and the sub-pixel precision of a far distance is fully exerted, so that the accuracy of point cloud segmentation is improved. In addition, by the scale transformation mode, the number of rows of the transformed three-dimensional points is close to the number of rows before transformation, excessive rows caused by scale transformation are avoided, the calculation force is greatly increased, and balance between the calculation force and the point cloud segmentation accuracy is realized.

In step 302, the candidate points selected in step 301 may be projected onto a v-disparity plane, which is equal in scale, that is, the number of rows of the projection points on the v-disparity plane corresponds to the number of rows of the disparity map, and the number of columns of the projection points on the v-disparity plane corresponds to the integer value of the valid range of the disparity value.

The plurality of candidate points may be searched on a v-disparity plane to determine a target candidate point located on the traveling surface of the movable platform among the plurality of candidate points. The following describes a process of determining target candidate points by taking a dynamic programming manner as an example. In practical applications, other ways may also be adopted to determine the target candidate point, and the description is not further provided herein.

For each of the plurality of candidate points, a search cost for the candidate point may be determined, and a target candidate point may be determined from the candidate points based on the search costs for the candidate point. And if the search cost of the candidate point is less than the preset cost, determining the candidate point as a target candidate point.

In some embodiments, the search cost comprises a first search cost and a second search cost; wherein the first search cost is used for characterizing whether a target candidate point is observed on the candidate point, and the second search cost is used for characterizing whether the candidate point and a neighborhood target candidate point of the candidate point are smooth.

Wherein the density cost can be calculated by the following method:

p is a point on the v-disparity image, cost is the density cost of the point, and th is related to the parameters of the vision sensor. For example, when the driving road surface of the movable platform is the ground, the width of the lane line on the ground is about 3 meters, and at a depth of 100 meters, a region with a width of 3 meters, which is captured by the vision sensor, generally includes 5 pixel points in one frame of three-dimensional point cloud image, and therefore, th may take a value of 5. In other cases, th may be set to other values according to actual conditions.

After the target candidate points are obtained, a model of the travel surface may be fitted based on the target candidate points in step 303. For example, a polynomial model of the traveling road surface may be obtained by performing polynomial fitting on the target candidate points based on a least square method. The resulting model can be expressed as:

y＝A+B*z+C*z ²+D*z ³+E*z ⁴

the slope of the tangent of the model at a certain point is:

y’＝B+2*C*z+3*D*z ²+4*E*z ³

where A, B, C, D and E are both constants and z is depth. The above model is only an exemplary illustration, and the above model can be adjusted to a cubic polynomial model or a quintic polynomial model, etc. according to the actual application scenario. Then, the slope of the model of the running road surface may be acquired; determining a second reference projected density of the driving road surface on the u-disparity plane based on the slope; and performing second point cloud segmentation on the three-dimensional point cloud based on the second reference projection density.

The method for performing the second point cloud segmentation on the three-dimensional point cloud based on the second reference projection density is similar to the method for performing the point cloud segmentation based on the first reference projection density, that is, in the second pixel grid on the u-disparity plane, if the ratio of the projection density of the three-dimensional point cloud on the u-disparity plane to the second reference projection density is greater than or equal to a second preset ratio, the point projected in the second pixel grid in the three-dimensional point cloud is segmented into the target point on the driving road surface. Here, the second preset ratio may be set to a value greater than or equal to 1, and the second preset value may be the same as or different from the first preset value.

Wherein the second reference projected density may be determined based on a model of the traveling road surface, a slope of the model, and a depth of the traveling road surface. For example, a product of the slope of the model and the depth of the running road surface may be calculated, a difference between the model of the running road surface and the product may be calculated, and the second reference projected density may be determined based on a ratio of the difference to a base length of the vision sensor, as follows:

Δv _g＝(y-z*y’)/b

the process comprises the steps of searching and model fitting on a v-disparity plane, point cloud segmentation on a u-disparity plane, and iteration of the two steps, wherein the signal-to-noise ratio of the ground candidate points can be improved through the segmentation of the u-disparity plane, so that the candidate points can be selected in a remote area (with less signal quantity), the searching distance and the searching precision on the v-disparity plane are improved, the obtained model can provide important information of reference density for the u-disparity segmentation, and the areas with high segmentation difficulty, such as a slope surface and a remote area, can be accurately segmented.

In some embodiments, labels may be further marked for each three-dimensional point in the three-dimensional point cloud based on the point cloud segmentation result, and the label of one three-dimensional point is used for representing the category of the three-dimensional point. The categories may include a first category for characterizing that the three-dimensional point belongs to a point on a moving platform travel surface and a second category for characterizing that the three-dimensional point belongs to a point on an obstacle. Further, the categories may further include a third category for characterizing that the three-dimensional point does not belong to either a point on the driving surface or a point on the obstacle. The third category of points may be inverted points, or points for which a category cannot be determined, etc.

In some embodiments, each three-dimensional point in the three-dimensional point cloud may be labeled based on the point cloud segmentation results and the height of each three-dimensional point in the three-dimensional point cloud. The three-dimensional points are labeled based on the heights of the three-dimensional points and the point cloud segmentation result, so that the labeling accuracy is improved. Specifically, if the height of a three-dimensional point is lower than the height of the driving road surface, the label of the three-dimensional point may be determined as a first label, and the first label is used for representing that the three-dimensional point is a reflection point. If the height of one three-dimensional point is not lower than that of the driving road surface, labeling can be further performed by combining the point cloud segmentation result.

For example, for each three-dimensional point in the three-dimensional point cloud, a first confidence that the three-dimensional point is a point on the driving surface may be determined based on the height of the three-dimensional point; determining a second confidence degree that the three-dimensional point is a point on the driving road surface based on the point cloud segmentation result; and labeling the three-dimensional point based on the first confidence degree and the second confidence degree of the three-dimensional point.

In the case where the height of one three-dimensional point is not lower than the height of the travel surface, if the height of one three-dimensional point is higher, the first confidence that the three-dimensional point is a point on the travel surface is lower, whereas the first confidence is higher. If the ratio of the projection density of one three-dimensional point to the second reference projection density is larger, the second confidence coefficient of the three-dimensional point as the point on the driving road surface is lower, and otherwise, the second confidence coefficient is higher.

Different modes can be selected for labeling in combination with different scenes. For example, in a case where the requirements on reliability and accuracy of the label are high, when both the first confidence and the second confidence of a three-dimensional point are greater than the corresponding confidence threshold, it may be determined that the label of the three-dimensional point is the first label, and the first label is used to indicate that the three-dimensional point is a point on the driving road surface. In the case that the requirements on the reliability and the accuracy of the label are low, when at least one of the first confidence and the second confidence of a three-dimensional point is greater than the corresponding confidence threshold, the label of the three-dimensional point may be determined to be the first label. The labels of the three-dimensional points may also be determined based on other ways, which are not listed here.

In practical application, the point cloud segmentation result can be used for a planning unit on the movable platform to plan the driving state of the movable platform. For example, the planning unit may determine whether an obstacle exists on the travel path based on the tag obtained from the point cloud segmentation result, thereby deciding whether the speed and attitude of the movable platform need to be controlled to avoid the obstacle.

It will be understood by those skilled in the art that in the method of the present invention, the order of writing the steps does not imply a strict order of execution and any limitations on the implementation, and the specific order of execution of the steps should be determined by their function and possible inherent logic.

The embodiment of the present disclosure further provides a point cloud segmentation apparatus, which includes a processor, where the processor is configured to execute the following steps:

acquiring a plurality of candidate points in the three-dimensional point cloud;

searching the candidate points on a v-disparity plane, and determining target candidate points on the traveling road surface of the movable platform in the candidate points;

and fitting a model of the driving road surface based on the target candidate points, and performing second point cloud segmentation on the three-dimensional point cloud on a u-disparity plane based on the model of the driving road surface.

In some embodiments, the processor is configured to: determining all three-dimensional points in the three-dimensional point cloud as candidate points; or performing semantic segmentation on the three-dimensional point cloud, and acquiring a plurality of candidate points in the three-dimensional point cloud based on a semantic segmentation result; or acquiring a plurality of candidate points in the three-dimensional point cloud based on the projection density of the three-dimensional point cloud on the u-disparity plane and the first reference projection density of the plane model of the driving road surface on the u-disparity plane.

In some embodiments, the processor is configured to: in a first pixel grid on the u-disparity plane, if the ratio of the projection density to the first reference projection density is greater than or equal to a first preset ratio, determining a point projected into the first pixel grid from the three-dimensional point cloud as a candidate point, where the first preset ratio is greater than 1.

In some embodiments, the first reference projected density of the planar model on the u-disparity plane is proportional to a disparity value of a point on the planar model.

In some embodiments, the three-dimensional point cloud is acquired by a vision sensor on the movable platform; the processor is configured to: acquiring an intercept of a first coordinate axis of the plane model in a coordinate system of the visual sensor, wherein the first coordinate axis is a coordinate axis in the height direction of the movable platform; determining the first reference projected density based on the intercept, a baseline length of the vision sensor, and a disparity value for a point on the planar model.

In some embodiments, the processor is configured to: calculating a ratio of the intercept to a baseline length of the vision sensor; determining the first reference projection density as a product of the ratio and a disparity value of a point on the planar model.

In some embodiments, the three-dimensional point cloud includes valid points and invalid points; the processor is configured to: and acquiring a plurality of candidate points from the effective points in the three-dimensional point cloud.

In some embodiments, the processor is configured to: filtering outlier points from the three-dimensional points; and acquiring a plurality of candidate points in the filtered three-dimensional point cloud.

In some embodiments, the processor is further configured to: acquiring preset scale transformation parameters; carrying out scale transformation on the u coordinate value of each three-dimensional point in the three-dimensional point cloud based on the scale transformation parameter; projecting the scaled three-dimensional point cloud onto the u-disparity plane.

In some embodiments, the scaling parameter for a three-dimensional point corresponds to a u-coordinate value for the three-dimensional point.

In some embodiments, if the u-coordinate value of a first three-dimensional point in the three-dimensional point cloud is smaller than a first preset coordinate value, the scale conversion parameter of the first three-dimensional point is greater than 1; and/or if the u coordinate value of the first three-dimensional point in the three-dimensional point cloud is larger than a second preset coordinate value, the scale transformation parameter of the first three-dimensional point is smaller than 1.

In some embodiments, the processor is configured to: determining, for each candidate point of the plurality of candidate points, a search cost of the candidate point; determining a target candidate point from the candidate points based on the search costs of the candidate points.

In some embodiments, the processor is configured to: and if the search cost of the candidate point is less than the preset cost, determining the candidate point as a target candidate point.

In some embodiments, the processor is configured to: and performing polynomial fitting on the target candidate points based on a least square method to obtain a polynomial model of the running road surface.

In some embodiments, the processor is configured to: acquiring the slope of the model of the driving road surface; determining a second reference projected density of the driving road surface on the u-disparity plane based on the slope; and performing second point cloud segmentation on the three-dimensional point cloud based on the second reference projection density.

In some embodiments, the processor is configured to: in a second pixel grid on the u-disparity plane, if the ratio of the projection density of the three-dimensional point cloud on the u-disparity plane to the second reference projection density is greater than or equal to a second preset ratio, dividing the point projected into the second pixel grid from the three-dimensional point cloud into a target point on the driving road surface, where the second preset ratio is greater than or equal to 1.

In some embodiments, the processor is configured to: determining a depth of the driving surface; determining the second reference projected density based on a model of the driving surface, a slope of the model, and a depth of the driving surface.

In some embodiments, the three-dimensional point cloud is acquired by a visual sensor on the movable platform; the processor is configured to: calculating a product of a slope of the model and a depth of the driving road surface; calculating a difference between the model of the driving road surface and the product; determining the second baseline projection density based on a ratio of the difference to a baseline length of the vision sensor.

In some embodiments, the processor is further configured to: and marking each three-dimensional point in the three-dimensional point cloud with a label based on the point cloud segmentation result, wherein the label of one three-dimensional point is used for representing the category of the three-dimensional point.

In some embodiments, the processor is configured to: and marking each three-dimensional point in the three-dimensional point cloud with a label based on the point cloud segmentation result and the height of each three-dimensional point in the three-dimensional point cloud.

In some embodiments, the processor is configured to: and if the height of the three-dimensional point is lower than that of the driving road surface, determining the label of the three-dimensional point as a first label, wherein the first label is used for representing that the three-dimensional point is a reflection point.

In some embodiments, the processor is configured to: determining, for each three-dimensional point in the three-dimensional point cloud, a first confidence that the three-dimensional point is a point on the driving road surface based on the height of the three-dimensional point; determining a second confidence degree that the three-dimensional point is a point on the driving road surface based on the point cloud segmentation result; and labeling the three-dimensional point based on the first confidence coefficient and the second confidence coefficient of the three-dimensional point.

In some embodiments, the processor is configured to: if at least one of the first confidence degree and the second confidence degree of the three-dimensional point is greater than a preset confidence degree, determining that the label of the three-dimensional point is a first label, wherein the first label is used for indicating that the three-dimensional point is a point on the driving road surface.

In some embodiments, the three-dimensional point cloud is acquired based on a vision sensor or a lidar mounted on the movable platform; and/or the point cloud segmentation result is used for a planning unit on the movable platform to plan the driving state of the movable platform.

For specific embodiments of the method executed by the processor in the point cloud segmentation apparatus according to the embodiments of the present disclosure, reference may be made to the foregoing method embodiments, and details are not repeated here.

Fig. 7 is a schematic diagram illustrating a more specific hardware structure of a data processing apparatus according to an embodiment of the present disclosure, where the apparatus may include: a processor 701, a memory 702, an input/output interface 703, a communication interface 704, and a bus 705. Wherein the processor 701, the memory 702, the input/output interface 703 and the communication interface 704 are communicatively connected to each other within the device via a bus 705.

The processor 701 may be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an Application Specific Integrated Circuit (ASIC), or one or more Integrated circuits, and is configured to execute related programs to implement the technical solutions provided in the embodiments of the present specification.

The Memory 702 may be implemented in the form of a ROM (Read Only Memory), a RAM (Random Access Memory), a static storage device, a dynamic storage device, or the like. The memory 702 may store an operating system and other application programs, and when the technical solution provided by the embodiments of the present specification is implemented by software or firmware, the relevant program codes are stored in the memory 702 and called to be executed by the processor 701.

The input/output interface 703 is used for connecting an input/output module to realize information input and output. The i/o module may be configured as a component in a device (not shown) or may be external to the device to provide a corresponding function. The input devices may include a keyboard, a mouse, a touch screen, a microphone, various sensors, etc., and the output devices may include a display, a speaker, a vibrator, an indicator light, etc.

The communication interface 704 is used for connecting a communication module (not shown in the figure) to realize communication interaction between the device and other devices. The communication module can realize communication in a wired mode (for example, USB, network cable, etc.), and can also realize communication in a wireless mode (for example, mobile network, WIFI, bluetooth, etc.).

Bus 705 includes a pathway for communicating information between various components of the device, such as processor 701, memory 702, input/output interface 703, and communication interface 704.

It should be noted that although the above-mentioned device only shows the processor 701, the memory 702, the input/output interface 703, the communication interface 704 and the bus 705, in a specific implementation, the device may also include other components necessary for normal operation. In addition, those skilled in the art will appreciate that the above-described apparatus may also include only those components necessary to implement the embodiments of the present description, and not necessarily all of the components shown in the figures.

As shown in fig. 8, embodiments of the present disclosure also provide a movable platform 800, including a housing 801; the point cloud acquisition device 802 is arranged on the shell 801 and is used for acquiring three-dimensional point cloud; and a three-dimensional point cloud segmentation device 803, which is arranged in the housing 801 and is used for executing the method according to any embodiment of the disclosure. Wherein, the movable platform 800 can be an unmanned aerial vehicle, an unmanned ship, a movable robot, etc., and the point cloud collection device 802 can be a vision sensor (such as a binocular vision sensor, a trinocular vision sensor, etc.) or a laser radar.

The embodiments of the present disclosure also provide a computer-readable storage medium, on which a computer program is stored, which when executed by a processor implements the steps performed by the second processing unit in the method according to any of the foregoing embodiments.

Computer-readable media, including both permanent and non-permanent, removable and non-removable media, may implement the information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), Digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device. As defined herein, a computer readable medium does not include a transitory computer readable medium such as a modulated data signal and a carrier wave.

From the above description of the embodiments, it is clear to those skilled in the art that the embodiments of the present disclosure can be implemented by software plus necessary general hardware platform. Based on such understanding, the technical solutions of the embodiments of the present specification may be essentially or partially implemented in the form of a software product, which may be stored in a storage medium, such as a ROM/RAM, a magnetic disk, an optical disk, etc., and includes several instructions for enabling a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the methods described in the embodiments or some parts of the embodiments of the present specification.

The systems, devices, modules or units illustrated in the above embodiments may be implemented by a computer chip or an entity, or by a product with certain functions. A typical implementation device is a computer, which may be in the form of a personal computer, laptop, cellular telephone, camera phone, smart phone, personal digital assistant, media player, navigation device, email messaging device, game console, tablet computer, wearable device, or a combination of any of these devices.

Various technical features in the above embodiments may be arbitrarily combined as long as there is no conflict or contradiction in the combination between the features, but the combination is limited by the space and is not described one by one, and therefore, any combination of various technical features in the above embodiments also belongs to the scope of the present disclosure.

Other embodiments of the present disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of the disclosure following, in general, the principles of the disclosure and including such departures from the present disclosure as come within known or customary practice within the art to which the disclosure pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims.

It will be understood that the present disclosure is not limited to the precise arrangements described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

The above description is only exemplary of the present disclosure and should not be taken as limiting the disclosure, as any modification, equivalent replacement, or improvement made within the spirit and principle of the present disclosure should be included in the scope of the present disclosure.

Claims

A three-dimensional point cloud segmentation method is used for performing point cloud segmentation on three-dimensional point cloud acquired by a movable platform, and the method comprises the following steps:

acquiring a plurality of candidate points in the three-dimensional point cloud;

searching the candidate points on a v-disparity plane, and determining a target candidate point on the traveling road surface of the movable platform in the candidate points;

and fitting a model of the driving road surface based on the target candidate points, and performing second point cloud segmentation on the three-dimensional point cloud on a u-disparity plane based on the model of the driving road surface to obtain a point cloud segmentation result.
The method of claim 1, wherein obtaining a plurality of candidate points in a three-dimensional point cloud comprises:

determining all three-dimensional points in the three-dimensional point cloud as candidate points; or

Performing semantic segmentation on the three-dimensional point cloud, and acquiring a plurality of candidate points in the three-dimensional point cloud based on a semantic segmentation result; or

And acquiring a plurality of candidate points in the three-dimensional point cloud based on the projection density of the three-dimensional point cloud on the u-disparity plane and the first reference projection density of the plane model of the driving road surface on the u-disparity plane.
The method of claim 2, wherein the obtaining a plurality of candidate points in the three-dimensional point cloud based on the projection density of the three-dimensional point cloud on a u-disparity plane and a first reference projection density of a plane model of the driving road surface on the u-disparity plane comprises:

in a first pixel grid on the u-disparity plane, if the ratio of the projection density to the first reference projection density is greater than or equal to a first preset ratio, determining a point projected into the first pixel grid from the three-dimensional point cloud as a candidate point, where the first preset ratio is greater than 1.
The method of claim 2, wherein the first reference projection density of the planar model on the u-disparity plane is proportional to the disparity value of the point on the planar model.
The method of claim 4, wherein the three-dimensional point cloud is acquired by a vision sensor on the movable platform; the first reference projection density is determined based on:

acquiring an intercept of a first coordinate axis of the plane model in a coordinate system of the visual sensor, wherein the first coordinate axis is a coordinate axis of the movable platform in the height direction;

determining the first reference projected density based on the intercept, a baseline length of the vision sensor, and a disparity value for a point on the planar model.
The method of claim 5, wherein determining the first reference projected density based on the intercept, a baseline length of the visual sensor, and a disparity value for a point on the planar model comprises:

calculating a ratio of the intercept to a baseline length of the vision sensor;

determining the first reference projection density as a product of the ratio and a disparity value of a point on the planar model.
The method of claim 1, wherein the three-dimensional point cloud comprises valid points and invalid points; the obtaining of the plurality of candidate points in the three-dimensional point cloud includes:

and acquiring a plurality of candidate points from the effective points in the three-dimensional point cloud.
The method of claim 1, wherein obtaining a plurality of candidate points in the three-dimensional point cloud comprises:

filtering outlier points from the three-dimensional points;

and acquiring a plurality of candidate points in the filtered three-dimensional point cloud.
The method according to claim 2, wherein before obtaining the plurality of candidate points in the three-dimensional point cloud based on the first projected density of the three-dimensional point cloud on the u-disparity plane and the first reference projected density of the plane model of the driving road surface on the u-disparity plane, the method further comprises:

acquiring preset scale transformation parameters;

carrying out scale transformation on the u coordinate value of each three-dimensional point in the three-dimensional point cloud based on the scale transformation parameter;

projecting the scaled three-dimensional point cloud onto the u-disparity plane.
The method of claim 9, wherein the scaling parameter for a three-dimensional point corresponds to a u-coordinate value for the three-dimensional point.
The method of claim 10, wherein if the u-coordinate of the first three-dimensional point in the three-dimensional point cloud is smaller than a first predetermined coordinate, the scaling parameter of the first three-dimensional point is greater than 1; and/or

And if the u coordinate value of the first three-dimensional point in the three-dimensional point cloud is greater than or equal to a second preset coordinate value, the scale transformation parameter of the first three-dimensional point is less than 1.
The method of claim 1, wherein searching the plurality of candidate points on a v-disparity plane comprises:

determining, for each candidate point of the plurality of candidate points, a search cost of the candidate point;

determining a target candidate point from the candidate points based on the search costs of the candidate points.
The method of claim 12, wherein determining a target candidate point from the candidate points based on the candidate point search costs comprises:

and if the search cost of the candidate point is less than the preset cost, determining the candidate point as a target candidate point.
The method of claim 12, wherein the search cost comprises a first search cost and a second search cost; wherein the first search cost is used for characterizing whether a target candidate point is observed on the candidate point, and the second search cost is used for characterizing whether the candidate point and a neighborhood target candidate point of the candidate point are smooth.
The method of claim 1, wherein fitting the model of the driving surface based on the target candidate points comprises:

and performing polynomial fitting on the target candidate points based on a least square method to obtain a polynomial model of the running road surface.
The method of claim 1, wherein the second point cloud segmentation of the three-dimensional point cloud based on the model of the driving surface comprises:

acquiring the slope of the model of the driving road surface;

determining a second reference projected density of the driving road surface on the u-disparity plane based on the slope;

and performing second point cloud segmentation on the three-dimensional point cloud based on the second reference projection density.
The method of claim 16, wherein the second point cloud segmentation of the three-dimensional point cloud based on the second reference projection density comprises:

in a second pixel grid on the u-disparity plane, if the ratio of the projection density of the three-dimensional point cloud on the u-disparity plane to the second reference projection density is greater than or equal to a second preset ratio, dividing the point projected into the second pixel grid from the three-dimensional point cloud into a target point on the driving road surface, where the second preset ratio is greater than or equal to 1.
The method of claim 16, wherein said determining a second reference projected density of said driving surface on said u-disparity plane based on said slope comprises:

determining a depth of the driving surface;

determining the second reference projected density based on a model of the driving surface, a slope of the model, and a depth of the driving surface.
The method of claim 18, wherein the three-dimensional point cloud is acquired by a vision sensor on the movable platform; the determining the second reference projected density based on the model of the driving surface, the slope of the model, and the depth of the driving surface includes:

calculating a product of a slope of the model and a depth of the driving road surface;

calculating a difference between the model of the driving road surface and the product;

determining the second baseline projection density based on a ratio of the difference to a baseline length of the vision sensor.
The method of claim 1, further comprising:

and marking each three-dimensional point in the three-dimensional point cloud with a label based on the point cloud segmentation result, wherein the label of one three-dimensional point is used for representing the category of the three-dimensional point.
The method of claim 20, wherein the labeling each three-dimensional point in the three-dimensional point cloud based on the point cloud segmentation results comprises:

and marking each three-dimensional point in the three-dimensional point cloud with a label based on the point cloud segmentation result and the height of each three-dimensional point in the three-dimensional point cloud.
The method of claim 21, wherein the labeling each three-dimensional point in the three-dimensional point cloud based on the point cloud segmentation results and the height of each three-dimensional point in the three-dimensional point cloud comprises:

and if the height of the three-dimensional point is lower than that of the driving road surface, determining the label of the three-dimensional point as a first label, wherein the first label is used for representing that the three-dimensional point is a reflection point.
The method of claim 21, wherein the labeling each three-dimensional point in the three-dimensional point cloud based on the point cloud segmentation results and the height of each three-dimensional point in the three-dimensional point cloud comprises:

determining, for each three-dimensional point in the three-dimensional point cloud, a first confidence that the three-dimensional point is a point on the driving road surface based on the height of the three-dimensional point;

determining a second confidence degree that the three-dimensional point is a point on the driving road surface based on the point cloud segmentation result;

and labeling the three-dimensional point based on the first confidence degree and the second confidence degree of the three-dimensional point.
The method of claim 23, wherein said labeling the three-dimensional point based on the first confidence level and the second confidence level of the three-dimensional point comprises:

if at least one of the first confidence degree and the second confidence degree of the three-dimensional point is greater than a preset confidence degree, determining that the label of the three-dimensional point is a first label, wherein the first label is used for indicating that the three-dimensional point is a point on the driving road surface.
The method of claim 1, wherein the three-dimensional point cloud is obtained based on a vision sensor or a lidar acquisition mounted on the movable platform; and/or

And the point cloud segmentation result is used for planning the driving state of the movable platform by a planning unit on the movable platform.
A three-dimensional point cloud segmentation device comprises a processor, and is characterized in that the three-dimensional point cloud segmentation device is used for performing point cloud segmentation on a three-dimensional point cloud acquired by a movable platform, and the processor is used for executing the following steps:

acquiring a plurality of candidate points in the three-dimensional point cloud;

searching the candidate points on a v-disparity plane, and determining a target candidate point on the traveling road surface of the movable platform in the candidate points;

and fitting a model of the driving road surface based on the target candidate points, and performing second point cloud segmentation on the three-dimensional point cloud on the u-disparity plane based on the model of the driving road surface.
The apparatus of claim 26, wherein the processor is configured to:

determining all three-dimensional points in the three-dimensional point cloud as candidate points; or

Performing semantic segmentation on the three-dimensional point cloud, and acquiring a plurality of candidate points in the three-dimensional point cloud based on a semantic segmentation result; or

And acquiring a plurality of candidate points in the three-dimensional point cloud based on the projection density of the three-dimensional point cloud on the u-disparity plane and the first reference projection density of the plane model of the driving road surface on the u-disparity plane.
The apparatus of claim 27, wherein the processor is configured to:

in a first pixel grid on the u-disparity plane, if the ratio of the projection density to the first reference projection density is greater than or equal to a first preset ratio, determining a point projected into the first pixel grid from the three-dimensional point cloud as a candidate point, where the first preset ratio is greater than 1.
The apparatus of claim 27, wherein a first reference projection density of the planar model on the u-disparity plane is proportional to a disparity value of a point on the planar model.
The apparatus of claim 29, wherein the three-dimensional point cloud is captured by a vision sensor on the movable platform; the processor is configured to:

acquiring an intercept of a first coordinate axis of the plane model in a coordinate system of the visual sensor, wherein the first coordinate axis is a coordinate axis in the height direction of the movable platform;

determining the first reference projected density based on the intercept, a baseline length of the vision sensor, and a disparity value for a point on the planar model.
The apparatus of claim 30, wherein the processor is configured to:

calculating a ratio of the intercept to a baseline length of the vision sensor;

determining the first reference projection density as a product of the ratio and a disparity value of a point on the planar model.
The apparatus of claim 26, wherein the three-dimensional point cloud comprises valid points and invalid points; the processor is configured to:

and acquiring a plurality of candidate points from the effective points in the three-dimensional point cloud.
The apparatus of claim 26, wherein the processor is configured to:

filtering outlier points from the three-dimensional points;

and acquiring a plurality of candidate points in the filtered three-dimensional point cloud.
The apparatus of claim 27, wherein the processor is further configured to:

acquiring preset scale transformation parameters;

carrying out scale transformation on the u coordinate value of each three-dimensional point in the three-dimensional point cloud based on the scale transformation parameter;

projecting the scaled three-dimensional point cloud onto the u-disparity plane.
The apparatus of claim 34, wherein the scaling parameter for a three-dimensional point corresponds to a u-coordinate value for the three-dimensional point.
The apparatus of claim 35, wherein if the u-coordinate of the first three-dimensional point in the three-dimensional point cloud is smaller than a first predetermined coordinate, the scaling parameter of the first three-dimensional point is greater than 1; and/or

And if the u coordinate value of the first three-dimensional point in the three-dimensional point cloud is larger than a second preset coordinate value, the scale transformation parameter of the first three-dimensional point is smaller than 1.
The apparatus of claim 26, wherein the processor is configured to:

determining, for each candidate point of the plurality of candidate points, a search cost of the candidate point;

determining a target candidate point from the candidate points based on the search costs of the candidate points.
The apparatus of claim 37, wherein the processor is configured to:

and if the search cost of the candidate point is less than the preset cost, determining the candidate point as a target candidate point.
The apparatus of claim 37, wherein the search cost comprises a first search cost and a second search cost; wherein the first search cost is used for characterizing whether a target candidate point is observed on the candidate point, and the second search cost is used for characterizing whether the candidate point and a neighborhood target candidate point of the candidate point are smooth.
The apparatus of claim 26, wherein the processor is configured to:

and performing polynomial fitting on the target candidate points based on a least square method to obtain a polynomial model of the running road surface.
The apparatus of claim 26, wherein the processor is configured to:

acquiring the slope of the model of the driving road surface;

determining a second reference projected density of the driving road surface on the u-disparity plane based on the slope;

and performing second point cloud segmentation on the three-dimensional point cloud based on the second reference projection density.
The apparatus of claim 41, wherein the processor is configured to:

in a second pixel grid on the u-disparity plane, if the ratio of the projection density of the three-dimensional point cloud on the u-disparity plane to the second reference projection density is greater than or equal to a second preset ratio, dividing the point projected into the second pixel grid from the three-dimensional point cloud into a target point on the driving road surface, where the second preset ratio is greater than or equal to 1.
The apparatus of claim 41, wherein the processor is configured to:

determining a depth of the driving surface;

determining the second reference projected density based on a model of the driving surface, a slope of the model, and a depth of the driving surface.
The apparatus of claim 43, wherein the three-dimensional point cloud is captured by a vision sensor on the movable platform; the processor is configured to:

calculating a product of a slope of the model and a depth of the driving road surface;

calculating a difference between the model of the driving road surface and the product;

determining the second baseline projection density based on a ratio of the difference to a baseline length of the vision sensor.
The apparatus of claim 26, wherein the processor is further configured to:

and marking each three-dimensional point in the three-dimensional point cloud with a label based on the point cloud segmentation result, wherein the label of one three-dimensional point is used for representing the category of the three-dimensional point.
The apparatus of claim 45, wherein the processor is configured to:

and marking each three-dimensional point in the three-dimensional point cloud with a label based on the point cloud segmentation result and the height of each three-dimensional point in the three-dimensional point cloud.
The apparatus of claim 46, wherein the processor is configured to:

and if the height of the three-dimensional point is lower than that of the driving road surface, determining the label of the three-dimensional point as a first label, wherein the first label is used for representing that the three-dimensional point is a reflection point.
The apparatus of claim 46, wherein the processor is configured to:

determining, for each three-dimensional point in the three-dimensional point cloud, a first confidence that the three-dimensional point is a point on the driving road surface based on the height of the three-dimensional point;

determining a second confidence degree that the three-dimensional point is a point on the driving road surface based on the point cloud segmentation result;

and labeling the three-dimensional point based on the first confidence degree and the second confidence degree of the three-dimensional point.
The apparatus of claim 48, wherein the processor is configured to:

if at least one of the first confidence degree and the second confidence degree of the three-dimensional point is greater than a preset confidence degree, determining that the label of the three-dimensional point is a first label, wherein the first label is used for indicating that the three-dimensional point is a point on the driving road surface.
The apparatus of claim 26, wherein the three-dimensional point cloud is acquired based on a vision sensor or a lidar mounted on the movable platform; and/or

And the point cloud segmentation result is used for planning the driving state of the movable platform by a planning unit on the movable platform.
A movable platform, comprising:

a housing;

the point cloud acquisition device is arranged on the shell and used for acquiring three-dimensional point cloud; and

a three-dimensional point cloud segmentation apparatus disposed within the housing for performing the method of any one of claims 1 to 25.
A computer-readable storage medium having stored thereon computer instructions which, when executed by a processor, implement the method of any one of claims 1 to 25.