WO2022099620A1

WO2022099620A1 - Three-dimensional point cloud segmentation method and apparatus, and mobile platform

Info

Publication number: WO2022099620A1
Application number: PCT/CN2020/128711
Authority: WO
Inventors: 李星河; 韩路新; 于亦奇
Original assignee: 深圳市大疆创新科技有限公司
Priority date: 2020-11-13
Filing date: 2020-11-13
Publication date: 2022-05-19
Also published as: CN114631124A

Abstract

A three-dimensional point cloud segmentation method and apparatus, and a mobile platform, said method and apparatus being used for performing point cloud segmentation on a three-dimensional point cloud collected by a mobile platform. Said method comprises: acquiring a plurality of candidate points in the three-dimensional point cloud (301); searching the plurality of candidate points on a v-disparity plane to determine target candidate points, located on a driving road surface of the mobile platform, among the plurality of candidate points (302); and obtaining, by fitting, a model of the driving road surface on the basis of the target candidate points, and on the u-disparity plane, performing second point cloud segmentation on the three-dimensional point cloud on the basis of the model of the driving road surface, so as to obtain a point cloud segmentation result (303).

Description

Three-dimensional point cloud segmentation method and device, and movable platform

technical field

The present disclosure relates to the technical field of computer vision, and in particular, to a three-dimensional point cloud segmentation method and device, and a movable platform.

Background technique

During the traveling process of the movable platform, a path planning module on the movable platform can perform decision planning on the traveling state (eg, pose and speed) of the movable platform. In order to enable the planning module to complete the decision-making planning, the point cloud acquisition device on the movable platform needs to collect the 3D point cloud of the surrounding environment, and perform point cloud segmentation to distinguish the ground and obstacles in the 3D point cloud, and further Distinguish dynamic and static objects from obstacles. Therefore, point cloud segmentation is an important part of decision planning for the driving state of the mobile platform.

Traditional point cloud segmentation methods are generally based on the local features of 3D point clouds to segment ground points and non-ground points. However, the above methods will obviously degenerate when dealing with 3D point clouds with a long distance, and the segmentation accuracy is low.

SUMMARY OF THE INVENTION

In view of this, the embodiments of the present disclosure propose a three-dimensional point cloud segmentation method and device, and a movable platform, so as to accurately perform point cloud segmentation on the three-dimensional point cloud collected by the movable platform.

According to a first aspect of the embodiments of the present disclosure, there is provided a method for segmenting a 3D point cloud, which is used to segment a 3D point cloud collected by a movable platform, the method comprising: acquiring multiple points in the 3D point cloud. search for the multiple candidate points on the v-disparity plane, and determine the target candidate point located on the road surface of the movable platform among the multiple candidate points; fit the target candidate point based on the The model of the driving road surface is obtained, and a second point cloud segmentation is performed on the three-dimensional point cloud based on the model of the driving road surface on the u-disparity plane to obtain a point cloud segmentation result.

According to a second aspect of the embodiments of the present disclosure, there is provided a three-dimensional point cloud segmentation device, including a processor, the three-dimensional point cloud segmentation device is configured to perform point cloud segmentation on a three-dimensional point cloud collected by a movable platform, and the processing The device is configured to perform the following steps: acquiring multiple candidate points in the three-dimensional point cloud; searching the multiple candidate points on the v-disparity plane, and determining that the multiple candidate points are located on the movable platform for driving target candidate points on the road surface; fit a model of the driving road surface based on the target candidate points, and perform a second point cloud segmentation on the three-dimensional point cloud based on the model of the driving road surface on the u-disparity plane.

According to a third aspect of the embodiments of the present disclosure, a movable platform is provided, which is characterized by comprising: a casing; a point cloud collecting device, disposed on the casing, for collecting a three-dimensional point cloud; and a three-dimensional point cloud A dividing device, which is arranged in the casing, is used for executing the method described in any embodiment of the present disclosure.

According to a fourth aspect of the embodiments of the present disclosure, there is provided a computer-readable storage medium on which a computer program is stored, and when the program is executed by a processor, implements the method described in any of the embodiments of the present disclosure.

Applying the solution of the embodiment of the present disclosure, firstly, the candidate points are searched on the v-disparity plane, the target candidate points located on the road surface of the movable platform among the plurality of candidate points are determined, and then based on the target candidate points The model of the driving road surface is combined, and the model is used as the benchmark for point cloud segmentation, and the second point cloud segmentation is performed on the three-dimensional point cloud based on the model of the driving road surface on the u-disparity plane, which improves the accuracy of point cloud segmentation. The accuracy makes it possible to accurately segment the areas that are difficult to segment, such as slopes and distances in the 3D point cloud.

Description of drawings

In order to illustrate the technical solutions in the embodiments of the present disclosure more clearly, the following briefly introduces the accompanying drawings used in the description of the embodiments. Obviously, the accompanying drawings in the following description are only some embodiments of the present disclosure. For those of ordinary skill in the art, other drawings can also be obtained from these drawings without creative labor.

Figure 1 is a schematic diagram of a point cloud segmentation process of some embodiments.

FIG. 2 is a schematic diagram of a decision-making planning process during travel of a mobile platform according to some embodiments.

FIG. 3 is a flowchart of a point cloud segmentation method according to an embodiment of the present disclosure.

4A and 4B are schematic diagrams of the uvd coordinate system according to an embodiment of the present disclosure, respectively.

FIG. 5 is a schematic diagram of a projection process of a u-disparity plane according to an embodiment of the present disclosure.

FIG. 6 is a schematic diagram of the relationship between parallax and depth according to an embodiment of the present disclosure.

FIG. 7 is a schematic diagram of a point cloud segmentation apparatus according to an embodiment of the present disclosure.

FIG. 8 is a schematic diagram of a movable platform according to an embodiment of the present disclosure.

Detailed ways

Exemplary embodiments will be described in detail herein, examples of which are illustrated in the accompanying drawings. Where the following description refers to the drawings, the same numerals in different drawings refer to the same or similar elements unless otherwise indicated. The implementations described in the illustrative examples below are not intended to represent all implementations consistent with this disclosure. Rather, they are merely examples of apparatus and methods consistent with some aspects of the present disclosure as recited in the appended claims.

The terminology used in the present disclosure is for the purpose of describing particular embodiments only and is not intended to limit the present disclosure. As used in this disclosure and the appended claims, the singular forms "a," "the," and "the" are intended to include the plural forms as well, unless the context clearly dictates otherwise. It will also be understood that the term "and/or" as used herein refers to and includes any and all possible combinations of one or more of the associated listed items.

It should be understood that although the terms first, second, third, etc. may be used in this disclosure to describe various pieces of information, such information should not be limited by these terms. These terms are only used to distinguish the same type of information from each other. For example, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information, without departing from the scope of the present disclosure. Depending on the context, the word "if" as used herein can be interpreted as "at the time of" or "when" or "in response to determining."

During the traveling process of the movable platform, a path planning module on the movable platform can be used to make decision planning on the traveling state of the movable platform. Among them, point cloud segmentation is an important part of decision-making planning for the driving state of the mobile platform. As shown in FIG. 1 , it is a schematic diagram of a point cloud segmentation process in some embodiments. In step 101, a 3D point cloud can be collected by a point cloud collection device on the movable platform, and then, in step 102, for a movable platform (such as an unmanned vehicle) running on the ground, the collected 3D point cloud can be collected. The point cloud is divided into ground points and non-ground points. For other types of mobile platforms (such as mobile robots), the collected 3D point cloud can be segmented to segment the 3D points in the 3D point cloud into points on the road that the mobile platform is driving on and those not driving on the mobile platform. point on the road. For the convenience of description, the following description will be made by taking the driving road as the ground. In step 103, if a 3D point is a ground point, step 104 is performed to add a ground point label to the 3D point; otherwise, step 105 is performed to perform dynamic and static segmentation on the 3D point, that is, segment the 3D point into stationary static point and dynamic point where motion occurs. In step 106, if a 3D point is a static point, step 107 is performed to add a static point label to the 3D point; otherwise, step 108 is performed to add a dynamic point label to the 3D point, and in step 109 the labelled output is output. 3D point cloud to downstream modules. Among them, all or part of the three-dimensional points in the three-dimensional point cloud can be marked. The label may include at least one of a first label used to characterize whether the 3D point is a ground point and a second label used to characterize whether the 3D point is a static point, and may also include a label used to characterize other information of the 3D point. Label.

The downstream module may be a planning module on a movable platform, such as an electronic control unit (Electronic Control Unit, ECU), a central processing unit (Central Processing Unit, CPU) and the like. After receiving the labeled 3D point cloud, the Planning module can make decision planning on the driving state of the movable platform based on the label of the 3D point. The driving state may include at least one of a pose and a speed of the movable platform. As shown in FIG. 2 , it is a schematic diagram of the decision planning process of some embodiments. In step 201 and step 202, the planning module can receive the 3D point cloud and read the tags carried in the 3D point cloud. In step 203, it may be determined whether the three-dimensional point in the three-dimensional point cloud is a point on the road (eg, ground) on which the movable platform travels based on the label. Taking the ground point as an example, if it is, go to step 204, identify the three-dimensional point belonging to the lane line from the ground point, and determine the posture of the movable platform according to the direction of the lane line, so that the movable platform can follow the direction of the lane line. drive in the direction. If it is a non-ground point, step 205 is executed to determine whether the non-ground point is a static point. If yes, step 206 is executed to determine the pose of the movable platform according to the orientation of the static point. For example, it is judged whether the static point is on the pre-planned travel path, and if so, the path is re-planned to avoid the movable platform colliding with the static point. If the non-ground point is a dynamic point, step 207 is executed to determine at least one of the attitude and speed of the movable platform according to the orientation and speed of the static point. For example, if the dynamic point is on the pre-planned travel path of the movable platform, and the moving speed of the dynamic point is less than or equal to the moving speed of the movable platform, control the movable platform to slow down, or adjust the posture of the movable platform, so that the movable platform bypasses the dynamic point. As another example, the movable platform can be controlled to travel at the same speed as the dynamic point.

It can be seen that point cloud segmentation is an important part of decision-making and planning for the driving state of the mobile platform, and accurate point cloud segmentation is helpful for accurate decision-making and planning of the driving state of the mobile platform. The current point cloud segmentation methods are mainly based on local features. Specifically, the three-dimensional point cloud is transformed into the xyz space, and rasterization or proximity search is performed in the xyz space. Find the adjacent points around the candidate point, and determine the probability that the candidate point belongs to the ground point according to the thickness, height and normal vector of the adjacent point cloud. This method will have obvious degradation when dealing with 3D point clouds with a long distance, the segmentation accuracy is low, and it is difficult to establish a global ground model, and it is impossible to make correct judgments for non-ground planes.

Based on this, the present disclosure provides a three-dimensional point cloud segmentation method, which is used to perform point cloud segmentation on a three-dimensional point cloud collected by a movable platform. As shown in FIG. 3 , the method includes:

Step 301: Obtain multiple candidate points in the 3D point cloud;

Step 302: Search the multiple candidate points on the v-disparity plane, and determine a target candidate point located on the road surface of the movable platform among the multiple candidate points;

Step 303: Fit the model of the driving road surface based on the target candidate points, and perform a second point cloud segmentation on the three-dimensional point cloud based on the model of the driving road surface on the u-disparity plane to obtain a point cloud segmentation result.

In step 301, a three-dimensional point cloud may be collected by a point cloud collection device (eg, lidar, vision sensor, etc.) on the movable platform. The movable platform may be an unmanned vehicle, an unmanned aerial vehicle, an unmanned ship, a movable robot, and the like.

The candidate points may be some or all of the points in the three-dimensional point cloud. Optionally, semantic segmentation may be performed on the three-dimensional point cloud, and multiple candidate points in the three-dimensional point cloud may be acquired based on the semantic segmentation result. Through semantic segmentation, the categories of multiple 3D points in the 3D point cloud can be obtained, such as vehicle category, traffic light category, pedestrian category, lane line category, etc. Then, candidate points can be determined based on the categories of the three-dimensional points. For example, three-dimensional points of the lane line category are determined as candidate points.

Optionally, the 3D point cloud may also be pre-segmented on the u-disparity plane, and candidate points are determined based on the pre-segmentation result. Specifically, the 3D point cloud can be obtained based on the projection density of the 3D point cloud on the u-disparity plane and the first reference projection density of the plane model of the driving road on the u-disparity plane. multiple candidate points. In this embodiment, the traveling road surface of the movable platform is assumed to be a plane, a first reference projection density is determined based on the plane, and then pre-segmentation is performed based on the first reference projection density to determine candidate points on the traveling surface of the movable platform. The pre-segmentation on the u-disparity plane can improve the signal-to-noise ratio of the candidate points, so that the candidate points can be selected in long-distance regions (with less signal amount), and the distance and accuracy of the search on the v-disparity plane can be improved.

Wherein, each coordinate axis in the uvd space can be determined based on the direction of the driving road surface of the movable platform. For example, in FIG. 4A , the movable platform 401 is driving on the horizontal ground 402, then the u-axis, v-axis and d-axis (ie, the disparity axis) can be the coordinate axes on the ground that are perpendicular to the traveling direction of the movable platform, and the A coordinate axis parallel to the traveling direction of the movable platform, and a coordinate axis in the height direction of the movable platform (ie, vertically upward). In FIG. 4B, the movable platform 404 is a glass-cleaning robot traveling on the vertical glass plane 403, then the u-axis, v-axis and d-axis may be the coordinate axes on the glass plane that are perpendicular to the traveling direction of the movable platform, The coordinate axis on the glass plane that is parallel to the traveling direction of the movable platform, and the coordinate axis in the height direction (ie, the horizontal direction) of the movable platform. In addition, each coordinate axis of the uvd space may also point in other directions, and the specific direction may be set according to actual needs, which is not limited in the present disclosure.

In some embodiments, the u-disparity plane may be pre-divided into multiple grids. In a first grid of the plurality of grids, if the ratio of the projection density to the first reference projection density is greater than or equal to a first preset ratio, project the three-dimensional point cloud to the Points in the first pixel grid are determined as candidate points. Wherein, the first preset ratio is greater than 1.

As shown in Figure 5, the u-disparity plane can be pre-divided into multiple grids, and each grid can be of the same size in order to compare projection densities. Each black point represents a projection point of a 3D point in the 3D point cloud on the u-disparity plane, and the number of projected points in a grid is equal to the number of 3D points in the 3D point cloud projected to the grid. The projected density within a grid can be determined as the ratio of the number of projected points within the grid to the area of the grid. Taking the vehicle driving on the road as an example, when the vehicle is driving, there is no obstacle in the area (called the first area) that is separated from the vehicle by d1 to d2, then each three-dimensional point in the first area is on the road. The three-dimensional points of , whose plane is parallel to the driving direction of the vehicle (that is, the direction of the disparity coordinate axis) or have a small included angle, these three-dimensional points extend along the disparity coordinate axis, and the parallax change range is large, that is, the first The density of three-dimensional points within a region is low. However, there is an obstacle in the area d3 to d4 away from the vehicle (referred to as the second area), and the plane where the obstacle is located is generally perpendicular to the driving direction of the vehicle or has a large included angle, which will hinder the vehicle from moving forward. That is to say, the parallax variation range of the three-dimensional points in the second area is small, and the density of the three-dimensional points in the second area is large. Therefore, as long as the first reference density of the travel plane of the movable platform is known, it can be roughly inferred whether a three-dimensional point is a point on the travel plane or a point outside the travel plane (eg, an obstacle).

Further, since the actual driving road may not be flat, the preset ratio λ (that is, the redundancy of the segmentation) is set to a value greater than 1 here, so as to provide a certain amount of redundancy and reduce the selection error of candidate points. . Among them, λ can be fixedly set according to the model of the vision sensor, or can also be dynamically set according to the actual application scenario. When the reliability of the vision sensor is low, λ can be set to a larger value, otherwise, λ can be set to a smaller value. For example, when the focal length of the vision sensor is long, or the surrounding light is dim, etc., λ can be set to a larger value.

In some embodiments, the first reference projection density of the plane model on the u-disparity plane is proportional to disparity values of points on the plane model. Specifically, the first coordinate axis may be determined according to the baseline length of the visual sensor, the ending of the first coordinate axis of the plane model in the coordinate system of the visual sensor, and the disparity value of points on the plane model. Baseline projected density. Assume that the plane model of the driving road is:

y=αz+h

Wherein, α is the slope of the driving road surface. A ratio of the intercept to a baseline length of the vision sensor may be calculated, and a product of the ratio and a disparity value of a point on the planar model may be determined as the first reference projection density. Then the first reference projection density of the points on the plane model on the u-disparity plane can be recorded as:

Wherein, h is the intercept, b is the baseline length, Δd is the density of projection points in a grid on the u-disparity plane, and Δv _g is the first reference projection density in the corresponding grid. The first coordinate axis is the coordinate axis in the height direction of the movable platform. For example, when the movable platform is driving on the ground shown in FIG. 4A , the first coordinate axis may be a vertically upward coordinate axis. For another example, when the movable platform travels on the glass plane shown in FIG. 4B , the first coordinate axis may be a coordinate axis in the horizontal direction.

Using the above principle, it is possible to obtain which points are points on the driving road when the model of the driving road is a plane model, so as to extract the points on the driving road as candidate points. It can be seen that the first reference projection density is only related to the intercept, baseline length and disparity value, but has nothing to do with the distance (z value) of the ground. Therefore, here, the driving road of the movable platform is assumed to be a plane, and then the first reference projection density is determined based on the plane model to segment on the u-disparity plane to determine candidate points. On the one hand, the amount of calculation is reduced, and on the other hand, the The signal-to-noise ratio of the candidate points is improved, so that the candidate points can be selected in the case of a long distance (less signal amount), which improves the subsequent search distance and accuracy on the v-disparity plane. Areas that are difficult to divide equally can also be accurately cut out.

In some embodiments, the point cloud acquisition device does not acquire a complete point cloud frame due to a clock reset or other reasons. Therefore, the acquired 3D point cloud may include both valid points and invalid points. In order to improve the reliability of acquiring candidate points, multiple candidate points may be acquired only from valid points in the three-dimensional point cloud. Among them, invalid points can be set as invalid to avoid invalid points being selected as candidate points.

In some embodiments, outlier points may also be filtered out of the three-dimensional points, and multiple candidate points in the filtered three-dimensional point cloud may be obtained. Outliers are points whose value range is outside the valid range. Outliers can be filtered out of 3D points by filtering.

In some embodiments, obtaining the Before multiple candidate points in the 3D point cloud, preset scale transformation parameters can be obtained, and based on the scale transformation parameters, scale transformation is performed on the u-coordinate values of each 3D point in the 3D point cloud, and the scaled transformation parameters are scaled. The 3D point cloud is projected onto the u-disparity plane.

The scale transformation parameter scale of a three-dimensional point is used to enlarge or reduce the u-coordinate value of the three-dimensional point. When the scale transformation parameter scale is greater than 1, it means that the u-coordinate value of the three-dimensional point is enlarged, that is, a row of projection points on the u-disparity plane is mapped to the transformed scale row. When the scale transformation parameter scale is less than 1, it means that the u-coordinate value of the three-dimensional point is reduced, that is, the scale row projection point on the u-disparity plane is mapped to the transformed first row.

The value of the scale transformation parameter of a three-dimensional point corresponds to the u-coordinate value of the three-dimensional point. For example, if the u-coordinate value of the first three-dimensional point in the three-dimensional point cloud is smaller than the first preset coordinate value, the scaling parameter of the first three-dimensional point is greater than 1. For another example, if the u-coordinate value of the first three-dimensional point in the three-dimensional point cloud is greater than or equal to the second preset coordinate value, the scaling parameter of the first three-dimensional point is less than 1. Wherein, the first preset coordinate value may be less than or equal to the second preset coordinate value. Further, if the first preset coordinate value is less than the second preset coordinate value, then when the u coordinate value of the first three-dimensional point in the three-dimensional point cloud is greater than or equal to the first preset coordinate value, and When it is less than the second preset coordinate value, the scale transformation parameter of the first three-dimensional point is equal to 1.

Further, multiple scaling parameters greater than 1 and/or multiple scaling parameters less than 1 may be set. For example, if the u-coordinate value of the first three-dimensional point is less than the third preset coordinate value, set the scale transformation parameter of the first three-dimensional point to the first parameter value; if the u-coordinate value of the first three-dimensional point is greater than or equal to the The third preset coordinate value is smaller than the first preset coordinate value, and the scale transformation parameter of the first three-dimensional point is set as the second parameter value. Wherein, both the first parameter value and the second parameter value are greater than 1, and the first parameter value is greater than the second parameter value, and the third preset coordinate value is less than the first preset coordinate value.

For another example, if the u-coordinate value of the first three-dimensional point is greater than or equal to the fourth preset coordinate value, the scale transformation parameter of the first three-dimensional point is set to the third parameter value, and if the u-coordinate value of the first three-dimensional point is less than The fourth preset coordinate value is greater than or equal to the second preset coordinate value, and the scale transformation parameter of the first three-dimensional point is set as the fourth parameter value. Wherein, the fourth preset coordinate value is greater than the second preset coordinate value, and the fourth parameter value is less than the third parameter value.

Table 1 shows the corresponding relationship between the scale parameter in some embodiments and the number of rows before and after transformation. Those skilled in the art can understand that Table 1 is only an exemplary description and is not used to limit the present disclosure. For example, when the scale transformation parameter scale is 4.0, the projection points of 1 row before transformation are mapped to the projection points of 4 rows after transformation. For example, the projection points of the 3rd row before transformation are mapped to the projection points of the 0th row to the 3rd row after transformation, and the projection points of the 4th row before transformation are mapped to the projection points of the 4th row to the 7th row after transformation, And so on.

It should be noted that the reason why the starting line number of the projection point before transformation is 3 is because the error corresponding to the point with too small parallax value is relatively large. Therefore, only the points with a disparity value greater than or equal to 3 are taken for processing here. Those skilled in the art can understand that the point before the third row can also be used without considering the error; in other cases, the starting row number of the projection point before transformation can also be set to a value greater than 3.

Table 1 The correspondence between the scale parameter and the number of rows before and after transformation

scalescale	4.04.0	2.02.0	1.01.0	0.50.5	0.250.25
变换后的行范围Transformed row range	[0,108)[0,108)	[108,128)[108,128)	[128,148)[128,148)	[148,162)[148,162)	[162,172)[162,172)
变换前的行范围range of lines before transformation	[3,30)[3,30)	[30,40)[30,40)	[40,60)[40,60)	[60,88)[60,88)	[88,128)[88,128)

Since the relationship between disparity and depth is:

z=b*f/d

Wherein, f is the focal length of the vision sensor used to collect the three-dimensional point cloud, b is the focal length of the vision sensor, d is the parallax value, and z is the depth. It can be seen that the relationship between z and d is inversely proportional, as shown in Figure 6. Here, the u-coordinate value is scaled by using the scale transformation parameter. The purpose is to counteract the characteristic of nonlinear variation of depth corresponding to disparity, compress the near high-resolution area, and give full play to the sub-pixel accuracy in the distance, thereby improving the point cloud. Segmentation accuracy. In addition, through the above scale transformation method, the number of rows of three-dimensional points after transformation is close to the number of rows before transformation, which avoids the situation that the number of rows is too large due to scale transformation, thereby greatly increasing the computing power, and realizes the computing power and the number of points. The balance between cloud segmentation accuracy.

In step 302, the candidate points selected in step 301 can be projected onto the v-disparity plane. Unlike the u-disparity plane, the v-disparity plane is of equal scale, that is, the number of rows of projection points on the v-disparity plane. One-to-one correspondence with the number of rows of the disparity map, and one-to-one correspondence between the number of columns of projection points on the v-disparity plane and the integer values of the valid range of disparity values.

The plurality of candidate points may be searched on the v-disparity plane to determine a target candidate point located on the road surface of the movable platform among the plurality of candidate points. The following takes the dynamic programming method as an example to describe the process of determining the target candidate point. In practical applications, other methods may also be adopted to determine the target candidate point, which will not be described here.

For each candidate point of the plurality of candidate points, a search cost of the candidate point may be determined, and a target candidate point may be determined from the candidate points based on the search cost of the candidate point. If the search cost of the candidate point is less than the preset cost, the candidate point is determined as the target candidate point.

In some embodiments, the search cost includes a first search cost and a second search cost; wherein the first search cost is used to characterize whether a target candidate point is observed on the candidate point, and the second search cost The cost is used to characterize whether the candidate point and the neighbor target candidate point of the candidate point are smooth.

Among them, the density cost can be calculated as follows:

p is a certain point on the v-disparity image, cost is the density cost of the point, and th is related to the parameters of the visual sensor. For example, when the driving surface of the movable platform is the ground, the width of the lane line on the ground is about 3 meters, and at a depth of 100 meters, the 3-meter wide area captured by the vision sensor is generally one frame of 3D points. The cloud image includes 5 pixels, so the value of th can be 5. In other cases, th can also be set to other values according to the actual situation.

After the target candidate points are obtained, in step 303, a model of the driving road surface may be fitted based on the target candidate points. For example, polynomial fitting may be performed on the target candidate points based on the least squares method to obtain a polynomial model of the driving road surface. The resulting model can be expressed as:

y=A+B*z+C*z ² +D*z ³ +E*z ⁴

The slope of the tangent to the model at a point is:

y'=B+2*C*z+3*D*z ² +4*E*z ³

where A, B, C, D, and E are all constants, and z is the depth. The above model is only an exemplary description, and the above model can be adjusted to a cubic polynomial model or a quintic polynomial model, etc. according to actual application scenarios. Then, the slope of the model of the driving road surface may be obtained; the second reference projection density of the driving road surface on the u-disparity plane may be determined based on the slope; Second point cloud segmentation.

The manner of performing the second point cloud segmentation on the 3D point cloud based on the second reference projection density is similar to the foregoing manner of performing the point cloud segmentation based on the first reference projection density, that is, the second pixel grid on the u-disparity plane , if the ratio of the projection density of the 3D point cloud on the u-disparity plane to the second reference projection density is greater than or equal to a second preset ratio, project the 3D point cloud to the second pixel The points in the grid are divided into target points on the road surface. Here, the second preset ratio may be set to a value greater than or equal to 1, and the second preset value may be the same as or different from the first preset value.

Wherein, the second reference projection density may be determined based on the model of the driving road surface, the slope of the model and the depth of the driving road surface. For example, a product of the slope of the model and the depth of the travel surface may be calculated, a difference between the model of the travel surface and the product may be calculated, and a determination may be made based on the ratio of the difference to the baseline length of the vision sensor is the second reference projection density, specifically as follows:

Δv _g =(yz*y')/b

The above process performs search and model fitting on the v-disparity plane, and performs point cloud segmentation on the u-disparity plane. The two steps are iteratively performed. The segmentation of the u-disparity plane can improve the signal-to-noise ratio of the ground candidate points, so that the The candidate points can be selected in long-distance regions (with less semaphore) to improve the search distance and accuracy on the v-disparity plane, and the resulting model can provide important information about the benchmark density for u-disparity segmentation, so that Areas that are difficult to segment, such as slopes and distances, can also be accurately cut out.

In some embodiments, each 3D point in the 3D point cloud may be labeled based on the point cloud segmentation result, and one 3D point label is used to represent the category of the 3D point. The categories may include a first category and a second category, wherein the first category is used to represent that the three-dimensional point belongs to a point on the road where the movable platform travels, and the second category is used to represent that the three-dimensional point belongs to a point on an obstacle. Further, the category may further include a third category, which is used to represent that the three-dimensional point does not belong to the point on the driving road, nor the point on the obstacle. The points of the third category may be reflection points, or points whose category cannot be determined, and the like.

In some embodiments, each 3D point in the 3D point cloud may be tagged based on the point cloud segmentation result and the height of each 3D point in the 3D point cloud. Here, the 3D points are labeled based on the height of the 3D point and the point cloud segmentation result, which improves the accuracy of the label. Specifically, if the height of a three-dimensional point is lower than the height of the driving road surface, the label of the three-dimensional point may be determined as a first label, and the first label is used to indicate that the three-dimensional point is a reflection point. If the height of a 3D point is not lower than the height of the driving road, it can be further combined with the point cloud segmentation result for labeling.

For example, for each 3D point in the 3D point cloud, a first confidence level that the 3D point is a point on the driving road may be determined based on the height of the 3D point; The three-dimensional point is the second confidence level of the point on the driving road surface; the three-dimensional point is labeled based on the first confidence level and the second confidence level of the three-dimensional point.

In the case that the height of a three-dimensional point is not lower than the height of the driving road, if the height of a three-dimensional point is higher, the first confidence that the three-dimensional point is a point on the driving road is lower, otherwise the first confidence is lower. high. If the ratio of the projected density of a three-dimensional point to the second reference projected density is larger, the second confidence level that the three-dimensional point is a point on the driving road is lower, otherwise the second confidence level is higher.

You can choose different ways to tag according to different scenarios. For example, in the case of high requirements for label reliability and accuracy, when both the first confidence level and the second confidence level of a 3D point are greater than the corresponding confidence threshold, the label of the 3D point may be determined as the first confidence level. A label, the first label is used to indicate that the three-dimensional point is a point on the driving road surface. In the case of low requirements on label reliability and accuracy, when at least one of the first confidence level and the second confidence level of a 3D point is greater than the corresponding confidence threshold, the label of the 3D point can be determined to be first label. The labels of the three-dimensional points may also be determined based on other methods, which will not be listed one by one here.

In practical applications, the point cloud segmentation result can be used by the planning unit on the movable platform to plan the driving state of the movable platform. For example, the planning unit can determine whether there are obstacles on the driving path based on the labels obtained from the segmentation results of the point cloud, so as to decide whether to control the speed and attitude of the movable platform to avoid obstacles.

Those skilled in the art can understand that in the above method of the specific implementation, the writing order of each step does not mean a strict execution order but constitutes any limitation on the implementation process, and the specific execution order of each step should be based on its function and possible Internal logic is determined.

An embodiment of the present disclosure further provides a point cloud segmentation device, including a processor, where the processor is configured to perform the following steps:

acquiring multiple candidate points in the three-dimensional point cloud;

searching the multiple candidate points on the v-disparity plane, and determining a target candidate point located on the road surface of the movable platform among the multiple candidate points;

A model of the driving road surface is fitted based on the target candidate points, and a second point cloud segmentation is performed on the three-dimensional point cloud based on the model of the driving road surface on the u-disparity plane.

In some embodiments, the processor is configured to: determine all 3D points in the 3D point cloud as candidate points; or perform semantic segmentation on the 3D point cloud, and obtain the 3D point cloud based on the semantic segmentation result. or based on the projection density of the 3D point cloud on the u-disparity plane and the first reference projection density of the plane model of the driving road on the u-disparity plane, obtain the 3D point Multiple candidate points in the cloud.

In some embodiments, the processor is configured to: in the first pixel grid on the u-disparity plane, if the ratio of the projected density to the first reference projected density is greater than or equal to a first preset A ratio, and a point projected into the first pixel grid in the three-dimensional point cloud is determined as a candidate point, and the first preset ratio is greater than 1.

In some embodiments, the first reference projection density of the plane model on the u-disparity plane is proportional to disparity values of points on the plane model.

In some embodiments, the three-dimensional point cloud is acquired by a vision sensor on the movable platform; the processor is configured to: acquire the first coordinate axis of the plane model in the coordinate system of the vision sensor intercept, the first coordinate axis is the coordinate axis in the height direction of the movable platform; the intercept is determined based on the intercept, the baseline length of the vision sensor and the parallax value of the point on the plane The first reference projected density.

In some embodiments, the processor is configured to: calculate a ratio of the intercept to a baseline length of the vision sensor; and determine a product of the ratio and a disparity value of a point on the plane model as the The first reference projected density.

In some embodiments, the three-dimensional point cloud includes valid points and invalid points; the processor is configured to: obtain a plurality of candidate points from the valid points in the three-dimensional point cloud.

In some embodiments, the processor is configured to: filter out outlier points from the three-dimensional points; and obtain a plurality of candidate points in the filtered three-dimensional point cloud.

In some embodiments, the processor is further configured to: obtain preset scaling parameters; perform scaling on the u-coordinate values of each 3D point in the 3D point cloud based on the scaling parameters; The transformed three-dimensional point cloud is projected onto the u-disparity plane.

In some embodiments, the scaling parameter of a 3D point corresponds to the u-coordinate value of the 3D point.

In some embodiments, if the u-coordinate value of the first three-dimensional point in the three-dimensional point cloud is smaller than the first preset coordinate value, the scaling parameter of the first three-dimensional point is greater than 1; and/or if the three-dimensional point The u-coordinate value of the first three-dimensional point in the point cloud is greater than the second preset coordinate value, and the scale transformation parameter of the first three-dimensional point is less than 1.

In some embodiments, the processor is configured to: for each candidate point in the plurality of candidate points, determine a search cost of the candidate point; determine from the candidate points based on the search cost of the candidate point target candidate point.

In some embodiments, the processor is configured to: if the search cost of the candidate point is less than a preset cost, determine the candidate point as the target candidate point.

In some embodiments, the processor is configured to: perform polynomial fitting on the target candidate points based on a least squares method to obtain a polynomial model of the driving road surface.

In some embodiments, the processor is configured to: obtain a slope of the model of the driving road surface; determine a second reference projection density of the driving road surface on the u-disparity plane based on the slope; The second point cloud segmentation is performed on the 3D point cloud by the two reference projection densities.

In some embodiments, the processor is configured to: in the second pixel grid on the u-disparity plane, if the projection density of the three-dimensional point cloud on the u-disparity plane is the same as the second reference projection The density ratio is greater than or equal to a second preset ratio, and the points projected into the second pixel grid in the three-dimensional point cloud are divided into target points on the driving road, and the second preset ratio is greater than or equal to 1.

In some embodiments, the processor is configured to: determine the depth of the driving surface; and determine the second reference projected density based on a model of the driving surface, a slope of the model, and the depth of the driving surface.

In some embodiments, the three-dimensional point cloud is collected by a vision sensor on the movable platform; the processor is configured to: calculate the product of the slope of the model and the depth of the driving surface; calculate the driving The difference between the model of the road surface and the product; the second reference projection density is determined based on the ratio of the difference to the baseline length of the vision sensor.

In some embodiments, the processor is further configured to: label each 3D point in the 3D point cloud based on the point cloud segmentation result, and a label of a 3D point is used to represent the category of the 3D point .

In some embodiments, the processor is configured to: label each 3D point in the 3D point cloud based on the point cloud segmentation result and the height of each 3D point in the 3D point cloud.

In some embodiments, the processor is configured to: if the height of the three-dimensional point is lower than the height of the driving road surface, determine the label of the three-dimensional point as a first label, and the first label is used to represent the The three-dimensional point is referred to as the reflection point.

In some embodiments, the processor is configured to: for each 3D point in the 3D point cloud, determine a first confidence level that the 3D point is a point on the driving road based on the height of the 3D point; A second confidence level of the three-dimensional point as a point on the driving road is determined based on the point cloud segmentation result; the three-dimensional point is labeled based on the first confidence level and the second confidence level of the three-dimensional point.

In some embodiments, the processor is configured to: if at least one of the first confidence level and the second confidence level of the 3D point is greater than a preset confidence level, determine that the label of the 3D point is the first A label, the first label is used to indicate that the three-dimensional point is a point on the driving road surface.

In some embodiments, the three-dimensional point cloud is acquired based on a vision sensor or lidar installed on the movable platform; and/or the point cloud segmentation result is used for the planning unit on the movable platform to pair The traveling state of the movable platform is planned.

For specific embodiments of the method executed by the processor in the point cloud segmentation apparatus according to the embodiment of the present disclosure, reference may be made to the foregoing method embodiments, which will not be repeated here.

FIG. 7 shows a schematic diagram of a more specific hardware structure of a data processing apparatus provided by an embodiment of this specification. The apparatus may include: a processor 701 , a memory 702 , an input/output interface 703 , a communication interface 704 and a bus 705 . The processor 701 , the memory 702 , the input/output interface 703 and the communication interface 704 realize the communication connection among each other within the device through the bus 705 .

The processor 701 can be implemented by a general-purpose CPU (Central Processing Unit, central processing unit), a microprocessor, an application specific integrated circuit (Application Specific Integrated Circuit, ASIC), or one or more integrated circuits, etc. program to implement the technical solutions provided by the embodiments of this specification.

The memory 702 can be implemented in the form of a ROM (Read Only Memory, read-only memory), a RAM (Random Access Memory, random access memory), a static storage device, a dynamic storage device, and the like. The memory 702 may store an operating system and other application programs. When implementing the technical solutions provided by the embodiments of this specification through software or firmware, the relevant program codes are stored in the memory 702 and invoked by the processor 701 for execution.

The input/output interface 703 is used to connect the input/output module to realize the input and output of information. The input/output/module can be configured in the device as a component (not shown in the figure), or can be externally connected to the device to provide corresponding functions. The input device may include a keyboard, a mouse, a touch screen, a microphone, various sensors, etc., and the output device may include a display, a speaker, a vibrator, an indicator light, and the like.

The communication interface 704 is used to connect a communication module (not shown in the figure), so as to realize the communication interaction between the device and other devices. The communication module may implement communication through wired means (eg, USB, network cable, etc.), or may implement communication through wireless means (eg, mobile network, WIFI, Bluetooth, etc.).

Bus 705 includes a path to transfer information between the various components of the device (eg, processor 701, memory 702, input/output interface 703, and communication interface 704).

It should be noted that although the above-mentioned device only shows the processor 701, the memory 702, the input/output interface 703, the communication interface 704 and the bus 705, in the specific implementation process, the device may also include necessary components for normal operation. other components. In addition, those skilled in the art can understand that, the above-mentioned device may only include components necessary to implement the solutions of the embodiments of the present specification, rather than all the components shown in the figures.

As shown in FIG. 8 , an embodiment of the present disclosure further provides a movable platform 800 , which includes a housing 801 ; a point cloud collecting device 802 , which is arranged on the housing 801 and is used to collect a three-dimensional point cloud; and a three-dimensional point cloud. The dividing device 803 is arranged in the casing 801 and is used for executing the method described in any embodiment of the present disclosure. Wherein, the movable platform 800 may be an unmanned aerial vehicle, an unmanned vehicle, an unmanned ship, a mobile robot, etc., and the point cloud collection device 802 may be a visual sensor (eg, a binocular vision sensor, a trinocular vision sensor, etc.) etc.) or lidar.

Embodiments of the present disclosure further provide a computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, implements the steps executed by the second processing unit in the method described in any of the foregoing embodiments.

Computer-readable media includes both persistent and non-permanent, removable and non-removable media, and storage of information may be implemented by any method or technology. Information may be computer readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read only memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), Flash Memory or other memory technology, Compact Disc Read Only Memory (CD-ROM), Digital Versatile Disc (DVD) or other optical storage, Magnetic tape cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission medium that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory computer-readable media, such as modulated data signals and carrier waves.

From the description of the above embodiments, those skilled in the art can clearly understand that the embodiments of the present specification can be implemented by means of software plus a necessary general hardware platform. Based on this understanding, the technical solutions of the embodiments of this specification or the parts that make contributions to the prior art may be embodied in the form of software products, and the computer software products may be stored in storage media, such as ROM/RAM, A magnetic disk, an optical disk, etc., includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the methods described in various embodiments or some parts of the embodiments in this specification.

The systems, devices, modules or units described in the above embodiments may be specifically implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer, which may be in the form of a personal computer, laptop computer, cellular phone, camera phone, smart phone, personal digital assistant, media player, navigation device, email sending and receiving device, game control desktop, tablet, wearable device, or a combination of any of these devices.

Various technical features in the above embodiments can be combined arbitrarily, as long as there is no conflict or contradiction between the combinations of features, but due to space limitations, they are not described one by one, so the various technical features in the above embodiments can be combined arbitrarily It is also within the scope of this disclosure.

Other embodiments of the present disclosure will readily occur to those skilled in the art upon consideration of the disclosure and practice of the specification disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common general knowledge or techniques in the technical field not disclosed by this disclosure . The specification and examples are to be regarded as exemplary only, with the true scope and spirit of the disclosure being indicated by the following claims.

It is to be understood that the present disclosure is not limited to the precise structures described above and illustrated in the accompanying drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

The above descriptions are only preferred embodiments of the present disclosure, and are not intended to limit the present disclosure. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present disclosure shall be included in the present disclosure. within the scope of protection.

Claims

A three-dimensional point cloud segmentation method, characterized in that it is used for point cloud segmentation on a three-dimensional point cloud collected by a movable platform, the method comprising:

acquiring multiple candidate points in the three-dimensional point cloud;

searching the multiple candidate points on the v-disparity plane, and determining a target candidate point located on the road surface of the movable platform among the multiple candidate points;

A model of the driving road surface is fitted based on the target candidate points, and a second point cloud segmentation is performed on the three-dimensional point cloud based on the model of the driving road surface on the u-disparity plane to obtain a point cloud segmentation result.
The method according to claim 1, wherein the acquiring a plurality of candidate points in the three-dimensional point cloud comprises:

Determining all 3D points in the 3D point cloud as candidate points; or

Perform semantic segmentation on the 3D point cloud, and obtain multiple candidate points in the 3D point cloud based on the semantic segmentation result; or

Obtain a plurality of candidate points in the 3D point cloud based on the projection density of the 3D point cloud on the u-disparity plane and the first reference projection density of the plane model of the driving road on the u-disparity plane .
The method according to claim 2, wherein the first reference based on the projection density of the three-dimensional point cloud on the u-disparity plane and the plane model of the driving road surface on the u-disparity plane Projection density, to obtain multiple candidate points in the 3D point cloud, including:

In the first pixel grid on the u-disparity plane, if the ratio of the projection density to the first reference projection density is greater than or equal to a first preset ratio, project the three-dimensional point cloud to the The points in the first pixel grid are determined as candidate points, and the first preset ratio is greater than 1.
The method according to claim 2, wherein the first reference projection density of the plane model on the u-disparity plane is proportional to the disparity value of points on the plane model.
The method according to claim 4, wherein the three-dimensional point cloud is collected by a vision sensor on the movable platform; and the first reference projection density is determined based on the following methods:

acquiring the intercept of the first coordinate axis of the plane model in the coordinate system of the vision sensor, where the first coordinate axis is the coordinate axis in the height direction of the movable platform;

The first reference projection density is determined based on the intercept, the baseline length of the vision sensor, and disparity values of points on the planar model.
6. The method of claim 5, wherein the determining the first reference projection density based on the intercept, the baseline length of the vision sensor, and the disparity value of points on the plane model, comprises: :

calculating the ratio of the intercept to the baseline length of the vision sensor;

The product of the ratio and the disparity value of the point on the plane model is determined as the first reference projection density.
The method according to claim 1, wherein the three-dimensional point cloud includes valid points and invalid points; and the acquiring a plurality of candidate points in the three-dimensional point cloud comprises:

A plurality of candidate points are obtained from valid points in the three-dimensional point cloud.
The method according to claim 1, wherein the acquiring a plurality of candidate points in the 3D point cloud comprises:

filtering out outliers from the three-dimensional points;

Obtain multiple candidate points in the filtered three-dimensional point cloud.
The method according to claim 2, wherein, based on the first projection density of the three-dimensional point cloud on the u-disparity plane and the first projection density of the plane model of the driving road on the u-disparity plane The reference projection density, before acquiring a plurality of candidate points in the three-dimensional point cloud, the method further includes:

Get the preset scaling parameters;

Based on the scale transformation parameter, scale transformation is performed on the u-coordinate value of each 3D point in the 3D point cloud;

The scaled three-dimensional point cloud is projected onto the u-disparity plane.
The method according to claim 9, wherein a scale transformation parameter of a three-dimensional point corresponds to a u-coordinate value of the three-dimensional point.
The method according to claim 10, wherein if the u-coordinate value of the first three-dimensional point in the three-dimensional point cloud is smaller than the first preset coordinate value, the scaling parameter of the first three-dimensional point is greater than 1; and / or

If the u-coordinate value of the first three-dimensional point in the three-dimensional point cloud is greater than or equal to the second preset coordinate value, the scale transformation parameter of the first three-dimensional point is less than 1.
The method according to claim 1, wherein the searching for the plurality of candidate points on the v-disparity plane comprises:

For each candidate point in the plurality of candidate points, determine the search cost of the candidate point;

A target candidate point is determined from the candidate points based on the search cost of the candidate points.
The method according to claim 12, wherein the determining the target candidate point from the candidate points based on the search cost of the candidate point comprises:

If the search cost of the candidate point is less than the preset cost, the candidate point is determined as the target candidate point.
The method according to claim 12, wherein the search cost includes a first search cost and a second search cost; wherein the first search cost is used to represent whether a target candidate is observed on the candidate point point, the second search cost is used to characterize whether the candidate point and the candidate point's neighborhood target candidate point are smooth.
The method according to claim 1, wherein the fitting the model of the driving road surface based on the target candidate points comprises:

Polynomial fitting is performed on the target candidate points based on the least squares method to obtain a polynomial model of the driving road surface.
The method according to claim 1, wherein the second point cloud segmentation of the three-dimensional point cloud based on the model of the driving road surface comprises:

obtaining the slope of the model of the driving road surface;

determining a second reference projected density of the driving road surface on the u-disparity plane based on the slope;

A second point cloud segmentation is performed on the three-dimensional point cloud based on the second reference projection density.
The method according to claim 16, wherein the performing the second point cloud segmentation on the three-dimensional point cloud based on the second reference projection density comprises:

In the second pixel grid on the u-disparity plane, if the ratio of the projection density of the 3D point cloud on the u-disparity plane to the second reference projection density is greater than or equal to a second preset ratio, The points projected into the second pixel grid in the three-dimensional point cloud are divided into target points on the driving road surface, and the second preset ratio is greater than or equal to 1.
The method according to claim 16, wherein the determining the second reference projection density of the driving road surface on the u-disparity plane based on the slope comprises:

determining the depth of the travel surface;

The second reference projected density is determined based on the model of the travel surface, the slope of the model, and the depth of the travel surface.
The method according to claim 18, wherein the three-dimensional point cloud is collected by a visual sensor on the movable platform; the model based on the driving road surface, the slope of the model and the driving The depth of the road surface, and the second reference projection density is determined, including:

calculating the product of the slope of the model and the depth of the road surface;

calculating the difference between the model of the driving surface and the product;

The second reference projected density is determined based on the ratio of the difference to the baseline length of the vision sensor.
The method according to claim 1, wherein the method further comprises:

Based on the point cloud segmentation result, each 3D point in the 3D point cloud is labeled, and one 3D point label is used to characterize the category of the 3D point.
The method according to claim 20, wherein, labeling each 3D point in the 3D point cloud based on the point cloud segmentation result, comprising:

Labeling each 3D point in the 3D point cloud based on the point cloud segmentation result and the height of each 3D point in the 3D point cloud.
The method according to claim 21, wherein, labeling each three-dimensional point in the three-dimensional point cloud based on the point cloud segmentation result and the height of each three-dimensional point in the three-dimensional point cloud, include:

If the height of the three-dimensional point is lower than the height of the driving road surface, the label of the three-dimensional point is determined as a first label, and the first label is used to indicate that the three-dimensional point is a reflection point.
The method according to claim 21, wherein, labeling each three-dimensional point in the three-dimensional point cloud based on the point cloud segmentation result and the height of each three-dimensional point in the three-dimensional point cloud, include:

For each three-dimensional point in the three-dimensional point cloud, determining a first confidence level that the three-dimensional point is a point on the driving road based on the height of the three-dimensional point;

determining a second confidence level that the three-dimensional point is a point on the driving road surface based on the point cloud segmentation result;

The three-dimensional point is tagged based on the first confidence level and the second confidence level of the three-dimensional point.
The method according to claim 23, wherein the labeling of the three-dimensional point based on the first confidence level and the second confidence level of the three-dimensional point comprises:

If at least one of the first confidence level and the second confidence level of the 3D point is greater than a preset confidence level, determine the label of the 3D point as a first label, and the first label is used to indicate the A three-dimensional point is a point on the driving road surface.
The method according to claim 1, wherein the three-dimensional point cloud is acquired based on a vision sensor or lidar installed on the movable platform; and/or

The point cloud segmentation result is used by the planning unit on the movable platform to plan the driving state of the movable platform.
A three-dimensional point cloud segmentation device, comprising a processor, characterized in that, the three-dimensional point cloud segmentation device is used to perform point cloud segmentation on a three-dimensional point cloud collected by a movable platform, and the processor is configured to perform the following steps:

acquiring multiple candidate points in the three-dimensional point cloud;

searching the multiple candidate points on the v-disparity plane, and determining a target candidate point located on the road surface of the movable platform among the multiple candidate points;

A model of the driving road surface is fitted based on the target candidate points, and a second point cloud segmentation is performed on the three-dimensional point cloud based on the model of the driving road surface on the u-disparity plane.
The apparatus of claim 26, wherein the processor is configured to:

Determining all 3D points in the 3D point cloud as candidate points; or

Perform semantic segmentation on the 3D point cloud, and obtain multiple candidate points in the 3D point cloud based on the semantic segmentation result; or

Obtain a plurality of candidate points in the 3D point cloud based on the projection density of the 3D point cloud on the u-disparity plane and the first reference projection density of the plane model of the driving road on the u-disparity plane .
The apparatus of claim 27, wherein the processor is configured to:

In the first pixel grid on the u-disparity plane, if the ratio of the projection density to the first reference projection density is greater than or equal to a first preset ratio, project the three-dimensional point cloud to the The points in the first pixel grid are determined as candidate points, and the first preset ratio is greater than 1.
The device according to claim 27, wherein the first reference projection density of the plane model on the u-disparity plane is proportional to the disparity value of points on the plane model.
The device according to claim 29, wherein the three-dimensional point cloud is acquired by a vision sensor on the movable platform; and the processor is used for:

acquiring the intercept of the first coordinate axis of the plane model in the coordinate system of the vision sensor, where the first coordinate axis is the coordinate axis in the height direction of the movable platform;

The first reference projection density is determined based on the intercept, a baseline length of the vision sensor, and disparity values of points on the planar model.
The apparatus of claim 30, wherein the processor is configured to:

calculating the ratio of the intercept to the baseline length of the vision sensor;

The product of the ratio and the disparity value of the point on the plane model is determined as the first reference projection density.
The device according to claim 26, wherein the three-dimensional point cloud includes valid points and invalid points; the processor is configured to:

A plurality of candidate points are obtained from valid points in the three-dimensional point cloud.
The apparatus of claim 26, wherein the processor is configured to:

filtering out outliers from the three-dimensional points;

Obtain multiple candidate points in the filtered three-dimensional point cloud.
The apparatus of claim 27, wherein the processor is further configured to:

Get the preset scaling parameters;

Based on the scale transformation parameter, scale transformation is performed on the u-coordinate value of each 3D point in the 3D point cloud;

The scaled three-dimensional point cloud is projected onto the u-disparity plane.
The apparatus according to claim 34, wherein a scale transformation parameter of a three-dimensional point corresponds to a u-coordinate value of the three-dimensional point.
The device according to claim 35, wherein if the u-coordinate value of the first three-dimensional point in the three-dimensional point cloud is smaller than the first preset coordinate value, the scaling parameter of the first three-dimensional point is greater than 1; and / or

If the u-coordinate value of the first three-dimensional point in the three-dimensional point cloud is greater than the second preset coordinate value, the scale transformation parameter of the first three-dimensional point is less than 1.
The apparatus of claim 26, wherein the processor is configured to:

For each candidate point in the plurality of candidate points, determine the search cost of the candidate point;

A target candidate point is determined from the candidate points based on the search cost of the candidate points.
The apparatus of claim 37, wherein the processor is configured to:

If the search cost of the candidate point is less than the preset cost, the candidate point is determined as the target candidate point.
The apparatus according to claim 37, wherein the search cost includes a first search cost and a second search cost; wherein the first search cost is used to represent whether a target candidate is observed at the candidate point point, and the second search cost is used to characterize whether the candidate point and the candidate point's neighborhood target candidate point are smooth.
The apparatus of claim 26, wherein the processor is configured to:

Polynomial fitting is performed on the target candidate points based on the least squares method to obtain a polynomial model of the driving road surface.
The apparatus of claim 26, wherein the processor is configured to:

obtaining the slope of the model of the driving road surface;

determining a second reference projected density of the driving road surface on the u-disparity plane based on the slope;

A second point cloud segmentation is performed on the three-dimensional point cloud based on the second reference projection density.
The apparatus of claim 41, wherein the processor is configured to:

In the second pixel grid on the u-disparity plane, if the ratio of the projection density of the 3D point cloud on the u-disparity plane to the second reference projection density is greater than or equal to a second preset ratio, The points projected into the second pixel grid in the three-dimensional point cloud are divided into target points on the driving road surface, and the second preset ratio is greater than or equal to 1.
The apparatus of claim 41, wherein the processor is configured to:

determining the depth of the travel surface;

The second reference projected density is determined based on the model of the travel surface, the slope of the model, and the depth of the travel surface.
The device according to claim 43, wherein the three-dimensional point cloud is acquired by a vision sensor on the movable platform; the processor is used for:

calculating the product of the slope of the model and the depth of the road surface;

calculating the difference between the model of the driving surface and the product;

The second reference projected density is determined based on the ratio of the difference to the baseline length of the vision sensor.
The apparatus of claim 26, wherein the processor is further configured to:

Based on the point cloud segmentation result, each 3D point in the 3D point cloud is labeled, and one 3D point label is used to represent the category of the 3D point.
The apparatus of claim 45, wherein the processor is configured to:

Labeling each 3D point in the 3D point cloud based on the point cloud segmentation result and the height of each 3D point in the 3D point cloud.
The apparatus of claim 46, wherein the processor is configured to:

If the height of the three-dimensional point is lower than the height of the driving road, the label of the three-dimensional point is determined as a first label, and the first label is used to indicate that the three-dimensional point is a reflection point.
The apparatus of claim 46, wherein the processor is configured to:

For each three-dimensional point in the three-dimensional point cloud, determining a first confidence level that the three-dimensional point is a point on the driving road based on the height of the three-dimensional point;

determining a second confidence level that the three-dimensional point is a point on the driving road surface based on the point cloud segmentation result;

The three-dimensional point is tagged based on the first confidence level and the second confidence level of the three-dimensional point.
The apparatus of claim 48, wherein the processor is configured to:

If at least one of the first confidence level and the second confidence level of the 3D point is greater than a preset confidence level, determine the label of the 3D point as a first label, and the first label is used to indicate the A three-dimensional point is a point on the driving road surface.
The device according to claim 26, wherein the three-dimensional point cloud is acquired based on a vision sensor or lidar installed on the movable platform; and/or

The point cloud segmentation result is used by the planning unit on the movable platform to plan the driving state of the movable platform.
A movable platform, characterized in that, comprising:

case;

a point cloud collecting device, arranged on the casing, for collecting three-dimensional point clouds; and

A three-dimensional point cloud segmentation device, arranged in the casing, is used to execute the method of any one of claims 1 to 25.
A computer-readable storage medium, characterized in that computer instructions are stored thereon, and when the instructions are executed by a processor, the method of any one of claims 1 to 25 is implemented.