WO2022099530A1

WO2022099530A1 - Motion segmentation method and apparatus for point cloud data, computer device and storage medium

Info

Publication number: WO2022099530A1
Application number: PCT/CN2020/128264
Authority: WO
Inventors: 吴伟
Original assignee: 深圳元戎启行科技有限公司
Priority date: 2020-11-12
Filing date: 2020-11-12
Publication date: 2022-05-19
Also published as: CN115066708A

Abstract

A motion segmentation method for point cloud data, comprising: acquiring multiple frames of point cloud data; performing point cloud motion segmentation on the point cloud data to obtain a point cloud motion segmentation result; performing ground filtering processing on each frame of point cloud data to obtain multiple frames of non-ground point cloud data; performing cluster analysis on the multiple frames of non-ground point cloud data to obtain a 3D bounding box corresponding to each frame of non-ground point cloud data; performing occlusion analysis on the 3D bounding box to determine a point cloud motion state corresponding to the 3D bounding box; and when the point cloud motion state is dynamic, updating the point cloud motion segmentation result according to the non-ground point cloud data corresponding to the 3D bounding box to obtain a target point cloud motion segmentation result.

Description

Point cloud data motion segmentation method, device, computer equipment and storage medium

technical field

The present application relates to a method, device, computer equipment and storage medium for motion segmentation of point cloud data.

Background technique

Lidar sensors can provide real-time and accurate 3D scene information, with a large ranging range and high accuracy, and the measured values are not affected by ambient light. The information collected by lidar sensors is usually presented in the form of a point cloud. In the process of unmanned driving, in order to ensure the safety of unmanned vehicles, it is necessary to monitor objects in the surrounding environment in real time, especially to detect and track moving objects such as pedestrians and moving vehicles. Data is segmented to identify moving objects.

In the traditional point cloud motion segmentation method, non-deep learning methods, such as based on grid maps, count the change status of point cloud data in grids corresponding to multiple consecutive frames, so as to identify dynamic points and static points in point cloud data. point. However, the traditional method only considers the individual features of each point, and it is difficult to directly correspond to the point cloud data of consecutive frames, resulting in low accuracy of point cloud motion segmentation.

SUMMARY OF THE INVENTION

According to various embodiments disclosed in the present application, a point cloud data segmentation method, apparatus, computer equipment and storage medium capable of accurate point cloud motion segmentation are provided.

A motion segmentation method for point cloud data, comprising:

Obtain multi-frame point cloud data;

Perform point cloud motion segmentation on the point cloud data to obtain the point cloud motion segmentation result;

Perform ground filtering on each frame of point cloud data to obtain multiple frames of non-ground point cloud data;

Perform cluster analysis on multiple frames of non-ground point cloud data, and obtain the three-dimensional bounding box corresponding to each frame of non-ground point cloud data;

Perform occlusion analysis on the 3D bounding box to determine the motion state of the point cloud corresponding to the 3D bounding box; and

When the motion state of the point cloud is dynamic, the point cloud motion segmentation result is updated according to the non-ground point cloud data corresponding to the 3D bounding box, and the target point cloud motion segmentation result is obtained.

A point cloud data motion segmentation device, comprising:

Communication module for acquiring multi-frame point cloud data;

The point cloud motion segmentation module is used to perform point cloud motion segmentation on the point cloud data, and obtain the point cloud motion segmentation result;

The ground filtering module is used to perform ground filtering processing on each frame of point cloud data to obtain multiple frames of non-ground point cloud data;

The cluster analysis module is used to perform cluster analysis on the multi-frame non-ground point cloud data, and obtain the three-dimensional bounding box corresponding to each frame of the non-ground point cloud data;

an occlusion analysis module for performing occlusion analysis on the 3D bounding box to determine the motion state of the point cloud corresponding to the 3D bounding box; and

The updating module is used to update the point cloud motion segmentation result according to the non-ground point cloud data corresponding to the three-dimensional bounding box when the point cloud motion state is dynamic, and obtain the target point cloud motion segmentation result.

A computer device includes a memory and one or more processors, the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, causes the one or more processors to perform the following steps:

Obtain multi-frame point cloud data;

One or more non-volatile computer-readable storage media storing computer-readable instructions that, when executed by one or more processors, cause the one or more processors to perform the following steps:

Obtain multi-frame point cloud data;

The details of one or more embodiments of the application are set forth in the accompanying drawings and the description below. Other features and advantages of the present application will be apparent from the description, drawings, and claims.

Description of drawings

In order to illustrate the technical solutions in the embodiments of the present application more clearly, the following briefly introduces the drawings required in the embodiments. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can also be obtained from these drawings without any creative effort.

FIG. 1 is an application environment diagram of a method for motion segmentation of point cloud data in one or more embodiments.

FIG. 2 is a schematic flowchart of a method for motion segmentation of point cloud data in one or more embodiments.

3 is a schematic flowchart of steps of performing occlusion analysis on a three-dimensional bounding box to determine a point cloud motion state corresponding to the three-dimensional bounding box in one or more embodiments.

FIG. 4 is a block diagram of an apparatus for motion segmentation of point cloud data in one or more embodiments.

5 is a block diagram of a computer device in one or more embodiments.

Detailed ways

In order to make the technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application, but not to limit the present application.

The point cloud data motion segmentation method provided in this application can be applied to the application environment shown in FIG. 1 . The vehicle-mounted sensor 102 sends the collected multi-frame point cloud data to the computer device 104 . For example, the onboard sensor could be a lidar. The computer device 104 performs point cloud motion segmentation on the point cloud data to obtain a point cloud motion segmentation result. The computer device 104 can also perform ground filtering processing on each frame of point cloud data to obtain multiple frames of non-ground point cloud data, and perform cluster analysis on the multiple frames of non-ground point cloud data to obtain a three-dimensional enclosure corresponding to each frame of non-ground point cloud data. box, so as to perform occlusion analysis on the 3D bounding box, and determine the motion state of the point cloud corresponding to the 3D bounding box. When the motion state of the point cloud is dynamic, the computer device 104 updates the point cloud motion segmentation result according to the non-ground point cloud data corresponding to the three-dimensional bounding box to obtain the target point cloud motion segmentation result.

In one of the embodiments, as shown in FIG. 2 , a method for motion segmentation of point cloud data is provided, and the method is applied to the computer equipment in FIG. 1 as an example to illustrate, including the following steps:

Step 202, acquiring multi-frame point cloud data.

The point cloud data can be the data recorded by the vehicle-mounted sensor in the form of point cloud by scanning the surrounding environment information. For example, the onboard sensor could be a lidar. A single frame of point cloud data refers to the original point cloud collected by the vehicle-mounted sensor rotating one circle horizontally. Multi-frame point cloud data refers to the original point cloud in which multiple consecutive frames exist in time sequence. The original point cloud is distributed on multiple circular scan lines with different vertical angles, and the point cloud data on each circular scan line is composed of a circle of laser points. The point cloud data may specifically include three-dimensional coordinates (x, y, z) of each point, laser reflection intensity (Intensity), color information (RGB), and the like. Three-dimensional coordinates are used to represent the position information of the target object surface in the surrounding environment. For example, the three-dimensional coordinates may be the coordinates of the point in the Cartesian coordinate system, and specifically include the horizontal, vertical, and vertical coordinates of the point in the Cartesian coordinate system. The Cartesian coordinate system is a three-dimensional space coordinate system established with lidar as the origin. The three-dimensional space coordinate system includes a horizontal axis (x axis), a vertical axis (y axis) and a vertical axis (z axis). The three-dimensional space coordinate system established with lidar as the origin satisfies the right-hand rule. The x-axis coordinate in the three-dimensional coordinates represents the longitudinal distance of the scanned target object surface relative to the lidar, the y-axis coordinate represents the lateral offset of the scanned target object surface relative to the lidar, and the z-axis coordinate represents the scanned target. The height of the object's surface relative to the lidar.

When the vehicle is unmanned, the surrounding environment can be scanned by the lidar installed on the vehicle to obtain the corresponding multi-frame point cloud data, and the collected multi-frame point cloud data can be transmitted to the computer equipment.

Step 204: Perform point cloud motion segmentation on the point cloud data to obtain a point cloud motion segmentation result.

Point cloud motion segmentation refers to classifying the motion of points in point cloud data and identifying whether each point is a dynamic point or a static point. After acquiring the multi-frame point cloud data transmitted by the vehicle sensor, the computer equipment can perform point cloud motion segmentation on the point cloud data. The point cloud motion segmentation result includes the motion state corresponding to each point in the point cloud data, including dynamic points and static points.

The way to perform point cloud motion segmentation on point cloud data may be to first divide the data space corresponding to each frame of point cloud data into multiple grids according to the preset grid size to obtain a grid map, so that the computer equipment counts multiple consecutive frames. For the quantitative change value of the point cloud data under each grid, the point corresponding to the grid with the quantitative change value greater than the threshold is determined as a dynamic point, and the point corresponding to the grid with the quantitative change value less than or equal to the threshold value is determined as a static point. . In one embodiment, the computer device may further establish a spatial grid containing the point cloud data, and set each grid to a preset size, such as 0.1 meters, so as to divide the spatial grid into a plurality of grids. When a raster has one or more points in the point cloud data at its corresponding location, the raster can be marked as occupied. When the position corresponding to the grid does not have any point, the grid is marked as unoccupied, thereby realizing the marking of each grid in the spatial grid. By marking the grid, the grid of the target object can be quickly determined, so as to perform point cloud motion segmentation. In one of the embodiments, the smaller the size of the grid, the higher the precision and the higher the resolution, that is, for the same spatial grid, the more grids there are, the higher the resolution of the spatial grid is. The higher the resolution rate of the spatial grid, the higher the accuracy of the raster map.

In one of the embodiments, the point cloud motion segmentation method may be any one of a variety of point cloud motion segmentation methods, such as a point-level occlusion analysis method, a point cloud motion segmentation method based on a deep learning model, and the like.

In one embodiment, performing point cloud motion segmentation on point cloud data, and obtaining a point cloud motion segmentation result includes: calling a pre-trained point cloud motion segmentation model, and inputting the point cloud data into the point cloud motion segmentation model; The cloud motion segmentation model classifies the point cloud data, outputs the corresponding motion state of each point in the point cloud data, and generates the point cloud motion segmentation result.

The point cloud motion segmentation model is trained by a large number of labeled sample data. For example, the point cloud motion segmentation model can be any one of pointnet, pointnet++, etc. The computer equipment can input the point cloud data into the point cloud motion segmentation model, classify the point cloud data through the point cloud motion segmentation model, and determine the motion state of each point in the point cloud data, that is, output each point as a dynamic point. Still a static point. The motion classification of point cloud data can improve the accuracy of point cloud motion segmentation by using a pre-established point cloud motion segmentation model.

In one of the embodiments, the computer device may further preprocess the point cloud data before the point cloud motion segmentation. The way of preprocessing can include downsampling and filtering processing operations. Specifically, computer equipment can use voxel grid filtering to downsample point cloud data, which can remove noise points, reduce the amount of redundant data, improve point cloud processing speed, and maintain details in the depth direction during filtering. Reduce the loss of detail in the depth direction. Therefore, the computer equipment performs through filtering processing on the downsampled point cloud data. Specifically, the depth direction data range of the point cloud data can be set, for example, between 0.6-1.0m, and the point cloud outside this range will be used as interference. Click to remove. Furthermore, the computer equipment adopts the statistical filtering method on the point cloud data after the pass-through filtering, performs a statistical analysis on the neighborhood of each point, and removes the point cloud outside the standard range, thereby reducing the isolated redundant point cloud. Specifically, the computer equipment can use the sparse outlier removal method to calculate the average distance from each point in the point cloud data to all adjacent points, determine the points whose average distance is outside the standard range as outliers, and extract the points from the point cloud data. removed from cloud data. The standard range is defined by the global distance mean and variance. By removing outliers, the accuracy of point cloud motion segmentation can be improved.

Step 206: Perform ground filtering processing on each frame of point cloud data to obtain multiple frames of non-ground point cloud data.

Step 208: Perform cluster analysis on multiple frames of non-ground point cloud data to obtain a three-dimensional bounding box corresponding to each frame of non-ground point cloud data.

Since the traditional point cloud motion segmentation algorithm only considers the individual features of each point and lacks the semantic information of the points, it is difficult to directly correspond to the point cloud data of consecutive frames. Therefore, the computer equipment can also perform point cloud motion on the point cloud data. At the same time of segmentation, 3D object-level occlusion analysis is performed on the point cloud data to assist the traditional point cloud motion segmentation method and improve the accuracy of point cloud motion segmentation.

Before performing occlusion analysis on the point cloud data at the 3D target level, the computer equipment needs to perform ground filtering, cluster analysis and identification of 3D bounding boxes on the point cloud data of each frame. Ground filtering refers to filtering out the ground points in the point cloud data, and the remaining points are non-ground points. The computer equipment can identify the ground points in each frame of point cloud data by performing ground segmentation on each frame of point cloud data, thereby filtering the ground points in each frame of point cloud data to obtain each frame of non-ground point cloud data. Specifically, the computer device may first divide the point cloud area where the point cloud data is located into a plurality of sub-areas. The point cloud area refers to the three-dimensional data space where the point cloud data is located. The division method may be grid division of the point cloud area, that is, division of the horizontal plane formed by the point cloud area in the x-axis direction and the y-axis direction. The grid division method may be equal division or random division. For example, for a vehicle-mounted sensor with a visible range of 100m, and the horizontal plane in the scanning area of the vehicle-mounted sensor is 100m*100m, the point cloud area can be equally divided into 10*10 horizontal grids. For each sub-region obtained by division, the computer device may use the least squares method to estimate the corresponding ground according to the preset plane equation, so as to obtain the corresponding ground of each sub-region. For example, the preset plane equation may be a ternary linear equation. The ground corresponding to each sub-region is embodied in the form of a ternary linear equation. The computer equipment traverses the coordinates of the points in the corresponding sub-regions in the equation corresponding to the ground, calculates the distance between each point and the corresponding ground, and determines the point as a ground point when the distance is less than a threshold. When the distance is greater than or equal to the threshold, the point is determined as a non-ground point. Threshold refers to the distance threshold used to judge whether the point is a ground point. The computer equipment further filters the ground points to obtain non-ground points in each frame of point cloud data, thereby obtaining each frame of non-ground point cloud data.

In one of the embodiments, the ground segmentation method may be any one of a depth image and geometric relationship-based method, a normal vector method, an absolute height method, an average height method, and the like.

The computer equipment performs cluster analysis on multiple frames of non-ground point cloud data, and obtains a three-dimensional bounding box corresponding to each target object in each frame of non-ground point cloud data. Cluster analysis refers to clustering the points corresponding to the same target object together, and identifying the three-dimensional bounding box corresponding to each target object. Cluster analysis can include two steps of clustering and identifying three-dimensional bounding boxes. For example, the clustering method may be any one of the clustering algorithms such as the connected domain clustering method and the DBSCAN (Density-Based Spatial Clustering of Applications with Noise, density clustering) algorithm. The three-dimensional bounding box corresponding to the identified target object may be any one of L-shape fitting method, principal component analysis method, and the like. The three-dimensional bounding box may include the center point coordinates, size, orientation, etc. of the target object. By identifying the three-dimensional bounding box corresponding to the target object, different target objects can be accurately distinguished, which is beneficial to improve the accuracy of point cloud motion segmentation.

Step 210: Perform occlusion analysis on the three-dimensional bounding box to determine the motion state of the point cloud corresponding to the three-dimensional bounding box.

Occlusion analysis refers to identifying the motion state of point cloud data according to the characteristics of the laser beam. The occlusion characteristic of the laser beam means that when multiple objects are on the same laser scanning zone, the lidar will directly measure the target object closest to the vehicle, but will not measure other target objects on the same laser scanning zone. There is no other target object between it and the measured target object. Among them, the laser scanning zone can be the scanning range of the laser beam emitted by the lidar at a certain time. Therefore, the computer equipment can use the characteristics of the laser beam emitted by the lidar to surround the target object corresponding to multiple frames of non-ground point cloud data. box for occlusion analysis. Specifically, the computer device can perform occlusion analysis on the middle position of the line connecting each 3D bounding box of the current frame and the 3D bounding box of the historical frame to the vehicle. If the bounding box is dynamic, all points in the three-dimensional bounding box are determined as dynamic points.

Step 212 , when the motion state of the point cloud is dynamic, update the point cloud motion segmentation result according to the non-ground point cloud data corresponding to the three-dimensional bounding box to obtain the target point cloud motion segmentation result.

Since the dynamic points obtained by occlusion analysis have high accuracy, the computer equipment can update the point cloud motion segmentation result according to the dynamic points obtained by the occlusion analysis, and can correct the wrong points in the dynamic point recognition in the point cloud motion segmentation results. Get the target point cloud segmentation result.

In this embodiment, the computer equipment performs point cloud motion segmentation on the acquired multi-frame point cloud data to obtain the point cloud motion segmentation result. At the same time, the computer equipment performs ground filtering processing on each frame of point cloud data to obtain effective cloud data. Clustering analysis of multiple frames of non-ground point cloud data can cluster the points belonging to the same target object together, and obtain the three-dimensional bounding box corresponding to each target object. The computer equipment performs occlusion analysis on the three-dimensional bounding box, and determines the points in the dynamic three-dimensional bounding box as dynamic points. Since the 3D bounding box is a real 3D target, the occlusion analysis of the 3D bounding box avoids the situation that the points corresponding to the same target object have different motion states, and can obtain a more accurate point cloud motion state, thereby obtaining a more accurate dynamic point. Further, the computer equipment updates the result of the point cloud motion segmentation according to the dynamic points, thereby improving the accuracy of the point cloud motion segmentation.

In one embodiment, as shown in FIG. 3 , the occlusion analysis is performed on the three-dimensional bounding box, and the step of determining the motion state of the point cloud corresponding to the three-dimensional bounding box includes:

Step 302 , in the three-dimensional bounding box, determine the current three-dimensional bounding box corresponding to the non-ground point cloud data of the current frame and the historical three-dimensional bounding box corresponding to the non-ground point cloud data of the historical frame.

Step 304: Perform occlusion analysis on the current 3D bounding box and the historical 3D bounding box, and identify the motion state of the point cloud corresponding to the current 3D bounding box.

The computer device may determine the current three-dimensional bounding box corresponding to the non-ground point cloud data of the current frame and the historical three-dimensional bounding box corresponding to the non-ground point cloud data of the historical frame in the three-dimensional bounding box obtained by the cluster analysis. The historical frame may be a certain historical frame, or may be multiple historical frames within a preset time period. The current 3D bounding box and the historical 3D bounding box may be one or multiple. Therefore, the computer device performs occlusion analysis on the current three-dimensional bounding box and the historical three-dimensional bounding box. In one embodiment, performing occlusion analysis on the current 3D bounding box and the historical 3D bounding box, and identifying the motion state of the point cloud corresponding to the current 3D bounding box includes: acquiring point cloud scan lines corresponding to each historical 3D bounding box; identifying the current 3D bounding box The positional relationship between the bounding box and the point cloud scan line; identify the motion state of the point cloud corresponding to the current bounding box according to the positional relationship. Specifically, the computer device acquires the point cloud scan line corresponding to the historical three-dimensional bounding box of each frame in the historical frame. The point cloud scan line refers to the connection between the historical 3D bounding box obtained on the laser scan band emitted by the lidar and the vehicle. The point cloud scan line includes the scan angle of the lidar. Therefore, the computer device recognizes the positional relationship between the current 3D bounding box and each point cloud scan line. Since there is no other target object in the connection between the historical 3D bounding box and the vehicle, when the computer device recognizes the current When there is a 3D bounding box located in the middle of the point cloud scan line in the 3D box, the 3D bounding box can be determined as a dynamic target, and the points in the 3D bounding box are all dynamic points. The computer device only needs to determine the three-dimensional bounding box as a dynamic target when detecting that there is a three-dimensional bounding box located in the middle of each point cloud scan line in the current three-dimensional bounding box, and determine the points in the three-dimensional bounding box as dynamic points. It can quickly judge the motion state of the point cloud.

For example, at time t-k, the lidar scans the target object z(n, 1), indicating that there is no other target object in the connection between the vehicle and the target object z(n, 1), while at time t, the lidar measures the target object z(n, 1). The target object z(m, 1) in the middle of the point cloud scan line formed by the vehicle to the target object z(n, 1) at time t-k indicates that the target object z(m, 1) is dynamic relative to time t-k. Therefore, the computer device can determine the points in the three-dimensional bounding box corresponding to the target object z(m, 1) as dynamic points.

In this embodiment, the computer device performs occlusion analysis on the current 3D bounding box and the historical 3D bounding box, and can quickly and accurately determine the motion state of the point cloud by using the characteristics of the laser beam. In addition, since the 3D bounding box is a real 3D target, the occlusion analysis of the 3D bounding box avoids the situation that the points corresponding to the same target object have different motion states, and can obtain more accurate dynamic points.

In one embodiment, the point cloud motion segmentation result is updated according to the non-ground point cloud data corresponding to the 3D bounding box, and obtaining the point cloud data segmentation result includes: extracting dynamic point cloud data corresponding to the 3D bounding box; In the segmentation result, the point cloud motion segmentation result corresponding to each dynamic point in the dynamic point cloud data is determined; when the determined point cloud motion segmentation result is a static point, the determined point cloud motion segmentation result is replaced with a dynamic point.

Since the accuracy of the dynamic points obtained through the occlusion analysis is high, the computer device can update the point cloud motion segmentation result according to the dynamic point cloud data corresponding to the dynamic three-dimensional bounding box. Specifically, the computer device extracts dynamic point cloud data corresponding to the dynamic three-dimensional bounding box, and the dynamic point cloud data includes a plurality of dynamic points. The computer device thus determines the point cloud motion segmentation result corresponding to each dynamic point in the point cloud motion segmentation result. When the determined point cloud motion segmentation result is a static point, the computer equipment updates the point cloud motion segmentation result to a dynamic point, thereby improving the recognition accuracy of the dynamic point in the point cloud motion segmentation result, thereby improving the detection of dynamic targets. accuracy.

In one embodiment, performing ground filtering processing on each frame of point cloud data to obtain multiple frames of non-ground point cloud data includes: dividing the point cloud area corresponding to each frame of point cloud data into multiple grids; The equation calculates the ground corresponding to the point cloud data in each grid; calculates the distance value between each point in the point cloud data of each grid and the corresponding ground; filters the points whose distance value is less than the first threshold to obtain multiple frames Non-ground point cloud data.

The point cloud area refers to the data space where the point cloud data of each frame is located. The computer equipment may divide the point cloud area corresponding to each frame of point cloud data into grids to obtain multiple grids. Grid division refers to dividing the point cloud area in the x-axis direction and the y-axis direction, that is, dividing the point cloud area on the xy level.

Specifically, the computer device may divide the data area corresponding to the extracted point cloud data in the x-axis direction and the y-axis direction according to preset parameters. The preset parameter may be a parameter for grid division of the data region where the extracted point cloud data is located. For example, the preset parameter may be length*width, indicating the length and width of each grid obtained after grid division. The length and width can be the same or different. The preset parameter may also be equal division. The height of multiple grids is the same. When dividing the point cloud region corresponding to each frame of point cloud data into a grid, the order of division in the x-axis direction and the y-axis direction is not limited. For example, the computer device may first divide the point cloud area corresponding to the extracted point cloud data in the x-axis direction according to preset parameters, and then divide the point cloud area corresponding to the extracted point cloud data in the y-axis direction according to the preset parameters. . The point cloud area corresponding to the extracted point cloud data may also be firstly divided in the y-axis direction according to preset parameters, and then the point cloud area corresponding to the extracted point cloud data may be divided in the x-axis direction according to the preset parameters.

The computer device obtains the preset plane equation, and calculates the ground corresponding to the point cloud data in each grid according to the preset plane equation and the least square method, so as to obtain the ground corresponding to each grid. The ground corresponding to each grid is embodied in the form of a ternary linear equation. The computer equipment traverses and inputs the point coordinates of each point in the corresponding grid in the equation corresponding to the ground, and calculates the distance value between each point and the corresponding ground. A first threshold for judging the type of the point is pre-stored in the computer device. Classes of points can include ground points as well as non-ground points. The computer device compares the distance value with the first threshold value, and when the distance value is smaller than the first threshold value, it indicates that the point is a ground point. When the distance value is greater than or equal to the first threshold, it indicates that the point is a non-ground point. The computer equipment filters the points whose distance value is less than the first threshold, and the remaining points constitute non-ground point cloud data.

In this embodiment, the point cloud area corresponding to each frame of point cloud data is divided into multiple grids, so as to calculate the ground corresponding to the point cloud data in each grid, and by calculating the distance between each point and the corresponding ground, Thereby filtering the ground points. Since the grid division only needs to divide the data area corresponding to the point cloud data in the x-axis direction and the y-axis direction, it can be quickly performed in the unmanned mode, when the computing resources of the computer equipment are limited and the real-time requirements are high. Ground filtration treatment.

In one embodiment, calculating the ground corresponding to the point cloud data in each grid according to the preset plane equation includes: selecting the point with the smallest height value in the point cloud data corresponding to each grid; calculating the point corresponding to each grid The height difference between each point in the cloud data and the point with the smallest height value; extract the points whose height difference is less than the second threshold, and perform plane fitting on the selected points according to the preset plane equation to obtain the points in each grid The ground corresponding to the cloud data.

After the computer equipment divides the point cloud area corresponding to each frame of point cloud data, and obtains multiple grids, it can select the point with the smallest height value in the point cloud data corresponding to each grid, and calculate the corresponding grid. The height difference between each point in the point cloud data and the point with the smallest height value. A second threshold for judging whether it is a plane fitting point is pre-stored in the computer device. When the height difference is less than the second threshold, it indicates that the point corresponding to the height difference is a plane fitting point. The computer equipment compares the height difference with the second threshold, selects points whose height difference is less than the second threshold, and performs plane fitting on the selected points according to the preset plane equation by the computer equipment to obtain the point cloud data in each grid. corresponding ground.

In this embodiment, the computer device selects the plane fitting by calculating the height difference between each point in the point cloud data corresponding to each grid and the point with the smallest height value, and comparing the height difference with the second threshold. Because the probability of the point with the smallest height value being the ground point is the largest, by calculating the height difference between each point and the point with the smallest height value, the plane fitting point can be determined more accurately, and the ground estimation efficiency can be improved, and then Improve ground filtration efficiency.

In one embodiment, as shown in FIG. 4 , a motion segmentation device for point cloud data is provided, including: a communication module 402 , a point cloud motion segmentation module 404 , a ground filtering module 406 , a cluster analysis module 408 , and an occlusion analysis module. module 410 and update module 412, where:

The communication module 402 is used for acquiring multi-frame point cloud data.

The point cloud motion segmentation module 404 is configured to perform point cloud motion segmentation on the point cloud data to obtain a point cloud motion segmentation result.

The ground filtering module 406 is configured to perform ground filtering processing on each frame of point cloud data to obtain multiple frames of non-ground point cloud data.

The cluster analysis module 408 is configured to perform cluster analysis on multiple frames of non-ground point cloud data to obtain a three-dimensional bounding box corresponding to each frame of non-ground point cloud data.

The occlusion analysis module 410 is configured to perform occlusion analysis on the three-dimensional bounding box, and determine the motion state of the point cloud corresponding to the three-dimensional bounding box.

The updating module 412 is configured to update the point cloud motion segmentation result according to the non-ground point cloud data corresponding to the three-dimensional bounding box when the motion state of the point cloud is dynamic, so as to obtain the target point cloud motion segmentation result.

In one embodiment, the occlusion analysis module 410 is further configured to determine, in the three-dimensional bounding box, the current three-dimensional bounding box corresponding to the non-ground point cloud data of the current frame and the historical three-dimensional bounding box corresponding to the non-ground point cloud data of the historical frame; The current 3D bounding box and the historical 3D bounding box are subjected to occlusion analysis to identify the motion state of the point cloud corresponding to the current 3D bounding box.

In one embodiment, the occlusion analysis module 410 is further configured to obtain the point cloud scan line corresponding to each historical 3D bounding box; identify the positional relationship between the current 3D bounding box and the point cloud scan line; identify the current bounding box according to the positional relationship The motion state of the point cloud corresponding to the box.

In one embodiment, the update module 412 is further configured to extract the dynamic point cloud data corresponding to the three-dimensional bounding box; determine the point cloud motion segmentation result corresponding to each dynamic point in the dynamic point cloud data in the point cloud motion segmentation result; when When the determined point cloud motion segmentation result is a static point, replace the determined point cloud motion segmentation result with a dynamic point.

In one embodiment, the ground filtering module 406 is further configured to divide the point cloud area corresponding to each frame of point cloud data into multiple grids; calculate the ground corresponding to the point cloud data in each grid according to a preset plane equation ; Calculate the distance value between each point in the point cloud data of each grid and the corresponding ground; filter the points whose distance value is less than the first threshold to obtain multiple frames of non-ground point cloud data.

In one embodiment, the ground filtering module 406 is further configured to select the point with the smallest height value in the point cloud data corresponding to each grid; calculate the point with the smallest height value in the point cloud data corresponding to each grid The height difference between the two points is extracted; the points whose height difference is less than the second threshold are extracted, and the selected points are fitted according to the preset plane equation to obtain the ground corresponding to the point cloud data in each grid.

In one embodiment, the point cloud motion segmentation module 404 is further configured to call a pre-trained point cloud motion segmentation model, and input the point cloud data into the point cloud motion segmentation model; Perform classification, output the motion state corresponding to each point in the point cloud data, and generate a point cloud motion segmentation result.

For the specific limitation of the point cloud data motion segmentation device, please refer to the limitation on the trajectory prediction method above, which will not be repeated here. Each module in the above-mentioned point cloud data motion segmentation device can be implemented in whole or in part by software, hardware and combinations thereof. The above modules can be embedded in or independent of the processor in the computer device in the form of hardware, or stored in the memory in the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

In one of the embodiments, a computer device is provided, the internal structure of which can be shown in FIG. 5 . The computer device includes a processor, memory, a communication interface, and a database connected by a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium, an internal memory. The non-volatile storage medium stores an operating system, computer readable instructions and a database. The internal memory provides an environment for the execution of the operating system and computer-readable instructions in the non-volatile storage medium. The database of the computer device is used to store the point cloud motion segmentation results, the target point cloud motion segmentation results, and the like. The communication interface of the computer device is used to connect and communicate with the vehicle sensor. The computer readable instructions, when executed by a processor, implement a method for motion segmentation of point cloud data.

Those skilled in the art can understand that the structure shown in FIG. 5 is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer equipment to which the solution of the present application is applied. Include more or fewer components than shown in the figures, or combine certain components, or have a different arrangement of components.

A computer device includes a memory and one or more processors, the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the one or more processors, causes the one or more processors to perform the following steps:

Obtain multi-frame point cloud data;

Perform occlusion analysis on the 3D bounding box to determine the motion state of the point cloud corresponding to the 3D bounding box;

In one embodiment, the processor further implements the following steps when executing the computer-readable instructions: determining, in the three-dimensional bounding box, the current three-dimensional bounding box corresponding to the non-ground point cloud data of the current frame and the historical frame corresponding to the non-ground point cloud data of the historical frame 3D bounding box: Perform occlusion analysis on the current 3D bounding box and historical 3D bounding box, and identify the point cloud motion state corresponding to the current 3D bounding box.

In one embodiment, when the processor executes the computer-readable instructions, it further implements the following steps: acquiring point cloud scan lines corresponding to each historical three-dimensional bounding box; identifying the positional relationship between the current three-dimensional bounding box and the point cloud scan line; The positional relationship identifies the motion state of the point cloud corresponding to the current bounding box.

In one embodiment, the processor further implements the following steps when executing the computer-readable instructions: extracting the dynamic point cloud data corresponding to the three-dimensional bounding box; determining the point corresponding to each dynamic point in the dynamic point cloud data in the point cloud motion segmentation result Cloud motion segmentation result; when the determined point cloud motion segmentation result is a static point, replace the determined point cloud motion segmentation result with a dynamic point.

In one embodiment, the processor further implements the following steps when executing the computer-readable instructions: dividing the point cloud area corresponding to each frame of point cloud data into a plurality of grids; calculating the points in each grid according to a preset plane equation The ground corresponding to the cloud data; calculate the distance value between each point in the point cloud data of each grid and the corresponding ground; filter the points whose distance value is less than the first threshold to obtain multiple frames of non-ground point cloud data.

In one embodiment, the processor further implements the following steps when executing the computer-readable instructions: selecting the point with the smallest height value in the point cloud data corresponding to each grid; calculating the difference between each point in the point cloud data corresponding to each grid and the The height difference between the points with the smallest height value; extract the points whose height difference is less than the second threshold, and perform plane fitting on the selected points according to the preset plane equation to obtain the ground corresponding to the point cloud data in each grid.

In one embodiment, the processor further implements the following steps when executing the computer-readable instructions: calling a pre-trained point cloud motion segmentation model, inputting point cloud data into the point cloud motion segmentation model; The point cloud data is classified, and the motion state corresponding to each point in the point cloud data is output to generate the point cloud motion segmentation result.

Obtain multi-frame point cloud data;

In one embodiment, when the computer-readable instructions are executed by the processor, the following steps are further implemented: determining, in the three-dimensional bounding box, the current three-dimensional bounding box corresponding to the non-ground point cloud data of the current frame and the non-ground point cloud data of the historical frame corresponding to the current frame. Historical 3D bounding box: Perform occlusion analysis on the current 3D bounding box and historical 3D bounding box, and identify the point cloud motion state corresponding to the current 3D bounding box.

In one embodiment, the computer-readable instructions further implement the following steps when executed by the processor: acquiring point cloud scan lines corresponding to each historical 3D bounding box; identifying the positional relationship between the current 3D bounding box and the point cloud scan lines; Identify the motion state of the point cloud corresponding to the current bounding box according to the positional relationship.

In one embodiment, when the computer-readable instructions are executed by the processor, the following steps are further implemented: extracting dynamic point cloud data corresponding to a three-dimensional bounding box; Point cloud motion segmentation result; when the determined point cloud motion segmentation result is a static point, replace the determined point cloud motion segmentation result with a dynamic point.

In one embodiment, when the computer-readable instructions are executed by the processor, the following steps are further implemented: dividing the point cloud area corresponding to each frame of point cloud data into a plurality of grids; The ground corresponding to the point cloud data; calculate the distance value between each point in the point cloud data of each grid and the corresponding ground; filter the points whose distance value is less than the first threshold to obtain multiple frames of non-ground point cloud data.

In one embodiment, when the computer-readable instructions are executed by the processor, the following steps are further implemented: selecting the point with the smallest height value in the point cloud data corresponding to each grid; calculating each point in the point cloud data corresponding to each grid The height difference between the point with the smallest height value; extract the points whose height difference is less than the second threshold, and perform plane fitting on the selected points according to the preset plane equation to obtain the ground corresponding to the point cloud data in each grid .

In one embodiment, when the computer-readable instructions are executed by the processor, the following steps are further implemented: calling a pre-trained point cloud motion segmentation model, inputting point cloud data into the point cloud motion segmentation model; Classify the point cloud data, output the motion state corresponding to each point in the point cloud data, and generate the point cloud motion segmentation result.

Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be implemented by instructing relevant hardware through computer-readable instructions, and the computer-readable instructions can be stored in a non-volatile computer. In the readable storage medium, the computer-readable instructions, when executed, may include the processes of the foregoing method embodiments. Wherein, any reference to memory, storage, database or other medium used in the various embodiments provided in this application may include non-volatile and/or volatile memory. Nonvolatile memory may include read only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous chain Road (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

The technical features of the above embodiments can be combined arbitrarily. In order to make the description simple, all possible combinations of the technical features in the above embodiments are not described. However, as long as there is no contradiction in the combination of these technical features It is considered to be the range described in this specification.

The above-mentioned embodiments only represent several embodiments of the present application, and the descriptions thereof are relatively specific and detailed, but should not be construed as a limitation on the scope of the invention patent. It should be pointed out that for those skilled in the art, without departing from the concept of the present application, several modifications and improvements can be made, which all belong to the protection scope of the present application. Therefore, the scope of protection of the patent of the present application shall be subject to the appended claims.

Claims

A motion segmentation method for point cloud data, comprising:

Obtain multi-frame point cloud data;

Perform point cloud motion segmentation on the point cloud data to obtain a point cloud motion segmentation result;

Perform ground filtering on each frame of point cloud data to obtain multiple frames of non-ground point cloud data;

Perform cluster analysis on the multi-frame non-ground point cloud data to obtain a three-dimensional bounding box corresponding to each frame of non-ground point cloud data;

performing occlusion analysis on the 3D bounding box to determine the motion state of the point cloud corresponding to the 3D bounding box; and

When the motion state of the point cloud is dynamic, the point cloud motion segmentation result is updated according to the non-ground point cloud data corresponding to the three-dimensional bounding box to obtain the target point cloud motion segmentation result.
The method according to claim 1, wherein the performing occlusion analysis on the three-dimensional bounding box and determining the motion state of the point cloud corresponding to the three-dimensional bounding box comprises:

In the 3D bounding box, determine the current 3D bounding box corresponding to the non-ground point cloud data of the current frame and the historical 3D bounding box corresponding to the non-ground point cloud data of the historical frame; and

Perform occlusion analysis on the current 3D bounding box and the historical 3D bounding box, and identify the point cloud motion state corresponding to the current 3D bounding box.
The method according to claim 2, wherein performing occlusion analysis on the current 3D bounding box and the historical 3D bounding box, and identifying the point cloud motion state corresponding to the current 3D bounding box comprises:

Obtain the point cloud scan lines corresponding to each historical 3D bounding box;

identifying the positional relationship between the current 3D bounding box and the point cloud scan line; and

Identify the motion state of the point cloud corresponding to the current bounding box according to the positional relationship.
The method according to claim 1, wherein the updating the point cloud motion segmentation result according to the non-ground point cloud data corresponding to the three-dimensional bounding box, and obtaining the point cloud data segmentation result comprises:

extracting dynamic point cloud data corresponding to the three-dimensional bounding box;

determining, in the point cloud motion segmentation result, a point cloud motion segmentation result corresponding to each dynamic point in the dynamic point cloud data; and

When the determined point cloud motion segmentation result is a static point, the determined point cloud motion segmentation result is replaced with a dynamic point.
The method according to claim 1, wherein the performing ground filtering processing on each frame of point cloud data to obtain multiple frames of non-ground point cloud data comprises:

Divide the point cloud area corresponding to each frame of point cloud data into multiple grids;

Calculate the ground corresponding to the point cloud data in each grid according to the preset plane equation;

Calculate the distance value between each point in the point cloud data of each grid and the corresponding ground; and

Filter the points whose distance value is less than the first threshold to obtain multi-frame non-ground point cloud data.
The method according to claim 5, wherein the calculating the ground corresponding to the point cloud data in each grid according to the preset plane equation comprises:

Select the point with the smallest height value in the point cloud data corresponding to each grid;

calculating the height difference between each point in the point cloud data corresponding to each grid and the point with the smallest height value; and

Extracting points whose height difference is less than the second threshold value, and performing plane fitting on the selected points according to a preset plane equation, to obtain the ground corresponding to the point cloud data in each grid.
The method according to any one of claims 1 to 6, wherein the performing point cloud motion segmentation on the point cloud data to obtain a point cloud motion segmentation result comprises:

calling a pre-trained point cloud motion segmentation model, and inputting the point cloud data into the point cloud motion segmentation model; and

The point cloud data is classified by the point cloud motion segmentation model, and the motion state corresponding to each point in the point cloud data is output to generate a point cloud motion segmentation result.
A point cloud data motion segmentation device, comprising:

Communication module for acquiring multi-frame point cloud data;

a point cloud motion segmentation module, which is used to perform point cloud motion segmentation on the point cloud data to obtain a point cloud motion segmentation result;

The ground filtering module is used to perform ground filtering processing on each frame of point cloud data to obtain multiple frames of non-ground point cloud data;

a cluster analysis module, configured to perform cluster analysis on the multiple frames of non-ground point cloud data, and obtain a three-dimensional bounding box corresponding to each frame of non-ground point cloud data;

an occlusion analysis module, configured to perform occlusion analysis on the three-dimensional bounding box, and determine the motion state of the point cloud corresponding to the three-dimensional bounding box; and

An update module, configured to update the point cloud motion segmentation result according to the non-ground point cloud data corresponding to the three-dimensional bounding box when the point cloud motion state is dynamic, to obtain the target point cloud motion segmentation result.
The device according to claim 8, wherein the occlusion analysis module is further configured to determine the current 3D bounding box corresponding to the non-ground point cloud data of the current frame and the non-ground point cloud data of the historical frame in the 3D bounding box corresponding historical 3D bounding box; and performing occlusion analysis on the current 3D bounding box and the historical 3D bounding box to identify the point cloud motion state corresponding to the current 3D bounding box.
The device according to claim 9, wherein the occlusion analysis module is further configured to obtain point cloud scan lines corresponding to each historical 3D bounding box; identify the distance between the current 3D bounding box and the point cloud scan line and identify the motion state of the point cloud corresponding to the current bounding box according to the positional relationship.
A computer device comprising a memory and one or more processors, the memory having computer-readable instructions stored in the memory that, when executed by the one or more processors, cause the one or more processors to Each processor performs the following steps:

Obtain multi-frame point cloud data;

Perform point cloud motion segmentation on the point cloud data to obtain a point cloud motion segmentation result;

Perform ground filtering on each frame of point cloud data to obtain multiple frames of non-ground point cloud data;

Perform cluster analysis on the multi-frame non-ground point cloud data to obtain a three-dimensional bounding box corresponding to each frame of non-ground point cloud data;

performing occlusion analysis on the 3D bounding box to determine the motion state of the point cloud corresponding to the 3D bounding box; and

When the motion state of the point cloud is dynamic, the point cloud motion segmentation result is updated according to the non-ground point cloud data corresponding to the three-dimensional bounding box to obtain the target point cloud motion segmentation result.
The computer device according to claim 11, wherein, when the processor executes the computer-readable instructions, the processor further executes the following step: determining, in the three-dimensional bounding box, a current three-dimensional image corresponding to the non-ground point cloud data of the current frame The bounding box and the historical 3D bounding box corresponding to the non-ground point cloud data of the historical frame; and performing occlusion analysis on the current 3D bounding box and the historical 3D bounding box to identify the point cloud motion state corresponding to the current 3D bounding box.
The computer device according to claim 12, wherein the processor further executes the following steps when executing the computer-readable instructions: acquiring point cloud scan lines corresponding to each historical three-dimensional bounding box; identifying the current three-dimensional bounding box The positional relationship between the frame and the point cloud scan line; and the point cloud motion state corresponding to the current bounding frame is identified according to the positional relationship.
The computer device according to claim 11, wherein when the processor executes the computer-readable instructions, the processor further executes the following steps: extracting dynamic point cloud data corresponding to the three-dimensional bounding box; In the segmentation result, determine the point cloud motion segmentation result corresponding to each dynamic point in the dynamic point cloud data; and when the determined point cloud motion segmentation result is a static point, replace the determined point cloud motion segmentation result with dynamic point.
The computer device according to claim 11, wherein the processor further executes the following steps when executing the computer-readable instructions: dividing the point cloud area corresponding to each frame of point cloud data into a plurality of grids; The preset plane equation calculates the ground corresponding to the point cloud data in each grid; calculates the distance value between each point in the point cloud data of each grid and the corresponding ground; and filters the points whose distance value is less than the first threshold , to obtain multi-frame non-ground point cloud data.
One or more non-volatile computer-readable storage media storing computer-readable instructions that, when executed by one or more processors, cause the one or more processors to perform the following steps:

Obtain multi-frame point cloud data;

Perform point cloud motion segmentation on the point cloud data to obtain a point cloud motion segmentation result;

Perform ground filtering on each frame of point cloud data to obtain multiple frames of non-ground point cloud data;

Perform cluster analysis on the multi-frame non-ground point cloud data to obtain a three-dimensional bounding box corresponding to each frame of non-ground point cloud data;

performing occlusion analysis on the 3D bounding box to determine the motion state of the point cloud corresponding to the 3D bounding box; and

When the motion state of the point cloud is dynamic, the point cloud motion segmentation result is updated according to the non-ground point cloud data corresponding to the three-dimensional bounding box to obtain the target point cloud motion segmentation result.
The storage medium according to claim 16, wherein when the computer-readable instructions are executed by the processor, the following step is further performed: determining the current frame corresponding to the non-ground point cloud data of the current frame in the three-dimensional bounding box The 3D bounding box and the historical 3D bounding box corresponding to the non-ground point cloud data of the historical frame; and performing occlusion analysis on the current 3D bounding box and the historical 3D bounding box to identify the point cloud motion state corresponding to the current 3D bounding box .
The storage medium according to claim 17, wherein the computer-readable instructions further perform the following steps when executed by the processor: acquiring point cloud scan lines corresponding to each historical three-dimensional bounding box; identifying the current three-dimensional the positional relationship between the bounding box and the point cloud scan line; and identifying the point cloud motion state corresponding to the current bounding box according to the positional relationship.
The storage medium according to claim 16, wherein when the computer-readable instructions are executed by the processor, the following steps are further performed: extracting dynamic point cloud data corresponding to the three-dimensional bounding box; In the motion segmentation result, determine the point cloud motion segmentation result corresponding to each dynamic point in the dynamic point cloud data; and when the determined point cloud motion segmentation result is a static point, replace the determined point cloud motion segmentation result is a dynamic point.
The storage medium according to claim 16, wherein the computer-readable instruction further performs the following steps when executed by the processor: dividing the point cloud area corresponding to each frame of point cloud data into a plurality of grids; Calculate the ground corresponding to the point cloud data in each grid according to the preset plane equation; calculate the distance value between each point in the point cloud data of each grid and the corresponding ground; Filter to get multi-frame non-ground point cloud data.