WO2022142628A1

WO2022142628A1 - Point cloud data processing method and device

Info

Publication number: WO2022142628A1
Application number: PCT/CN2021/125567
Authority: WO
Inventors: 金磊; 杨鑫
Original assignee: 华为技术有限公司
Priority date: 2020-12-29
Filing date: 2021-10-22
Publication date: 2022-07-07
Also published as: CN114693696A

Abstract

The present application provides a point cloud data processing method and device. The method comprises: obtaining initial point cloud data, and determining a target area of the initial point cloud data, wherein the target area comprises all point clouds in the initial point cloud data; performing grid division on the target area to obtain multiple grids, wherein feature data of each point cloud in each grid satisfies a feature distribution condition, and the feature data is used for representing the spatial geometric relationship between the point cloud and neighboring points of the point cloud; selecting a target point cloud of each grid from the point clouds comprised in each grid, and deleting the point clouds except the target point cloud in each grid; and obtaining target point cloud data according to the target point cloud of each grid. According to the point cloud data processing method provided by the present application, firstly, point cloud data is subjected to grid division according to the geometric distribution features of point clouds, and then a target point cloud of each grid is used as a representative point of the grid, so that when the point clouds are compressed, damage to the shape features of the point clouds can be reduced, and the compression effect of the point cloud data is improved.

Description

Method and device for processing point cloud data

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims the priority of the Chinese patent application with the application number 202011588423.1 and the application title "a point cloud data processing method and device", which was submitted to the China Patent Office on December 29, 2020, the entire contents of which are incorporated herein by reference Applying.

technical field

The present application relates to the technical field of data processing, and in particular, to a method and device for processing point cloud data.

Background technique

Three-dimensional (3-dimension, 3D) point cloud data is important data to be collected in various scenarios such as autonomous driving and high-precision map production. At present, 3D point cloud data is generally collected by high-precision laser radar. The size of a file containing 3D point cloud data is about 1.8 megabytes (MB), and the 3D point cloud data collected for about ten minutes will get 10 gigabytes (gigabytes). , GB) size file. Therefore, the 3D point cloud data collected by the existing method has a large amount of data, which will occupy a large amount of memory space and disk space, and the data calculation amount in the process of playing the 3D point cloud in real time is also large. Therefore, it is necessary to compress the collected 3D point cloud data.

At present, the commonly used 3D point cloud compression method is the proportional voxel filtering method, which reduces the number of point clouds by filtering the collected 3D point cloud data, thereby realizing point cloud compression. However, this method damages the point cloud structure greatly, and the shape deviation between the compressed point cloud data and the original point cloud data is large, so the point cloud compression effect is poor.

SUMMARY OF THE INVENTION

The present application provides a point cloud data processing method and device, which are used to reduce the damage to the shape features of the point cloud data and improve the point cloud compression effect during point cloud compression.

In a first aspect, the present application provides a point cloud data processing method, the method includes: acquiring initial point cloud data, and determining a target area of the initial point cloud data, wherein the target area includes the initial point cloud data Each point cloud in the grid; the target area is divided into grids to obtain multiple grids, wherein the feature data of the point cloud in each grid satisfies the feature distribution condition, and the feature data is used to represent the point cloud and all the grids. Describe the spatial geometric relationship between the neighboring points of the point cloud; select the target point cloud of each grid from the point clouds contained in each grid, and delete other point clouds except the target point cloud in each grid ; According to the target point cloud of each grid, get the target point cloud data.

In this method, the target area of the point cloud data is divided into grids, and then the target point cloud in each grid is used to replace all point clouds in the grid, which can effectively reduce the number of point clouds and realize point cloud compression. At the same time, the spatial geometric characteristics of the point cloud in the divided grid must meet the characteristic distribution conditions, that is, the grid division in this method is based on the geometric distribution characteristics of the point cloud, which can ensure that while compressing the point cloud, It avoids major damage to the geometric distribution characteristics of the point cloud, thereby reducing the damage to the shape characteristics of the point cloud data and improving the point cloud compression effect. In addition, the geometric distribution feature of the above point cloud is a local feature with invariant attitude, so the rotation and translation invariance of the point cloud can be guaranteed during grid division, the stability is improved, and the error of point cloud data processing can be reduced.

In a possible design, the dividing the target area into grids includes: dividing the target area into multiple grids according to a set grid size; dividing each grid that does not meet the dividing conditions The following steps are performed on each grid that satisfies the dividing conditions: the target grid is divided into multiple grids, and the target grid is each grid that meets the dividing conditions; wherein, the The division condition is that the feature data of the point cloud in the grid does not satisfy the feature distribution condition.

In this method, when dividing the grid, by setting the dividing conditions, the grid can be selectively divided, and finally non-uniform grid division can be realized. The point cloud is selectively simplified, thereby reducing the elimination of some point clouds with obvious features and reducing the damage to the shape features of the point cloud data.

In a possible design, whether the feature data of the point cloud in the grid satisfies the feature distribution condition is determined according to the following methods: calculating the first target parameter according to the feature data of all point clouds in the grid; if it is determined that the If the first target parameter is not greater than the set threshold, it is determined that the feature data of the point cloud in the grid satisfies the feature distribution condition; otherwise, it is determined that the feature data of the point cloud in the grid does not meet the feature distribution condition.

In this method, by determining the relevant parameters reflecting the geometric distribution characteristics of the point cloud in the grid according to the feature data of the extracted point cloud, it is possible to quickly determine whether the divided grid meets the requirements according to the relevant parameters, and then it can be determined whether The grid is further divided.

In a possible design, after obtaining the target point cloud data according to the target point cloud of each grid, the method further includes: adopting a random sampling consensus algorithm to perform a random sampling on the point cloud included in the target point cloud data. Plane fitting to obtain at least one fitting plane; in the at least one fitting plane, determine a first target fitting plane whose distance from the target plane in the target coordinate system is lower than the set distance value; if the at least one fitting plane is If there is a second target fitting plane that satisfies the setting condition in the fitting plane, then perform voxel filtering on the point cloud data included in the second target fitting plane; wherein the setting condition includes: perpendicular to the For the first target fitting plane, the length along the target direction is greater than the set length value, and the number of point clouds located in the plane is greater than the set value.

In this method, after removing some point clouds in the initial point cloud data to obtain the target point cloud data, some planes with more point clouds are further selected in the target point cloud data, and the selected planes are filtered. The general distribution characteristics of the point cloud are relatively smooth, so removing some of the redundant points will not cause great damage to the shape characteristics of the point cloud data, and at the same time, it can further simplify the point cloud data and reduce the size of the point cloud data file. .

In a possible design, before determining the target area of the initial point cloud data, the method further includes: filtering the initial point cloud data to remove outliers in the initial point cloud data.

In this method, the initially collected point cloud data generally contains a certain amount of noise points and measurement error points. The existence of these outlier points will affect the local distribution characteristics of the point cloud in the initial point cloud data. Therefore, by removing the outlier points It can improve the accuracy of point cloud feature extraction in point cloud data, thereby improving the accuracy of point cloud data processing.

In a possible design, after selecting the target point cloud of each grid from the point clouds contained in each grid, the method further includes: adjusting the second target parameter of the target point cloud in each grid is the average value of the second target parameter for all point clouds contained in this grid.

In this method, when the target point cloud of the grid is used as the representative point of all point clouds in the grid, by adjusting the relevant parameters of the target point cloud to the average value of all the relevant parameters of the point clouds in the grid where it is located, it is possible to relatively More accurately reflect the overall point cloud parameter characteristics of the grid, and avoid deviations caused by some extreme parameter values.

In a possible design, the target point cloud of each grid is the centroid point in the point cloud contained in each grid.

In this method, the centroid point of the point cloud in the grid is used as the representative point of the grid, which can better preserve the spatial layout characteristics of the point cloud in the point cloud data and reduce the damage to the point cloud distribution characteristics of the point cloud data.

In a second aspect, the present application provides a point cloud data processing device, the device includes an acquisition unit and a processing unit; the acquisition unit is used to acquire initial point cloud data; the processing unit is used to determine the value of the initial point cloud data a target area, wherein the target area includes each point cloud in the initial point cloud data; the processing unit is further configured to perform grid division on the target area to obtain multiple grids, wherein each grid The feature data of the point cloud in the grid satisfies the feature distribution condition, and the feature data is used to represent the spatial geometric relationship between the point cloud and the neighboring points of the point cloud; select each point cloud contained in each grid. The target point cloud of the grid is deleted, and other point clouds except the target point cloud in each grid are deleted; according to the target point cloud of each grid, the target point cloud data is obtained.

In a possible design, the processing unit divides the target area into grids, including: dividing the target area into multiple grids according to a set grid size; The grid is not divided; perform the following steps on each grid that meets the dividing conditions: divide the target grid into multiple grids, and the target grid is each grid that meets the dividing conditions; wherein, The division condition is that the feature data of the point cloud in the grid does not satisfy the feature distribution condition.

In a possible design, the processing unit determines whether the feature data of the point cloud in the grid satisfies the feature distribution condition according to the following manner: calculating the first target parameter according to the feature data of all point clouds in the grid; If it is determined that the first target parameter is not greater than the set threshold, it is determined that the feature data of the point cloud in the grid satisfies the feature distribution condition; otherwise, it is determined that the feature data of the point cloud in the grid does not satisfy the feature distribution condition. Feature distribution conditions.

In a possible design, after obtaining the target point cloud data according to the target point cloud of each grid, the processing unit is further configured to: adopt a random sampling consensus algorithm to analyze the points included in the target point cloud data Perform plane fitting on the cloud to obtain at least one fitting plane; in the at least one fitting plane, determine the first target fitting plane whose distance from the target plane in the target coordinate system is lower than the set distance value; if the If there is a second target fitting plane that satisfies the setting condition in at least one fitting plane, then perform voxel filtering on the point cloud data included in the second target fitting plane; wherein, the setting condition includes: perpendicular to the The length of the first target fitting plane along the target direction is greater than the set length value, and the number of point clouds located in the plane is greater than the set value.

In a possible design, before determining the target area of the initial point cloud data, the processing unit is further configured to: filter the initial point cloud data to remove outliers in the initial point cloud data point.

In a possible design, after the processing unit selects the target point cloud of each grid from the point clouds contained in each grid, the processing unit is further configured to: convert the second target of the target point cloud in each grid The parameter is adjusted to the average value of the second target parameter of all point clouds contained in this grid.

In a third aspect, the present application provides a point cloud data processing device, including a memory and a processor; the memory is used to store a computer program; the processor is used to execute the calculation program stored in the memory, so as to realize the above-mentioned first aspect or the method described in any possible design of the first aspect.

In a fourth aspect, the present application provides a point cloud data processing device, including at least one processor and an interface; the interface is used to provide program instructions or data for the at least one processor; the at least one processor is used to execute The program instructions implement the method described in the first aspect or any possible design of the first aspect.

In a fifth aspect, the present application provides a computer-readable storage medium, where a computer program is stored in the computer-readable storage medium, and when the computer program is executed on a data processing apparatus, the data processing apparatus is made to execute the above-mentioned first The method described in the aspect or any possible design of the first aspect.

In a sixth aspect, the present application provides a computer program product, the computer program product includes a computer program or an instruction, when the computer program or instruction is executed by a data processing apparatus, realizes the first aspect or any one of the first aspects. possible designs of the method described.

In a seventh aspect, the present application provides a chip system, the chip system includes at least one processor and an interface, the interface is used to provide program instructions or data for the at least one processor, and the at least one processor is used to execute all the The program instructions are used to implement the method described in the first aspect or any possible design of the first aspect.

In a possible design, the chip system further includes a memory for storing program instructions and data.

In one possible design, the chip system consists of chips, or includes chips and other discrete devices.

For the beneficial effects of the second aspect to the seventh aspect, please refer to the description of the beneficial effects of the first aspect, which will not be repeated here.

Description of drawings

1 is a schematic diagram of a possible application scenario to which a point cloud data processing method provided by an embodiment of the present application is applicable;

2 is a schematic diagram of a point cloud data processing method provided by an embodiment of the present application;

3 is a schematic diagram of a fitting plane processing method provided by an embodiment of the present application;

4 is a schematic diagram of a method for dividing a point cloud grid according to an embodiment of the present application;

5 is a schematic diagram of comparison before and after point cloud data processing provided by an embodiment of the present application;

6a is a schematic diagram of a method for limiting the display range of a point cloud provided by an embodiment of the present application;

6b is a schematic diagram of another method for limiting the display range of a point cloud provided by an embodiment of the present application;

FIG. 7a is a schematic diagram of original point cloud data provided by an embodiment of the present application;

7b is a schematic diagram of a processed point cloud data provided by an embodiment of the present application;

FIG. 8 is a schematic diagram of a data processing apparatus according to an embodiment of the present application;

FIG. 9 is a schematic diagram of a data processing apparatus according to an embodiment of the present application.

Detailed ways

In order to make the objectives, technical solutions and advantages of the embodiments of the present application more clear, the embodiments of the present application will be described in further detail below with reference to the accompanying drawings. Among them, in the description of the embodiments of the present application, the terms "first" and "second" are only used for description purposes, and should not be understood as indicating or implying relative importance or indicating the number of technical features indicated. . Thus, a feature defined as "first" or "second" may expressly or implicitly include one or more of that feature.

For ease of understanding, descriptions of concepts related to the present application are exemplarily given for reference.

1) Point cloud data: The set of point data on the surface of an object measured by measuring equipment can be called point cloud data. Point cloud data is a collection of points obtained after obtaining the spatial coordinates of each sampling point on the surface of the object, also known as the mass point collection of the surface characteristics of the target object.

The point cloud data (also referred to as laser point cloud data) measured based on the principle of laser measurement includes information such as three-dimensional coordinates and laser reflection intensity. The point cloud data obtained based on the principle of photogrammetry includes information such as three-dimensional coordinates and color, wherein the color information may be color data in red, green, blue, RGB format. Combining the principles of laser measurement and photogrammetry, point cloud data including three-dimensional coordinates, laser reflection intensity and color information are obtained.

2), point feature histograms (point feature histograms, PFH): The point feature histogram is a local feature with invariant attitude, which is based on the relationship between the point cloud contained in the point cloud data and its neighboring points and their estimation. Normals are used to describe the geometric features of local point cloud data. PFH uses parameterization to query the spatial difference between the point cloud and the neighboring points, and forms a multi-dimensional histogram to describe the neighborhood geometric properties of the point cloud. The high-dimensional hyperspace in which the histogram resides provides a measurable information space for feature representation, is invariant to the 6-dimensional pose of the surface corresponding to the point cloud, and is robust to different sampling densities or noise levels in the neighborhood . Fast point feature histograms (FPFH) are a simplified form of how PFH is calculated.

It should be understood that, in the embodiments of the present application, "at least one" refers to one or more, and "a plurality" refers to two or more. "And/or", which describes the association relationship of the associated objects, means that there can be three kinds of relationships, for example, A and/or B, it can mean that A exists alone, A and B exist at the same time, and B exists alone, where A, B can be singular or plural. The character "/" generally indicates that the associated objects are an "or" relationship. "At least one (item) of the following" or its similar expression refers to any combination of these items, including any combination of single item (item) or plural item (item). For example, at least one (a) of a, b or c may represent: a, b, c, a and b, a and c, b and c, or a, b and c, where a, b, c Can be single or multiple.

The specific operation methods in the method embodiments may also be applied to the apparatus embodiments or the system embodiments.

3D point cloud data is an important data that needs to be collected in various scenarios such as autonomous driving and high-precision map production. For example, in the field of intelligent driving, high-precision electronic map acquisition systems or automatic driving systems and assisted driving systems usually use radar to collect point cloud data, for example, use LiDAR (light detection and ranging, LiDAR) to obtain 3D point clouds with reflection intensity data, and then obtain the corresponding environmental information according to the 3D point cloud data.

Lidar is a radar system that emits a laser beam to detect the position, speed and other characteristic quantities of a target. Its working principle is to transmit a detection signal such as a laser beam to the target, and then receive the signal reflected from the target, such as target echo and emission. Signals are compared, and relevant information of the target is obtained after appropriate processing, such as the target's distance, orientation, height, speed, attitude, and even shape and other parameters. If the laser beam is scanned according to a certain trajectory by the lidar, the reflected laser point information will be recorded while scanning. Since the scanning is extremely fine, a large number of laser points can be obtained during the scanning process, thereby forming point cloud data.

At present, the amount of 3D point cloud data collected by radar is very large, which requires a lot of memory and disk space for storage, which is not conducive to point cloud data transmission, point cloud data playback and other processing. Therefore, it is necessary to compress or compress point cloud data. Streamlined, thereby reducing the data volume of point cloud data.

Point cloud compression needs to keep the shape characteristics of point clouds while reducing the number of point clouds and reducing the size of point cloud data files. At present, the commonly used point cloud compression method is the proportional voxel filtering method, which homogenizes and sparses the point cloud in the point cloud data, so it is easy to lose the shape or distribution characteristics of the point cloud, resulting in the effect of point cloud compression. poor.

In view of this, an embodiment of the present application provides a point cloud data processing method, which selectively sparse and simplifies point cloud data according to point cloud characteristics, which can compress the data volume of point cloud data and reduce the need for point cloud data. The destruction of the structure, thereby improving the point cloud compression effect.

It should be noted that, in this embodiment of the present application, compressing point cloud data can be understood as selectively deleting a part of the point cloud data from the collected point cloud data and retaining the other part of the point cloud, thereby reducing the amount of data contained in the point cloud data. The number of point clouds reduces the file size of point cloud data and realizes compression of point cloud data. Therefore, compressing point cloud data can also be understood as streamlining the point cloud data.

The point cloud data processing method provided in the embodiment of the present application can compress the point cloud data collected by the radar, and the method can be applied to a data processing device with data processing capability, and the data processing device can be a vehicle with data processing function, Or the on-board equipment with data processing function in the vehicle, or set in the sensor with the function of collecting and processing point cloud data. Vehicle-mounted devices may include, but are not limited to, vehicle-mounted terminals, vehicle-mounted controllers, vehicle-mounted modules, vehicle-mounted modules, vehicle-mounted components, vehicle-mounted chips, vehicle-mounted units, vehicle-mounted radars, and other devices. The data processing device may also be other electronic devices with data processing functions, including but not limited to smart home devices (such as TVs, etc.), smart robots, mobile terminals (such as mobile phones, tablet computers, etc.), wearable devices (such as smart Watches, etc.) and other smart devices. The data processing device may also be a controller, a chip, a radar and other devices in the smart device.

The method for processing point cloud data provided by the embodiments of the present application will be described in detail below with reference to the accompanying drawings. It can be understood that the embodiments described below are only a part of the embodiments of the present application, rather than all the embodiments.

FIG. 1 is a schematic diagram of a possible application scenario to which a point cloud data processing method provided by an embodiment of the present application is applicable. As shown in FIG. 1 , the application scenario of the point cloud data processing method provided by the embodiment of the present application may be an assisted driving scenario. In this scenario, the vehicle is in a certain natural environment, for example, the vehicle is driving on a road, and the setting on the vehicle is There is radar, which can take measurements of the surrounding environment to obtain point cloud data of the surrounding environment of the vehicle.

The radar can send the collected point cloud data to the vehicle where it is located or the vehicle-mounted device on the vehicle, so that the vehicle or the vehicle-mounted device can perform the point cloud data processing method provided in the embodiment of the present application on the collected point cloud data; or, the radar Execute the point cloud data processing method provided by the embodiment of the present application on the obtained point cloud data, and send the processed point cloud data to the vehicle or the vehicle-mounted device on the vehicle, so that the vehicle or the vehicle-mounted device can process the processed point cloud data. Perform subsequent operations, such as playing point cloud data, etc.

In FIG. 1 , the radar is just an example of a device that can collect point cloud data. The device that collects point cloud data in this embodiment of the present application is not limited to radar, but can also be any other device that can collect point cloud data. For example, it can also be A device such as a camera is not limited in this embodiment of the present application. The position of the radar on the vehicle is just an example, and the specific position of the radar on the vehicle is not limited thereto.

Of course, FIG. 1 is just an example, and the application scenarios of the embodiments of the present application are not limited thereto. For example, the data processing device used to execute the point cloud data processing method provided by the embodiments of the present application may not be a radar but other equipment, and the device may not be installed on the vehicle, but may be installed on other equipment, or the Devices can also be set individually.

The method for processing point cloud data provided by the present application will be described in detail below with reference to specific embodiments.

FIG. 2 is a schematic diagram of a point cloud data processing method provided by an embodiment of the present application.

For ease of introduction, hereinafter, the method for processing point cloud data provided by the present application is performed by a data processing apparatus as an example for description. The data processing apparatus may be, but is not limited to, the apparatus with data processing capability provided in this embodiment of the present application, for example, may be a radar, a vehicle, or an in-vehicle device in the scenario shown in FIG.

As shown in Figure 2, the point cloud data processing method provided by this application includes:

S201: The data processing apparatus acquires initial point cloud data, and determines a target area of the initial point cloud data, wherein the target area includes each point cloud in the initial point cloud data.

In the embodiments of the present application, the radar is used as an example for the device for collecting initial point cloud data for description.

The data of the point cloud in the initial point cloud data collected by the radar includes at least position information of the point cloud, and the position information may be the three-dimensional coordinates of the point cloud. The point cloud data may also include point cloud color parameters such as RGB color values and reflection intensity parameters.

The initial point cloud data is obtained by the radar measuring the environment in the scene. Exemplarily, when the radar is the radar shown in FIG. 1, and the scene where the radar is located is the scene where the vehicle is driving on the road shown in FIG. 1, the radar measures and collects data on the surrounding environment to obtain initial point cloud data, and sent to the data processing device. Among them, the radar can collect point cloud data of all objects including vehicles, roads, other vehicles on the road, signs on both sides of the road, buildings, etc., or collect point cloud data of specific objects or specific ranges according to actual needs .

In this embodiment of the present application, the target area of the initial point cloud data needs to include all the point clouds in the initial point cloud data, and the target area may be the smallest circumscribed area of all the point clouds in the initial point cloud data. Wherein, the shape of the region may be any shape or a shape set arbitrarily, for example, it may be a rectangular parallelepiped, a cube, a triangular prism, a triangular pyramid, a sphere, and the like.

In the embodiment of the present application, the shape of the target area is a cuboid as an example to introduce and describe the data processing method provided by the present application. For the processing method when the target area is other shapes, refer to the processing method adopted when the target area is a cuboid.

When the shape of the target area of the initial point cloud data is a cuboid, the target area can be understood as the circumscribed cuboid of the initial point cloud data. The following takes the circumscribed cuboid as the smallest circumscribed cuboid that can contain all the point clouds in the initial point cloud data as an example. illustrate.

After acquiring the initial point cloud data collected by the radar, the data processing device determines the smallest cuboid that can contain all point clouds in the initial point cloud data, and determines the smallest cuboid as the circumscribed cuboid of the initial point cloud data. Wherein, when the lengths of each side of the circumscribed cuboid are equal, the circumscribed cuboid may also be called a circumscribed cube. A circumscribed cuboid (or a circumscribed cube) is a frame surrounding the object being processed, and can be understood as an imaginary frame surrounding the initial point cloud data in this application.

In some embodiments of the present application, after acquiring the initial point cloud data collected by the radar, the data processing device may also determine the smallest cube that can contain all the point clouds in the initial point cloud data, and determine the smallest cube as the size of the initial point cloud data. The circumscribed cube is subsequently processed as described below.

Radar scanning usually generates point cloud sets with different point cloud densities. In addition, the noise points generated in the measurement and the sparse outliers due to measurement errors will affect the distribution of local features of the point cloud data, which may destroy the processing results. . Therefore, in some embodiments of the present application, before determining the circumscribed cuboid of the initial point cloud data, the data processing device may first perform filtering processing on the initial point cloud data, and perform a statistical analysis on the neighborhood of each point cloud through the filtering processing. , and remove some point clouds that do not meet the standard, so as to ensure the accuracy of subsequent processing.

In some embodiments of the present application, the data processing apparatus may use a statistical outlier removal filter to determine and delete outlier points from the point cloud included in the initial point cloud data. During specific implementation, the data processing device may set the parameters of the statistical outlier elimination filter as follows: the search neighborhood for the point cloud is the set neighborhood value, and the reference standard deviation multiple is the set multiple value, for example, the set neighborhood value It can be 50, and the standard deviation multiple can be 1. For a certain point cloud within the set search neighborhood, if the average distance between the point cloud and other point clouds in the search neighborhood is greater than the standard range of point cloud distribution in the search neighborhood, the point cloud is determined to be an outlier. and removed from the initial point cloud data. In the specific implementation, calculate the average distance from each point cloud to all its neighbor points in the search neighborhood. Assuming that the obtained result is a Gaussian distribution whose shape is determined by the mean and standard deviation, the corresponding average distance is Points outside the standard range are determined as outliers, where the standard range is the range corresponding to the product of the standard deviation and the multiple of the set standard deviation.

S202: The data processing device divides the target area into grids to obtain a plurality of grids, wherein the feature data of the point cloud in each grid satisfies the feature distribution condition, and the feature data is used to represent the point cloud and the neighborhood of the point cloud Spatial geometric relationship between points.

Specifically, when the data processing apparatus divides the target area into grids, the following methods may be adopted: first, according to the set grid size, the target area is divided into multiple grids; in the obtained multiple grids, for each grid The grids that do not meet the division conditions will not be divided; perform the following steps for each grid that meets the division conditions: divide the target grid into multiple grids, where the target grid is each grid that meets the division conditions grid.

The division condition is that the characteristic data of the point cloud in the grid does not meet the characteristic distribution condition, and the data processing device determines whether the characteristic data of the point cloud in the grid meets the characteristic distribution condition in the following way: According to the characteristic data of all point clouds in the grid Calculate the first target parameter; if it is determined that the first target parameter is not greater than the set threshold, it is determined that the feature data of the point cloud in the grid satisfies the feature distribution condition; otherwise, it is determined that the feature data of the point cloud in the grid does not meet the feature distribution condition.

The above-mentioned first target parameter may be a variance parameter, a standard deviation parameter, etc., wherein, in the following embodiments of the present application, the first target parameter is taken as an example to introduce the solution.

A detailed description will be given below.

After the data processing device determines the circumscribed cuboid of the acquired initial point cloud data, according to the set grid size, the circumscribed cuboid is evenly divided, and the circumscribed cuboid is divided into multiple grids. The length of each side of the grid corresponding to the set grid size may be the same or different.

Among them, the grid can also be understood as a fictitious outer frame that contains part of the point cloud or does not contain the point cloud. Set the grid size to one of the following:

1) Set the grid size to the set fixed size.

Exemplarily, the set grid size may be a fixed 40×30×20 cm, that is, the grid corresponding to the set grid size is a cuboid with side lengths of 40, 30, and 20 cm, respectively, and the data processing device will The circumscribed cuboid is evenly divided into multiple grids with a size of 40×30×20 cm.

2) Set the grid size to the size determined according to the size of the circumscribed cuboid.

Exemplarily, the product of each side length of the circumscribed cuboid and the set coefficient may be respectively determined as the grid side length corresponding to the set grid size, and the set coefficient is greater than 0 and not greater than 1.

For example, if the set coefficient is 0.1%, when the size of the circumscribed cuboid determined by the data processing device is 100×80×40 meters, the side lengths of the grids corresponding to the set grid size are respectively 100×0.1%= 0.1m, 80×0.1%=0.08m, 40×0.1%=0.04m, that is, the grid size is set to 10×8×4 cm, then the data processing device divides the circumscribed cuboid into a plurality of 10×8 For example, when the length of each side of the circumscribed cuboid determined by the data processing device is 50 meters, that is, when the circumscribed cuboid is a cube at the same time, the grid size is set to 5×5×5 cm.

In this embodiment of the present application, the grid may also be referred to as a voxel grid.

After the above-mentioned data processing device preliminarily divides the circumscribed cuboid, it respectively judges whether each grid obtained by the division satisfies the division conditions, and if so, continues to divide the grid, and continues to judge each grid obtained after division Whether the division conditions are met and continue to divide when the division conditions are met, and so on, until all the divided grids do not meet the division conditions, stop dividing the grid.

Specifically, after the data processing device preliminarily divides the circumscribed cuboid into a plurality of grids, it continues to perform the following grid division steps:

Step 1: The data processing apparatus takes each grid of the divided grids as a target grid.

Step 2: The data processing device judges whether the target grid satisfies the division conditions, and if yes, executes Step 3, otherwise, the target grid is not divided.

Step 3: The data processing apparatus divides the target grid into a plurality of grids, and uses each grid of the plurality of grids obtained by division as a target grid, and performs step 2.

The local geometric features of the point cloud include PFH or FPFH. PFH or FPFH is a local feature with invariant attitude, which has feature invariance to the 6-DOF pose of the corresponding surface of the point cloud, and is in different sampling densities or neighborhoods. Feature extraction under noise level is robust. Therefore, in the grid division process, it can be determined whether to further divide the grid according to the PFH or FPFH characteristics of the point cloud in the grid.

Specifically, in the above step 2, when judging whether the target grid satisfies the division conditions, the data processing device firstly determines the feature data of each point cloud in the target grid according to the point cloud contained in the target grid, wherein the feature data It is used to represent the positional relationship between the point cloud and the neighboring points of the point cloud; then, the data processing device calculates the variance of the feature data of all point clouds in the target grid, and judges whether the obtained variance is greater than the set threshold, if the If the variance is greater than the set threshold, it is determined that the feature data of the point cloud in the target grid does not meet the feature distribution conditions, then the target grid satisfies the division conditions; otherwise, it is determined that the feature data of the point cloud in the target grid meets the feature distribution conditions, then the target grid The raster does not meet the division criteria.

As an optional implementation manner, the variance calculated above is the variance calculated according to the PFH of the point cloud within the target grid range. Specifically, the feature data of the above point cloud includes a distance parameter and three angle parameters, wherein one distance parameter is the Euclidean distance between the point cloud and a certain neighborhood point, and the three angle parameters are used to represent the point cloud. The angular deviation between the normal of and the normal of one of its neighbor points. The data processing device uses the point cloud and the neighboring points of the point cloud to form point pairs for each point cloud in the target grid, and calculates the distance parameters and angle parameters corresponding to each group of point pairs. After the calculation is completed, the PFH is obtained, and Find the variance of the feature data contained in the PFH. Finally, it is determined whether the target grid satisfies the division conditions according to the obtained variance.

As another optional implementation manner, the variance calculated above is the variance calculated according to the FPFH of the point cloud within the target grid range. Specifically, the feature data of the above point cloud includes three angle parameters, which are used to represent the angle deviation between the normal of the point cloud and the normal of its neighboring points. For each point cloud in the target grid, the data processing device uses the point cloud and the neighboring points of the point cloud to form point pairs, and calculates the angle parameters corresponding to each group of point pairs. After the calculation is completed, the FPFH is obtained. The feature data contained in the FPFH is varianced. Finally, it is determined whether the target grid satisfies the division conditions according to the obtained variance.

After determining the variance corresponding to the grid, compare the variance with the set threshold. If the variance is less than the set threshold, it means that the PFH or FPFH characteristics of the point cloud in the grid change little, and the grid does not need to be further divided. If the variance is greater than or equal to the set threshold, it means that the PFH or FPFH characteristics of the point cloud in the grid change drastically, and the grid can be further divided.

The point cloud in general point cloud data is three-dimensional, and the data of three-dimensional point cloud is unstructured data, which has the characteristics of sparsity, disorder, non-uniform distribution, and large number of changes. In the above method of this embodiment, the feature data of the point cloud is determined according to the point pair formed by the point cloud and its neighboring points, which has a certain anti-interference, such as anti-rotation, etc., so more accurate and robust features can be obtained. data.

In the above step 3, as an optional implementation manner, when the data processing apparatus divides the target grid into multiple grids, the value of each side length of the target grid may be reduced to 1/n of the original side length value Then, taking the obtained size as the side length, and dividing the target grid according to the corresponding grid size, the target grid can be evenly divided into n ³ grids, where n is an integer not less than 2.

For example, when the target grid is a cube with a side length of 40 centimeters and n is 2, the side length value of 40 centimeters is reduced to 1/2 of the original value and then becomes 20 centimeters, then the data processing device divides the target grid into side lengths. For a grid of 20 cm, a total of 8 smaller grids are obtained. When the target grid is a 90×60×30 cm cube and the value of n is 3, the size of the target grid is 30×20×10 cm after the value of each side is reduced to 1/3 of the original, then the data processing device will The grid is evenly divided into grids of size 30 × 20 × 10 cm, resulting in a total of 27 smaller grids.

When the data processing device finishes performing the grid division step and determines that all grids obtained by division do not meet the division conditions, it determines that the grid division is completed, and continues to perform the point cloud reduction process described below.

S203: The data processing apparatus selects the target point cloud of each grid from the point clouds contained in each grid, and deletes other point clouds except the target point cloud in each grid.

After the data processing device divides the grid, the point cloud data can be simplified. Specifically, the data processing apparatus selects the target point cloud of each grid from the point clouds contained in each grid.

In some embodiments of the present application, the target point cloud is the centroid point in the point cloud included in the grid, wherein the coordinates of the centroid point in the grid are obtained by calculating the average value of the coordinates of all the point clouds in the grid . In some embodiments of the present application, the target point cloud may also be the center of gravity point in the point cloud included in the grid.

For each grid, after determining the target point cloud in the grid, the data processing device deletes other point clouds in the grid except the target point cloud, and only replaces the grid with the target point cloud. At the same time, the second target parameter of the target point cloud in the grid can also be adjusted to the average value of the second target parameters of all point clouds contained in the grid, wherein the second target parameter includes the color parameter of the point cloud and/or Or the reflection intensity parameter, which can be an RGB color value. Specifically, the data processing device may adjust the RGB color parameters of the target point cloud in the grid to the average value of the red, green and blue color parameters of all point clouds contained in the grid; and/or, the target point in the grid The reflection intensity parameter of the cloud is adjusted to the average of the reflection intensity parameters of all point clouds contained in this grid.

S204: The data processing device obtains target point cloud data according to the target point cloud of each grid.

After the data processing device performs the above-mentioned processing on each grid, the target point cloud data composed of all retained target point clouds is obtained, and the target point cloud data is the point cloud data after the initial point cloud data is simplified or compressed.

In the embodiment of the present application, the target point cloud data obtained by simplifying the initial point cloud data may also include many points with smooth but redundant features. Taking the scene shown in Figure 1 as an example, in the target point cloud data obtained by simplifying the initial point cloud data collected by the radar, the points with smooth but redundant features are reflected in the road surface and building plane around the vehicle. The road surface generally contains There is more traffic sign information, so its point cloud data can be retained without excessive processing, but the redundant building plane data on both sides of the road can be further simplified. Therefore, in some embodiments of the present application, the target point cloud data may be further downsampled by plane fitting, so as to remove redundant point clouds on the plane of buildings on both sides of the road, and further reduce the number of point clouds.

Specifically, after the data processing device obtains the target point cloud data, a random sampling consensus algorithm can be used to perform plane fitting on the point cloud included in the target point cloud data to obtain at least one fitted plane, and then in the at least one fitted plane, Determine the first target fitting plane whose distance from the target plane in the target coordinate system is lower than the set distance value; The point cloud data contained in the combined plane is filtered; wherein, the setting conditions include: perpendicular to the first target fitting plane, the length along the target direction is greater than the set length value, and the number of point clouds in the plane is greater than the set value , the target coordinate system is the coordinate system adopted by the device for collecting initial point cloud data, and the coordinate system is a coordinate system with the center of the device (such as radar) collecting initial point cloud data as the origin.

A detailed description will be given below.

The data processing device may perform random sample consensus (RANSAC) plane fitting on the point cloud contained in the target point cloud data. Among them, the plane fitting takes N points as the fitting condition, and determines the plane containing the number of point clouds greater than N, where N is a set positive integer.

As an optional implementation manner, the data processing device may first randomly select three point clouds from the target point cloud data, form a plane by these three point clouds, and then calculate the target point cloud data separately from other point clouds to The distance of the plane, if the distance from the point cloud to the plane is less than the preset distance value, the point cloud is considered to be a point located in this plane, otherwise, the point cloud is considered not to be located in the plane. Finally, if it is determined that the number of point clouds located in the plane is greater than N, the plane is determined as a fitting plane obtained, and all point clouds located in the fitting plane are marked as matched, and then never marked as Continue to select three point clouds from the matched point clouds, and determine the fitting plane based on the selected three point clouds. If it is determined that the number of point clouds located on the plane is not greater than N, directly continue to select three point clouds from the point clouds not marked as matched, and determine the fitting plane based on the selected three point clouds. The data processing device iteratively performs plane fitting in the above-mentioned manner until the termination condition is satisfied, and at least one fitted plane is obtained. The termination condition is that the number of point clouds contained in the determined plane after M iterations is less than N, or three point clouds not marked as matched cannot be found. Among them, M is the set number of iterations, which is a positive integer.

After determining at least one fitting plane through the above-mentioned plane fitting, the data processing device selects a plane whose number of point clouds needs to be further reduced from the at least one fitting plane, and further reduces the selected plane. The following description will be made with reference to FIG. 3 .

FIG. 3 is a schematic diagram of a fitting plane processing method provided by an embodiment of the present application. As shown in Figure 3, the X axis, the Y axis and the Z axis are the three coordinate axes of the three-dimensional coordinate system adopted by the device for collecting the initial point cloud data, wherein any two coordinate axes are perpendicular to each other, and point O is the three-dimensional coordinate system of the three-dimensional coordinate system. origin.

When applied to the scene shown in FIG. 1 , the X axis can be the same as the direction in which the vehicle is traveling, the XOY plane can be the horizontal plane in the natural coordinate system, and the Z axis represents the height. Assuming that the distribution of the point cloud contained in the target point cloud data in the three-dimensional coordinate system is shown in Figure 3, the plane obtained by the data processing device performing plane fitting on the target point cloud data includes the points O, P, The plane L1 corresponding to Q and R, the plane L2 corresponding to the points P1, P2, P3, and P4, and the plane L3 corresponding to the points P3, P4, and P5.

The target plane is the XOY plane. The data processing device first selects the plane whose distance from the target plane in the coordinate system is lower than the set distance value from the planes L1, L2, and L3, that is, the plane with the lowest plane height. Plane L1 serves as the ground plane. Then select a plane whose height along the Z-axis direction is higher than the set height value, the number of point clouds contained is greater than the set number value, and is perpendicular to the plane L1 from the rest of the planes except the plane L1. After the plane L2, the plane L2 is determined as the building plane. Then, the resolution of the plane L2 is reduced to a set value through filtering, for example, the set value may be one tenth of the original resolution of the plane L2. In specific implementation, a voxel grid filter can be used to filter the point cloud contained in the plane, wherein when the voxel filter filters the point cloud contained in the plane, the circumscribed cuboid of the point cloud contained in the plane is uniform Divide it into multiple grids, and the length of each side of each grid is one-tenth of the corresponding side length of the circumscribed cuboid, and then use the target point cloud in each grid as the representative point of the grid, and delete the Non-target point cloud in raster.

In the above method, by determining the building plane in the point cloud of the vehicle driving environment and performing down-sampling processing on it, redundant data of the building plane on both sides of the road can be effectively removed, and the amount of point cloud data can be further reduced.

It should be noted that, the step numbers in the various embodiments described in the embodiments of the present application are only an example of the execution flow, and do not constitute a limitation on the sequence of execution of the steps. There is no strict order of execution between the steps of timing dependencies. For example, in the above step S203, when it is determined that a grid does not meet the grid division requirements, the data processing device in the above step S204 may select the target point cloud from the point clouds contained in the grid and delete the grid. For the processing steps of other point clouds except the target point cloud, it is also possible to perform step S204 for each grid after the execution of step S203 and determine that all grids do not meet the grid division requirements, that is, when steps S203 and S204 are executed , the step S203 may be performed prior to the step S204, or the two steps may be selectively performed simultaneously.

In the above embodiment, in the process of streamlining and compressing point cloud data, by dividing the point cloud data into grids, the target point cloud determined in each grid is used to replace the grid, which can reduce the number of point clouds and realize the realization of point clouds. compression. At the same time, when dividing the grid, by setting the dividing conditions, the grid can be selectively divided, and the non-uniform grid division can reduce the elimination of some point clouds with obvious characteristics, thereby ensuring the shape of the point cloud data. feature to improve point cloud compression. Among them, the division conditions are set according to the feature parameters reflecting the discrete degree of point cloud distribution in the grid, which can be combined with the geometric features of the point cloud data to divide the grid to ensure the pose invariance of the point cloud, improve the robustness, and ensure the point cloud. The accuracy of the features can avoid grid division errors caused by changes such as translation and rotation of the point cloud, and further improve the effect of point cloud compression.

The grid division method in the foregoing embodiments of the present application will be described in detail below with reference to specific examples.

Referring to FIG. 4 , it is a schematic diagram of a method for dividing a point cloud grid according to an embodiment of the present application. As shown in (a) of FIG. 4 , it is assumed that the initial point cloud data acquired by the data processing device is a point cloud set S.

The following is an example of determining the circumscribed cube of the initial point cloud data, and taking the centroid point in the point cloud included in the grid as the target point cloud of the grid as an example for description.

When the data processing device processes the initial point cloud data, it first determines the circumscribed cube of the initial point cloud data. Grid S1.

The data processing apparatus divides the grid S1 into a plurality of grids according to the set grid size. As an optional implementation manner, the data processing apparatus determines and sets the grid size according to the size of the grid S1. For example, as shown in (a) of FIG. 4 , the data processing apparatus may determine half of the side length of the grid S1 as the side length corresponding to the set grid size, then the grid S1 is divided into 8 equal-sized grids After division, the side length of each grid is half of the side length of grid S1, as shown in grid S2, where grids A, B, C, D, E, F, G, and H are 8 small grids obtained by dividing the grid S2.

After the data processing device divides the grid S1 to obtain the grid S2, for each grid obtained by dividing the grid S2, it is respectively determined whether the grid satisfies the dividing conditions. In the figure, grid D and grid F are used as examples for illustration. Specifically, for grid D, as shown in (b) in FIG. 4 , if the data processing device determines that grid D does not meet the division conditions, then Grid D is divided, but the centroid point is selected in the point cloud contained in grid D, and other point clouds except the centroid point in grid D are deleted to obtain grid D1, which is shown in grid D1 The point cloud of is the centroid point of the reserved grid D. The grid D2 shown in (b) of FIG. 4 is the grid after the grid D is simplified, and is the same as the grid D1. For the grid F, as shown in (b) in FIG. 4 , the data processing apparatus determines that the grid F satisfies the division conditions, and further divides the grid F, for example, as shown in (b) in FIG. 4 As shown, using the same method as dividing the grid S1, the grid F is divided into 2 ³ =8 small grids to obtain the grid F1, wherein the grids F11, F12, F13, F14, F15, F16, F17, F18 is the 8 small grids obtained by dividing the grid F1.

After the data processing device divides the grid F to obtain the grid F1, for each grid obtained by dividing the grid F1, it is respectively determined whether the grid satisfies the division conditions. In the figure, grid F11, grid F16 and grid F17 are used as examples for illustration. Specifically, if the data processing device determines that grid F11, grid F16 and grid F17 do not meet the division conditions, grid F11 will not be processed any more. , grid F16 and grid F17 to divide, but select the centroid point in the point clouds contained in grid F11, grid F16 and grid F17 respectively, and delete other point clouds except the centroid point in the grid respectively , the grid F21, grid F26 and grid F27 shown in (b) of FIG. 4 are obtained respectively. After the data processing device performs the operations of selecting centroid points and deleting non-centroid points for each grid in the grid F1, the grid F2 shown in (b) in FIG. 4 is obtained, and the grid F2 is a pair of grids. Grid F1 for the reduced grid.

For processing manners of other grids not shown in FIG. 4 , reference may be made to the foregoing method, and details are not described herein again.

The data processing device divides the grid S1 corresponding to the circumscribed cube of the initial point cloud data according to the above grid division method, until all grids obtained by the latest division do not meet the division conditions, it is determined that the point cloud simplification is completed, and the latest division is obtained. The point clouds in all the grids of , make up the target point cloud data, as shown in grid S3. The point cloud in the grid S3 is the point cloud after the point cloud in the grid S1 is simplified.

For specific implementations of the above-mentioned partial methods, reference may be made to the relevant descriptions in the above-mentioned first embodiment, which will not be repeated here.

FIG. 5 is a schematic diagram of comparison before and after point cloud data processing according to an embodiment of the present application.

Grid W in FIG. 5 shows the point cloud distribution of the original simulated point cloud data before the point cloud data processing, and grid W1 shows the point cloud data using the method provided by the above embodiments of the present application. The point cloud distribution of the point cloud data obtained after processing. Comparing the distribution of point clouds in the grid W and the grid W1, it can be seen that after the point cloud processing method provided in the embodiment of the present application is used to process the point cloud data with a large amount of data, the number of point clouds can be effectively reduced, thereby reducing the number of point clouds. The size of the cloud data file realizes point cloud compression; at the same time, compared with the point cloud before processing shown in grid W1, the processed point cloud shown in grid W1 is denser in shape features and point cloud distribution. The differences in the degree and other aspects are relatively small, so the point cloud processing method provided by the embodiment of the present application damages the shape features of the point cloud very little, and can ensure a better point cloud compression effect.

In the above embodiments of the present application, when dividing the point cloud into a grid, the initial point cloud data is first divided into a uniform grid, and then the grid obtained by the initial division is selectively further divided according to the dividing conditions. When each grid is further divided, the uniform division method is still used. Through the above method, the non-uniform rasterization of the point cloud is finally realized, and the point cloud compression is realized by replacing the centroid point, which can reduce the damage to the shape and distribution characteristics of the point cloud, thereby improving the point cloud compression effect.

In the fields of high-precision electronic map collection, automatic driving, and assisted driving, the initial point cloud data of the scene collected by the radar may contain sensitive building features and other data that are not suitable for display, and relevant regulations require that these sensitive data should not be disclosed or large-scale. It is held for a long time, so the viewing of point cloud data is greatly restricted.

In practical applications, point cloud data is mainly used for driving perception. For point clouds that are far away from the radar, it is not very helpful for driving perception, especially in the field of intelligent driving such as automatic driving and assisted driving. Generally, it mainly relies on It is based on the point cloud data within a certain range with the vehicle center as the origin for driving perception. Therefore, the range of point cloud data collected by radar and subsequently displayed can be limited, so as to avoid or blur the point cloud of sensitive buildings. characteristics, and at the same time, it will not have much impact on driving perception.

For example, based on the above embodiment, when the radar collects the point cloud data of the surrounding environment of the vehicle, the point cloud data outside the set range can be deleted, only the point cloud data within the set range can be retained, and the set range The point cloud data inside is used as the initial point cloud data, and the initial point cloud data is processed by using the point cloud data processing method provided by the above-mentioned embodiment of the present application, and then the point cloud data is effectively simplified, and the compliance erasure is invalid. point cloud in the range.

FIG. 6a is a schematic diagram of a method for limiting the display range of a point cloud provided by an embodiment of the present application. As shown in Figure 6a, when the radar is located on the top of the vehicle, the radar is used as the origin, and in the vertical direction, only the point cloud data within the range corresponding to the set distance on both sides of the radar is retained, that is, the dotted line L4 and the dotted line L5 in Figure 6a The point cloud data within the limited range, for the point cloud data beyond this range, the radar does not perform point cloud collection or deletes the point cloud data that has been collected.

In view of the fact that the height of general small vehicles is basically below 2 meters, and the maximum height of large trucks is not more than 4.2 meters, the height distance value can be set to 2.5 meters, and only the point cloud data within the range corresponding to 2.5 on both sides of the radar is retained, that is The limit between the two dashed lines in Figure 6a is a range of 5 meters in height, which can cover both the ground and the height of basically all models. The buildings on both sides of the driveway will be shielded because of the height limit of 2.5 meters, and will not be displayed too much, so as to prevent all sensitive building information from being disclosed.

FIG. 6b is a schematic diagram of another method for limiting the display range of a point cloud provided by an embodiment of the present application. As shown in Figure 6b, on the basis of the limited range shown in Figure 6a, only the point cloud data within the range corresponding to the set distances on the left and right sides of the radar and the front and rear sides of the radar are retained, that is, the range surrounded by the dotted box in Figure 6b. For the point cloud data in this range, the radar does not collect the point cloud or deletes the point cloud data that has been collected. The x-axis shown in Figure 6b is the same as the vehicle's driving direction, and the y-axis is perpendicular to the x-axis.

Generally, the distribution of point clouds in the range beyond 100 meters before and after the vehicle is relatively sparse, which has no reference value and can be removed. Therefore, the set distance in the front and rear directions of the radar can be 100 meters. The relevant standards stipulate that the maximum width of the standard lane is 3.75 meters, and the width of the 8-lane one-way is less than 16 meters. The point cloud beyond the range of 30 meters to the left and right of the radar does not affect the target detection and can also be removed. The set distance in the direction can be 30 meters.

By retaining the point cloud data within the range of 100 meters before and after the radar, 30 meters on the left and right, and 2.5 meters on the upper and lower sides of the radar, the necessary detection information in the lane and on both sides of the road can be retained, and the collection of invalid information can be reduced.

In this embodiment, in combination with the methods shown in Figures 6a and 6b above, the point cloud data collected around the vehicle only includes point cloud data within a set range around the vehicle, which can cover the useful information range while shielding the road. The building information on both sides can avoid the disclosure of sensitive building information, further reduce the amount of point cloud data, and improve the efficiency of point cloud data transmission and playback.

FIG. 7a is a schematic diagram of original point cloud data provided by an embodiment of the present application. As shown in Figure 7a, it is the point cloud data of the actual environment collected by the radar. The white points in the figure are point clouds. It can be seen that the number of point clouds in the point cloud data in the figure is relatively large.

FIG. 7b is a schematic diagram of processed point cloud data according to an embodiment of the present application. As shown in FIG. 7b, in order to use the point cloud data processing method provided by the above embodiments of the present application, the point cloud data obtained after processing the original point cloud data shown in FIG. 7a, the white points in the figure are point clouds.

Comparing Fig. 7a and Fig. 7b, it can be seen that after the point cloud data with a large amount of data is processed by the point cloud processing method provided in the embodiment of the present application, the number of point clouds is reduced to about one tenth of the original. Therefore, the solution of the present application It can effectively reduce the number of point clouds and realize point cloud compression; at the same time, compared with the original point cloud shown in Figure 7a, the processed point cloud shown in Fig. The differences are relatively small, so the point cloud processing method provided in the embodiment of the present application damages the shape features of the point cloud very little, and can ensure a better point cloud compression effect.

Based on the above embodiments and the same concept, an embodiment of the present application further provides a data processing apparatus. As shown in FIG. 8 , the data processing apparatus 800 may include: an acquisition unit 801 and a processing unit 802 .

The acquiring unit 801 is used for acquiring initial point cloud data.

The processing unit 802 is configured to determine a target area of the initial point cloud data, wherein the target area includes each point cloud in the initial point cloud data.

The processing unit 802 is further configured to perform grid division on the target area to obtain a plurality of grids, wherein the feature data of the point cloud in each grid satisfies the feature distribution condition, and the feature data is used to represent the point cloud. The spatial geometric relationship with the neighboring points of the point cloud; select the target point cloud of each grid from the point clouds contained in each grid, and delete the other than the target point cloud in each grid Point cloud: According to the target point cloud of each grid, the target point cloud data is obtained.

In a possible design, the processing unit 802 divides the target area into grids, including: dividing the target area into multiple grids according to a set grid size; The grids that meet the conditions are not divided; perform the following steps for each grid that satisfies the dividing conditions: divide the target grid into multiple grids, and the target grid is each grid that meets the dividing conditions; wherein , and the division condition is that the feature data of the point cloud in the grid does not satisfy the feature distribution condition.

In a possible design, the processing unit 802 determines whether the feature data of point clouds in the grid satisfies the feature distribution condition according to the following manner: calculating the first target parameter according to the feature data of all point clouds in the grid ; If it is determined that the first target parameter is not greater than the set threshold, then it is determined that the feature data of the point cloud in the grid satisfies the feature distribution condition, otherwise, it is determined that the feature data of the point cloud in the grid does not meet the requirements Describe the characteristic distribution conditions.

In a possible design, after obtaining the target point cloud data according to the target point cloud of each grid, the processing unit 802 is further configured to: adopt a random sampling consensus algorithm to analyze the data contained in the target point cloud data Perform plane fitting on the point cloud to obtain at least one fitting plane; in the at least one fitting plane, determine the first target fitting plane whose distance from the target plane in the target coordinate system is lower than the set distance value; If there is a second target fitting plane that satisfies the setting condition in the at least one fitting plane, then perform voxel filtering on the point cloud data included in the second target fitting plane; wherein, the setting condition includes: vertical In the first target fitting plane, the length along the target direction is greater than the set length value, and the number of point clouds in the plane is greater than the set value.

In a possible design, before determining the target area of the initial point cloud data, the processing unit 802 is further configured to: filter the initial point cloud data to remove the distances in the initial point cloud data group point.

In a possible design, after selecting the target point cloud of each grid from the point clouds contained in each grid, the processing unit 802 is further configured to: The target parameters are adjusted to the average of the second target parameters of all point clouds contained in the grid.

As an implementation manner, the data processing apparatus 800 may further include a storage unit 803 for storing program codes and data of the data processing apparatus 800 . The processing unit 802 may be a processor or a controller, for example, a general-purpose central processing unit (CPU), a general-purpose processor, a digital signal processing (DSP), an application-specific integrated circuit (application). specific integrated circuits, ASIC), field programmable gate array (FPGA), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It may implement or execute the various exemplary logical blocks, modules, etc. described in connection with this disclosure. The processor may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, and the like. The storage unit 803 may be a memory. The acquiring unit 801 may be an interface circuit of the data processing device, and is used for receiving data from other devices, for example, receiving initial point cloud data sent by a point cloud data acquisition device. When the data processing device is implemented in the form of a chip, the transceiver unit 801 may be an interface circuit used by the chip to receive data from or send data to other chips or devices.

The division of units in the embodiments of the present application is schematic, and is only a logical function division. In actual implementation, there may be other division methods. In addition, the functional units in the various embodiments of the present application may be integrated into one processing unit. In the device, it can also exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated units may be implemented in the form of hardware, or may be implemented in the form of software functional units.

Only one or more of the various elements in FIG. 8 may be implemented in software, hardware, firmware, or a combination thereof. The software or firmware includes, but is not limited to, computer program instructions or code, and can be executed by a hardware processor. The hardware includes, but is not limited to, various types of integrated circuits, such as a central processing unit (CPU), a digital signal processor (DSP), a field programmable gate array (FPGA), or an application specific integrated circuit (ASIC).

Based on the above embodiments and the same concept, the embodiments of the present application further provide a data processing apparatus for implementing the point cloud data processing method provided by the embodiments of the present application. As shown in FIG. 9, the data processing apparatus 900 may include: one or more processors 901, a memory 902, and one or more computer programs (not shown in the figure). As an implementation manner, the above devices may be coupled through one or more communication lines 903 . One or more computer programs are stored in the memory 902, and the one or more computer programs include instructions; the processor 901 invokes the instructions stored in the memory 902, so that the data processing apparatus 900 executes the instructions provided by the embodiments of the present application. Point cloud data processing methods.

In this embodiment of the present application, the processor may be a general-purpose processor, a digital signal processor, an application-specific integrated circuit, a field programmable gate array or other programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component, which can implement or The methods, steps and logic block diagrams disclosed in the embodiments of this application are executed. A general purpose processor may be a microprocessor or any conventional processor or the like. The steps of the methods disclosed in conjunction with the embodiments of the present application may be directly embodied as executed by a hardware processor, or executed by a combination of hardware and software modules in the processor.

In the embodiments of the present application, the memory may be a volatile memory or a nonvolatile memory, or may include both volatile and nonvolatile memory. The non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically programmable Erase programmable read-only memory (electrically EPROM, EEPROM) or flash memory. Volatile memory may be random access memory (RAM), which acts as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous DRAM, SDRAM), double data rate synchronous dynamic random access memory (double data rate SDRAM, DDR SDRAM), enhanced synchronous dynamic random access memory (enhanced SDRAM, ESDRAM), synchronous link dynamic random access memory (synchlink DRAM, SLDRAM) ) and direct memory bus random access memory (direct rambus RAM, DR RAM). It should be noted that the memory of the systems and methods described herein is intended to include, but not be limited to, these and any other suitable types of memory. The memory in this embodiment of the present application may also be a circuit or any other device capable of implementing a storage function.

As an implementation manner, the data processing apparatus 900 may further include a communication interface 904 for communicating with other apparatuses through a transmission medium. For example, when the apparatus for collecting initial point cloud data is not the data processing apparatus 900, the The data processing apparatus 900 may communicate with the apparatus for collecting initial point cloud data through the communication interface 904, so as to receive the initial point cloud data collected by the apparatus. In this embodiment of the present application, the communication interface may be a transceiver, a circuit, a bus, a module, or other types of communication interfaces. In this embodiment of the present application, when the communication interface is a transceiver, the transceiver may include an independent receiver and an independent transmitter; it may also be a transceiver integrating a transceiver function, or an interface circuit.

In some embodiments of the present application, the processor 901, the memory 902 and the communication interface 904 may be connected to each other through a communication line 903; the communication line 903 may be a Peripheral Component Interconnect (PCI for short) bus or an extension Industry standard structure (Extended Industry Standard Architecture, referred to as EISA) bus and so on. The communication line 903 can be divided into an address bus, a data bus, a control bus, and the like. For ease of presentation, only one thick line is used in FIG. 9, but it does not mean that there is only one bus or one type of bus.

The methods provided in the embodiments of the present application may be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer may be a general purpose computer, a special purpose computer, a computer network, network equipment, user equipment, or other programmable apparatus. The computer instructions may be stored in or transmitted from one computer readable storage medium to another computer readable storage medium, for example, the computer instructions may be downloaded from a website site, computer, server or data center Transmission to another website site, computer, server or data center by means of wired (such as coaxial cable, optical fiber, digital subscriber line, DSL for short) or wireless (such as infrared, wireless, microwave, etc.) A computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. that contains an integration of one or more available media. The available media can be magnetic media (eg, floppy disks, hard disks, magnetic tape), optical media (eg, digital video disc (DVD) for short), or semiconductor media (eg, SSD), and the like.

Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

As will be appreciated by those skilled in the art, the embodiments of the present application may be provided as a method, a system, or a computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein.

The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to the present application. It will be understood that each flow and/or block in the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to the processor of a general purpose computer, special purpose computer, embedded processor or other programmable data processing device to produce a machine such that the instructions executed by the processor of the computer or other programmable data processing device produce Means for implementing the functions specified in a flow or flow of a flowchart and/or a block or blocks of a block diagram.

These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory result in an article of manufacture comprising instruction means, the instructions The apparatus implements the functions specified in the flow or flow of the flowcharts and/or the block or blocks of the block diagrams.

These computer program instructions can also be loaded on a computer or other programmable data processing device to cause a series of operational steps to be performed on the computer or other programmable device to produce a computer-implemented process such that The instructions provide steps for implementing the functions specified in the flow or blocks of the flowcharts and/or the block or blocks of the block diagrams.

In the several embodiments provided in the present application, it should be understood that the disclosed apparatus and method may be implemented in other manners without exceeding the scope of the present application. For example, the above-described embodiments are only illustrative. For example, the division of the modules or units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components may be combined. Either it can be integrated into another system, or some features can be omitted, or not implemented. The unit described as a separate component may or may not be physically separated, and the component displayed as a unit may or may not be a physical unit, that is, it may be located in one place, or may be distributed to multiple network units . Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution in this embodiment. Those of ordinary skill in the art can understand and implement it without creative effort.

In addition, the described apparatus and methods, as well as the schematic diagrams of the various embodiments, may be combined or integrated with other systems, modules, techniques or methods without departing from the scope of this application. On the other hand, the shown or discussed mutual coupling or direct coupling or communication connection may be through some interfaces, indirect coupling or communication connection of devices or units, which may be in electronic, mechanical or other forms.

The above are only specific embodiments of the present application, but the protection scope of the present application is not limited to this. should be covered within the scope of protection of this application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

A method for processing point cloud data, comprising:

acquiring initial point cloud data, and determining a target area of the initial point cloud data, wherein the target area includes each point cloud in the initial point cloud data;

Perform grid division on the target area to obtain a plurality of grids, wherein the feature data of the point cloud in each grid satisfies the feature distribution condition, and the feature data is used to represent the point cloud and the neighborhood of the point cloud Spatial geometric relationship between points;

Select the target point cloud of each grid in the point cloud contained in each grid, and delete other point clouds except the target point cloud in each grid;

According to the target point cloud of each grid, the target point cloud data is obtained.
The method according to claim 1, wherein the performing grid division on the target area comprises:

Divide the target area into a plurality of grids according to the set grid size;

For each grid that does not meet the division conditions, no division processing is performed;

Perform the following steps on each grid that satisfies the division conditions: divide the target grid into a plurality of grids, and the target grid is each grid that meets the dividing conditions;

Wherein, the division condition is that the feature data of the point cloud in the grid does not satisfy the feature distribution condition.
The method according to claim 1 or 2, characterized in that whether the feature data of the point cloud in the grid satisfies the feature distribution condition is determined according to the following manner:

Calculate the first target parameter according to the feature data of all point clouds in the grid;

If it is determined that the first target parameter is not greater than the set threshold, it is determined that the feature data of the point cloud in the grid satisfies the feature distribution condition; otherwise, it is determined that the feature data of the point cloud in the grid does not satisfy the feature distribution condition. Feature distribution conditions.
The method according to any one of claims 1 to 3, wherein after obtaining the target point cloud data according to the target point cloud of each grid, the method further comprises:

A random sampling consensus algorithm is used to perform plane fitting on the point cloud included in the target point cloud data to obtain at least one fitted plane;

In the at least one fitting plane, determine a first target fitting plane whose distance from the target plane in the target coordinate system is lower than the set distance value;

If there is a second target fitting plane that satisfies the set condition in the at least one fitting plane, performing voxel filtering on the point cloud data included in the second target fitting plane;

The setting conditions include: perpendicular to the first target fitting plane, the length along the target direction is greater than the set length value, and the number of point clouds in the plane is greater than the set value.
The method according to any one of claims 1 to 4, wherein before determining the target area of the initial point cloud data, the method further comprises:

Filtering the initial point cloud data to remove outliers in the initial point cloud data.
The method according to any one of claims 1 to 5, wherein after selecting the target point cloud of each grid from the point clouds contained in each grid, the method further comprises:

The second target parameter of the target point cloud in each grid is adjusted to the average value of the second target parameter of all point clouds contained in the grid.
The method according to any one of claims 1 to 6, wherein the target point cloud of each grid is the centroid point in the point cloud included in each grid.
A data processing device, characterized in that it includes an acquisition unit and a processing unit;

The acquisition unit is used to acquire initial point cloud data;

The processing unit is configured to determine a target area of the initial point cloud data, wherein the target area includes each point cloud in the initial point cloud data;

The processing unit is further configured to perform grid division on the target area to obtain a plurality of grids, wherein the feature data of the point cloud in each grid satisfies the feature distribution condition, and the feature data is used to represent the difference between the point cloud and the grid. The spatial geometric relationship between the neighboring points of the point cloud; select the target point cloud of each grid from the point clouds contained in each grid, and delete other points in each grid except the target point cloud Cloud; according to the target point cloud of each grid, get the target point cloud data.
The apparatus according to claim 8, wherein the processing unit performs grid division on the target area, comprising:

Divide the target area into a plurality of grids according to the set grid size;

For each grid that does not meet the division conditions, no division processing is performed;

Perform the following steps on each grid that satisfies the division conditions: divide the target grid into a plurality of grids, and the target grid is each grid that meets the dividing conditions;

Wherein, the division condition is that the feature data of the point cloud in the grid does not satisfy the feature distribution condition.
The device according to claim 8 or 9, wherein the processing unit determines whether the feature data of the point cloud in the grid satisfies the feature distribution condition according to the following manner:

Calculate the first target parameter according to the feature data of all point clouds in the grid;

If it is determined that the first target parameter is not greater than the set threshold, it is determined that the feature data of the point cloud in the grid satisfies the feature distribution condition; otherwise, it is determined that the feature data of the point cloud in the grid does not satisfy the feature distribution condition. Feature distribution conditions.
The device according to any one of claims 8 to 10, wherein after obtaining the target point cloud data according to the target point cloud of each grid, the processing unit is further configured to:

A random sampling consensus algorithm is used to perform plane fitting on the point cloud included in the target point cloud data to obtain at least one fitted plane;

In the at least one fitting plane, determine a first target fitting plane whose distance from the target plane in the target coordinate system is lower than the set distance value;

If there is a second target fitting plane that satisfies the set condition in the at least one fitting plane, performing voxel filtering on the point cloud data included in the second target fitting plane;

The setting conditions include: perpendicular to the first target fitting plane, the length along the target direction is greater than the set length value, and the number of point clouds in the plane is greater than the set value.
The apparatus according to any one of claims 8 to 11, wherein before determining the target area of the initial point cloud data, the processing unit is further configured to:

Filtering the initial point cloud data to remove outliers in the initial point cloud data.
The apparatus according to any one of claims 8 to 12, wherein after selecting the target point cloud of each grid from the point clouds included in each grid, the processing unit is further configured to:

The second target parameter of the target point cloud in each grid is adjusted to the average value of the second target parameter of all point clouds contained in the grid.
The device according to any one of claims 8 to 13, wherein the target point cloud of each grid is a centroid point in the point cloud included in each grid.
A data processing device, comprising a memory and a processor;

the memory is used to store computer programs;

The processor is configured to execute the calculation program stored in the memory to implement the method according to any one of claims 1-7.
A data processing device, comprising at least one processor and an interface;

the interface is used to provide program instructions or data for the at least one processor;

The at least one processor is configured to execute the program instructions, implementing the method of any of claims 1-7.
A computer-readable storage medium, characterized in that, the computer-readable storage medium stores a computer program, and when the computer program runs on a data processing device, the data processing device is made to execute the above claims 1- The method of any one of 7.