WO2022017147A1 - Point cloud data processing method and apparatus, radar apparatus, electronic device, and computer readable storage medium - Google Patents

Abstract

The present invention provides a point cloud data processing method and apparatus, a radar apparatus, an electronic device, and a computer readable storage medium. The method comprises: obtaining point cloud data corresponding to a target scene; rasterizing the obtained point cloud data to obtain point cloud datasets respectively corresponding to multiple rasters, the point cloud dataset in each raster being sequentially stored in a storage medium, and each of the point cloud datasets comprising data of at least one point; and determining, according to the point cloud dataset of each raster, point cloud feature information corresponding to the raster. According to the present invention, point cloud feature extraction is divided into a point cloud rasterization process and a raster feature extraction process, and in this way, the overall point cloud feature extraction is performed on the point cloud data set of each raster by taking a raster as a unit, and thus the efficiency of point cloud feature extraction is improved.

Description

Method and device for processing point cloud data, radar device, electronic device and computer-readable storage medium

cross reference statement

The present invention claims the priority of the Chinese Patent Application No. 202010713994.7 filed with the Chinese Patent Office on July 22, 2020, the entire contents of which are incorporated herein by reference.

technical field

The present disclosure relates to the technical field of point cloud data processing, and in particular, to a point cloud data processing method, a point cloud data processing device, a radar device, an electronic device, and a computer-readable storage medium.

Background technique

With the continuous development of lidar technology, since the point cloud data collected by lidar includes the accurate position information of the target object, the application of lidar to collect point cloud data is widely used in various fields, such as target detection, 3D target reconstruction, automatic driving Wait. Before the above application, it is usually necessary to perform feature extraction on the collected point cloud data.

However, since the amount of point cloud data generated by lidar is large and updated in real time, if the point cloud data collected is directly extracted, the feature extraction speed will be slow.

SUMMARY OF THE INVENTION

The embodiments of the present disclosure provide at least one solution for processing point cloud data, which divides point cloud feature extraction into two stages of grid processing and grid feature extraction in combination with a data reading and writing mechanism, which improves the efficiency of point cloud feature extraction.

It mainly includes the following aspects:

In a first aspect, an embodiment of the present disclosure provides a method for processing point cloud data, the method comprising:

Obtain the point cloud data corresponding to the target scene;

Perform grid processing on the acquired point cloud data to obtain point cloud datasets corresponding to multiple grids; the point cloud datasets in each grid are sequentially stored in the storage medium; each of the point cloud datasets The cloud dataset contains data of at least one point;

According to the point cloud dataset of each grid, point cloud feature information corresponding to the grid is determined.

By using the above processing method for point cloud data, it can first obtain the point cloud data corresponding to the target scene, and then perform grid processing on the acquired point cloud data, that is, determine the grid corresponding to the point cloud data, and convert the grid to the grid. The gridded point cloud data is sequentially stored in the storage medium in grid units. In this way, before the feature extraction is performed for each grid, the point cloud dataset of each grid can be sequentially read from the storage medium, and then the corresponding point cloud dataset corresponding to each grid can be determined according to each read point cloud dataset. point cloud feature information. The above extraction method divides point cloud feature extraction into a point cloud rasterization process and a grid feature extraction process. In this way, by taking the grid as a unit, the overall point cloud feature extraction is performed on the point cloud dataset of each grid to improve the performance of the grid. The efficiency of point cloud feature extraction is improved.

In one embodiment, the rasterizing processing on the acquired point cloud data includes:

Based on the conversion relationship between the radar coordinate system and the Cartesian coordinate system, the acquired radar coordinate information of each point in the point cloud data is respectively converted into corresponding Cartesian coordinate information;

Determine the coordinate range of all grids according to the set radar detection range and preset grid size;

The points in the point cloud data are divided into corresponding grids according to their respective Cartesian coordinate information.

In one embodiment, the dividing the points in the point cloud data into corresponding grids according to their respective Cartesian coordinate information, including:

For each point in the point cloud data, determine the grid to be divided by the point based on the Cartesian coordinate information of the point and the coordinate range of all grids;

If the number of grids that have divided points is less than the first preset number, then determine whether the number of divided points in the grid to be divided by the point is less than the second preset number;

If it is less than the second preset number, the points are divided into grids to which the points are to be divided.

In an embodiment, according to the point cloud dataset of each grid, determining the point cloud feature information corresponding to the grid includes:

For the point cloud data set of each grid, based on the coordinate information of each point in the point cloud data set, feature enhancement is performed on the point, and the enhanced feature information of the point is determined; the enhanced feature information has the first feature dimension;

Based on the enhanced feature information of each point in the point cloud dataset, the encoded feature information of the point cloud dataset is determined; the encoded feature information has a second feature dimension; the number of dimensions of the second feature dimension is greater than the The number of dimensions of the first feature dimension;

Based on the encoded feature information of the point cloud dataset, point cloud feature information corresponding to the grid is determined.

Here, in order to achieve grid feature extraction, the embodiment of the present disclosure may first perform feature enhancement on each point in the point cloud dataset based on the coordinate information of the point, and obtain the encoded feature information of the feature-enhanced point cloud dataset, which is equivalent to The features of multiple points in a grid are fused.

In an embodiment, the feature enhancement is performed on the point based on the coordinate information of each point in the point cloud dataset, and the enhanced feature information of the point is determined, including:

Determine the center coordinate information of the point cloud dataset based on the coordinate information of each point in the point cloud dataset;

For each of the points, first deviation information corresponding to the point is determined based on the coordinate information of the point and the determined center coordinate information; and based on the coordinate information of the point and the point Coordinate information of other points other than the point in the cloud data set, determining second deviation information between the point and the other points;

The coordinate information of the point, the first deviation information, and the second deviation information are used as enhanced feature information of the point.

In one embodiment, the encoded feature information of the point cloud dataset is determined based on the enhanced feature information of each point in the point cloud dataset by using a convolutional neural network, wherein the convolutional neural network includes a convolution layer and activation layers, including:

Use the convolution check of the convolution layer to perform convolution operations on the enhanced feature information of each point in the point cloud data set, to obtain convolution feature information corresponding to each point in the point cloud data set;

Combining the convolution feature information corresponding to each point to obtain the convolution feature information corresponding to the point cloud data set;

The convolution feature information corresponding to the point cloud data set is processed by using the activation function of the activation layer to obtain the encoded feature information of the point cloud data set.

In an embodiment, the activation function of the activation layer is used to process the convolution feature information corresponding to the point cloud dataset to obtain the encoded feature information of the point cloud dataset, including:

Perform quantization processing on the convolution feature information corresponding to the point cloud data set according to the first preset bit width to obtain the quantized convolution feature information; the first preset bit width is the target detection network deployed to the running The quantized bit width of the platform, the target detection network is used to detect the target object according to the point cloud feature information corresponding to each grid;

The quantized convolution feature information is processed by using the activation function of the activation layer to obtain the encoded feature information of the point cloud data set.

Here, in order to adapt to the subsequent operation of the target detection network, after the convolution feature information corresponding to the point cloud dataset is determined, the above-mentioned convolution feature information can be quantized based on the first preset bit width of the target detection network to obtain Feature encoding is performed using the quantized convolutional feature information.

In an implementation manner, using the coordinate information, the first deviation information, and the second deviation information of the point as the enhanced feature information of the point includes:

The coordinate information, the first deviation information and the second deviation information of the point are respectively quantized according to the second preset bit width to obtain the quantized coordinate information, the first deviation information and the second deviation information. deviation information; the second preset bit width is the bit width quantified when the convolutional neural network is deployed to the running platform; the convolutional neural network determines the The encoded feature information of the point cloud dataset;

The quantized coordinate information, the first deviation information, and the second deviation information are used as the enhanced feature information of the point.

Here, in order to adapt to the convolutional neural network used for feature encoding, after determining the first deviation information corresponding to each point and the enhanced feature information such as the second deviation information, the second preset bit width of the convolutional neural network can be based on Quantize the enhanced feature information to determine the enhanced feature information after quantization.

In one embodiment, after determining the point cloud feature information corresponding to the grid, the method further includes:

A target object is detected based on the point cloud feature information corresponding to each grid.

Here, considering that the point cloud feature information corresponding to each grid presents the spatial distribution of the relevant target objects, the point cloud feature information extracted by the grid feature extraction process can be used as the input information of the target detection network to carry out the target detection. In the detection of object information, under the premise of faster point cloud feature extraction, the efficiency of target detection can also be improved.

In one embodiment, the detecting a target object based on the point cloud feature information corresponding to each grid includes:

The point cloud feature information of the multiple grids under the same feature dimension is used as a set of input information, and different sets of input information corresponding to different feature dimensions are input into the target detection network in parallel to obtain the target in the target scene. object.

In order to better extract the target object information, before inputting the point cloud feature information to the target detection network, the point cloud feature information of the grid can be decomposed in the feature dimension, and the decomposed point cloud feature information can present different features The influence of dimension on the target object detection result ensures the detection accuracy.

In one embodiment, detecting a target object based on the point cloud feature information corresponding to each grid includes:

For each grid, based on the coordinate information of each point in the point cloud dataset of the grid, determine the coordinate information of the target point representing the grid position;

Taking the point cloud feature information corresponding to the grid as the point cloud feature information corresponding to the target point;

The coordinate information of the target points representing the multiple grid positions and the corresponding point cloud feature information are input into the target detection network to obtain the target object in the target scene.

Here, considering the correspondence between grids and points, the point cloud feature information of the grid can be mapped to a target point, and the target point can be used to represent the grid for target object detection, thereby improving detection efficiency.

In a second aspect, an embodiment of the present disclosure further provides a radar device, including: a radar controller, a radar component, and a field programmable gate array FPGA; wherein the radar controller is connected to the radar component and the FPGA circuit respectively. connect;

the radar controller, configured to control the radar component to scan the target scene and obtain point cloud data corresponding to the target scene;

The FPGA includes an on-chip processor, and the on-chip processor is used to perform grid processing on the point cloud data to obtain point cloud data sets corresponding to multiple grids; A point cloud dataset, which determines the point cloud feature information corresponding to the grid; the point cloud dataset in each grid is sequentially stored in the storage medium; each of the point cloud datasets contains data of at least one point .

In one embodiment, the FPGA further includes an on-chip logic circuit;

The on-chip logic circuit is used for detecting a target object from the point cloud feature information corresponding to each grid.

In a third aspect, an embodiment of the present disclosure further provides an apparatus for processing point cloud data, the apparatus comprising:

The acquisition module is used to acquire the point cloud data corresponding to the target scene;

a processing module, configured to perform grid processing on the acquired point cloud data to obtain point cloud datasets corresponding to multiple grids; the point cloud datasets in each grid are sequentially stored in the storage medium; Each of the point cloud datasets contains data of at least one point;

A determination module, configured to determine the point cloud feature information corresponding to the grid according to the point cloud data set of each grid.

In a fourth aspect, embodiments of the present disclosure further provide an electronic device, including: a processor, a memory, and a bus; the memory stores machine-readable instructions executable by the processor; when the electronic device runs, Communication between the processor and the memory is through the bus; when the machine-readable instructions are executed by the processor, the processing of the point cloud data according to any one of the first aspect and its various embodiments is performed. steps of the method.

In a fifth aspect, an embodiment of the present disclosure further provides a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and the computer program is executed by a processor when the first aspect and various aspects thereof are executed. The steps of the method for processing point cloud data described in any one of the embodiments.

For a description of the effects of the above-mentioned point cloud data processing apparatus, electronic device, and computer-readable storage medium, reference may be made to the above-mentioned description of the point cloud data processing method, which will not be repeated here.

In order to make the above-mentioned objects, features and advantages of the present disclosure more obvious and easy to understand, the preferred embodiments are exemplified below, and are described in detail as follows in conjunction with the accompanying drawings.

Description of drawings

In order to explain the technical solutions of the embodiments of the present disclosure more clearly, the following briefly introduces the accompanying drawings required in the embodiments, which are incorporated into the specification and constitute a part of the specification. The drawings illustrate embodiments consistent with the present disclosure, and together with the description serve to explain the technical solutions of the present disclosure. It should be understood that the following drawings only show some embodiments of the present disclosure, and therefore should not be regarded as limiting the scope. Other related figures are obtained from these figures.

1 shows a flowchart of a method for processing point cloud data provided by Embodiment 1 of the present disclosure;

FIG. 2 shows a specific example diagram of a rasterization process in the method for processing point cloud data provided by Embodiment 1 of the present disclosure;

3 shows a specific example diagram of a feature enhancement process in the method for processing point cloud data provided by Embodiment 1 of the present disclosure;

Figure 4(a) shows a specific example diagram before feature encoding in the method for processing point cloud data provided in Embodiment 1 of the present disclosure;

FIG. 4(b) shows a specific example diagram after feature encoding in the method for processing point cloud data provided by Embodiment 1 of the present disclosure;

FIG. 5 shows a schematic diagram of an apparatus for processing point cloud data according to Embodiment 2 of the present disclosure;

FIG. 6 shows a schematic diagram of an electronic device according to Embodiment 3 of the present disclosure.

detailed description

In order to make the purposes, technical solutions and advantages of the embodiments of the present disclosure more clear, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only These are some, but not all, embodiments of the present disclosure. The components of the disclosed embodiments generally described and illustrated in the drawings herein may be arranged and designed in a variety of different configurations. Therefore, the following detailed description of the embodiments of the disclosure provided in the accompanying drawings is not intended to limit the scope of the disclosure as claimed, but is merely representative of selected embodiments of the disclosure. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without creative work fall within the protection scope of the present disclosure.

It has been found through research that in related technologies, since the amount of point cloud data generated by lidar is large and updated in real time, if the feature extraction is performed directly on the collected point cloud data, the feature extraction speed will be slow.

Based on the above research, the present disclosure provides at least one point cloud data processing solution, which combines the data reading and writing mechanism to divide point cloud feature extraction into two stages: rasterization processing and grid feature extraction, which improves the performance of point cloud feature extraction. efficient.

The defects existing in the above solutions are all the results obtained by the inventor after practice and careful research. Therefore, the discovery process of the above problems and the solutions to the above problems proposed by the present disclosure hereinafter should be the inventors Contributions made to this disclosure during the course of this disclosure.

It should be noted that like numerals and letters refer to like items in the following figures, so once an item is defined in one figure, it does not require further definition and explanation in subsequent figures.

In order to facilitate the understanding of this embodiment, a method for processing point cloud data disclosed in the embodiment of the present disclosure is first introduced in detail. The execution subject of the method for processing point cloud data provided by the embodiment of the present disclosure generally has a certain computing capability. electronic equipment, the electronic equipment includes, for example: terminal equipment or server or other processing chips, processing chips such as Microcontroller Unit (MCU), System-on-a-Chip (SoC), application-specific integrated circuit ( Application Specific Integrated Circuit, ASIC), field-programmable gate array (FPGA, field-programmable gate array), terminal equipment such as lidar equipment, user equipment (User Equipment, UE), mobile equipment, user terminals (such as smart phones), Terminals, handheld devices, computing devices, vehicle-mounted devices, wearable devices, etc. In some possible implementations, the method for processing point cloud data may be implemented by a processor invoking computer-readable instructions stored in a memory.

The processing method of the point cloud data provided by the embodiments of the present disclosure will be described below.

Example 1

Referring to FIG. 1 , which is a flowchart of a method for processing point cloud data provided by an embodiment of the present disclosure, the method includes steps S101 to S103 .

S101. Acquire point cloud data corresponding to a target scene.

S102. Perform grid processing on the acquired point cloud data to obtain point cloud data sets corresponding to multiple grids; the point cloud data sets in each grid are sequentially stored in a storage medium; each point cloud data set Data that contains at least one point in the set.

S103. According to the point cloud data set of each grid, determine the point cloud feature information corresponding to the grid.

Here, in order to facilitate understanding of the method for processing point cloud data provided by the embodiments of the present disclosure, a specific application scenario of the extraction method is first described in detail below. The method for processing point cloud data provided by the embodiments of the present disclosure can be mainly applied to the fields of target object detection, three-dimensional target reconstruction, and the like. Here, the target object detection is taken as an example for illustration. In the related art, in order to determine the location and other information related to the target object, after acquiring the data information (eg point cloud data) related to the application scene, the target object detection can be realized based on a pre-trained target detection network. Here, considering that in the process of relying on the target detection network for target object detection, due to the large amount of point cloud data collected by lidar and updated in real time, this will lead to slower feature extraction and slower feature extraction. The speed directly leads to the low efficiency of target object detection, especially in applications such as automatic driving that require high real-time target object detection, the low target object detection efficiency directly affects driving safety.

It is to solve the above problem that the embodiments of the present disclosure provide a point cloud data processing method that divides point cloud feature extraction into a point cloud rasterization process and a raster feature extraction process based on different data reading and writing mechanisms. The previous process reads the point cloud data for rasterization, and writes the rasterized point data sequentially in raster units, that is, the point cloud data in the same raster can be written in the storage. A contiguous address range of the medium, thus facilitating the sequential reading of the point cloud dataset for each raster in the latter process. The two processes cooperate with each other to ensure the rapid realization of point cloud feature extraction. In this way, when the above extraction method is applied to target object detection and other fields, the efficient realization of the application can be ensured.

The point cloud rasterization process in the embodiment of the present disclosure is the process of performing rasterization processing on the point cloud data corresponding to the target scene, and the raster feature extraction process may be the feature extraction process on the point cloud data set of the raster. the process of.

Here, considering that in practical applications, the amount of raw point cloud data collected by lidar is usually large, and it is usually stored in off-chip memory, such as Double Data Rate Synchronous Dynamic Random Access Memory, DDR SDRAM) in DDR3\DDR4, and from the perspective of processing efficiency, the point cloud rasterization and grid feature extraction in the point cloud data processing method provided by the embodiment of the present disclosure may depend on the on-chip memory. Therefore, in the process of rasterization, firstly, according to the reading mechanism of the first storage medium (corresponding to the off-chip memory), multiple points corresponding to the point cloud data can be read, and then the corresponding points of the point cloud data can be read for each The point determines the grid to which it is mapped, so that the point can be written to the storage address corresponding to the second storage medium (corresponding to the on-chip memory) according to the grid information.

It should be noted that, in the process of writing each point into the second storage medium, the coordinates of each point falling into a grid can be sorted first, for example, each point is sorted in ascending order of horizontal coordinate value. Sorting is performed so that sequential writes can be performed according to the sorted results.

Wherein, the process of writing the data at the point involves out-of-order access to the storage medium. This is mainly to consider that after reading the data of a point, it is necessary to first determine the grid to which the point belongs. Here, before writing the above point into the storage medium corresponding to the grid, the grid to which the point belongs out-of-order access to other points. After the sorting operation, the sorted points can be sequentially written to the storage medium, thereby facilitating the sequential reading of the data in the grid in the subsequent grid feature extraction stage.

In this embodiment of the present disclosure, the initial feature of a point may be its coordinate information, and the coordinate information may include a horizontal coordinate value, a vertical coordinate value, a height coordinate value, and a depth coordinate value, which can be corresponding to {X, Y, Z, I}. Representing the above coordinate values (as shown in the example of point P1 shown in FIG. 2 ), it can be seen that the feature dimension of the initial feature of a point is 4 dimensions. For example, in the case of determining a total of 16 points in a grid, the sorted 16 points {P1, P2..., P16} can be sequentially written to the storage address where a grid is located, as shown in Figure 2, No specific example is given for information such as the number of points written in other grids.

In addition, in the process of extracting grid features, it may be a process of uniformly analyzing the point cloud dataset corresponding to each grid. In specific applications, the point cloud dataset of each grid can be read first. Then, feature enhancement is performed on each point in the point cloud dataset corresponding to the grid, where the feature dimension of the point before feature enhancement is smaller than the feature dimension of the point after feature enhancement, that is, the feature information of each point in the grid is highlighted. Finally, the feature encoding of the corresponding point cloud dataset can be implemented based on the enhanced feature information of each point, thereby realizing the feature extraction of the grid.

It should be noted that, based on the above-mentioned sequential writing operation of points, before the grid feature extraction is performed, the related data of the relevant points can be sequentially read and written.

Here, corresponding interfaces can be set for the above point cloud rasterization processing operation and raster feature extraction operation respectively. For example, an input interface for the point cloud rasterization operation can be set, and the point cloud data in the storage medium can be randomly accessed using the input interface. For another example, an interface between the point cloud rasterization operation and the raster feature extraction operation can be set, and the interface can be used to realize sequential access to the rasterized point cloud data stored in the storage medium.

The above-mentioned storage medium may be an on-chip memory or an off-chip memory. When the on-chip cache cannot meet the access requirements, off-chip memory can be used. On-chip memory can be directly used on chips with sufficient on-chip cache resources, without the need to read and write off-chip memory, thereby reducing latency and improving real-time access.

Here, point cloud rasterization and grid feature extraction are key steps of the point cloud data processing method provided by the embodiments of the present disclosure, which will be described separately below.

First, the process of point cloud rasterization can be described in detail.

The process of point cloud rasterization in the embodiment of the present disclosure may depend on the conversion relationship between the radar coordinate system and the Cartesian coordinate system, and may be specifically implemented by the following steps:

Step 1: Based on the conversion relationship between the radar coordinate system and the Cartesian coordinate system, the radar coordinate information of each point in the acquired point cloud data is converted into the corresponding Cartesian coordinate information respectively;

Step 2: Determine the coordinate range of all grids according to the set radar detection range and the preset grid size;

Step 3: Divide the points in the point cloud data into corresponding grids according to their respective Cartesian coordinate information.

The Cartesian coordinate system here may be established based on the detection plane (eg, the ground), and the origin of the established Cartesian coordinate system may be coincident with the origin of the radar coordinate system. In this way, when the radar coordinate information of each point in the point cloud data is projected onto the ground, the Cartesian coordinate information corresponding to the point can be determined. The Cartesian coordinate information can be the actual distance from the lidar device when the point is projected onto the detection plane. After dividing multiple grids based on the set radar detection range and preset grid size, the grid to which each point is divided can be determined, thereby realizing point cloud rasterization.

It should be noted that, when the set radar detection range is the same, the larger the preset grid size is, the less the number of divided grids, and the more points each grid falls into. Similarly, the smaller the preset grid size is, the more grids are divided, and the fewer points each grid falls into.

In the embodiment of the present disclosure, considering that the grid feature extraction process may be based on the grid-based point cloud dataset processing, the number of grids and the number of points included in the grid point cloud dataset will be Directly affects the speed of raster feature extraction. Here, in order to improve the efficiency of grid feature extraction as much as possible on the premise of ensuring the accuracy of feature extraction, the grid may be divided based on the limitation of the number of grids and the number of points in the grid. Specifically, it can be achieved by performing the following steps for each point in the point cloud data:

Step 1: Determine the grid to be divided at the point based on the Cartesian coordinate information of the point and the coordinate range of all grids;

Step 2: If the number of grids with divided points is less than the first preset number, then determine whether the number of divided points in the grid to be divided at the point is less than the second preset number;

Step 3: If the number is less than the second preset number, divide the point into the grid to be divided by the point.

Here, in order to facilitate the understanding of the above grid division process, the total number of grids is limited to 20 and the points in one grid are limited to 16 as an example for illustration.

For an acquired point, if it is determined based on the Cartesian coordinate information of the point and the coordinate range of all the grids, the grid to be divided for the point is the fifth grid. Here, if it is determined that the number of grids with divided points is 4, and the upper limit of 20 has not been reached, then the fifth grid will be used as the grid of the new divided points, and the above points will be used as the new divided grid. The first point in the grid of points. If it is determined that the number of grids with divided points is 5, and the upper limit of 20 has not been reached, then the fifth grid will be used as the grid of divided points, and the above points will be used as the grid of divided points. A non-first point within the grid. Whether to divide this point into the 5th grid may be determined based on the judgment result of whether the number of divided points in the 5th grid is less than or equal to 16. If the number of points in the 5th grid is already equal to 16, discard the point, otherwise divide the point into the 5th grid.

For an acquired point, it is determined that the grid to be divided for this point is the 21st grid based on the Cartesian coordinate information corresponding to the point and the coordinate range of all grids. Here, since it is determined that the number of grids with divided points is 20, the upper limit of 20 has been reached. At this point, the point is discarded, and the process of point cloud rasterization ends.

It can be seen that the embodiments of the present disclosure implement grid division based on the limitation of the number of grids and the number of points in the grid, thereby ensuring the efficiency of subsequent grid feature extraction.

Next, the grid feature extraction process provided by the embodiments of the present disclosure can be described in detail. The grid feature extraction in the embodiment of the present disclosure mainly includes two processes of feature enhancement for points and feature encoding for grids, and specifically includes performing the following steps for the point cloud dataset of each grid:

Step 1, based on the coordinate information of each point in the point cloud data set, perform feature enhancement on the point, and determine the enhanced feature information of the point; the enhanced feature information has a first feature dimension;

Step 2: Determine the encoded feature information of the point cloud dataset based on the enhanced feature information of each point in the point cloud dataset; the encoded feature information has a second feature dimension; the number of dimensions of the second feature dimension is greater than that of the first feature dimension. number of dimensions;

Step 3: Based on the encoded feature information of the point cloud dataset, determine the point cloud feature information corresponding to the grid.

Here, the process of feature enhancement for points may be a process of amplifying the features of each point in each point cloud data set, which may be implemented by the following steps:

Step 1: For the point cloud dataset of each grid, determine the center coordinate information of the point cloud dataset based on the coordinate information of each point in the point cloud dataset;

Step 2, for each point, based on the coordinate information of the point and the determined center coordinate information, determine the first deviation information corresponding to the point; and based on the coordinate information of the point and other than the point in the point cloud data set. Coordinate information of other points to determine the second deviation information between the point and other points;

Step 3: Use the coordinate information, the first deviation information and the second deviation information of each point as the enhanced feature information of the point.

Here, for the read point cloud data set of each grid, first, based on the coordinate information of each point in the point cloud data set, the center coordinate information corresponding to a point cloud data set may be determined. In a specific application, after summing the coordinate information of each point, the quotient can be obtained with the number of points contained in the point cloud data set to determine the center coordinate information. The center coordinate information can represent the centralized distribution trend of each point in the point cloud dataset to a certain extent.

In the case of determining the center coordinate information of a grid, a first deviation amount from the above-mentioned center coordinate can be determined for any point falling within the grid, and a second deviation of any point from other points in the grid can also be determined quantity.

Here, for any point, the first deviation information corresponding to the point can be determined by calculating the difference between the coordinate information of the point and the above-mentioned center coordinate information, and the first deviation information can represent the degree of dispersion of the data distribution in the point cloud data set to a certain extent. In addition, for any point, the difference calculation between the coordinate information of the point and the coordinate information of other points can also be performed to determine the second deviation information corresponding to the point. It can characterize the degree of dispersion of the data distribution in the point cloud dataset.

In a specific application, in addition to determining the enhanced feature information of a point based on the above-mentioned first deviation information and second deviation information, other enhanced feature information can also be determined by other processing of the coordinate information of each point. No specific restrictions are imposed.

Here, still taking the feature dimension of the initial feature of the point as 4 dimensions as an example, after feature enhancement, the corresponding enhanced feature information may be 9 dimensions (corresponding to the first feature dimension), that is, under the premise of the initial dimension Extended 5-dimensional features.

As shown in FIG. 3 , the feature enhancement operation is performed after the points of the 4-dimensional initial feature are input in the form of serial data stream, and a specific example of the 9-dimensional enhanced feature information is obtained. For each point included in a point cloud dataset in a grid, the coordinate information of each point may be read sequentially. In this way, in the process of feature enhancement, data enhancement processing may be performed on the sequentially read point information in a serial data stream manner.

In order to facilitate data enhancement, when feature enhancement is performed for any point, the grid index table generated in the rasterization process can be combined to realize the enhancement. Based on the grid index table, the related information of other points belonging to the same grid as the point can be determined, and then the above-mentioned related deviation information can be calculated to determine the enhanced feature information corresponding to any point. On the premise that the point information is connected to the data enhancement processing operation in a serial data stream mode, the output enhancement feature information for a point may also be output in a serial data stream mode.

Here, the process of feature encoding for the grid may be a process of fusing the features of each point in the point cloud dataset corresponding to a grid.

In the embodiment of the present disclosure, for the point cloud data set of each grid read, the enhanced feature information of each point in the point cloud data set can be used as the input information of the convolutional neural network, and the convolution operation can be used to realize the point cloud data set. Feature encoding for cloud datasets.

It should be noted that, in order to adapt to the second preset bit width (such as 4 bits) of the above-mentioned convolutional neural network, the coordinate information, first deviation information and second The deviation information is separately quantized, and the quantized coordinate information, the first deviation information, and the second deviation information are used as the enhanced feature information of the point.

Wherein, the above-mentioned convolutional neural network may include a convolutional layer and an activation layer. Based on the convolutional neural network, the feature encoding of the point cloud dataset of each grid can be realized by the following steps:

Step 1, using the convolution kernel of the convolution layer to perform a convolution operation on the enhanced feature information of each point in the point cloud data set, to obtain the convolution feature information corresponding to each point in the point cloud data set;

Step 2: Combine the convolution feature information corresponding to each point to obtain the convolution feature information corresponding to the point cloud data set;

Step 3: Use the activation function of the activation layer to process the convolution feature information corresponding to the point cloud data set to obtain the encoded feature information of the point cloud data set.

Here, for a point cloud dataset of a grid, the convolution feature information corresponding to each point in the point cloud dataset may be determined first. Here, still take the point cloud dataset in one grid including 16 points, and the dimension of the enhanced feature information of each point is 9 as an example, the convolution operation can be performed on the 16 points respectively, for example, 1x1 convolution can be used to perform the convolution operation. The convolution operation is performed to obtain the convolution feature information corresponding to each point.

The convolutional feature information corresponding to the point cloud dataset can be obtained by summing the 16 convolutional feature information. At this time, an activation function can be used to process the point cloud data set to obtain corresponding encoded feature information, for example, encoded feature information of 32 dimensions (corresponding to the second feature dimension) can be encoded.

Figures 4(a) to 4(b) are specific example diagrams of feature encoding for one grid in the 32 grids after gridding. As shown in Fig. 4(a), the three axes respectively indicate the number of grids, as shown by W=32, the number of points in the grid, as shown by H=16, the enhanced feature of each point after enhancement The dimension of the information (as indicated by the enhanced feature information of point P1), as indicated by C=9. In this way, after the encoding is implemented according to the above feature encoding method, W=32, H=1, and C=32 respectively indicated by the three axes can be determined, that is, each point included in the point cloud dataset corresponding to a grid is realized. The fusion of features corresponds to the encoded feature information.

In a specific application, the activation function of a linear rectification function (Rectified Linear Unit, ReLU) can be used for encoding processing. This function can encode the point cloud data set indicating that the convolution eigenvalue is less than or equal to zero as 0, and the point cloud data set indicating that the convolution eigenvalue is greater than zero corresponds to 1, and the obtained encoded feature information at the position of 1 can be It corresponds to the target object, and the position of 0 can correspond to the background, so as to facilitate the subsequent detection of the target object.

In a specific application, the above-mentioned convolutional neural network for implementing feature encoding can be implemented by using a Voxel Feature Encoding (VFE) network. The VFE network can use one-dimensional/streaming sparse convolution technology to implement feature encoding. Since this technology fits the sparse characteristics of lidar data itself, it can adapt to the data flow density, improve encoding efficiency, and further reduce processing. Delay and improve performance.

It should be noted that, in order to adapt to the first preset bit width (such as 8bit) of the target detection network, at this time, the convolution feature information of the above point cloud dataset can be quantified according to this bit width, and then the The quantized convolutional feature information is encoded to obtain encoded feature information.

In the embodiment of the present disclosure, after the point cloud feature information corresponding to each grid is determined according to the above method, the point cloud feature information can be used as the input information of the target detection network to realize the detection of the target object.

Wherein, based on different application scenarios, the determined target objects are also different. For example, for automatic driving, the above-mentioned target object may be a person in front of the self-driving car, or may be a vehicle in front of the car, which is not specifically limited in this embodiment of the present disclosure.

In specific applications, the training of the target detection network can be realized by using the convolutional neural network. In the case where the above-mentioned encoded feature information is input to the convolutional neural network as a sparse graph and participates in the convolution operation, the convolution operation can be performed only on the grid encoded as 1, thereby reducing the amount of convolution operation.

In the embodiment of the present disclosure, in order to adapt to the data receiving mode of the target detection network, feature disassembly may also be performed. Here, the point cloud feature information of each grid under the same feature dimension can be used as a set of input information, and different sets of input information corresponding to different feature dimensions can be input into the trained target detection network in parallel, and determined from the target scene. target.

For example, if the dimension of the point cloud feature information of the grid is determined to be 32 dimensions, the point cloud feature information of each grid under the same feature dimension can be used as a group of input information, and different feature dimensions will correspond to different groups of input information. information, corresponding to 32 groups of input information in total. The target object can be determined by inputting these 32 sets of input information into the target detection network.

In the embodiment of the present disclosure, in order to adapt to the data receiving method of the target detection network, the remapping of the grid scatter data to the bird's-eye view detection plane may also be performed, which specifically includes the following steps:

Step 1: Determine the coordinate information of the target point representing the position of the grid based on the coordinate information of each point in the point cloud data set of each grid;

Step 2, taking the point cloud feature information corresponding to the grid as the point cloud feature information corresponding to the target point;

Step 3: Input the coordinate information of the target point representing each grid position and the corresponding point cloud feature information into the target detection network to obtain the target object in the target scene.

Here, before rasterizing, there is a many-to-one relationship between points and rasters. After the grid feature extraction is performed, the features of each point in a grid are fused (corresponding to the feature encoding process), and at this time, the grid can be mapped back to the corresponding target point. The target object detection can be realized based on the coordinate information of the mapped target point and the corresponding point cloud feature information.

In a specific application, a corresponding interface can also be set between the above feature encoding operation and the above mapping operation, and the encoded encoded feature information is written to the storage medium in a sequential manner, and can be sequentially read from the storage medium before the mapping operation. Encoding feature information.

The above-mentioned mapping relationship between grids and points corresponds to the remapping process from grid scatter data to a bird's-eye view detection plane. At this time, the coordinate information of the mapped target point can be marked at the corresponding position of the corresponding bird's-eye view, and zero can be automatically filled for points without grid mapping.

It should be noted that the sparse feature map corresponding to the grid discrete data may be sparsely arranged. Therefore, when the post-stage convolutional neural network can support sparse convolution, the amount of grid feature output data is greatly reduced, which can further reduce the amount of computation.

In addition, when the lidar device used in this method is a rotating radar, the rotating radar collects point cloud data according to a set angle (such as 15°), and the point cloud data collected in one circle (corresponding to 360°) can be collected here. As a frame of point cloud data, point cloud rasterization and grid feature extraction can also be performed, point cloud data collected in a half circle (corresponding to 180°) can also be used as a frame of point cloud data for point cloud rasterization and grid feature extraction. , the point cloud data collected in a quarter circle (corresponding to 90°) can also be used as a frame of point cloud data for point cloud rasterization and raster feature extraction.

In order to extract the point cloud feature information corresponding to a complete circle of point cloud data faster, here, two grid feature extraction modules can be used to accelerate the two half-circle point cloud data in parallel, or four grid features can be used. The extraction module performs parallel acceleration on the four quarter-circle point cloud data, thereby increasing the processing frame rate and further improving the real-time feature extraction.

Relative to related technologies, the outstanding contributions of the method for processing point cloud data provided by the embodiments of the present disclosure mainly lie in the following points:

First, in the process of realizing grid feature extraction, the feature enhancement and feature encoding in the embodiments of the present disclosure are sequentially accelerated in the on-chip pipeline, and the feature enhancement and feature encoding can be accelerated separately in parallel;

Second, in the process of realizing point cloud rasterization, the embodiment of the present disclosure adopts the point cloud raster index table and the compressed data form of features for rasterization. Feature extraction in the form of data.

Based on the method for processing point cloud data provided by the foregoing embodiments, an embodiment of the present disclosure further provides a radar apparatus applying the foregoing processing method, the radar apparatus including: a radar controller, a radar component, and a field programmable gate array FPGA; wherein , the radar controller is electrically connected with the radar component and the FPGA respectively;

The radar controller is used to control the radar component to scan the target scene and obtain the point cloud data corresponding to the target scene;

The FPGA is used to rasterize the point cloud data to obtain point cloud datasets corresponding to multiple grids; and is used to determine the point cloud features corresponding to the grid according to the point cloud dataset of each grid Information; wherein the point cloud dataset in each grid is sequentially stored in the storage medium, and the point cloud dataset contains data of at least one point.

Here, the radar component is responsible for driving the lidar and storing the original point cloud obtained by the lidar ranging to off-chip memory such as DDR3\DDR4 or on-chip memory. The FPGA obtains the original point cloud of the relevant lidar from the on-chip or off-chip memory, and calls the PS side of the FPGA heterogeneous system, that is, the ARM side (a kind of microprocessor) kernel Central Processing Unit (CPU) as the point cloud feature extractor , extract the features of the original point cloud of the lidar, and store the feature information in the on-chip memory. Call the PL side of the FPGA heterogeneous system, that is, the logic circuit as an artificial neural network accelerator, perform convolution calculation on the feature information, and output the detection results to the radar component through on-chip or off-chip memory, and finally the radar component is responsible for sending the detection results through the network port. output to the user.

Wherein, the above-mentioned FPGA includes an on-chip processor and an on-chip logic circuit.

The on-chip processor performs grid processing on the point cloud data to obtain point cloud datasets corresponding to multiple grids; and is used to determine the point cloud corresponding to the grid according to the point cloud dataset of each grid Feature information; the point cloud dataset in each grid is sequentially stored in the storage medium, and the point cloud dataset contains data of at least one point;

On-chip logic to detect target objects based on point cloud feature information corresponding to each grid. Among them, the on-chip logic circuit can be used to accelerate the target detection network.

Here, the point cloud data corresponding to the target scene can be stored in off-chip memory or on-chip memory, and the storage medium stored in the point cloud data set in the same grid can be on-chip memory.

In this embodiment of the present disclosure, a target detection network such as a neural network may be used to detect target objects based on the point cloud feature information corresponding to each grid.

The network parameters of the above neural network can be stored in off-chip memory. When in use, read from the off-chip memory to the on-chip memory, the on-chip processor and the on-chip logic circuit can perform data processing based on network parameters.

Those skilled in the art can understand that in the above method of the specific implementation, the writing order of each step does not mean a strict execution order but constitutes any limitation on the implementation process, and the specific execution order of each step should be based on its function and possible Internal logic is determined.

Based on the same inventive concept, the embodiment of the present disclosure also provides a point cloud data processing device corresponding to the method for processing point cloud data. The processing methods are similar, so the implementation of the device can refer to the implementation of the method, and the repetition will not be repeated.

Embodiment 2

Referring to FIG. 5 , which is a schematic structural diagram of a device for processing point cloud data according to an embodiment of the present disclosure, the device includes: an acquisition module 501 , a processing module 502 , and a determination module 503 .

The acquiring module 501 is configured to acquire point cloud data corresponding to the target scene.

The processing module 502 is configured to perform grid processing on the acquired point cloud data to obtain point cloud data sets corresponding to multiple grids respectively. The point cloud datasets in each grid are sequentially stored in the storage medium; each point cloud dataset contains data of at least one point.

The determining module 503 is configured to determine the point cloud feature information corresponding to the grid according to the point cloud data set of each grid.

The above processing device divides the point cloud feature extraction into a point cloud rasterization process and a grid feature extraction process. In this way, by taking the grid as a unit, the overall point cloud feature extraction is performed on the point cloud data set of each grid to improve the performance. The efficiency of point cloud feature extraction is improved.

In one embodiment, the processing module 502 is configured to perform rasterization processing on the acquired point cloud data, including:

Based on the conversion relationship between the radar coordinate system and the Cartesian coordinate system, the radar coordinate information of each point in the acquired point cloud data is converted into the corresponding Cartesian coordinate information respectively;

Determine the coordinate range of all grids according to the set radar detection range and preset grid size;

The points in the point cloud data are divided into corresponding grids according to their respective Cartesian coordinate information.

In one embodiment, the processing module 502 is configured to divide the points in the point cloud data into corresponding grids according to their respective Cartesian coordinate information, including for each point in the point cloud data:

Determine the grid to be divided by the point based on the Cartesian coordinate information corresponding to the point and the coordinate range of all grids;

If the number of grids that have divided points is less than the first preset number, determine whether the number of divided points in the grid to be divided at the point is less than the second preset number;

If it is less than the second preset number, the point is divided into the grid to be divided by the point.

In one embodiment, the determining module 503 is configured to determine the point cloud feature information corresponding to the grid according to the point cloud dataset of each grid, including the point cloud dataset for each grid:

Based on the coordinate information of each point in the point cloud dataset, feature enhancement is performed on the point, and the enhanced feature information of the point is determined; the enhanced feature information has a first feature dimension;

Based on the enhanced feature information of each point in the point cloud dataset, the encoded feature information of the point cloud dataset is determined; the encoded feature information has a second feature dimension; the number of dimensions of the second feature dimension is greater than the number of dimensions of the first feature dimension;

Based on the encoded feature information of the point cloud dataset, the point cloud feature information corresponding to the grid is determined.

In one embodiment, the determining module 503 is configured to perform feature enhancement on the point based on the coordinate information of each point in the point cloud data set, and determine the enhanced feature information of the point, including:

Based on the coordinate information of each point in the point cloud dataset of each grid, determine the center coordinate information of the point cloud dataset;

For each point, determine the first deviation information corresponding to the point based on the coordinate information of the point and the determined center coordinate information; and based on the coordinate information of the point and other points in the point cloud dataset except the point Coordinate information to determine the second deviation information between the point and other points;

The coordinate information, the first deviation information and the second deviation information of the point are used as the enhanced feature information of the point.

In one embodiment, the determining module 503 is configured to use a convolutional neural network to determine the encoded feature information of each point cloud dataset based on the enhanced feature information of each point in the point cloud dataset, wherein the convolutional neural network includes a convolutional neural network. Buildup and activation layers, including for:

Convolution operation is performed on the enhanced feature information of each point in the point cloud dataset by using the convolution kernel of the convolution layer to obtain the convolution feature information corresponding to each point in the point cloud dataset;

Combine the convolution feature information corresponding to each point to obtain the convolution feature information corresponding to the point cloud dataset;

The convolution feature information corresponding to the point cloud dataset is processed by the activation function of the activation layer, and the encoded feature information of the point cloud dataset is obtained.

In one embodiment, the determining module 503 is configured to process the convolution feature information corresponding to the point cloud dataset by using the activation function of the activation layer to obtain the encoded feature information of the point cloud dataset, including:

Perform quantization processing on the convolution feature information corresponding to the point cloud data set according to the first preset bit width to obtain the quantized convolution feature information; the first preset bit width is the quantization when the target detection network is deployed to the running platform The bit width of the target detection network is used to detect the target object according to the point cloud feature information corresponding to each grid;

The quantized convolution feature information is processed by the activation function of the activation layer, and the encoded feature information of the point cloud dataset is obtained.

In one embodiment, the determining module 503 is configured to use the coordinate information, the first deviation information and the second deviation information of each point as the enhanced feature information of the point, including:

The coordinate information, the first deviation information and the second deviation information of the point are respectively quantized according to the second preset bit width to obtain the quantized coordinate information, the first deviation information and the second deviation information; the second preset The bit width is the quantized bit width when the convolutional neural network is deployed to the running platform; the convolutional neural network determines the encoded feature information of a point cloud dataset based on the enhanced feature information of each point in the point cloud dataset;

The quantized coordinate information, the first deviation information, and the second deviation information are used as the enhanced feature information of the point.

In one embodiment, the above device further includes:

The detection module 504 is configured to detect the target object based on the point cloud feature information corresponding to each grid.

In one embodiment, the detection module 504 is configured to detect the target object based on the point cloud feature information corresponding to each grid, including:

The point cloud feature information of the multiple grids under the same feature dimension is used as a set of input information, and different sets of input information corresponding to different feature dimensions are input into the target detection network in parallel to obtain the target object in the target scene.

In one embodiment, the detection module 504 is configured to detect the target object based on the point cloud feature information corresponding to each grid, including:

Determine the coordinate information of the target point representing the position of the grid based on the coordinate information of each point in the point cloud dataset of each grid;

Taking the point cloud feature information corresponding to the grid as the point cloud feature information corresponding to the target point;

The coordinate information of the target points representing the multiple grid positions and the corresponding point cloud feature information are input into the target detection network to obtain the target object in the target scene.

For the description of the processing flow of each module in the apparatus and the interaction flow between the modules, reference may be made to the relevant descriptions in the foregoing method embodiments, which will not be described in detail here.

Embodiment 3

An embodiment of the present disclosure further provides an electronic device. As shown in FIG. 6 , a schematic structural diagram of the electronic device provided by the embodiment of the present disclosure includes: a processor 601 , a memory 602 , and a bus 603 . The memory 602 stores machine-readable instructions executable by the processor 601 (in the apparatus for processing point cloud data as shown in FIG. 5 , the correspondingly executed instructions of the acquisition module 501 , the processing module 502 and the determination module 503 ). When the electronic device is running, the processor 601 communicates with the memory 602 through the bus 603, and the machine-readable instructions are executed by the processor 601 to perform the following processing:

Obtain the point cloud data corresponding to the target scene;

After rasterizing the acquired point cloud data, point cloud datasets corresponding to multiple grids are obtained; the point cloud datasets in each grid are sequentially stored in the storage medium; each point cloud dataset contains data for at least one point;

According to the point cloud dataset of each grid, determine the point cloud feature information corresponding to the grid.

Embodiments of the present disclosure further provide a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium. When the computer program is run by the processor 601, the steps of the method for processing point cloud data in the foregoing method embodiments are executed. . Wherein, the computer-readable storage medium may be a volatile or non-volatile computer-readable storage medium.

The computer program product of the method for processing point cloud data provided by the embodiments of the present disclosure includes a computer-readable storage medium storing program codes, and the instructions included in the program codes can be used to execute the point clouds described in the above method embodiments. For the steps of the data processing method, reference may be made to the foregoing method embodiments, and details are not described herein again.

Embodiments of the present disclosure further provide a computer program, which implements any one of the methods in the foregoing embodiments when the computer program is executed by a processor. The computer program product can be specifically implemented by hardware, software or a combination thereof. In an optional embodiment, the computer program product is embodied as a computer storage medium, and in another optional embodiment, the computer program product is embodied as a software product, such as a software development kit (Software Development Kit, SDK), etc. Wait.

Those skilled in the art can clearly understand that, for the convenience and brevity of description, for the specific working process of the system and device described above, reference may be made to the corresponding process in the foregoing method embodiments, which will not be repeated here. In the several embodiments provided by the present disclosure, it should be understood that the disclosed system, apparatus and method may be implemented in other manners. The apparatus embodiments described above are only illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components may be combined or Can be integrated into another system, or some features can be ignored, or not implemented. On the other hand, the shown or discussed mutual coupling or direct coupling or communication connection may be through some communication interfaces, indirect coupling or communication connection of devices or units, which may be in electrical, mechanical or other forms.

The units described as separate components may or may not be physically separated, and components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution in this embodiment.

In addition, each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, or each unit may exist physically alone, or two or more units may be integrated into one unit.

The functions, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a processor-executable non-volatile computer-readable storage medium. Based on such understanding, the technical solutions of the present disclosure can be embodied in the form of software products in essence, or the parts that contribute to the prior art or the parts of the technical solutions. The computer software products are stored in a storage medium, including Several instructions are used to cause an electronic device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present disclosure. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disk or optical disk and other media that can store program codes .

Finally, it should be noted that the above-mentioned embodiments are only specific implementations of the present disclosure, and are used to illustrate the technical solutions of the present disclosure rather than limit them. The protection scope of the present disclosure is not limited thereto, although referring to the foregoing The embodiments describe the present disclosure in detail. Those of ordinary skill in the art should understand that: any person skilled in the art can still modify the technical solutions described in the foregoing embodiments within the technical scope disclosed by the present disclosure. Changes can be easily thought of, or equivalent replacements are made to some of the technical features; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present disclosure, and should be covered in the present disclosure. within the scope of protection. Therefore, the protection scope of the present disclosure should be based on the protection scope of the claims.

Claims (17)

1. A method for processing point cloud data, comprising:

   Obtain the point cloud data corresponding to the target scene;

   Perform grid processing on the acquired point cloud data to obtain point cloud datasets corresponding to multiple grids; the point cloud datasets in each grid are sequentially stored in the storage medium; each of the point cloud datasets The cloud dataset contains data of at least one point;

   According to the point cloud dataset of each grid, point cloud feature information corresponding to the grid is determined.
2. The processing method according to claim 1, wherein the performing rasterization processing on the acquired point cloud data comprises:

   Based on the conversion relationship between the radar coordinate system and the Cartesian coordinate system, the acquired radar coordinate information of each point in the point cloud data is respectively converted into corresponding Cartesian coordinate information;

   Determine the coordinate range of all grids according to the set radar detection range and preset grid size;

   The points in the point cloud data are divided into corresponding grids according to their respective Cartesian coordinate information.
3. The processing method according to claim 2, wherein the dividing the points in the point cloud data into corresponding grids according to their respective Cartesian coordinate information, comprising:

   For each point in the point cloud data,

   Determine the grid to be divided by the point based on the Cartesian coordinate information of the point and the coordinate range of all grids;

   If the number of grids that have divided points is less than the first preset number, determine whether the number of divided points in the grid to be divided by the point is less than the second preset number;

   If it is less than the second preset number, the points are divided into grids to which the points are to be divided.
4. The processing method according to any one of claims 1 to 3, wherein determining the point cloud feature information corresponding to the grid according to the point cloud data set of each grid, comprising:

   the point cloud dataset for each raster,

   Based on the coordinate information of each point in the point cloud data set, feature enhancement is performed on the point, and enhanced feature information of the point is determined; the enhanced feature information has a first feature dimension;

   Based on the enhanced feature information of each point in the point cloud dataset, the encoded feature information of the point cloud dataset is determined; the encoded feature information has a second feature dimension; the number of dimensions of the second feature dimension is greater than the The number of dimensions of the first feature dimension;

   Based on the encoded feature information of the point cloud dataset, point cloud feature information corresponding to the grid is determined.
5. The processing method according to claim 4, wherein the feature enhancement is performed on the point based on the coordinate information of each point in the point cloud data set, and the enhanced feature information of the point is determined, comprising: :

   Determine the center coordinate information of the point cloud dataset based on the coordinate information of each point in the point cloud dataset;

   For each of said points,

   Determine the first deviation information corresponding to the point based on the coordinate information of the point and the determined center coordinate information; and divide the point cloud dataset based on the coordinate information of the point and the point cloud dataset coordinate information of other points other than the point, and determine second deviation information between the point and the other points;

   The coordinate information of the point, the first deviation information and the second deviation information are used as the enhanced feature information of the point.
6. The processing method according to claim 4 or 5, wherein the encoded feature information of the point cloud dataset is determined based on the enhanced feature information of each point in the point cloud dataset by using a convolutional neural network, wherein the The convolutional neural network described above includes convolutional layers and activation layers, including:

   Use the convolution check of the convolution layer to perform a convolution operation on the enhanced feature information of each point in the point cloud data set, to obtain convolution feature information corresponding to each point in the point cloud data set;

   Combining the convolution feature information corresponding to each point to obtain the convolution feature information corresponding to the point cloud data set;

   The convolution feature information corresponding to the point cloud data set is processed by using the activation function of the activation layer to obtain the encoded feature information of the point cloud data set.
7. The processing method according to claim 6, wherein the convolution feature information corresponding to the point cloud data set is processed by using the activation function of the activation layer to obtain the encoded feature information of the point cloud data set, include:

   Perform quantization processing on the convolution feature information corresponding to the point cloud data set according to the first preset bit width to obtain the quantized convolution feature information; the first preset bit width is the target detection network deployed to the running The quantized bit width of the platform, the target detection network is used to detect the target object according to the point cloud feature information corresponding to each grid;

   The quantized convolution feature information is processed by using the activation function of the activation layer to obtain the encoded feature information of the point cloud data set.
8. The processing method according to claim 5, wherein the taking the coordinate information, the first deviation information and the second deviation information of the point as the enhanced feature information of the point, further comprising: :

   The coordinate information, the first deviation information and the second deviation information of the point are respectively quantized according to the second preset bit width to obtain the quantized coordinate information, the first deviation information and the second deviation information. deviation information; the second preset bit width is the bit width quantified when the convolutional neural network is deployed to the running platform; the convolutional neural network determines the The encoded feature information of the point cloud dataset;

   The quantized coordinate information, the first deviation information, and the second deviation information are used as the enhanced feature information of the point.
9. The processing method according to any one of claims 1 to 8, wherein after determining the point cloud feature information corresponding to the grid, the method further comprises:

   A target object is detected based on the point cloud feature information corresponding to each grid.
10. The processing method according to claim 9, wherein the detecting a target object based on the point cloud feature information corresponding to each grid comprises:

    The point cloud feature information of the multiple grids under the same feature dimension is used as a set of input information, and different sets of input information corresponding to different feature dimensions are input into the target detection network in parallel to obtain the target in the target scene. object.
11. The processing method according to claim 9, wherein the detecting a target object based on the point cloud feature information corresponding to each grid comprises:

    For each raster,

    Determine the coordinate information of the target point representing the position of the grid based on the coordinate information of each point in the point cloud data set of the grid;

    Taking the point cloud feature information corresponding to the grid as the point cloud feature information corresponding to the target point;

    The coordinate information of the target points representing the multiple grid positions and the corresponding point cloud feature information are input into the target detection network to obtain the target object in the target scene.
12. A radar device, comprising: a radar controller, a radar assembly, and a field programmable gate array FPGA; wherein the radar controller is electrically connected to the radar assembly and the FPGA, respectively;

    the radar controller, configured to control the radar component to scan the target scene and obtain point cloud data corresponding to the target scene;

    The FPGA includes an on-chip processor, and the on-chip processor is used to perform grid processing on the point cloud data to obtain point cloud data sets corresponding to multiple grids; point cloud dataset, determining the point cloud feature information corresponding to the grid; wherein the point cloud dataset in each grid is sequentially stored in the storage medium, and each point cloud dataset contains data of at least one point .
13. The radar device according to claim 12, wherein the FPGA further comprises an on-chip logic circuit;

    The on-chip logic circuit is configured to detect a target object based on the point cloud feature information corresponding to each grid.
14. A processing device for point cloud data, comprising:

    The acquisition module is used to acquire the point cloud data corresponding to the target scene;

    a processing module, configured to perform grid processing on the acquired point cloud data to obtain point cloud datasets corresponding to multiple grids; the point cloud datasets in each grid are sequentially stored in the storage medium; Each of the point cloud datasets contains data of at least one point;

    A determination module, configured to determine the point cloud feature information corresponding to the grid according to the point cloud data set of each grid.
15. An electronic device, comprising: a processor, a memory and a bus;

    the memory stores machine-readable instructions executable by the processor;

    When the electronic device is running, the processor and the memory communicate through the bus;

    When the machine-readable instructions are executed by the processor, the steps of the method for processing point cloud data according to any one of claims 1 to 11 are performed.
16. A computer-readable storage medium, on which a computer program is stored, and when the computer program is run by a processor, the steps of the method for processing point cloud data according to any one of claims 1 to 11 are executed. .
17. A computer program, the computer program comprising computer readable codes, which can implement the steps of the point cloud data processing method according to any one of claims 1 to 11 when the computer readable codes are executed by a processor.

Images