CN117253209B

CN117253209B - Automatic driving point cloud detection method, device, communication equipment and storage medium

Info

Publication number: CN117253209B
Application number: CN202311532044.4A
Authority: CN
Inventors: 詹景麟; 刘铁军; 张晶威; 韩大峰
Original assignee: Suzhou Metabrain Intelligent Technology Co Ltd
Current assignee: Suzhou Metabrain Intelligent Technology Co Ltd
Priority date: 2023-11-16
Filing date: 2023-11-16
Publication date: 2024-02-20
Anticipated expiration: 2043-11-16
Also published as: CN117253209A

Abstract

The embodiment of the application provides an automatic driving point cloud detection method, an automatic driving point cloud detection device, communication equipment and a storage medium, wherein the automatic driving point cloud detection method comprises the following steps: collecting original point cloud data of a target vehicle; voxelized processing is carried out on the original point cloud data to obtain voxelized point cloud feature vectors; constructing a three-dimensional convolutional neural network block according to a three-dimensional convolutional calculation layer which is generated in advance; inputting the voxelized feature vector into the three-dimensional convolutional neural network block to obtain a target three-dimensional point cloud feature vector; converting the target three-dimensional point cloud feature vector into a target two-dimensional point cloud feature vector; and inputting the target two-dimensional point cloud feature vector into a target detection neural network constructed based on two-dimensional convolution to obtain a detection result of target objects around the target vehicle. According to the embodiment of the application, the three-dimensional convolution calculation layer is utilized to achieve efficient extraction of three-dimensional information of point cloud data, the method is used for achieving accurate perception and understanding of an automatic driving automobile on a complex environment in a task of identifying objects around the automobile, and meanwhile calculation power consumption of a vehicle-mounted platform is saved.

Description

Automatic driving point cloud detection method, device, communication equipment and storage medium

Technical Field

The application relates to the technical field of automatic driving, in particular to an automatic driving point cloud detection method, an automatic driving point cloud detection device, communication equipment and a storage medium.

Background

The point cloud data is a discrete three-dimensional data representation consisting of a large number of three-dimensional points. Each three-dimensional point has three spatial coordinates, x, y, and z, for describing the exact location of a point in three-dimensional space. Point cloud data is often used to represent three-dimensional objects and scenes in the real world, and is an important data source for three-dimensional reconstruction and understanding. In the field of automatic driving, the point cloud data is used as one of the best data sources for realizing environment perception, can provide surrounding rich three-dimensional space information for an automatic driving automobile, is used for realizing the identification of other traffic participants, including pedestrians, bicycles, other vehicles and other target objects, and obtains basic information of the target objects, including categories, positions, sizes, movement speeds and the like. Compared with image data, the point cloud data is not influenced by conditions such as illumination, and the position, speed and other information of static and dynamic obstacles can be reflected more accurately, so that the point cloud data is very important for planning a route and avoiding the obstacles of an automatic driving automobile.

Three-dimensional convolution is used as an effective mechanism for directly operating a three-dimensional space, and can simultaneously consider the association between all dimensions in the three-dimensional space, so that the spatial characteristics and the structure of the three-dimensional data can be learned more accurately. The introduction of three-dimensional convolution enables three-dimensional information processing to be changed from two-dimensional manual feature extraction to end-to-end three-dimensional space learning, and as the three-dimensional information processing can directly act on three-dimensional input data, the three-dimensional convolution greatly simplifies information processing flow, is widely applied to automatic driving point cloud detection based on a deep learning technology, and lays a foundation for identifying target objects (including pedestrians, bicycles, other vehicles and the like) around a vehicle as a core means for extracting three-dimensional space and structural information of point cloud data.

The traditional three-dimensional convolution operation is directly applied to three-dimensional point cloud data, unstructured features are difficult to extract, the operation is low in efficiency, the feature perception range expansion is limited, efficient and accurate three-dimensional point cloud feature analysis cannot be achieved, the recognition accuracy of an automatic driving system to peripheral target objects is greatly affected, and meanwhile, the serious waste of computational power resources of a vehicle-mounted hardware platform is caused.

Disclosure of Invention

An object of an embodiment of the present application is to provide an automatic driving point cloud detection method, an automatic driving point cloud detection device, a communication device and a storage medium, and the specific technical scheme is as follows:

in a first aspect of the present application, there is first provided an autopilot point cloud detection method applied to an autopilot system, the autopilot point cloud detection method including:

collecting original point cloud data of a target vehicle;

voxelized processing is carried out on the original point cloud data to obtain voxelized point cloud feature vectors;

constructing a three-dimensional convolutional neural network block according to a pre-generated three-dimensional convolutional calculation layer, wherein the pre-generated three-dimensional convolutional calculation layer is constructed based on self-adaptive node retrieval and local voxel grid tree;

inputting the voxelized point cloud feature vector into the three-dimensional convolutional neural network block to obtain a target three-dimensional point cloud feature vector;

Converting the target three-dimensional point cloud feature vector into a target two-dimensional point cloud feature vector;

and inputting the target two-dimensional point cloud feature vector into a target detection neural network constructed based on two-dimensional convolution to obtain a detection result of target objects around the target vehicle.

Optionally, the inputting the voxelized point cloud feature vector to the three-dimensional convolutional neural network block, to obtain a target three-dimensional point cloud feature vector includes:

determining a first channel number of the input voxelized point cloud characteristic vector and a second channel number of the output voxelized point cloud characteristic vector;

determining the corresponding quantity and the corresponding size of the multi-channel three-dimensional convolution kernels;

traversing voxel grids in an input channel corresponding to a first channel number in the input three-dimensional feature vector, constructing key value pairs, and storing root node feature values calculated according to the local non-empty voxel grid tree;

reading a target three-dimensional convolution kernel, wherein the target three-dimensional convolution kernel is determined based on the first channel number and the second channel number;

acquiring a three-dimensional cross-correlation calculation result of the input three-dimensional feature vector corresponding to the target three-dimensional convolution kernel and the first channel number corresponding to the target three-dimensional convolution kernel at a preset position, wherein the three-dimensional cross-correlation calculation result comprises a feature value of the output three-dimensional feature vector corresponding to the second channel number at the preset position;

Accumulating the characteristic values corresponding to the input characteristic vectors with the number corresponding to the first channel number, and splicing the characteristic values corresponding to the output characteristic vectors with the number corresponding to the second channel number to obtain the characteristic values corresponding to the output three-dimensional characteristic vectors with the number corresponding to the second channel number at the preset position;

and outputting the feature values of the three-dimensional feature vectors corresponding to the number of the second channels at the preset positions according to the feature values of the three-dimensional point cloud feature vectors corresponding to the number of the second channels.

Optionally, the obtaining the three-dimensional cross-correlation calculation result of the input three-dimensional feature vector corresponding to the target three-dimensional convolution kernel and the first channel number corresponding to the target three-dimensional convolution kernel at the preset position includes:

determining a local three-dimensional feature vector according to the corresponding size of the three-dimensional convolution kernel;

setting each position in the local three-dimensional feature vector as a root node, and acquiring a root node index;

according to the root node index and the three-dimensional convolution kernel, sliding a preset step number along three dimensions, and solving the absolute voxel grid position of the root node in an input three-dimensional feature vector;

initializing a target root node according to the absolute voxel grid position in the input three-dimensional feature vector, and constructing a local non-empty voxel grid tree corresponding to the target root node;

Updating a characteristic value of a target root node in a local non-empty voxel grid tree corresponding to the target root node;

updating the root node index until the root node index reaches the size corresponding to the three-dimensional convolution kernel;

and weighting and updating the characteristic values contained in the voxel grids in the local input three-dimensional characteristic vectors according to the weight values stored in the three-dimensional convolution kernel to obtain a three-dimensional cross-correlation calculation result.

Optionally, traversing voxel grids in an input channel corresponding to the first channel number in the input three-dimensional feature vector, and constructing key value pairs to store the root node feature value calculated according to the local non-empty voxel grid tree includes:

constructing a local non-empty voxel grid tree for each root node;

updating the characteristic value of the root node in the local non-empty voxel grid number;

and constructing a key value to store the characteristic value of the root node.

Optionally, the storing the root node feature value by the build key value includes:

the input channel index, the voxel grid absolute coordinates are stored in a key store, and the root node feature values are stored in a value store.

Optionally, the constructing a local non-empty voxel grid tree for each root node includes:

And constructing a local non-empty voxel grid tree aiming at each voxel grid in the input three-dimensional feature vector covered by the current sliding three-dimensional convolution kernel by using a node searching mode, wherein the root node is a single voxel grid in the input three-dimensional feature vector.

Optionally, the constructing a local non-empty voxel grid tree for each voxel grid in the input three-dimensional feature vector covered by the current sliding three-dimensional convolution kernel by using a node searching mode includes:

determining storage information corresponding to each node, wherein the storage information comprises a shortest offset vector from a root node to each current node, an absolute voxel grid position corresponding to the current node in the input three-dimensional feature vector, a feature value of a voxel grid corresponding to the input three-dimensional feature vector, and a parent node at the upper level, the shortest offset vector is used for determining a contribution weight of the feature value of the current node to the feature value of the root node, and the absolute voxel grid position in the input three-dimensional feature vector is used for extracting the feature value corresponding to the current node from the input three-dimensional vector;

initializing a local voxel grid tree root node;

initializing a primary node, wherein the primary node is a child node corresponding to the root node;

Traversing the primary nodes, and searching nodes for each primary node until the construction of the local non-empty voxel grid tree is completed.

Optionally, initializing the local voxel grid tree root node comprises:

acquiring an initial absolute voxel grid position of the root node in the input three-dimensional feature vector;

setting the shortest offset vector corresponding to the root node as an initial vector value;

and reading the characteristic value corresponding to the initial absolute voxel grid position from the input three-dimensional characteristic vector as an initial characteristic value.

Optionally, the primary node is a child node corresponding to the root node, and the initializing the primary node includes:

the shortest offset vector of each level one node corresponds to one element in a set of unit offset vectors;

initializing the absolute voxel grid position corresponding to each primary node according to the shortest offset vector and the initial absolute voxel grid position;

extracting a corresponding characteristic value from the input three-dimensional characteristic vector as an initial characteristic value of each primary node;

initializing the parent node of the previous level corresponding to the primary node as the root node;

each of the primary nodes is added to a local non-empty voxel grid tree.

Optionally, traversing the primary nodes, and searching nodes for each primary node until the construction of the local non-empty voxel grid tree is completed includes:

traversing the primary nodes, and completing searching of child nodes with preset layers aiming at each primary node, wherein the preset layers are determined based on the preset maximum level number of the local non-empty voxel grid tree;

storing the shortest offset vector of each level of child nodes corresponding to the root node, the absolute voxel grid position, the initial characteristic value and the parent node of the previous level;

traversing all nodes corresponding to any level, and searching nodes for all nodes of the next level corresponding to any level until the construction of the local non-empty voxel grid tree is completed.

Optionally, traversing all nodes corresponding to any level, and searching nodes for all nodes of a next level corresponding to any level until building of the local non-empty voxel grid tree is completed includes:

performing expandability and validity check on all nodes corresponding to any level according to the parameter states corresponding to the positioned voxel grids to determine whether the current node meets preset conditions;

Under the condition that the preset condition is detected to be met, acquiring a characteristic value corresponding to the positioned voxel grid, wherein the characteristic value corresponding to the positioned voxel grid is used for calculating a root node characteristic value containing irregular three-dimensional topological structure information;

and searching nodes for all nodes of the next level corresponding to any level until the construction of the local non-empty voxel grid tree is completed.

Optionally, the updating the root node feature value in the local non-empty voxel grid number includes:

the feature values of all nodes in different levels are updated step by step in the local non-empty voxel grid tree based on the sequence from bottom to top;

and carrying out weighted accumulation on the characteristic values of all nodes in different levels, and updating the characteristic value of the root node in the local non-empty voxel grid number.

Optionally, before the step of determining the stored information corresponding to each node, the method includes:

presetting the corresponding size of an input three-dimensional feature vector and the maximum level number of the local non-empty voxel grid number;

an anisotropic unit offset vector set is obtained.

Optionally, the unit offset vector set is used to construct the local non-empty voxel grid tree, and the obtaining the anisotropic unit offset vector set includes:

Determining an adaptive expansion search coefficient;

setting an initial search unit offset vector set, wherein the initial search unit offset vector set comprises unit movement operation in a preset direction;

reading the absolute position of a voxel grid in a three-dimensional input feature vector corresponding to the root node;

obtaining unit offset vector expansion coefficients respectively corresponding to the preset directions according to the absolute positions of the voxel grids;

and adaptively acquiring an anisotropic unit offset vector set according to the unit offset vector expansion coefficient.

Optionally, the unit offset vector expansion coefficient is obtained by the following formula:

wherein,representing the unit offset vector expansion coefficient, the Ceil function represents the rounding up operation, ++>The function is the dirac function, when +.>When empty, the person is left with->The function value is 1, K represents the maximum level number of the local non-empty voxel grid tree, and E represents the adaptive expansion search coefficient.

Optionally, the constructing the three-dimensional convolutional neural network block according to the three-dimensional convolutional calculation layer generated in advance includes:

the method comprises the steps of connecting a plurality of three-dimensional convolution computing layers which are generated in advance in series to obtain a plurality of three-dimensional convolution computing layers which are connected in series;

and constructing a three-dimensional convolutional neural network block according to the plurality of three-dimensional convolutional calculation layers after the series connection.

Optionally, the data format corresponding to the three-dimensional point cloud feature vector is that the number of the output three-dimensional feature channels is represented, and the three-dimensional space is uniformly divided into individual element grids;

the collecting the original point cloud data of the target vehicle comprises the following steps:

and acquiring original point cloud data of the target vehicle through a laser radar, wherein a data format corresponding to the original point cloud data is as follows.

Optionally, inputting the target two-dimensional point cloud feature vector into a target detection neural network constructed based on two-dimensional convolution, and obtaining a detection result of a target object around the target vehicle includes:

inputting the target two-dimensional point cloud feature vector into a target detection neural network constructed based on two-dimensional convolution to obtain a detection result of a target object around a target vehicle, wherein the detection result comprises at least one of category information, position information and size information corresponding to the target object.

In still another aspect of the present application, there is further provided an automatic driving point cloud detection apparatus, including:

the acquisition module is used for acquiring original point cloud data of the target vehicle;

the processing module is used for carrying out voxelization on the original point cloud data to obtain voxelization point cloud feature vectors;

The construction module is used for constructing a three-dimensional convolutional neural network block according to a pre-generated three-dimensional convolutional calculation layer, wherein the pre-generated three-dimensional convolutional calculation layer is a three-dimensional convolutional calculation layer constructed based on self-adaptive node retrieval and local voxel grid tree;

the input module is used for inputting the voxelized point cloud characteristic vector into the three-dimensional convolutional neural network block to obtain a target three-dimensional point cloud characteristic vector;

the conversion module is used for converting the target three-dimensional point cloud characteristic vector into a target two-dimensional point cloud characteristic vector;

the detection module is used for inputting the target two-dimensional point cloud feature vector into a target detection neural network constructed based on two-dimensional convolution to obtain a detection result of a target object around the target vehicle.

In yet another aspect of the present application, there is provided a communication device including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory complete communication with each other through the communication bus;

a memory for storing a computer program;

and the processor is used for realizing any one of the automatic driving point cloud detection methods when executing the program stored in the memory.

In yet another aspect of the application implementation, there is also provided a computer readable storage medium having instructions stored therein that, when executed on a computer, cause the computer to perform any one of the above-described methods of automatic driving point cloud detection.

According to the automatic driving point cloud detection method, original point cloud data of a target vehicle are collected; voxelized processing is carried out on the original point cloud data to obtain voxelized point cloud feature vectors; constructing a three-dimensional convolutional neural network block according to a pre-generated three-dimensional convolutional calculation layer, wherein the pre-generated three-dimensional convolutional calculation layer is constructed based on self-adaptive node retrieval and local voxel grid tree; inputting the voxelized point cloud feature vector into the three-dimensional convolutional neural network block to obtain a target three-dimensional point cloud feature vector; converting the target three-dimensional point cloud feature vector into a target two-dimensional point cloud feature vector; and inputting the target two-dimensional point cloud feature vector into a target detection neural network constructed based on two-dimensional convolution to obtain a detection result of target objects around the target vehicle. According to the embodiment of the application, the original point cloud data are processed to obtain the voxelized point cloud feature vector, the voxelized point cloud feature vector is input to the three-dimensional convolution calculation layer based on the pre-generation to construct the three-dimensional convolution neural network block, so that the efficient extraction of the three-dimensional information of the point cloud data is realized by using the efficient three-dimensional convolution calculation layer based on the self-adaptive node retrieval and supporting the multi-channel feature input and output, the efficient three-dimensional convolution calculation layer is used in the recognition task of the objects around the vehicle, the accurate perception and understanding of the automatic driving automobile on the complex environment are realized, and meanwhile, the calculation power consumption of the vehicle-mounted platform is saved.

Drawings

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below.

Fig. 1 shows a step flowchart of an automatic driving point cloud detection method provided in an embodiment of the present application;

FIG. 2 is a flowchart illustrating steps of another method for detecting an autopilot point cloud according to an embodiment of the present application;

fig. 3 is a step flowchart of step 206 in the step flowchart of another automatic driving point cloud detection method provided in fig. 2 according to an embodiment of the present application;

fig. 4 is a step flowchart of step 208 in the step flowchart of another automatic driving point cloud detection method provided in fig. 2 according to an embodiment of the present application;

fig. 5 shows a device block diagram of an automatic driving point cloud detection device according to an embodiment of the present application;

fig. 6 shows a block diagram of a communication device according to an embodiment of the present application;

FIG. 7 is a schematic diagram of a conventional three-dimensional cross-correlation operation according to an embodiment of the present application;

FIG. 8 illustrates a schematic diagram of a local non-empty voxel grid tree construction provided by an embodiment of the present application;

FIG. 9 is a schematic diagram of a three-dimensional feature calculation method of a root node voxel grid according to an embodiment of the present application;

FIG. 10 is a schematic diagram of a three-dimensional cross-correlation calculation method based on local voxel grid tree construction according to an embodiment of the present application;

FIG. 11 is a schematic diagram of an adaptive anisotropic local unit offset vector generation method according to an embodiment of the present application;

FIG. 12 is a schematic diagram showing a three-dimensional convolution calculation step based on adaptive node retrieval according to an embodiment of the present disclosure;

fig. 13 shows a schematic diagram of an automatic driving point cloud detection system according to an embodiment of the present application.

Detailed Description

The technical solutions in the embodiments of the present application will be described below with reference to the drawings in the embodiments of the present application.

For the purposes of making the objects, technical solutions and advantages of the embodiments of the present application more apparent, the following detailed description of each embodiment of the present application will be given with reference to the accompanying drawings. However, those of ordinary skill in the art will understand that in various embodiments of the present application, numerous technical details have been set forth in order to provide a better understanding of the present application. However, the technical solutions claimed in the present application can be implemented without these technical details and with various changes and modifications based on the following embodiments. The following embodiments are divided for convenience of description, and should not be construed as limiting the specific implementation of the present application, and the embodiments may be mutually combined and referred to without contradiction.

Referring to fig. 1, a step flowchart of an automatic driving point cloud detection method provided in an embodiment of the present application is shown, where the method may include:

step 101, collecting original point cloud data of a target vehicle;

further, the collecting the original point cloud data of the target vehicle includes: collecting original point cloud data of a target vehicle through a laser radar, wherein a data format corresponding to the original point cloud data is as follows。

It should be noted that, according to the embodiment of the application, a set of automatic driving point cloud detection system with high precision and low calculation power consumption can be realized, efficient feature extraction of three-dimensional point cloud data is realized, the recognition capability of the vehicle-mounted system on peripheral target objects is improved, and the calculation power consumption of the vehicle-mounted platform is saved.

Therefore, firstly, the automatic driving system collects point cloud data through the laser radar to obtain the data with the format ofIs described.

102, voxelized processing is carried out on the original point cloud data to obtain a voxelized point cloud characteristic vector;

the original point cloud data is subjected to voxel processing to obtain a data format ofIs described.

Step 103, constructing a three-dimensional convolutional neural network block according to a pre-generated three-dimensional convolutional calculation layer, wherein the pre-generated three-dimensional convolutional calculation layer is a three-dimensional convolutional calculation layer constructed based on self-adaptive node retrieval and local voxel grid tree;

Further, step 103 includes that said constructing a three-dimensional convolutional neural network block according to a pre-generated three-dimensional convolutional calculation layer includes: the method comprises the steps of connecting a plurality of three-dimensional convolution computing layers which are generated in advance in series to obtain a plurality of three-dimensional convolution computing layers which are connected in series; and constructing a three-dimensional convolutional neural network block according to the plurality of three-dimensional convolutional calculation layers after the series connection.

Further, the pre-generated three-dimensional convolution calculation layer is a three-dimensional convolution calculation layer based on adaptive node retrieval.

In the embodiment of the application, three-dimensional convolutional neural network blocks based on adaptive node retrieval are constructed by using three-dimensional convolutional calculation layers of N layers in series.

In the application, the three-dimensional convolution calculation layer generated in advance is a three-dimensional convolution calculation layer based on self-adaptive node retrieval.

Specifically, the three-dimensional convolution calculation layer based on the self-adaptive node retrieval acts on the voxelized point cloud data, so that different characteristic abstract requirements of a point cloud related algorithm are supported, and the waste of hardware resources caused by repeated calculation can be avoided by pre-constructing and storing characteristic values corresponding to the root nodes of the local non-empty voxel grid tree.

With particular reference to the following description.

Step 104, inputting the voxelized point cloud feature vector into the three-dimensional convolutional neural network block to obtain a target three-dimensional point cloud feature vector;

it should be noted that, inputting the above voxel point cloud data into the network block to implement feature abstraction of the point cloud data, and obtaining a data format as followsIs a target three-dimensional point cloud feature vector.

Further, the data format corresponding to the three-dimensional point cloud feature vector is as followsWherein->Representing the number of channels of the output three-dimensional feature, < >>Representing a uniform division of the three-dimensional space into +.>A voxel grid.

Step 105, converting the target three-dimensional point cloud feature vector into a target two-dimensional point cloud feature vector;

it should be noted that, the target three-dimensional point cloud feature vector is converted into a target two-dimensional point cloud feature vector according to formula 1.

Further, the converting the target three-dimensional point cloud feature vector into the target two-dimensional point cloud feature vector is by the following formula:

(equation 1)

Wherein, in the above formula 1,the input three-dimensional feature vector is of the size of two-dimensional point cloud feature vector。

And 106, inputting the target two-dimensional point cloud feature vector into a target detection neural network constructed based on two-dimensional convolution to obtain a detection result of a target object around the target vehicle.

Further, the inputting the target two-dimensional point cloud feature vector into a target detection neural network constructed based on two-dimensional convolution, and obtaining a detection result of a target object around the target vehicle comprises: inputting the target two-dimensional point cloud feature vector into a target detection neural network constructed based on two-dimensional convolution to obtain a detection result of a target object around a target vehicle, wherein the detection result comprises at least one of category information, position information and size information corresponding to the target object.

In the embodiment of the present application, the two-dimensional point cloud feature vector is input into a target detection neural network constructed based on two-dimensional convolution, so as to obtain a recognition result of an object around the vehicle, where the recognition result includes information such as a category, a position, a size, and the like of the target object.

For the target detection neural network, the application is not limited to a specific network architecture, and classical two-dimensional convolutional neural networks such as Yolo and RCNN can be utilized.

In addition, the above steps 101-106 may be described with reference to FIG. 13.

According to the automatic driving point cloud detection method, original point cloud data of a target vehicle are collected; voxelized processing is carried out on the original point cloud data to obtain voxelized point cloud feature vectors; constructing a three-dimensional convolutional neural network block according to a three-dimensional convolutional calculation layer which is generated in advance; inputting the voxelized point cloud feature vector into the three-dimensional convolutional neural network block to obtain a target three-dimensional point cloud feature vector; converting the target three-dimensional point cloud feature vector into a target two-dimensional point cloud feature vector; and inputting the target two-dimensional point cloud feature vector into a target detection neural network constructed based on two-dimensional convolution to obtain a detection result of target objects around the target vehicle. According to the embodiment of the application, the original point cloud data are processed to obtain the voxelized point cloud feature vector, the voxelized point cloud feature vector is input to the three-dimensional convolution calculation layer based on the pre-generation to construct the three-dimensional convolution neural network block, so that the efficient extraction of the three-dimensional information of the point cloud data is realized by using the efficient three-dimensional convolution calculation layer based on the self-adaptive node retrieval and supporting the multi-channel feature input and output, the efficient three-dimensional convolution calculation layer is used in the recognition task of the objects around the vehicle, the accurate perception and understanding of the automatic driving automobile on the complex environment are realized, and meanwhile, the calculation power consumption of the vehicle-mounted platform is saved.

Referring to fig. 2, a flowchart illustrating steps of another method for detecting an autopilot point cloud according to an embodiment of the present application is shown, where the method may include:

step 201, collecting original point cloud data of a target vehicle;

step 202, voxelized processing is carried out on the original point cloud data to obtain a voxelized point cloud characteristic vector;

it should be noted that, the steps 201-203 are discussed with reference to the foregoing, and are not repeated herein.

The following steps may be described with reference to fig. 12.

Step 203, constructing a three-dimensional convolutional neural network block according to a pre-generated three-dimensional convolutional calculation layer, wherein the pre-generated three-dimensional convolutional calculation layer is a three-dimensional convolutional calculation layer constructed based on self-adaptive node retrieval and local voxel grid tree;

step 204, determining a first channel number of the input voxelized point cloud characteristic vector and a second channel number of the output voxelized point cloud characteristic vector;

firstly, voxelization processing is carried out on original point cloud data, and the point cloud data is formatted fromConversion to: uniformly dividing three-dimensional space into +.>A voxel grid, for each voxel grid, if data points are contained therein, < +|all data points therein are used>The dimension average feature is taken as +. >And (3) marking the voxel grid as empty if the dimension features do not contain data points.

Step 205, determining the number and the size corresponding to the multi-channel three-dimensional convolution kernels;

second, determining the number of channels for inputting the three-dimensional featureAnd outputting the number of three-dimensional characteristic channels->I.e. the input three-dimensional feature vector of the three-dimensional convolution calculation layer contains +.>Personal characteristic value->The output three-dimensional feature vector containsPersonal characteristic value->。

Determining the number and size of multi-channel three-dimensional convolution kernels requires, in order to support multi-channel three-dimensional convolution calculations, the followingPersonal->A three-dimensional convolution kernel of size.

Step 206, traversing voxel grids in the input channels corresponding to the first channel number in the input three-dimensional feature vector, constructing key value pairs, and storing root node feature values calculated according to the local non-empty voxel grid tree;

further, traversing the input three-dimensional feature vectorCorresponding +.>And (3) individual voxel grids, constructing key value pairs, and storing root node characteristic value updating results obtained by calculation according to the local non-empty voxel grid tree.

Further, as shown in fig. 3, step 206 includes:

step 2061, constructing a local non-empty voxel grid tree for each root node;

construction of the following examplesA local non-empty voxel grid tree.

Further, the constructing a local non-empty voxel grid tree comprises: and constructing a local non-empty voxel grid tree aiming at each voxel grid in the input three-dimensional feature vector covered by the current sliding three-dimensional convolution kernel by utilizing a node searching mode.

Further, the constructing a local non-empty voxel grid tree for each voxel grid in the input three-dimensional feature vector covered by the current sliding three-dimensional convolution kernel by using a node searching mode includes: determining storage information corresponding to each node, wherein the storage information comprises the shortest offset vector from the root node to the node, the absolute voxel grid position in the input three-dimensional feature vector, the feature value of the voxel grid corresponding to the original input three-dimensional feature vector and the parent node at the upper level; initializing a local voxel grid tree root node; initializing a primary node, wherein the primary node is a child node corresponding to the root node; traversing the primary nodes, and searching nodes for each primary node until the construction of the local non-empty voxel grid tree is completed.

In the embodiment of the present application, referring to fig. 8, a local non-empty voxel grid tree is constructed for each voxel grid in the input three-dimensional feature vector covered by the current sliding three-dimensional convolution kernel by using a node search method, so as to obtain a random three-dimensional topology of point cloud data. In the method, a reasonable node information storage mode is planned, a judging rule of node expandability and legitimacy is introduced, node searching efficiency is optimized, and redundant searching and computing are avoided.

Specifically, the method comprises the following steps:

let the input three-dimensional feature vector beLet the maximum number of levels of the local non-empty voxel grid tree be +.>Let the unit offset vector set be:

wherein each element represents a unit grid offset operation of up, down, front, back, left, and right, respectively.

Defining information stored by the node: each node stores the shortest offset vector from the root node to that nodeAbsolute voxel grid position in input three-dimensional feature vector +.>Eigenvalue +.of voxel grid corresponding to original input three-dimensional eigenvector>Parent node of the previous level->。/>And->The method is used for positioning the corresponding voxel grid from the input three-dimensional feature vector, judging node expandability and legitimacy according to the properties of whether the voxel grid is empty, whether the voxel grid is out of range or not and the like, extracting the feature value of the voxel grid for calculating the root node feature value containing irregular three-dimensional topological structure information, and storing the local non-empty voxel grid three-dimensional topology through the father node.

Initializing local voxel grid tree root nodes: obtaining absolute voxel grid position of root node in input three-dimensional feature vector>Wherein x, y and z are integers and represent voxel grid subscripts; setting the shortest offset vector of the node Is->The method comprises the steps of carrying out a first treatment on the surface of the Reading the grid position from the input three-dimensional feature vector>Corresponding characteristic value->Setting the initial characteristic value corresponding to the node; parent node of the upper level->。

Initializing 6 primary nodes: shortest offset vector of 6 primary nodes +.>Respectively corresponding to the set->6 elements of (a); absolute voxel grid position from shortest offset vector and root node +.>Initializing absolute voxel grid position of each level node>The method comprises the steps of carrying out a first treatment on the surface of the Accordingly, the corresponding characteristic value is extracted from the original three-dimensional input characteristic vector to serve as the initial characteristic value of each level 6 node>If->The grid is empty, then +.>Initializing to 0; initializing a parent node of a level above 6 level nodes +.>The 6 primary nodes are added to the local non-empty voxel grid tree.

Traversing the 6 primary nodes, and completing the process for each primary nodeSearching of layer child nodes and storing shortest offset vector +.>Absolute voxel grid position +.>Initial characteristic value +.>Parent node of the upper level->Wherein->Representing node hierarchy (++>），/>Identification level->Is a node of the network. Traverse->All nodes of the hierarchy->Obtain->The steps of acquiring all nodes of the stage are as follows:

Judgment nodeWhether it is an expandable node, when->And->When the node is in the process, the node can be expanded;

when (when)For expandable node, according to the unit offset vector set +.>Preliminary acquisition by->Expanding the obtained child nodeThe subsections arePoint corresponds root node shortest offset vector +.>，/>Absolute voxel grid positionExtracting corresponding eigenvalue +.>If the corresponding location grid is empty, then +.>Initialized to 0, assigned->Parent node of the upper level->。

For a pair ofAnd expanding the obtained 6 child nodes to perform validity check one by one. Specifically, by->By->A unit offset vector->The resulting child node->The validity check of (2) is as shown in formula (2):

i.e. the child nodeThe absolute voxel grid position of (2) is in the space range of the three-dimensional input feature vector, and is not present in the upper node, the child node is legal, and the node is reserved as +.>Level node storing the corresponding +.>、/>、/>、If the child node does not meet the formula 2, the child node is an illegal node, and the node is deleted. In particular, the second criterion of equation 2 is mainly used to ensure that the current search path is the shortest offset path from the node to the root node.

TraversingGet->Level all node set +.>And updating the local voxel grid tree according to the father node relation stored in each node.

UpdatingIs->Repeating the above steps to complete->And constructing a local voxel grid tree of each hierarchy.

Through the operation, based on the node retrieval mode, a single voxel grid in the input three-dimensional feature vector is taken as a root node, the local three-dimensional structural feature of the point cloud data is obtained and used for updating the root node feature value, and local three-dimensional topological information is introduced into three-dimensional cross-correlation calculation.

Further, the invention aims at the unit offset vector in the construction process of the local non-empty voxel grid tree, comprehensively considers the anisotropic characteristic of the non-empty voxel grid distribution, introduces the unit offset vector expansion coefficient in the node retrieval process to realize the dynamic adjustment of the node spacing, expands the three-dimensional characteristic perception range and improves the calculation efficiency of three-dimensional convolution.

Specifically, before the step of determining the stored information corresponding to each node, the method includes:

an anisotropic unit offset vector set is obtained.

Further, the set of unit offset vectors is used to construct the local non-empty voxel grid tree, and the obtaining the set of anisotropic unit offset vectors includes:

Determining an adaptive expansion search coefficient;

Specifically, the unit offset vector expansion coefficient is obtained by the following formula:

Wherein,is a positive integer>。

Specifically, as shown in fig. 11, the following are included:

determining a maximum number of levels of a local non-empty voxel grid treeDetermining adaptive expansion search coefficient +.>(positive integer,/-Suo)>）。

Definition setStore up, down, left, right, front, back 6 directional unit move operations:

Reading local voxel grid tree root nodeAbsolute voxel grid position +.>。

Searching a set of motion vectors for an extensionElement->Searching for non-empty voxel grid distribution in the direction of movement based on +.>The number of non-empty voxel grids in the voxel grids determines the unit-shift-vector expansion coefficient +.>：

Where the Ceil () function represents a rounding up operation,the function is the dirac function, when +.>When empty, the person is left with->The function value was 1.

Repeating the operation d until the unit offset vector expansion coefficients in 6 directions are obtained, according to the above coefficientsCalculating to obtain the root node of the local voxel grid treeCorresponding 6 unit offset vector sets +.>：

Introducing unit offset vector expansion coefficients into node retrieval by using the steps, and calculating anisotropic unit offset vector set for root nodes represented by each voxel grid in the input three-dimensional feature vectorThe node distance is adaptively changed according to the distribution condition of the non-empty voxel grids, and the method is applied to the construction method of the local non-empty voxel grid tree proposed in section 2.2.1, so that the retrieval efficiency is ensured, and the characteristic sensing range of three-dimensional cross-correlation calculation is effectively expanded.

Step 2062, updating the root node characteristic value in the local non-empty voxel grid number;

updatingCharacteristic values of the individual root nodes.

Further, the updating the root node characteristic value in the local non-empty voxel grid number includes:

After constructing the local voxel grid tree, as shown in fig. 9, node feature values are constructedAnd (3) introducing the local three-dimensional topology into root node characteristic value calculation by designing a weight coefficient related to the hierarchy.

Specifically, the characteristic values of the nodes in different levels are calculated and updated step by step in a weighted accumulation mode according to the sequence from bottom to top. Let the current node be +.>Wherein->Representing the number of levels of the current node.

When (when)When child nodes exist, the characteristic value +.>Update according to equation 4, when +.>When there is no child node, then->And not updated.

Wherein:

a weight factor of each level is defined in equation 4 So that the calculated +.>Comprising local three-dimensional topological structure information, i.e. distance node +.>Child node pair with deeper hierarchical span ++>The smaller the effect of the result, the use of +.>The normalization processing of all the level weights is realized, and the characteristic values of the child nodes are avoided to be +.>Medium duty cycle imbalance:

used in equation 5Calculate and->Characteristic values associated with all child nodes are set to be +.>Number of level sharing nodes->Then->The calculation formula is as follows:

wherein the method comprises the steps ofIs a dirac function for realizing the screening of child nodes, i.e. from +.>Personal->Get +.>Is included in the set of sub-nodes.

Due to updatedIs prepared from->The characteristic values of the current node and the characteristic information of all the corresponding sub-nodes and the local three-dimensional topological structure information are calculated, and the characteristic values are used for three-dimensional cross-correlation calculation, so that the understanding capability of unstructured point cloud data is improved.

In step 2063, a key value is built to store the root node feature value.

Further, the storing the root node feature value by the build key value includes:

Further, the constructing a local non-empty voxel grid tree for each root node comprises:

Further, the constructing a local non-empty voxel grid tree for each voxel grid in the input three-dimensional feature vector covered by the current sliding three-dimensional convolution kernel by using a node searching mode includes: determining storage information corresponding to each node, wherein the storage information comprises a shortest offset vector from a root node to each current node, an absolute voxel grid position corresponding to the current node in the input three-dimensional feature vector, a feature value of a voxel grid corresponding to the input three-dimensional feature vector, and a parent node at the upper level, the shortest offset vector is used for determining a contribution weight of the feature value of the current node to the feature value of the root node, and the absolute voxel grid position in the input three-dimensional feature vector is used for extracting the feature value corresponding to the current node from the input three-dimensional vector; initializing a local voxel grid tree root node; initializing a primary node, wherein the primary node is a child node corresponding to the root node; traversing the primary nodes, and searching nodes for each primary node until the construction of the local non-empty voxel grid tree is completed.

Further, the initializing the local voxel grid tree root node comprises: acquiring an initial absolute voxel grid position of the root node in the input three-dimensional feature vector; setting the shortest offset vector corresponding to the root node as an initial vector value; and reading the characteristic value corresponding to the initial absolute voxel grid position from the input three-dimensional characteristic vector as an initial characteristic value.

Further, the primary node is a child node corresponding to the root node, and the initializing the primary node includes: the shortest offset vector of each level one node corresponds to one element in a set of unit offset vectors; initializing the absolute voxel grid position corresponding to each primary node according to the shortest offset vector and the initial absolute voxel grid position; extracting a corresponding characteristic value from the input three-dimensional characteristic vector as an initial characteristic value of each primary node; initializing the parent node of the previous level corresponding to the primary node as the root node; each of the primary nodes is added to a local non-empty voxel grid tree.

Further, traversing the primary nodes, and searching nodes for each primary node until the construction of the local non-empty voxel grid tree is completed, wherein the steps include:

Further, traversing all nodes corresponding to any level, and searching nodes for all nodes of a next level corresponding to any level until the construction of the local non-empty voxel grid tree is completed, wherein the steps include:

Further, the root node is a single voxel grid in the input three-dimensional feature vector.

It should be noted that, the above calculation result is stored in the data format by means of the key value, wherein the "key" stores the input channel indexThe absolute coordinates x, y and z of the voxel grid, and storing the corresponding characteristic value ++of the root node of the local non-empty voxel grid tree by the' value->. In the subsequent three-dimensional convolution operation, a large number of repeated calculations can be avoided by a method for directly reading the characteristic values, and the resource consumption of the method is greatly reduced.

Step 207, reading a target three-dimensional convolution kernel, wherein the target three-dimensional convolution kernel is determined based on the first channel number and the second channel number;

read the firstThree-dimensional convolution kernel, use +.>Representing the weight values stored at each position of the convolution kernel, sliding the convolution kernel along the x-axis, the y-axis and the z-axis respectively +.>、/>、/>Step, convolution kernel->The absolute voxel grid position in the three-dimensional input feature vector is +.>Then calculate +.>Three-dimensional convolution kernel and the firstDimension input three-dimensional feature vector at position +. >Three-dimensional cross-correlation calculation results at:

step 208, obtaining a three-dimensional cross-correlation calculation result of the input three-dimensional feature vector corresponding to the target three-dimensional convolution kernel and the first channel number corresponding to the target three-dimensional convolution kernel at a preset position, wherein the three-dimensional cross-correlation calculation result comprises a feature value of the output three-dimensional feature vector corresponding to the second channel number at the preset position;

further, as shown in fig. 4, step 208 includes:

step 2081, determining a local three-dimensional feature vector according to the size corresponding to the three-dimensional convolution kernel;

step 2082, setting each position in the local three-dimensional feature vector as a root node, and obtaining a root node index;

step 2083, according to the root node index and the three-dimensional convolution kernel, sliding a preset step number along three dimensions, and solving the absolute voxel grid position of the root node in the input three-dimensional feature vector;

step 2084, initializing a target root node according to the absolute voxel grid position in the input three-dimensional feature vector, and constructing a local non-empty voxel grid tree corresponding to the target root node;

step 2085, updating the characteristic value of the target root node in the local non-empty voxel grid tree corresponding to the target root node;

Step 2086, updating the root node index until the root node index reaches the size corresponding to the three-dimensional convolution kernel;

step 2087, updating the feature values contained in the voxel grids in the local input three-dimensional feature vectors according to the weight values stored in the three-dimensional convolution kernel in a weighting manner to obtain a three-dimensional cross-correlation calculation result;

in the steps 2081 to 2087, the three-dimensional convolution kernel size and the local input three-dimensional feature vector for three-dimensional cross-correlation calculation are determined, each position in the local input three-dimensional feature vector is used as a root node, the construction of the local non-empty voxel grid tree of the root node and the updating of the feature value are respectively completed, the feature value comprises the local three-dimensional topology, and the analysis capability of the three-dimensional cross-correlation calculation on the irregular point cloud data is improved.

Specifically, as shown in fig. 10, the following are included:

determining the size of a three-dimensional convolution kernel and performing three-dimensional cross-correlation calculation on the size of the three-dimensional convolution kernel and a local input three-dimensional feature vector: sizing a three-dimensional convolution kernelPosition in three-dimensional convolution kernel>Corresponds to a weight value->The three-dimensional convolution kernel slides on the input three-dimensional feature vector along the sequence from top to bottom, from left to right and from front to back, and in the sliding process, the three-dimensional convolution kernel carries out cross-correlation operation with the local input three-dimensional feature vector covered by the three-dimensional convolution kernel, and the position in the local input three-dimensional feature vector is >Corresponds to a characteristic value +.>。

Each position in the obtained local input three-dimensional feature vector is regarded as a root node, and a root node index is definedFor traversing the root node, index ∈>And->The conversion relation of (2) is:

in equation 8 "//" is a integer division operation, and "%" is a remainder operation.

According to the root node indexThree-dimensional convolution kernel slides step number +_ along three dimensions>Solving absolute voxel grid position of root node in original input three-dimensional feature vector>：

Based on absolute voxel grid position in original input three-dimensional feature vectorInitializing->A root node, constructing the +.1 according to the construction method of the local non-empty voxel grid tree set forth in 1)>And a local non-empty voxel grid tree corresponding to each root node.

According to the firstA local non-empty voxel grid tree corresponding to a root node according to the inclusion set forth in 2)Root node characteristic value calculating method of local three-dimensional topological information updates characteristic value corresponding to root node>According to index +.>And->Conversion relation of (1) is obtained->。

UpdatingRepeating the steps c-e until +.>。

And weighting the feature values contained in the voxel grids in the updated local input three-dimensional feature vectors by using the weight values stored in the three-dimensional convolution kernel to obtain a three-dimensional cross-correlation calculation result:

The three-dimensional cross-correlation calculation method realized by the steps integrates the point cloud data local three-dimensional topological structure information into the characteristic value of each voxel grid of the input three-dimensional characteristic vector, and has remarkable advantages in improving the understanding capability of unstructured data compared with the traditional three-dimensional cross-correlation calculation.

Step 209, accumulating the feature values corresponding to the input feature vectors with the number corresponding to the first channel number, and splicing the feature values corresponding to the output feature vectors with the number corresponding to the second channel number to obtain the feature values corresponding to the output three-dimensional feature vectors with the number corresponding to the second channel number at the preset position;

in particular, for accumulation ofCharacteristic values corresponding to the input characteristic vectors are spliced>The feature value corresponding to the output feature vector is obtained>Output three-dimensional feature vector of number of channels +.>Characteristic values corresponding to the positions:

step 210, outputting a feature value corresponding to the three-dimensional feature vector at the preset position according to the number of the second channels.

It should be noted that, in the above steps 203 to 210, the calculation flow corresponding to the three-dimensional convolution calculation layer is based on the adaptive node search.

Step 211, converting the target three-dimensional point cloud feature vector into a target two-dimensional point cloud feature vector;

and 212, inputting the target two-dimensional point cloud feature vector into a target detection neural network constructed based on two-dimensional convolution to obtain a detection result of a target object around the target vehicle.

It should be noted that, the step 211 is discussed with reference to the foregoing, and is not repeated here.

In addition, in the embodiment of the application, through a construction method of a local non-empty voxel grid tree, namely, a single voxel grid in an input three-dimensional feature vector is taken as a root node, local three-dimensional structural features of point cloud data are obtained based on a node retrieval mode, and a judgment rule of node expandability and legitimacy is introduced by planning a reasonable node information storage mode, so that node search efficiency is optimized, and redundant retrieval and calculation are avoided.

According to the root node characteristic value calculation method comprising the local three-dimensional topology information, a calculation strategy of a node characteristic value f is constructed according to a local voxel grid tree, and the local three-dimensional topology is introduced into root node characteristic value calculation through designing a weight coefficient related to a hierarchy.

By the three-dimensional cross-correlation calculation method, in the three-dimensional cross-correlation calculation, the characteristic value containing the local three-dimensional topological information of the point cloud data is used, and compared with the traditional calculation method, the understanding capability of unstructured data can be remarkably improved.

In addition, the invention provides a self-adaptive anisotropic local unit offset vector generation method, which is used for further three-dimensional cross-correlation calculation, introducing a unit offset vector expansion coefficient in node retrieval and being used for adapting to the anisotropy of point cloud data distribution, and effectively expanding the characteristic sensing range of the three-dimensional cross-correlation calculation while ensuring the retrieval efficiency.

In addition, the invention provides a high-efficiency three-dimensional convolution calculation layer which is based on self-adaptive node retrieval and supports multi-channel characteristic input and output and is used for meeting different characteristic abstraction requirements. And the feature values corresponding to the root nodes of the local non-empty voxel grid tree are pre-constructed and stored, so that the waste of hardware resources caused by repeated calculation is avoided.

The invention designs an automatic driving point cloud detection system, which utilizes a high-efficiency three-dimensional convolution calculation layer which is based on self-adaptive node retrieval and supports multi-channel characteristic input and output to realize high-efficiency extraction of three-dimensional information of point cloud data, is used for identifying objects around a vehicle, realizes accurate perception and understanding of an automatic driving automobile on a complex environment, and simultaneously saves the calculation power consumption of a vehicle-mounted platform.

Referring to fig. 5, fig. 5 shows an automatic driving point cloud detection device provided in an embodiment of the present application, where the device includes:

the acquisition module 501 is used for acquiring original point cloud data of a target vehicle;

the processing module 502 is configured to voxel the original point cloud data to obtain a voxel point cloud feature vector;

a construction module 503, configured to construct a three-dimensional convolutional neural network block according to a pre-generated three-dimensional convolutional calculation layer, where the pre-generated three-dimensional convolutional calculation layer is a three-dimensional convolutional calculation layer constructed based on adaptive node retrieval and local voxel grid tree;

The input module 504 is configured to input the voxelized point cloud feature vector to the three-dimensional convolutional neural network block, to obtain a target three-dimensional point cloud feature vector;

the conversion module 505 is configured to convert the target three-dimensional point cloud feature vector into a target two-dimensional point cloud feature vector;

the detection module 506 is configured to input the target two-dimensional point cloud feature vector into a target detection neural network constructed based on two-dimensional convolution, so as to obtain a detection result of a target object around the target vehicle.

According to the automatic driving point cloud detection device, original point cloud data of a target vehicle are collected; voxelized processing is carried out on the original point cloud data to obtain voxelized point cloud feature vectors; constructing a three-dimensional convolutional neural network block according to a three-dimensional convolutional calculation layer which is generated in advance; inputting the voxelized point cloud feature vector into the three-dimensional convolutional neural network block to obtain a target three-dimensional point cloud feature vector; converting the target three-dimensional point cloud feature vector into a target two-dimensional point cloud feature vector; and inputting the target two-dimensional point cloud feature vector into a target detection neural network constructed based on two-dimensional convolution to obtain a detection result of target objects around the target vehicle. According to the embodiment of the application, the original point cloud data are processed to obtain the voxelized point cloud feature vector, the voxelized point cloud feature vector is input to the three-dimensional convolution calculation layer based on the pre-generation to construct the three-dimensional convolution neural network block, so that the efficient extraction of the three-dimensional information of the point cloud data is realized by using the efficient three-dimensional convolution calculation layer based on the self-adaptive node retrieval and supporting the multi-channel feature input and output, the efficient three-dimensional convolution calculation layer is used in the recognition task of the objects around the vehicle, the accurate perception and understanding of the automatic driving automobile on the complex environment are realized, and meanwhile, the calculation power consumption of the vehicle-mounted platform is saved.

The embodiment of the present application also provides a communication device, as shown in fig. 6, including a processor 601, a communication interface 602, a memory 603, and a communication bus 604, where the processor 601, the communication interface 602, and the memory 603 perform communication with each other through the communication bus 904,

a memory 603 for storing a computer program;

the processor 601, when executing the program stored in the memory 603, may implement the following steps:

collecting original point cloud data of a target vehicle;

The communication bus mentioned by the above terminal may be a peripheral component interconnect standard (Peripheral Component Interconnect, abbreviated as PCI) bus or an extended industry standard architecture (Extended Industry Standard Architecture, abbreviated as EISA) bus, etc. The communication bus may be classified as an address bus, a data bus, a control bus, or the like. For ease of illustration, the figures are shown with only one bold line, but not with only one bus or one type of bus.

The communication interface is used for communication between the terminal and other devices.

The memory may include random access memory (Random Access Memory, RAM) or non-volatile memory (non-volatile memory), such as at least one disk memory. Optionally, the memory may also be at least one memory device located remotely from the aforementioned processor.

The processor may be a general-purpose processor, including a central processing unit (Central Processing Unit, CPU for short), a network processor (Network Processor, NP for short), etc.; but also digital signal processors (Digital Signal Processing, DSP for short), application specific integrated circuits (Application Specific Integrated Circuit, ASIC for short), field-programmable gate arrays (Field-Programmable Gate Array, FPGA for short) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

In yet another embodiment provided herein, there is also provided a computer readable storage medium having instructions stored therein that, when run on a computer, cause the computer to perform the automatic driving point cloud detection of any of the above embodiments.

In yet another embodiment provided herein, there is also provided a computer program product containing instructions that, when run on a computer, cause the computer to perform the automatic driving point cloud detection of any of the above embodiments.

In the above embodiments, it may be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loaded and executed on a computer, produces a flow or function in accordance with embodiments of the present application, in whole or in part. The computer may be a general purpose computer, a special purpose computer, a computer network, or other programmable apparatus. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from one website, computer, server, or third database to another website, computer, server, or third database by a wired (e.g., coaxial cable, fiber optic, digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The computer readable storage medium may be any available medium that can be accessed by a computer or a data storage device including one or more servers, third databases, etc. that can be integrated with the available medium. The usable medium may be a magnetic medium (e.g., floppy Disk, hard Disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid State Disk (SSD)), etc.

It is noted that relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.

In this specification, each embodiment is described in a related manner, and identical and similar parts of each embodiment are all referred to each other, and each embodiment mainly describes differences from other embodiments. In particular, for system embodiments, since they are substantially similar to method embodiments, the description is relatively simple, as relevant to see a section of the description of method embodiments.

The foregoing description is only of the preferred embodiments of the present application and is not intended to limit the scope of the present application. Any modifications, equivalent substitutions, improvements, etc. that are within the spirit and principles of the present application are intended to be included within the scope of the present application.

Claims

1. An automatic driving point cloud detection method, which is characterized by being applied to an automatic driving system, comprises the following steps:

collecting original point cloud data of a target vehicle;

inputting the target two-dimensional point cloud feature vector into a target detection neural network constructed based on two-dimensional convolution to obtain a detection result of target objects around the target vehicle;

Inputting the voxelized point cloud feature vector to the three-dimensional convolutional neural network block, and obtaining a target three-dimensional point cloud feature vector comprises:

2. The method for detecting an autopilot point cloud according to claim 1, wherein the obtaining the three-dimensional cross-correlation calculation result of the input three-dimensional feature vector corresponding to the target three-dimensional convolution kernel and the first channel number corresponding to the target three-dimensional convolution kernel at the preset position includes:

3. The method of claim 1, wherein traversing voxel grids in an input channel corresponding to a first channel number in the input three-dimensional feature vector, constructing key-value pairs to store root node feature values calculated from a local non-empty voxel grid tree comprises:

constructing a local non-empty voxel grid tree for each root node;

4. The automatic driving point cloud detection method of claim 3, wherein the storing of the root node feature value by the build key value comprises:

5. The method of automatic driving point cloud detection as claimed in claim 3, wherein said constructing a local non-empty voxel grid tree for each root node comprises:

6. The method for detecting an autopilot cloud of claim 5 wherein constructing a local non-empty voxel grid tree for each voxel grid in the input three-dimensional feature vector covered by the current sliding three-dimensional convolution kernel using a node search method comprises:

initializing a local voxel grid tree root node;

7. The method of automatic driving point cloud detection as recited in claim 6, wherein initializing local voxel grid tree root nodes comprises:

8. The method for detecting an autopilot cloud of claim 6 wherein the primary node is a child node corresponding to the root node, the initializing the primary node comprising:

each of the primary nodes is added to a local non-empty voxel grid tree.

9. The method of automatic driving point cloud detection according to claim 6, wherein traversing the primary nodes, and performing node search for each primary node until construction of a local non-empty voxel grid tree is completed comprises:

10. The method for detecting the cloud of the automatic driving point according to claim 9, wherein traversing all nodes corresponding to any level, and searching nodes for all nodes of a next level corresponding to any level until the construction of the local non-empty voxel grid tree is completed comprises:

11. The method of claim 3, wherein updating the root node feature value in the local non-empty voxel grid number comprises:

12. The automatic driving point cloud detection method according to claim 6, wherein before the step of determining the stored information corresponding to each node, the method includes:

an anisotropic unit offset vector set is obtained.

13. The method of automatic driving point cloud detection of claim 12, wherein the set of unit offset vectors is used to construct the local non-empty voxel grid tree, the obtaining the set of anisotropic unit offset vectors comprising:

determining an adaptive expansion search coefficient;

14. The automatic driving point cloud detection method of claim 13, wherein the unit offset vector expansion coefficient is obtained by the following formula:

wherein,representing the unit offset vector expansion coefficient, the Ceil function represents the rounding up operation, ++ >The function is the dirac function, when +.>When empty, the person is left with->The function value is 1, K represents the maximum level number of the local non-empty voxel grid tree, and E represents the adaptive expansion search coefficient.

15. The automatic driving point cloud detection method according to claim 1, wherein the constructing a three-dimensional convolutional neural network block from a three-dimensional convolutional calculation layer generated in advance includes:

16. The automatic driving point cloud detection method according to claim 1, wherein the data format corresponding to the three-dimensional point cloud feature vector isWherein->Representing the number of channels of the output three-dimensional feature, < >>Representing a uniform division of the three-dimensional space into +.>A voxel grid;

collecting original point cloud data of a target vehicle through a laser radar, wherein the method comprises the following steps ofThe data format corresponding to the original point cloud data is that。

17. The method for detecting an autopilot point cloud according to claim 1, wherein the inputting the target two-dimensional point cloud feature vector into a target detection neural network constructed based on two-dimensional convolution, the obtaining a detection result of a target object around a target vehicle comprises:

18. An autopilot point cloud detection apparatus, characterized by being applied to a baseboard management controller, the autopilot point cloud detection apparatus comprising:

The detection module is used for inputting the target two-dimensional point cloud feature vector into a target detection neural network constructed based on two-dimensional convolution to obtain a detection result of a target object around the target vehicle;

19. A communication device, comprising: a transceiver, a memory, a processor, and a program stored on the memory and executable on the processor;

the processor is configured to read a program in the memory to implement the automatic driving point cloud detection method according to any one of claims 1 to 17.

20. A readable storage medium storing a program, wherein the program when executed by a processor implements the automatic driving point cloud detection method according to any one of claims 1 to 17.