CN111768493B - Point cloud processing method based on distribution parameter coding - Google Patents
Point cloud processing method based on distribution parameter coding Download PDFInfo
- Publication number
- CN111768493B CN111768493B CN202010575561.XA CN202010575561A CN111768493B CN 111768493 B CN111768493 B CN 111768493B CN 202010575561 A CN202010575561 A CN 202010575561A CN 111768493 B CN111768493 B CN 111768493B
- Authority
- CN
- China
- Prior art keywords
- point cloud
- neural network
- distribution
- cloud data
- parameters
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Software Systems (AREA)
- General Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Computing Systems (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Biomedical Technology (AREA)
- Molecular Biology (AREA)
- Biophysics (AREA)
- General Engineering & Computer Science (AREA)
- Artificial Intelligence (AREA)
- Mathematical Physics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Health & Medical Sciences (AREA)
- Computer Graphics (AREA)
- Geometry (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a point cloud processing method based on distribution parameter coding, which expresses the probability density function of current point cloud distribution by training a neural network to fit the distribution state of the point cloud, takes the parameters at the moment as the characteristics expressing the current point cloud data, and successfully compresses a large amount of point cloud data to the magnitude of a parameter space. Meanwhile, aiming at the condition of continuously obtaining the point cloud, the invention can take the point cloud parameters as the output of a circulation unit in the neural network, thereby predicting the transformation trend of the point cloud data distribution along with the change of the external environment.
Description
Technical Field
The invention belongs to the technical field of point cloud data compression processing, and particularly relates to a point cloud processing method based on distribution parameter coding.
Background
With the rapid development of three-dimensional data acquisition equipment, it becomes more convenient and faster to describe a three-dimensional object and reproduce a three-dimensional scene by using a 3D point cloud. The 3D point cloud is a new data format and is used for recording and representing the surface information of the three-dimensional object; point cloud data is a collection of points in space that contain three-dimensional coordinates and one or more attribute information, such as color, normal vector, reflection intensity, etc. Compared with other three-dimensional data formats, the point cloud has the advantages of convenience in acquisition, simplicity in processing and the like, and therefore, the point cloud is widely applied to various emerging fields such as Augmented Reality (AR), automatic driving, 3D printing and the like.
The current common point cloud processing methods include the following three types:
(1) discretizing the received point cloud position into three-dimensional voxels by means of discretization, representing the current point cloud state by using the number or density of points in each voxel in a three-dimensional space, and then processing the point cloud state by using a 3D convolutional neural network or by means of clustering.
(2) Directly using 3D radar data as the input of a neural network, namely directly using a structure similar to PointNet and using a pooling operation to solve the problem of overlarge point cloud number.
(3) Projecting the three-dimensional point cloud onto a two-dimensional grid space; methods such as spherical mapping (spherical Map), Camera-Plane Map (CPM), Bird's-Eye View (BEV) and the like are generally used, and then a convolutional neural network is used to extract features from the projected two-dimensional image.
The first of these schemes uses conventional statistical methods, which increase the amount of computation in larger spaces or when the number of dimensions of the processed data is higher. The second method uses pooling operation to process information contained in the point cloud, and many pieces of point cloud information which are not transferred can be lost in the process, and the method is mainly used for classifying and segmenting the object represented by the point cloud data; in such a scenario, local features are usually determined by extreme values and edge values, and the loss of information of non-maximum values has a small influence on the final judgment. The third scheme uses a projection mode to project three-dimensional information to two dimensions, so that part of information in a three-dimensional space is lost.
In a specific scenario, the point cloud data may be used to represent the distribution of objects around the collector or their features, and the above-mentioned centralized approach may not be able to completely find the surrounding situation because part of the information is lost. For example, in the process of automatic driving, the acquired point cloud data is usually the surrounding obstacles, each point in the data has a small practical meaning, but the distribution of the whole point cloud data indicates the distribution condition of the obstacles around the current vehicle. In some cases, point cloud data under continuous time can be obtained, the change of the distribution of the point cloud data represents the change of obstacles around the vehicle, the point cloud distribution change is smaller between continuous sampling samples, and if the distribution can be represented by using parameters, the change of the parameters is more stable.
The point cloud data is represented by using the distribution parameters, so that the discretization step can be skipped, continuous data can be obtained, and meanwhile, the global information and the three-dimensional information of the point cloud data including special points can be kept; however, it also has some disadvantages: firstly, the calculation amount is large, and a neural network needs to be trained according to current point cloud data; secondly, it cannot guarantee the liphoz continuity of point cloud data on the parameter space; finally, the parameters used to represent the point cloud distribution are less interpretable.
Disclosure of Invention
In view of the above, the present invention provides a point cloud processing method based on distribution parameter coding, which uses a neural network model to fit the distribution of current point cloud data, uses model parameters to represent the distribution parameters of the current point cloud state distribution, and directly uses a large amount of point cloud data to represent the distribution parameters with less dimensions, so as to achieve the effect of compressing the data to the parameter space thereof, thereby facilitating the subsequent use.
A point cloud processing method based on distribution parameter coding specifically comprises the following steps: the method comprises the steps of representing a probability density function of point cloud distribution by using a neural network, taking real point cloud data acquired in the prior art as a positive sample, taking point cloud data generated by random sampling in an environment as a negative sample, training the neural network by using a large number of positive and negative samples, and taking model parameters of the trained neural network as distribution parameters (codes) of the current point cloud distribution state.
Further, when the negative sample point cloud data is generated by random sampling, the uniform distribution of the value range of each dimension of the positive sample point cloud data can be used as the sampling probability, or the normal distribution of the mean value and the standard deviation can be used as the sampling probability by calculating the mean value and the standard deviation of the positive sample point cloud data.
Further, the neural network adopts a three-layer fully-connected neural network, cross entropy is used as a loss function of training, and a random gradient descent method is used for training the neural network.
Further, since the point cloud data does not only contain the current position information thereof, when the neural network is trained, other attribute information (such as color, reflection intensity, etc.) of the point cloud data is used as an additional output of the neural network, wherein the other attribute information of the negative sample is set to 0.
Furthermore, because the collected and obtained real point cloud data is influenced by the external environment, relevant external environment variables are input into a network structure with a gating cycle unit (GRU), and the output of the network structure is embedded into a neural network to predict the change rule of point cloud coding parameters under the action of the external environment.
Furthermore, the output of the network structure with the gating cycle unit is used as a weight coefficient of the first layer of the neural network, and the weight coefficient is directly subjected to matrix multiplication with the point cloud data.
Further, the output of the network structure with the gating cycle unit and the point cloud data input by the neural network are projected to the N dimension to carry out element-wise multiplication, and then the result is input into the neural network structure.
Furthermore, during training, the loss is calculated through the neural network and is transmitted to the network structure with the gate control cycle unit, and parameters of the two networks are updated simultaneously, so that the distribution parameters of the point cloud data at each moment and the variation trend of the parameters under external input can be trained simultaneously; according to the network structure, the future values of the point cloud coding parameters under the condition of given external input can be predicted, so that the distribution function of the point cloud at the future time can be predicted.
Although the point cloud data expressed by the distribution parameters has certain errors and cannot completely express the real point cloud distribution, the characteristics can be effectively extracted when the required information is the distribution state of the point cloud and the attributes thereof, and the data volume is reduced. Therefore, the invention has the following beneficial technical effects:
1. the invention uses a neural network to fit the distribution function of the point cloud and serializes the discrete data.
2. The invention uses the neural network parameters as the encoding parameters of the point cloud, and compresses the point cloud data.
3. The method can predict the change trend of the point cloud distribution by using structures such as a cyclic neural network and the like under the environment of continuously acquiring the point cloud data, thereby predicting the future distribution of the point cloud.
Drawings
FIG. 1 is a schematic view of a process of encoding point cloud distribution parameters according to the present invention.
Fig. 2 is a schematic diagram of a multi-output neural network structure.
FIG. 3 is a schematic diagram of a neural network structure for processing continuous point cloud data.
Fig. 4 is a schematic diagram of a network structure combining point cloud parameters and point cloud data.
FIG. 5 is a schematic diagram of another network structure combining point cloud parameters and point cloud data.
Detailed Description
In order to more specifically describe the present invention, the following detailed description is provided for the technical solution of the present invention with reference to the accompanying drawings and the specific embodiments.
The invention is based on the point cloud processing method of the distribution parameter coding, can represent the point cloud Data dynamic distribution in real time, can also compress the point cloud Data that exists, this method is through importing the position Data [ x, y, z ] of the point cloud of the present moment at first, presume the random point cloud distribution of environment of the present moment is Pr ([ x, y, z ]), produce the random point cloud Data of environment of certain quantity according to this probability distribution function at random, print the label 1 to the point cloud Data imported before, print the label 0 to the point cloud Data that is produced from the environment at random; and then training a neural network with a given structure to serve as a classifier of the two types of data. And outputting the trained parameters as the parameter representation of the distribution of the current point cloud.
When generating the point cloud distribution, the uniform distribution of the value range of each dimension of the current point cloud can be used as the sampling probability:
the sample mean value mu and standard deviation sigma can also be calculated from all point cloud data, and normal distribution Pr ([ x, y, z)]) N (μ, σ) as the sampling probability.
In addition, the point cloud data not only includes the current position information, but also includes other data such as color, reflection intensity, etc.; for them, output nodes can be added at the output layer of the neural network to fit these data as well.
In some cases, the acquisition of the point cloud data is chronological, such as point cloud data acquired using LiDAR on an autonomous vehicle. In this case, a network structure (referred to as a network 2) having a cyclic unit, such as a cyclic neural network, which outputs parameters of a neural network (referred to as a network 1) that classifies point cloud data using parameters, with the input of a factor that can cause a change in the point cloud distribution, may be used. During training, the loss is calculated by the network 1 and is transmitted to the network 2, and the parameters of the two networks are updated. Therefore, the distribution parameters of the point cloud data at each moment and the variation trend of the parameters under external input can be trained simultaneously, and the future value of the point cloud coding parameters under the condition of given external input can be predicted according to the structure, so that the distribution function of the point cloud at the future moment can be predicted.
The following technical scheme can be implemented by aiming at the problem that the point cloud data acquired by a vehicle is acquired and processed when the vehicle is driven on a CARLA platform, and the specific steps are shown in FIG. 1:
step 1: simulation experiments were performed on the cara platform, with LIDAR sensors set on the control vehicle. The set position is [2.0,0.0,1.4 ] based on the vehicle position]The pitch angle is-15 degrees, and the yaw angle and the roll angle are both set to be 0 degree; automatically driving the vehicle, collecting point cloud data in real time, wherein the sampling length of each section is 640 frames, 2500 point cloud samples are collected in each frame, and the obtained point cloud data is { [ x ] 1 ,y 1 ,z 1 ]…[x n ,y n ,z n ]And setting the corresponding label as 1.
Step 2: calculating the sample mean value mu and standard deviation sigma of the collected point cloud data, using normal distribution N (mu, sigma) as the collection distribution of the negative sample, and using the data collected from the negative sample { [ x ] 1 ′,y 1 ′,z 1 ′]…[x n ′,y n ′,z n ′]The label of 0, a uniform distribution can also be used as the distribution of negative samples.
And step 3: setting the current neural network structure as a full connection layer neural network of [128, 1] three layers, and adopting the calculation cross entropy as a loss function for the data in the step 1 and the step 2 as follows:
and then training a neural network by using random gradient descent, and updating network parameters so that the network is used as a classifier F of the data in the step 1 and the step 2 θ (x,y,z)。
And 4, step 4: the sampling process of step 2 and the training process of step 3 can be repeated without changing the neural network structure.
And 5: and outputting the neural network parameter theta as a representation parameter of the current point cloud distribution.
When the point cloud contains other attributes, the structure of fig. 2 can be used, and the neural network fits all attribute data given currently while outputting and judging whether the data belongs to point cloud distribution; the values of the negative samples on the characteristics can be set to be zero, and normal distribution of corresponding attributes can be used to ensure the consistency of the distributed data. In specific implementation, the first dimension of the neural network outputs the probability that the current data belongs to the real point cloud data, the other dimensions output the predicted values of the corresponding features, and for the first dimension, the Loss function still uses the cross entropy Loss 1 (ii) a For other data such as class characteristics, Softmax Loss may be used as a Loss Loss i Continuous data such as illumination intensity, the square error can be used as Loss i . Finally, different weights w can be set for different characteristics i To obtain the final loss
Replacing the Loss function of step 3 in the above step with Loss total The task of extracting distribution characteristics when the point cloud contains other attributes can be solved.
When the point cloud can be continuously acquired in a time sequence, a gate control loop unit GRU with an input as an external variable and an output as a judgment neural network parameter can be trained by using the structure shown in fig. 3, wherein the external variable is determined by the acquisition condition of the point cloud data. In this embodiment, when the collection source of the point cloud is LiDAR on an autonomous vehicle, the input of the gate control cycle unit is set to the current speed, acceleration, direction, angular velocity, etc. of the autonomous vehicle.
When the point cloud can be continuously obtained in time series, the output parameters of the cyclic unit need to be embedded as the parameters of the network 1 to increase the prediction of the change of the distribution parameters, and there are various ways to accomplish this function. The first method is as follows: as shown in fig. 4, the output point cloud parameters of the cyclic unit may be used as the weight coefficients of the first layer of the network 1, and the point cloud parameters and the point cloud data may be directly subjected to matrix multiplication. The second method comprises the following steps: or, as shown in fig. 5, the point cloud data and the point cloud parameters output by the circulation unit are projected to the N-dimension to perform element-wise multiplication, and then the result is input to the network. Besides the above, there are other means to integrate the point cloud parameters output by the cyclic unit into the first discriminant network, and these solutions conform to the description of this specification, and the implementation is not listed here.
In a reinforcement learning task with point cloud as modal data input, the distribution parameters of the current distribution of the point cloud can be directly used as characteristics to train the whole strategy network. Optionally, in the reinforcement learning task, the network parameters may be continuously fine-tuned and updated according to the loss of the point cloud classifier.
In summary, the invention uses the distribution parameters as the codes of the point cloud data by learning the distribution probability function parameters of the point cloud, thereby compressing a large amount of point cloud data to the parameter space, and meanwhile, under the condition that the point cloud data can be continuously obtained, the variation trend of the parameters under the environmental influence can be trained by the circulation units such as the GRU, etc., thereby predicting the variation trend of the distribution parameters of the point cloud distribution.
The embodiments described above are intended to facilitate one of ordinary skill in the art in understanding and using the invention. It will be readily apparent to those skilled in the art that various modifications to the above-described embodiments may be made, and the generic principles defined herein may be applied to other embodiments without the use of inventive faculty. Therefore, the present invention is not limited to the above embodiments, and those skilled in the art should make improvements and modifications to the present invention based on the disclosure of the present invention within the protection scope of the present invention.
Claims (6)
1. A point cloud processing method based on distribution parameter coding is characterized in that: representing a probability density function of point cloud distribution by using a neural network, taking real point cloud data acquired in the prior art as a positive sample, taking point cloud data generated by random sampling in an environment as a negative sample, training the neural network by using a large number of positive and negative samples, and taking model parameters of the trained neural network as distribution parameters of the current point cloud distribution state;
because the collected and obtained real point cloud data is influenced by the external environment, relevant external environment variables are input into a network structure with a gating cycle unit, and the output of the network structure is embedded into a neural network to predict the change rule of point cloud coding parameters under the action of the external environment;
during training, calculating loss through a neural network, transmitting the loss to a network structure with a gating circulation unit, and updating parameters of the two networks simultaneously, so that the distribution parameters of point cloud data at each moment and the variation trend of the parameters under external input can be trained simultaneously; according to the network structure, the future values of the point cloud coding parameters under the condition of given external input can be predicted, so that the distribution function of the point cloud at the future time can be predicted.
2. The point cloud processing method of claim 1, wherein: when the negative sample point cloud data is generated by random sampling, the uniform distribution of the value range of each dimension of the positive sample point cloud data can be used as the sampling probability, or the normal distribution of the mean value and the standard deviation can be used as the sampling probability by calculating the mean value and the standard deviation of the positive sample point cloud data.
3. The point cloud processing method of claim 1, wherein: the neural network adopts a three-layer fully-connected neural network, cross entropy is used as a loss function of training, and a random gradient descent method is used for training the neural network.
4. The point cloud processing method of claim 1, wherein: because the point cloud data does not only contain the current position information of the point cloud data, when the neural network is trained, other attribute information of the point cloud data including color, normal vector and reflection intensity is used as the additional output of the neural network, wherein the other attribute information of the negative sample is set to be 0.
5. The point cloud processing method of claim 1, wherein: and taking the output of the network structure with the gated cyclic unit as a weight coefficient of a first layer of the neural network, and directly performing matrix multiplication on the weight coefficient and the point cloud data.
6. The point cloud processing method of claim 1, wherein: and projecting the output of the network structure with the gate control circulation unit and the point cloud data input by the neural network to the N dimension to carry out element-wise multiplication, and inputting the result into the neural network structure.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010575561.XA CN111768493B (en) | 2020-06-22 | 2020-06-22 | Point cloud processing method based on distribution parameter coding |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010575561.XA CN111768493B (en) | 2020-06-22 | 2020-06-22 | Point cloud processing method based on distribution parameter coding |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111768493A CN111768493A (en) | 2020-10-13 |
| CN111768493B true CN111768493B (en) | 2022-08-05 |
Family
ID=72721460
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010575561.XA Active CN111768493B (en) | 2020-06-22 | 2020-06-22 | Point cloud processing method based on distribution parameter coding |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111768493B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112528016B (en) * | 2020-11-19 | 2024-05-07 | 重庆兆光科技股份有限公司 | Text classification method based on low-dimensional spherical projection |
Family Cites Families (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110321910B (en) * | 2018-03-29 | 2021-05-28 | 中国科学院深圳先进技术研究院 | Point cloud-oriented feature extraction method, device and equipment |
| DE102018210937A1 (en) * | 2018-07-03 | 2020-01-09 | Robert Bosch Gmbh | Realistic simulation of physical measurement data |
| CN110246112B (en) * | 2019-01-21 | 2021-04-30 | 厦门大学 | Laser scanning SLAM indoor three-dimensional point cloud quality evaluation method based on deep learning |
| CN110442917B (en) * | 2019-07-09 | 2023-06-20 | 武汉工程大学 | Parameterized model reconstruction method based on point cloud |
| CN110689576B (en) * | 2019-09-29 | 2023-04-07 | 桂林电子科技大学 | Automatic ware-based dynamic 3D point cloud normal distribution AGV positioning method |
| CN111062343B (en) * | 2019-12-20 | 2023-04-18 | 东南大学 | Three-dimensional human ear recognition method based on average curvature and normal distribution transformation |
| CN111199206A (en) * | 2019-12-30 | 2020-05-26 | 上海眼控科技股份有限公司 | Three-dimensional target detection method and device, computer equipment and storage medium |
-
2020
- 2020-06-22 CN CN202010575561.XA patent/CN111768493B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN111768493A (en) | 2020-10-13 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109410307B (en) | Scene point cloud semantic segmentation method | |
| CN110472483B (en) | SAR image-oriented small sample semantic feature enhancement method and device | |
| CN111832655B (en) | Multi-scale three-dimensional target detection method based on characteristic pyramid network | |
| US11922569B2 (en) | Generating realistic point clouds | |
| CN110766063B (en) | Image classification method based on compressed excitation and tightly connected convolutional neural network | |
| CN111862101A (en) | 3D point cloud semantic segmentation method under aerial view coding visual angle | |
| CN113052106B (en) | Airplane take-off and landing runway identification method based on PSPNet network | |
| CN117157678A (en) | Method and system for graph-based panorama segmentation | |
| CN112950780B (en) | Intelligent network map generation method and system based on remote sensing image | |
| CN115861619A (en) | Airborne LiDAR (light detection and ranging) urban point cloud semantic segmentation method and system of recursive residual double-attention kernel point convolution network | |
| CN115082885A (en) | Point cloud target detection method, device, equipment and storage medium | |
| DE102022100360A1 (en) | MACHINE LEARNING FRAMEWORK APPLIED IN A SEMI-SUPERVISED SETTING TO PERFORM INSTANCE TRACKING IN A SEQUENCE OF IMAGE FRAMES | |
| CN111768493B (en) | Point cloud processing method based on distribution parameter coding | |
| CN116681895A (en) | Method, system, equipment and medium for segmenting airplane grid model component | |
| CN115147798A (en) | Method, model and device for predicting travelable area and vehicle | |
| CN113536920B (en) | Semi-supervised three-dimensional point cloud target detection method | |
| Kulkarni et al. | Task-driven compression for collision encoding based on depth images | |
| CN110348311B (en) | Deep learning-based road intersection identification system and method | |
| CN112132207A (en) | Target detection neural network construction method based on multi-branch feature mapping | |
| CN116051561A (en) | Lightweight pavement disease inspection method based on vehicle-mounted edge equipment | |
| CN115249269A (en) | Object detection method, computer program product, storage medium, and electronic device | |
| CN113837215B (en) | Point cloud semantic and instance segmentation method based on conditional random field | |
| DUMAN et al. | YOLOv5-based Vehicle Objects Detection Using UAV Images | |
| CN117252821A (en) | Image detection method and device, storage medium and electronic equipment | |
| CN117053804A (en) | Map construction method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |