CN114782672B - Point cloud pose adjustment method and device - Google Patents
Point cloud pose adjustment method and device Download PDFInfo
- Publication number
- CN114782672B CN114782672B CN202210374345.8A CN202210374345A CN114782672B CN 114782672 B CN114782672 B CN 114782672B CN 202210374345 A CN202210374345 A CN 202210374345A CN 114782672 B CN114782672 B CN 114782672B
- Authority
- CN
- China
- Prior art keywords
- point cloud
- matrix
- processed
- rotation
- center
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 173
- 239000011159 matrix material Substances 0.000 claims abstract description 119
- 238000004364 calculation method Methods 0.000 claims abstract description 37
- 238000012545 processing Methods 0.000 claims abstract description 19
- 239000013598 vector Substances 0.000 claims description 34
- 238000004590 computer program Methods 0.000 claims description 16
- 238000000605 extraction Methods 0.000 claims description 8
- 230000008569 process Effects 0.000 abstract description 10
- 238000002474 experimental method Methods 0.000 description 21
- 238000012549 training Methods 0.000 description 17
- 230000000694 effects Effects 0.000 description 15
- 238000012360 testing method Methods 0.000 description 10
- 230000011218 segmentation Effects 0.000 description 7
- 230000008901 benefit Effects 0.000 description 6
- 238000004422 calculation algorithm Methods 0.000 description 5
- 238000004891 communication Methods 0.000 description 5
- 238000010586 diagram Methods 0.000 description 5
- 238000013519 translation Methods 0.000 description 5
- 238000012937 correction Methods 0.000 description 4
- 238000011156 evaluation Methods 0.000 description 4
- 238000013527 convolutional neural network Methods 0.000 description 3
- 230000007547 defect Effects 0.000 description 3
- 238000000513 principal component analysis Methods 0.000 description 3
- 230000003044 adaptive effect Effects 0.000 description 2
- 238000004141 dimensional analysis Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 230000010354 integration Effects 0.000 description 2
- 238000002372 labelling Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000005457 optimization Methods 0.000 description 2
- 230000004044 response Effects 0.000 description 2
- 238000005070 sampling Methods 0.000 description 2
- 238000012952 Resampling Methods 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 239000002243 precursor Substances 0.000 description 1
- 238000012847 principal component analysis method Methods 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Computing Systems (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Life Sciences & Earth Sciences (AREA)
- Molecular Biology (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Image Processing (AREA)
- Image Analysis (AREA)
Abstract
The invention provides a point cloud pose adjusting method and device. The method comprises the following steps: acquiring a point cloud to be processed, performing center zeroing operation on the point cloud to be processed, and determining a point cloud origin of the point cloud to be processed after center zeroing; extracting the distance from each point in the point cloud to be processed to the point cloud origin as a one-dimensional rotation invariant feature after the center is zeroed; performing weighted calculation on the point cloud to be processed after the center is zeroed to obtain a weighted result; wherein the weight of the weighted calculation is calculated according to the one-dimensional rotation invariant feature; carrying out standard orthogonalization processing on the weighted result to obtain a rotation matrix of the point cloud to be processed after the center is zeroed; and based on the rotation matrix, carrying out pose adjustment on the point cloud to be processed after the center is zeroed. According to the technical scheme, the point cloud which is not aligned is aligned, the calculated amount is small in the position and posture adjustment process of the point cloud, and the point cloud after the alignment has strict rotation invariance.
Description
Technical Field
The invention relates to the technical field of artificial intelligence, in particular to a point cloud pose adjusting method and device.
Background
Some features of the point cloud are independent of the direction of the point cloud, a property known as rotational invariance of the point cloud. For example, a point cloud describing a cup, however rotated, is still described as a cup. Many existing point cloud processing methods, while having good results on the aligned point clouds, tend to severely degrade their performance once the direction of the point clouds changes. How to deal with rotational invariance of point clouds becomes a key issue.
Existing methods can be broadly divided into three general categories: multi-view methods, spherical convolution methods, and rotation invariant methods.
The multi-view method can only obtain rotational invariance in one discrete rotation group, in other words it does not have strict rotational invariance on SO (3), SO robustness cannot be guaranteed in more general scenarios.
In the spherical convolution method, the calculation of convolution needs to calculate a numerical integral on the spherical surface, and the calculation of the numerical integral needs to disperse the spherical surface, so that the spherical surface has unavoidable numerical errors and does not have strict rotation invariance in theory; in addition, to reduce the numerical error, it can be achieved only by increasing the calculation amount.
There are many of the rotation invariant methods, which have different disadvantages: some methods lose part of the structural information while calculating the rotation invariants, and the part of the information plays an important role in downstream tasks; some methods do not have theoretical strict rotation invariance, and still require a training phase to perform a great deal of data enhancement to improve the performance of the algorithm; the method for fixing the regular direction based on principal component analysis has ambiguity naturally, and meanwhile, the parameters cannot be optimized in a training mode, so that effective optimization cannot be performed for different downstream tasks; the self-adaptive regular direction method at the present stage can only be applied to a data set with a labeling direction, and cannot be effectively expanded to a large-scale scene data set lacking relevant labeling.
Disclosure of Invention
The invention provides a point cloud pose adjustment method, which is used for solving the defects that in the prior art, the calculated amount is large in the process of adjusting the pose of the point cloud, and the point cloud after being adjusted does not have strict rotation invariance, and realizing the technical effects of reducing the calculated amount in the process of adjusting the pose of the point cloud and simultaneously ensuring that the point cloud after being adjusted has strict rotation invariance.
The invention provides a point cloud pose adjusting method, which comprises the following steps:
acquiring a point cloud to be processed, performing center zeroing operation on the point cloud to be processed, and determining a point cloud origin of the point cloud to be processed after center zeroing;
Extracting the distance from each point in the point cloud to be processed to the point cloud origin as a one-dimensional rotation invariant feature after the center is zeroed;
performing weighted calculation on the point cloud to be processed after the center is zeroed to obtain a weighted result; wherein the weight of the weighted calculation is calculated according to the one-dimensional rotation invariant feature;
Carrying out standard orthogonalization processing on the weighted result to obtain a rotation matrix of the point cloud to be processed after the center is zeroed;
and based on the rotation matrix, carrying out pose adjustment on the point cloud to be processed after the center is zeroed.
According to the point cloud pose adjustment method provided by the invention, the point cloud to be processed is an N multiplied by 3 matrix, wherein N is the number of points in the point cloud to be processed;
the one-dimensional rotation invariant feature is a matrix of n×1;
the weight is a matrix of Nx2;
the weighted result is a matrix of 3×2;
the rotation matrix is a 3×3 matrix;
the weight of the weighted calculation is calculated according to the one-dimensional rotation invariant feature, and specifically comprises the following steps:
And converting the N multiplied by 1 matrix of the one-dimensional rotation invariant feature into the N multiplied by 2 matrix of the weight through a multi-layer perceptron.
According to the method for adjusting the pose of the point cloud, the weighted result is subjected to standard orthogonalization to obtain the rotation matrix of the point cloud to be processed after center zeroing, and the method specifically comprises the following steps:
Carrying out standard orthogonalization processing on two columns in the weighting result matrix to obtain an x-axis rotation vector which represents the point cloud around the x-axis direction and a y-axis rotation vector which represents the point cloud around the y-axis direction in the rotation matrix;
Based on the characteristics of the rotation matrix, the x-axis rotation vector and the y-axis rotation vector, a z-axis rotation vector which represents the point cloud around the z-axis direction in the rotation matrix is obtained;
obtaining the rotation matrix according to the x-axis rotation vector, the y-axis rotation vector and the z-axis rotation vector;
the characteristics of the rotation matrix are specifically as follows:
The transposed product of the rotation matrix and the rotation matrix is a unitary matrix;
The determinant of the rotation matrix is 1.
According to the method for adjusting the pose of the point cloud provided by the invention, the pose of the point cloud to be processed after zeroing is adjusted based on the rotation matrix, specifically:
Multiplying the rotation matrix by the point cloud to be processed after zeroing to obtain a point cloud matrix after pose adjustment.
According to the point cloud pose adjustment method provided by the invention, the point cloud to be processed after the center is zeroed is subjected to weighted calculation to obtain a weighted result, and the weighted result is specifically as follows:
Multiplying the matrix of the weight with the transpose of the point cloud matrix to be processed after the center is zeroed, and obtaining the matrix of the weighted result.
According to the point cloud pose adjustment method provided by the invention, in the weighted result orthogonalization processing, the orthogonalization method used in the weighted result orthogonalization processing meets the rotation equivalence.
The invention also provides a pose adjusting device of the point cloud, which comprises:
the center zeroing module is used for acquiring point clouds to be processed, carrying out center zeroing operation on the point clouds to be processed, and determining the point cloud origin of the point clouds to be processed after center zeroing;
the feature extraction module is used for extracting the distance from each point in the point cloud to be processed to the point cloud origin after the center is zeroed as a one-dimensional rotation invariant feature;
the weighting module is used for carrying out weighting calculation on the point cloud to be processed after the center is zeroed to obtain a weighting result; wherein the weight of the weighted calculation is calculated according to the one-dimensional rotation invariant feature;
The standard orthogonalization module is used for carrying out standard orthogonalization processing on the weighted result to obtain a rotation matrix of the point cloud to be processed after the center is zeroed;
and the pose adjustment module is used for carrying out pose adjustment on the point cloud to be processed after the center is zeroed based on the rotation matrix.
The invention also provides electronic equipment, which comprises a memory, a processor and a computer program stored on the memory and capable of running on the processor, wherein the processor realizes the point cloud pose adjustment method according to any one of the above when executing the program.
The present invention also provides a non-transitory computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements a point cloud pose adjustment method as described in any of the above.
The invention also provides a computer program product comprising a computer program which, when executed by a processor, implements a point cloud pose adjustment method according to any one of the above.
The invention provides a point cloud pose adjusting method and a device, wherein the method comprises the following steps: acquiring a point cloud to be processed, performing center zeroing operation on the point cloud to be processed, and determining a point cloud origin of the point cloud to be processed after center zeroing; extracting the distance from each point in the point cloud to be processed to the point cloud origin as a one-dimensional rotation invariant feature after the center is zeroed; carrying out weighted calculation on the point cloud to be processed after the center is zeroed to obtain a weighted result; wherein the weight of the weighted calculation is calculated according to the one-dimensional rotation invariant feature; carrying out standard orthogonalization treatment on the weighted result to obtain a rotation matrix of the point cloud to be treated after the center is zeroed; and based on the rotation matrix, carrying out pose adjustment on the point cloud to be processed after the center is zeroed. According to the technical scheme, firstly, the point cloud to be processed is subjected to center zeroing operation, the influence of translation on the point cloud to be processed is eliminated, then a rotation matrix is obtained through calculation, and the rotation matrix is multiplied by the point cloud to be processed after center zeroing, so that the point cloud is subjected to rotation adjustment. According to the technical scheme, the point cloud which is not aligned is aligned, the calculated amount is small in the position and posture adjustment process of the point cloud, and the point cloud after the alignment has strict rotation invariance.
Drawings
In order to more clearly illustrate the invention or the technical solutions of the prior art, the following description will briefly explain the drawings used in the embodiments or the description of the prior art, and it is obvious that the drawings in the following description are some embodiments of the invention, and other drawings can be obtained according to the drawings without inventive effort for a person skilled in the art.
FIG. 1 is a schematic flow chart of a point cloud pose adjustment method provided by the invention;
FIG. 2 is a second flow chart of the method for adjusting the pose of the point cloud according to the present invention;
FIG. 3 is a schematic diagram of a different orthogonalization method provided by the present invention;
FIG. 4 is a schematic diagram of a point cloud pose adjustment method and a baseline method provided by the invention compared on a point cloud reconstruction task;
fig. 5 is a schematic structural diagram of a pose adjustment device for point cloud provided by the invention;
Fig. 6 is a schematic structural diagram of an electronic device provided by the present invention.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the present invention more apparent, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings, and it is apparent that the described embodiments are some embodiments of the present invention, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
The point cloud operations such as point cloud alignment and point cloud registration are essentially to perform rotational translation on the point cloud, so that the point cloud is transformed to a target pose. The prior art point cloud alignment process can be broadly divided into three main categories: multi-view methods, spherical convolution methods, and rotation invariant methods.
The multi-view method is a method by using a set of multi-view data as a descriptor of a three-dimensional shape, and has a long history in three-dimensional analysis. After the convolutional neural network becomes the optimal method in the image field, the multi-view image method based on the convolutional neural network also becomes a method with better effect in three-dimensional analysis. MVCNN is a precursor to such a method, which processes multi-view pictures of a three-dimensional model through a convolutional neural network to perform feature extraction on the three-dimensional model. Although such methods were not originally specifically proposed as methods with rotational invariance, they have a good effect on the rotational point cloud. With the advent of methods for directly processing point clouds, there are also some methods that have turned to the processing of multi-view point clouds. For example, rot-SO-Net is to obtain discrete rotational invariance by directly processing the multi-view point cloud.
The spherical convolution method is a method of firstly converting a three-dimensional model into a spherical signal and then processing the spherical signal to obtain rotational invariance. There are a number of defined ways of spherical convolution, but these can be used for feature extraction with rotational invariance. Spherical convolution is not directly applicable to discrete point clouds because it requires the integration to be calculated over a continuous area. To solve this problem, PRIN and SPRIN combine the means of density adaptive sampling and point cloud resampling by introducing spherical voxel convolution, thereby expanding the spherical convolution method to point clouds.
The rotation invariant method is a method which is widely applied and has better robustness at present. The method comprises the steps of firstly converting the point cloud from an original coordinate which is easy to be influenced by rotation into an invariant which is not influenced by rotation, and then processing the invariant. SRINet by first selecting the points furthest and closest from the center and their cross products in the point cloud as three projection coordinate axes, and then projecting the coordinates as rotation invariant; clusterNet is then the conversion of the original coordinates into a K-nearest neighbor map with rotation invariance by a more complex rotation invariance operator. Among such methods, there are some methods to obtain rotational invariance by constructing orthonormal basis, where such methods are equivalent to rotating a point cloud to a canonical direction first and then processing the canonical-direction point cloud to obtain rotational invariance. There are both fixed canonical direction calculation methods based on principal component analysis and adaptive canonical direction calculation methods, such as RTN and RotPredictor.
Due to the defects existing in the prior art, the invention provides a novel point cloud pose adjusting method.
The following describes a point cloud pose adjustment method according to the present invention with reference to fig. 1 to 2.
Fig. 1 is a flow chart of a method for adjusting pose of point cloud according to an embodiment of the present invention, as shown in fig. 1, the method includes the following steps:
step 110, acquiring a point cloud to be processed, performing center zeroing operation on the point cloud to be processed, and determining a point cloud origin of the point cloud to be processed after center zeroing. A point cloud is a data set of points in a certain coordinate system. The points contain rich information including three-dimensional coordinates X, Y, Z, color, classification values, intensity values, time, etc., to name but a few. In this embodiment, the point cloud to be processed is an n×3 matrix, where N is the number of points in the point cloud to be processed. Three columns in the matrix are coordinate values of each point X, Y and Z.
Specifically, it is assumed that the input point cloud to be processed is represented in a matrix formCenter zeroing is performed on the point cloud to be processed in step 110: /(I)Wherein 1 represents a length of/>And are all column vectors of 1. This step changes the input point cloud to a center-zeroed point cloud to remove the effects of translating the point cloud.
And 120, extracting the distance from each point in the point cloud to be processed to the point cloud origin after the center is zeroed as a one-dimensional rotation invariant feature.
Specifically, step 120 is feature extraction: . This step is performed by a function/> Feature extraction is carried out on the point cloud with the center zeroed, and the difference/>Corresponding to different characteristics with rotational invariance. In the implementation of the method, the distance from each point in the point cloud to the origin of the point cloud is selected as a one-dimensional rotation invariant feature. This feature, while simple, has linear complexity, computes faster, and the effect of the experiment is also better. Wherein the one-dimensional rotation invariant feature is a matrix of N x 1.
130, Carrying out weighted calculation on the point cloud to be processed after the center is zeroed to obtain a weighted result; wherein the weight of the weighted calculation is calculated according to the one-dimensional rotation invariant feature. And multiplying the weighted matrix by the transpose of the point cloud matrix to be processed after the center is zeroed to obtain a matrix of weighted results, wherein the weighted results are 3 multiplied by 2.
Specifically, step 130 is feature regression: . The present invention uses a multi-layer perceptron to implement the function/> This step is to further process the rotation invariant feature obtained in the previous step to obtain the weight/>. Wherein the weight is a matrix of n×2. And converting the N multiplied by 1 matrix of the one-dimensional rotation invariant feature into an N multiplied by 2 matrix of the weight through a multi-layer perceptron.
And 140, carrying out standard orthogonalization processing on the weighted result to obtain a rotation matrix of the point cloud to be processed after the center is zeroed. In the orthogonalization of the weighted result, the orthogonalization method used satisfies the rotation equivalence.
In some embodiments, as shown in fig. 2, step 140 specifically includes the steps of:
step 210, carrying out standard orthogonalization processing on two columns in the weighted result matrix to obtain an x-axis rotation vector which represents the point cloud around the x-axis direction and a y-axis rotation vector which represents the point cloud around the y-axis direction in the rotation matrix;
step 220, based on the characteristics of the rotation matrix and the x-axis rotation vector and the y-axis rotation vector, obtaining a z-axis rotation vector which represents the point cloud around the z-axis direction in the rotation matrix;
step 230, obtaining the rotation matrix according to the x-axis rotation vector, the y-axis rotation vector and the z-axis rotation vector;
the rotation matrix specifically has the following characteristics:
The transposed product of the rotation matrix and the rotation matrix is an identity matrix;
the determinant of the rotation matrix is 1.
Step 140 is orthogonalization: . This step uses the weighted weights obtained in the previous step/> Point cloud/>Weighting to obtain/>. Pair/>Orthogonalization is performed on the two columns of the matrix to obtain two coordinate axes/>. The orthonormal method used here must satisfy the rotational equivalence, that is:
There are many standard orthogonalization methods satisfying this requirement, such as Gram-Schmidt method, etc., but are not employed here. The reason for this is mainly that the Gram-Schmidt method results in two vectors of orthogonalization that are excessively dependent on one of the directions, namely:
In this case, two directions obtained by orthogonalization Greatly receives/>Is the effect of balance/>The method adopts the following method to calculate:
After the second method is adopted, for the final For/>With the same effect, the two methods differ as shown in fig. 3. Wherein the left side in FIG. 3 is the Gram-Schmidt method and the right side is the orthogonalization method in the method of the invention.
Since the final rotation matrix satisfiesLast coordinate axis/>Can directly pass throughCalculating;
Assembling a rotation matrix: . The three coordinate axes obtained in the step are directly spliced into a rotation matrix. Wherein the rotation matrix is a3×3 matrix.
And 150, adjusting the pose of the point cloud to be processed after the center is zeroed based on the rotation matrix. Specifically, multiplying the rotation matrix by the zeroed point cloud to be processed to obtain a point cloud matrix with adjusted pose.
In the technical scheme of the invention, a key problem is whether the point cloud pose adjustment method has strict rotation invariance.
To answer this question, the method establishes the following theoretical basis:
for the present method, if it is for an input point cloud Output/>Then for the input point cloud/>Output/>Wherein/>Is an arbitrary point cloud,/>Is an arbitrary permutation,/>Is arbitrarily rotated,/>Is any translation.
The theoretical basis shows that the rotation matrix calculated by the method has strict rotation invariance, which ensures consistency and consistency in the training process.
The key points of the point cloud pose adjustment method of the invention are as follows:
1. The method has strict rotational invariance over the SO (3) rotor group: unlike the multi-view method with rotational invariance only on discrete rotating subgroups, the method obtains rotational invariance by calculating the regular direction, and has strict rotational invariance on SO (3) because the regular direction can be unchanged for any direction input;
2. The method has less calculated amount: compared with a spherical convolution method which needs to project a point cloud onto a spherical surface and calculate numerical integration on the spherical surface, the method has the advantages that the main calculated amount is only a small multi-layer perceptron, and the calculated amount is less;
3. The method fully reserves the structural information of the point cloud: the point cloud with the regular direction calculated by the method is only different from the original point cloud by one rotation matrix, so that the point cloud structure information mainly based on the relative position relationship is completely reserved;
4. The method does not need a large amount of data enhancement in the training stage of the algorithm: in fact, only one module of the method needs to be added at the front end of the existing model, and the existing model without rotation invariance can be helped to obtain rotation invariance without additional adjustment;
5. the method has no ambiguity: the method does not obtain the regular directions through a principal component analysis method, so that ambiguity does not exist naturally, and an additional module is not needed to screen a plurality of regular directions;
6. The method is a self-adaptive regular direction calculation method with strict rotation invariance: the conventional regular direction calculation method can not perform corresponding self-adaptive optimization aiming at different downstream tasks, or can only achieve approximate rotation invariance in the range of a training data set. The method uses a simple multi-layer perceptron to carry out regression on the weight, and simultaneously overcomes the two defects, namely, the method can realize self-adaptive adjustment on the downstream task while having strict rotation invariance;
7. The performance of the method is optimized in the existing method. The method achieves the best performance on datasets such as ModelNet, SHAPENETPART and the like which are commonly used in the field.
The technical scheme of the invention has the advantages that:
1. The method has less calculated amount: the calculated amount of the main parts in the method is only proportional to the number of points, and only has linear complexity;
2. The method does not require data enhancement: only training on point clouds which are not subjected to data enhancement is needed, so that point clouds in all directions can be covered;
3. The method has no ambiguity: the method can generate point clouds in different regular directions according to the needs, is not limited by methods such as principal component analysis and the like, does not have ambiguity naturally, and does not need an additional module to screen a plurality of regular directions;
4. the method has good performance. The method has good effects on the tasks of point cloud classification, point cloud reconstruction, point cloud component segmentation, point cloud scene segmentation and the like.
In order to illustrate the effect of the method provided by the invention, the technical scheme of the invention performs effect verification on four tasks, and is listed as follows:
(a) Point cloud classification
ModelNet40 the data set is a widely used three-dimensional shape data set that includes a total of forty classes of three-dimensional models. There were 9843 models in the training dataset and 2468 models in the test dataset. The goal of this task is to predict the class of the point cloud from the point cloud data. In order to fairly evaluate the performance of the algorithm, the technical scheme of the invention uses the sampling data in PointNet for training and testing. The training was performed without any rotation data enhancement, and the testing was performed with any rotation-enhanced data. The experimental results are shown in table 1.
TABLE 1 Point cloud Classification experiment results
It can be seen from table 1 that the performance of the present method under this test condition is optimal for all methods at present. Further, the results of the present method are more pronounced considering that the present method does not use any rotation enhancement (the aforementioned method mostly uses rotation enhancement around one axis) and that the network backbone uses only the most common PointNet ++.
(B) Point cloud reconstruction
ShapeNetCore is a subset of ShapeNet that includes some independent clean three-dimensional model data. The experiment selects the five categories with the largest number of models in ShapeNetCore for the experiment, and aims to learn a point cloud characteristic representation which is robust to rotation through self-supervision training on the data. To verify the effectiveness of the present method, this experiment compares the baseline method without posture correction with the control method for posture correction using the present method. The experiment adopts two minimum matching distances MMD-CD and MMD-EMD as evaluation indexes, and the smaller the values of the two indexes, the better the reconstruction effect is.
TABLE 2 Point cloud reconstruction experiment results
It can be seen from table 2 that the method has a decisive advantage in reconstructing the minimum matching distance over all five categories. The present application visualizes the results of the point cloud reconstruction on five categories, as shown in fig. 4. FIG. 4 is divided into two columns, wherein the left side represents the same direction of the test data and the training data, and the right side represents the different directions of the test data and the training data; each column is divided into three columns, and the three columns are respectively input point cloud, point cloud reconstructed by a baseline method and point cloud reconstructed by the method from left to right. As can be seen from the figure, the baseline method works well when the input point cloud tested is in the same direction as during training, and even performs a little better on some test samples; however, once the direction of the input point cloud is different from the direction during training, the effect of the baseline method is severely degraded, and the output results either collapse (e.g., first and fifth columns in right column) or become completely independent of the input (e.g., second, third and fourth columns in right column). This phenomenon illustrates that in the point cloud reconstruction task, the baseline method encodes a large amount of direction information, which makes it not robust to point clouds of different directions, in contrast to the method, which does not encode any direction information, which makes its performance as little affected by the direction of the point cloud as possible.
(C) Point cloud component segmentation
SHAPENETPART is a data set specific to a part segmentation experiment, which includes 14006 training samples and 2874 test samples. The purpose of the experiment is to predict the component class for each point in the input data. As with the point cloud classification task, the present application uses PointNet sampled datasets for fair comparison.
IoU is an evaluation index commonly used in such experiments, and is mainly used for measuring the proximity of a predicted value and a true value, and the higher the value, the better the effect of the algorithm is. In this experiment, there are two indicators of the performance of the evaluation algorithm that have been done in the past. One part calculates an instance average IoU and another part calculates a category average IoU. Since SHAPENETPART is not a class-balanced dataset, the results of the instance average and class average are not identical. For a fair comparison with the previous method, the experiment was performed under two conditions, respectively: non-class equalization (higher instance average IoU is obtained) and class equalization (higher class average IoU is obtained). The results of the experiment are shown in table 3.
TABLE 3 Point cloud component segmentation experiment Effect
As can be seen from table 3, the present method trained under non-class equalization conditions reached the highest on the example average IoU, while the present method trained under class equalization conditions reached the highest on the class average IoU, which fully demonstrates the advantages of the present method.
(D) Point cloud semantic segmentation
ScanNet is a large set of scene video data that includes millions of view data from 1513 scenes. The task of this experiment was to predict the semantic tags for each point in the scene with the color data removed.
In the past rotation invariance method, the data set is taken as the rotation invariance experimental data set by very few works, mainly because the number of point clouds in the experiment reaches millions, so that most of the past methods cannot be effectively applied. Correspondingly, the overall complexity of the method is proportional to the number of point clouds, which makes it easy to expand into large scale scene data sets. The present experiment compares a baseline method without posture correction with a control method for posture correction using the present method, using a similar setup as the point cloud reconstruction experiment. The accuracy under the conditions of no rotation and arbitrary rotation is adopted as an evaluation index of the experiment, and the experimental results are shown in table 4.
Table 4 Point cloud semantic segmentation experiment Effect
As can be seen from table 4, the baseline method is even stronger than the present method without rotation, since the directions of the test data and the training data are substantially the same. However, once the direction of the test data is different from the training data, the performance of the baseline method is severely degraded, while the method remains substantially unchanged. This fully illustrates the advantages of the present method.
In order to verify the time efficiency of the method, the application also provides response time data of the method on the four experiments, mainly comprising the time required for correcting the posture of the method and the time required for extracting the characteristics of the rest, as shown in table 5.
TABLE 5 comparison of response times for the present method
As can be seen from table 5, the time required for correcting the posture part of the method is significantly lower than that required for the rest part, which fully demonstrates that the method can be given the rotational invariance of the existing method without significantly increasing the calculation amount.
As can be seen from the above, the technical scheme of the invention eliminates the influence of translation on the point cloud to be processed by performing the center zeroing operation on the point cloud to be processed, obtains the rotation matrix through calculation, and multiplies the rotation matrix with the point cloud to be processed after center zeroing, thereby performing rotation adjustment on the point cloud. According to the technical scheme, the point cloud which is not aligned is aligned, the calculated amount is small in the position and posture adjustment process of the point cloud, and the point cloud after the alignment has strict rotation invariance.
The pose adjusting device of the point cloud provided by the invention is described below, and the pose adjusting device of the point cloud described below and the pose adjusting method of the point cloud described above can be correspondingly referred to each other.
As shown in fig. 5, an embodiment of the present invention provides a pose adjustment device for a point cloud, the device including: a center zeroing module 510, a feature extraction module 520, a weighting module 530, a orthogonalization module 540, and a pose adjustment module 550.
Specifically, the center zeroing module 510 is configured to obtain a point cloud to be processed, perform a center zeroing operation on the point cloud to be processed, and determine a point cloud origin of the point cloud to be processed after center zeroing.
The feature extraction module 520 is configured to extract a distance from each point in the point cloud to be processed to an origin of the point cloud after the center is zeroed as a one-dimensional rotation invariant feature.
The weighting module 530 is configured to perform a weighting calculation on the point cloud to be processed after the center is zeroed, so as to obtain a weighted result; wherein the weight of the weighted calculation is calculated according to the one-dimensional rotation invariant feature.
The orthonormal module 540 is configured to orthonormal the weighted result to obtain a rotation matrix of the point cloud to be processed after the center is zeroed.
The pose adjustment module 550 is configured to perform pose adjustment on the point cloud to be processed after the center is zeroed based on the rotation matrix.
According to the technical scheme, firstly, the point cloud to be processed is subjected to center zeroing operation, the influence of translation on the point cloud to be processed is eliminated, then a rotation matrix is obtained through calculation, and the rotation matrix is multiplied by the point cloud to be processed after center zeroing, so that the point cloud is subjected to rotation adjustment. According to the technical scheme, the point cloud which is not aligned is aligned, the calculated amount is small in the position and posture adjustment process of the point cloud, and the point cloud after the alignment has strict rotation invariance.
Fig. 6 illustrates a physical schematic diagram of an electronic device, as shown in fig. 6, which may include: processor 610, communication interface (Communications Interface) 620, memory 630, and communication bus 640, wherein processor 610, communication interface 620, memory 630 communicate with each other via communication bus 640. The processor 610 may call logic instructions in the memory 630 to perform a point cloud pose adjustment method comprising the steps of: acquiring a point cloud to be processed, performing center zeroing operation on the point cloud to be processed, and determining a point cloud origin of the point cloud to be processed after center zeroing; extracting the distance from each point in the point cloud to be processed to the point cloud origin as a one-dimensional rotation invariant feature after the center is zeroed; carrying out weighted calculation on the point cloud to be processed after the center is zeroed to obtain a weighted result; wherein the weight of the weighted calculation is calculated according to the one-dimensional rotation invariant feature; carrying out standard orthogonalization treatment on the weighted result to obtain a rotation matrix of the point cloud to be treated after the center is zeroed; and based on the rotation matrix, carrying out pose adjustment on the point cloud to be processed after the center is zeroed.
Further, the logic instructions in the memory 630 may be implemented in the form of software functional units and stored in a computer-readable storage medium when sold or used as a stand-alone product. Based on this understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art or in a part of the technical solution, in the form of a software product stored in a storage medium, comprising several instructions for causing a computer device (which may be a personal computer, a server, a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a usb disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a magnetic disk, or an optical disk, or other various media capable of storing program codes.
In another aspect, the present invention also provides a computer program product, where the computer program product includes a computer program, where the computer program can be stored on a non-transitory computer readable storage medium, and when the computer program is executed by a processor, the computer can execute the point cloud pose adjustment method provided by the above methods, and the method includes the following steps: acquiring a point cloud to be processed, performing center zeroing operation on the point cloud to be processed, and determining a point cloud origin of the point cloud to be processed after center zeroing; extracting the distance from each point in the point cloud to be processed to the point cloud origin as a one-dimensional rotation invariant feature after the center is zeroed; carrying out weighted calculation on the point cloud to be processed after the center is zeroed to obtain a weighted result; wherein the weight of the weighted calculation is calculated according to the one-dimensional rotation invariant feature; carrying out standard orthogonalization treatment on the weighted result to obtain a rotation matrix of the point cloud to be treated after the center is zeroed; and based on the rotation matrix, carrying out pose adjustment on the point cloud to be processed after the center is zeroed.
In still another aspect, the present invention further provides a non-transitory computer readable storage medium having stored thereon a computer program which, when executed by a processor, is implemented to perform the method for adjusting a point cloud pose provided by the above methods, the method comprising the steps of: acquiring a point cloud to be processed, performing center zeroing operation on the point cloud to be processed, and determining a point cloud origin of the point cloud to be processed after center zeroing; extracting the distance from each point in the point cloud to be processed to the point cloud origin as a one-dimensional rotation invariant feature after the center is zeroed; carrying out weighted calculation on the point cloud to be processed after the center is zeroed to obtain a weighted result; wherein the weight of the weighted calculation is calculated according to the one-dimensional rotation invariant feature; carrying out standard orthogonalization treatment on the weighted result to obtain a rotation matrix of the point cloud to be treated after the center is zeroed; and based on the rotation matrix, carrying out pose adjustment on the point cloud to be processed after the center is zeroed.
The apparatus embodiments described above are merely illustrative, wherein the elements illustrated as separate elements may or may not be physically separate, and the elements shown as elements may or may not be physical elements, may be located in one place, or may be distributed over a plurality of network elements. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art will understand and implement the present invention without undue burden.
From the above description of the embodiments, it will be apparent to those skilled in the art that the embodiments may be implemented by means of software plus necessary general hardware platforms, or of course may be implemented by means of hardware. Based on this understanding, the foregoing technical solution may be embodied essentially or in a part contributing to the prior art in the form of a software product, which may be stored in a computer readable storage medium, such as ROM/RAM, a magnetic disk, an optical disk, etc., including several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the method described in the respective embodiments or some parts of the embodiments.
Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present invention, and are not limiting; although the invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims (9)
1. The method for adjusting the pose of the point cloud is characterized by comprising the following steps:
acquiring a point cloud to be processed, performing center zeroing operation on the point cloud to be processed, and determining a point cloud origin of the point cloud to be processed after center zeroing;
Extracting the distance from each point in the point cloud to be processed to the point cloud origin as a one-dimensional rotation invariant feature after the center is zeroed;
performing weighted calculation on the point cloud to be processed after the center is zeroed to obtain a weighted result; wherein the weight of the weighted calculation is calculated according to the one-dimensional rotation invariant feature;
Carrying out standard orthogonalization processing on the weighted result to obtain a rotation matrix of the point cloud to be processed after the center is zeroed;
Based on the rotation matrix, carrying out pose adjustment on the point cloud to be processed after the center is zeroed;
the step of carrying out standard orthogonalization processing on the weighted result to obtain a rotation matrix of the point cloud to be processed after center zeroing, specifically comprises the following steps:
The weighted result is a matrix of 3 multiplied by 2, and two columns in the weighted result matrix are subjected to standard orthogonalization processing to obtain an x-axis rotation vector which represents the point cloud around the x-axis direction and a y-axis rotation vector which represents the point cloud around the y-axis direction in the rotation matrix;
Based on the characteristics of the rotation matrix, the x-axis rotation vector and the y-axis rotation vector, a z-axis rotation vector which represents the point cloud around the z-axis direction in the rotation matrix is obtained;
obtaining the rotation matrix according to the x-axis rotation vector, the y-axis rotation vector and the z-axis rotation vector;
the characteristics of the rotation matrix are specifically as follows:
The transposed product of the rotation matrix and the rotation matrix is a unitary matrix;
The determinant of the rotation matrix is 1.
2. The method for adjusting the pose of the point cloud according to claim 1, wherein the pose adjustment of the point cloud to be processed after zeroing is performed based on the rotation matrix, specifically:
Multiplying the rotation matrix by the point cloud to be processed after zeroing to obtain a point cloud matrix after pose adjustment.
3. The method for adjusting the pose of the point cloud according to claim 1, wherein the weighting calculation is performed on the point cloud to be processed after the center is zeroed, so as to obtain a weighted result, which specifically is:
Multiplying the matrix of the weight with the transpose of the point cloud matrix to be processed after the center is zeroed, and obtaining the matrix of the weighted result.
4. The method for adjusting the pose of a point cloud according to claim 1, wherein,
The point cloud to be processed is an N multiplied by 3 matrix, wherein N is the number of points in the point cloud to be processed;
the one-dimensional rotation invariant feature is a matrix of n×1;
the weight is a matrix of Nx2;
the rotation matrix is a 3×3 matrix;
the weight of the weighted calculation is calculated according to the one-dimensional rotation invariant feature, and specifically comprises the following steps:
And converting the N multiplied by 1 matrix of the one-dimensional rotation invariant feature into the N multiplied by 2 matrix of the weight through a multi-layer perceptron.
5. The point cloud pose adjustment method according to claim 1, characterized in that in orthogonalizing the weighted result, the orthogonalization method used satisfies rotational equivalence.
6. A pose adjustment device for a point cloud, the device comprising:
the center zeroing module is used for acquiring point clouds to be processed, carrying out center zeroing operation on the point clouds to be processed, and determining the point cloud origin of the point clouds to be processed after center zeroing;
the feature extraction module is used for extracting the distance from each point in the point cloud to be processed to the point cloud origin after the center is zeroed as a one-dimensional rotation invariant feature;
the weighting module is used for carrying out weighting calculation on the point cloud to be processed after the center is zeroed to obtain a weighting result; wherein the weight of the weighted calculation is calculated according to the one-dimensional rotation invariant feature;
The standard orthogonalization module is used for carrying out standard orthogonalization processing on the weighted result to obtain a rotation matrix of the point cloud to be processed after the center is zeroed;
The pose adjustment module is used for carrying out pose adjustment on the point cloud to be processed after the center is zeroed based on the rotation matrix;
the standard orthogonalization module is specifically configured to:
The weighted result is a matrix of 3 multiplied by 2, and two columns in the weighted result matrix are subjected to standard orthogonalization processing to obtain an x-axis rotation vector which represents the point cloud around the x-axis direction and a y-axis rotation vector which represents the point cloud around the y-axis direction in the rotation matrix;
Based on the characteristics of the rotation matrix, the x-axis rotation vector and the y-axis rotation vector, a z-axis rotation vector which represents the point cloud around the z-axis direction in the rotation matrix is obtained;
obtaining the rotation matrix according to the x-axis rotation vector, the y-axis rotation vector and the z-axis rotation vector;
the characteristics of the rotation matrix are specifically as follows:
The transposed product of the rotation matrix and the rotation matrix is a unitary matrix;
The determinant of the rotation matrix is 1.
7. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the point cloud pose adjustment method according to any of claims 1 to 5 when the program is executed by the processor.
8. A non-transitory computer readable storage medium having stored thereon a computer program, wherein the computer program when executed by a processor implements the point cloud pose adjustment method according to any of claims 1 to 5.
9. A computer program product comprising a computer program, characterized in that the computer program, when executed by a processor, implements the point cloud pose adjustment method according to any of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210374345.8A CN114782672B (en) | 2022-04-11 | 2022-04-11 | Point cloud pose adjustment method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210374345.8A CN114782672B (en) | 2022-04-11 | 2022-04-11 | Point cloud pose adjustment method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114782672A CN114782672A (en) | 2022-07-22 |
| CN114782672B true CN114782672B (en) | 2024-06-21 |
Family
ID=82430021
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210374345.8A Active CN114782672B (en) | 2022-04-11 | 2022-04-11 | Point cloud pose adjustment method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114782672B (en) |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111402328A (en) * | 2020-03-17 | 2020-07-10 | 北京图森智途科技有限公司 | Pose calculation method and device based on laser odometer |
| CN111915661A (en) * | 2020-07-24 | 2020-11-10 | 广州大学 | Point cloud registration method and system based on RANSAC algorithm and computer readable storage medium |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP3693922A1 (en) * | 2019-02-11 | 2020-08-12 | Siemens Aktiengesellschaft | An apparatus and a method for performing a data driven pairwise registration of three-dimensional point clouds |
| US20210133990A1 (en) * | 2019-11-05 | 2021-05-06 | Nvidia Corporation | Image aligning neural network |
-
2022
- 2022-04-11 CN CN202210374345.8A patent/CN114782672B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111402328A (en) * | 2020-03-17 | 2020-07-10 | 北京图森智途科技有限公司 | Pose calculation method and device based on laser odometer |
| CN111915661A (en) * | 2020-07-24 | 2020-11-10 | 广州大学 | Point cloud registration method and system based on RANSAC algorithm and computer readable storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114782672A (en) | 2022-07-22 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109784149B (en) | Method and system for detecting key points of human skeleton | |
| Yuan et al. | Factorization-based texture segmentation | |
| CN113095333B (en) | Unsupervised feature point detection method and unsupervised feature point detection device | |
| CN110827297A (en) | Insulator segmentation method for generating countermeasure network based on improved conditions | |
| CN113570508A (en) | Image restoration method and device, storage medium and terminal | |
| CN112634163A (en) | Method for removing image motion blur based on improved cycle generation countermeasure network | |
| CN111598087B (en) | Irregular character recognition method, device, computer equipment and storage medium | |
| CN113658040A (en) | Face super-resolution method based on prior information and attention fusion mechanism | |
| US20220415007A1 (en) | Image normalization processing | |
| CN111784699B (en) | Method and device for carrying out target segmentation on three-dimensional point cloud data and terminal equipment | |
| CN112164008A (en) | Training method of image data enhancement network, and training device, medium, and apparatus thereof | |
| CN112836820B (en) | Deep convolution network training method, device and system for image classification task | |
| CN108960280A (en) | A kind of picture similarity detection method and system | |
| CN112149521A (en) | Palm print ROI extraction and enhancement method based on multitask convolutional neural network | |
| CN113763535A (en) | Characteristic latent code extraction method, computer equipment and storage medium | |
| JP7277341B2 (en) | Curved object segmentation with noise antecedents | |
| CN113298931A (en) | Reconstruction method and device of object model, terminal equipment and storage medium | |
| CN114118303B (en) | Face key point detection method and device based on prior constraint | |
| CN104573738A (en) | Signal processing method and device thereof | |
| CN117132461B (en) | Method and system for whole-body optimization of character based on character deformation target body | |
| CN114662666A (en) | Decoupling method and system based on beta-GVAE and related equipment | |
| CN114782672B (en) | Point cloud pose adjustment method and device | |
| CN117576497A (en) | Training method and device for memory Dirichlet process Gaussian mixture model | |
| CN112001865A (en) | Face recognition method, device and equipment | |
| CN116071625A (en) | Training method of deep learning model, target detection method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |