CN115170626A - Unsupervised method for robust point cloud registration based on depth features - Google Patents

Abstract

The invention discloses an unsupervised method for carrying out robust point cloud registration based on depth characteristics, which comprises the following steps: 1) Acquiring point cloud data; 2) Converting; 3) Extracting characteristics; 4) Point cloud registration; 5) And (5) training. The method is an unsupervised network, the registration framework is trained in an unsupervised mode by combining global and local high-level features to learn and extract the deep features, expensive calculation is not needed to be carried out on point correspondence, and great advantages are achieved in the aspects of precision, initialization robustness and calculation efficiency.

Description

An unsupervised method for robust point cloud registration based on deep features

technical field

The invention relates to three-dimensional reconstruction and positioning of objects, in particular to an unsupervised method for robust point cloud registration based on depth features.

Background technique

Point cloud registration is the problem of estimating a rigid transformation that aligns two point clouds. It has many applications in various fields such as autonomous driving, motion and pose estimation, 3D reconstruction, simultaneous localization and mapping (SLAM), and augmented reality.

Recently, some deep learning (DL) based methods have been proposed to deal with large rotation angles. (Y. Wang and J.M. Solomon, “Deep closest point: Learning representations for point cloud registration,” in Proceedings of the IEEE International Conferenceon Computer Vision (ICCV), 2019, pp. 3523–3532.) (Y. Aoki, H. Goforth , R.A. Srivatsan, and S. Lucey, "Pointnetlk: Robust&efficient point cloud registration using pointnet," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2019, pp. 7163–7172) (V. Sarode, X. Li , H. Goforth, Y. Aoki, R. A. Srivatsan, S. Lucey, and H. Choset, "Pcrnet: point cloud registration network using pointnet encoding," arXiv preprint arXiv: 1908.07906, 2019.) (X. Huang, G.Mei, and J. Zhang, “Feature-metric registration: A fast semi-supervised approach for robust point cloud registration without correspondences,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition(CVPR), 2020, pp.11 366–11 374. )

Roughly speaking, they can be divided into two categories: supervised methods that rely on ground-truth correspondences or class labels, and unsupervised methods. The deep nearest point (DCP for short) calculates the rigid transformation through singular value decomposition (SVD for short), in which the correspondence is constructed by learning the soft matching map. PointNetLK uses classical alignment techniques such as the Lucas-Kanade (LK) algorithm to align PointNet features, yielding good generalization ability to shapes not seen during training. However, they rely on a large amount of registration label data, which makes the algorithm impractical because 3D registration labels are very labor-intensive. In contrast, achieving registration from unlabeled point cloud data without ground truth counterparts is a significant challenge. PCRNet alleviates the pose bias shown in PointNetLK by replacing the Lucas-Kanade module with a multilayer perceptron. PCRNet recovers transformation parameters directly from concatenated global descriptors of source and target point clouds. FMR-Net implements an unsupervised framework with an encoder-decoder task, while enabling registration by minimizing feature metric projection errors. Although these methods exhibit more prominent advantages of unsupervised learning, they mainly rely on the deep features of global representation and ignore the deep features of local representation. Therefore, the depth features of the point cloud are not fully utilized for registration, and the registration effect is not perfect.

SUMMARY OF THE INVENTION

The purpose of the present invention is to provide an unsupervised method for robust point cloud registration based on depth features, aiming at the deficiencies in the prior art. This method is an unsupervised network that learns to extract deep features by combining global and local high-level features, trains the registration framework in an unsupervised manner, and this method does not require expensive computation of point correspondences, in terms of accuracy, There are great advantages in initialization robustness and computational efficiency.

The technical scheme that realizes the object of the present invention is:

An unsupervised method for robust point cloud registration based on depth features, including the following steps:

1) Obtain point cloud data: obtain the point cloud data to be registered, and evenly sample 1024 point data on the point cloud surface;

2) Conversion: Convert the sampled data types of the obtained two point clouds into tensors, the size of which is 1024×3, and then input them into the deep learning framework;

3) Feature extraction: The encoder module in the deep learning framework performs point cloud depth feature extraction on the input tensor, and finally outputs a one-dimensional tensor representing the point cloud depth feature. The specific process is as follows:

3-1) The input tensor 1024×3 passes through the EdgeConv module, and uses each point as the center point to represent the edge features between it and each neighboring point, and then aggregates these features to obtain a new representation of the point, that is, by constructing each point. The field of each vertex obtains the local features of the point cloud, and the specific steps are:

h _θ (x _i ,x _j )=h _θ (x _i ,x _j -x _i ) (1),

Among them, x _i is in the vertex set X={x ₁ ,...x _n }∈RF, and ^F represents the feature space dimension information of the point output by a certain layer of the neural network, and then sends it into a perceptron to obtain the edge feature:

e' _ijm =RELU(θ _m ·(x _j -x _i )+φ _m ·x _i ) (2),

in:

Θ=(θ ₁ , ..., θ _M , φ ₁ ..., φ _M ) (3),

Θ is a learnable parameter, and then aggregates the features of each adjacent edge:

表示聚合操作，Ω表示以点x_i为中心的构成邻边的点对集合，具体聚合操作为：in

represents the aggregation operation, and Ω represents the set of point pairs forming adjacent edges centered on the point x _i . The specific aggregation operation is:

Finally, the size of the output tensor in the EdgeConv module is 6×1024×20 and input to the next convolutional layer;

3-2) Perform five convolutions on the tensors obtained with local features. The first convolution passes through 64 convolutional layers with two-dimensional convolution kernels of size 1. The output tensor size is 64×1024×20;

3-3) Inspired by the feature that the attention mechanism significantly improves the effect of deep learning, the output of the first layer is connected to the CBAM attention module, and the importance of each channel and space is weighted, and the same goes through 64, 128, 256 size convolutional layers;

3-4) Splicing the output tensors of the first four layers and modifying the size of the tensor to 512×1024×1, then entering 512 convolution kernels with a two-dimensional size of 1 for convolution and modifying the dimension to obtain a tensor size of 512 ×1024, use the flatten function to flatten the tensor and the output tensor size is 512, thus obtaining deep features including local and global descriptors;

4) Point cloud registration: To register two point clouds, that is, to obtain a rigid transformation matrix G∈SE(3), which is divided into a rotation matrix R∈SO(3) and a translation matrix t∈R ³ , and then solve To minimize the objective function F(G(R,t)):

Where ψ:R ^3×N →R ^K is the feature extraction function learned by the encoder, and k is the feature dimension;

4-1) G(R, t) is regarded as a special Lie group, which is represented by exponential mapping as follows:

Among them, T is the generator of the exponential mapping G, ε is a Lie algebra, which is mapped to the transformation matrix G through the exponential;

4-2) The point cloud registration problem is described as φ(PT)=φ(G·PT), and solving the matrix G is to obtain the derivative of G by taking the derivative of the Lie algebra ε, so as to directly adjust the Lie algebra ε for indirect optimization The optimal transformation matrix G and the minimization of the feature space projection error of the two point clouds are used for the point cloud registration task. Inspired by the (IC) formula, the inverse transformation of φ(PT)=φ(G·PT) is first performed and its Linearization on the right:

4-3) Transformation Estimation Iteratively runs the computation increment ε, by running the inverse synthesis (IC) algorithm to estimate ε for each step:

ε = (J ^T J) ^-1 (J ^T δ) (9),

是投影误差δ相对于变换参数ε的雅可比矩阵；where δ=φ(P _S )-φ(P _T ) is the projection error of the feature space of the two point clouds,

is the Jacobian matrix of the projection error δ relative to the transformation parameter ε;

4-4) In order to effectively calculate the Jacobian matrix, different finite gradients are used to replace the traditional stochastic gradient method to calculate the Jacobian matrix:

Among them, t _i is the infinitesimal disturbance of the transformation parameters during the calculation. When t _i is set to a fixed value, a better effect will be obtained. For t _i , three angle parameters are set for rotation and three disturbance parameters are used for translation. t _i size is set to 2*e ^-2 ;

4-5) Through the iterative operation formula (9), the transformation matrix is optimized to shorten the projection distance between the source point cloud and the template point cloud in the feature space:

_ΔG ·PS→ _PS (11),

为公式(9)计算出变换参数ε得到的变换矩阵，经过多次迭代，不断变换P_S与P_T配准，最终输出最佳变换矩阵G_est：in

Calculate the transformation matrix obtained by the transformation parameter ε for formula (9). After many iterations, the registration of P _S and P _T is continuously transformed, and the optimal transformation matrix G _est is finally output:

G _est =ΔG _n ··· ΔG ₁ ΔG ₀ (12),

5) Training: In order to realize the training of the deep learning framework in an unsupervised manner, a decoder consisting of four fully connected layers is introduced; at the same time, ReLU is selected as the activation function of the first three layers, and the tanh function is the fourth layer. The activation function of ;

5-1) In the unsupervised case, use the alignment error between the transformed source point cloud and the target point cloud instead of the ground truth transformation for model training, using a robust loss function:

是卷积层参数中的第i个分量，Φ是原始输入的三维点。where P _T ∈ Θ is a set of points sampled from the unit square [0, 1] ² , x is the point cloud feature,

is the ith component in the parameters of the convolutional layer, and Φ is the 3D point of the original input.

This method has high accuracy, high computational efficiency, strong initialization robustness, and does not require expensive computation of point correspondences.

Description of drawings

1 is a schematic diagram of a point cloud registration network architecture in an embodiment;

Fig. 2 is the schematic diagram and effect diagram of carrying out the three-dimensional point cloud registration of bottle in the embodiment;

3 is a schematic diagram and an effect diagram of three-dimensional point cloud registration of a toilet in an embodiment;

4 is a schematic diagram and an effect diagram of performing three-dimensional point cloud registration of a noisy mobile phone according to an embodiment;

5 is a schematic diagram and an effect diagram of performing three-dimensional point cloud registration of a real indoor scene according to an embodiment;

FIG. 6 is a schematic diagram and an effect diagram of the point cloud registration of Armadillo in Stanford 3DMatch according to the embodiment.

Detailed ways

The invention will be further described in detail below with reference to the accompanying drawings and specific embodiments, but it is not intended to limit the invention.

Example:

Referring to Figure 1, an unsupervised method for robust point cloud registration based on depth features includes the following steps:

1) Obtain point cloud data: obtain the point cloud data to be registered, and evenly sample 1024 point data on the point cloud surface;

2) Conversion: Convert the sampled data types of the obtained two point clouds to tensors with a size of 1024×3, and then input them into the deep learning framework;

3) Feature extraction: The encoder module in the deep learning framework performs point cloud depth feature extraction on the input tensor, and finally outputs a one-dimensional tensor representing the point cloud depth feature. The specific process is as follows:

3-1) Unlike other unsupervised methods that rely on global descriptors, the method in this example continues to focus on local features based on the unsupervised training method. The input tensor 1024×3 passes through the EdgeConv module, and uses each point as The center point is used to characterize the edge features between it and each neighboring point, and then these features are aggregated to obtain a new representation of the point, that is, the local features of the point cloud are obtained by constructing the field of each vertex. The specific steps are:

h _θ (x _i ,x _j )=h _θ (x _i ,x _j -x _i ) (1),

Among them, x _i is in the vertex set X={x ₁ ,...x _n }∈RF, and ^F represents the feature space dimension information of the point output by a certain layer of the neural network, and then sends it into a perceptron to obtain the edge feature:

e' _ijm =RELU(θ _m ·(x _j -x _i )+φ _m ·x _i ) (2),

in:

Θ=(θ ₁ , ..., θ _M , φ ₁ ..., φ _M ) (3),

Θ is a learnable parameter, and then aggregates the features of each adjacent edge:

表示聚合操作，Ω表示以点x_i为中心的构成邻边的点对集合，具体聚合操作为：in

represents the aggregation operation, and Ω represents the set of point pairs forming adjacent edges centered on the point x _i . The specific aggregation operation is:

Finally, the size of the output tensor in the EdgeConv module is 6×1024×20 and input to the next convolutional layer;

3-2) Perform five convolutions on the tensors obtained with local features. The first convolution passes through 64 convolutional layers with two-dimensional convolution kernels of size 1. The output tensor size is 64×1024×20;

3-3) Inspired by the feature that the attention mechanism significantly improves the effect of deep learning, the output of the first layer is connected to the CBAM attention module, and the importance of each channel and space is weighted, and the same goes through 64, 128, 256 size convolutional layers;

3-4) Splicing the output tensors of the first four layers and modifying the size of the tensor to 512×1024×1, then entering 512 convolution kernels with a two-dimensional size of 1 for convolution and modifying the dimension to obtain a tensor size of 512 ×1024, use the flatten function to flatten the tensor and the output tensor size is 512, thus obtaining deep features including local and global descriptors;

4) Point cloud registration: To register two point clouds, that is, to obtain a rigid transformation matrix G∈SE(3), which is divided into a rotation matrix R∈SO(3) and a translation matrix t∈R ³ , and then solve To minimize the objective function F(G(R,t)):

Where ψ:R ^3×N →R ^K is the feature extraction function learned by the encoder, and k is the feature dimension;

4-1) G(R, t) is regarded as a special Lie group, which is represented by exponential mapping as follows:

Among them, T is the generator of the exponential mapping G, ε is a Lie algebra, which is mapped to the transformation matrix G through the exponential;

4-2) The point cloud registration problem is described as φ(PT)=φ(G·PT), and solving the matrix G is to obtain the derivative of G by taking the derivative of the Lie algebra ε, so as to directly adjust the Lie algebra ε for indirect optimization The optimal transformation matrix G and the minimization of the feature space projection error of the two point clouds are used for the point cloud registration task. Inspired by the (IC) formula, the inverse transformation of φ(PT)=φ(G·PT) is first performed and its Linearization on the right:

4-3) Transformation Estimation Iteratively runs the computation increment ε, by running the inverse synthesis (IC) algorithm to estimate ε for each step:

ε = (J ^T J) ^-1 (J ^T δ) (9),

是投影误差δ相对于变换where δ=φ(P _S )-φ(P _T ) is the projection error of the feature space of the two point clouds,

is the projection error δ with respect to the transformation

the Jacobian of the parameter ε;

4-4) In order to effectively calculate the Jacobian matrix, different finite gradients are used to replace the traditional stochastic gradient method to calculate the Jacobian matrix:

Among them, t _i is the infinitesimal disturbance of the transformation parameters during the calculation. When t _i is set to a fixed value, a better effect will be obtained. For t _i , three angle parameters are set for rotation and three disturbance parameters are used for translation. t _i size is set to 2*e ^-2 ;

4-5) Through the iterative operation formula (9), the transformation matrix is optimized to shorten the projection distance between the source point cloud and the template point cloud in the feature space:

_ΔG ·PS→ _PS (11),

为公式(9)计算出变换参数ε得到的变换矩阵，经过多次迭代，不断变换P_S与P_T配准，最终输出最佳变换矩阵G_est：in

Calculate the transformation matrix obtained by the transformation parameter ε for formula (9). After many iterations, the registration of P _S and P _T is continuously transformed, and the optimal transformation matrix G _est is finally output:

G _est =ΔG _n ··· ΔG ₁ ΔG ₀ (12),

5) Training: In order to realize the training of the deep learning framework in an unsupervised manner, a decoder consisting of four fully connected layers is introduced; at the same time, ReLU is selected as the activation function of the first three layers, and the tanh function is the fourth layer. The activation function of , the number of layers is shown in Figure 1;

5-1) In the unsupervised case, use the alignment error between the transformed source point cloud and the target point cloud instead of the ground truth transformation for model training, using a robust loss function:

是卷积层参数中的第i个分量，Φ是原始输入的三维点。where P _T ∈ Θ is a set of points sampled from the unit square [0, 1] ² , x is the point cloud feature,

is the ith component in the parameters of the convolutional layer, and Φ is the 3D point of the original input.

In order to illustrate the effectiveness of the deep learning framework, the most widely used ModelNet40 is used as the pre-training set. ModelNet40 is a dataset with 12311 CAD models and 40 object categories. The deep learning framework is used in the top 20 categories of ModelNet40. Train and test on the last 20 classes. In the process of training and testing, the ground truth transformation matrix G _gt is obtained by randomly generating the rigid transformation matrix G, and at the same time, the source point cloud passes through G to generate the target point cloud. Select the axis arbitrarily, set the rotation component in G in the range of [0, 45] degrees, and initialize the translation component in the range of [-0.5, 0.5], with the actual rotation and translation matrix G _gt and predicted rotation and translation matrix G _est . The mean squared error (Mean Squared Error, referred to as MSE), the root mean squared error (Root Mean Squared Error, referred to as RMSE) and the absolute mean error (Mean Absolute Error, referred to as MAE) of the rotation and translation components between the two are used as evaluation indicators. The smaller the value is, the higher the registration accuracy is. At the same time, the unit of measurement of the angle in the experiment is degrees, and the unit of running registration time is seconds. The results are shown in the following table:

Specific examples are shown in Figures 2-6.

Claims (1)

1. an unsupervised method for robust point cloud registration based on depth features, is characterized in that, comprises the steps: 1) Obtain point cloud data: obtain the point cloud data to be registered, and evenly sample 1024 point data on the point cloud surface; 2) Conversion: Convert the sampled data types of the obtained two point clouds to tensors with a size of 1024×3, and then input them into the deep learning framework; 3) Feature extraction: The encoder module in the deep learning framework performs point cloud depth feature extraction on the input tensor, and finally outputs a one-dimensional tensor representing the point cloud depth feature. The specific process is as follows: 3-1) The input tensor 1024×3 passes through the EdgeConv module, and uses each point as the center point to represent the edge features between it and each neighboring point, and then aggregates these features to obtain a new representation of the point, that is, by constructing each point. The field of each vertex obtains the local features of the point cloud, and the specific steps are: h _θ (x _i ,x _j )=h _θ (x _i ,x _j -x _i ) (1), Among them, x _i is in the vertex set X={x ₁ ,...x _n }∈RF, and ^F represents the feature space dimension information of the point output by a certain layer of the neural network, and then sends it into a perceptron to obtain the edge feature: e' _ijm =RELU(θ _m ·(x _j -x _i )+φ _m ·x _i ) (2), in: Θ=(θ ₁ , ..., θ _M , φ ₁ ..., φ _M ) (3), Θ is a learnable parameter, and then aggregates the features of each adjacent edge:

in

represents the aggregation operation, and Ω represents the set of point pairs forming adjacent edges with the point _xi as the center. The specific aggregation operation is:

Finally, the size of the output tensor in the EdgeConv module is 6×1024×20 and input to the next convolutional layer; 3-2) Perform five convolutions on the tensors obtained with local features. The first convolution passes through 64 convolutional layers with two-dimensional convolution kernels of size 1. The output tensor size is 64×1024×20; 3-3) Inspired by the feature that the attention mechanism significantly improves the effect of deep learning, the output of the first layer is connected to the CBAM attention module, and the importance of each channel and space is weighted, and the same goes through 64, 128, 256 size convolutional layers; 3-4) Splicing the output tensors of the first four layers and modifying the size of the tensor to 512×1024×1, then entering 512 convolution kernels with a two-dimensional size of 1 for convolution and modifying the dimension to obtain a tensor size of 512 ×1024, use the flatten function to flatten the tensor and the output tensor size is 512, thus obtaining deep features including local and global descriptors; 4) Point cloud registration: To register two point clouds, that is, to obtain a rigid transformation matrix G∈SE(3), which is divided into a rotation matrix R∈SO(3) and a translation matrix t∈R ³ , and then solve To minimize the objective function F(G(R,t)):

Where ψ:R ^3×N →R ^K is the feature extraction function learned by the encoder, and k is the feature dimension; 4-1) G(R, t) is regarded as a special Lie group, which is represented by exponential mapping as follows:

Among them, T is the generator of the exponential mapping G, ε is a Lie algebra, which is mapped to the transformation matrix G through the exponential; 4-2) The point cloud registration problem is described as φ(PT)=φ(G·PT). To solve the matrix G, the derivative of G is obtained by taking the derivation of the Lie algebra ε, so as to directly adjust the Lie algebra ε to indirectly optimize the optimum. To optimize the transformation matrix G and minimize the feature space projection error of the two point clouds to perform the point cloud registration task, inspired by the (IC) formula, first inverse transform φ(PT)=φ(G PT) and right it Lateral linearization:

4-3) Transformation Estimation Iteratively runs the computation increment ε, by running the inverse synthesis (IC) algorithm to estimate ε for each step: ε = (J ^T J) ^-1 (J ^T δ) (9), where δ=φ(P _S )-φ(P _T ) is the projection error of the feature space of the two point clouds,

is the Jacobian matrix of the projection error δ relative to the transformation parameter ε; 4-4) In order to effectively calculate the Jacobian matrix, different finite gradients are used to replace the traditional stochastic gradient method to calculate the Jacobian matrix:

Among them, t _i is the infinitesimal disturbance of the transformation parameters during the calculation. When t _i is set to a fixed value, a better effect will be obtained. For t _i , three angle parameters are set for rotation and three disturbance parameters are used for translation. t _i size is set to 2*e ^-2 ; 4-5) Through the iterative operation formula (9), the transformation matrix is optimized to shorten the projection distance between the source point cloud and the template point cloud in the feature space: _ΔG ·PS→ _PS (11), in

Calculate the transformation matrix obtained by the transformation parameter ε for formula (9). After many iterations, the registration of P _S and P _T is continuously transformed, and the optimal transformation matrix G _est is finally output: G _est =ΔG _n ··· ΔG ₁ ΔG ₀ (12), 5) Training: In order to realize the training of the deep learning framework in an unsupervised manner, a decoder consisting of four fully connected layers is introduced; at the same time, ReLU is selected as the activation function of the first three layers, and the tanh function is the fourth layer. The activation function of ; 5-1) In the unsupervised case, use the alignment error between the transformed source point cloud and the target point cloud instead of the ground truth transformation for model training, using a robust loss function:

where P _T ∈ Θ is a set of points sampled from the unit square [0, 1] ² , x is the point cloud feature,

is the ith component in the parameters of the convolutional layer, and Φ is the 3D point of the original input.

Images