CN114782503A - Point cloud registration method and system based on multi-scale feature similarity constraint - Google Patents

Point cloud registration method and system based on multi-scale feature similarity constraint Download PDF

Info

Publication number
CN114782503A
CN114782503A CN202210337610.5A CN202210337610A CN114782503A CN 114782503 A CN114782503 A CN 114782503A CN 202210337610 A CN202210337610 A CN 202210337610A CN 114782503 A CN114782503 A CN 114782503A
Authority
CN
China
Prior art keywords
point cloud
point
source
matrix
module
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202210337610.5A
Other languages
Chinese (zh)
Inventor
舒禹程
侯宗壮
肖斌
李伟生
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Chongqing University of Post and Telecommunications
Original Assignee
Chongqing University of Post and Telecommunications
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Chongqing University of Post and Telecommunications filed Critical Chongqing University of Post and Telecommunications
Priority to CN202210337610.5A priority Critical patent/CN114782503A/en
Publication of CN114782503A publication Critical patent/CN114782503A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
    • G06T7/33—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/084—Backpropagation, e.g. using gradient descent
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10028—Range image; Depth image; 3D point clouds
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]

Abstract

The invention belongs to the field of computer vision graphics, and particularly relates to a point cloud registration method and system based on multi-scale feature similarity constraint.

Description

Point cloud registration method and system based on multi-scale feature similarity constraint
Technical Field
The invention belongs to the field of computer vision graphics, and particularly relates to a point cloud registration method and system based on multi-scale feature similarity constraint.
Background
The goal of 3D point cloud registration is to find an optimal spatial transformation to align one point cloud with another. Point cloud registration is an important research topic in the field of computer vision, and many applications are established on the basis, such as augmented reality, attitude estimation, three-dimensional scene reconstruction, and the like. The most widely affected conventional registration method is ICP (iterative Closest point) algorithm, which mainly assigns each point in one set to the nearest point in the other set continuously, and then finds a rigid transformation that meets the least squares through iterative computation, but ICP depends heavily on good initial conditions and easily falls into local optima.
With the development of deep learning, a point cloud registration method based on data driving is continuously proposed, features are usually extracted from data by the method, and then a corresponding relation is established according to feature similarity, so that the method has better registration accuracy, calculation efficiency and generalization capability compared with the traditional method. There are still some problems behind, such as not ideal enough for point cloud registration that only shares partial regions of coincidence. On one hand, a large number of outliers exist among the point clouds to bring interference to point cloud registration, and on the other hand, after the corresponding relation is established, wrong point pairs participate in reasoning. In order to solve the problem of local-to-local point cloud registration, it is usually necessary to screen key points and then establish a corresponding point relationship for the key points, although some researchers have proposed related methods such as an iterative multi-stage point elimination method, a mask eliminating the influence of non-coincident points on global characteristics of point clouds, and the like, these methods have problems of poor registration accuracy, easy overfitting, and the like for point clouds with low coincidence rate.
Disclosure of Invention
In order to solve the problems, the invention provides a point cloud registration method and system based on multi-scale feature similarity constraint.
In the first aspect, a point cloud registration method based on multi-scale feature similarity constraint is characterized by constructing and training a point cloud registration model, inputting data to be registered into the trained point cloud registration model, introducing multi-scale consistency constraint to obtain point pair weight, and then calculating a registration result;
the training process of the point cloud registration model comprises the following steps:
s1, acquiring a source point cloud and a target point cloud;
s2, extracting the original features of each point in the two groups of point clouds, exchanging the original features of the two groups of point clouds, obtaining the deep features of each point through MLP, performing maximum pooling on the deep features to obtain global features of the two groups of point clouds, splicing the global features of the two groups of point clouds with the deep features of each point, and outputting the confidence coefficient of each point through a fully-connected network; setting a source point cloud confidence coefficient loss function;
s3, local structural features of the two groups of point clouds are extracted by using GNN, GNN output features are recorded as first scale feature representations, and final feature representations of the two groups of point clouds after enhancement are respectively obtained through an attention mechanism and recorded as second scale feature representations;
s4, randomly selecting K points in the source point cloud in the two groups of point cloud overlapping areas to form a first point set, and searching out a second scale feature representation corresponding to the first point set;
s5, multiplying second scale feature representation corresponding to the first point set with second scale feature representation of the target point cloud to obtain a similar matrix;
s6, normalizing the similar matrix according to rows to obtain a corresponding distribution matrix of the first point set and the target point cloud, and multiplying the corresponding distribution matrix and the target point cloud to obtain a first virtual point set matched with the first point set; setting corresponding loss functions for the first virtual point set and the real corresponding point set;
s7, selecting the maximum element of each row in the corresponding distribution matrix, and carrying out normalization processing on the maximum element to obtain the weight of the matching point pair between the first point set and the first virtual point set;
s8, solving a rotation matrix and a translation matrix between the source point cloud and the target point cloud by adopting an SVD algorithm based on the corresponding distribution matrix and the weight of the matching point pair;
s9, transforming the source point cloud according to the rotation matrix and the translation matrix, returning to the step S2, and iterating for 3 times;
and S10, starting a gradient reverse propagation mechanism, optimizing a loss function, updating network parameters, and storing the model after the model converges or reaches the set epoch times.
Further, introducing consistency constraint in the trained point cloud registration model comprises:
calculating a similar matrix from the source point cloud to the target point cloud according to the first scale feature representation, and recording the similar matrix as M1, and calculating a similar matrix from the source point cloud to the target point cloud according to the second scale feature representation, and recording the similar matrix as M2;
multiplying M1 and M2 by bit to obtain a new similar matrix, and normalizing the similar matrix by rows to obtain a corresponding point set of the source point cloud in the target point cloud;
selecting the maximum element of each row in the new similar matrix, aligning and normalizing to obtain the weight of a matching point pair between the source point cloud and the corresponding point set;
based on the new similarity matrix and the weight of the matching point pair, solving a rotation matrix and a translation matrix between the source point cloud and the target point cloud by adopting an SVD (singular value decomposition) algorithm; and calculating to obtain a final transformation matrix.
Furthermore, a bidirectional loss function is used in the training stage, so that the registration process from the source point cloud to the target point cloud is optimized, and the registration process from the target point cloud to the source point cloud is also optimized.
Further, the two-way loss function in the point cloud registration model training stage is represented as:
Figure BDA0003577198440000031
wherein, the first and the second end of the pipe are connected with each other,
Figure BDA0003577198440000032
representing the loss of confidence in the source point cloud,
Figure BDA0003577198440000033
representing the loss of confidence in the target point cloud,
Figure BDA0003577198440000034
representing the loss between a virtual point in the target point cloud corresponding to the source point cloud and a real point,
Figure BDA0003577198440000035
representing pairs of source point clouds and target point cloudsThe loss between the corresponding virtual point and the real point.
Further, the confidence loss of the source point cloud is calculated by using the typical cross entropy loss, and the formula is as follows:
Figure BDA0003577198440000036
wherein GT (x)i) Represents point xiThe real label of (2) for representing the point xiWhether the point belongs to the overlapped region or the non-overlapped region of the two groups of point clouds is represented by 1 and 0 respectively, P (x)i) Representing a point x in a source point cloudiThe confidence of (c).
Further, the loss function error between the virtual point and the real point in the target point cloud corresponding to the source point cloud
Figure BDA0003577198440000037
Expressed as:
Figure BDA0003577198440000038
wherein x isiRepresenting a point in the source point cloud and,
Figure BDA0003577198440000039
is denoted by xiGenerated virtual corresponding points, (R)XYxi+tXY) Representing the sum x in the target point cloudiThe corresponding real point.
In a second aspect, the present invention provides a point cloud registration system based on multi-scale feature similarity constraint, including:
the data collection module is used for acquiring source point cloud data and target point cloud data input into the model;
the down-sampling module is used for screening out point sets participating in registration;
the characteristic exchange module is used for extracting the characteristics of each point in the source point cloud and the target point cloud and then exchanging the characteristic information of the two groups of point clouds;
the MLP module is used for extracting deep features of the two groups of point clouds by adopting a plurality of MLPs based on the output of the feature exchange module;
the pooling module is used for performing maximum pooling on the deep features of the two groups of point clouds to obtain the global features of the source point clouds and the global features of the target point clouds;
the splicing module is used for fusing the characteristic information of the points and splicing the global characteristics of the source point cloud and the target point cloud with the deep characteristics of each point;
the confidence coefficient calculation module is used for inputting the splicing characteristics of each point into the full-connection layer to obtain the confidence coefficient of each point;
the local structure characteristic acquisition module is used for acquiring first scale characteristic representation of source point cloud and target point cloud;
the attention module is used for acquiring association information between the point clouds and enhancing point cloud feature representation, namely processing the first scale feature representation according to a self-attention mechanism and a cross-attention mechanism to obtain a second scale feature representation of the two groups of point clouds;
the candidate point set module is used for selecting K points with the highest confidence degree from the source point cloud to form a first point set according to the output result of the confidence degree calculation module;
the index module is used for screening different scale feature representations of the first point set in the registration process;
the matching module is used for acquiring a corresponding point set matched with the first point set from the target point cloud;
the consistency constraint module is used for introducing consistency constraint into the point cloud registration model to obtain a corresponding distribution matrix of the first point set and the target point cloud;
the SVD solving module is used for calculating a rotation matrix and a translation matrix between the source point cloud and the target point cloud according to the result of the virtual matching module and the weight obtained by the consistency constraint module;
and the transformation module is used for transforming the source point cloud according to the rotation matrix and the translation matrix.
The invention has the beneficial effects that:
the invention provides a point cloud registration method and a point cloud registration system based on multi-scale feature similarity constraint, which are characterized in that the matching reliability of points is actively evaluated through a multi-scale feature exchange network, meanwhile, a network of an attention fusion mechanism is adopted to extract distinctive features, a bidirectional alternative corresponding search mechanism and a flexible dislocation loss function are used to perform robust learning, a point pair weight distribution scheme based on the feature similarity consistency constraint established by multiple scales is introduced in an inference stage, and finally, a transformation matrix is solved by weighted SVD (singular value decomposition) so as to improve the registration precision.
Drawings
FIG. 1 is a diagram of a point cloud registration structure based on multi-scale feature similarity constraint according to the present invention;
FIG. 2 is a graph illustrating the correspondence between the registration result and the high score based on the consistency constraint according to the present invention;
fig. 3 is a graph of the registration result according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be described clearly and completely with reference to the accompanying drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The invention provides a point cloud registration method based on multi-scale feature similarity constraint, as shown in FIG. 1, comprising the following steps:
s1, acquiring source point cloud and target point cloud, wherein the maximum epoch frequency is set to be 100 in the embodiment, and the initial learning rate is 0.0001;
s2, extracting original features of each point in the two groups of point clouds, exchanging the original features of the two groups of point clouds, finally obtaining deep features of each point through a plurality of MLPs, performing maximum pooling on the deep features of each point to obtain global features of the source point cloud and global features of the target point cloud, splicing the global features of the two groups of point clouds with the deep features of each point, and outputting the confidence coefficient of each point through a fully-connected network;
and (3) representing the high-reliability matching degree between the point clouds by using the confidence coefficient, wherein after the model is trained, the higher the confidence coefficient is, the more possible the point is the point of the overlapping area of the source point cloud and the target point cloud, and the higher the matching accuracy is.
Specifically, under difficult conditions such as noise, outliers or large transformations, the use of potential corresponding points with high matching reliability is beneficial to the robustness of registration, but in local-local point cloud registration, non-coincident regions do not have a correspondence in another point cloud, so a multi-scale feature exchange network is proposed to focus on the global semantics of the coincident trusted regions of two sets of point clouds, and the original features of the two point clouds are exchanged using a transform-based mechanism, and the formula can be expressed as:
fx＝fx+ζ(fx,fy)；
fy＝fy+ζ(fy,fx)；
where ζ (-) is the residual term based on Transformer model learning, fxRepresenting the original features of the source point cloud, fyRepresenting the original features of the target point cloud.
Specifically, the spliced points are sent to the full-link layer, and semantic information of each spliced point is obtained, that is, the confidence of each point is expressed as:
Figure BDA0003577198440000061
Figure BDA0003577198440000062
wherein G isXGlobal features representing source point clouds, GYRepresenting global features of the target point cloud, SxSet of confidence levels, S, representing each point of the source point cloudySet of confidence values, ρ, representing each point of the target point clouds() Representing a fully connected network, r () represents a repeat of a global feature.
S3, local structural features of the two groups of point clouds are extracted by using GNN, GNN output features are recorded as first scale feature representations, and final feature representations of the two groups of point clouds after enhancement are respectively obtained through an attention mechanism and recorded as second scale feature representations;
specifically, the first scale feature representation of the two groups of point clouds is input into an attention mechanism to obtain two groups of results, the two groups of results are input into a cross attention mechanism to output a second scale feature representation of the two groups of point clouds, wherein the source point clouds represent the first scale feature representation through the two layers of attention mechanisms, and the process of obtaining the second scale feature representation is represented as follows:
Figure BDA0003577198440000063
Figure BDA0003577198440000064
wherein psiXThe first scale features representing the source point cloud represent the result after the autofocusing mechanism, ΦXA second scale feature representation representing the source point cloud,
Figure BDA0003577198440000071
a self-attention mechanism is shown in which,
Figure BDA0003577198440000072
representing a cross-attention mechanism, the first scale features of the target point cloud represent the same process as the source point cloud through a two-layer attention mechanism.
S4, randomly selecting K points in the source point cloud in the two groups of point cloud overlapping areas to form a first point set, and searching out second scale feature representations corresponding to the first point set;
in the training process, points in the overlapping area of the source point cloud and the target point cloud are randomly selected for training, so that the points of the source point cloud and the target point cloud can be ensured to establish a corresponding relation and can be coupled with a subsequent process; after training is completed, when the model is used for registration, the first K points with the highest confidence level are selected from the source point cloud to form a first point set.
S5, multiplying second scale feature representation corresponding to the first point set with second scale feature representation of the target point cloud to obtain a similar matrix;
s6, normalizing the similar matrix according to rows to obtain a corresponding distribution matrix of the first point set and the target point cloud, and multiplying the corresponding distribution matrix and the target point cloud to obtain a first virtual point set matched with the first point set; setting corresponding loss functions for the first virtual point set and the real corresponding point set;
s7, selecting the maximum element of each row in the corresponding distribution matrix, and carrying out normalization processing on the maximum elements to obtain the weight of a matching point pair between the first point set and the first virtual point set;
s8, solving a rotation matrix and a translation matrix between the source point cloud and the target point cloud by adopting an SVD algorithm based on the corresponding distribution matrix and the weight of the matching point pair;
s9, transforming the source point cloud according to the rotation matrix and the translation matrix, returning to the step S2, and iterating for 3 times;
and S10, starting a gradient reverse propagation mechanism, optimizing a loss function, updating network parameters, and storing the model after the model converges or reaches the set epoch times.
Specifically, the point cloud registration model is a bidirectional model, that is, when the source point cloud is registered to the target point cloud, the target point cloud is registered to the source point cloud by the same method.
In the point cloud registration process, corresponding points are established based on the feature similarity of the points to form a pair of point pairs, and the derivation process is inevitably influenced by wrong point-to-point relationships, so that after the corresponding relationships are established for K key points in a first point set, correct corresponding points cannot be found out, and in order to alleviate the problem, the reliable point-to-point relationships are further screened based on the consistency thought, and the reliable point-to-point relationships comprise the following steps:
according to the first scale feature representation, calculating a similarity matrix from the source point cloud to the target point cloud, and recording as M1, according to the second scale feature representation, calculating a similarity matrix from the source point cloud to the target point cloud, and recording as M2;
multiplying M1 and M2 by bit to obtain a new similar matrix, and normalizing the similar matrix by rows to obtain a corresponding point set of the source point cloud in the target point cloud;
selecting the maximum element of each row in the new similar matrix, aligning and normalizing to obtain the weight of a matching point pair between the source point cloud and the corresponding point set;
based on the new similar matrix and the weight of the matching point pair, solving a rotation matrix and a translation matrix between the source point cloud and the target point cloud by adopting an SVD (singular value decomposition) algorithm; and calculating to obtain a final transformation matrix.
Specifically, the fusion of the new similarity matrix is represented as:
Figure BDA0003577198440000081
wherein the content of the first and second substances,
Figure BDA0003577198440000082
a matrix of M1 is represented,
Figure BDA0003577198440000083
indicating an M2 matrix, indicating a bit-wise multiplication of matrix elements,
Figure BDA0003577198440000084
representing the feature representation of the source point cloud at the first scale by indexing the first set of points,
Figure BDA0003577198440000085
a first scale feature representation representing N points in the target point cloud,
Figure BDA0003577198440000086
representing a feature representation of the source point cloud indexing the first set of points in the second scale feature representation,
Figure BDA0003577198440000087
expressing the second scale feature representation of N points in the target point cloud, and based on the consistency of feature similarity, the inconsistent distribution of the two matrixes can cause the similarity after normalization to be obtainedScore reduction, so to eliminate the effect of this unreliable correspondence on reasoning, C is reselectedXYThe maximum element of each row in the Chinese character is normalized again and then is according to CXYThe certainty of each row yields the weight of the contribution of the matching pairs to the calculation:
Figure BDA0003577198440000088
wherein, XKRepresenting a set of points, Y, obtained by downsampling a source point cloudNA point cloud of the target point is represented,
Figure BDA0003577198440000089
is X in SVDKTo YNA corresponding point pair weight is established and,
Figure BDA00035771984400000810
is a function of the value 0 or 1, with YKRepresenting a collection of points, X, obtained by sampling the target point cloudMRepresenting a source point cloud, by YKTo XMThe weights are calculated in the same way.
In one embodiment, four corresponding loss functions are provided, which together form the overall loss function of the present invention, expressed as:
Figure BDA0003577198440000091
wherein the content of the first and second substances,
Figure BDA0003577198440000092
representing the loss of confidence in the source point cloud,
Figure BDA0003577198440000093
representing the loss of confidence in the target point cloud,
Figure BDA0003577198440000094
representing the error between the virtual point and the real point in the target point cloud corresponding to the source point cloud,
Figure BDA0003577198440000095
and representing the error between the virtual point and the real point corresponding to the target point cloud in the source point cloud. In the embodiment, the point cloud registration model is trained by adopting the bidirectional loss function, and the registration accuracy of the model is further improved while the training sample is expanded.
Specifically, the confidence loss of the source point cloud is calculated by using a typical cross entropy loss, and the formula is as follows:
Figure BDA0003577198440000096
wherein GT (x)i) Represents point xiTrue tag of, P (x)i) Representing a source point cloud midpoint xiThe confidence of (c).
Specifically, the error between a virtual point and a real point in the target point cloud corresponding to the source point cloud
Figure BDA0003577198440000097
Expressed as:
Figure BDA0003577198440000098
wherein x isiRepresenting a point in the source point cloud and,
Figure BDA0003577198440000099
is denoted by xiGenerated virtual corresponding points, (R)XYxi+tXY) Representing the sum x in the target point cloudiThe corresponding real point.
The invention provides a point cloud registration system based on multi-scale feature similarity constraint, which comprises the following steps:
the data collection module is used for acquiring source point cloud data and target point cloud data input into the model;
the down-sampling module is used for screening out point sets participating in registration;
the characteristic exchange module is used for extracting the characteristics of each point in the source point cloud and the target point cloud and then exchanging the characteristic information of the two groups of point clouds;
the MLP module is used for extracting deep features of the two groups of point clouds by adopting a plurality of MLPs based on the output of the feature exchange module;
the pooling module is used for performing maximum pooling on the deep features of the two groups of point clouds to obtain the global feature of the source point cloud and the global feature of the target point cloud;
the splicing module is used for fusing the characteristic information of the points and splicing the global characteristics of the source point cloud and the target point cloud with the deep characteristics of each point;
the confidence coefficient calculation module is used for inputting the splicing characteristics of each point into the full-connection layer to obtain the confidence coefficient of each point;
the local structure characteristic acquisition module is used for acquiring first scale characteristic representation of source point cloud and target point cloud;
the attention module is used for acquiring the association information between the point clouds and enhancing the point cloud feature representation, namely processing the first scale feature representation according to a self-attention mechanism and a cross-attention mechanism to obtain a second scale feature representation of the two groups of point clouds;
the candidate point set module is used for selecting K points with the highest confidence level from the source point cloud to form a first point set according to the output result of the confidence level calculation module;
the index module is used for screening different scale feature representations of the first point set in the registration process;
the matching module is used for acquiring a corresponding point set matched with the first point set from the target point cloud;
the consistency constraint module is used for introducing consistency constraint into the point cloud registration model to obtain a corresponding distribution matrix of the first point set and the target point cloud;
the SVD solving module is used for calculating a rotation matrix and a translation matrix between the source point cloud and the target point cloud according to the result of the matching module and the weight obtained by the consistency constraint module;
and the transformation module is used for transforming the source point cloud according to the rotation matrix and the translation matrix.
In one embodiment, the experiment is performed using a dataset (ModelNet40), which is a composite dataset that is widely used for registration tasks, and which contains 12311 CAD models, typically 9843 for training sets and 2468 for testing sets. For obtaining two point clouds in registration, downsampling and analog rotation translation are needed for sample preparation. And marking each original sample as a source point cloud, randomly generating three Euler angles in [0,45 degrees ] along three X/Y/Z axes in order to obtain a target point cloud, setting translation parameters between [ -0.5,0.5], and applying rotational translation to the source point cloud to obtain the target point cloud. In order to obtain partial point cloud pairs, two random sampling points are set, and 768 points are reserved respectively. The experiments were divided into three groups, the first: training on all categories, and then testing in a test set; second group: training in the first 20 classes and testing in the last twenty classes; third group: training in noisy data and then testing the focused test. The Root Mean Square Error (RMSE) and the Mean Absolute Error (MAE) are used to measure the quality of the registration, and table 1 shows the experimental results obtained by performing a second set of tests in a different manner.
TABLE 1 test results on unseen categories
Method RMSE(R) MAE(R) RMSE(t) MAE(t)
ICP 37.9513 26.7103 0.29395 0.2507
DCP 9.5585 6.9228 0.0461 0.0348
PRNet 6.0438 3.6295 0.0425 0.0291
GNN+IDAM 37.5242 17.9965 0.1028 0.0549
As can be seen from table 1, the errors of the rotation angle (R) and the translation parameter (t) obtained by the present invention are greatly reduced by performing tests with different algorithms, and in addition, as shown in fig. 2, the first column is two input point clouds, the second column is the registration result of the present invention based on the consistency constraint, the third column is the real registration result, and the last column represents the high-weight correspondence established based on the consistency constraint, which indicates that the present invention can effectively screen more reliable point pairs based on the feature similarity consistency constraint, and fig. 3 is the visualization of the registration result in three sets of experiments, and as can be seen from the results of fig. 2 and 3, the present invention is effective.
Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that changes, modifications, substitutions and alterations can be made in these embodiments without departing from the principles and spirit of the invention, the scope of which is defined in the appended claims and their equivalents.

Claims (7)

1. A point cloud registration method based on multi-scale feature similarity constraint is characterized in that a point cloud registration model is constructed and trained, data to be registered are input into the trained point cloud registration model, multi-scale consistency constraint is introduced to obtain point pair weight, and then a registration result is calculated;
the training process of the point cloud registration model comprises the following steps:
s1, acquiring source point cloud and target point cloud;
s2, extracting the original features of each point in the two groups of point clouds, exchanging the original features of the two groups of point clouds, obtaining the deep features of each point through MLP, performing maximum pooling on the deep features to obtain global features of the two groups of point clouds, splicing the global features of the two groups of point clouds with the deep features of each point, and outputting the confidence coefficient of each point through a fully-connected network; setting a source point cloud confidence coefficient loss function;
s3, local structural features of the two groups of point clouds are extracted by using GNN, GNN output features are recorded as first scale feature representations, and final feature representations of the two groups of point clouds after enhancement are respectively obtained through an attention mechanism and recorded as second scale feature representations;
s4, randomly selecting K points in the source point cloud in the two groups of point cloud overlapping areas to form a first point set, and indexing a second scale feature representation corresponding to the first point set;
s5, multiplying second scale characteristic representation corresponding to the first point set and second scale characteristic representation of the target point cloud to obtain a similar matrix;
s6, normalizing the similar matrix according to rows to obtain a corresponding distribution matrix of the first point set and the target point cloud, and multiplying the corresponding distribution matrix and the target point cloud to obtain a first virtual point set matched with the first point set; setting corresponding loss functions for the first virtual point set and the real corresponding point set;
s7, selecting the maximum element of each row in the corresponding distribution matrix, and carrying out normalization processing on the maximum element to obtain the weight of the matching point pair between the first point set and the first virtual point set;
s8, solving a rotation matrix and a translation matrix between the source point cloud and the target point cloud by adopting an SVD algorithm based on the corresponding distribution matrix and the weight of the matching point pair;
s9, transforming the source point cloud according to the rotation matrix and the translation matrix, returning to the step S2, and iterating for 3 times;
and S10, starting a gradient reverse propagation mechanism, optimizing a loss function, updating network parameters, and storing the model after the model converges or reaches the set epoch times.
2. The point cloud registration method based on multi-scale feature similarity constraint according to claim 1, wherein introducing consistency constraint in the trained point cloud registration model comprises:
according to the first scale feature representation, calculating a similarity matrix from the source point cloud to the target point cloud, and recording as M1, according to the second scale feature representation, calculating a similarity matrix from the source point cloud to the target point cloud, and recording as M2;
multiplying M1 and M2 by bit to obtain a new similar matrix, and normalizing the similar matrix by rows to obtain a corresponding point set of the source point cloud in the target point cloud;
selecting the maximum element of each row in the new similar matrix, and normalizing the maximum element to obtain the weight of a matching point pair between the source point cloud and the corresponding point set;
and based on the new similarity matrix and the weight of the matching point pair, solving a rotation matrix and a translation matrix between the source point cloud and the target point cloud by adopting an SVD (singular value decomposition) algorithm, and calculating to obtain a final transformation matrix.
3. The method of claim 1, wherein the training phase uses a bi-directional loss function to optimize both the source point cloud to target point cloud registration process and the target point cloud to source point cloud registration process.
4. The point cloud registration method based on multi-scale feature similarity constraint of claim 3, wherein the bidirectional loss function in the training stage of the point cloud registration model is represented as:
Figure FDA0003577198430000021
wherein the content of the first and second substances,
Figure FDA0003577198430000022
representing the loss of confidence in the source point cloud,
Figure FDA0003577198430000023
representing the loss of confidence in the target point cloud,
Figure FDA0003577198430000024
representing the loss between a virtual point in the target point cloud corresponding to the source point cloud and a real point,
Figure FDA0003577198430000025
representing the loss between the virtual point in the source point cloud corresponding to the target point cloud and the real point.
5. The point cloud registration method based on multi-scale feature similarity constraint of claim 4, wherein the confidence coefficient loss of the source point cloud is calculated by using a typical cross entropy loss, and the formula is as follows:
Figure FDA0003577198430000026
wherein GT (x)i) Represents point xiTrue tag of (2), P (x)i) Representing the cloud midpoint x of the source pointiThe confidence of (c).
6. The point cloud registration method based on multi-scale feature similarity constraint of claim 4, whereinCharacterized by a loss between a virtual point corresponding to the source point cloud and a real point in the target point cloud
Figure FDA0003577198430000031
Expressed as:
Figure FDA0003577198430000032
wherein x isiRepresenting a point in the source point cloud and,
Figure FDA0003577198430000033
is denoted by xiGenerated virtual corresponding points, (R)XYxi+tXY) Representing the sum x in a target point cloudiThe corresponding real point.
7. A point cloud registration system based on multi-scale feature similarity constraint is characterized by comprising:
the data collection module is used for acquiring source point cloud data and target point cloud data input into the model;
the down-sampling module is used for screening out point sets participating in registration;
the characteristic exchange module is used for extracting the characteristics of each point in the source point cloud and the target point cloud and then exchanging the characteristic information of the two groups of point clouds;
the MLP module is used for extracting deep features of the two groups of point clouds by adopting a plurality of MLPs based on the output of the feature exchange module;
the pooling module is used for performing maximum pooling on the deep features of the two groups of point clouds to obtain the global features of the source point clouds and the global features of the target point clouds;
the splicing module is used for fusing the characteristic information of the points and splicing the global characteristics of the source point cloud and the target point cloud with the deep characteristics of each point;
the confidence coefficient calculation module is used for inputting the splicing characteristics of each point into the full-connection layer to obtain the confidence coefficient of each point;
the local structural feature acquisition module is used for acquiring first scale feature representations of the source point cloud and the target point cloud;
the attention module is used for acquiring the association information between the point clouds and enhancing the point cloud feature representation, namely processing the first scale feature representation according to a self-attention mechanism and a cross-attention mechanism to obtain a second scale feature representation of the two groups of point clouds;
the candidate point set module is used for selecting K points with the highest confidence level from the source point cloud to form a first point set according to the output result of the confidence level calculation module;
the index module is used for screening different scale feature representations of the first point set in the registration process;
the matching module is used for acquiring a corresponding point set matched with the first point set from the target point cloud;
the consistency constraint module is used for introducing consistency constraint into the point cloud registration model to obtain a corresponding distribution matrix of the first point set and the target point cloud;
the SVD solving module is used for calculating a rotation matrix and a translation matrix between the source point cloud and the target point cloud according to the result of the matching module and the weight obtained by the consistency constraint module;
and the transformation module is used for transforming the source point cloud according to the rotation matrix and the translation matrix.
CN202210337610.5A 2022-04-01 2022-04-01 Point cloud registration method and system based on multi-scale feature similarity constraint Pending CN114782503A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210337610.5A CN114782503A (en) 2022-04-01 2022-04-01 Point cloud registration method and system based on multi-scale feature similarity constraint

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210337610.5A CN114782503A (en) 2022-04-01 2022-04-01 Point cloud registration method and system based on multi-scale feature similarity constraint

Publications (1)

Publication Number Publication Date
CN114782503A true CN114782503A (en) 2022-07-22

Family

ID=82426641

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210337610.5A Pending CN114782503A (en) 2022-04-01 2022-04-01 Point cloud registration method and system based on multi-scale feature similarity constraint

Country Status (1)

Country Link
CN (1) CN114782503A (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115272433A (en) * 2022-09-23 2022-11-01 武汉图科智能科技有限公司 Light-weight point cloud registration method and system for automatic obstacle avoidance of unmanned aerial vehicle
CN116228825A (en) * 2023-01-29 2023-06-06 重庆邮电大学 Point cloud registration method based on significant anchor point geometric embedding

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115272433A (en) * 2022-09-23 2022-11-01 武汉图科智能科技有限公司 Light-weight point cloud registration method and system for automatic obstacle avoidance of unmanned aerial vehicle
CN115272433B (en) * 2022-09-23 2022-12-09 武汉图科智能科技有限公司 Light-weight point cloud registration method and system for automatic obstacle avoidance of unmanned aerial vehicle
CN116228825A (en) * 2023-01-29 2023-06-06 重庆邮电大学 Point cloud registration method based on significant anchor point geometric embedding
CN116228825B (en) * 2023-01-29 2024-01-23 武汉天宝耐特科技有限公司 Point cloud registration method based on significant anchor point geometric embedding

Similar Documents

Publication Publication Date Title
CN112926396B (en) Action identification method based on double-current convolution attention
CN109919245B (en) Deep learning model training method and device, training equipment and storage medium
CN110659727A (en) Sketch-based image generation method
CN114782503A (en) Point cloud registration method and system based on multi-scale feature similarity constraint
CN114241273A (en) Multi-modal image processing method and system based on Transformer network and hypersphere space learning
CN116049467A (en) Non-supervision image retrieval method and system based on label visual joint perception
CN111091010A (en) Similarity determination method, similarity determination device, network training device, network searching device and storage medium
CN115222998B (en) Image classification method
CN113096169A (en) Non-rigid multimode medical image registration model establishing method and application thereof
CN113763442A (en) Deformable medical image registration method and system
CN107451617B (en) Graph transduction semi-supervised classification method
CN112364747A (en) Target detection method under limited sample
CN114612902A (en) Image semantic segmentation method, device, equipment, storage medium and program product
CN114219824A (en) Visible light-infrared target tracking method and system based on deep network
CN108805280B (en) Image retrieval method and device
Kim et al. Self-supervised keypoint detection based on multi-layer random forest regressor
CN116975350A (en) Image-text retrieval method, device, equipment and storage medium
CN112905894B (en) Collaborative filtering recommendation method based on enhanced graph learning
CN113536925A (en) Crowd counting method based on attention guide mechanism
CN116071410A (en) Point cloud registration method, system, equipment and medium based on deep learning
CN117036760A (en) Multi-view clustering model implementation method based on graph comparison learning
CN116958324A (en) Training method, device, equipment and storage medium of image generation model
CN115601745A (en) Multi-view three-dimensional object identification method facing application end
CN112559640B (en) Training method and device of atlas characterization system
CN114612535A (en) Image registration method, system, device and medium based on partial differential countermeasure learning

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination