CN114037743B - Three-dimensional point cloud robust registration method for Qin warriors based on dynamic graph attention mechanism - Google Patents
Three-dimensional point cloud robust registration method for Qin warriors based on dynamic graph attention mechanism Download PDFInfo
- Publication number
- CN114037743B CN114037743B CN202111245398.1A CN202111245398A CN114037743B CN 114037743 B CN114037743 B CN 114037743B CN 202111245398 A CN202111245398 A CN 202111245398A CN 114037743 B CN114037743 B CN 114037743B
- Authority
- CN
- China
- Prior art keywords
- point cloud
- qin
- registration
- network
- dimensional point
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 230000007246 mechanism Effects 0.000 title claims abstract description 24
- 238000000034 method Methods 0.000 title claims abstract description 22
- 230000008859 change Effects 0.000 claims abstract description 7
- 239000011159 matrix material Substances 0.000 claims abstract description 5
- 238000005070 sampling Methods 0.000 claims description 6
- 238000000605 extraction Methods 0.000 claims description 3
- 230000009191 jumping Effects 0.000 claims 1
- 230000006870 function Effects 0.000 description 10
- 238000010586 diagram Methods 0.000 description 9
- 230000009466 transformation Effects 0.000 description 5
- 238000004364 calculation method Methods 0.000 description 4
- 238000010606 normalization Methods 0.000 description 4
- 208000037170 Delayed Emergence from Anesthesia Diseases 0.000 description 3
- 238000012512 characterization method Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000002776 aggregation Effects 0.000 description 1
- 238000004220 aggregation Methods 0.000 description 1
- 125000004122 cyclic group Chemical group 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000008034 disappearance Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000004880 explosion Methods 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 238000000844 transformation Methods 0.000 description 1
- 230000007704 transition Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
- G06T7/33—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods
- G06T7/344—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods involving models
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10028—Range image; Depth image; 3D point clouds
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Artificial Intelligence (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Life Sciences & Earth Sciences (AREA)
- General Engineering & Computer Science (AREA)
- Processing Or Creating Images (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a three-dimensional point cloud robust registration method of Qin warriors based on a dynamic graph attention mechanism, which comprises the following steps: step 1, acquiring three-dimensional point clouds of Qin figurines with different resolutions through a three-dimensional scanner; step 2, in the U-Net network, replacing the convolution layer with B-NHN-Conv, replacing the deconvolution layer with B-NHN-ConvTr, and embedding a residual error module and a dynamic graph attention mechanism into the U-Net network to obtain a point cloud registration network; step 3, three-dimensional point clouds of the Qin warriors with different resolutions are input into a point cloud registration network, and training is carried out under supervision of a circle loss function and an overlap loss function; and 4, extracting three-dimensional point cloud characteristics of the Qin figurines by using a trained point cloud registration network, estimating a change matrix between a source point cloud and a target point cloud by combining a RANSAC algorithm, and finishing registration of the three-dimensional point cloud of the Qin figurines. The registration method provided by the invention can still learn robust features and well finish the registration of the point cloud under the condition of unmatched point cloud resolution and large amount of noise.
Description
Technical Field
The invention relates to a three-dimensional point cloud model registration technology, in particular to a three-dimensional point cloud robust registration method for Qin figurines based on a dynamic graph attention mechanism.
Background
The point cloud registration technology plays an important role in projects such as virtual restoration of Qin warriors, intelligent museums of Qinling, and the like, and the accuracy of a point cloud registration result is a key of subsequent three-dimensional reconstruction. The point cloud registration aims at obtaining a perfect coordinate transformation through calculation, and integrating the point cloud data under different view angles into a specified coordinate system through rotation translation and other rigid transformations.
At present, most of registration methods of the Qin warriors and related cultural relics are based on optimized traditional registration methods, wherein a more classical method is an iterative closest point (Iterative Closest Point) algorithm. The method mainly comprises two stages: corresponding search and transform estimates. These two phases will be iterated to find the best transition between the point clouds. However, when the method is used for processing scenes with large initial position difference, unmatched point cloud resolution, strong noise interference and small overlapping degree, the method is easy to fall into local optimum. In recent years, researchers have proposed a deep learning-based method to learn robust feature computation correspondence points, and finally determine a transformation matrix through RANSAC or SVD algorithm, without requiring iteration between correspondence estimation and transformation estimation. However, such algorithms cannot handle quickly and robustly in the face of challenges such as partial overlap, density variations, noise, etc.
Disclosure of Invention
Aiming at the defects existing in the prior art, the invention aims to provide the three-dimensional point cloud robust registration method of the Qin warriors based on the dynamic graph attention mechanism, which can be robustly processed when facing partial overlapping and density change.
In order to achieve the above purpose, the invention is realized by adopting the following technical scheme:
a three-dimensional point cloud robust registration method of Qin warriors based on a dynamic graph attention mechanism comprises the following steps:
step 1, acquiring three-dimensional point clouds of Qin figurines with different resolutions through a three-dimensional scanner;
step 2, in the U-Net network, replacing the convolution layer with B-NHN-Conv, replacing the deconvolution layer with B-NHN-ConvTr, and embedding a residual error module and a dynamic graph attention mechanism into the U-Net network to obtain a point cloud registration network;
step 3, three-dimensional point clouds of the Qin warriors with different resolutions are input into a point cloud registration network, and the established point cloud registration network is trained under the common supervision of a circle loss function and an overlap loss function;
and 4, extracting three-dimensional point cloud characteristics of the Qin figurines by using a trained point cloud registration network, estimating a change matrix between a source point cloud and a target point cloud by combining a RANSAC algorithm, and finishing registration of the three-dimensional point cloud of the Qin figurines.
Further, the U-Net network in step 2 further comprises an encoder module and a decoder module, the encoder module obtaining a plurality of scale features through a plurality of downsampling, all the full connection layers in the multi-layer perceptron are converted into a series of full convolution layers with the kernel size of 1 multiplied by 1, and the channel number is 64,128,256,512; and each time the decoder module performs up-sampling, the information loss caused by down-sampling is reduced by fusing the features of the same scale corresponding to the channel number through the jump connection structure and the feature extraction part.
Further, the residual module in the step 2 is formed by connecting a plurality of residual blocks in series, and is used for jumping-connecting the input information to the output information.
Further, the dynamic graph attention mechanism in the step 2 comprises a combination module of a multi-layer self-attention module and a cross-attention module, and the dynamic graph attention mechanism is based on the attention weight a ij Find the top k edges for each query node and construct the graph using only the top k edges and corresponding nodes and based on the changed attention weights a at each level of the dynamic graph attention mechanism ij A new graph is constructed.
Compared with the prior art, the invention has the following technical effects:
the invention uses U-Net as a backbone architecture, and solves the problems of gradient disappearance or gradient explosion by embedding a residual block; local features and contextual features of the point clouds are aggregated by embedding a dynamic graph attention mechanism, so that multi-level semantic characterization with more abundant semantic characterization can be obtained, and a network can be better helped to determine possible overlapping areas between the point clouds; the potential change of the characteristic mean value and the standard deviation is reduced or removed through the B-NHN-Conv convolution operation, so that the robustness of the three-dimensional characteristic to the point density change is improved; the point cloud registration network constructed by the method can learn robust features and well complete the registration of the point clouds under the condition of unmatched point cloud resolutions and a large amount of noise.
Further, all the full-connection layers in the multi-layer perceptron are converted into a series of full-convolution layers with the kernel size of 1 multiplied by 1, so that the efficiency of network processing can be accelerated.
Drawings
Fig. 1 is a flow chart of a three-dimensional point cloud robust registration method of Qin warriors based on a dynamic diagram attention mechanism;
FIG. 2 is a graph of a point cloud registration network model;
FIG. 3 is a schematic diagram of the structure of the dynamic diagram attention mechanism;
FIG. 4 is a schematic diagram of a residual module;
fig. 5 is an initial pose diagram of point clouds at two heads of the Qin warrior;
fig. 6 is a graph of the registration results of two head point clouds of Qin warriors;
fig. 7 is an initial pose diagram of two foot point clouds of the Qin warrior;
fig. 8 is a graph of the registration results of the two foot points of the Qin warriors.
Detailed Description
The following examples illustrate the invention in further detail.
Referring to fig. 1-4, the embodiment provides a three-dimensional point cloud robust registration method of Qin figurines based on a dynamic diagram attention mechanism, which comprises the following steps:
step 1, acquiring three-dimensional point clouds of Qin figurines with different resolutions through a three-dimensional scanner;
step 2, in the U-Net network, replacing the convolution layer with B-NHN-Conv, replacing the deconvolution layer with B-NHN-ConvTr, and embedding a residual error module and a dynamic graph attention mechanism into the U-Net network to obtain a point cloud registration network; wherein,
B-NHN-Conv is an operation combining B-NHN normalization and subsequent three-dimensional sparse convolution, and the B-NHN normalization and the three-dimensional sparse convolution are tightly coupled; B-NHN-ConvTr is obtained by transposing B-NHN-Conv convolution operation, namely combining B-NHN normalization operation with a three-dimensional sparse transpose convolution function; the B-NHN normalization operation can reduce or eliminate potential variation of feature mean and standard deviation, and improve robustness of three-dimensional feature to point density variation.
The residual module is formed by connecting a plurality of residual blocks in series and is used for jumping-connecting input information to output information; the problem of losing effective characteristic information can be relieved, so that the network is more easily optimized, and the establishment of the corresponding relation between two point clouds is more facilitated.
The dynamic graph attention mechanism includes a combination of multiple layers of self-attention modules (self-attention) and cross-attention modules (cross-attention), also known as dynamic graph attention modules. The dynamic graph attention mechanism is based on attention weight a ij Find the top k edges for each query node and construct the graph using only the top k edges and corresponding nodes and based on the changed attention weights a at each level of the dynamic graph attention mechanism ij A new graph is constructed.
The dynamic graph attention mechanism first calculates a query vector for feature nodes in the graph by linear projectionKey vector->Sum vector->Wherein->Representing a real set, b is the dimension of the feature for graph structure update and attention aggregation, as shown in the following formula:
q i ＝W 1 (l) f i Q +b 1
wherein the method comprises the steps of，{Q,S}∈{X,Y} 2 When q=s, self-attention is indicated; when Q is not equal to S, the cross attention is indicated; w and b are the projection parameters that can be learned, (l) f i Q the feature representing the key points in the dynamic graph meaning module l layer in the point cloud Q can be represented as query nodes, and all the nodes in the point cloud S can be represented as source nodes.
By calculating q i And each k j To obtain each k j Corresponding v j Weight coefficient alpha of (2) ij Then to v j And carrying out weighted summation to obtain a final Attention value, and calculating the message by a weighted average value, wherein the calculation formula is as follows:
wherein the weight coefficientRepresenting feature (l) f i Q For characteristics->Is concerned with the degree of care of (2); epsilon { epsilon } self ,ε cross }。
Once all layers are aggregated, the final characteristics of the node can be expressed as:
f i ＝Wf i +b
the U-Net network also comprises an encoder module for extracting features and a decoder module for fusing the features, wherein the encoder module obtains a plurality of scale features through a plurality of downsampling, and converts all full-connection layers in the multi-layer perceptron into a series of full-convolution layers with kernel sizes of 1 multiplied by 1, and the number of channels is (64,128,256,512); and each time the decoder module performs up-sampling, the information loss caused by down-sampling is reduced by fusing the features of the same scale corresponding to the channel number through the jump connection structure and the feature extraction part.
Step 3, inputting three-dimensional point clouds of the Qin warriors with different resolutions into a point cloud registration network, and training the constructed point cloud registration network under the common supervision of a circle loss function (cyclic loss function) and an overlap loss function (overlapping loss function);
circle loss is a variation of the triplet loss common in point cloud registration and the loss function formula is shown below:
wherein the overlapping point cloud pair X, Y is aligned, n x Representing the number of point clouds epsilon after random sampling from point cloud X x (x i ) Expressed in terms of point x i As the center, radius r x All points in the interior belonging to the point cloud Y, ε n (x i ) Expressed in terms of point x i As the center, radius r x All points outside that belong to the point cloud Y,representing the distance, delta, between two features in feature space x And delta n Respectively represent positive and negative sample intervals and weight->From super parameters gamma>Sample positive sample spacing delta x Determined, the same weightFrom super parameters gamma>Sample negative sample interval delta n Determined (I)>The calculation method is the same, so the final loss function of circle loss is +.>
The estimation of the overlap region is converted into a binary classification, supervised using an overlap loss, the loss function is as follows:
wherein the tag is authenticRepresenting point x i Whether or not overlap area->A label representing a network prediction.The calculation method is the same.
And 4, inputting the source point cloud and the target point cloud into a trained point cloud registration network, sequentially passing through an encoder module and a dynamic graph attention module, outputting the extracted features through a decoder module, and calculating a transformation matrix by combining with a RANSAC algorithm to finish registration.
Fig. 5 and 7 are respectively an initial pose graph of two head point clouds of the warrior and an initial pose graph of two foot point clouds of the warrior, which are obtained by a three-dimensional scanner; fig. 6 and 8 are respectively a two-head point cloud registration result diagram of the warrior and a two-foot point cloud registration result diagram of the warrior after registration by the method of the invention. From the figure, the method of the invention can still process the point cloud part in a robust way when aiming at the point cloud part overlapping and density change.
Claims (1)
1. A three-dimensional point cloud robust registration method of Qin warriors based on a dynamic graph attention mechanism is characterized by comprising the following steps:
step 1, acquiring three-dimensional point clouds of Qin figurines with different resolutions through a three-dimensional scanner;
step 2, in the U-Net network, replacing the convolution layer with B-NHN-Conv, replacing the deconvolution layer with B-NHN-ConvTr, and embedding a residual error module and a dynamic graph attention mechanism into the U-Net network to obtain a point cloud registration network;
the U-Net network also comprises an encoder module and a decoder module, wherein the encoder module obtains a plurality of scale characteristics through a plurality of downsampling, and converts all full-connection layers in the multi-layer perceptron into a series of full-convolution layers with kernel sizes of 1 multiplied by 1, and the number of channels is 64,128,256,512; the decoder module fuses the features with the same scale corresponding to the channel number through the jump connection structure and the feature extraction part after sampling every time, and reduces the information loss caused by downsampling;
the residual error module is formed by connecting a plurality of residual error blocks in series and is used for jumping and connecting input information to output information;
the dynamic graph attention mechanism comprises a combination module of a multi-layer self-attention module and a cross-attention module, and is based on the attention weight a ij Find the top k edges for each query node and construct the graph using only the top k edges and corresponding nodes and based on the changed attention weights α at each level of the dynamic graph attention mechanism ij Constructing a new graph;
step 3, three-dimensional point clouds of the Qin warriors with different resolutions are input into a point cloud registration network, and the established point cloud registration network is trained under the common supervision of a circle loss function and an overlap loss function;
and 4, extracting three-dimensional point cloud characteristics of the Qin figurines by using a trained point cloud registration network, estimating a change matrix between a source point cloud and a target point cloud by combining a RANSAC algorithm, and finishing registration of the three-dimensional point cloud of the Qin figurines.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111245398.1A CN114037743B (en) | 2021-10-26 | 2021-10-26 | Three-dimensional point cloud robust registration method for Qin warriors based on dynamic graph attention mechanism |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111245398.1A CN114037743B (en) | 2021-10-26 | 2021-10-26 | Three-dimensional point cloud robust registration method for Qin warriors based on dynamic graph attention mechanism |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114037743A CN114037743A (en) | 2022-02-11 |
| CN114037743B true CN114037743B (en) | 2024-01-26 |
Family
ID=80135357
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111245398.1A Active CN114037743B (en) | 2021-10-26 | 2021-10-26 | Three-dimensional point cloud robust registration method for Qin warriors based on dynamic graph attention mechanism |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114037743B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115631221B (en) * | 2022-11-30 | 2023-04-28 | 北京航空航天大学 | Low-overlapping-degree point cloud registration method based on consistency sampling |
| CN115631341A (en) * | 2022-12-21 | 2023-01-20 | 北京航空航天大学 | Point cloud registration method and system based on multi-scale feature voting |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108596961A (en) * | 2018-04-17 | 2018-09-28 | 浙江工业大学 | Point cloud registration method based on Three dimensional convolution neural network |
| CN111046781A (en) * | 2019-12-09 | 2020-04-21 | 华中科技大学 | Robust three-dimensional target detection method based on ternary attention mechanism |
| CN111882593A (en) * | 2020-07-23 | 2020-11-03 | 首都师范大学 | Point cloud registration model and method combining attention mechanism and three-dimensional graph convolution network |
| CN112837356A (en) * | 2021-02-06 | 2021-05-25 | 湖南大学 | WGAN-based unsupervised multi-view three-dimensional point cloud joint registration method |
| WO2021104056A1 (en) * | 2019-11-27 | 2021-06-03 | 中国科学院深圳先进技术研究院 | Automatic tumor segmentation system and method, and electronic device |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10824862B2 (en) * | 2017-11-14 | 2020-11-03 | Nuro, Inc. | Three-dimensional object detection for autonomous robotic systems using image proposals |
-
2021
- 2021-10-26 CN CN202111245398.1A patent/CN114037743B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108596961A (en) * | 2018-04-17 | 2018-09-28 | 浙江工业大学 | Point cloud registration method based on Three dimensional convolution neural network |
| WO2021104056A1 (en) * | 2019-11-27 | 2021-06-03 | 中国科学院深圳先进技术研究院 | Automatic tumor segmentation system and method, and electronic device |
| CN111046781A (en) * | 2019-12-09 | 2020-04-21 | 华中科技大学 | Robust three-dimensional target detection method based on ternary attention mechanism |
| CN111882593A (en) * | 2020-07-23 | 2020-11-03 | 首都师范大学 | Point cloud registration model and method combining attention mechanism and three-dimensional graph convolution network |
| CN112837356A (en) * | 2021-02-06 | 2021-05-25 | 湖南大学 | WGAN-based unsupervised multi-view three-dimensional point cloud joint registration method |
Non-Patent Citations (2)
| Title |
|---|
| 图像特征检测与匹配方法研究综述;唐灿;唐亮贵;刘波;;南京信息工程大学学报(自然科学版)(03);全文 * |
| 基于邻域几何特征约束的植株三维形态配准方法研究;马福峰;耿楠;张志毅;;计算机应用与软件(09);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114037743A (en) | 2022-02-11 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108510532B (en) | Optical and SAR image registration method based on deep convolution GAN | |
| CN114037743B (en) | Three-dimensional point cloud robust registration method for Qin warriors based on dynamic graph attention mechanism | |
| WO2019203231A1 (en) | Three-dimensional point cloud label learning device, three-dimensional point cloud label estimating device, three-dimensional point cloud label learning method, three-dimensional point cloud label estimating method, and program | |
| CN111046900B (en) | Semi-supervised generation confrontation network image classification method based on local manifold regularization | |
| CN105930382A (en) | Method for searching for 3D model with 2D pictures | |
| CN109447897B (en) | Real scene image synthesis method and system | |
| CN115222998B (en) | Image classification method | |
| CN114419464A (en) | Twin network change detection model based on deep learning | |
| CN113988147A (en) | Multi-label classification method and device for remote sensing image scene based on graph network, and multi-label retrieval method and device | |
| CN108805280B (en) | Image retrieval method and device | |
| CN110083734B (en) | Semi-supervised image retrieval method based on self-coding network and robust kernel hash | |
| CN112036511A (en) | Image retrieval method based on attention machine mapping convolutional neural network | |
| CN114663880A (en) | Three-dimensional target detection method based on multi-level cross-modal self-attention mechanism | |
| CN114066871A (en) | Method for training new coronary pneumonia focus region segmentation model | |
| CN112329818B (en) | Hyperspectral image non-supervision classification method based on graph convolution network embedded characterization | |
| CN111242003B (en) | Video salient object detection method based on multi-scale constrained self-attention mechanism | |
| CN112488221A (en) | Road pavement abnormity detection method based on dynamic refreshing positive sample image library | |
| CN111046958A (en) | Image classification and recognition method based on data-dependent kernel learning and dictionary learning | |
| CN114168782B (en) | Deep hash image retrieval method based on triplet network | |
| CN111797903B (en) | Multi-mode remote sensing image registration method based on data-driven particle swarm optimization | |
| Hu et al. | Loop closure detection for visual slam based on deep learning | |
| CN113591685A (en) | Geographic object spatial relationship identification method and system based on multi-scale pooling | |
| CN113313202A (en) | Single-domain generalization method based on progressive unknown domain expansion | |
| Li et al. | HTDFormer: Hyperspectral Target Detection Based on Transformer With Distributed Learning | |
| Zhan et al. | An Advanced JPEG Steganalysis Method with Balanced Depth and Width Based on Fractal Residual Network |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |