CN114170410A - Point cloud part level segmentation method based on PointNet graph convolution and KNN search - Google Patents
Point cloud part level segmentation method based on PointNet graph convolution and KNN search Download PDFInfo
- Publication number
- CN114170410A CN114170410A CN202111442718.2A CN202111442718A CN114170410A CN 114170410 A CN114170410 A CN 114170410A CN 202111442718 A CN202111442718 A CN 202111442718A CN 114170410 A CN114170410 A CN 114170410A
- Authority
- CN
- China
- Prior art keywords
- point cloud
- net
- point
- network
- information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000011218 segmentation Effects 0.000 title claims abstract description 31
- 238000000034 method Methods 0.000 title claims abstract description 24
- 238000012549 training Methods 0.000 claims abstract description 14
- 239000011159 matrix material Substances 0.000 claims description 48
- 238000012545 processing Methods 0.000 claims description 19
- PXFBZOLANLWPMH-UHFFFAOYSA-N 16-Epiaffinine Natural products C1C(C2=CC=CC=C2N2)=C2C(=O)CC2C(=CC)CN(C)C1C2CO PXFBZOLANLWPMH-UHFFFAOYSA-N 0.000 claims description 12
- 230000009466 transformation Effects 0.000 claims description 12
- 238000011176 pooling Methods 0.000 claims description 9
- 238000000605 extraction Methods 0.000 claims description 5
- 230000004913 activation Effects 0.000 claims description 3
- 238000005259 measurement Methods 0.000 claims description 3
- 238000010606 normalization Methods 0.000 claims description 3
- 230000006870 function Effects 0.000 description 6
- 230000008569 process Effects 0.000 description 4
- 238000011160 research Methods 0.000 description 3
- 238000004458 analytical method Methods 0.000 description 2
- 238000013528 artificial neural network Methods 0.000 description 2
- 238000013135 deep learning Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000007423 decrease Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 238000010191 image analysis Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
- 239000013598 vector Substances 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T19/00—Manipulating 3D models or images for computer graphics
- G06T19/20—Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/22—Matching criteria, e.g. proximity measures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2413—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on distances to training or reference patterns
- G06F18/24133—Distances to prototypes
- G06F18/24143—Distances to neighbourhood prototypes, e.g. restricted Coulomb energy networks [RCEN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/25—Fusion techniques
- G06F18/253—Fusion techniques of extracted features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/047—Probabilistic or stochastic networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/02—Affine transformations
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10028—Range image; Depth image; 3D point clouds
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20021—Dividing image into blocks, subimages or windows
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Computation (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Computational Linguistics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Biology (AREA)
- Molecular Biology (AREA)
- Biophysics (AREA)
- Bioinformatics & Cheminformatics (AREA)
- General Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Probability & Statistics with Applications (AREA)
- Architecture (AREA)
- Computer Graphics (AREA)
- Computer Hardware Design (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a point cloud part level segmentation method based on graph convolution of PointNet and KNN search, which comprises the following steps of firstly, ensuring the point cloud rotation invariance through a micro network T-Net; then, calculating Euclidean distance between every two point pairs, selecting k points with the nearest distance, and taking point piConstructing a k near neighborhood graph for the central point to extract local features, and dynamically updating the k neighborhood graph for each layer in subsequent model training to obtain local feature information; next, aligning local characteristic data of different point clouds by designing a new T-Net micro network; and finally, splicing the output multilayer local features and the global features, and obtaining a part level segmentation result of the three-dimensional point cloud object through a plurality of MLP operations.
Description
Technical Field
The invention belongs to the technical field of point cloud object segmentation, and particularly relates to a point cloud part level segmentation method based on PointNet graph convolution and KNN search.
Background
Analysis and understanding of three-dimensional shapes has been an important research topic in computer graphics. In recent years, the popularization of depth sensors and three-dimensional laser scanners has promoted the rapid development of three-dimensional point cloud processing methods. The part-level segmentation of the three-dimensional point cloud object is used as the basis of 3D scene understanding and analysis, becomes the research focus in the fields of navigation positioning, medical image analysis, mode recognition and the like, and has important research value and wide application prospect.
The point cloud part level segmentation is to divide the point cloud into a plurality of regions with detailed semantic categories. The basic point cloud contains 6-dimensional features, XYZ coordinates and RGB color information, respectively. With the appearance of a large-scale data set, the reduction of computer hardware cost and the improvement of GPU parallel computing capability, deep learning gradually occupies an absolute dominant position in the point cloud segmentation field, deep learning taking a PointNet network as a pioneer is researched and used by more scholars, the network model is very simple, the original point cloud data can be directly operated according to the characteristics of the point cloud data, meanwhile, MaxPoint is used as a symmetric function to process the orderless of the point cloud, and two T-net networks are adopted to carry out rotation invariance processing on the model.
The PointNet network mainly processes points independently on a local scale, arrangement invariance is kept, but interaction relation of adjacent points is not considered, and therefore local characteristics of geometric relation among nodes are lost. Based on the method, the PointNet network is optimized, and the neighborhood characteristics of the point cloud are more efficiently aggregated by the part level segmentation method based on the graph convolution and KNN search of PointNet.
Disclosure of Invention
The invention aims to provide a point cloud part level segmentation method based on graph convolution and KNN search of PointNet, which solves the problem of low semantic segmentation precision caused by extracting the characteristics of single points only by a PointNet network.
The technical scheme adopted by the invention is that the point cloud part level segmentation method based on PointNet graph convolution and KNN search is implemented according to the following steps:
step 1, point cloud space alignment: inputting point cloud data, predicting an affine transformation matrix by using a T-Net micro network, and performing coordinate alignment on the input point cloud data by using the matrix;
step 2, local feature extraction: in order to capture local features, each point in the step 1 is taken as a vertex, k points with the nearest distance are selected to construct a k nearest neighborhood graph, edge convolution is applied to edges connecting adjacent point pairs to obtain edge feature information between every two point pairs, and then local feature matrix information is extracted through MLP operation of a multilayer perceptron network;
step 4, point cloud feature transformation: designing a new T-Net mini network, inputting the updated local feature matrix information into the new T-Net network for coordinate alignment, and ensuring the rotation invariance of feature points;
and 5, fusing local features and global features: and carrying out MaxPolling maximum pooling operation on the aligned and updated local feature matrix information, and processing the global feature information obtained by the operation through a plurality of MLPs to obtain the category fraction of each object, thereby realizing the part-level segmentation of the three-dimensional object.
The present invention is also characterized in that,
the step 1 specifically comprises the following steps:
step 1.1, inputting point cloud data P ═ { P) of a three-dimensional object1,p2,…,pi,…,pNIn which p isiIndicating the position information of the ith point, where N is point _ num, which is the number of points of the object, N is 2048, the batch size is 4, and the one-time training period epoch is 200;
step 1.2, in order to make the model have arrangement invariance to the input, the information of each point is aggregated by using a symmetric function to obtain an affine matrix A:
f({p1,…,pN})≈g(h(p1),…,h(pN)), (1)
in the formula (1), h represents a multilayer perceptron network, g represents a maximum pooling function, and f represents the characteristic information of a captured point cloud set;
and step 1.3, after the input point cloud data is multiplied by the affine matrix A, the input point cloud coordinate alignment is realized through a plurality of MLP processing.
The step 2 specifically comprises the following steps:
step 2.1, selecting k nearest neighbor points k as 20 by using Euclidean distance as a measurement standard;
step 2.2, each point p of the point cloud data obtained after the alignment in the step 1 is processediAnd its k nearest neighbors qjJ-1, 2, …, k constructs a k-neighborhood domain map, whereRFRepresenting an F-dimensional input point cloud matrix.
Step 2.3, the k-neighborhood map structure is G ═ (V, E), where V ═ { p ═ piI | ═ 1, …, N } represents a set of vertices, E ═ Ei=(ei1,ei2,…,eij,…eik) I | ═ 1,2, …, N } represents the set of edges between vertices, eijRepresents a point piDirected edges with k neighbor points;
and 2.4, performing MLP convolution processing on the k near neighborhood map obtained after updating in the step 2.3 to obtain local characteristic matrix information of the three-dimensional point cloud object.
The step 3 is as follows:
step 3.1, the MLP comprises two convolution layers, a batch processing normalization layer and an activation layer, wherein the sizes of convolution kernels of the two convolution layers are 64 and 64 from left to right, and output point cloud data of each layer of the MLP network isl represents the l-th layer of the MLP network;
step 3.2, obtaining a different output k near-neighborhood graph G for each MLP layer according to the output point cloud data of each MLP network layerl=(Vl,El),GlK-neighborhood graph, V, representing the output of the l-th layerlRepresents GlCorresponding set of vertices, ElRepresents GlA set of edges between corresponding vertices;
and 3.3, acquiring local feature matrix information of each layer according to the k near neighborhood graph output by each layer.
The new designed T-Net network is characterized in that a regularization item shown in a formula (3) is added in the softmax training loss of a conventional T-Net network framework, and a characteristic transformation matrix is limited to be close to an orthogonal matrix;
in the formula (3), A is an affine matrix predicted by the T-Net network, so that input information cannot be lost by orthogonal transformation, I is an identity matrix, and F represents a point cloud dimension.
The step 5 is as follows:
step 5.1, using the aligned and updated local feature matrix information as input, and performing convolution operation processing through 3 spatial _ transform modules to respectively obtain three local feature information nets1,net2,net3;
Step 5.2, respectively converting three local features corresponding to the 3 spatial _ transform modules: net1,net2,net3Splicing, and outputting out;
step 5.3, carrying out MaxPolling maximum pooling operation on the output out data to obtain the global characteristics of the current layer;
step 5.4, splicing the global features of the current layer with the tag information of the input point cloud in the data set to obtain global feature global _ feature of the whole network;
step 5.5, global _ feature and net of global feature1,net2,net3Splicing is carried out;
and 5.6, carrying out MLP multi-layer perceptron processing on the data spliced in the step 5.5 to obtain the category score conditions of all the objects, and realizing the part-level segmentation of the three-dimensional objects.
The invention has the beneficial effects that:
the invention relates to a point cloud part level segmentation method based on PointNet graph convolution and KNN search.A point cloud rotation invariance is ensured through a micro network T-Net (input _ transform); then, calculating Euclidean distance between every two point pairs, selecting k points with the nearest distance, and taking point piConstructing a k near neighborhood graph for the central point to extract local features, and dynamically updating the k neighborhood graph for each layer in subsequent model training to obtain local feature information; next, aligning the local feature data of different point clouds through a T-Net (feature _ transform) micro network; and finally, splicing the output multilayer local features and the global features, and then obtaining a part grade segmentation result of the three-dimensional point cloud object through a plurality of MLP (multilayer perceptron) operations.
Drawings
FIG. 1 is a network architecture diagram of a point cloud part level segmentation method based on PointNet's graph convolution and KNN search;
FIG. 2 is a graph convolution based on PointNet and KNN search point cloud part level segmentation method improved graph convolution and KNN search edge feature (edge _ feature) extraction network graph;
FIG. 3 is a result of part-level segmentation of a three-dimensional object in a ShapeNetCore dataset;
FIG. 4(a) is a graph of loss during ShapeNetCore dataset training
Figure 4(b) is a graph of accuracy during the training of the ShapeNetCore dataset.
Detailed Description
The present invention will be described in detail with reference to the following embodiments.
Examples
The embodiment provides a point cloud part level segmentation method based on PointNet graph convolution and KNN search, which is specifically implemented according to the following steps as shown in FIG. 1:
step 1, point cloud space alignment: inputting point cloud data, predicting an affine transformation matrix by using a T-Net micro network, and performing coordinate alignment on the input point cloud data by using the matrix;
step 1.1, inputting point cloud data P ═ { P) of a three-dimensional object1,p2,…,pi,…,pNIn which p isiIndicating the position information of the ith point, where N is point _ num, which is the number of points of the object, N is 2048, the batch size is 4, and the one-time training period epoch is 200;
step 1.2, in order to make the model have arrangement invariance to the input, the information of each point is aggregated by using a symmetric function to obtain an affine matrix A:
f({p1,…,pN})≈g(h(p1),…,h(pN)), (1)
in the formula (1), h represents a multilayer perceptron network, g represents a maximum pooling function, and f represents the characteristic information of a captured point cloud set;
and step 1.3, after the input point cloud data is multiplied by the affine matrix A, the input point cloud coordinate alignment is realized through a plurality of MLP processing.
Step 2, local feature extraction: in order to capture local features, each point in the step 1 is taken as a vertex, k points with the nearest distance are selected to construct a k nearest neighborhood graph, edge convolution is applied to edges connecting adjacent point pairs to obtain edge feature information between every two point pairs, and then local feature matrix information is extracted through MLP operation of a multilayer perceptron network;
step 2.1, selecting k nearest neighbor points k as 20 by using Euclidean distance as a measurement standard;
step 2.2, each point p of the point cloud data obtained after the alignment in the step 1 is processediAnd its k nearest neighbors qjJ-1, 2, …, k constructs a k-neighborhood domain map, whereRFRepresenting an F-dimensional input point cloud matrix.
Step 2.3, the k-neighborhood map structure is G ═ (V, E), where V ═ { p ═ piI | ═ 1, …, N } represents a set of vertices, E ═ Ei=(ei1,ei2,…,eij,…eik) I | ═ 1,2, …, N } represents the set of edges between vertices, eijRepresents a point piDirected edges with k neighbor points;
and 2.4, performing MLP convolution processing on the k near neighborhood map obtained after updating in the step 2.3 to obtain local characteristic matrix information of the three-dimensional point cloud object.
step 3.1, the MLP comprises two convolution layers, a batch processing normalization layer and an activation layer, wherein the sizes of convolution kernels of the two convolution layers are 64 and 64 from left to right, and output point cloud data of each layer of the MLP network isl represents the l-th layer of the MLP network;
step 3.2, obtaining a different output k near-neighborhood graph G for each MLP layer according to the output point cloud data of each MLP network layerl=(Vl,El),GlK-neighborhood graph, V, representing the output of the l-th layerlRepresents GlA set of corresponding vertices is created and is,Elrepresents GlA set of edges between corresponding vertices;
and 3.3, acquiring local feature matrix information of each layer according to the k near neighborhood graph output by each layer.
Step 4, point cloud feature transformation: designing a new T-Net mini network, inputting the updated local feature matrix information into the new T-Net network for coordinate alignment, and ensuring the rotation invariance of feature points;
the new designed T-Net network is characterized in that a regularization item shown in a formula (3) is added in the softmax training loss of a conventional T-Net network framework, and a characteristic transformation matrix is limited to be close to an orthogonal matrix;
in the formula (3), A is an affine matrix predicted by the T-Net network, so that input information cannot be lost by orthogonal transformation, I is an identity matrix, and F represents a point cloud dimension.
And 5, fusing local features and global features: and carrying out MaxPolling maximum pooling operation on the aligned and updated local feature matrix information, and processing the global feature information obtained by the operation through a plurality of MLPs to obtain the category fraction of each object, thereby realizing the part-level segmentation of the three-dimensional object.
Step 5.1, using the aligned and updated local feature matrix information as input, and performing convolution operation processing through 3 spatial _ transform modules to respectively obtain three local feature information nets1,net2,net3The spatial _ transform module is shown in fig. 2, wherein the spatial _ transform module is composed of an edge convolution module and a plurality of MLP multi-layer perceptron modules;
step 5.2, respectively converting three local features corresponding to the 3 spatial _ transform modules: net1,net2,net3Splicing, and outputting out;
step 5.3, carrying out MaxPolling maximum pooling operation on the output out data to obtain the global characteristics of the current layer;
step 5.4, splicing the global features of the current layer with the tag information of the input point cloud in the data set to obtain global feature global _ feature of the whole network;
step 5.5, global _ feature and net of global feature1,net2,net3Splicing is carried out;
and 5.6, carrying out MLP multi-layer perceptron processing on the data spliced in the step 5.5 to obtain the category score conditions of all the objects, and realizing the part-level segmentation of the three-dimensional objects as shown in FIG. 3.
In the training process, the cross entropy loss function is used for learning parameters, so that the model reaches a convergence state, and the error of the predicted value of the model is reduced. As shown in fig. 4, fig. 4(a) shows a loss curve during the training process, fig. 4(b) shows an accuracy curve, and as the accuracy of the training set increases with the increase of epoch, loss steadily decreases, which indicates that the learning model of the present invention has good adaptability to the training set.
According to the method, due to the fact that the point cloud is lack of topological information essentially, the graph convolution neural network can run on the graph structure directly and can capture dependency relations in the graph by means of information transfer among nodes in the graph, and aiming at the limitation of local feature extraction in the PointNet framework, the method utilizes the graph convolution neural network to extract features of a central point and edge vectors of the central point and k adjacent domain points of the central point to obtain local features of the point cloud, and the problem that the PointNet network cannot extract the local structure is effectively solved.
Claims (6)
1. The point cloud part level segmentation method based on PointNet graph convolution and KNN search is characterized by comprising the following steps:
step 1, point cloud space alignment: inputting point cloud data, predicting an affine transformation matrix by using a T-Net micro network, and performing coordinate alignment on the input point cloud data by using the matrix;
step 2, local feature extraction: in order to capture local features, each point in the step 1 is taken as a vertex, k points with the nearest distance are selected to construct a k nearest neighborhood graph, edge convolution is applied to edges connecting adjacent point pairs to obtain edge feature information between every two point pairs, and then local feature matrix information is extracted through MLP operation of a multilayer perceptron network;
step 3, dynamically updating the local neighborhood map: calculating and updating a k neighbor graph of each layer of the MLP according to the embedding sequence, and extracting updated local feature matrix information;
step 4, point cloud feature transformation: designing a new T-Net mini network, inputting the updated local feature matrix information into the new T-Net network for coordinate alignment, and ensuring the rotation invariance of feature points;
and 5, fusing local features and global features: and carrying out MaxPolling maximum pooling operation on the aligned and updated local feature matrix information, and processing the global feature information obtained by the operation through a plurality of MLPs to obtain the category fraction of each object, thereby realizing the part-level segmentation of the three-dimensional object.
2. The point cloud part level segmentation method based on PointNet graph convolution and KNN search of claim 1, wherein the step 1 specifically comprises:
step 1.1, inputting point cloud data P ═ { P) of a three-dimensional object1,p2,…,pi,…,pNIn which p isiIndicating the position information of the ith point, where N is point _ num, which is the number of points of the object, N is 2048, the batch size is 4, and the one-time training period epoch is 200;
step 1.2, in order to make the model have arrangement invariance to the input, the information of each point is aggregated by using a symmetric function to obtain an affine matrix A:
f({p1,…,pN})≈g(h(p1),…,h(pN)), (1)
in the formula (1), h represents a multilayer perceptron network, g represents a maximum pooling function, and f represents the characteristic information of a captured point cloud set;
and step 1.3, after the input point cloud data is multiplied by the affine matrix A, the input point cloud coordinate alignment is realized through a plurality of MLP processing.
3. The point cloud part level segmentation method based on PointNet's atlas convolution and KNN search of claim 2, wherein the step 2 specifically is:
step 2.1, selecting k nearest neighbor points k as 20 by using Euclidean distance as a measurement standard;
step 2.2, each point p of the point cloud data obtained after the alignment in the step 1 is processediAnd its k nearest neighbors qjJ-1, 2, …, k constructs a k-neighborhood domain map, whereRFRepresenting an F-dimensional input point cloud matrix;
step 2.3, the k-neighborhood map structure is G ═ (V, E), where V ═ { p ═ piI | ═ 1, …, N } represents a set of vertices, E ═ Ei=(ei1,ei2,…,eij,…eik) I | ═ 1,2, …, N } represents the set of edges between vertices, eijRepresents a point piDirected edges with k neighbor points;
and 2.4, performing MLP convolution processing on the k near neighborhood map obtained after updating in the step 2.3 to obtain local characteristic matrix information of the three-dimensional point cloud object.
4. The point cloud part level segmentation method based on PointNet graph convolution and KNN search of claim 1, wherein the step 3 specifically comprises:
step 3.1, the MLP comprises two convolution layers, a batch processing normalization layer and an activation layer, wherein the sizes of convolution kernels of the two convolution layers are 64 and 64 from left to right, and output point cloud data of each layer of the MLP network isl represents the l-th layer of the MLP network;
step 3.2, obtaining a different output k near-neighborhood graph G for each MLP layer according to the output point cloud data of each MLP network layerl=(Vl,El),GlDenotes the l-th layerOutput k-neighborhood graph, VlRepresents GlCorresponding set of vertices, ElRepresents GlA set of edges between corresponding vertices;
and 3.3, acquiring local feature matrix information of each layer according to the k near neighborhood graph output by each layer.
5. The point cloud part level segmentation method based on PointNet graph convolution and KNN search as claimed in claim 1, wherein the new T-Net network designed in step 4 is to add a regularization term as shown in formula (3) to the softmax training loss of the conventional T-Net network framework, and limit the feature transformation matrix to be close to the orthogonal matrix;
in the formula (3), A is an affine matrix predicted by the T-Net network, so that input information cannot be lost by orthogonal transformation, I is an identity matrix, and F represents a point cloud dimension.
6. The point cloud part level segmentation method based on PointNet graph convolution and KNN search of claim 1, wherein the step 5 is as follows:
step 5.1, using the aligned and updated local feature matrix information as input, and performing convolution operation processing through 3 spatial _ transform modules to respectively obtain three local feature information nets1,net2,net3;
Step 5.2, respectively converting three local features corresponding to the 3 spatial _ transform modules: net1,net2,net3Splicing, and outputting out;
step 5.3, carrying out MaxPolling maximum pooling operation on the output out data to obtain the global characteristics of the current layer;
step 5.4, splicing the global features of the current layer with the tag information of the input point cloud in the data set to obtain global feature global _ feature of the whole network;
step 5.5, global _ feature and net of global feature1,net2,net3Splicing is carried out;
and 5.6, carrying out MLP multi-layer perceptron processing on the data spliced in the step 5.5 to obtain the category score conditions of all the objects, and realizing the part-level segmentation of the three-dimensional objects.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202111442718.2A CN114170410A (en) | 2021-11-30 | 2021-11-30 | Point cloud part level segmentation method based on PointNet graph convolution and KNN search |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202111442718.2A CN114170410A (en) | 2021-11-30 | 2021-11-30 | Point cloud part level segmentation method based on PointNet graph convolution and KNN search |
Publications (1)
Publication Number | Publication Date |
---|---|
CN114170410A true CN114170410A (en) | 2022-03-11 |
Family
ID=80481933
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202111442718.2A Pending CN114170410A (en) | 2021-11-30 | 2021-11-30 | Point cloud part level segmentation method based on PointNet graph convolution and KNN search |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN114170410A (en) |
Cited By (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN114693932A (en) * | 2022-04-06 | 2022-07-01 | 南京航空航天大学 | Large aircraft large component point cloud semantic segmentation method |
CN114743123A (en) * | 2022-04-29 | 2022-07-12 | 电子科技大学 | Scene understanding method based on implicit function three-dimensional representation and graph neural network |
CN114973422A (en) * | 2022-07-19 | 2022-08-30 | 南京应用数学中心 | Gait recognition method based on three-dimensional human body modeling point cloud feature coding |
CN116128835A (en) * | 2023-02-02 | 2023-05-16 | 南方电网数字电网研究院有限公司 | Point cloud analysis-based power transmission wire sag measurement method, device and equipment |
CN116164648A (en) * | 2023-04-20 | 2023-05-26 | 武汉嘉晨电子技术有限公司 | Automatic detection method for BDU automobile harness connector terminal |
Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2020173297A1 (en) * | 2019-02-25 | 2020-09-03 | 腾讯科技(深圳)有限公司 | Point cloud segmentation method, computer-readable storage medium and computer device |
CN112488210A (en) * | 2020-12-02 | 2021-03-12 | 北京工业大学 | Three-dimensional point cloud automatic classification method based on graph convolution neural network |
CN113628217A (en) * | 2021-08-12 | 2021-11-09 | 江南大学 | Three-dimensional point cloud segmentation method based on image convolution and integrating direction and distance |
CN115049833A (en) * | 2022-06-30 | 2022-09-13 | 西安理工大学 | Point cloud component segmentation method based on local feature enhancement and similarity measurement |
-
2021
- 2021-11-30 CN CN202111442718.2A patent/CN114170410A/en active Pending
Patent Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2020173297A1 (en) * | 2019-02-25 | 2020-09-03 | 腾讯科技(深圳)有限公司 | Point cloud segmentation method, computer-readable storage medium and computer device |
CN112488210A (en) * | 2020-12-02 | 2021-03-12 | 北京工业大学 | Three-dimensional point cloud automatic classification method based on graph convolution neural network |
CN113628217A (en) * | 2021-08-12 | 2021-11-09 | 江南大学 | Three-dimensional point cloud segmentation method based on image convolution and integrating direction and distance |
CN115049833A (en) * | 2022-06-30 | 2022-09-13 | 西安理工大学 | Point cloud component segmentation method based on local feature enhancement and similarity measurement |
Non-Patent Citations (5)
Title |
---|
YAODONG CUI 等: "Deep Learning for Image and Point Cloud Fusion in Autonomous Driving: A Review", IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS, vol. 23, no. 2, 17 March 2021 (2021-03-17), pages 722 - 739 * |
宁小娟 等: "结合语义分割与模型匹配的室内场景重建方法", 中国图象图形学报, vol. 28, no. 10, 16 October 2023 (2023-10-16), pages 3149 - 3162 * |
张新良;付鹏飞;赵运基;谢恒;王琬如;: "融合图卷积和差异性池化函数的点云数据分类分割模型", 中国图象图形学报, no. 06, 16 June 2020 (2020-06-16), pages 137 - 144 * |
点云PCL博主: "结合代码理解Pointnet网络结构", Retrieved from the Internet <URL:https://cloud.tencent.com/developer/article/1605590> * |
韩姗姗;黄远程;白穆;: "基于DGCNN语义分割的倾斜摄影测量三维点云建筑物分类", 测绘标准化, no. 03, 25 September 2020 (2020-09-25), pages 25 - 30 * |
Cited By (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN114693932A (en) * | 2022-04-06 | 2022-07-01 | 南京航空航天大学 | Large aircraft large component point cloud semantic segmentation method |
CN114743123A (en) * | 2022-04-29 | 2022-07-12 | 电子科技大学 | Scene understanding method based on implicit function three-dimensional representation and graph neural network |
CN114973422A (en) * | 2022-07-19 | 2022-08-30 | 南京应用数学中心 | Gait recognition method based on three-dimensional human body modeling point cloud feature coding |
CN116128835A (en) * | 2023-02-02 | 2023-05-16 | 南方电网数字电网研究院有限公司 | Point cloud analysis-based power transmission wire sag measurement method, device and equipment |
CN116128835B (en) * | 2023-02-02 | 2023-09-15 | 南方电网数字电网研究院有限公司 | Point cloud analysis-based power transmission wire sag measurement method, device and equipment |
CN116164648A (en) * | 2023-04-20 | 2023-05-26 | 武汉嘉晨电子技术有限公司 | Automatic detection method for BDU automobile harness connector terminal |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN111583263B (en) | Point cloud segmentation method based on joint dynamic graph convolution | |
CN114170410A (en) | Point cloud part level segmentation method based on PointNet graph convolution and KNN search | |
CN113205466B (en) | Incomplete point cloud completion method based on hidden space topological structure constraint | |
US11328172B2 (en) | Method for fine-grained sketch-based scene image retrieval | |
CN112633350B (en) | Multi-scale point cloud classification implementation method based on graph convolution | |
CN111489358A (en) | Three-dimensional point cloud semantic segmentation method based on deep learning | |
CN112288011B (en) | Image matching method based on self-attention deep neural network | |
CN111625667A (en) | Three-dimensional model cross-domain retrieval method and system based on complex background image | |
CN110674741A (en) | Machine vision gesture recognition method based on dual-channel feature fusion | |
CN110263855B (en) | Method for classifying images by utilizing common-basis capsule projection | |
CN112329801B (en) | Convolutional neural network non-local information construction method | |
CN110490915B (en) | Point cloud registration method based on convolution-limited Boltzmann machine | |
CN115908517B (en) | Low-overlapping point cloud registration method based on optimization of corresponding point matching matrix | |
CN113011568A (en) | Model training method, data processing method and equipment | |
CN115049833A (en) | Point cloud component segmentation method based on local feature enhancement and similarity measurement | |
CN113989340A (en) | Point cloud registration method based on distribution | |
CN116740527A (en) | Remote sensing image change detection method combining U-shaped network and self-attention mechanism | |
Lee et al. | Connectivity-based convolutional neural network for classifying point clouds | |
CN111368733A (en) | Three-dimensional hand posture estimation method based on label distribution learning, storage medium and terminal | |
CN117252928B (en) | Visual image positioning system for modular intelligent assembly of electronic products | |
Ni et al. | Category-level assignment for cross-domain semantic segmentation in remote sensing images | |
CN113538474A (en) | 3D point cloud segmentation target detection system based on edge feature fusion | |
CN111291651B (en) | Multi-task neural network framework for remote sensing scene classification and classification method | |
CN111898756A (en) | Multi-target information associated neural network loss function calculation method and device | |
CN116977265A (en) | Training method and device for defect detection model, computer equipment and storage medium |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination |