CN115731137A - Outdoor large scene point cloud segmentation method based on A-EdgeConv - Google Patents
Outdoor large scene point cloud segmentation method based on A-EdgeConv Download PDFInfo
- Publication number
- CN115731137A CN115731137A CN202211513938.4A CN202211513938A CN115731137A CN 115731137 A CN115731137 A CN 115731137A CN 202211513938 A CN202211513938 A CN 202211513938A CN 115731137 A CN115731137 A CN 115731137A
- Authority
- CN
- China
- Prior art keywords
- point
- super
- points
- characteristic
- point cloud
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 230000011218 segmentation Effects 0.000 title claims abstract description 42
- 238000000034 method Methods 0.000 title claims abstract description 32
- 230000007246 mechanism Effects 0.000 claims abstract description 9
- 239000013598 vector Substances 0.000 claims description 33
- 230000006870 function Effects 0.000 claims description 20
- 238000004364 calculation method Methods 0.000 claims description 19
- 238000005520 cutting process Methods 0.000 claims description 12
- 238000009499 grossing Methods 0.000 claims description 11
- 238000000605 extraction Methods 0.000 claims description 10
- 230000008859 change Effects 0.000 claims description 9
- 238000010586 diagram Methods 0.000 claims description 8
- 238000012549 training Methods 0.000 claims description 6
- 101100295776 Drosophila melanogaster onecut gene Proteins 0.000 claims description 5
- 230000015572 biosynthetic process Effects 0.000 claims description 5
- 238000000354 decomposition reaction Methods 0.000 claims description 5
- 239000011159 matrix material Substances 0.000 claims description 5
- 238000003062 neural network model Methods 0.000 claims description 5
- 238000007781 pre-processing Methods 0.000 claims description 5
- 230000008569 process Effects 0.000 claims description 5
- 238000003786 synthesis reaction Methods 0.000 claims description 5
- 230000002776 aggregation Effects 0.000 abstract description 2
- 238000004220 aggregation Methods 0.000 abstract description 2
- 238000013528 artificial neural network Methods 0.000 abstract description 2
- 125000004122 cyclic group Chemical group 0.000 abstract description 2
- 238000012545 processing Methods 0.000 abstract description 2
- 230000009286 beneficial effect Effects 0.000 description 2
- 239000000463 material Substances 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 238000004891 communication Methods 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000009826 distribution Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 238000011176 pooling Methods 0.000 description 1
- 238000000611 regression analysis Methods 0.000 description 1
Images
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Image Analysis (AREA)
Abstract
The invention belongs to the technical field of point cloud image processing, and particularly relates to an outdoor large-scene point cloud segmentation method based on A-EdgeConv, which combines local geometric information and a graph segmentation algorithm to achieve the acquisition of a super-point, adopts a local adjacency graph and an attention mechanism to more accurately extract a super-point characteristic, and achieves the characteristic aggregation of the super-point and the super-edge through a GRU (generalized regression) cyclic neural network.
Description
Technical Field
The invention belongs to the technical field of point cloud image processing, and particularly relates to an outdoor large scene point cloud segmentation method based on A-EdgeConv.
Background
In recent years, laser scanning technology is rapidly developed, becomes an important way for rapidly acquiring three-dimensional information, and has wide application in the fields of automatic driving, smart city construction and the like. The efficient and accurate semantic segmentation of the point cloud is a precondition of the application, and the accurate extraction of the ground object target in an outdoor large scene with huge data volume and uneven scene distribution is always a challenging problem.
The information of the valid features is a direct factor in obtaining the correct semantic label. In a large outdoor scene, the requirement on detail characteristic information is more urgent due to the complex relation between the ground class and the space, and how to quickly acquire the effective information of the object becomes a research hotspot. Aiming at the extraction of object feature information, most researches adopt a PointNet model to extract point cloud features, low-dimensional information is projected to a high-dimensional space by using a shared multilayer perceptron, and the point cloud features are extracted by combining a maximum pooling function. The information extraction mode does not consider the point cloud space relation in the neighborhood, weakens the feature expression of different ground objects in a complex environment, and loses a large amount of effective information.
Disclosure of Invention
In view of the above problems, the present invention provides an a-EdgeConv-based outdoor large scene point cloud segmentation method, which overcomes or at least partially solves the above problems, and can efficiently and accurately segment the outdoor large scene point cloud.
Specifically, the invention provides an outdoor large scene point cloud segmentation method based on A-EdgeConv, which comprises the following steps:
step 2, carrying out geometric homogeneity division on the point cloud in a graph cutting mode to obtain a super point;
step 3, constructing a local adjacency graph based on the over points obtained in the step 2, describing local characteristics, and highlighting key information through an attention mechanism algorithm;
step 4, constructing a super-point diagram by utilizing the adjacency relation of the super-points to acquire super-edge characteristics;
and 5, combining the GRU gating cycle unit and the GNN graph neural network model to aggregate the characteristics of the super point and the super edge, and realizing accurate semantic segmentation of the point cloud.
Optionally, the step 1 includes:
step 1.1, preprocessing outdoor large scene point cloud and removing noise points;
step 1.2, selecting an optimal neighborhood by minimizing the characteristic entropy;
step 1.3, combining each optimal neighborhood of the point cloud, obtaining a characteristic value by calculating a covariance matrix of adjacent points, and calculating linear L according to the covariance characteristic value λ Flatness P λ Degree of scattering S λ All-round difference O λ Change of curvature C λ Five characteristic indexes.
Optionally, in step 1.2, the method for calculating the minimum characteristic entropy is as shown in formula (1):
E λ ＝-λ 1 ln(λ 1 )-λ 2 ln(λ 2 )-λ 3 ln(λ 3 ) (1)
wherein λ is 1 、λ 2 、λ 3 Respectively representing the covariance eigenvalues of the neighborhood of the target point.
Optionally, in step 1.3, L is linear λ Flatness P λ Degree of scattering S λ All-round difference O λ Change of curvature C λ The calculation methods of the five characteristic indexes are shown in formulas (2) to (6):
optionally, the step 2 includes:
step 2.1, constructing a minimum energy function according to the point cloud local information described in the step 1;
and 2.2, solving the problem of minimizing energy by adopting a one-cut algorithm, adaptively adjusting the size of the divided segments according to the local geometric complexity, and realizing the acquisition of the over point through graph cutting.
Optionally, in step 2.1, the energy minimization function is as shown in equation (7):
E(G)＝∑ p∈P D p (G p )+λ∑ (p,q)∈E V p,q (G p ,G q ) (7)
wherein D is p (G p ) Is a data cost term, which is the sum of all points and their penalties for assigning tag numbers, D p (G p ) Indicates a label G p Penalty in assigning to node p; sigma (p,q)∈E V p,q (G p ,G q ) For the smoothing cost term, is the sum of penalties for all neighboring labels to be different, V p,q (G p ,G q ) The penalty of different label numbers between adjacent nodes p and q is represented; lambda is a weight coefficient, and the numerical value reflects the proportion of a data cost item and a smooth cost item in graph cutting; suppose that the feature of point p is denoted f p Which is associated with a label L p Data cost D of p (G p ) The calculation is shown in equation (8):
D p (G p )＝(g p -f p ) 2 (8)
wherein, g p Indicates a tag number G p Point cloud characteristics corresponding to the segmented object;
the smoothing cost is used to penalize the inconsistency of label numbers between adjacent nodes p and q, and generally speaking, if the distance between the nodes p and q is smaller, the smoothing cost V between the nodes p and q is smaller p,q (G p ,G q ) The larger the probability that the node p and q label numbers are the same; smoothing cost V between adjacent nodes p,q (G p ,G q ) The calculation formula is shown in formulas (9) to (10):
V p,q (G p ,G q )＝exp(-ds) (9)
wherein f is p 、f q Respectively representing the geometric features of the adjacent point clouds p and q.
Optionally, the step 3 includes:
step 3.1, constructing a local adjacency graph in the super-point by taking the super-point obtained in the step 2 as a basis;
step 3.2, randomly selecting a clustering initial point x from the three-dimensional point cloud data; finding a neighborhood feature point set M of an initial point x, assuming that there are n neighboring points, defined as: x = X 1 ,X 2 ,…X n Performing convolution operation on the obtained data;
step 3.3, for n neighborhood points X in the neighborhood range of the point cloud data X in the super point ij1 ,X ij2 ,…,X ijn The neighborhood point feature vector is expressed as:
a unique attention score is learned for the feature vector of each neighborhood point by a sharing function g.
Step 3.4, when the neighborhood relationship also exists between the points q and p in the point set M in space, grouping q and p into one class; weight coefficient
Expressing the importance of each feature vector, and performing weighted summation on each feature vector to obtain the final rich feature vector representing the over-point information
Step 3.5, performing the operation on all the over points in the point set to obtain all over point characteristics;
and 3.6, overlapping a plurality of layers of A-EdgeConv modules to realize layer-by-layer extraction and summarization of local information, and realizing accurate description of the over point by overlapping different characteristic spatial information.
Optionally, in step 3.3, the definition of the sharing function g is shown in formula (11):
wherein, W is the weight learnable by the multi-layer perceptron,
as feature vectors
And obtaining the weight coefficient through the sharing function.
Optionally, in the step 3.4, the feature vector
The calculation is shown in equation (12):
optionally, the step 4 includes:
step 4.1, constructing a super-point map based on the super-points obtained in step 2, and assuming that adjacent super-points S 'and T' exist, if one or more edges formed by three-dimensional points in the two super-points S 'and T' belong to E, the offset set between the super-points is as shown in formula (13):
δ(S′,T′)＝{(p i -p j )∣(i,j)∈E∩(S′×T′)} (13)
in the formula, p i 、p j Respectively representing point clouds on two sides of an edge formed by adjacent super points;
step 4.2, the super-edge characteristics are obtained according to the relation of the super-point graph, and the characteristic value decomposition is carried out on the three-dimensional point set in the adjacent super-points to obtain the characteristic value lambda 1 ≥λ 2 ≥λ 3 ≥0；
And 4.3, obtaining the one-dimensional and/or three-dimensional super-edge characteristics through different combinations of the characteristic values.
Optionally, the step 5 includes:
step 5.1, embedding the over-point characteristic of the step 3 and the over-edge characteristic of the step 4 into a circulation unit as initial states;
step 5.2, inputting the over-point characteristic and the over-edge characteristic as initial values h of the hidden state 0 And input x 0 The method determines whether the candidate state at the current time needs to depend on the network state at the last time and how much the candidate state needs to depend on by resetting the gate Wr, and determines h in the state for controlling the output at the current time by the updating gate Wz t How many historical states h to keep t-1 And how many candidate states at the current time are retained
Continuously updating the input hidden state h at the next moment t ；
And 5.3, matching the aggregated features with the point labels, and optimizing by minimizing the residual error between the synthesis and the real labels in the training process to realize accurate segmentation of the point cloud.
The invention has the beneficial effects that:
1. according to the method, the point cloud is segmented in an object-oriented mode, so that the dependence on an optimal neighborhood is reduced to a certain extent, the influence of noise in the point cloud on feature calculation is reduced, the calculation of geometric features is facilitated, and the uncertainty of information extraction is reduced; secondly, context information between some new characteristic attributes derived by the object and the contained point clouds is beneficial to improving the classification precision, and good effect is shown in a complex outdoor scene;
2. the A-EdgeConv (Attention-EdgeConv) -based outdoor large scene point cloud segmentation method combines the local neighborhood characteristics of the hyper-points to realize the extraction of context information between point clouds, provides more local details and provides more information for the outdoor scene point cloud segmentation. Meanwhile, the attention mechanism module can focus on key points more effectively, the problem that key feature information is lost due to the fact that a maximum/average pool is used for hard integration of adjacent features is avoided, the problems that the complexity of model training is increased due to the fact that the key feature information is lost, the feature information is difficult to extract fully and the like are solved, and the segmentation precision is improved.
The above and other objects, advantages and features of the present invention will become more apparent to those skilled in the art from the following detailed description of specific embodiments thereof, taken in conjunction with the accompanying drawings.
Drawings
Some specific embodiments of the invention will be described in detail hereinafter, by way of illustration and not limitation, with reference to the accompanying drawings. The same reference numbers in the drawings identify the same or similar elements or components. Those skilled in the art will appreciate that the drawings are not necessarily drawn to scale. In the drawings:
FIG. 1 is a flow chart of the method for segmenting the outdoor large scene point cloud based on A-EdgeConv according to the present invention;
fig. 2 is a schematic diagram of step 3 of the a-EdgeConv-based outdoor large scene point cloud segmentation method according to the embodiment of the present invention;
FIG. 3 is a schematic diagram of the A-EdgeConv module of the present invention;
FIG. 4 is a diagram of the result of semantic segmentation of the outdoor large scene according to the present invention.
Detailed Description
Reference will now be made in detail to embodiments of the present invention, examples of which are illustrated in the accompanying drawings. The embodiments described below with reference to the accompanying drawings are illustrative and intended to explain the present invention and should not be construed as limiting the present invention.
The following describes an outdoor large scene point cloud segmentation method based on the a-EdgeConv according to the embodiment of the invention with reference to fig. 1 to 4.
As shown in fig. 1, an embodiment of the present invention provides an outdoor large scene point cloud segmentation method based on a-EdgeConv, including the following steps:
step 1.1, preprocessing outdoor large scene point cloud and removing noise points;
step 1.2, selecting an optimal neighborhood by minimizing the characteristic entropy;
step 1.3, combining each optimal neighborhood of the point cloud, obtaining an eigenvalue by calculating a covariance matrix of adjacent points, and calculating linear L according to the covariance eigenvalue λ Flatness P λ Degree of scattering S λ All-round difference O λ Change of curvature C λ Five characteristic indexes;
step 2, carrying out geometric homogeneity division on the point cloud in a graph cutting mode to obtain a super point;
step 2.1, constructing a minimum energy function according to the point cloud local information described in the step 1;
step 2.2, solving the problem of minimizing energy by adopting a one-cut algorithm, adaptively adjusting the size of a divided segment according to the local geometric complexity, and realizing the acquisition of the over point by graph cutting;
step 3, as shown in fig. 2, constructing a local adjacency graph based on the over points obtained in the step 2, describing local characteristics, and highlighting key information through an attention mechanism algorithm;
step 3.1, constructing a local adjacency graph inside the super-point by taking the super-point obtained in the step 2 as a basis;
step 3.2, randomly selecting a clustering initial point x from the three-dimensional point cloud data; finding a neighborhood feature point set M of an initial point x, assuming that there are n neighboring points, defined as: x = X 1 ,X 2 ,…X n Performing convolution operation on the obtained data;
step 3.3, for over-point interiorN neighborhood points X in neighborhood range of partial point cloud data X ij1 ,X ij2 ,…,X ijn The neighborhood point feature vector is expressed as:
a unique attention score is learned for the feature vector of each neighborhood point by a sharing function g.
Step 3.4, when the point q and the point p in the point set M also have a neighborhood relationship in space, the q and the p are gathered into one type; weight coefficient
Expressing the importance of each feature vector, and performing weighted summation on each feature vector to obtain the final rich feature vector representing the over-point information
Step 3.5, performing the operation on all the over points in the point set to obtain all over point characteristics;
step 3.6, overlapping a plurality of layers of A-EdgeConv modules to realize layer-by-layer extraction and summarization of local information, and realizing accurate description of the over point by overlapping different characteristic space information;
step 4, constructing a super-point diagram by utilizing the adjacency relation of the super-points to acquire super-edge characteristics;
step 4.1, constructing a hyper-point map based on the hyper-points obtained in step 2, assuming that there are adjacent hyper-points S 'and T', and if there is one or more edges formed by three-dimensional points in two hyper-points S 'and T' belonging to E, the offset set δ (S ', T') between the hyper-points is as shown in formula (13):
δ(S′,T′)＝{(p i -p j )∣(i,j)∈E∩(S′×T′)} (13)
in the formula, p i 、p j Respectively representing point clouds on two sides of an edge formed by adjacent super points;
step 4.2, the super-edge characteristics are obtained according to the relation of the super-point graph, and the characteristic value decomposition is carried out on the three-dimensional point set in the adjacent super-points to obtain the characteristic valueλ 1 ≥λ 2 ≥λ 3 ≥0；
4.3, obtaining one-dimensional and/or three-dimensional super-edge characteristics through different combinations of characteristic values;
step 5, combining a GRU gating cycle unit and a GNN graph neural network model to aggregate the characteristics of the super point and the super edge to realize accurate semantic segmentation of the point cloud;
step 5.1, embedding the over-point characteristic in the step 3 and the over-edge characteristic in the step 4 into a circulation unit as initial states;
step 5.2, inputting the over-point characteristic and the over-edge characteristic as initial values h of the hidden state 0 And input x 0 The reset gate Wr determines whether the candidate state at the current moment needs to depend on the network state at the last moment and how much the candidate state needs to depend on, and the update gate Wz determines h in the state for controlling the output at the current moment t How many historical states h to keep t-1 And how many candidate states at the current time are retained
Continuously updating the input hidden state h at the next moment t As shown in fig. 3;
and 5.3, matching the aggregated features with the point labels, and optimizing through minimizing the residual error between the synthesis and the real labels in the training process to realize accurate segmentation of the point cloud, wherein the result of the semantic segmentation of the outdoor large scene is shown in FIG. 4.
Another embodiment of the present invention provides an outdoor large scene point cloud segmentation method based on a-EdgeConv, including the following steps:
step 1.1, preprocessing outdoor large scene point cloud and removing noise points;
step 1.2, selecting the optimal neighborhood through the minimized characteristic entropy, wherein the calculation method of the minimized characteristic entropy is shown as a formula (1):
E λ ＝-λ 1 ln(λ 1 )-λ 2 ln(λ 2 )-λ 3 ln(λ 3 ) (1)
wherein λ is 1 、λ 2 、λ 3 Respectively representing the covariance eigenvalues of the neighborhood of the target point.
Step 1.3, combining each optimal neighborhood of the point cloud, obtaining an eigenvalue by calculating a covariance matrix of adjacent points, and calculating linear L according to the covariance eigenvalue λ Flatness P λ Degree of scattering S λ All formula (O) λ Change of curvature C λ Five characteristic indexes, linearity L λ Flatness P λ Degree of scattering S λ All formula (O) λ Change of curvature C λ The calculation methods of the five characteristic indexes are shown in formulas (2) to (6):
step 2, carrying out geometric homogeneity division on the point cloud in a graph cutting mode to obtain a super point;
step 2.1, constructing a minimum energy function according to the point cloud local information described in the step 1;
step 2.2, solving the problem of minimizing energy by adopting a one-cut algorithm, adaptively adjusting the size of a division segment according to the local geometric complexity, and realizing the acquisition of the over point by graph cutting;
step 3, as shown in fig. 2, constructing a local adjacency graph based on the over points obtained in the step 2, describing local features, and highlighting key information through an attention mechanism algorithm;
step 3.1, constructing a local adjacency graph inside the super-point by taking the super-point obtained in the step 2 as a basis;
step 3.2, randomly selecting a clustering initial point x from the three-dimensional point cloud data; finding a neighborhood feature point set M of an initial point x, assuming that there are n neighboring points, defined as: x = X 1 ,X 2 ,…X n Performing convolution operation on the obtained data;
step 3.3, for n neighborhood points X in the neighborhood range of the point cloud data X in the super point ij1 ,X ij2 ,…,X ijn The neighborhood point feature vector is expressed as:
learning a unique attention score for the feature vector of each neighborhood point by a sharing function g, which is defined as formula (11):
wherein W is the weight learnable by the multi-layer perceptron,
as feature vectors
And obtaining the weight coefficient through the sharing function.
Step 3.4, when the point q and the point p in the point set M also have a neighborhood relationship in space, the q and the p are gathered into one type; weight coefficient
Expressing the importance of each feature vector, and performing weighted summation on each feature vector to obtainFinal rich feature vector characterizing the hyper-point information
Feature vector
The calculation is shown in equation (12):
step 3.5, performing the operation on all the over points in the point set to obtain all over point characteristics;
step 3.6, overlapping a plurality of layers of A-EdgeConv modules to realize layer-by-layer extraction and summarization of local information, and realizing accurate description of the over point by overlapping different characteristic space information;
step 4, constructing a super-point diagram by utilizing the adjacency relation of the super-points to acquire super-edge characteristics;
step 4.1, constructing a hyper-point map based on the hyper-points obtained in step 2, assuming that there are adjacent hyper-points S 'and T', and if there is one or more edges formed by three-dimensional points in two hyper-points S 'and T' belonging to E, the offset set δ (S ', T') between the hyper-points is as shown in formula (13):
δ(S′,T′)＝{(p i -p j )∣(i,j)∈E∩(S′×T′)} (13)
in the formula, p i 、p j Respectively representing point clouds on two sides of an edge formed by adjacent super points;
step 4.2, the super-edge characteristics are obtained according to the relation of the super-point graph, and the characteristic value decomposition is carried out on the three-dimensional point set in the adjacent super-points to obtain the characteristic value lambda 1 ≥λ 2 ≥λ 3 ≥0；
4.3, obtaining one-dimensional and/or three-dimensional super-edge characteristics through different combinations of characteristic values;
step 5, combining a GRU gating cycle unit and a GNN graph neural network model to aggregate the characteristics of the super point and the super edge, and realizing accurate semantic segmentation of the point cloud;
step 5.1, embedding the over-point characteristic of the step 3 and the over-edge characteristic of the step 4 into a circulation unit as initial states;
step 5.2, inputting the over-point characteristic and the over-edge characteristic as initial values h of the hidden state 0 And input x 0 The reset gate Wr determines whether the candidate state at the current moment needs to depend on the network state at the last moment and how much the candidate state needs to depend on, and the update gate Wz determines h in the state for controlling the output at the current moment t How many historical states h to keep t-1 And how many candidate states at the current time are retained
Continuously updating the input hidden state h at the next moment t As shown in fig. 3;
and 5.3, matching the aggregated features with the point labels, and optimizing through minimizing the residual error between the synthesis and the real labels in the training process to realize accurate segmentation of the point cloud, wherein the result of the semantic segmentation of the outdoor large scene is shown in FIG. 4.
Another embodiment of the present invention provides an outdoor large scene point cloud segmentation method based on a-EdgeConv, including the following steps:
step 1.1, preprocessing outdoor large scene point cloud and removing noise points;
step 1.2, selecting the optimal neighborhood through the minimized characteristic entropy, wherein the calculation method of the minimized characteristic entropy is shown as a formula (1):
E λ ＝-λ 1 ln(λ 1 )-λ 2 ln(λ 2 )-λ 3 ln(λ 3 ) (1)
wherein λ is 1 、λ 2 、λ 3 Respectively representing covariance eigenvalues of the neighborhood of the target point.
Step 1.3, combining each optimal neighborhood of the point cloud, obtaining an eigenvalue by calculating a covariance matrix of adjacent points, and calculating linear L according to the covariance eigenvalue λ Flatness P λ Degree of scattering S λ All formula (O) λ Change in curvature C λ Five characteristic indexes, linearity L λ Flatness P λ Degree of scattering S λ All-round difference O λ Change of curvature C λ The calculation methods of the five characteristic indexes are shown in formulas (2) to (6):
step 2, carrying out geometric homogeneity division on the point cloud in a graph cutting mode to obtain a super point;
step 2.1, constructing a minimum energy function according to the point cloud local information described in the step 1; the minimum energy function is shown in equation (7):
E(G)＝∑ p∈P D p (G p )+λ∑ (p,q)∈E V p,q (G p ,G q ) (7)
wherein D is p (G p ) Is a data cost term which is the sum of all points and their penalties for assigning tag numbers, D p (G p ) Indicates the label G p Penalty when assigning to node p; sigma (p,q)∈E V p,q (G p ,G q ) To be smoothedCost term is the sum of penalties for all neighboring tags being different, V p,q (G p ,G q ) Represents the penalty of different label numbers between adjacent nodes p and q; lambda is a weight coefficient, and the numerical value reflects the proportion of the data cost item and the smooth cost item in graph cut; let the characteristic of the point p be denoted f p Which is associated with a label L p Data cost D of p (G p ) The calculation is shown in equation (8):
D p (G p )＝(g p -f p ) 2 (8)
wherein, g p Indicates a label number G p Point cloud characteristics corresponding to the segmented object;
the smoothing cost is used to penalize the inconsistency of label numbers between adjacent nodes p and q, and generally speaking, if the distance between the nodes p and q is smaller, the smoothing cost V between the nodes p and q is smaller p,q (G p ,G q ) The larger the probability that the node p and q label numbers are the same; smoothing cost V between adjacent nodes p,q (G p ,G q ) The calculation formula is shown in formulas (9) to (10):
V p,q (G p ,G q )＝exp(-ds) (9)
wherein f is p 、f q Respectively representing the geometric features of the adjacent point clouds p and q.
Step 2.2, solving the problem of minimizing energy by adopting a one-cut algorithm, adaptively adjusting the size of a division segment according to the local geometric complexity, and realizing the acquisition of the over point by graph cutting;
step 3, as shown in fig. 2, constructing a local adjacency graph based on the over points obtained in the step 2, describing local features, and highlighting key information through an attention mechanism algorithm;
step 3.1, constructing a local adjacency graph inside the super-point by taking the super-point obtained in the step 2 as a basis;
step 3.2, random three-dimensionalSelecting a clustering initial point x from the point cloud data; finding a neighborhood feature point set M of an initial point x, assuming that there are n neighboring points, defined as: x = X 1 ,X 2 ,…X n Performing convolution operation on the data;
step 3.3, for n neighborhood points X in the neighborhood range of the point cloud data X in the super point ij1 ,X ij2 ,…,X ijn The neighborhood point feature vector is expressed as:
learning a unique attention score for the feature vector of each neighborhood point by a sharing function g, which is defined as formula (11):
wherein, W is the weight learnable by the multi-layer perceptron,
as feature vectors
And obtaining the weight coefficient through the sharing function.
Step 3.4, when the neighborhood relationship also exists between the points q and p in the point set M in space, grouping q and p into one class; weight coefficient
Expressing the importance of each feature vector, and performing weighted summation on each feature vector to obtain the final rich feature vector representing the over-point information
Feature vector
The calculation is shown in equation (12):
step 3.5, performing the operation on all the over points in the point set to obtain all over point characteristics;
step 3.6, overlapping a plurality of layers of A-EdgeConv modules to realize layer-by-layer extraction and summarization of local information, and realizing accurate description of the over point by overlapping different characteristic space information;
step 4, constructing a super-point diagram by utilizing the adjacency relation of the super-points to acquire super-edge characteristics;
step 4.1, constructing a hyper-point map based on the hyper-points obtained in step 2, assuming that there are adjacent hyper-points S 'and T', and if there is one or more edges formed by three-dimensional points in two hyper-points S 'and T' belonging to E, the offset set δ (S ', T') between the hyper-points is as shown in formula (13):
δ(S′,T′)＝{(p i -p j )∣(i,j)∈E∩(S′×T′)} (13)
in the formula, p i 、p j Respectively representing point clouds on two sides of an edge formed by adjacent super points;
step 4.2, the super-edge characteristics are obtained according to the relation of the super-point graph, and the characteristic value decomposition is carried out on the three-dimensional point set in the adjacent super-points to obtain the characteristic value lambda 1 ≥λ 2 ≥λ 3 ≥0；
And 4.3, obtaining one-dimensional and/or three-dimensional super-edge characteristics through different combinations of the characteristic values, wherein the super-edge characteristics are shown in a table 1:
TABLE 1 superceding characteristics
Step 5, combining a GRU gating cycle unit and a GNN graph neural network model to aggregate the characteristics of the super point and the super edge, and realizing accurate semantic segmentation of the point cloud;
step 5.1, embedding the over-point characteristic of the step 3 and the over-edge characteristic of the step 4 into a circulation unit as initial states;
step 5.2, inputting the over-point characteristic and the over-edge characteristic as the initial value h of the hidden state 0 And input x 0 The reset gate Wr determines whether the candidate state at the current moment needs to depend on the network state at the last moment and how much the candidate state needs to depend on, and the update gate Wz determines h in the state for controlling the output at the current moment t How many historical states h to keep t-1 And how many candidate states at the current time are retained
Continuously updating the input hidden state h at the next moment t As shown in fig. 3;
and 5.3, matching the aggregated features with the point labels, and optimizing through minimizing the residual error between the synthesis and the real labels in the training process to realize accurate segmentation of the point cloud, wherein the result of the semantic segmentation of the outdoor large scene is shown in FIG. 4.
According to the method, local geometric information and a graph cut algorithm are combined to achieve the acquisition of the super point, the local adjacent graph and an attention mechanism are combined to extract the super point characteristics more accurately, the characteristic aggregation of the super point and the super edge is achieved through a GRU (generalized regression analysis unit) cyclic neural network, the point cloud segmentation speed in a large scene is greatly improved, the characteristic expression is improved due to the combination of the local adjacent graph and the attention mechanism, the effective information and the point cloud local structure are concerned, the segmentation precision is improved, and the optimization complexity is reduced.
In the description of the present invention, furthermore, the terms "first", "second" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implying any number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one such feature. In the description of the present invention, "a plurality" means at least two, e.g., two, three, etc., unless specifically limited otherwise.
In the present invention, unless otherwise expressly stated or limited, the terms "mounted," "connected," "secured," and the like are to be construed broadly and can, for example, be fixedly connected, detachably connected, or integrally formed; may be mechanically coupled, may be electrically coupled or may be in communication with each other; they may be directly connected or indirectly connected through intervening media, or they may be connected internally or in any other suitable relationship, unless expressly stated otherwise. The specific meanings of the above terms in the present invention can be understood according to specific situations by those of ordinary skill in the art.
In the present disclosure, the terms "one embodiment," "some embodiments," "an example," "a specific example," or "some examples" and the like mean that a specific feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the present disclosure. In this specification, the schematic representations of the terms used above are not necessarily intended to refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Moreover, various embodiments or examples and features of various embodiments or examples described in this specification can be combined and combined by one skilled in the art without being mutually inconsistent.
Thus, it should be appreciated by those skilled in the art that while a number of exemplary embodiments of the invention have been illustrated and described in detail herein, many other variations or modifications consistent with the principles of the invention may be directly determined or derived from the disclosure of the present invention without departing from the spirit and scope of the invention. Accordingly, the scope of the invention should be understood and interpreted to cover all such other variations or modifications.
Claims (10)
1. An outdoor large scene point cloud segmentation method based on A-EdgeConv is characterized by comprising the following steps:
step 1, describing local information of a point cloud by using local geometric features;
step 2, carrying out geometric homogeneity division on the point cloud in a graph cutting mode to obtain a super point;
step 3, constructing a local adjacency graph based on the super points obtained in the step 2, describing local characteristics, and highlighting key information through an attention mechanism algorithm;
step 4, constructing a super-point diagram by utilizing the adjacency relation of the super-points to acquire super-edge characteristics;
and 5, combining the GRU gating cycle unit and the GNN graph neural network model to aggregate the characteristics of the super point and the super edge, and realizing accurate semantic segmentation of the point cloud.
2. The segmentation method according to claim 1, wherein the step 1 comprises:
step 1.1, preprocessing outdoor large scene point cloud and removing noise points;
step 1.2, selecting an optimal neighborhood by minimizing the characteristic entropy;
step 1.3, combining each optimal neighborhood of the point cloud, obtaining an eigenvalue by calculating a covariance matrix of adjacent points, and calculating linear L according to the covariance eigenvalue λ Flatness P λ Degree of scattering S λ All formula (O) λ Change of curvature C λ Five characteristic indexes.
3. Segmentation method according to claim 2, characterized in that in step 1.2, the characteristic entropy E is minimized λ The calculation method is shown in formula (1):
E λ ＝-λ 1 ln(λ 1 )-λ 2 ln(λ 2 )-λ 3 ln(λ 3 ) (1)
wherein λ is 1 、λ 2 、λ 3 Respectively representing the covariance eigenvalues of the neighborhood of the target point.
4. Segmentation method according to claim 2, characterized in that in step 1.3, the L is linear λ Flatness P λ Degree of scattering S λ All-round difference O λ Change of curvature C λ The calculation methods of the five characteristic indexes are shown in formulas (2) to (6):
5. the segmentation method according to claim 1, wherein the step 2 comprises:
step 2.1, constructing a minimized energy function according to the point cloud local information described in the step 1;
and 2.2, solving the problem of minimizing energy by adopting a one-cut algorithm, adaptively adjusting the size of the divided segments according to the local geometric complexity, and realizing the acquisition of the over point through graph cutting.
6. Segmentation method according to claim 5, characterized in that in step 2.1, the minimization of the energy function E (G) is constructed as shown in equation (7):
E(G)＝∑ p∈P D p (G p )+λ∑ (p,q)∈E V p,q (G p ,G q ) (7)
wherein D is p (G p ) Is a data cost term, which is the sum of all points and their penalties for assigning tag numbers, D p (G p ) Indicates a label G p Penalty in assigning to node p; sigma (p,q)∈E V p,q (G p ,G q ) For the smoothing cost term, is the sum of penalties for all neighboring labels to be different, V p,q (G p ,G q ) Represents the penalty of different label numbers between adjacent nodes p and q; lambda is a weight coefficient, and the numerical value reflects the proportion of a data cost item and a smooth cost item in graph cutting; suppose that the feature of point p is denoted f p Which is associated with a label L p Data cost D of p (G p ) The calculation is shown in equation (8):
D p (G p )＝(g p -f p ) 2 (8)
wherein, g p Indicates a label number G p Point cloud characteristics corresponding to the segmented object;
the smoothing cost is used to punish the inconsistency of label number between the adjacent nodes p and q, generally speaking, if the distance between the nodes p and q is smaller, the smoothing cost V between the nodes p and q is p,q (G p ,G q ) The greater the probability that the node p and q tag numbers are the same; smoothing cost V between adjacent nodes p,q (G p ,G q ) The calculation formula is shown in formulas (9) to (10):
V p,q (G p ,G q )＝exp(-ds) (9)
wherein f is p 、f q Respectively representing the geometrical characteristics of the adjacent point clouds p and q.
7. The segmentation method according to claim 1, wherein the step 3 comprises:
step 3.1, constructing a local adjacency graph inside the super-point by taking the super-point obtained in the step 2 as a basis;
step 3.2, randomly selecting a clustering initial point x from the three-dimensional point cloud data; finding a neighborhood feature point set M of an initial point x, assuming that there are n neighboring points, defined as: x = X 1 ,X 2 ,…X n Performing convolution operation on the obtained data;
step 3.3, for n neighborhood points X in the neighborhood range of the point cloud data X in the super point ij1 ,X ij2 ,…,X ijn The neighborhood point feature vector is expressed as:
learning a unique attention score for the feature vector of each neighborhood point through a sharing function g;
step 3.4, when the point q and the point p in the point set M also have a neighborhood relationship in space, the q and the p are gathered into one type; weight coefficient
Expressing the importance of each feature vector, and performing weighted summation on each feature vector to obtain the final rich feature vector representing the over-point information
Step 3.5, performing the operation on all the over points in the point set to obtain all over point characteristics;
and 3.6, overlapping a plurality of layers of A-EdgeConv modules to realize layer-by-layer extraction and summarization of local information, and accurately describing the super point by overlapping different characteristic space information.
8. The segmentation method according to claim 7, wherein in step 3.3, the definition of the sharing function g is shown in formula (11):
wherein, W is the weight learnable by the multi-layer perceptron,
as feature vectors
A weight coefficient obtained by a sharing function;
in said step 3.4, the feature vectors
The calculation is shown in equation (12):
9. the segmentation method according to claim 1, wherein the step 4 comprises:
step 4.1, constructing a hyper-point map based on the hyper-points obtained in step 2, assuming that there are adjacent hyper-points S 'and T', and if there is one or more edges formed by three-dimensional points in two hyper-points S 'and T' belonging to E, the offset set δ (S ', T') between the hyper-points is as shown in formula (13):
δ(S′,T′)＝{(p i -p j )∣(i,j)∈E∩(S′×T′)} (13)
in the formula, p i 、p j Respectively representing point clouds on two sides of an edge formed by adjacent super points;
step 4.2, the super-edge characteristics are obtained according to the relation of the super-point graph, and the characteristic value decomposition is carried out on the three-dimensional point set in the adjacent super-points to obtain the characteristic value lambda 1 ≥λ 2 ≥λ 3 ≥0；
And 4.3, obtaining the one-dimensional and/or three-dimensional super-edge characteristics through different combinations of the characteristic values.
10. The segmentation method according to claim 1, wherein the step 5 comprises:
step 5.1, embedding the over-point characteristic of the step 3 and the over-edge characteristic of the step 4 into a circulation unit as initial states;
step 5.2, inputting the over-point characteristic and the over-edge characteristic to doIs an initial value h of a hidden state 0 And input x 0 The reset gate Wr determines whether the candidate state at the current moment needs to depend on the network state at the last moment and how much the candidate state needs to depend on, and the update gate Wz determines h in the state for controlling the output at the current moment t How many historical states h to keep t-1 And how many candidate states at the current time are retained
Continuously updating the input hidden state h at the next moment t ；
And 5.3, matching the aggregated features with the point labels, and optimizing by minimizing the residual error between the synthesis and the real labels in the training process to realize accurate segmentation of the point cloud.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211513938.4A CN115731137B (en) | 2022-11-29 | 2022-11-29 | Outdoor large scene point cloud segmentation method based on A-EdgeConv |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211513938.4A CN115731137B (en) | 2022-11-29 | 2022-11-29 | Outdoor large scene point cloud segmentation method based on A-EdgeConv |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115731137A true CN115731137A (en) | 2023-03-03 |
| CN115731137B CN115731137B (en) | 2024-07-26 |
Family
ID=85299105
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211513938.4A Active CN115731137B (en) | 2022-11-29 | 2022-11-29 | Outdoor large scene point cloud segmentation method based on A-EdgeConv |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115731137B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117392386A (en) * | 2023-10-13 | 2024-01-12 | 浙江省测绘科学技术研究院 | Classification training method and device for superside mask generation network based on instance segmentation |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108319957A (en) * | 2018-02-09 | 2018-07-24 | 深圳市唯特视科技有限公司 | A kind of large-scale point cloud semantic segmentation method based on overtrick figure |
| CN110097556A (en) * | 2019-04-29 | 2019-08-06 | 东南大学 | Large-scale point cloud semantic segmentation algorithm based on PointNet |
| CN113674286A (en) * | 2021-08-31 | 2021-11-19 | 浙江工商大学 | Dental model point cloud segmentation method based on cross-image attention machine mechanism and cost function learning |
| US20220076432A1 (en) * | 2020-05-06 | 2022-03-10 | Luminar, Llc | Neural network for object detection and tracking |
| US20220222824A1 (en) * | 2020-09-15 | 2022-07-14 | Sri International | Fully automated multimodal system architecture for semantic segmentation of large-scale 3d outdoor point cloud data |
| CN114758129A (en) * | 2022-04-12 | 2022-07-15 | 西安理工大学 | RandLA-Net outdoor scene semantic segmentation method based on local feature enhancement |
-
2022
- 2022-11-29 CN CN202211513938.4A patent/CN115731137B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108319957A (en) * | 2018-02-09 | 2018-07-24 | 深圳市唯特视科技有限公司 | A kind of large-scale point cloud semantic segmentation method based on overtrick figure |
| CN110097556A (en) * | 2019-04-29 | 2019-08-06 | 东南大学 | Large-scale point cloud semantic segmentation algorithm based on PointNet |
| US20220076432A1 (en) * | 2020-05-06 | 2022-03-10 | Luminar, Llc | Neural network for object detection and tracking |
| US20220222824A1 (en) * | 2020-09-15 | 2022-07-14 | Sri International | Fully automated multimodal system architecture for semantic segmentation of large-scale 3d outdoor point cloud data |
| CN113674286A (en) * | 2021-08-31 | 2021-11-19 | 浙江工商大学 | Dental model point cloud segmentation method based on cross-image attention machine mechanism and cost function learning |
| CN114758129A (en) * | 2022-04-12 | 2022-07-15 | 西安理工大学 | RandLA-Net outdoor scene semantic segmentation method based on local feature enhancement |
Non-Patent Citations (1)
| Title |
|---|
| YEN-PO LIN等: "Attention EdgeConv For 3D Point Cloud Classification", PROCEEDINGS, APSIPA ANNUAL SUMMIT AND CONFERENCE 2021, 17 December 2021 (2021-12-17), pages 1 - 5 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117392386A (en) * | 2023-10-13 | 2024-01-12 | 浙江省测绘科学技术研究院 | Classification training method and device for superside mask generation network based on instance segmentation |
| CN117392386B (en) * | 2023-10-13 | 2024-05-10 | 浙江省测绘科学技术研究院 | Classification training method and device for superside mask generation network based on instance segmentation |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115731137B (en) | 2024-07-26 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| He et al. | Dyco3d: Robust instance segmentation of 3d point clouds through dynamic convolution | |
| CN108132968B (en) | Weak supervision learning method for associated semantic elements in web texts and images | |
| CN107066559B (en) | Three-dimensional model retrieval method based on deep learning | |
| Lin et al. | Active and incremental learning for semantic ALS point cloud segmentation | |
| Oliva et al. | Scene-centered description from spatial envelope properties | |
| CN105809672B (en) | A kind of image multiple target collaboration dividing method constrained based on super-pixel and structuring | |
| Ren et al. | 3d-a-nets: 3d deep dense descriptor for volumetric shapes with adversarial networks | |
| Xia et al. | Exploiting deep features for remote sensing image retrieval: A systematic investigation | |
| CN108897791B (en) | Image retrieval method based on depth convolution characteristics and semantic similarity measurement | |
| Chen et al. | Learning attentive and hierarchical representations for 3d shape recognition | |
| Amer et al. | Monocular extraction of 2.1 d sketch using constrained convex optimization | |
| Yang et al. | Local label descriptor for example based semantic image labeling | |
| CN112364747B (en) | Target detection method under limited sample | |
| Zhao et al. | DAR-Net: Dynamic aggregation network for semantic scene segmentation | |
| CN115731137A (en) | Outdoor large scene point cloud segmentation method based on A-EdgeConv | |
| Wen et al. | Learnable skeleton-aware 3d point cloud sampling | |
| CN112860936B (en) | Visual pedestrian re-identification method based on sparse graph similarity migration | |
| Mukilan et al. | Human and object detection using hybrid deep convolutional neural network | |
| Ghosal et al. | A comparative study among clustering techniques for leaf segmentation in rosette plants | |
| Fabijanska | Graph Convolutional Networks for Semi-Supervised Image Segmentation | |
| Shyr et al. | Supervised hierarchical Pitman-Yor process for natural scene segmentation | |
| Guo | Research on sports video retrieval algorithm based on semantic feature extraction | |
| Li et al. | A non-rigid 3D model retrieval method based on scale-invariant heat kernel signature features | |
| CN111488923B (en) | Enhanced anchor point image semi-supervised classification method | |
| Özkan et al. | Comparing feature extraction techniques for urban land‐use classification |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |