CN114511571A - Point cloud data semantic segmentation method and system and related components - Google Patents
Point cloud data semantic segmentation method and system and related components Download PDFInfo
- Publication number
- CN114511571A CN114511571A CN202011168398.1A CN202011168398A CN114511571A CN 114511571 A CN114511571 A CN 114511571A CN 202011168398 A CN202011168398 A CN 202011168398A CN 114511571 A CN114511571 A CN 114511571A
- Authority
- CN
- China
- Prior art keywords
- point cloud
- point
- cloud data
- features
- feature
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 78
- 230000011218 segmentation Effects 0.000 title claims abstract description 77
- 238000003860 storage Methods 0.000 claims abstract description 10
- 238000012545 processing Methods 0.000 claims description 21
- 230000008569 process Effects 0.000 claims description 13
- 239000013598 vector Substances 0.000 claims description 13
- 238000004590 computer program Methods 0.000 claims description 11
- 230000006870 function Effects 0.000 claims description 9
- 238000011156 evaluation Methods 0.000 claims description 8
- 238000004422 calculation algorithm Methods 0.000 claims description 5
- 230000002708 enhancing effect Effects 0.000 claims description 4
- 238000005457 optimization Methods 0.000 claims description 4
- 230000001174 ascending effect Effects 0.000 claims description 3
- 239000000126 substance Substances 0.000 claims 1
- 238000005070 sampling Methods 0.000 abstract description 10
- 230000009286 beneficial effect Effects 0.000 abstract description 4
- 238000012800 visualization Methods 0.000 description 9
- 230000000875 corresponding effect Effects 0.000 description 8
- 230000000694 effects Effects 0.000 description 5
- 238000004364 calculation method Methods 0.000 description 4
- 230000009471 action Effects 0.000 description 3
- 238000013528 artificial neural network Methods 0.000 description 3
- 238000012549 training Methods 0.000 description 3
- 230000001788 irregular Effects 0.000 description 2
- 230000008447 perception Effects 0.000 description 2
- 238000000638 solvent extraction Methods 0.000 description 2
- 230000002776 aggregation Effects 0.000 description 1
- 238000004220 aggregation Methods 0.000 description 1
- 238000013527 convolutional neural network Methods 0.000 description 1
- 230000002596 correlated effect Effects 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 238000009826 distribution Methods 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
- 238000010845 search algorithm Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/23—Clustering techniques
- G06F18/232—Non-hierarchical techniques
- G06F18/2321—Non-hierarchical techniques using statistics or function optimisation, e.g. modelling of probability density functions
- G06F18/23213—Non-hierarchical techniques using statistics or function optimisation, e.g. modelling of probability density functions with fixed number of clusters, e.g. K-means clustering
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2413—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on distances to training or reference patterns
- G06F18/24147—Distances to closest patterns, e.g. nearest neighbour classification
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10028—Range image; Depth image; 3D point clouds
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10032—Satellite or aerial image; Remote sensing
- G06T2207/10044—Radar image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30181—Earth observation
- G06T2207/30188—Vegetation; Agriculture
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Evolutionary Computation (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Engineering & Computer Science (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computational Linguistics (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Probability & Statistics with Applications (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Image Analysis (AREA)
Abstract
The application discloses a point cloud data segmentation method, which comprises the steps of defining descriptors of the characteristics of each point cloud data in a point cloud space, wherein the characteristics comprise linear characteristics, flatness characteristics, scattering characteristics and perpendicularity characteristics; calculating descriptors of all points in a neighborhood space corresponding to each point cloud data; cascading all the features to obtain new features of the point cloud data; cascading the new features with descriptors of the central point of the neighborhood space to obtain target point features; dividing the characteristics of the target point to obtain m point sets with different sizes, wherein m is a positive integer; acquiring a point characteristic value of each cluster of point sets, and segmenting point cloud data into m types through MLP; and performing near point interpolation according to the class and the nearest interpolation method to complete point cloud semantic segmentation. According to the method and the device, point cloud information loss caused by random sampling can be reduced by constructing redundant point cloud features. The application also discloses a point cloud data segmentation system, a point cloud data segmentation device and a computer readable storage medium, which have the beneficial effects.
Description
Technical Field
The application relates to the field of forestry tree measurement, in particular to a point cloud data segmentation method, a point cloud data segmentation system and related components.
Background
The forestry-oriented laser radar point cloud data is often abnormally large in data level, and is difficult to process by directly using a deep learning method, while the traditional voxelization method also has the problems of long Processing time and limitation of a Graphic Processing Unit (GPU) memory. Early methods typically employed three-dimensional convolutional neural networks based on three-dimensional point clouds, the three-dimensional graph typically being represented by a probability distribution of binary variables over a voxel grid. While good performance has been achieved, these methods do not scale well to dense three-dimensional data because the computational and memory footprint grows in cubes with increasing resolution. At present, some schemes for directly processing large-scale point clouds exist, for example, SPG uses a hypergraph (super graph) and a hypergpoint (super points) to represent a large-scene point cloud, and methods such as FCPN and PCT combine the advantages of volume and point to process the large-scale point cloud. Although these methods also achieve good segmentation effect, most methods have too large preprocessing calculation amount or high memory occupation, and are difficult to deploy in practical application.
The realization of efficient and accurate semantic segmentation of large-scene three-dimensional point clouds is one of the key problems of current three-dimensional scene understanding and environment intelligent perception, and an important task is to effectively reduce the large-scale point clouds in terms of the area of hundreds of meters collected by a mobile laser radar and the large-scale scene point clouds in tens of millions of orders. The random sampling method has the advantages of low calculation cost, small GPU memory occupation, high operation efficiency and the like, and has no requirement on the input number of the point clouds, so that the point clouds with any size can be directly input into a network for training, but the method can cause the loss of effective information of the point clouds, thereby often generating errors when the extracted data is segmented and applied in the next step, and reducing the accuracy. We construct redundant features of the point cloud to reduce the loss of valid information.
Therefore, how to provide a solution to the above technical problem is a problem that needs to be solved by those skilled in the art.
Disclosure of Invention
The application aims to provide a point cloud data segmentation method, a point cloud data segmentation system, a point cloud data segmentation device and a computer readable storage medium, which can reduce point cloud information loss caused by random sampling by constructing redundant point cloud features.
In order to solve the technical problem, the application provides a point cloud data semantic segmentation method, which comprises the following steps:
defining descriptors of features of each point cloud data in a point cloud space, the features including linear features, flatness features, scattering features, and perpendicularity features;
calculating the descriptors of all points in a neighborhood space corresponding to each point cloud data;
cascading all the features to obtain new features of the point cloud data;
cascading the new features with the descriptors of the central points of the neighborhood space to obtain target point features;
the target point features are segmented to obtain m point sets with different sizes, wherein m is a positive integer;
acquiring a point characteristic value of each cluster of the point set, and segmenting the point cloud data into m types through MLP;
and performing near point interpolation according to the class and the nearest interpolation method to complete point cloud segmentation.
Preferably, before the calculating the descriptor of each point in the neighborhood corresponding to each point cloud data, the point cloud data segmentation method further includes:
and determining K adjacent points of the neighborhood space of each point cloud data through a K adjacent algorithm.
Preferably, the process of concatenating all the features includes:
learning an attribute score for each of the features via a first relationship;
learning a separate attention score for each of the point cloud data using learnable parameters of MLP;
and obtaining the weighted sum of each feature point level of the neighborhood space according to the soft mask of the automatically selected features.
Preferably, after the descriptor of the feature of each point cloud data in the point cloud space, the point cloud data segmentation method further includes:
and enhancing the characteristics of each point cloud data through rotation.
Preferably, after obtaining the target point feature, the point cloud data segmentation method further includes:
performing energy optimization processing on the target point characteristics according to an energy function, wherein the energy function is
Preferably, the process of obtaining the point feature value of each cluster of the point set includes:
performing MLP of shared parameters on each cluster of point sets, and performing dimension ascending to obtain a feature vector of a preset dimension;
performing maxporoling on the point set of each cluster to obtain the local characteristics of the preset dimensionality of the point set of each cluster;
performing maxporoling on the local features corresponding to all the point sets to obtain global features;
and copying the global feature n times so as to connect the target point feature to the global feature with dimensions of n multiplied by 1024 to obtain a plurality of point feature values.
Preferably, the point cloud data segmentation method further includes:
and evaluating the semantic segmentation data result.
In order to solve the above technical problem, the present application further provides a point cloud data segmentation system, including:
a definition module for defining descriptors of features of each point cloud data in a point cloud space, the features including linear features, flatness features, scattering features, and perpendicularity features;
the computing module is used for computing the descriptors of all points in a neighborhood space corresponding to each point cloud data;
the first cascade module is used for cascading all the characteristics to obtain new characteristics of the point cloud data;
the second cascading module is used for cascading the new feature and the descriptor of the central point of the neighborhood space to obtain a target point feature;
and the segmentation module is used for segmenting the target point features to obtain m point sets with different sizes, acquiring the point feature value of each cluster of the point sets, segmenting the point cloud data into m classes through MLP (Multi level processing), and performing near point interpolation according to the classes and a nearest interpolation method to complete point cloud segmentation, wherein m is a positive integer.
In order to solve the above technical problem, the present application further provides a point cloud data segmentation apparatus, including:
a memory for storing a computer program;
a processor for implementing the steps of the point cloud data segmentation method as described in any one of the above when the computer program is executed.
To solve the above technical problem, the present application further provides a computer-readable storage medium, on which a computer program is stored, and the computer program, when executed by a processor, implements the steps of the point cloud data segmentation method according to any one of the above items.
The application provides a point cloud data semantic segmentation method, redundant point cloud features are obtained by processing and cascading various features, point cloud information loss caused by random sampling is reduced by constructing the redundant point cloud features, a geometric feature descriptor of the point cloud is fused into a subsequent deep neural network, so that real-time rapid down-sampling can be realized, effective information of the geometric features of the point cloud can be maximally reserved, and a better partitioning effect is achieved on sparse and irregular point clouds. The application also provides a point cloud data segmentation system, a point cloud data segmentation device and a computer readable storage medium, and the point cloud data segmentation method has the same beneficial effects as the point cloud data segmentation method.
Drawings
In order to more clearly illustrate the embodiments of the present application, the drawings needed for the embodiments will be briefly described below, and it is obvious that some of the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings without creative efforts.
FIG. 1 is a diagram of a semantic segmentation pointLAE network architecture provided herein;
fig. 2 is a plurality of point cloud feature visualization processing results provided in the present application, wherein (a) Semantic3D data set point cloud geometric features are visualized (b) S3DIS data set point cloud geometric features are visualized (c) Nuscene data set point cloud geometric features are visualized (d) bjfumap data set main building area point cloud geometric features are visualized (e) bjfumap data set main road area point cloud geometric features are visualized;
fig. 3 is several super-segmentation visualization processing results provided in the present application, wherein (a) a Semantic3D data set point cloud super-segmentation visualization (b) an S3DIS data set point cloud super-segmentation visualization (c) a Nuscene data set point cloud super-segmentation visualization (d) a bjfumap data set main building region point cloud super-segmentation visualization (e) a bjfumap data set main road region point cloud super-segmentation visualization;
FIG. 4 is a visualization of several data segmentation effects provided herein, wherein (a) pointLAE visualizes the segmentation effect of a dataset in a semantic3D (b) pointent + + visualization (c) semantic3D label group project;
fig. 5 is a summary attached drawing provided by the present application, and relates to a flow chart of the point cloud data semantic segmentation processing invention.
Detailed Description
The core of the application is to provide a point cloud data segmentation method, a system, a device and a computer readable storage medium, which can reduce point cloud information loss caused by random sampling by constructing redundant point cloud features.
In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are some embodiments of the present application, but not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
The present application provides a step flow of a point cloud data segmentation method, that is, the point cloud data segmentation method includes:
s101: defining descriptors of the characteristics of each point cloud data in the point cloud space, wherein the characteristics comprise linear characteristics, flatness characteristics, scattering characteristics and verticality characteristics;
specifically, the method includes the steps of firstly conducting random downsampling on the whole large-scale point cloud space, finding K neighbor points of a neighborhood space for each point p through a K neighbor search algorithm knn, and coding the geometric features, positions and relative positions of the neighbor points. In order to limit the burden of calculation amount, the application only defines descriptors of local geometric features of 4 point clouds, and the descriptors are used for describing the spatial feature structure, linearity, planarity, scattering property and verticality of the point clouds.
Linearity: degree of stretch elongation of point cloud neighborhood
Flatness degree: evaluating its fit to a plane
Scattering property: corresponding to the spherical neighborhood, isotropic features are described.
Verticality descriptor: the vertical nature of the best neighborhood is an important feature to distinguish between roads and facades and between polygons and vertical object point clouds.
The first three features are collectively called the dimensional characteristics of the point cloud, the vertical feature descriptors are also obtained from the above feature vectors and feature values, let a1, a2, and a3 be three feature vectors associated with λ 1, λ 2, and λ 3, respectively, and the present application defines the unitary vector of the principal direction in the three vectors as the sum of absolute values of coordinates of the feature vectors weighted according to their feature values: the application considers that the vertical component of this vector characterizes the verticality of a knn point neighborhood. For a horizontal neighborhood it reaches a minimum value (equal to zero) and for a linear vertical neighborhood it reaches a maximum value (equal to 1), which facilitates the division of the horizontal neighborhood and the linear vertical neighborhood.
In each knn neighborhood, the present application calculates the point cloud covariance eigenvalue λ 1 ≧ λ 2 ≧ λ 3, and the neighborhood size is chosen to minimize the eigen-1/λ, λ 2/λ, λ 3/λ, and ^ P3 i ═ 1 λ i, according to the Weinmann et al's best neighborhood principle. These features allow the present application to best describe the geometry of the local neighborhood by the following vector.
S102: calculating descriptors of all points in a neighborhood space corresponding to each point cloud data;
specifically, the K nearest neighbor algorithm is used for each point piFinding the nearest k neighborhood points in Euclidean space
And calculating the geometric feature descriptor f of the central point of each neighborhood according to the formulas (1), (2) and (3)i L，fi s， fi P，fiV and geometric feature descriptor of its neighborhood points
S103: cascading all the features to obtain new features of the point cloud data;
s104: cascading the new features with descriptors of the central point of the neighborhood space to obtain target point features;
specifically, MLP with shared weight is adopted to cascade neighborhood geometric features of points together to obtain new point features
The characteristic is a redundancy characteristic of points, and the loss of effective information can be effectively reduced in the random sampling process. And by the method of concat will
And point geometric feature descriptor fi L，fi s，fi P，fiV are cascaded to obtain point features
In order to aggregate the above features together, max posing method can be used, but this drastic method results in many loss of useful information, and the present application wants to automatically learn and select the useful information in the concatenated features of aggregation 1 by the method of attention, as shown in formula (6), and the present application uses function α to learn attention score for each feature, and learns a separate attention score for each point by sharing W of learnable parameters of MLP, and obtains weighted summation calculation of the neighborhood feature point level by this automatically selected soft mask of the features, as shown in (7).
S105: dividing the characteristics of the target point to obtain m point sets with different sizes, wherein m is a positive integer;
s106: acquiring a point characteristic value of each cluster of point sets, and segmenting point cloud data into m types through MLP;
s107: and performing near point interpolation according to the class and the nearest interpolation method to complete point cloud segmentation.
Specifically, as shown in fig. 1, a point cloud of any scale is randomly down-sampled and processed by the ATSE module to obtain a D-dimensional feature
Combining the inherent X, Y and Z three-dimensional characteristics of the point cloud and the n X (3+ D) characteristics, inputting the characteristics into an MLP module of a basic pointnet, performing characteristic dimension increasing to obtain n X64 characteristic vectors, wherein at the moment, if the subsequent module processing of the pointnet is continuously performed, the problem of the pointnet receptive field is solved, and considering that the input point cloud is greatly down-sampled, the significant increase of the perception of each point is also very necessary.
In order to keep the overall geometric details of the input point cloud even if the characteristics of some points are discarded in a network, the invention provides a method for effectively dividing the local geometric receptive field of the point cloud by using a super-division module, which is also a method combined with unsupervised learning.
The first step of the algorithm is to divide the point cloud into geometrically simple but meaningful shapes. By the module, high-quality point cloud super-segmentation can be generated, which is equivalent to pre-segmentation with high reliability in a geometric sense and has the following three attributes:
object non-overlapping: the point clouds on different objects are not overlapped with each other, especially under the condition that the represented semantics are different;
boundary property: overlapping the boundary between the super-segmented point cloud cluster and the object;
regularity: the super-segmentation shape and profile must be simple.
We represent the down-sampled point cloud with an undirected graph G ═ V, E, where V represents nodes, representing points of the point cloud, and E represents edges, i.e., the process of encoding the mutual proximity of nodes V. We compute only 10 nearest neighbor maps, for each point we will have its local geometric feature vector fi ∈ R4 (dimension and perpendicularity, i.e. the geometric feature descriptor f computed in the foregoingiL， fi s，fi P，fiV) are correlated, and a piecewise constant is calculated
And constructing a function from the graph G
Defined as a vector of RV × 4, the following Potts segmentation energies are minimized by optimizing the following energy functions.
Therefore, according to the embodiment, redundant point cloud features are obtained by processing and cascading the features, point cloud information loss caused by random sampling is reduced by constructing the redundant point cloud features, and geometric feature descriptors of the point cloud are fused into a subsequent deep neural network, so that real-time and rapid down-sampling can be realized, effective information of the geometric features of the point cloud can be maximally reserved, and a better partitioning effect is achieved on sparse and irregular point clouds.
On the basis of the above-mentioned embodiment, as a preferred embodiment, after the descriptor of the feature of each point cloud data in the point cloud space, the point cloud data segmentation method further includes:
and enhancing the characteristics of each point cloud data through rotation.
Specifically, data enhancement processing is performed through rotation, a data set is added on the basis of a sematic 3D training result for training, and random recall processing is performed on dense point clouds by adopting a random dropout method, so that generalization capability of the point cloud sparse uneven scene can be enhanced.
As a preferred embodiment, the point cloud data segmentation method further includes:
and evaluating the semantic segmentation data.
We applied the recall IOU, per IOU type, union intersection, total accuracy OA to evaluate the data set used following the evaluation index of sematic 3D. Wherein, cijIs the number of samples predicted as class j from class i groudtruth. A. the-IoU is an evaluation index for each class, IoU is an overall evaluation index for the data set, and OA is an overall accuracy evaluation index for the data set.
The structure of a point cloud data segmentation system provided by the application comprises:
the system comprises a definition module, a data processing module and a data processing module, wherein the definition module is used for defining descriptors of characteristics of each point cloud data in a point cloud space, and the characteristics comprise linear characteristics, flatness characteristics, scattering characteristics and perpendicularity characteristics;
the computing module is used for computing descriptors of all points in a neighborhood space corresponding to each point cloud data;
the first cascade module is used for cascading all the characteristics to obtain new characteristics of the point cloud data;
the second cascade module is used for cascading the new features with the descriptors of the central points of the neighborhood space to obtain target point features;
and the segmentation module is used for segmenting the characteristics of the target point to obtain m point sets with different sizes, acquiring the point characteristic value of each cluster of point sets, segmenting the point cloud data into m classes through MLP (Multi level processing), and performing near point interpolation according to the classes and a nearest interpolation method to complete point cloud segmentation, wherein m is a positive integer.
Therefore, in the embodiment, redundant point cloud features are obtained by processing and cascading the features, point cloud information loss caused by random sampling is reduced by constructing the redundant point cloud features, and geometric feature descriptors of the point cloud are fused into a subsequent deep neural network.
As a preferred embodiment, the point cloud data segmentation system further comprises:
and the determining module is used for determining K adjacent points of the neighborhood space of each point cloud data through a K adjacent algorithm.
As a preferred embodiment, the process of cascading all features includes:
learning an attribute score for each feature via a first relationship;
learning a separate attention score for each point cloud data using the learnable parameters of the MLP;
and according to the soft mask of the automatically selected features, obtaining the weighted sum of each feature point level of the neighborhood space.
As a preferred embodiment, the point cloud data segmentation system further includes:
and the enhancement processing module is used for enhancing the characteristics of each point cloud data through rotation.
As a preferred embodiment, the point cloud data segmentation apparatus further includes:
the energy optimization module is used for carrying out energy optimization processing on the target point characteristics according to an energy function, wherein the energy function is
As a preferred embodiment, the process of obtaining the point feature value of each cluster of point sets includes:
performing MLP of shared parameters on each cluster of point sets, and performing dimension ascending to obtain a feature vector of a preset dimension;
performing maxporoling on each cluster point set to obtain local characteristics of preset dimensionality of each cluster point set;
performing maxporoling on local features corresponding to all point sets to obtain global features;
the global feature is copied n times to connect the target point feature to the n × 1024-dimensional global feature, resulting in a plurality of point feature values.
As a preferred embodiment, the point cloud data segmentation system further comprises
And the evaluation module is used for evaluating the semantic segmentation data.
On the other hand, the present application further provides a point cloud data segmentation apparatus, including:
a memory for storing a computer program;
a processor for implementing the steps of the point cloud data segmentation method as described in any one of the above embodiments when executing the computer program.
Please refer to the above embodiments for the introduction of the point cloud data segmentation apparatus provided in the present application, which is not described herein again.
The point cloud data segmentation device has the same beneficial effects as the point cloud data segmentation method.
In order to solve the above technical problem, the present application further provides a computer-readable storage medium, on which a computer program is stored, and the computer program, when executed by a processor, implements the steps of the point cloud data segmentation method as described in any one of the above.
For the introduction of a computer-readable storage medium provided in the present application, please refer to the above embodiments, which are not described herein again.
The computer-readable storage medium provided by the application has the same beneficial effects as the point cloud data segmentation method.
It is further noted that, in the present specification, relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the application. Thus, the present application is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims (10)
1. A point cloud data segmentation method is characterized by comprising the following steps:
defining descriptors of features of each point cloud data in a point cloud space, the features including linear features, flatness features, scattering features, and perpendicularity features;
calculating the descriptors of all points in a neighborhood space corresponding to each point cloud data;
cascading all the features to obtain new features of the point cloud data;
cascading the new features with the descriptors of the central points of the neighborhood space to obtain target point features, and segmenting the target point features to obtain m point sets with different sizes, wherein m is a positive integer;
acquiring a point characteristic value of each cluster of the point set, and segmenting the point cloud data into m types through MLP;
and performing near point interpolation according to the class and the nearest interpolation method to complete point cloud segmentation.
2. The method of point cloud data segmentation according to claim 1, wherein before the computing the descriptor of each point in the neighborhood corresponding to each point cloud data, the method further comprises:
and determining K adjacent points of the neighborhood space of each point cloud data through a K adjacent algorithm.
3. The point cloud data segmentation method of claim 1, wherein the process of concatenating all the features comprises:
learning an attribute score for each of the features via a first relationship;
learning a separate attention score for each of the point cloud data using learnable parameters of MLP;
and obtaining the weighted sum of each feature point level of the neighborhood space according to the soft mask of the automatically selected features.
4. The point cloud data segmentation method of claim 1, wherein the descriptor of the feature of each point cloud data in the point cloud space is followed by the point cloud data segmentation method further comprising:
and enhancing the characteristics of each point cloud data through rotation.
5. The point cloud data segmentation method of claim 1, wherein after obtaining the target point feature, the point cloud data segmentation method further comprises:
performing energy optimization processing on the target point characteristics according to an energy function, wherein the energy function is optimizedIs composed of
。
6. The point cloud data segmentation method according to claim 5, wherein the process of obtaining the point feature value of the point set for each cluster comprises:
performing MLP of shared parameters on each cluster of point sets, and performing dimension ascending to obtain a feature vector of a preset dimension;
performing maxporoling on the point set of each cluster to obtain the local characteristics of the preset dimensionality of the point set of each cluster;
performing maxporoling on the local features corresponding to all the point sets to obtain global features;
and copying the global feature n times so as to connect the target point feature to the global feature with dimensions of n multiplied by 1024 to obtain a plurality of point feature values.
7. The point cloud data segmentation method of any one of claims 1-6, wherein the point cloud data semantic segmentation method further comprises:
evaluating the semantic segmentation result data;
evaluating the used data set by applying recall IOU, each type of IOU, joint intersection and total precision OA according to the evaluation index of sematic 3D;
wherein the content of the first and second substances,
is the number of samples predicted as class j from class i groudtruth;
8. A point cloud data segmentation system, comprising:
a definition module for defining descriptors of features of each point cloud data in a point cloud space, the features including linear features, flatness features, scattering features, and perpendicularity features;
the computing module is used for computing the descriptors of all points in a neighborhood space corresponding to each point cloud data;
the first cascade module is used for cascading all the characteristics to obtain new characteristics of the point cloud data;
the second cascading module is used for cascading the new feature and the descriptor of the central point of the neighborhood space to obtain a target point feature;
and the segmentation module is used for segmenting the target point features to obtain m point sets with different sizes, acquiring the point feature value of each cluster of the point sets, segmenting the point cloud data into m classes through MLP (Multi level processing), and performing near point interpolation according to the classes and a nearest interpolation method to complete point cloud segmentation, wherein m is a positive integer.
9. A point cloud data semantic segmentation device is characterized by comprising:
a memory for storing a computer program;
a processor for implementing the steps of the point cloud data segmentation method according to any one of claims 1 to 7 when executing the computer program.
10. A computer-readable storage medium, characterized in that the computer-readable storage medium has stored thereon a computer program which, when being executed by a processor, carries out the steps of the point cloud data segmentation method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011168398.1A CN114511571A (en) | 2020-10-28 | 2020-10-28 | Point cloud data semantic segmentation method and system and related components |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011168398.1A CN114511571A (en) | 2020-10-28 | 2020-10-28 | Point cloud data semantic segmentation method and system and related components |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114511571A true CN114511571A (en) | 2022-05-17 |
Family
ID=81547390
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011168398.1A Pending CN114511571A (en) | 2020-10-28 | 2020-10-28 | Point cloud data semantic segmentation method and system and related components |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114511571A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116503390A (en) * | 2023-06-25 | 2023-07-28 | 深圳市智宇精密五金塑胶有限公司 | Hardware part defect detection method based on computer vision |
-
2020
- 2020-10-28 CN CN202011168398.1A patent/CN114511571A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116503390A (en) * | 2023-06-25 | 2023-07-28 | 深圳市智宇精密五金塑胶有限公司 | Hardware part defect detection method based on computer vision |
| CN116503390B (en) * | 2023-06-25 | 2023-09-22 | 深圳市智宇精密五金塑胶有限公司 | Hardware part defect detection method based on computer vision |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Ji et al. | A novel simplification method for 3D geometric point cloud based on the importance of point | |
| CN111627065B (en) | Visual positioning method and device and storage medium | |
| JP2015506026A (en) | Image classification | |
| WO2022193335A1 (en) | Point cloud data processing method and apparatus, and computer device and storage medium | |
| CN108540988B (en) | Scene division method and device | |
| CN115661374B (en) | Rapid retrieval method based on space division and model voxelization | |
| Wang et al. | The isotropic organization of DEM structure and extraction of valley lines using hexagonal grid | |
| CN111310821A (en) | Multi-view feature fusion method, system, computer device and storage medium | |
| Sainju et al. | A hidden markov contour tree model for spatial structured prediction | |
| CN114241388A (en) | Video instance segmentation method and segmentation device based on space-time memory information | |
| CN115457492A (en) | Target detection method and device, computer equipment and storage medium | |
| CN112241676A (en) | Method for automatically identifying terrain sundries | |
| Xiao et al. | A point selection method in map generalization using graph convolutional network model | |
| CN113436223B (en) | Point cloud data segmentation method and device, computer equipment and storage medium | |
| CN115170859A (en) | Point cloud shape analysis method based on space geometric perception convolutional neural network | |
| Xu et al. | Robust resistance to noise and outliers: Screened Poisson Surface Reconstruction using adaptive kernel density estimation | |
| CN114511571A (en) | Point cloud data semantic segmentation method and system and related components | |
| CN117237660A (en) | Point cloud data processing and segmentation method based on deep learning feature aggregation | |
| Fellegara et al. | Terrain trees: a framework for representing, analyzing and visualizing triangulated terrains | |
| CN113903016B (en) | Bifurcation point detection method, bifurcation point detection device, computer equipment and storage medium | |
| CN116246119A (en) | 3D target detection method, electronic device and storage medium | |
| Yaghouti et al. | Determining optimal value of the shape parameter $ c $ in RBF for unequal distances topographical points by Cross-Validation algorithm | |
| JP6725310B2 (en) | Image processing device and program | |
| CN113658338A (en) | Point cloud tree monomer segmentation method and device, electronic equipment and storage medium | |
| JP7310932B2 (en) | 3D point group identification device, learning device, 3D point group identification method, learning method, and program |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| WD01 | Invention patent application deemed withdrawn after publication | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20220517 |