CN109166128B - Non-equivalent three-dimensional point cloud segmentation method - Google Patents
Non-equivalent three-dimensional point cloud segmentation method Download PDFInfo
- Publication number
- CN109166128B CN109166128B CN201810859511.7A CN201810859511A CN109166128B CN 109166128 B CN109166128 B CN 109166128B CN 201810859511 A CN201810859511 A CN 201810859511A CN 109166128 B CN109166128 B CN 109166128B
- Authority
- CN
- China
- Prior art keywords
- point cloud
- segmentation
- point
- model
- points
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10028—Range image; Depth image; 3D point clouds
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
Landscapes
- Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Image Analysis (AREA)
Abstract
The invention provides a non-equivalent three-dimensional point cloud segmentation method, which comprises the steps of firstly constructing an original point cloud data set and dividing a training set and a test set, secondly, preprocessing the training set data to change the number of cloud points of each point into the same specification, designing a convolutional neural network model, respectively calculating the distance between each point in the cloud points and the point which is not in the same segmentation area with the point, selecting the minimum distance value between the two points to construct a distance matrix, by designing a penalty function and acting on the distance matrix, the value of a point close to the boundary of the divided area is increased, and the value of the point far away from the boundary is reduced, and the penalized distance matrix is multiplied by the model loss function, the method has the advantages that loss calculation is optimized in the training process of the segmentation model, error feedback of the intersection point of the segmentation area is strengthened, point cloud segmentation precision is improved, and finally the segmentation performance of the designed model is evaluated by using the test set. The method can realize the quick segmentation of the nonequivalent three-dimensional point cloud and has high segmentation precision.
Description
Technical Field
The invention relates to the technical field of three-dimensional point cloud segmentation, in particular to a non-equivalent three-dimensional point cloud segmentation method.
Background
Point cloud segmentation is a process of dividing point clouds into a plurality of homogeneous regions, points in the same region have the same attribute, and the method mainly comprises two methods: the first method is to use a pure mathematical model and a geometric reasoning technology for segmentation, such as region growing or model fitting, and the like, and the method has simple principle and easy programming, but has low automation degree and inaccurate segmentation result; the second method is segmentation using machine learning techniques, which improves the point cloud segmentation accuracy, but takes a long time and may lead to excessive segmentation results.
In recent years, a deep learning algorithm represented by a convolutional neural network has a good effect in many fields, and a new method is provided for point cloud segmentation. At present, most of researches on point cloud segmentation by deep learning are developed aiming at the characteristics of changeability and disorder of the number of point clouds, and the nonequivalence problem of the point clouds in the actual segmentation process is not considered, namely the influence degree of points at the junctions of point cloud segmentation areas on segmentation results is greater than that of points far away from the junctions. Therefore, when the point cloud segmentation is performed by using the deep learning method, the influence of the non-equivalence of the point cloud on the segmentation precision needs to be solved.
Disclosure of Invention
The technical problem to be solved by the invention is as follows: when a deep learning method is adopted for point cloud segmentation, how to avoid the influence on segmentation precision caused by point cloud non-equivalence is avoided.
In order to solve the technical problem, the technical scheme of the invention is to provide a non-equivalent three-dimensional point cloud segmentation method, which is characterized by comprising the following steps:
step 1: determining a nonequivalent three-dimensional point cloud model needing to be segmented, and constructing a point cloud data set of the model;
step 2: dividing the point cloud data set into a training set and a test set, wherein the point cloud data in the training set is used for training a segmentation model, and the point cloud data in the test set is used for evaluating the segmentation performance of the segmentation model;
and step 3: randomly sampling point cloud data in the training set to change the number of points in each point cloud into the same specification;
and 4, step 4: designing a point cloud segmentation model based on a convolutional neural network;
and 5: constructing a distance matrix, specifically: traversing all points of each point cloud model in the training set, respectively calculating Euclidean distances between each point and points which are not in the same segmentation area, selecting the minimum distance value between the two points to construct a distance matrix, and finally enabling each point of each point cloud model in the training set to have a distance value in the distance matrix corresponding to the distance value;
step 6: designing a penalty function, and taking the distance value in the distance matrix as the input of the penalty function, so that the smaller the distance value is, the larger the output value is after the action of the penalty function, and the larger the distance value is, the smaller the output value is after the action of the penalty function;
and 7: the distance matrix after the punishment function is multiplied by the loss function of the segmentation model to realize the weighting processing of the loss function, so that the closer point to the junction of the segmentation region is, the greater the loss increase degree caused by inaccurate prediction is, and the model can optimize the point at the junction of the segmentation region in an emphatic way in the training process;
and 8: and training the weighted segmentation model by using the point cloud data in the training set, minimizing the error between the prediction output and the actual output of the model by using a small-batch random gradient descent method in the training process, further obtaining an optimal segmentation model, and evaluating the segmentation performance of the model by using the point cloud data in the test set.
Preferably, in the step 1, the non-equivalence of the three-dimensional point cloud is defined as: in the point cloud segmentation process, the segmentation accuracy is determined by the points at the junctions of the areas, and the farther the points are from the junctions of the areas, the lower the influence on the segmentation accuracy, so the distance between each point in the point cloud and the junctions of the segmented areas causes the inequivalence of the point cloud.
Preferably, in the step 3, the cloud models of the points are simplified by a random sampling method, so that the points are consistent, specifically according to the following method: firstly, determining the number N of points of each point cloud modeliDetermining the sample size N, N to be samplediN is positive integer, and the points in the point cloud models are sampled without being put back, so that the possibility of sampling each point is N/Ni。
Preferably, in step 4, the designed point cloud segmentation model based on the convolutional neural network extracts the point cloud features through 5 convolutional layers, then down-samples the point cloud through the pooling layer, and finally combines the features of each convolutional layer and the pooling layer, and outputs the segmentation result through 3 convolutional layers.
Preferably, in the step 5, when the euclidean distances between each point and the point not in the same partition area are calculated, whether each point is in the same partition area is judged according to the label corresponding to each point, and then the distance calculation is performed according to the sequence of the points in the point cloud data file, so that the point in the point cloud data file corresponds to the value in the distance matrix one to one.
Preferably, in the step 6, the penalty function expression is designed as
Wherein d isiThe distance value of each point in the distance matrix is obtained; c is a penalty factor which represents the degree of penalty for distance; miThe magnitude of the corresponding output value after the distance value in the distance matrix is acted by a penalty function;
when d isiWhen greater than C, output MiA real number greater than 0 and less than 1;
when d isiWhen equal to C, output MiIs 1;
when d isiWhen less than C, output MiA real number greater than 1;
and finally, determining the value of C by analyzing the distribution of each distance interval in the distance matrix and testing the segmentation effect of the point cloud model under each C value.
The invention adopts a deep learning method to carry out point cloud segmentation, solves the problems of low automation degree and long time consumption in the traditional point cloud method, considers the influence of point cloud non-equivalence on the segmentation result in the segmentation process, and improves the precision of point cloud segmentation.
Drawings
Fig. 1 is a flowchart of a non-equivalent three-dimensional point cloud segmentation method provided in this embodiment;
FIG. 2 is a point cloud segmentation model architecture diagram based on a convolutional neural network;
fig. 3 is a point cloud model for segmentation experiments, wherein (a) is an origin cloud object model, and (b) is the point cloud object model after segmentation according to the present invention.
Detailed Description
The invention will be further illustrated with reference to the following specific examples.
Fig. 1 is a schematic diagram of a non-equivalent three-dimensional point cloud segmentation method provided in this embodiment, where the non-equivalent three-dimensional point cloud segmentation method includes the following steps:
firstly, a point cloud model sample for segmentation needs to be obtained, a point cloud data set of the model is constructed, the point cloud data set comprises three-dimensional coordinates of each point in the point cloud and a corresponding label, and fig. 3(a) shows an object to be subjected to point cloud segmentation, the object comprises two segmentation areas, and the two segmentation areas are respectively positioned at the left side and the right side of the point cloud model.
And dividing the point cloud data set into a training set and a testing set, wherein the data in the training set is used for model training, the data in the testing set is used for evaluating the segmentation performance of the model, the ratio of the training set to the model in the testing set is 8: 2, and the point cloud model in the training set is preprocessed in a random sampling mode to change the number of the point cloud model points into the same specification.
Designing a convolutional neural network model shown in fig. 2 to perform point cloud segmentation, wherein the model firstly extracts point cloud features through 5 convolutional layers, then downsamples the point cloud through a pooling layer, and finally combines the features of each convolutional layer and the pooling layer and outputs segmentation results through 3 convolutional layers.
Traversing all the points of the point cloud models in the training set, respectively calculating Euclidean distances between each point and the points which are not in the same segmentation area, selecting the minimum distance value between the two points to construct a distance matrix, and finally enabling each point of each point cloud model in the training set to have a distance value in the distance matrix corresponding to the distance value.
And designing a penalty function by analyzing each distance distribution in the distance matrix, and applying the penalty function to the distance matrix, so that the value of a point close to the boundary of the divided area is further enlarged, and the value of a point far away from the boundary of the divided area is further reduced.
Designing penalty function
Wherein d isiAnd C is a penalty factor. When d isiWhen the value is larger than C, outputting a real number which is larger than 0 and smaller than 1; when d isiWhen the C is equal, 1 is output; when in usediAnd when the value is less than C, outputting a real number which is greater than 1. And C value is determined by analyzing the distribution of each distance interval in the distance matrix and testing the segmentation effect of the segmentation model under each value.
In the training process of the segmentation model, the distance matrix acted by the penalty function is multiplied by the loss function of the segmentation model to realize the weighting processing of the loss function, so that the closer points to the junction of the point cloud segmentation area are inaccurate in prediction, the greater the degree of loss increase is, a small batch random gradient descent method is used to minimize the error between the prediction output and the actual output of the model, the optimal segmentation model is further obtained, and the point cloud data in the test set is used for evaluating the segmentation performance of the model. Fig. 3(a) is an origin cloud object model, and fig. 3(B) is a point cloud model after segmentation according to the present invention, in which a represents a first segmented region, B represents a second segmented region, and C represents an undivided region.
It is not difficult to find that the method firstly reads and preprocesses the total data information of the model, secondly designs a convolution neural network model which can be used for point cloud segmentation, then calculates the Euclidean distance between each point in the point cloud and the point which is not in the same segmentation area, selects the minimum distance value between the two points to construct a distance matrix, and designs a penalty function to act on the distance matrix, so that the value of the point which is close to the boundary of the point cloud segmentation area is further amplified, the value of the point which is far from the boundary of the segmentation area is further reduced, and multiplies the distance matrix acted by the penalty function with a model loss function, so that the point of the boundary of the segmentation area is optimized in the training process of the segmentation model, thereby solving the influence of the nonequivalence of the point cloud on the segmentation precision.
The invention adopts a deep learning method to carry out point cloud segmentation, solves the problems of low automation degree and long time consumption in the traditional point cloud method, considers the influence of point cloud non-equivalence on the segmentation result in the segmentation process and improves the segmentation precision.
While the invention has been described with respect to a preferred embodiment, it will be understood by those skilled in the art that the foregoing and other changes, omissions and deviations in the form and detail thereof may be made without departing from the scope of this invention. Those skilled in the art can make various changes, modifications and equivalent arrangements, which are equivalent to the embodiments of the present invention, without departing from the spirit and scope of the present invention, and which may be made by utilizing the techniques disclosed above; meanwhile, any changes, modifications and variations of the above-described embodiments, which are equivalent to those of the technical spirit of the present invention, are within the scope of the technical solution of the present invention.
Claims (5)
1. A non-equivalent three-dimensional point cloud segmentation method is characterized by comprising the following steps:
step 1: determining a nonequivalent three-dimensional point cloud model needing to be segmented, and constructing a point cloud data set of the model;
step 2: dividing the point cloud data set into a training set and a test set, wherein the point cloud data in the training set is used for training a segmentation model, and the point cloud data in the test set is used for evaluating the segmentation performance of the segmentation model;
and step 3: randomly sampling point cloud data in the training set to change the number of points in each point cloud into the same specification;
and 4, step 4: designing a point cloud segmentation model based on a convolutional neural network;
and 5: constructing a distance matrix, specifically: traversing all points of each point cloud model in the training set, respectively calculating Euclidean distances between each point and points which are not in the same segmentation area, selecting the minimum distance value between the two points to construct a distance matrix, and finally enabling each point of each point cloud model in the training set to have a distance value in the distance matrix corresponding to the distance value;
step 6: designing a penalty function, and taking the distance value in the distance matrix as the input of the penalty function, so that the smaller the distance value is, the larger the output value is after the action of the penalty function, and the larger the distance value is, the smaller the output value is after the action of the penalty function;
and 7: the distance matrix after the punishment function is multiplied by the loss function of the segmentation model to realize the weighting processing of the loss function, so that the closer point to the junction of the segmentation region is, the greater the loss increase degree caused by inaccurate prediction is, and the model can optimize the point at the junction of the segmentation region in an emphatic way in the training process;
and 8: training the weighted segmentation model by using point cloud data in a training set, minimizing the error between model prediction output and actual output by using a small-batch random gradient descent method in the training process so as to obtain an optimal segmentation model, and evaluating the segmentation performance of the model by using the point cloud data in a test set;
in the step 3, the designed point cloud segmentation model based on the convolutional neural network firstly extracts the point cloud features through 5 convolutional layers, then downsamples the point cloud through the pooling layers, finally combines the features of each convolutional layer and the pooling layers, and outputs the segmentation result through 3 convolutional layers.
2. The method of claim 1, wherein the method comprises: in the step 1, the non-equivalence of the three-dimensional point cloud is defined as: in the point cloud segmentation process, the segmentation accuracy is determined by the points at the junctions of the areas, and the farther the points are from the junctions of the areas, the lower the influence on the segmentation accuracy, so the distance between each point in the point cloud and the junctions of the segmented areas causes the inequivalence of the point cloud.
3. The method of claim 1, wherein the method comprises: in the step 3, the cloud models of the points are simplified by adopting a random sampling method, so that the points are consistent, and the method specifically comprises the following steps: firstly, determining the point number Ni of each point cloud model, determining the volume n of samples to be extracted, wherein Ni and n are positive integers, and sampling the points in the point cloud models without replacing the points, so that the possibility of sampling each point is n/Ni.
4. The method of claim 1, wherein the method comprises: in the step 5, when the euclidean distances between each point and the points which are not in the same partition area are calculated, whether each point is in the same partition area is judged according to the corresponding label of each point, and then the distance calculation is performed according to the sequence of the points in the point cloud data file, so that the point in the point cloud data file corresponds to the value in the distance matrix one by one.
5. The method of claim 1, wherein the method comprises: in the step 6, the designed penalty function expression is Mi ═ 1/edi-C, where di is the size of the distance value in the distance matrix corresponding to each point; c is a penalty factor which represents the degree of penalty for distance; mi is the size of the corresponding output value of the distance value in the distance matrix after the action of a penalty function;
when di is larger than C, the output Mi is a real number which is larger than 0 and smaller than 1;
when di is equal to C, the output Mi is 1;
when di is smaller than C, outputting Mi as a real number larger than 1;
and finally, determining the value of C by analyzing the distribution of each distance interval in the distance matrix and testing the segmentation effect of the point cloud model under each C value.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810859511.7A CN109166128B (en) | 2018-07-31 | 2018-07-31 | Non-equivalent three-dimensional point cloud segmentation method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810859511.7A CN109166128B (en) | 2018-07-31 | 2018-07-31 | Non-equivalent three-dimensional point cloud segmentation method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109166128A CN109166128A (en) | 2019-01-08 |
| CN109166128B true CN109166128B (en) | 2021-11-26 |
Family
ID=64898435
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810859511.7A Active CN109166128B (en) | 2018-07-31 | 2018-07-31 | Non-equivalent three-dimensional point cloud segmentation method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109166128B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111223101B (en) * | 2020-01-17 | 2023-08-11 | 湖南视比特机器人有限公司 | Point cloud processing method, point cloud processing system and storage medium |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104197897A (en) * | 2014-04-25 | 2014-12-10 | 厦门大学 | Urban road marker automatic sorting method based on vehicle-mounted laser scanning point cloud |
| US9710714B2 (en) * | 2015-08-03 | 2017-07-18 | Nokia Technologies Oy | Fusion of RGB images and LiDAR data for lane classification |
| CN107025642A (en) * | 2016-01-27 | 2017-08-08 | 百度在线网络技术（北京）有限公司 | Vehicle's contour detection method and device based on cloud data |
| CN108230329A (en) * | 2017-12-18 | 2018-06-29 | 孙颖 | Semantic segmentation method based on multiple dimensioned convolutional neural networks |
-
2018
- 2018-07-31 CN CN201810859511.7A patent/CN109166128B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104197897A (en) * | 2014-04-25 | 2014-12-10 | 厦门大学 | Urban road marker automatic sorting method based on vehicle-mounted laser scanning point cloud |
| US9710714B2 (en) * | 2015-08-03 | 2017-07-18 | Nokia Technologies Oy | Fusion of RGB images and LiDAR data for lane classification |
| CN107025642A (en) * | 2016-01-27 | 2017-08-08 | 百度在线网络技术（北京）有限公司 | Vehicle's contour detection method and device based on cloud data |
| CN108230329A (en) * | 2017-12-18 | 2018-06-29 | 孙颖 | Semantic segmentation method based on multiple dimensioned convolutional neural networks |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109166128A (en) | 2019-01-08 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110969250B (en) | Neural network training method and device | |
| CN107247989B (en) | Real-time computer vision processing method and device | |
| Bode | Neural networks for cost estimation: simulations and pilot application | |
| Singh et al. | PI-LSTM: Physics-infused long short-term memory network | |
| CN111860982A (en) | Wind power plant short-term wind power prediction method based on VMD-FCM-GRU | |
| CN104063876A (en) | Interactive image segmentation method | |
| Shahrokhi et al. | A surrogate assisted evolutionary optimization method with application to the transonic airfoil design | |
| Oh et al. | Quantum convolutional neural network for resource-efficient image classification: A quantum random access memory (QRAM) approach | |
| JP2018092349A (en) | Reduction condition determining method, reduction condition determining program, and reduction condition determining apparatus | |
| Li et al. | Graphfit: Learning multi-scale graph-convolutional representation for point cloud normal estimation | |
| CN112949944A (en) | Underground water level intelligent prediction method and system based on space-time characteristics | |
| CN113779385B (en) | Friend attention metric ordering method and system based on complex network graph embedding | |
| CN109166128B (en) | Non-equivalent three-dimensional point cloud segmentation method | |
| Chen et al. | Mahler measuring the genetic code of amoebae | |
| Sun et al. | An improved spatial point cloud simplification algorithm | |
| Díaz-Morales et al. | Deep learning combined with singular value decomposition to reconstruct databases in fluid dynamics | |
| CN116629099A (en) | Sheet metal part multi-objective optimization method based on mixed agent model | |
| CN115937516B (en) | Image semantic segmentation method and device, storage medium and terminal | |
| CN116975631A (en) | Model training, target detection method, electronic device and computer storage medium | |
| CN116826734A (en) | Photovoltaic power generation power prediction method and device based on multi-input model | |
| CN116977265A (en) | Training method and device for defect detection model, computer equipment and storage medium | |
| Selvan | On the effect of shape parameterization on aerofoil shape optimization | |
| CN113722951B (en) | Scatterer three-dimensional finite element grid optimization method based on neural network | |
| CN115310590A (en) | Graph structure learning method and device | |
| CN113822293A (en) | Model processing method, device and equipment for graph data and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |