CN113298952B - Incomplete point cloud classification method based on data expansion and similarity measurement - Google Patents
Incomplete point cloud classification method based on data expansion and similarity measurement Download PDFInfo
- Publication number
- CN113298952B CN113298952B CN202110652559.2A CN202110652559A CN113298952B CN 113298952 B CN113298952 B CN 113298952B CN 202110652559 A CN202110652559 A CN 202110652559A CN 113298952 B CN113298952 B CN 113298952B
- Authority
- CN
- China
- Prior art keywords
- point cloud
- incomplete
- loss
- complete
- network
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
- G06T17/20—Finite element generation, e.g. wire-frame surface description, tesselation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/22—Matching criteria, e.g. proximity measures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Computation (AREA)
- General Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Evolutionary Biology (AREA)
- Software Systems (AREA)
- Biophysics (AREA)
- Mathematical Physics (AREA)
- Computing Systems (AREA)
- Molecular Biology (AREA)
- General Health & Medical Sciences (AREA)
- Computational Linguistics (AREA)
- Biomedical Technology (AREA)
- Health & Medical Sciences (AREA)
- Computer Graphics (AREA)
- Geometry (AREA)
- Image Analysis (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention belongs to the field of three-dimensional computer vision, and particularly relates to a defect point cloud classification network based on data expansion and similarity measurement. The invention aims to solve the problem of how to expand a complete point cloud classification network to incomplete point clouds, thereby providing an excellent solution for the classification problem of the incomplete point clouds. Therefore, the invention provides a new classification network IPC-Net aiming at incomplete point cloud on the basis of the traditional point-based classification network, can solve the problems of low classification precision and poor network robustness of the incomplete point cloud based on data expansion and similarity measurement, and in addition, utilizes an auxiliary loss function to combine with an attention mechanism to help solve the related problems.
Description
Technical Field
The invention belongs to the field of three-dimensional computer vision, and particularly relates to a incomplete point cloud classification method based on data expansion and similarity measurement.
Background
The point cloud is different from a regular grid structure of a two-dimensional image, but is a set of unordered 3D points, so the point cloud classification has certain challenges. In recent years, many deep learning methods applied to three-dimensional point clouds have appeared. The method for accurately classifying the point cloud generally depends on the richness and the availability of data, and compared with a mass data set MS COCO and an ImageNet data set which are commonly used in the field of two-dimensional images, the scale of a three-dimensional point cloud data set is usually much smaller, and the richness of a sample is insufficient. The common point cloud classification dataset has a complete structure and clear definition, and the point cloud data sources in practical application mainly have two modes: firstly, scanning to obtain discrete point cloud data by using a laser radar in a mode of measuring the distance between a sensor and a measured object; and secondly, acquiring point cloud data by using an RGB-D camera in a mode of measuring the distance between the camera and an object. No matter which way to collect data, because shelter from between the object in the scene, light reflection, the restriction of sensor resolution ratio and visual angle can cause the loss of geometry and semantic information, lead to the cloud of points to appear the incomplete of different degrees. In the identification and classification, the regional loss can cause the point cloud to lose partial characteristics, and the accuracy of shape classification is influenced, for example, in a VKITT data set, some houses cut off by the waist are easily judged as trucks by network errors.
Disclosure of Invention
The invention aims to provide a incomplete point cloud-oriented classification method based on data expansion and similarity measurement, which is used for expanding a complete point cloud classification network to incomplete point clouds so as to provide an excellent solution for the classification problem of the incomplete point clouds.
The purpose of the invention is realized by the following steps:
a incomplete point cloud classification method based on data expansion and similarity measurement comprises the following steps:
s1: preprocessing the original point cloud;
firstly, normalizing non-normalized data to keep the data range in [ -1,1] all the time so as to facilitate subsequent tasks;
s2: a data capacity expansion mode facing to three-dimensional point cloud, namely random area erasing, is provided;
the step can give consideration to the study of semantic information of complete point clouds and incomplete point clouds, randomly removes point clouds in partial areas aiming at the complete point clouds, and strips off local information; the process simulates the problems of shielding, missing and the like in the actual point cloud acquisition process, increases the complexity and richness of data, and enhances the robustness and generalization performance of the network; the method comprises the following specific steps:
s21: in the training process, original point cloud data is input into a network and then copied to obtain a 'copy' which is the same as the original point cloud data and is used as input data of a random area erasing module;
s22: the random area erasing method randomly selects a point in a 'duplicate' point set as an erased area center;
s23: the random area erasing module is provided with an erasing range, and any size in the range is randomly selected as an erasing radius to form a spherical area for erasing;
s24: the point cloud data in the erasing sphere is cleared, meanwhile, the module needs to judge whether the number of points reserved by the erased point cloud meets the requirement lower limit of a subsequent classification network, the input number of the classification network is 1024, and therefore when the number of remaining points after erasing is less than 1024, the network gives up performing random area erasing on the point cloud data;
s3: designing a point similarity measurement module, and introducing an attention mechanism to complete the design of an auxiliary loss function;
the similarity measurement module in the step takes the complete point cloud and the incomplete point cloud output by S2 as input, sets evaluation indexes and judges the similarity between the complete point cloud and the incomplete point cloud; applying an attention mechanism to the complete point cloud to obtain weight values between different points, wherein the weight values are used as parameters to participate in constructing similarity measurement loss; introducing a similarity loss function into a total loss function in an auxiliary loss mode, introducing the similarity loss function in the training process of the network model, and removing the similarity loss function in the testing link of the network; the method comprises the following specific steps:
s31: the complete point cloud and the incomplete point cloud with different data volumes are subjected to down sampling together, the down sampling method is the sampling of the farthest point,
1024 sampling points are reserved for the complete point cloud and the incomplete point cloud which are sampled through the farthest point;
s32: applying an attention mechanism to the complete point cloud after down-sampling to obtain the weight among all points, wherein the formula for obtaining the attention is as follows:
A＝g(Qi,W)
wherein Qi＝{qjJ is more than or equal to 1 and less than or equal to i represents complete point cloud, and W is learnable weight of the shared multilayer perceptron;
s33: evaluating the relation between incomplete point cloud and complete point cloud by using evaluation indexes in point cloud completion, wherein the method mainly comprises the following steps:
s331: evaluating the relation between the complete point cloud Q and the incomplete point cloud P by adopting a Chamfering Distance (CD) in point cloud completion, wherein the CD represents the average distance from each point in one point cloud to the nearest neighbor point in the other point cloud, so that the CD loss is formed, and the specific formula is as follows:
s332: the method adopts the Earth Moving Distance (EMD) in point cloud completion to evaluate the relationship between the complete point cloud Q and the incomplete point cloud P, wherein the EMD distance represents the minimum consumption of one point cloud under the optimal planning path of moving into the other point cloud, and the EMD loss is formed by the following specific formula:
where φ is a bijective that minimizes the average distance between corresponding points, and the sum must be the same size;
s333: the CD loss and the EMD loss are combined, the CD calculation efficiency is high, the time complexity is low, the EMD has better discrimination on the individual characteristics and the local details of a single point cloud, the CD calculation efficiency and the EMD are combined to jointly construct similarity measurement loss, and the loss function formula is as follows:
s34: passing the complete point cloud throughThe weight obtained by the attention module is used as a parameter, and the distance d between the complete point cloud and the incomplete point cloud is introducedijIn the calculation of (c), dijIs the core part of the similarity measure loss, and the formula is as follows:
dij＝A·||Q-P||
s4: classifying the complete point cloud and the incomplete point cloud;
in the step, the expanded data obtained in S2 is used as the input of a classification network, and the classification network part continues the encoder-decoder classification structure of PointNet + +; the multi-level feature extraction structure captures local and global features, and classification loss and similarity measurement loss are combined in a specific ratio to form an overall loss function of the network; the method comprises the following specific steps:
s41: inputting the complete point cloud and the incomplete point cloud after expansion into a classification network together to realize incomplete point cloud classification; the method specifically comprises the following steps:
s411: the method comprises the following steps of utilizing an SA layer in a coding part of PointNet + + to realize feature extraction, wherein the SA layer specifically comprises three modules of Sampling, Grouping and PointNet, and the method comprises the following specific steps:
s4111: a Sampling module in an SA layer firstly carries out down-Sampling on the point cloud of an input module, the Sampling mode is Sampling of the farthest point, and the number of the Sampling points is 1024;
s4112: the Grouping module in the SA layer selects a proper number of points in a fixed area to form a local point set by using the selected points as a mass center through a neighborhood algorithm, and the number of the selected points in each neighborhood is 32;
s4113: the PointNet part of the SA layer uses maximum pooling (max-pooling) as a symmetric function to realize the arrangement invariance of point clouds, and an MLP module is used for extracting local features, and the formula of the process is as follows:
f({x1,...,xn})≈g(h(x1),...,h(xn))
in this implementation, h represents the MLP, g represents the max-firing function, which represents the maximum in the feature and satisfies the permutation invariance, which is given by the formula:
max(x1,x2,x3,...)≡max(xα1,xα2,xα3,...),xi∈RD
s412: repeating the process of S411 twice, and gradually obtaining local features and global features of different levels;
s413: the decoding part takes the global features obtained by the down-sampling of the coding part as input and introduces the global features into a fully-connected network, and completes classification through Softmax;
s42: combining the loss function obtained by the classification network with the similarity measurement loss obtained by the S3 to form the loss function of IPC-Net, and classifying the loss function L of the network partCLSIs NLLLoss; the similarity measurement loss is obtained in S333, and is applied to the training process of the model in the form of an auxiliary function, and is removed in the testing link; the over-parameter alpha is used for balancing two losses, the alpha value is determined to be 0.05 through experiments, and the loss function of IPC-Net is as follows:
LCON＝0.95LCLS+0.05LSM。
compared with the prior art, the invention has the beneficial effects that:
1. the invention provides a classification network IPC-Net of incomplete point cloud. The method can expand the complete point cloud classification network to the incomplete point cloud, thereby ensuring the robustness of point cloud classification with different integrality.
2. The invention designs a data expansion method aiming at 3D point cloud, which emphasizes data expansion on the semantic level and has practical significance. We propose a point similarity measurement module in which attention mechanisms are introduced and refined to design the secondary penalty functions.
3. The invention is independent of a specific data set, has universality on different input data sets, and achieves excellent classification effect on a ModelNet40 data set.
Drawings
FIG. 1 is a flow chart of the incomplete point cloud classification network IPC-Net based on data expansion and similarity measurement proposed by the present invention;
FIG. 2 is a block diagram of the modules of the IPC-Net network;
FIG. 3 is a flow chart of random area erase;
FIG. 4 is a visualization of a ModelNet40 dataset randomly erased;
FIG. 5 is a homemade data set visualization chart for judging IPC-Net generalization performance.
Detailed Description
The following description of the embodiments of the present invention will be made with reference to the accompanying drawings.
The point cloud mentioned in the application is a massive point set which expresses target space distribution and target surface characteristics under the same space reference system, and after the space coordinate of each sampling point on the surface of an object is obtained, the point set is obtained.
The point cloud classification mentioned in the present application is to assign a semantic tag to each point. The point cloud classification is to classify the point cloud into different point cloud sets, and the same point cloud set has similar or same attributes.
The point cloud completion method is widely applied to the field of detection and identification of three-dimensional objects and is used for repairing missing point clouds, and complete point clouds are estimated from the missing point clouds, so that higher-quality point clouds are obtained. The incomplete point cloud oriented classification network is a further popularization of the point cloud classification network, and the common analysis method of the incomplete point cloud is mainly a point cloud completion network. Point cloud completion is a prior task for realizing incomplete point cloud classification, and shape classification is carried out after complete point cloud data are obtained. However, in the point cloud classification task, the point cloud completion operation is redundant. Because the network can achieve classification by learning the semantic information of the incomplete point cloud without generating local details with unobtrusive features. In addition, adding a point cloud before classifying the network to complete the network will also increase the complexity of the model. Meanwhile, in the research of deep learning and analysis of the 3D point cloud, the training data plays a very important role. However, the actual training data usually has the problems of small volume, low complexity and poor richness. Therefore, data expansion is a common training method. Data expansion is a form of display of regularization, which aims to expand the number and richness of training sets using data only by manual operations. Due to the operation conversion in the process, the semantic information which can be learned by the network is maximized by data expansion, and the semantic information becomes a key part for preventing the convolutional neural network model from being over-fitted.
The invention mainly comprises a data expansion module and a point similarity measurement module. The data expansion part is mainly realized by random area erasure. The random area erasing process simulates the shielding of point cloud in practical application, thereby improving the generalization performance of the network. The point similarity metric module combines an attention mechanism with the similarity metric to guide learning of incomplete point features to the original point features. Meanwhile, the invention obtains the weight between different points of the original point cloud by using attention, so that the network pays more attention to the learning characteristic with high information content. The invention also uses the layered classification structure of PointNet + + as a reference, demonstrates IPC-Net and demonstrates the effectiveness of the IPC-Net on geometric tasks.
The invention is further described in detail in the following with reference to the accompanying drawings.
FIG. 1 is a flow chart of the incomplete point cloud classification network IPC-Net based on data expansion and similarity measurement, and FIG. 2 is a specific structure diagram of each module of the IPC-Net network. The method specifically comprises the following steps:
s1, preprocessing input data;
s2, performing data expansion on the original point cloud;
s3, evaluating the difference between the complete point cloud and the incomplete point cloud by using a similarity measurement module, and outputting a similarity measurement loss;
and S4, the incomplete point cloud classification is realized by utilizing a classification network.
The incomplete point cloud classification network based on data expansion and point cloud similarity measurement is described in more detail below with reference to the accompanying drawings.
Step S1, normalizing the original point cloud, normalizing the point cloud data of different sizes to the range of [ -1,1], and the visualization result of the original point cloud is shown in fig. 5.
In step S2, data expansion is performed on the original point cloud with a complete structure, the data expansion process is mainly implemented by random area erasure, and the flow of random area erasure is shown in fig. 3, and the specific steps are as follows:
s2.1, in the training process, after the original point cloud data is input into the network, copying the original point cloud data to obtain a 'copy' which is the same as the original point cloud data and used as input data of a random area erasing module.
S2.2, the random area erasing method randomly selects a point in the copy point set as the center of the erased area.
S2.3, the random area erasing module sets an erasing range, and randomly selects any size in the range as an erasing radius to form a spherical area for erasing.
And S2.4, removing the point cloud data in the erasing sphere, and meanwhile, judging whether the number of points reserved by the erased point cloud meets the requirement lower limit of a subsequent classification network by the module, wherein the number of input points of the classification network is 1024, so that when the number of remaining points after erasing is less than 1024, the network gives up carrying out random area erasing on the point cloud data.
The data capacity expansion module of the invention ensures the maximum random principle, firstly, the position of the central point of the erasing area is random, secondly, the radius of the erasing area is also random, and finally, whether the erasing operation can be executed for a point cloud data is also random due to the phenomenon that the possible points are highly concentrated in the erasing area, and the complete process of random erasing is visually displayed in figure 4.
Step S3, the input point cloud sampled by the farthest point is introduced into a similarity measurement module, the complete point cloud and the incomplete point cloud are linked by using an evaluation index, and the point cloud similarity measurement loss is obtained by evaluating the difference between the complete point cloud and the incomplete point cloud, and the process specifically comprises the following steps:
and S3.1, carrying out downsampling on the complete point cloud and the incomplete point cloud with different data volumes together, wherein the downsampling method is sampling at a farthest point, and 1024 sampling points are reserved for the complete point cloud and the incomplete point cloud which are sampled at the farthest point.
S3.2, applying an attention mechanism to the complete point cloud after down-sampling so as to obtain the weight among all points, wherein the formula for obtaining the attention is as follows:
A＝g(Qi,W)
wherein Qi＝{qjJ is more than or equal to 1 and less than or equal to i represents complete point cloud, and W is learnable weight of the shared multilayer perceptron.
S3.3, evaluating the relation between incomplete point cloud and complete point cloud by using evaluation indexes in point cloud completion, wherein the evaluation indexes mainly comprise the following steps:
s3.3.1, evaluating the relationship between the complete point cloud Q and the incomplete point cloud P by using the Chamfering Distance (CD) in the point cloud completion, wherein the CD represents the average distance from each point in one point cloud to the nearest neighbor point in the other point cloud, thereby forming the CD loss, and the specific formula is as follows:
s3.3.2, using the Earth Moving Distance (EMD) in the point cloud completion to evaluate the relationship between the complete point cloud Q and the incomplete point cloud P, the EMD distance represents the minimum consumption of the optimal planning path of one point cloud moving into another point cloud, thus forming the EMD loss, the specific formula is:
where phi is a bijection that minimizes the average distance between corresponding points, and the sum must be the same size.
S3.3.3, combining CD loss and EMD loss, CD calculation efficiency is high, time complexity is low, EMD has better discrimination to individual characteristics and local details of single point cloud, combining the two to construct similarity measurement loss, the loss function formula is:
s3.4, taking the weight of the complete point cloud obtained by the attention module as a parameter, and introducing the complete point cloud and the incomplete point cloudA distance d betweenijIn the calculation of (c), dijIs the core part of the similarity measure loss, and the formula is as follows:
dij＝A·||Q-P||
and S4.1, inputting the expanded complete point cloud and the incomplete point cloud into a classification network together to realize incomplete point cloud classification. The method comprises the following specific steps:
s4.1.1, utilizing an SA layer in a coding part of PointNet + + to realize feature extraction, wherein the SA layer specifically comprises three modules of Sampling, Grouping and PointNet, and the specific steps are as follows:
s4.1.1.1, the Sampling module in the SA layer firstly down-samples the point cloud of the input module in the way of Sampling the farthest point, and the number of the Sampling points is 1024 points.
S4.1.1.2, the Grouping module in the SA layer selects a proper number of points in a fixed area to form a local point set by using the selected points as the mass center and a neighborhood algorithm, and the number of points selected in each neighborhood is 32.
S4.1.1.3, the PointNet part of the SA layer uses the maximum pooling (max-posing) as a symmetric function to realize the arrangement invariance of the point cloud, and uses the MLP module to extract the local features, and the formula of the process is as follows:
f({x1,...,xn})≈g(h(x1),...,h(xn))
in this process, h represents MLP, g represents max-firing function, which represents the maximum in the feature and satisfies the permutation invariance, and the formula is:
max(x1,x2,x3,...)≡max(xα1,xα2,xα3,...),xi∈RD
s4.1.2, repeating the process of S411 twice, and gradually obtaining local features and global features of different levels.
S4.1.3, the decoding part takes the global feature obtained by the down sampling of the coding part as the input to the full-connection network and completes the classification by Softmax.
S4.2, combining the loss function obtained by the classification network with the similarity measurement loss obtained by the S3 to form IPLoss function of C-Net, loss function L of classification network partCLSIs NLLLoss. The similarity measure loss is obtained in S333, and is applied to the training process of the model in the form of an auxiliary function, and is removed in the testing process. The over parameter alpha is used for balancing two losses, the alpha value in the invention is determined to be 0.05 through experiments, and the loss function of IPC-Net is as follows:
LCON＝0.95LCLS+0.05LSM
by combining the incomplete point cloud classification method, when the input point cloud is complete point cloud or incomplete point cloud with different defect degrees, the ICP-Net network provided by the invention can realize better classification effect, and the network has stronger robustness.
Although the present invention has been described with reference to the preferred embodiments, it is not intended to be limited thereto. Those skilled in the art can make various changes and modifications without departing from the spirit and scope of the invention. Therefore, the protection scope of the present invention should be determined by the claims.
Claims (1)
1. A incomplete point cloud classification method based on data expansion and similarity measurement is characterized by comprising the following steps: the method comprises the following steps:
s1: preprocessing the original point cloud;
firstly, normalizing non-normalized data to keep the data range in [ -1,1] all the time so as to facilitate subsequent tasks;
s2: a data capacity expansion mode facing to three-dimensional point cloud, namely random area erasing, is provided;
the step can give consideration to learning the semantic information of complete point cloud and incomplete point cloud, randomly remove the point cloud in partial area aiming at the complete point cloud, and strip off the local information; the process simulates the problems of shielding and missing in the actual point cloud acquisition process, increases the complexity and richness of data, and enhances the robustness and generalization performance of the network; the method comprises the following specific steps:
s21: in the training process, original point cloud data is input into a network and then copied to obtain a 'copy' which is the same as the original point cloud data and is used as input data of a random area erasing module;
s22: the random area erasing method randomly selects a point in a 'duplicate' point set as an erased area center;
s23: the random area erasing module is provided with an erasing range, and any size in the range is randomly selected as an erasing radius to form a spherical area for erasing;
s24: the point cloud data in the erasing sphere is cleared, meanwhile, the module needs to judge whether the number of points reserved by the erased point cloud meets the requirement lower limit of a subsequent classification network, the input number of the classification network is 1024, and therefore when the number of remaining points after erasing is less than 1024, the network gives up performing random area erasing on the point cloud data;
s3: designing a point similarity measurement module, and introducing an attention mechanism to complete the design of an auxiliary loss function;
the similarity measurement module in the step takes the complete point cloud and the incomplete point cloud output by S2 as input, sets evaluation indexes and judges the similarity between the complete point cloud and the incomplete point cloud; applying an attention mechanism to the complete point cloud to obtain weight values between different points, wherein the weight values are used as parameters to participate in constructing similarity measurement loss; introducing a similarity loss function into a total loss function in an auxiliary loss mode, introducing the similarity loss function in the training process of a network model, and removing the similarity loss function in the testing link of the network; the method comprises the following specific steps:
s31: carrying out down-sampling on complete point clouds and incomplete point clouds with different data volumes together, wherein the down-sampling method is farthest point sampling, and 1024 sampling points are reserved for the complete point cloud and the incomplete point cloud which are subjected to the farthest point sampling;
s32: applying an attention mechanism to the complete point cloud after down-sampling to obtain the weight among all points, wherein the formula for obtaining the attention is as follows:
A＝g(Qi,W)
wherein Qi＝{qjJ is more than or equal to 1 and less than or equal to i represents complete point cloud, and W is learnable weight of the shared multilayer perceptron;
s33: evaluating the relation between incomplete point cloud and complete point cloud by using evaluation indexes in point cloud completion, wherein the method mainly comprises the following steps:
s331: the relation between complete point cloud Q and incomplete point cloud P is evaluated by adopting a chamfering distance CD in point cloud completion, the CD represents the average distance from each point in one point cloud to the nearest neighbor point in the other point cloud, and therefore CD loss is formed, and the specific formula is as follows:
s332: the method adopts the earth moving distance EMD in point cloud completion to evaluate the relationship between the complete point cloud Q and the incomplete point cloud P, and the EMD distance represents the minimum consumption of one point cloud under the optimal planning path of moving into the other point cloud, so that the EMD loss is formed, and the specific formula is as follows:
where φ is a bijective that minimizes the average distance between corresponding points, and the sum must be the same size;
s333: the CD loss and the EMD loss are combined, the CD calculation efficiency is high, the time complexity is low, the EMD has better discrimination on the individual characteristics and the local details of a single point cloud, the CD calculation efficiency and the EMD are combined to jointly construct similarity measurement loss, and the loss function formula is as follows:
s34: the weight of the complete point cloud obtained by the attention module is taken as a parameter, and the distance d between the complete point cloud and the incomplete point cloud is introducedijIn the calculation of (c), dijIs the core part of the similarity measure loss, and the formula is as follows:
dij＝A·||Q-P||
s4: classifying the complete point cloud and the incomplete point cloud;
in the step, the expanded data obtained in S2 is used as the input of a classification network, and the classification network part continues the encoder-decoder classification structure of PointNet + +; the multi-level feature extraction structure captures local and global features, and the classification loss and the similarity measurement loss are combined in a specific proportion to form an overall loss function of the network; the method comprises the following specific steps:
s41: inputting the complete point cloud and the incomplete point cloud after expansion into a classification network together to realize incomplete point cloud classification; the method specifically comprises the following steps:
s411: the method comprises the following steps of utilizing an SA layer in a coding part of PointNet + + to realize feature extraction, wherein the SA layer specifically comprises three modules of Sampling, Grouping and PointNet, and specifically comprises the following steps:
s4111: a Sampling module in an SA layer firstly carries out down-Sampling on the point cloud of an input module, wherein the Sampling mode is Sampling of a farthest point, and the number of the Sampling points is 1024;
s4112: the Grouping module in the SA layer selects a proper number of points in a fixed area to form a local point set by using the selected points as a mass center through a neighborhood algorithm, and the number of the selected points in each neighborhood is 32;
s4113: the PointNet part of the SA layer uses maximum pooling max-pooling as a symmetric function to realize the arrangement invariance of point clouds, an MLP module is used for extracting local features, and the formula of the process is as follows:
f({x1,...,xn})≈g(h(x1),...,h(xn))
in this process, h represents MLP, g represents max-firing function, which represents the maximum in the feature and satisfies the permutation invariance, and the formula is:
max(x1,x2,x3,...)≡max(xα1,xα2,xα3,...),xi∈RD
s412: repeating the process of S411 twice, and gradually obtaining local features and global features of different levels;
s413: the decoding part takes the global features obtained by the down-sampling of the coding part as input and introduces the global features into a fully-connected network, and completes classification through Softmax;
s42: combining the loss function obtained by the classification network with the similarity measurement loss obtained by the S3 to form the loss function of IPC-Net, and classifying the loss function L of the network partCLSIs NLLLoss; the similarity measurement loss is obtained in S333, and is applied to the training process of the model in the form of an auxiliary function, and is removed in the testing link; the over-parameter alpha is used for balancing two losses, the alpha value is determined to be 0.05 through experiments, and the loss function of IPC-Net is as follows:
LCON＝0.95LCLS+0.05LSM。
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110652559.2A CN113298952B (en) | 2021-06-11 | 2021-06-11 | Incomplete point cloud classification method based on data expansion and similarity measurement |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110652559.2A CN113298952B (en) | 2021-06-11 | 2021-06-11 | Incomplete point cloud classification method based on data expansion and similarity measurement |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113298952A CN113298952A (en) | 2021-08-24 |
| CN113298952B true CN113298952B (en) | 2022-07-15 |
Family
ID=77328010
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110652559.2A Active CN113298952B (en) | 2021-06-11 | 2021-06-11 | Incomplete point cloud classification method based on data expansion and similarity measurement |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113298952B (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112132958A (en) * | 2020-09-23 | 2020-12-25 | 哈尔滨工程大学 | Underwater environment three-dimensional reconstruction method based on binocular vision |
| CN112614071A (en) * | 2020-12-29 | 2021-04-06 | 清华大学 | Self-attention-based diverse point cloud completion method and device |
| CN112785526A (en) * | 2021-01-28 | 2021-05-11 | 南京大学 | Three-dimensional point cloud repairing method for graphic processing |
| CN112927359A (en) * | 2021-03-22 | 2021-06-08 | 南京大学 | Three-dimensional point cloud completion method based on deep learning and voxels |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10176589B2 (en) * | 2017-01-31 | 2019-01-08 | Mitsubishi Electric Research Labroatories, Inc. | Method and system for completing point clouds using planar segments |
| CN112488210A (en) * | 2020-12-02 | 2021-03-12 | 北京工业大学 | Three-dimensional point cloud automatic classification method based on graph convolution neural network |
-
2021
- 2021-06-11 CN CN202110652559.2A patent/CN113298952B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112132958A (en) * | 2020-09-23 | 2020-12-25 | 哈尔滨工程大学 | Underwater environment three-dimensional reconstruction method based on binocular vision |
| CN112614071A (en) * | 2020-12-29 | 2021-04-06 | 清华大学 | Self-attention-based diverse point cloud completion method and device |
| CN112785526A (en) * | 2021-01-28 | 2021-05-11 | 南京大学 | Three-dimensional point cloud repairing method for graphic processing |
| CN112927359A (en) * | 2021-03-22 | 2021-06-08 | 南京大学 | Three-dimensional point cloud completion method based on deep learning and voxels |
Non-Patent Citations (4)
| Title |
|---|
| MinkLoc3D: Point Cloud Based Large-Scale Place Recognition;Jacek Komorowski;《https://arxiv.org/abs/2011.04530》;20201109;第1-10页 * |
| 基于时空注意力机制的视频行人再识别方法研究;李明哲;《中国优秀博硕士学位论文全文数据库(硕士) 信息科技辑》;20210515(第05期);第I138-811页 * |
| 基于生成对抗网络的点云形状保结构补全;缪永伟 等;《中国科学:信息科学》;20200420;第50卷(第05期);第675–691页 * |
| 基于神经网络的点云残缺数据修补方法;刘俊 等;《机械工程师》;20070210(第02期);第32-34页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113298952A (en) | 2021-08-24 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110120097B (en) | Semantic modeling method for airborne point cloud of large scene | |
| CN110111345B (en) | Attention network-based 3D point cloud segmentation method | |
| CN101271469B (en) | Two-dimension image recognition based on three-dimensional model warehouse and object reconstruction method | |
| CN114821342B (en) | Remote sensing image road extraction method and system | |
| CN114092697B (en) | Building facade semantic segmentation method with attention fused with global and local depth features | |
| CN112347987A (en) | Multimode data fusion three-dimensional target detection method | |
| CN115512216A (en) | City functional area fine recognition method coupling block space-time characteristics and ensemble learning | |
| CN113838109B (en) | Low-coincidence point cloud registration method | |
| CN113345082A (en) | Characteristic pyramid multi-view three-dimensional reconstruction method and system | |
| Su et al. | DLA-Net: Learning dual local attention features for semantic segmentation of large-scale building facade point clouds | |
| CN113345106A (en) | Three-dimensional point cloud analysis method and system based on multi-scale multi-level converter | |
| CN116704137B (en) | Reverse modeling method for point cloud deep learning of offshore oil drilling platform | |
| CN113627440A (en) | Large-scale point cloud semantic segmentation method based on lightweight neural network | |
| CN115272599A (en) | Three-dimensional semantic map construction method oriented to city information model | |
| CN104463962B (en) | Three-dimensional scene reconstruction method based on GPS information video | |
| Sun et al. | Two-stage deep regression enhanced depth estimation from a single RGB image | |
| Lin et al. | An Unsupervised Transformer-based Multivariate Alteration Detection Approach for Change Detection in VHR Remote Sensing Images | |
| Dong et al. | RETRACTED ARTICLE: Research on North Gulf distributed big data submarine 3D terrain computing system based on remote sensing and multi-beam | |
| Chen et al. | Class-aware domain adaptation for coastal land cover mapping using optical remote sensing imagery | |
| Chen et al. | Point-attention Net: a graph attention convolution network for point cloudsegmentation | |
| CN113298952B (en) | Incomplete point cloud classification method based on data expansion and similarity measurement | |
| CN114565730A (en) | Method and device for processing map vector sample and computer readable storage medium | |
| Sayed et al. | Point clouds reduction model based on 3D feature extraction | |
| CN114155524A (en) | Single-stage 3D point cloud target detection method and device, computer equipment and medium | |
| CN116977572B (en) | Building elevation structure extraction method for multi-scale dynamic graph convolution |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |