CN113191973A - Cultural relic point cloud data denoising method based on unsupervised network framework - Google Patents

Cultural relic point cloud data denoising method based on unsupervised network framework Download PDF

Info

Publication number
CN113191973A
CN113191973A CN202110476219.9A CN202110476219A CN113191973A CN 113191973 A CN113191973 A CN 113191973A CN 202110476219 A CN202110476219 A CN 202110476219A CN 113191973 A CN113191973 A CN 113191973A
Authority
CN
China
Prior art keywords
point cloud
cloud data
network
noise
denoising
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202110476219.9A
Other languages
Chinese (zh)
Other versions
CN113191973B (en
Inventor
周明全
刘一萍
张海波
寇姣姣
周蓬勃
鱼跃华
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Northwestern University
Original Assignee
Northwestern University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Northwestern University filed Critical Northwestern University
Priority to CN202110476219.9A priority Critical patent/CN113191973B/en
Publication of CN113191973A publication Critical patent/CN113191973A/en
Application granted granted Critical
Publication of CN113191973B publication Critical patent/CN113191973B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/70—Denoising; Smoothing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/088—Non-supervised learning, e.g. competitive learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10028—Range image; Depth image; 3D point clouds

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Biophysics (AREA)
  • Evolutionary Computation (AREA)
  • Artificial Intelligence (AREA)
  • Biomedical Technology (AREA)
  • Health & Medical Sciences (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Image Processing (AREA)

Abstract

The invention discloses a cultural relic point cloud data denoising method based on an unsupervised network framework, which comprises the following steps of 1) constructing a two-layer network model based on an unsupervised network framework; 2) carrying out supervised training on the first layer network in the step 1); 3) carrying out unsupervised training on the second-layer network in the step 1); 4) taking the output of the first layer network trained in the step 2) as the input of the second layer network trained in the step 3), and constructing and obtaining a complete cultural relic point cloud data denoising network; 5) displaying a three-dimensional point cloud data image according to the cultural relic point cloud data denoising network obtained in the step 4); the network trains data under the condition of only acquiring real outlier labels, paired training data is not needed, outlier noise points in noise point cloud data can be better removed, a better denoising effect is achieved, the requirement of the network on the training data is less compared with other denoising algorithms, the operation efficiency is higher, and the denoising effect is better.

Description

Cultural relic point cloud data denoising method based on unsupervised network framework
Technical Field
The invention belongs to the field of three-dimensional point cloud denoising, and particularly relates to a cultural relic point cloud data denoising method based on an unsupervised network framework.
Background
The method is very key to the cultural relic protection work by acquiring the spatial point cloud data information which is as close to the real cultural relic object as possible. With the technological development in recent years, the three-dimensional scanning technology is becoming more mature, and three-dimensional laser scanning is increasingly applied to acquiring three-dimensional point cloud data, but limited by the sensor of the scanner, and affected by factors such as reflection of surrounding objects, illumination, environmental changes, factors considered, even inherent noise of the acquisition equipment, and the like, the initially acquired point cloud data always has inevitable partial noise. A large number of noise points have great obstacles to the registration, segmentation, reconstruction and other processing of the later-stage point cloud data, so that the initial point cloud must be denoised to obtain accurate point cloud. On the basis of keeping the geometric characteristics, removing noise points in the point cloud data as much as possible has also become an important research topic in the engineering field.
The purpose of denoising the point cloud data is to efficiently remove noise points as much as possible on the basis of keeping the geometric characteristics of the point cloud data, and meanwhile, keep the geometric texture, the edge and other detailed characteristics of the point cloud data, so as to obtain more accurate point cloud data.
The preservation of the geometrical characteristics of the details in the point cloud data denoising process is a difficult point, because the blurring of the details is caused by the loss of the high-order geometrical properties of the surface of the point cloud data model, the problem is caused because the large noise points cover the geometrical characteristics of the details of the point cloud data model on one hand, and the geometrical details are removed while denoising in the denoising process on the other hand.
The existing point cloud data denoising algorithm is divided into a traditional method and an algorithm combined with deep learning, and the traditional method is often low in efficiency and often unbalanced in large-scale point cloud data operation time and in point cloud data geometric feature retention and denoising; the algorithm combining deep learning mostly adopts supervised network training data and needs paired point cloud data, however, the paired point cloud data is not easy to obtain, and meanwhile, the problems of complex network structure, unobvious denoising effect and the like exist.
Disclosure of Invention
In order to overcome the defects of the prior art, the invention aims to provide the cultural relic point cloud data denoising method based on the unsupervised network framework, which has the advantages of less training data requirement, high operation efficiency and good denoising effect.
In order to achieve the purpose, the invention adopts the technical scheme that:
a cultural relic point cloud data denoising method based on an unsupervised network framework specifically comprises the following steps;
step 1: constructing a two-layer network model based on an unsupervised network architecture;
step 2: inputting noise cultural relic point cloud data with abnormal values and containing outlier noise points into the first layer network constructed in the step 1, constraining the output of the first layer network by using the actual outlier noise point labels of the noise cultural relic point cloud data, continuously updating the parameters of neurons in the first layer network by adopting supervised repeated iterative training, and terminating the training when the loss function of the first layer network reaches a set threshold value;
and step 3: after the noise cultural relic point cloud data is input into the second-layer network constructed in the step 1, the output of the second-layer network is restrained by using a set loss function, the parameters of neurons in the second-layer network are continuously updated by adopting unsupervised repeated iterative training, and the training is terminated when the loss function reaches a set threshold or the maximum iterative times;
and 4, step 4: taking the output of the first layer network trained in the step 2 as the input of the second layer network trained in the step 3, and constructing and obtaining a complete cultural relic point cloud data denoising network;
and 5: and (4) denoising the cultural relic point cloud data according to the cultural relic point cloud data denoising network obtained in the step (4).
Further, the first layer network is constructed by modifying a network layer and the output size on the basis of a pointenet network architecture;
the second layer network is obtained by modifying network input to construct on the basis of the totaldenosing network architecture.
Further, the construction and training process of the first layer network is as follows:
inputting noise point cloud data, representing the noise point cloud data as a point set, firstly obtaining local features, processing feature dimensionality by a layer of mlp, then performing maxporoling to obtain global features, fusing the global features and the local features, obtaining a point set by two layers of mlp, obtaining estimated outliers by the point set through a function g, and finally outputting the point cloud data of the noise point cloud data without the estimated outliers;
in the first layer network, a nonlinear function g is obtained through network training modeling, and is used for obtaining outlier noise points from input noise point cloud data:
Figure BDA0003047173670000031
p' in the formula (1) is input noise point cloud data, Pi' is the ith point in the noise point cloud data,
Figure BDA0003047173670000032
the point is the estimated probability value of the outlier noise point, and after all outliers in the input noise point cloud data are obtained, the input noise point cloud data are removed from the outlier set to obtain the point cloud to be further processed after the outliers are removed
Figure BDA0003047173670000033
Figure BDA0003047173670000034
At the moment, the real outlier label is adopted to constrain the estimation outlier set, and an L1 distance is used as a loss function between the real outlier label and the estimation outlier; the expression of the loss function is:
Figure BDA0003047173670000035
in the formula (2)
Figure BDA0003047173670000036
For de-noising the point cloud data, PiIn order to be a clean point cloud,
Figure BDA0003047173670000037
to obtain estimated outliers, QiAre true outlier tags.
Further, the construction and training process of the second layer network is as follows:
after point cloud data to be processed is input, the point cloud data is processed through a second network layer, the output is the denoised point cloud data, a spatial neighborhood in the point cloud data is converted into a potential code defined on an encoder, then the potential code is sampled to a decoder, and two-stage encoding is executed; introducing a spatial prior term to guide the observation to converge to the only closest one of a plurality of modes on the manifold;
the prior term q (z | y) to calculate the likelihood that the observed point y may be a clean point z; the calculation formula is expressed as: q (z | y) ═ p (z | S) × k (z-y) (3);
Figure BDA0003047173670000038
in formula (3), S is the surface of the clean point cloud data, σ is the bandwidth of k, W is diag (W), W is a diagonal weight matrix, W is 1/α r, r is 5% of the model diameter, and α is a scale factor;
Figure BDA0003047173670000039
when the formula (5) is optimized, the observed point cloud data can be converged; wherein f is a parameter ofΘIs obtained by network training, l is fΘThe loss function employed between (y) and q is the annealing loss function L0。
Compared with the prior art, the invention has the beneficial effects that:
1) the method constructs a first layer of network which is mainly used for removing outlier noise points in noise point cloud data, adopts supervised iterative training and only needs to use loss1 as a loss function between the cultural relic point cloud data containing paired outlier noise points and labels of real outlier noise points to restrain the output of the first layer of network, and has less requirements on training data; the second network only needs to input noise point cloud data and does not need paired clean cultural relic point cloud data by adopting unsupervised training, only needs to use a loss function to restrict the output of the second network, has less requirements on training data and has higher operation efficiency;
then, taking the output of the first layer network as input, and carrying out denoising treatment again to finally obtain point cloud data close to clean point cloud data; the constructed cultural relic point cloud data denoising network trains data under the condition of only acquiring a real outlier label, even does not need paired training data, can better remove outlier noise points in the noise point cloud data, achieves a better denoising effect, has less requirements on the training data compared with other denoising algorithms, and has higher operation efficiency and better denoising effect.
2) The method can achieve the effect of accurate classification on similar models; the traditional method has the defects of describing the model, and the method considers the global shape information and the local characteristic information of the three-dimensional model and has the effect of integrating the global information; meanwhile, the method uses the mean value of the Fourier coefficient difference as a similarity measurement criterion, can accurately convert the similarity of the two models into a numerical value, and judges a clustering result through a threshold value.
Drawings
FIG. 1 is a technical route diagram of a point cloud data denoising network in the present invention;
FIG. 2 is a schematic diagram of a first layer network architecture of the present invention;
FIG. 3 is a schematic diagram of a second tier network architecture of the present invention;
FIG. 4 is a schematic structural diagram of the whole point cloud data denoising network according to the present invention;
FIG. 5(a) is a clean point cloud data map of the publicly available point cloud data model bunny in the Stanford university public dataset of the present invention;
FIG. 5(b) is a clean point cloud data map of a publicly available point cloud data model dragon in the Stanford university public data set of the present invention;
FIG. 5(c) is a clean point cloud data map of a publicly available point cloud data model drill from the Stanford university public data set of the present invention;
FIG. 6(a) is a data diagram of a point cloud to be denoised according to the present invention, wherein the data diagram of the point cloud to be denoised is obtained by adding Gaussian noise with a standard deviation of 0.03% to a clean point cloud in FIG. 5 (a);
FIG. 6(b) is a data diagram of the point cloud to be denoised in FIG. 5(b) in which Gaussian noise with a standard deviation of 0.05% is added to the clean point cloud;
FIG. 6(c) is a data diagram of the point cloud to be denoised in FIG. 5(c) with a standard deviation of 0.07% Gaussian noise added to the clean point cloud;
FIG. 7(a) is a comparison graph of the denoising result of FIG. 6(a) according to the present invention using reference [4 ];
FIG. 7(b) is a comparison graph of the denoising result of FIG. 6(b) according to the present invention using reference [4 ];
FIG. 7(c) is a comparison graph of the denoising results of FIG. 6(c) using reference [4] in the present invention;
FIG. 8(a) is a comparison graph of the denoising result of FIG. 6(a) according to the present invention using reference [5 ];
FIG. 8(b) is a comparison graph of the denoising result of FIG. 6(b) according to the present invention using reference [5 ];
FIG. 8(c) is a comparison graph of the denoising results of FIG. 6(c) using reference [5] in the present invention;
FIG. 9(a) is a comparison graph of the denoising result of FIG. 6(a) according to the present invention using reference [6 ];
FIG. 9(b) is a comparison graph of the denoising result of FIG. 6(b) according to the present invention using reference [6 ];
FIG. 9(c) is a comparison graph of the denoising results of FIG. 6(c) using reference [6] in the present invention;
FIG. 10(a) is a comparison graph of the denoising results of FIG. 6(a) using reference [7] in the present invention;
FIG. 10(b) is a comparison graph of the denoising result of FIG. 6(b) according to the present invention using reference [7 ];
FIG. 10(c) is a comparison graph of the denoising results of FIG. 6(c) using reference [7] in the present invention;
FIG. 11(a) is a comparison graph of the denoising result of FIG. 6(a) according to the present invention;
FIG. 11(b) is a comparison graph of the denoising result of FIG. 6(b) according to the present invention;
FIG. 11(c) is a graph comparing the denoising results of FIG. 6(c) according to the present invention;
FIG. 12(a) is a data diagram of the point cloud of the soldier puppet to be denoised by one hand of the soldier puppet of the invention;
FIG. 12(b) is a graph comparing the results of the present invention denoising FIG. 12(a) using the method of the present application;
FIG. 13(a) is a data diagram of the point cloud of the head of the soldier figurine to be denoised;
FIG. 13(b) is a comparison graph of the denoising result of FIG. 13(a) according to the present invention;
FIG. 14(a) is a point cloud data diagram to be denoised on the upper half of a soldier figurine according to the invention;
FIG. 14(b) is a comparison graph of the denoising result of FIG. 14(a) according to the present invention;
FIG. 15(a) is a data diagram of the point cloud of the collar of the soldier warrior to be denoised;
FIG. 15(b) is a comparison of the denoising results of FIG. 15(a) according to the present invention;
FIG. 16(a) is a data diagram of point cloud of the soldiers to be denoised by hands of two hands of the soldier;
FIG. 16(b) is a comparison of the denoising results of FIG. 16(a) according to the present invention using the method of the present application.
Detailed Description
The present invention will be described in further detail with reference to the following examples, which are not intended to limit the invention thereto.
As shown in fig. 1, the invention provides a cultural relic point cloud data denoising method based on an unsupervised network framework, which specifically comprises the following steps;
step 1: modifying a network layer and constructing an output size to obtain a first layer of network on the basis of a pointet (reference [1]) network architecture; and modifying network input on the basis of totaldenosing (reference [4]) network architecture to construct and obtain a second-layer network.
Step 2: the first layer of network structure is a supervised network mechanism (refer to fig. 2, wherein Q is a real outlier noise point set label) which is mainly used for removing outlier noise points in noise point cloud data; the first layer of network structure inputs noise point cloud data on the basis of a pointenet network architecture, the noise point cloud data is represented as a point set of n x 3, wherein n represents the number of point clouds, 3 represents xyz coordinate information, local features of n x 128 are obtained through a layer of mlp, feature dimension increasing is carried out to 512 through a layer of mlp processing features, maxpopooling is carried out to obtain global features, the global features and the local features are fused, the features are extracted through two layers of mlp to obtain a point set of n x 3, the point set is subjected to a function g to obtain estimated outliers, and finally the point cloud data from which the estimated outliers are removed from the noise point cloud data is output. Wherein the estimated outliers obtained by the first layer network are marked with real outliers.
In the first layer network, a nonlinear function g is obtained through network training modeling, and is used for obtaining outlier noise points from input noise point cloud data:
Figure BDA0003047173670000061
p' in the formula (1) is input noise point cloud data, Pi' is the ith point in the noise point cloud data,
Figure BDA0003047173670000062
estimated probability value of the point as an outlier noise point
Figure BDA0003047173670000063
Then, the point is determined as an outlier noise point and added to the estimated outlier set
Figure BDA0003047173670000064
After all outliers in the input noise point cloud data are obtained, the input noise point cloud data are subjected to outlier removing to obtain point cloud to be further processed after the outliers are removed
Figure BDA0003047173670000065
Figure BDA0003047173670000066
At this time, only the real outlier tag is needed to constrain the estimated outlier set, so that the L1 distance is used as a loss function between the real outlier tag and the estimated outlier; the expression of the loss function is:
Figure BDA0003047173670000067
in the formula (2)
Figure BDA0003047173670000071
For de-noising the point cloud data, PiIn order to obtain clean point cloud data,
Figure BDA0003047173670000072
to obtain estimated outliers, QiAre true outlier tags.
And step 3: the second layer network structure takes the output of the first layer network as input and utilizes the unsupervised network structure to denoise again. Starting from the input point cloud data to be processed, two-stage unstructured encoding is performed, the field of view is reduced, and then two-stage decoding using transposed unstructured convolution is used. The coding and decoding in the layer network structure are realized based on Monte Carlo convolution. The network layer is of an unsupervised network structure (refer to fig. 3), point cloud data to be processed are input and then processed through the network layer, and output is the point cloud data after denoising.
The second layer network structure converts the spatial neighborhood in the point cloud data into a potential code defined on an encoder, then samples the potential code to a decoder, and executes two-stage encoding, wherein the reception field of the first stage is 5%, the reception field of the second stage is 10%, and the radius of the pooled Poisson discs of the first stage and the second stage is half of the respective reception field.
Because three-dimensional point cloud data is easily influenced by noise to generate coordinate deviation and a reliable pixel network is not provided, multiple convergence exists in an observation, and in order to overcome the problem, a spatial prior term is introduced to guide the observation to converge to the only closest one of multiple modes in manifold. The prior term q (z | y) to calculate the likelihood that the observed point y may be a clean point z; the calculation formula is expressed as: q (z | y) ═ p (z | S) × k (z-y) (3);
Figure BDA0003047173670000075
in equation (3), S is the surface of the clean point cloud data, σ is the bandwidth of k, W ═ diag (W), and W is a diagonal weight matrix, where W ═ 1/α r, r is 5% of the model diameter, and α is a scale factor.
Figure BDA0003047173670000076
When equation (5) is optimized, the observed point cloud data can be made to converge to a pattern closest to the real point cloud data. Wherein f is a parameter ofΘIs obtained by network training, l is fΘLoss function adopted between (y) and q, used in the present application by reference [3]The annealing loss function L as set forth in0，(|fΘ(y)-q|+ε)γWherein ε is 10-8And gamma is from 2 to 0 during training.
And 4, step 4: the output of the first layer network trained in the step 2 is used as the input of the second layer network trained in the step 3, so that a complete cultural relic point cloud data denoising network can be constructed and obtained (referring to fig. 4, P' in the figure represents noise point cloud data,
Figure BDA0003047173670000073
representing a set of estimated outlier noise points obtained across the network layer,
Figure BDA0003047173670000074
to remove point cloud data to be further denoised after estimating the outlier set,
Figure BDA0003047173670000081
point cloud data after complete denoising through a network).
And 5: and (4) denoising the cultural relic point cloud data according to the cultural relic point cloud data denoising network obtained in the step (4).
The invention aims to achieve the effect of accurate classification by adopting the method when the similar models are difficult to be accurately classified. The traditional method has the defects of model description, the invention considers the global shape information and the local characteristic information of the three-dimensional model, and has the effect of integrating the global information. In addition, the method uses the mean value of the Fourier coefficient difference as the criterion of similarity measurement, can accurately convert the similarity of two models into a numerical value, and judges the clustering result through a threshold value.
Example 1
Firstly, a two-layer point cloud data denoising network is constructed according to a point net and totaldenoisign network architecture.
Further, supervised training was performed on the constructed first layer network, using a data set containing 28 different shapes, 18 of which were used for training, and 10 were left for testing. 100K points are randomly sampled from the surface of each starting original triangular mesh to generate clean point cloud data.
For the outlier removal task, gaussian noise with a standard deviation of 20% was added to the random point subset in order to generate outliers. Starting at 10% to 90% of the point cloud data in the training set, each 10% apart point is converted to an outlier, and the final true outlier label, only selects outliers that are further from the surface than the standard deviation of the noise distribution. The experiment was repeated 2000 epochs, the batch size was 64, and the initial learning rate was set at 0.0001.
Further, unsupervised training is carried out on the constructed second-layer network, a minimization formula (5) is selected for training, before training, samples are extracted according to prior, and for uniform random zeta epsilon (0,1), a random point is selected from Y in r
Figure BDA0003047173670000082
And is arranged at
Figure BDA0003047173670000083
Under the conditions of (a) to (b). All the following operations are performed on a batch process with the size of the point cloud data, and a specific training algorithm is as follows:
1: for all points R do of the noise point cloud data;
2: xi by outputting (0,1) a random value from the uniform distribution;
3: obtaining Q through (R, xi) prior sampling, wherein Q is a group of prior samples of all points in the point cloud data;
4: minimizing fΘ(R)-Q||0Obtaining an updated parameter theta;
5：end for。
the data set adopts 15 different categories in ModelNet-40, wherein each category has 7 models with different shapes, 5 models are used for training, and 2 models are used for testing; in a simple noise model, sampling each 3D grid by using Poisson disc sampling to obtain clean point cloud data of 13K and 190K, wherein 2200 ten thousand points are used for training and 1000 ten thousand points are used for testing; then, Gaussian three-dimensional noise is added, and the standard deviation of the Gaussian three-dimensional noise is 0.5%, 1% and 1.5% of the diagonal line of the bounding box; in the training process, an Adam optimizer is adopted to optimize the loss function, the initial learning rate is set to be 0.005, and along with the training process, the learning rate can slowly and automatically decrease.
And further, outputting the trained first-layer network as the trained second-layer network input, and fusing the two layers of networks to form a complete point cloud data denoising network.
The invention aims to realize point cloud data denoising and simultaneously keep the detailed geometric characteristics of the point cloud data as much as possible under the condition of only obtaining a real outlier label, and improve the denoising efficiency of large-scale point cloud data. For the problems that the conventional point cloud data denoising network training has high data requirement and poor outlier noise point removing effect, the invention considers the combination of an unsupervised network and a pointenet network architecture, thereby realizing the training of the data under the condition of only acquiring a real outlier label, even without paired training data, and better removing the outlier noise points in the noise point cloud data. In addition, compared with other denoising algorithms, the method has the advantages that the requirement of the network on training data is less, the operation efficiency is higher, and the denoising effect is better.
Comparative experiment 1
In order to objectively verify the experimental results of the present application, the following experiments were performed according to references [4], [5], [6], and [7 ]. First, using publicly available point cloud data models bunny (see fig. 5(a)), dragon (see fig. 5(b)), and drill (see fig. 5(c)) in three stanford university public data sets, and adding gaussian noise having a standard deviation of 0.03% (see fig. 6(a)), 0.05% (see fig. 6(b)), and 0.07% (see fig. 6(c)) to clean point cloud data as noise point cloud data to perform an experiment, and using cd values and mse values as objective evaluation criteria for the experimental results. The results of the experimental data are shown in tables 1 and 2. When gaussian noise with a standard deviation of 0.05% is added to clean point cloud data as noise point cloud data, the denoising comparison results are shown in fig. 7(a) -7 (c), fig. 8(a) -8 (c), fig. 9(a) -9 (c), fig. 10(a) -10 (c), and fig. 11(a) -11 (c).
As can be seen from fig. 7(a) -7 (c), 8(a) -8 (c), 9(a) -9 (c), 10(a) -10 (c), and 11(a) -11 (c), the denoising algorithm of the present application is still clear in detail features of the dragon tail, dragon scale, and faucet part, and there is no excessive fairing, even in fig. 6(b) containing the most complicated features, as shown in fig. 11 (b). In general, compared with references [4], [5], [6] and [7], for noise point cloud data containing features with different complexity, a point cloud data model denoised by the algorithm is always closest to a clean point cloud data model, geometric features of details of the point cloud data model are not lost while denoising, edges of the denoised point cloud data model are clearer, surface noise points are obviously reduced, the geometric features are more obvious, and the surface is smoother.
TABLE 1 comparison of denoising effects of different algorithms with cd value as evaluation criterion
Figure BDA0003047173670000101
TABLE 2 comparison of denoising effects of different algorithms with cd value as evaluation criterion
Figure BDA0003047173670000111
As can be seen from the parameters in tables 1 and 2, the cd distance and the MSE are respectively used as evaluation standards in the references [4], [5], [6] and [7], and the chamfer distance and the Mse distance calculated after denoising are smaller than those of other algorithms, so that the mean error after denoising is the minimum, and the difference between the denoised point cloud data and the original clean point cloud data is the minimum.
Comparative experiment 2
Selecting cultural relic point cloud data for experiment, the application adopts a soldier warrior point cloud data model to carry out denoising effect display, and the point cloud data model has 325728 points in total.
After gaussian noise with a standard deviation of 0.03% is added to the soldier puppet model, the network structure denoising effect obtained by using the method of the application is shown in fig. 12(a) -12 (b), fig. 13(a) -13 (b), fig. 14(a) -14 (b), fig. 15(a) -15 (b), and fig. 16(a) -16 (b).
From fig. 12(a) -12 (b), 13(a) -13 (b), 14(a) -14 (b), 15(a) -15 (b), 16(a) -16 (b), it can be clearly observed that the details of the body of the soldier puppet, such as the hands, the head and the neckline, become blurred after noise is added, and after the network structure obtained by the method is denoised, the body edge of the soldier puppet is clearer, the detailed geometrical characteristics of the body part are more obvious, and the phenomenon of excessive smoothness caused by denoising does not occur.
The point cloud data model of the soldier warrior contains a plurality of complex geometric features and is large in number, and belongs to a large-scale point cloud data model.
Reference to the literature
[1]Qi C R,Su H,Mo K,et al.PointNet:Deep Learning on Point Sets for 3D Classification and Segmentation[C]//2017IEEE Conference on Computer Vision and Pattern Recognition(CVPR).IEEE,2017.
[2]Liu Liheng,Zhao Fuqun,Tang Hui,et al.A point cloud denoising algorithm for cultural relics with geometric feature preservation[J].Data Acquisition and Processing,2020,v.35；No.160(02):183-190.
[3]J.Lehtinen,J.Munkberg,J.Hasselgren,S.Laine,T.Kar-ras,M.Aittala,and T.Aila.Noise2Noise:Learning image restoration without clean data.2018.
[4]P.Casajus,T.Ritschel and T.Ropinski,"Total Denoising:Unsupervised Learning of3D Point Cloud Cleaning,"in 2019IEEE/CVF International Conference on Computer Vision(ICCV),Seoul,Korea(South),2019pp.52-60.
[5]Marie-Julie Rakotosaona,Barbera V L,Guerrero P,et al.PointCleanNet:Learning to Denoise and Remove Outliers from Dense Point Clouds[J].Computer Graphics Forum,2020,39(1).
[6]Duan C,Chen S,Kovacevic J.3D Point Cloud Denoising via Deep Neural Network based Local Surface Estimation[J].2019.
[7]Li Pengfei,Wu Haie,Jing Junfeng,et al.Noise classification and denoising algorithm of point cloud model[J].Computer Enginee-ring and Application,2016,52(020):188-192.

Claims (4)

1. A cultural relic point cloud data denoising method based on an unsupervised network framework is characterized by comprising the following steps;
step 1: constructing a two-layer network model based on an unsupervised network architecture;
step 2: inputting noise cultural relic point cloud data with abnormal values and containing outlier noise points into the first layer network constructed in the step 1, constraining the output of the first layer network by using the actual outlier noise point labels of the noise cultural relic point cloud data, continuously updating the parameters of neurons in the first layer network by adopting supervised repeated iterative training, and terminating the training when the loss function of the first layer network reaches a set threshold value;
and step 3: after the noise cultural relic point cloud data is input into the second-layer network constructed in the step 1, the output of the second-layer network is restrained by using a set loss function, the parameters of neurons in the second-layer network are continuously updated by adopting unsupervised repeated iterative training, and the training is terminated when the loss function reaches a set threshold or the maximum iterative times;
and 4, step 4: taking the output of the first layer network trained in the step 2 as the input of the second layer network trained in the step 3, and constructing and obtaining a complete cultural relic point cloud data denoising network;
and 5: and (4) denoising the cultural relic point cloud data according to the cultural relic point cloud data denoising network obtained in the step (4).
2. The unsupervised network framework-based cultural relic point cloud data denoising method as claimed in claim 1, wherein: the first layer network is obtained by modifying a network layer and the output size on the basis of a pointenet network architecture;
the second layer network is obtained by modifying network input to construct on the basis of the totaldenosing network architecture.
3. The method for denoising cultural relic point cloud data based on an unsupervised network framework as claimed in claim 2, wherein: the construction and training process of the first layer network is as follows:
inputting noise point cloud data, representing the noise point cloud data as a point set, firstly obtaining local features, processing feature dimensionality by a layer of mlp, then performing maxporoling to obtain global features, fusing the global features and the local features, obtaining a point set by two layers of mlp, obtaining estimated outliers by the point set through a function g, and finally outputting the point cloud data of the noise point cloud data without the estimated outliers;
in the first layer network, a nonlinear function g is obtained through network training modeling, and is used for obtaining outlier noise points from input noise point cloud data:
Figure FDA0003047173660000021
p' in the formula (1) is input noise point cloud data, Pi' is the ith point in the noise point cloud data,
Figure FDA0003047173660000022
the point is the estimated probability value of the outlier noise point, and after all outliers in the input noise point cloud data are obtained, the input noise point cloud data are removed from the outlier set to obtain the point cloud to be further processed after the outliers are removed
Figure FDA0003047173660000023
At the moment, the real outlier label is adopted to constrain the estimation outlier set, and an L1 distance is used as a loss function between the real outlier label and the estimation outlier; the expression of the loss function is:
Figure FDA0003047173660000024
in the formula (2)
Figure FDA0003047173660000025
For de-noising the point cloud data, PiIn order to be a clean point cloud,
Figure FDA0003047173660000026
to obtain estimated outliers, QiAre true outlier tags.
4. The unsupervised network framework-based cultural relic point cloud data denoising method as claimed in claim 3, wherein: the construction and training process of the second-layer network is as follows:
after point cloud data to be processed is input, the point cloud data is processed through a second network layer, the output is the denoised point cloud data, a spatial neighborhood in the point cloud data is converted into a potential code defined on an encoder, then the potential code is sampled to a decoder, and two-stage encoding is executed; introducing a spatial prior term to guide the observation to converge to the only closest one of a plurality of modes on the manifold;
the prior term q (z | y) to calculate the likelihood that the observed point y may be a clean point z; the calculation formula is expressed as: q (z | y) ═ p (z | S) × k (z-y) (3);
Figure FDA0003047173660000027
in formula (3), S is the surface of the clean point cloud data, σ is the bandwidth of k, W is diag (W), W is a diagonal weight matrix, W is 1/α r, r is 5% of the model diameter, and α is a scale factor;
Figure FDA0003047173660000028
when the formula (5) is optimized, the observed point cloud data can be converged; wherein f is a parameter ofΘIs obtained by network training, l is fΘThe loss function employed between (y) and q is the annealing loss function L0。
CN202110476219.9A 2021-04-29 2021-04-29 Cultural relic point cloud data denoising method based on unsupervised network framework Active CN113191973B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110476219.9A CN113191973B (en) 2021-04-29 2021-04-29 Cultural relic point cloud data denoising method based on unsupervised network framework

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110476219.9A CN113191973B (en) 2021-04-29 2021-04-29 Cultural relic point cloud data denoising method based on unsupervised network framework

Publications (2)

Publication Number Publication Date
CN113191973A true CN113191973A (en) 2021-07-30
CN113191973B CN113191973B (en) 2023-09-01

Family

ID=76980716

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110476219.9A Active CN113191973B (en) 2021-04-29 2021-04-29 Cultural relic point cloud data denoising method based on unsupervised network framework

Country Status (1)

Country Link
CN (1) CN113191973B (en)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114170113A (en) * 2022-01-20 2022-03-11 湖北晓雲科技有限公司 Three-dimensional modeling method and system for unmanned aerial vehicle aerial surveying and mapping
CN114862692A (en) * 2022-04-01 2022-08-05 华南理工大学 Three-dimensional point cloud denoising method based on standardized flow theory
WO2023020289A1 (en) * 2021-08-16 2023-02-23 北京百度网讯科技有限公司 Processing method and apparatus for network model, and device and storage medium
CN115984801A (en) * 2023-03-07 2023-04-18 安徽蔚来智驾科技有限公司 Point cloud target detection method, computer equipment, storage medium and vehicle

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20200334897A1 (en) * 2019-04-18 2020-10-22 Zebra Medical Vision Ltd. Systems and methods for reconstruction of 3d anatomical images from 2d anatomical images
CN112085821A (en) * 2020-08-17 2020-12-15 万申(北京)科技有限公司 Semi-supervised-based CBCT (cone beam computed tomography) and laser scanning point cloud data registration method
CN112435239A (en) * 2020-11-25 2021-03-02 南京农业大学 Scindapsus aureus leaf shape parameter estimation method based on MRE-PointNet and self-encoder model
CN112464994A (en) * 2020-11-05 2021-03-09 北京航天控制仪器研究所 Boat stern wave identification and removal method based on PointNet network
CN112669230A (en) * 2020-12-23 2021-04-16 天津博迈科海洋工程有限公司 Point cloud data denoising method based on convolutional neural network

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20200334897A1 (en) * 2019-04-18 2020-10-22 Zebra Medical Vision Ltd. Systems and methods for reconstruction of 3d anatomical images from 2d anatomical images
CN112085821A (en) * 2020-08-17 2020-12-15 万申(北京)科技有限公司 Semi-supervised-based CBCT (cone beam computed tomography) and laser scanning point cloud data registration method
CN112464994A (en) * 2020-11-05 2021-03-09 北京航天控制仪器研究所 Boat stern wave identification and removal method based on PointNet network
CN112435239A (en) * 2020-11-25 2021-03-02 南京农业大学 Scindapsus aureus leaf shape parameter estimation method based on MRE-PointNet and self-encoder model
CN112669230A (en) * 2020-12-23 2021-04-16 天津博迈科海洋工程有限公司 Point cloud data denoising method based on convolutional neural network

Non-Patent Citations (4)

* Cited by examiner, † Cited by third party
Title
PEDRO H ET AL: "《Total Denoising: Unsupervised Learning of 3D Point Cloud Cleaning》", 《ARXIV》 *
PEDRO H ET AL: "《Total Denoising: Unsupervised Learning of 3D Point Cloud Cleaning》", 《ARXIV》, 17 October 2019 (2019-10-17) *
杨炜静: "《基于三维激光扫描技术的原木三维图像重构》", 《中国优秀硕士学位论文全文数据库 农业科技辑》 *
杨炜静: "《基于三维激光扫描技术的原木三维图像重构》", 《中国优秀硕士学位论文全文数据库 农业科技辑》, 15 February 2019 (2019-02-15) *

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2023020289A1 (en) * 2021-08-16 2023-02-23 北京百度网讯科技有限公司 Processing method and apparatus for network model, and device and storage medium
CN114170113A (en) * 2022-01-20 2022-03-11 湖北晓雲科技有限公司 Three-dimensional modeling method and system for unmanned aerial vehicle aerial surveying and mapping
CN114862692A (en) * 2022-04-01 2022-08-05 华南理工大学 Three-dimensional point cloud denoising method based on standardized flow theory
CN114862692B (en) * 2022-04-01 2024-04-02 华南理工大学 Three-dimensional point cloud denoising method based on standardized flow theory
CN115984801A (en) * 2023-03-07 2023-04-18 安徽蔚来智驾科技有限公司 Point cloud target detection method, computer equipment, storage medium and vehicle

Also Published As

Publication number Publication date
CN113191973B (en) 2023-09-01

Similar Documents

Publication Publication Date Title
CN113191973A (en) Cultural relic point cloud data denoising method based on unsupervised network framework
CN112200750B (en) Ultrasonic image denoising model establishing method and ultrasonic image denoising method
CN107679462B (en) Depth multi-feature fusion classification method based on wavelets
CN112446429B (en) CGAN (Carrier grade Access network) -based routing inspection image data small sample expansion method
CN112508991B (en) Panda photo cartoon method with separated foreground and background
CN112150354B (en) Single image super-resolution method combining contour enhancement and denoising statistical prior
CN111080591A (en) Medical image segmentation method based on combination of coding and decoding structure and residual error module
CN113066025B (en) Image defogging method based on incremental learning and feature and attention transfer
CN106157254A (en) Rarefaction representation remote sensing images denoising method based on non local self-similarity
CN109002771B (en) Remote sensing image classification method based on recurrent neural network
Cao et al. Latent tangent space representation for normal estimation
CN113506224A (en) Image restoration method based on multi-scale generation countermeasure network
CN112596104A (en) Seismic data denoising method combining tensor decomposition and total variation
CN112801945A (en) Depth Gaussian mixture model skull registration method based on dual attention mechanism feature extraction
CN112488934A (en) Finger vein image denoising method based on CS-TCGAN
CN112967210B (en) Unmanned aerial vehicle image denoising method based on full convolution twin network
Xia et al. Meta-learning based degradation representation for blind super-resolution
CN109003247B (en) Method for removing color image mixed noise
CN107292855A (en) A kind of image de-noising method of the non local sample of combining adaptive and low-rank
CN108090914B (en) Color image segmentation method based on statistical modeling and pixel classification
CN111340741B (en) Particle swarm optimization gray image enhancement method based on quaternion and L1 norm
CN108510591A (en) A kind of improvement Poisson curve reestablishing method based on non-local mean and bilateral filtering
CN109584194B (en) Hyperspectral image fusion method based on convolution variation probability model
CN114708353B (en) Image reconstruction method and device, electronic equipment and storage medium
CN109165551B (en) Expression recognition method for adaptively weighting and fusing significance structure tensor and LBP characteristics

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant