CN116977187A - Depth point set resampling method based on gradient field - Google Patents
Depth point set resampling method based on gradient field Download PDFInfo
- Publication number
- CN116977187A CN116977187A CN202210392112.0A CN202210392112A CN116977187A CN 116977187 A CN116977187 A CN 116977187A CN 202210392112 A CN202210392112 A CN 202210392112A CN 116977187 A CN116977187 A CN 116977187A
- Authority
- CN
- China
- Prior art keywords
- point
- point cloud
- gradient
- gradient field
- network
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 64
- 238000012952 Resampling Methods 0.000 title claims abstract description 17
- 238000012549 training Methods 0.000 claims abstract description 26
- 238000013528 artificial neural network Methods 0.000 claims abstract description 17
- 238000000605 extraction Methods 0.000 claims abstract description 11
- 230000006870 function Effects 0.000 claims description 13
- 230000015556 catabolic process Effects 0.000 claims description 11
- 238000006731 degradation reaction Methods 0.000 claims description 11
- 238000005457 optimization Methods 0.000 claims description 11
- 239000011159 matrix material Substances 0.000 claims description 10
- 238000004590 computer program Methods 0.000 claims description 8
- 238000009826 distribution Methods 0.000 claims description 8
- 238000000137 annealing Methods 0.000 claims description 5
- 230000001174 ascending effect Effects 0.000 claims description 3
- 239000000284 extract Substances 0.000 claims description 3
- 238000012804 iterative process Methods 0.000 claims description 3
- 125000004122 cyclic group Chemical group 0.000 claims description 2
- 238000003860 storage Methods 0.000 claims description 2
- 230000000630 rising effect Effects 0.000 abstract description 6
- 230000000875 corresponding effect Effects 0.000 description 22
- 230000008569 process Effects 0.000 description 17
- 230000008439 repair process Effects 0.000 description 10
- 238000005070 sampling Methods 0.000 description 4
- 238000013135 deep learning Methods 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- 238000013527 convolutional neural network Methods 0.000 description 2
- 230000007423 decrease Effects 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 238000006073 displacement reaction Methods 0.000 description 2
- 230000002452 interceptive effect Effects 0.000 description 2
- 206010023126 Jaundice Diseases 0.000 description 1
- 238000000342 Monte Carlo simulation Methods 0.000 description 1
- 239000000654 additive Substances 0.000 description 1
- 230000000996 additive effect Effects 0.000 description 1
- 230000004931 aggregating effect Effects 0.000 description 1
- 230000002776 aggregation Effects 0.000 description 1
- 238000004220 aggregation Methods 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 230000002596 correlated effect Effects 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 229910001092 metal group alloy Inorganic materials 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000006116 polymerization reaction Methods 0.000 description 1
- 238000004321 preservation Methods 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/77—Retouching; Inpainting; Scratch removal
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/0464—Convolutional networks [CNN, ConvNet]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10028—Range image; Depth image; 3D point clouds
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Biophysics (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Biomedical Technology (AREA)
- Health & Medical Sciences (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Life Sciences & Earth Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a depth point set resampling method based on a gradient field, which comprises the following steps: training a neural network using the training data set; the method for training the neural network comprises the following steps: inputting the degenerated point cloud into a context feature extraction network, and acquiring each point in the degenerated point cloud by the context feature extraction networkCorresponding feature h i And inputs it to a gradient field estimation network; gradient field estimation network basis pointsContext point cloud, pointAnd its characteristic h i Obtaining a pointCorresponding gradientAccording toCorresponding true gradients in a clean point cloudCalculating a loss functionInputting the degenerated point cloud X to be sampled into a trained neural network to obtain a gradient field corresponding to each point in the degenerated point cloud X; and then, according to the gradient field of each point, carrying out iterative updating on the points in the cloud X through gradient rising, and completing the restoration of the degraded point cloud X.
Description
Technical Field
The invention belongs to the technical field of computer software, relates to point cloud restoration, and in particular relates to a depth point set resampling method based on a gradient field.
Background
Increasingly sophisticated depth sensing, laser scanning and image processing techniques enable people to more conveniently obtain a three-dimensional point cloud from a real-world scene. Three-dimensional point clouds, which consist of discrete three-dimensional points irregularly sampled from a continuous surface, have attracted increasing attention as an effective representation of three-dimensional shapes, and are widely used in autopilot, robotics, and immersive interactive telepresence. However, due to inherent limitations of the scanning device, or matching ambiguity in reconstruction from the image, the point cloud is often disturbed by noise or affected by low density. Thus, point cloud repair, such as denoising and upsampling, is critical to related three-dimensional vision applications.
The point cloud repair method can be divided into two types, namely an optimization-based repair method and a deep learning-based repair method. Optimization-based methods rely heavily on geometric prior knowledge, sometimes making it difficult to balance the effects of detail preservation and repair. Recently, due to the advent of neural network architecture specifically designed for point clouds, deep learning-based methods have emerged and good repair performance has been achieved. For point cloud denoising, most denoising models based on deep learning are to predict the displacement of noise points and potential surfaces and then move the points back to the corresponding potential surfaces. Such methods face mainly two problems, namely point cloud shrinkage or the occurrence of outliers, which result from overestimation or underestimation of displacement. Whereas for the up-sampling task of point clouds, complex regularization terms or fine-tuning operations are often required to prevent the trivial result of point clouds coming together.
Disclosure of Invention
Aiming at the problems existing in the prior art, the invention aims to provide a depth point set resampling method based on a gradient field. Firstly, estimating a distributed global gradient field from an input degradation point cloud; and then, gradient rising is carried out by using the estimated gradient field, the points are converged to the potential surface, and the point cloud restoration is completed.
Three-dimensional point clouds obtained by scanning real-world objects or scenes have found wide application in recent years, including immersive interactive telepresence, autopilot, surveillance, and the like. However, the sampled point cloud often encounters noise or low density effects. The invention provides a new point cloud repair paradigm (Deep Point Set Resampler, deep RS) which enables points to approach corresponding potential surfaces by learning continuous gradient fields of the point cloud. In particular, the invention represents the point cloud by its gradient field, i.e. the gradient of the logarithmic probability density function, and makes this gradient field continuous, thus guaranteeing the continuity of the solvable optimization model. The invention fits this gradient field through a neural network, based on which a gradient-based Markov chain Monte Carlo Method (MCMC) is performed on the noisy or sparse point cloud of inputs. In addition, the invention also provides a regularization introducing process in the MCMC process in the point cloud restoration process. This is essentially an iterative improvement to the intermediate resampled point cloud and introduces various a priori knowledge in the resampling process. A large number of experiments show that the point cloud resampling method achieves the most advanced performance in representative repair tasks including point cloud denoising and up-sampling.
The technical scheme of the invention is as follows:
a depth point set resampling method based on gradient field includes the steps:
training a neural network using the training data set; wherein the training data set comprises a degraded point cloud and a corresponding clean point cloud; the neural network comprises a context feature extraction network and a gradient field estimation network; the method for training the neural network comprises the following steps:
inputting a degraded point cloud into the context feature extraction network, wherein the context feature extraction network acquires each point in the degraded point cloudCorresponding feature h i And inputting it into the gradient field estimation network; the gradient field estimation network is based on the point +.>Context point cloud, point->And its characteristic h i Get the spot->Corresponding gradient->According to->Corresponding real gradient +.>Calculating a loss function->Wherein S represents the point cloud distribution of the degraded point cloud X; training the neural network by minimizing the loss function L when damagedAfter the loss function converges or reaches the set training cycle times, training is completed;
inputting the degenerated point cloud X to be sampled into the trained neural network to obtain a gradient field g (X) corresponding to each point X in the degenerated point cloud X; and then, according to the gradient field of each point in the degradation point cloud X, carrying out iterative updating on the points in the degradation point cloud X through gradient ascending until reaching a set upper limit or convergence, and completing the restoration of the degradation point cloud X.
Further, the method for iteratively updating the points in the degradation point cloud to be sampled by gradient ascent comprises the following steps:wherein T is the set total number of iterative loops, alpha t Is a set super parameter; />Is the ith acquisition point X in the degenerated point cloud X i Coordinate value after updating at t-th iteration cycle,/->And updating the gradient field of the corresponding point calculated for the obtained coordinates according to the t-1 th cyclic iteration.
Furthermore, a regularization term is added in the iterative updating, namely gradient rising and optimization based on the regularization term are alternately carried out; the iterative process isWherein I represents an identity matrix, lambda is a super parameter, and L is a Laplacian matrix of a k-nearest neighbor graph generated based on an input point cloud.
Further, a dot is obtainedCorresponding gradient->The method of (1) is as follows: the gradient field estimation network first extractsEvery point in the degenerate point cloud +.>Each neighbor in the upper and lower Wen Dian cloud +.>Is>Then according to the adjacent point->Distance point->Distance of (2) is relative feature->Giving corresponding weight to obtain the point->Is characterized by the aggregation of (3)Then will->Inputting the global multi-layer perceptron, and estimating to obtain the point +.>Corresponding gradient->Wherein (1)>To at the point->Is the set of points in the neighborhood with a center radius r.
Further, neighboring pointsDistance point->The farther apart the relative features are +.>The smaller the weight of (c).
Further, a cosine annealing method is used for determining relative characteristicsIs a weight of (2).
A server comprising a memory and a processor, the memory storing a computer program configured to be executed by the processor, the computer program comprising instructions for performing the steps of the above method.
A computer readable storage medium having stored thereon a computer program, characterized in that the computer program when executed by a processor realizes the steps of the above method.
The invention has the following advantages:
1) The invention unifies the point cloud restoration problems of denoising, up-sampling and the like into the resampling of the point cloud, and provides an integrated solution form.
2) Compared with the prior method, the method analyzes the continuity of the distribution modeling, and proposes a continuous model by utilizing a cosine annealing method, thereby ensuring that the optimization based on the gradient is solvable.
3) According to the method, regularization is introduced into the resampling process of the point set, and the point cloud in the middle process can be repeatedly enhanced in the sampling process by a specific regularization method.
Drawings
Fig. 1 is a schematic diagram of a point cloud repairing method according to the present invention.
Fig. 2 is a network schematic diagram of deep rs.
Fig. 3 is a network structure of deep rs.
Detailed Description
The invention will now be described in further detail with reference to the accompanying drawings, which are given by way of illustration only and are not intended to limit the scope of the invention.
(1) Algorithm framework:
first, the present invention regards the undegraded point cloud Y as a sample from the three-dimensional distribution p (Y). Considering the degraded point cloud of the input acquisition, the invention marks the degraded point cloud asWhere H is the corresponding degradation function, such as downsampling, blurring, etc.; n is additive noise from a certain noise distribution (e.g., gaussian distribution, etc.); />The convolution operation is represented. Assuming that the distribution q (X) corresponding to X is known, the mode of the general degradation function is 0, and the left and right sides of the above formula are derived, the readily available q (X) reaches a maximum value when on the corresponding potential surface. Thus, reconstruct the point cloud->Equivalent to maximizing Sigma i logq(x i ) Where M is the number of points of the point cloud. This can be done by a pattern of gradient rises until q (x) converges. This gradient rising process is only associated with +.>I.e. the first derivative of the logarithmic density function. Thus gradient field->Always pointing to a clean surface. Also because q (x) is unknown during the test. Rather than estimating q (x) from degraded observations, the present invention chooses to estimate its gradient because it is easier to operate. To sum upThe model of the invention aims at learning the gradient field g (x) such that Σ i logq(x i ) Maximising, i.e. max g(x) ∑ i logq(x i )。
It can be seen that repairing a point cloud is equivalent to solving the equation g (x) =0, so the model must have continuity to ensure that this equation can be solved by gradient iteration. The present invention uses cosine annealing to make the estimation of gradient field continuous to the center point. In particular, since the present invention is directed to a local neighborhood N with radius r r (x) To estimate the gradient of a point x, other points may suddenly enter or leave the neighborhood N when the position of x changes during resampling r (x) This will result in a discontinuity. Thus, before aggregating the features of nearby points, the present invention assigns each point a corresponding weight that decays as the distance from x increases. Formally, the polymerization characteristic of x isWherein x is j ∈N r (x) Represents x j In the neighborhood of x with radius r, f j (x) Is x j Calculated features with respect to x. Essentially, this formula ensures that the feature weight decreases with increasing distance from x, and eventually decreases to 0 when the distance equals or exceeds r.
The depth point set resampling method provided by the invention firstly learns the gradient field from training data, and then carries out point cloud restoration through gradient rising. This framework allows the present invention to introduce regularization during gradient ascent for further fine tuning based on a priori knowledge. Existing in-service regularization can only be considered during the training phase, typically by taking it into account the loss function. The framework of the invention introduces regularization in the repair process, and is therefore more flexible for designing various prior parameters for different downstream tasks. Adding regularization terms can enable the recovered point cloud to have specific attributes according to corresponding prior knowledge, and a formula can be written as:x and ZRepresenting the input degenerate point cloud and the repaired point cloud respectively, H (·) represents the degenerate operator defined on Z, and P (Z) represents the regularization term. Specifically, the invention mainly uses two regular terms commonly used by optimization-based algorithms, namely a Graph Laplace Regularization (GLR) and a weighted graph Laplace regularization (RGLR), to carry out point cloud restoration, and the invention specifically introduces the use modes of the two algorithms in a network model part.
The Graph (Graph) provides a structurally adaptable, accurate and compact representation for a point cloud. Thus, the present invention represents each point in the point cloud as a node in graph G and connects points that are neighbors to each other to construct a graph, e.g., a k-neighbor (kNN) graph, with each point connected to its nearest k neighbor nodes. The mathematical formula for GLR is generally written as:where L is the graph laplace matrix, the connectivity of the encoded graph and the degree of each node. i-j means that points i, j are connected, meaning that the two points are highly correlated in the point cloud. If the GLR is small, this means that the graph signal is smooth, since z is correspondingly greater if the weight is greater i And z j Should be similar. In GLR, the laplace matrix L is fixed. The invention can also consider the laplace matrix as a learnable function of the graph signal Z, expanding it to RGLR: />Wherein w is ij (z i ,z j ) Can be adaptively learned during the optimization process. RGLR helps to promote the sheet-like smoothness of the point cloud, so that the repaired point cloud possesses this better property.
The invention is next presented how regularization terms are added to the process of the invention. The present invention is primarily concerned with regularizing items that are microscopically. In the above formula, H may be assumed to be an identity matrix for simplicity. Derivative Z and let derivative be 0, 2 (X-Z) +λp' (Z) =0. The invention thus allows a very simple solution to Z, for example when GLR is selected a prioriThere is z= (i+λl) -1 ) X is a metal alloy. And (3) alternately carrying out gradient rising and optimization based on a regularization term in the resampling process to complete the reconstruction process.
(2) Network model
Given an acquired degenerate point cloud containing M pointsThe object of the invention is to repair it by the above-mentioned point set resampling method, such as denoising, upsampling, etc. To achieve this goal, the present invention devised a neural network for gradient field training and point cloud resampling. The network is composed of a context feature extraction network and a gradient field estimation network. The overall structure is shown in fig. 2, and the specific network composition is shown in fig. 3.
a) Context feature extraction network
To estimate the global gradient field, a context point cloud is needed for the auxiliary operation. The point cloud may be the same as the degenerate point cloud or a more intimate point cloud may be selected. Given a contextual point cloudThe invention firstly extracts the characteristics corresponding to each point through the context characteristics. The network is based on a dynamic graph convolutional neural network (DGCNN; reference Wang Y, sun Y, liu Z, et al dynamic graph cnn for learning on point clouds [ J ]]Acm Transactions On Graphics (tog), 2019,38 (5): 1-12) enables multi-scale and local and non-local features to be extracted for each point and further features with richer background information to be obtained by densely connecting the convolutional neural networks described above. Specifically, a contextual point cloud is input into the feature extraction network, which first builds a k-neighbor graph for this point cloud, each point being considered a vertex of the graph and connected to its k nearest neighbors. Then, it uses the dynamic graph convolution network to extract the corresponding feature of each point, note the point +.>Extracted featuresIs h i (including multi-scale information, i.e., local features and non-local features) and saving and inputting the extracted features into the gradient field estimation network of the next stage.
b) Gradient field estimation network
After obtaining the above h i The present invention then attempts to estimate the gradient field from a global perspective using a gradient field estimation network. The network is composed of a plurality of multi-layer perceptrons (MLPs). In this section, the input point cloud is a contextual point cloud and its features and degenerated point cloud. For each point in the degenerate point cloudThe gradient field estimation network first extracts every point +.>K-neighbor in upper and lower Wen Dian cloud +.>Is>Wherein F is implemented by using a multi-layer perceptron (MLP) and an edge convolution method proposed in a dynamic graph convolution neural network, h j To be from the point->Extracted features, cryptophan jaundice>The meaning of (c) is already mentioned in the chapter of the algorithm framework. Then, the extracted relative features are subjected to cosine annealing method introduced by the second section in the algorithm framework section, and corresponding weights are given to each point to ensure the continuity of the model, so that the model is obtainedFinally, will->Inputting the global multi-layer perceptron G, and estimating to obtain gradient corresponding to each point>The overall learning process can be written as: />
(3) Training and application of network
(a) Training process
In the training process, the invention firstly obtains a training data set, which comprises degenerated (such as sparse or noisy) point clouds and corresponding clean point clouds. First, the present invention calculates the gradient of the degenerate point cloud according to the method described in section (2). Generally, the data amount is limited in the training process, and for convenience, the context point cloud is simply taken as the degradation point cloud, so that the taking method is enough to obtain good experimental results. Recording pointThe final calculated gradient is +.>We define the true gradient of each point +.>Wherein Y represents the corresponding clean point cloud, < >>Representing distance points +.in a clean point cloud>The nearest point. Then the optimization objective (loss function) defined by the invention>Wherein S represents that the point in the point cloud is +.>Distribution in space (three-dimensional vector space). The invention trains the model by minimizing the loss function, and the model is trained after the loss function converges or reaches a certain training cycle number.
(b) Application process
In the application stage, since the input data only has the degraded point cloud, the degraded point cloud itself is also necessarily used as the context point cloud. And inputting the gradient field g (x) corresponding to each point x into the network model of the invention. According to the obtained gradient field, under the condition of no regularization, the method carries out iterative updating on the points through simple gradient ascending until reaching an upper limit or convergence, and the repaired point cloud is obtained through a resampling method. Namely: wherein t is the number of iterative cycles, alpha t The super parameter is a manually set super parameter, and the super parameter can be changed along with the change of the circulation times. />Acquisition Point x, a degenerate Point cloud i And (5) circulating the updated coordinate values at the t-th iteration.The gradient field of the corresponding point calculated from the coordinates obtained from the iteration update of the t-1 th loop is then used. If it is desired to add regularization terms, the gradient ascent flow simply needs to be modified, i.e., gradient ascent and regularization term-based optimization are alternated. Taking the graph Laplacian as an example, the iterative process at this time is +.>Wherein I represents an identity matrix, lambda is a super parameter, and L is a Laplacian matrix of a k-nearest neighbor graph generated based on an input point cloud. />Representing the intermediate result obtained in the t-th cycle, the remaining labels have the same meaning as the labels of the formula without regularization. Tables 1, 2, 3 show the effects of the present invention, and it can be seen that the experimental results of the present invention are superior to the previous methods in both the denoising and upsampling tasks.
Table 1: denoising results of Gaussian noise on PUNet and PCNet datasets
Table 2: denoising results for other noise on a PUNet dataset
Table 3: upsampling results on PU-GAN and MPU datasets
Note that: ours (Gen) and OursRefers to the method of the present invention.
Although specific embodiments of the invention have been disclosed for illustrative purposes, it will be appreciated by those skilled in the art that the invention may be implemented with the help of a variety of examples: various alternatives, variations and modifications are possible without departing from the spirit and scope of the invention and the appended claims. Therefore, it is intended that the invention not be limited to the particular embodiment disclosed as the best mode contemplated for carrying out this invention, but that the invention will have the scope indicated by the scope of the appended claims.
Claims (8)
1. A depth point set resampling method based on gradient field includes the steps:
training a neural network using the training data set; wherein the training data set comprises a degraded point cloud and a corresponding clean point cloud; the neural network comprises a context feature extraction network and a gradient field estimation network; the method for training the neural network comprises the following steps:
inputting a degraded point cloud into the context feature extraction network, wherein the context feature extraction network acquires each point in the degraded point cloudCorresponding feature h i And inputting it into the gradient field estimation network; the gradient field estimation network is based on the point +.>Context point cloud, point->And its characteristic h i Get the spot->Corresponding gradient->According to->Corresponding real gradient +.>Calculating a loss function->Wherein S represents the point cloud distribution of the degraded point cloud X; training the neural network by minimizing the loss function L, and after the loss function converges or reaches the set training cycle times, finishing training;
inputting the degenerated point cloud X to be sampled into the trained neural network to obtain a gradient field g (X) corresponding to each point X in the degenerated point cloud X; and then, according to the gradient field of each point in the degradation point cloud X, carrying out iterative updating on the points in the degradation point cloud X through gradient ascending until reaching a set upper limit or convergence, and completing the restoration of the degradation point cloud X.
2. The method according to claim 1, wherein the iterative updating of points in the degraded point cloud to be sampled by gradient ascent is:wherein T is the set total number of iterative loops, alpha t Is a set super parameter; />Is the ith acquisition point X in the degenerated point cloud X i Coordinate value after updating at t-th iteration cycle,/->And updating the gradient field of the corresponding point calculated for the obtained coordinates according to the t-1 th cyclic iteration.
3. The method according to claim 2, characterized in that regularization terms are added in the iterative update, i.e. gradient ascent and optimization based on regularization terms are performed alternately; the iterative process is Wherein I represents an identity matrix, lambda is a super parameter, and L is a Laplacian matrix of a k-nearest neighbor graph generated based on an input point cloud. />Representing the intermediate result obtained in the t-th cycle, the remaining labels have the same meaning as in 2.
4. The method according to claim 1, wherein a dot is obtainedCorresponding gradient->The method of (1) is as follows: the gradient field estimation network first extracts each point +.>Each neighbor in the upper and lower Wen Dian cloud +.>Is>Then according to the adjacent point->Distance point->Distance of (2) is relative feature->Giving corresponding weight to obtain the point->Is>Then will->Inputting the global multi-layer perceptron, and estimating to obtain the point +.>Corresponding gradient->Wherein (1)>To at the point->Is the set of points in the neighborhood with a center radius r.
5. The method of claim 4, wherein the neighbor pointsDistance point->The farther apart the relative features areThe smaller the weight of (c).
6. The method of claim 4 or 5, wherein the relative features are determined using cosine annealingIs a weight of (2).
7. A server comprising a memory and a processor, the memory storing a computer program configured to be executed by the processor, the computer program comprising instructions for performing the steps of the method of any of claims 1 to 6.
8. A computer readable storage medium, on which a computer program is stored, characterized in that the computer program, when being executed by a processor, implements the steps of the method of any of claims 1 to 6.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210392112.0A CN116977187A (en) | 2022-04-14 | 2022-04-14 | Depth point set resampling method based on gradient field |
| PCT/CN2022/132439 WO2023197601A1 (en) | 2022-04-14 | 2022-11-17 | Gradient field-based point cloud repair method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210392112.0A CN116977187A (en) | 2022-04-14 | 2022-04-14 | Depth point set resampling method based on gradient field |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116977187A true CN116977187A (en) | 2023-10-31 |
Family
ID=88328861
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210392112.0A Pending CN116977187A (en) | 2022-04-14 | 2022-04-14 | Depth point set resampling method based on gradient field |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN116977187A (en) |
| WO (1) | WO2023197601A1 (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117789198B (en) * | 2024-02-28 | 2024-05-14 | 上海几何伙伴智能驾驶有限公司 | Method for realizing point cloud degradation detection based on 4D millimeter wave imaging radar |
| CN117853484B (en) * | 2024-03-05 | 2024-05-28 | 湖南建工交建宏特科技有限公司 | Intelligent bridge damage monitoring method and system based on vision |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP3457357B1 (en) * | 2017-09-13 | 2021-07-07 | Tata Consultancy Services Limited | Methods and systems for surface fitting based change detection in 3d point-cloud |
| CN113920271A (en) * | 2021-09-07 | 2022-01-11 | 北京理工大学 | Three-dimensional point cloud completion method, device and equipment |
| CN114067075A (en) * | 2021-09-27 | 2022-02-18 | 中国地质大学（武汉） | Point cloud completion method and device based on generation of countermeasure network |
| CN114048845B (en) * | 2022-01-14 | 2022-06-03 | 深圳大学 | Point cloud repairing method and device, computer equipment and storage medium |
-
2022
- 2022-04-14 CN CN202210392112.0A patent/CN116977187A/en active Pending
- 2022-11-17 WO PCT/CN2022/132439 patent/WO2023197601A1/en unknown
Also Published As
| Publication number | Publication date |
|---|---|
| WO2023197601A1 (en) | 2023-10-19 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Riegler et al. | A deep primal-dual network for guided depth super-resolution | |
| Min et al. | Depth video enhancement based on weighted mode filtering | |
| Duan et al. | 3D point cloud denoising via deep neural network based local surface estimation | |
| Chantas et al. | Bayesian restoration using a new nonstationary edge-preserving image prior | |
| CN111902825A (en) | Polygonal object labeling system and method for training object labeling system | |
| Potetz | Efficient belief propagation for vision using linear constraint nodes | |
| CN116977187A (en) | Depth point set resampling method based on gradient field | |
| Elad et al. | Image denoising: The deep learning revolution and beyond—a survey paper | |
| Jalobeanu et al. | An adaptive Gaussian model for satellite image deblurring | |
| Jellal et al. | LS-ELAS: Line segment based efficient large scale stereo matching | |
| Chen et al. | SAR image despeckling based on combination of fractional-order total variation and nonlocal low rank regularization | |
| CN110969105B (en) | Human body posture estimation method | |
| Pistilli et al. | Learning robust graph-convolutional representations for point cloud denoising | |
| EP2817783A1 (en) | Method and apparatus for mesh simplification | |
| CN112634149B (en) | Point cloud denoising method based on graph convolution network | |
| Chen et al. | Deep point set resampling via gradient fields | |
| CN115222625A (en) | Laser radar point cloud denoising method based on multi-scale noise | |
| He et al. | Grad-pu: Arbitrary-scale point cloud upsampling via gradient descent with learned distance functions | |
| CN111178142A (en) | Hand posture estimation method based on space-time context learning | |
| CN113177592A (en) | Image segmentation method and device, computer equipment and storage medium | |
| Angelopoulou et al. | Fast 2d/3d object representation with growing neural gas | |
| Firouznia et al. | Adaptive chaotic sampling particle filter to handle occlusion and fast motion in visual object tracking | |
| Lin et al. | A-SATMVSNet: An attention-aware multi-view stereo matching network based on satellite imagery | |
| CN116258877A (en) | Land utilization scene similarity change detection method, device, medium and equipment | |
| Amiri et al. | A fast video super resolution for facial image |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |