CN109147040A - Human body dot cloud hole method for repairing and mending based on template - Google Patents
Human body dot cloud hole method for repairing and mending based on template Download PDFInfo
- Publication number
- CN109147040A CN109147040A CN201810988717.XA CN201810988717A CN109147040A CN 109147040 A CN109147040 A CN 109147040A CN 201810988717 A CN201810988717 A CN 201810988717A CN 109147040 A CN109147040 A CN 109147040A
- Authority
- CN
- China
- Prior art keywords
- template
- human body
- point
- cloud
- point cloud
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 55
- 239000000203 mixture Substances 0.000 claims abstract description 8
- 230000011218 segmentation Effects 0.000 claims abstract description 8
- 238000012952 Resampling Methods 0.000 claims abstract description 7
- 239000000284 extract Substances 0.000 claims abstract 3
- 239000011159 matrix material Substances 0.000 claims description 25
- 238000005259 measurement Methods 0.000 claims description 17
- 230000006870 function Effects 0.000 claims description 14
- 238000005457 optimization Methods 0.000 claims description 13
- 230000009466 transformation Effects 0.000 claims description 13
- PXFBZOLANLWPMH-UHFFFAOYSA-N 16-Epiaffinine Natural products C1C(C2=CC=CC=C2N2)=C2C(=O)CC2C(=CC)CN(C)C1C2CO PXFBZOLANLWPMH-UHFFFAOYSA-N 0.000 claims description 9
- 230000008569 process Effects 0.000 claims description 8
- 238000004364 calculation method Methods 0.000 claims description 5
- 238000009499 grossing Methods 0.000 claims description 3
- 238000010606 normalization Methods 0.000 claims description 3
- 238000005070 sampling Methods 0.000 claims description 3
- 230000003044 adaptive effect Effects 0.000 claims description 2
- 238000012679 convergent method Methods 0.000 claims description 2
- 238000012804 iterative process Methods 0.000 claims description 2
- 230000009467 reduction Effects 0.000 claims description 2
- 230000008439 repair process Effects 0.000 abstract description 5
- 230000007704 transition Effects 0.000 abstract description 3
- 230000008901 benefit Effects 0.000 abstract description 2
- 230000037237 body shape Effects 0.000 abstract description 2
- 238000013459 approach Methods 0.000 description 5
- 230000008859 change Effects 0.000 description 2
- 238000010276 construction Methods 0.000 description 2
- 230000004069 differentiation Effects 0.000 description 2
- 238000000605 extraction Methods 0.000 description 2
- 230000006872 improvement Effects 0.000 description 2
- 230000001788 irregular Effects 0.000 description 2
- 238000013507 mapping Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000003796 beauty Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000007812 deficiency Effects 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 238000002310 reflectometry Methods 0.000 description 1
- 238000001228 spectrum Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
- G06T7/33—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10028—Range image; Depth image; 3D point clouds
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20212—Image combination
- G06T2207/20221—Image fusion; Image merging
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30196—Human being; Person
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Computer Graphics (AREA)
- Geometry (AREA)
- Software Systems (AREA)
- Image Processing (AREA)
Abstract
The invention discloses a kind of human body dot cloud hole method for repairing and mending based on template, comprising: human body grid model template is split according to the perforated in cloud, and some crucial character pair points in template and point cloud are marked；It extracts the human body point cloud with confidence level and expands boundary point set；By the template and human body point cloud local progress Rigid Registration after segmentation；Rigid Registration rear pattern plate is deformed, is allowed to match with human body point cloud local；The vertex extracted in deformed template constitutes template point cloud, and removes the template point cloud part Chong Die with original human body point cloud；Overlapping template point cloud resampling will be gone to original human body point cloud density, and smooth blend is to original human body dot cloud hole part.The dot cloud hole that human body dot cloud hole method for repairing and mending provided by the invention encounters when can carry out three-dimensional modeling to human body is repaired and guarantees that hole repair result meets body shape priori knowledge, has the advantages that repairing perforated transitions smooth, operation efficiency are high.
Description
Technical field
The present invention relates to the field of three-dimension modeling towards real-world object in computer graphics, and in particular to one kind is based on
The human body dot cloud hole method for repairing and mending of template.
Background technique
Equipment is obtained using three dimensional surface data, and various fields have been widely used in the object modeling of real world
In, a large amount of manual modelling operability can be saved and capture the details of more multi-model.And use three dimensional surface data data acquisition
Equipment model having and be paid close attention to due to it has huge application value by people to human body.It is set using three-dimensional data acquisition
Standby, such as triplex scanner, the three dimensional surface data of collected object is usually point cloud data, is by some discrete three
The set of dimension point composition.It generallys use three dimensional surface data and obtains the original human body data cloud that equipment obtains, because being set by acquisition
Standby visual angle limitation, human body surface rise and fall generate block, the influence for the factors such as regional area difference in reflectivity is larger, lead to people
Body surface face subregion is unable to measure, and incomplete measurement data forms hole.Such as under oxter, hip with regions such as soles be compared with
For common typical perforated, the hole in these regions has the characteristics that missing area is big, in irregular shape.These are by endless
Entire data formed hole not only influence beauty, but also the quality of the subsequent processings such as mesh reconstruction, texture mapping can be caused compared with
It is big to influence.Therefore, it needs rationally to be filled up using absent region of the holes repairing technology to data before practical application, it is ensured that
The integrality of data.
A considerable amount of various hole repair algorithms are currently, there are, such as the Hole Repairing in Triangular Meshes that Liepa is proposed is calculated
The hole repair algorithm etc. based on Poisson resurfacing that method, Centin et al. are proposed.In Publication No. CN107610061A
State's patent document discloses a kind of guarantor's edge point cloud hole repair method based on two-dimensional projection, by determining orderly dot cloud hole side
Boundary's point protects edge lengths projection in the fit Plane of dot cloud hole boundary point, then by subpoint using edge lengths sciagraphy is protected
It is sequentially connected, is formed to protect plane polygon view field of the edge lengths subpoint as vertex；Then area is carried out to view field
Domain is decomposed and point setization filling, bent using Interpolation Property of Radial Basis Function mapping techniques construction dot cloud hole according to dot cloud hole boundary point
Face, then filling point reflection is mapped on the dot cloud hole curved surface of construction, complete the repairing of dot cloud hole.However including above-mentioned algorithm
Inside, there are sizable deficiencies when handling large area, irregular holes for existing many algorithms.This is primarily due to these
Algorithm is to be estimated by the information of hole zone boundary neighborhood filling up for perforated, the hole area filled up if necessary
The original surface undulation in domain is larger, and perforated boundary neighborhood, which is not enough to provide, generates accurate hole repairing information needed.
On the other hand, the algorithm that fitting algorithm, such as Allen based on template deformation et al. proposes and Stoll et al. mention
Algorithm out is fitted collection point cloud using a complete template, and exports the template after fitting, reaches and closes to large area hole
Manage the effect of filling.However these methods are not only local comprising hole using the whole point cloud of complete body templates fitting
Region, this makes the robustness of algorithm lower, is easily trapped into locally optimal solution, causes fitting result that cannot be bonded input point everywhere
Cloud can destroy the data accuracy of non-perforated；In addition, the calculation amount being fitted to whole point cloud is larger, so that algorithm
Operation efficiency it is lower.
Summary of the invention
The purpose of the present invention is to provide a kind of human body dot cloud hole method for repairing and mending based on template, people provided by the invention
The dot cloud hole that body dot cloud hole method for repairing and mending encounters when can carry out three-dimensional modeling to human body is repaired and guarantees that hole is repaired
It mends result and meets body shape priori knowledge, have the advantages that repair perforated transitions smooth and operation efficiency is high.
The technical solution adopted by the present invention are as follows:
A kind of human body dot cloud hole method for repairing and mending based on template, comprising:
(1) human body grid model template is split according to the perforated in original human body point cloud, and to segmentation after
Human body grid model template be marked with the crucial character pair point in original human body point cloud；
(2) the original human body point cloud from original human body data reduction with confidence level expands boundary point set；
(3) by the human body grid model template and original human body point cloud local progress Rigid Registration after segmentation；
(4) the human body grid model template deformation after the Rigid Registration for obtaining step (3) is allowed to and original human body point cloud
Part matches；
(5) vertex in extraction step (4) in deformed human body grid model template constitutes template point cloud, and according to arriving
The method of boundary point set distance threshold removes part Chong Die with original human body point cloud in template point cloud, obtains overlapping template point
Cloud；
(6) go overlapping template point cloud resampling to original human body point cloud density for what step (5) obtained, and smooth blend arrives
Complete cloud of hole has been filled up in original human body dot cloud hole part.
In step (2), the method that the original human body point cloud with confidence level expands boundary point set is extracted are as follows:
(2-1) carries out boundary point differentiation using original point set of the combination distinguishing criterion to human body point cloud: using maximum angle
Amount criterion, semi-disc measurement criterion and shape measurements criterion calculate separately boundary point probability, and weighted sum obtains probability-weighted value,
Probability-weighted value is compared with given threshold value, then thinks that the point is boundary point greater than threshold value, obtains boundary point set
(2-2) is to boundary point setIt is expanded according to distance threshold, and calculates each pixel confidence cp, obtain with confidence
The original human body point cloud of degree expands boundary point set.
In step (2-1), the probability-weighted value are as follows:
P (p)=w1P1(p)+w2P2(p)+w3P3(p)
P1(p)、P2(p) and P3It (p) is quasi- using maximum angular measurement criterion, semi-disc measurement criterion and shape measurements respectively
The boundary point probability then calculated, w1、w2And w3For the corresponding weight of each criterion.
In step (2-1), the basic thought of maximum angular measurement criterion is: if the adjoint point collection N (p) of data point p is in office
Biggish notch is formd between subpoint in portion's tangent plane, then it is assumed that data point p is boundary point；Conversely, if N (p) was at p weeks
It encloses and is uniformly distributed, then it is assumed that p is interior point.
First according to data point p and its adjoint point NpConstruct local tangential plane of the point set curved surface at p；Then by adjoint point collection Np
It projects in tangent plane, obtains subpoint p 'i.Next each subpoint p ' is calculatediThe angle that line between p point is formed
Gather { αi, it is ranked up, maximum angle β=max (α is found outi), boundary point probability value is finally obtained according to the following formula:
In step (2-1), semi-disc measurement criterion is that hole boundary point is judged by the local space structure of point set
's.On two-dimensional manifold, a point p on curved surface is taken, if the point is interior point, neighborhood homeomorphism is in disk, then point p and its neighborhood
Average point μpDeviation should very little；, whereas if the point is boundary point, neighborhood homeomorphism is in semi-disc, then μpIt should be able to be with p
Position have relatively large deviation.
The adjoint point N of data point ppProjection in local tangential plane may be considered on two-dimensional manifold, by number to be determined
The neighboring mean value μ of strong point ppIt is compared with the centroid position of the ideal semi-disc in tangent plane, and with distance between the two
To measure the probability that p point belongs to boundary point.Wherein, using gaussian kernel function
To calculate neighborhood NpWeighted average μp, wherein σ depend on data point p to its adjoint point average distance rp, usually
It takesThus it can calculate
And to μpCalculate its and plane on projectionIt is then available that boundary is belonged to according to semi-disc criterion measurement p
The probability value of point:
In step (2-1), shape measurements criterion is the adjoint point N using data point ppConstruct corresponding covariance matrix Cp,
And boundary point is detected according to three of them characteristic value.For hole boundary point and internal point, association that neighborhood is constituted
Three feature Distribution values of variance matrix are different, so as to differentiate accordingly to boundary point.Its basic step is as follows:
First according to data point p and its adjoint point NpConstruct corresponding covariance matrix
Obtain three of them eigenvalue λ0≥λ1≥λ2>=0, composition decision vector is normalized to characteristic valueWherein α=λ0+λ1+λ2.The then corresponding four kinds of different decision vector distribution characteristics of the type of point p, point
The type of p is boundary point, interior point, angle point/noise spot, the point on straight line, and corresponding decision vector is respectively Λ4=(1,0,0).
Four kinds of distribution characteristics points of Λ can just form the triangle T comprising all probable values of ΛΛ, can pass through
Calculating space kernel function belongs to certain type of probability to data point and estimates near characteristic point.Use Gaussian kernel gσ,
Variance isBy normalization, it can be deduced that data point belongs to the general of boundary point
Rate value:
In step (2-2), expands and calculate each pixel confidence cpMethod are as follows:
A distance threshold τ is provided in advanced, and confidence interval [cL,cU], in the distance d≤τ for arriving holedRange
Interior, closer to hole boundary point, confidence level is lower, and boundary pixel confidence is set as cL；Further away from the point of hole, confidence level
It is higher；Distance is more than τdPoint, confidence level is set as maximum cU；Define point set any pointDistance to hole is should
Point arrives boundary point setThe minimum value of the distance of middle all the points, i.e.,
Then expand boundary point setAre as follows:
And on point set any point p confidence level cpMeet about dpFunctional relation,
cp=C (dp)=min (f (dp)(cU-cL)+cL,cU)
The valuation functions f (d) of distance to confidence level is that codomain is [0,1] monotonically increasing function,
By seeking original point setAll the points confidence level, available boundary point set?On with τdFor maximum
The expansion point set of distanceAnd have
In step (4), the mode of calculation template deformation is that iteration carries out corresponding points lookup and minimizes with minor function:
Iteration is restrained until T, i.e., iterates to calculate out twice | | T(k+1)-T(k)| | < ε, ε are the error threshold pre-set
Value.
Wherein, α, β, γ are pre-set parameter, V=[v1,…,vn]TThe square constituted for apex coordinates all in template
Battle array, U=[u1,…,un]TFor the matrix that the corresponding points coordinate that vertex in V is searched in cloud is constituted, W=diag
(w1,…,wn) be template to point cloud each group of corresponding points weighted value, Metzler matrix indicates the adjacent pass in template between vertex
System, for connection vertex to (vi,vj) side r, M r row nonzero term be Mri=-1, Mrj=1,For Kronecker product, I4
For 4 × 4 unit matrixs, VLWith ULRespectively the row of correspondence markings point in V is taken out, and the row of the corresponding points in U is taken out,
The matrix of composition.
Deform the principle of fitting are as follows: for each vertex v in templateiThere is corresponding 3 × 4 affine transformation square respectively
Battle array Ti, each affine transformation matrix includes 12 freedom degrees.Target is to find one group of affine transformation, can be by body templates song
FaceIn all the points be moved to deformed curved surfaceOn, so thatThe reference surface defined with human body point cloudSufficiently connect
Closely.
Enable T=[T1,…,Tn]TFor the affine transformation matrix T of vertex correspondence each in templateiThe matrix of the 4n connected into × 3,
The mode for calculating T is that iterative search deforms rear pattern plate vertex TiviCorresponding points u on to cloudi, and minimize above-mentioned energy letter
Number.
When minimizing energy function, it is noted that A is sequency spectrum, and Hessian matrix is nonsingular, therefore this is
One well-posed problem.By making the derivative of E (T) equal to 0, available system of linear equations, and E (T) is solved in T=(ATA)- 1ATB obtains minimum value.
In step (4), when being iterated optimization, to weighted value wiFollowing strategy is taken to be adjusted:
(4-1) is if uiFor the boundary point on original point cloud, then it is assumed that viLack part on corresponding original point cloud, will be right
Answer matched weighted value wiIt is set to 0 (thinking that there is no corresponding points in original point cloud)；
(4-2) is if TiviWith uiNormal vector angle it is excessive (be greater than specified threshold), then it is assumed that matching is unreliable, will be right
Answer matched weighted value wiIt is set to 0；
(4-3) for other matchings, use obtained in step (2) on point cloud every pixel confidence after normalization as
Weighted value, i.e.,
Because searching a point T on the deforming template being calculated using closest approach when being iterated optimizationiviTo a cloud
On corresponding points uiMatching result be not necessarily reliably, therefore take strategy it is adjusted.
Deformation fitting is carried out for each perforated by using the local template of segmentation respectively, we can reduce list
The scale of secondary Solve problems.And this method can be eliminated when the template far from perforated is fitted input point cloud pair
The contribution of energy function, so that the convergence rate of local fit is faster, error is smaller.
It further include accelerating convergence by multiresolution method and in optimization process in iterative process in step (4)
Different phase uses different weight parameters to energy term.
Because above-mentioned energy function E (T) has higher locality, convergence rate is slower, is easily trapped into local minimum, right
This can accelerate to restrain by multiresolution method；Simultaneously in the different phase of optimization process, the weight used energy term
Parameter is also different.
Described accelerates convergent method by multiresolution method are as follows: by carrying out differentiate to template mesh model more
Rate auto-adaptive parameter obtains the corresponding relationship in the version and these versions of different resolution between vertex；First at low point
Above-mentioned optimization process is carried out on the down-sampling of resolution template and original point cloud point cloud；Then affine transformation matrix is upsampled to
In high-resolution template, optimized as initial transformation, then to high-resolution template.
The method that the different phase in optimization process uses different weight parameters to energy term are as follows: rigid in iteration
When beginning, α is gradually increased as iteration carries out using lesser α and biggish β, γ, is missed by minimizing closest approach matching
Poor item allow template and human body original point cloud as far as possible close to.
In step (5), deformation is removed using according to the method for the expansion boundary point set distance threshold obtained to step (2)
The part Chong Die with human body point cloud on human body network model template afterwards.
The method of the removing template point cloud part Chong Die with original human body point cloud are as follows: for the point v on template point cloudi, make
It is searched to obtain its corresponding points u on original human body point cloud with closest approachiIf dist (vi,ui) < τmAndThen will
It is rejected；Wherein τmIt is a specified distance threshold,τdIt is
Specified expansion radius.
Using boundary expansion algorithm identical with step (2),For original point cloudBoundary point composition set, use
Radius τdBoundary is expanded, the expansion boundary point set with certain " thickness " is obtainedWherein τdIt is specified expansion radius.
The distance threshold τmSelection be sufficiently large, enable and correspond to the part of non-perforated well in template
It is removed, and because without rejecting against expansion boundary point setPart, point cloud by rejecting and original point cloud exist
Hole boundary has a degree of coincidence.
In step (6), the method for the complete cloud for having been filled up hole are as follows:
The expansion boundary point set that (6-1) obtains step (2)Chong Die template point cloud is gone to make with what step (5) obtained
For input, is handled using MLS smoothing algorithm, obtain merging point cloud；
(6-2) and by merging point cloud resampling, keeps the averag density of merging point cloud consistent with original point cloud, obtain a cloud
(6-3) will put cloud againBoundary point set is expanded with eliminatingOriginal point cloudMerge, has been filled up hole
Complete cloud:
In the present invention, original human body point cloud is also known as original point cloud or human body point cloud.
Compared with prior art, the beneficial effects of the present invention are embodied in following aspect:
(1) present invention is by combining expansion weighting boundary point set and being filled out using the method for template deformation to perforated
It fills, using the prior information of template, can handle the hole of complicated shape, and smooth in hole border transition, it can be preferably
Real human body is described in the shape in the region.
(2) by being split to manikin, by perforated piecemeal handle, and use multiresolution method with
Interim parameter, accelerates the convergence rate of deformation algorithm, and reduces the matched error of deformation.
Detailed description of the invention
Fig. 1 is the flow diagram of human body dot cloud hole method for repairing and mending provided by the invention.
Specific embodiment
Method provided by the invention is further illustrated with reference to the accompanying drawings and embodiments.
As shown in Figure 1, the implementation steps of the human body dot cloud hole method for repairing and mending provided in this embodiment based on template are as follows:
Step (1): human body grid model template is split according to the perforated in cloud, and to template and human body
Some crucial character pair points in point cloud are marked.
In the present embodiment, the typical hole in the human body point cloud human body three-dimensional reconstructing system based on passive vision obtained
Hole region is divided, such as: under the crown, oxter, hand, hip, sole.And 7-12 group is marked in each cut zone
Character pair point.
Step (2): the human body point cloud with confidence level is extracted from original human body point cloud and expands boundary point set.
(2-1) carries out boundary point differentiation using original point set of the combination distinguishing criterion to human body point cloud: using maximum angle
Amount criterion, semi-disc measurement criterion and shape measurements criterion calculate separately boundary point probability, and weighted sum obtains probability-weighted value,
Probability-weighted value is compared with given threshold value, then thinks that the point is boundary point greater than threshold value, obtains boundary point set
For the every bit in human body point cloud, using calculating maximum angular measurement criterion, semi-disc measurement criterion and shape degree
Amount criterion calculates separately the probability P that the point is boundary point1(p)、P2(p)、P3(p), weighted sum:
P (p)=w1P1(p)+w2P2(p)+w3P3(p)
And be compared probability-weighted value with given threshold value, then think that the point is boundary point greater than threshold value.It chooses herein
Weight is w1=w2=w3=1/3, threshold value 0.9.
(2-2) is to boundary point setIt is expanded according to distance threshold, and calculates each pixel confidence cp, obtain with confidence
The human body point cloud of degree expands boundary point set.
When calculating the expansion boundary point set for having confidence level, according to the equalization point spacing selected distance threshold value of cloud, this reality
Apply τ in exampled=12, confidence interval is set as [0,1]；Define point set any pointDistance to hole is the point to boundary
Point setThe minimum value of the distance of middle all the points, i.e.,
Then expand boundary point setAre as follows:
And on point set any point p confidence level cpMeet about dpFunctional relation,
Step (3): by the human body grid model template and original human body point cloud local progress Rigid Registration after segmentation.
The template after segmentation first specially is calculated to human body point cloud local using the corresponding points marked in step (1)
Thick rigid transformation matrix reuses ICP algorithm and is iterated optimization, obtain more accurate Rigid Registration as initial value.
Step (4): the human body grid model template deformation after the Rigid Registration that step (3) is obtained is allowed to and primitive man
Body point cloud local matches.
The mode of calculation template deformation is that iterative search deforms rear pattern plate vertex TiviCorresponding points u on to cloudi, and most
Smallization is with minor function:
Wherein, T=[T1,…,Tn]TFor the affine transformation matrix T of vertex correspondence each in templateiThe square of the 4n connected into × 3
Battle array, α, β, γ are pre-set parameter, V=[v1,…,vn]TFor the matrix that apex coordinates all in template are constituted, U=
[u1,…,un]TFor the matrix that the corresponding points coordinate that vertex in V is searched in cloud is constituted, W=diag (w1,…,wn) be
Template is to the weighted value for each group of corresponding points for putting cloud, and Metzler matrix indicates the syntople in template between vertex, for connection top
Point is to (vi,vj) side r, M r row nonzero term be Mri=-1, Mrj=1,For Kronecker product, I4For 4 × 4 unit squares
Battle array, VLWith ULRespectively the row of correspondence markings point in V is taken out, and the row of the corresponding points in U is taken out, the matrix of composition.
Closest approach lookup is accelerated using kd-tree, and the minimum of energy function passes through iterative calculation T=(ATA)- 1ATB is carried out, and until T convergence, i.e., is iterated to calculate out twice | | T(k+1)-T(k)| | iteration ends when < ε choose ε=10 herein-6。
When being iterated optimization, take following strategy to weighted value wiIt is adjusted:
1. if uiFor the boundary point on original point cloud, by the weighted value w of Corresponding matchingiIt is set to 0；
2. if TiviWith uiNormal vector angle be greater than threshold value, by the weighted value w of Corresponding matchingiIt is set to 0, the present embodiment
In the threshold value position 90 °；
3. for other matchings, use obtained in step (2) on point cloud every pixel confidence as weighted value, i.e. wi=ci。
To accelerate convergence, using multiresolution method: by carrying out multi-resolution adaptive parameter to template mesh model
Change, obtains the corresponding relationship in the version and these versions of different resolution between vertex.First in low resolution template and
Above-mentioned optimization process is carried out on the down-sampling point cloud of original point cloud, affine transformation matrix is then upsampled to high-resolution template
On, it is optimized as initial transformation, then to high-resolution template.In the present embodiment, using the template of Resolutions: face
The primary template of low resolution template and full resolution that the piece number halves.
To accelerate convergence, in the different phase of optimization process, different weight parameters is used to energy term.In the present embodiment
In, the parameter for having used 8 groups of unique steps to change, wherein α is from 0.4 to 1, and β is from 1 to 0.2, and γ is from 1 to 0.
Step (5): the vertex in extraction step (4) in deformed human body grid model template constitutes template point cloud, and
Part Chong Die with original human body point cloud in template point cloud is removed according to the method to boundary point set distance threshold, obtains being overlapped
Template point cloud.
Template point cloud is made of all vertex in deformed template.For the point v on template point cloudi, using being based on
The closest approach of kd-tree is searched to obtain its corresponding points u on original point cloudi.If dist (vi,ui) < τmAnd
Then rejected.Wherein τmIt is preset distance threshold, τ in the present embodimentm=1.5,For the boundary according to step (2)
The expansion boundary point set that Extension algorithm obtains.
Step (6): go overlapping template point cloud resampling to original human body point cloud density for what step (5) obtained, and smooth
It is fused to original human body dot cloud hole part, has been filled up complete cloud of hole.
The expansion boundary point set that step (2) is obtainedThe template point cloud obtained with step (5) is used as input
MLS smoothing algorithm is handled, and obtains merging point cloud, and by its resampling, is kept its equalization point spacing consistent with original point cloud, is obtained
To a cloudFinally, by its with eliminate expansion boundary point setOriginal point cloudMerge, has been filled up the complete of hole
Integral point cloud:
The foregoing is merely the preferred embodiment of the present invention, protection scope of the present invention is not limited in above-mentioned embodiment party
Formula, all technical solutions for belonging to the principle of the invention all belong to the scope of protection of the present invention.For those skilled in the art and
Speech, several improvements and modifications carried out without departing from the principles of the present invention, these improvements and modifications also should be regarded as this
The protection scope of invention.
Claims (9)
1. a kind of human body dot cloud hole method for repairing and mending based on template, comprising:
(1) human body grid model template is split according to the perforated in original human body point cloud, and to the people after segmentation
The template of volume mesh model is marked with the crucial character pair point in original human body point cloud；
(2) the original human body point cloud with confidence level is extracted from original human body data reduction expand boundary point set；
(3) by the template and original human body point cloud local progress Rigid Registration after segmentation；
(4) template deformation after the Rigid Registration for obtaining step (2) is allowed to match with original human body point cloud local；
(5) it extracts the vertex in deformed template and constitutes template point cloud, and according to the side for arriving expansion boundary point set distance threshold
Method removes part Chong Die with original human body point cloud in template point cloud, obtains overlapping template point cloud；
(6) go overlapping template point cloud resampling to original human body point cloud density for what step (5) obtained, and smooth blend is to original
Complete cloud of hole has been filled up in human body dot cloud hole part.
2. the human body dot cloud hole method for repairing and mending according to claim 1 based on template, which is characterized in that in step (2)
In, extract the method that the original human body point cloud with confidence level expands boundary point set are as follows:
(2-1) calculates separately boundary point probability using maximum angular measurement criterion, semi-disc measurement criterion and shape measurements criterion, adds
Power summation obtains probability-weighted value, and probability-weighted value is compared with given threshold value, then thinks that the point is boundary greater than threshold value
Point obtains boundary point set
(2-2) is to boundary point setIt is expanded according to distance threshold, and calculates each pixel confidence cp, obtain with confidence level
Original human body point cloud expands boundary point set.
3. the human body dot cloud hole method for repairing and mending according to claim 2 based on template, which is characterized in that in step (2-
2) it in, expands and calculates each pixel confidence cpMethod are as follows:
A distance threshold τ is provided in advanced, and confidence interval [cL,cU], in the distance d≤τ for arriving holedIn the range of, more
Point close to hole boundary, confidence level is lower, and boundary pixel confidence is set as cL；Further away from the point of hole, confidence level is higher；
Distance is more than τdPoint, confidence level is set as maximum cU；Define point set any pointDistance to hole is the point to side
Boundary's point setThe minimum value of the distance of middle all the points, i.e.,
Then expand boundary point setAre as follows:
And on point set any point p confidence level cpMeet about dpFunctional relation,
cp=C (dp)=min (f (dp)(cU-cL)+cL,cU)
The valuation functions f (d) of distance to confidence level is that codomain is [0,1] monotonically increasing function,
4. the human body dot cloud hole method for repairing and mending according to claim 1 based on template, which is characterized in that in step (4)
In, the mode of calculation template deformation is that iteration carries out corresponding points lookup and minimizes with minor function:
Wherein, T=[T1,…,Tn]TFor the affine transformation matrix T of vertex correspondence each in templateiThe matrix of the 4n connected into × 3, α,
β, γ are pre-set parameter, V=[v1,…,vn]TFor the matrix that apex coordinates all in template are constituted, U=[u1,…,un
]TFor the matrix that the corresponding points coordinate that vertex in V is searched in cloud is constituted, W=diag (w1,…,wn) it is template to point
The weighted value of each group of corresponding points of cloud, Metzler matrix indicate the syntople in template between vertex, for connection vertex to (vi,
vj) side r, M r row nonzero term be Mri=-1, Mrj=1,For Kronecker product, I4For 4 × 4 unit matrixs, VLWith UL
Respectively the row of correspondence markings point in V is taken out and the row of the corresponding points in U is taken out into the matrix formed；The termination item of iteration
Part are as follows: T convergence iterates to calculate out twice | | T(k+1)-T(k)| | < ε, ε are the error threshold pre-set.
5. the human body dot cloud hole method for repairing and mending according to claim 4 based on template, which is characterized in that in step (4)
In, when being iterated optimization, to weighted value wiFollowing strategy is taken to be adjusted:
(4-1) is if uiFor the boundary point on original point cloud, then it is assumed that viLack part on corresponding original point cloud, by corresponding
The weighted value w matchediIt is set to 0；
(4-2) is if TiviWith uiNormal vector angle it is excessive, then it is assumed that matching it is unreliable, by the weighted value w of Corresponding matchingiIt is set to
0；
(4-3) uses every pixel confidence on the expansion boundary point set of original human body point cloud obtained in step (2) for other matchings
Weighted value is used as after normalization, i.e.,
6. the human body dot cloud hole method for repairing and mending according to claim 4 based on template, which is characterized in that in step (4)
In, it further include accelerating to restrain by multiresolution method and making in the different phase of optimization process to energy term in iterative process
With different weight parameters.
7. the human body dot cloud hole method for repairing and mending according to claim 6 based on template, which is characterized in that described passes through
Multiresolution method accelerates convergent method are as follows: by carrying out multi-resolution adaptive parametrization to template mesh model, obtains
To the corresponding relationship in the version of different resolution and these versions between vertex；It is first in low resolution template and original
Above-mentioned optimization process is carried out on the down-sampling point cloud of human body point cloud；Then affine transformation matrix is upsampled to high-resolution template
On, it is optimized as initial transformation, then to high-resolution template.
8. the human body dot cloud hole method for repairing and mending according to claim 1 based on template, which is characterized in that in step (5)
In, the method for the removal template point cloud part Chong Die with original human body point cloud are as follows: for the point v on template point cloudi, using most
Near point is searched to obtain its corresponding points u on original human body point cloudiIf dist (vi,ui) < τmAndThen picked
It removes；Wherein τmIt is a specified distance threshold,τdIt is specified
Expansion radius.
9. the human body dot cloud hole method for repairing and mending according to claim 1 based on template, which is characterized in that in step (6)
In, the method for the complete cloud for having been filled up hole are as follows:
The expansion boundary point set that (6-1) obtains step (2)Go Chong Die template point cloud as defeated with what step (5) obtained
Enter, handled using MLS smoothing algorithm, obtains merging point cloud；
(6-2) and by merging point cloud resampling, makes the averag density and original point cloud consistent in density of merging point cloud, obtain a cloud
(6-3) will put cloud againBoundary point set is expanded with eliminatingOriginal human body point cloudMerge, has been filled up hole
Complete cloud:
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810988717.XA CN109147040B (en) | 2018-08-28 | 2018-08-28 | Template-based human point cloud hole repairing method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810988717.XA CN109147040B (en) | 2018-08-28 | 2018-08-28 | Template-based human point cloud hole repairing method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109147040A true CN109147040A (en) | 2019-01-04 |
| CN109147040B CN109147040B (en) | 2020-06-30 |
Family
ID=64828639
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810988717.XA Active CN109147040B (en) | 2018-08-28 | 2018-08-28 | Template-based human point cloud hole repairing method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109147040B (en) |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111444811A (en) * | 2020-03-23 | 2020-07-24 | 复旦大学 | Method for detecting three-dimensional point cloud target |
| CN111539446A (en) * | 2020-03-04 | 2020-08-14 | 南京航空航天大学 | 2D laser hole site detection method based on template matching |
| CN112215763A (en) * | 2020-09-01 | 2021-01-12 | 青岛秀山移动测量有限公司 | Planar internal missing point cloud repairing method based on image data assistance |
| CN112465903A (en) * | 2020-12-21 | 2021-03-09 | 上海交通大学宁波人工智能研究院 | 6DOF object attitude estimation method based on deep learning point cloud matching |
| CN112529891A (en) * | 2020-12-21 | 2021-03-19 | 深圳辰视智能科技有限公司 | Hollow hole identification and contour detection method and device based on point cloud and storage medium |
| CN114266798A (en) * | 2021-12-14 | 2022-04-01 | 上海应用技术大学 | Point cloud repairing method based on boundary point iteration |
| CN114333482A (en) * | 2022-01-07 | 2022-04-12 | 山东众阳健康科技集团有限公司 | Virtual anatomy teaching system based on mixed reality technology |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1885349A (en) * | 2006-07-05 | 2006-12-27 | 东南大学 | Point cloud hole repairing method for three-dimensional scanning |
| US8749551B2 (en) * | 2009-12-29 | 2014-06-10 | Hong Fu Jin Precision Industry (Shenzhen) Co., Ltd. | System and method for constructing triangular mesh surface of point cloud |
| CN104504660A (en) * | 2014-12-23 | 2015-04-08 | 北京数码大方科技股份有限公司 | Data processing method and data processing device used for removing voids in grid model |
| CN107464223A (en) * | 2017-07-19 | 2017-12-12 | 西安理工大学 | A kind of dot cloud hole method for repairing and mending based on section |
| CN107590825A (en) * | 2017-07-31 | 2018-01-16 | 东南大学 | A kind of dot cloud hole method for repairing and mending based on SFM |
| CN107610061A (en) * | 2017-08-30 | 2018-01-19 | 西安理工大学 | A kind of guarantor's edge point cloud hole repair method based on two-dimensional projection |
| CN107767457A (en) * | 2017-10-09 | 2018-03-06 | 东南大学 | A kind of STL digital-to-analogue generation methods quickly rebuild based on a cloud |
-
2018
- 2018-08-28 CN CN201810988717.XA patent/CN109147040B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1885349A (en) * | 2006-07-05 | 2006-12-27 | 东南大学 | Point cloud hole repairing method for three-dimensional scanning |
| US8749551B2 (en) * | 2009-12-29 | 2014-06-10 | Hong Fu Jin Precision Industry (Shenzhen) Co., Ltd. | System and method for constructing triangular mesh surface of point cloud |
| CN104504660A (en) * | 2014-12-23 | 2015-04-08 | 北京数码大方科技股份有限公司 | Data processing method and data processing device used for removing voids in grid model |
| CN107464223A (en) * | 2017-07-19 | 2017-12-12 | 西安理工大学 | A kind of dot cloud hole method for repairing and mending based on section |
| CN107590825A (en) * | 2017-07-31 | 2018-01-16 | 东南大学 | A kind of dot cloud hole method for repairing and mending based on SFM |
| CN107610061A (en) * | 2017-08-30 | 2018-01-19 | 西安理工大学 | A kind of guarantor's edge point cloud hole repair method based on two-dimensional projection |
| CN107767457A (en) * | 2017-10-09 | 2018-03-06 | 东南大学 | A kind of STL digital-to-analogue generation methods quickly rebuild based on a cloud |
Non-Patent Citations (1)
| Title |
|---|
| 曾露露: "基于从运动中恢复结构的三维点云孔洞修补算法研究", 《光学学报》 * |
Cited By (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111539446A (en) * | 2020-03-04 | 2020-08-14 | 南京航空航天大学 | 2D laser hole site detection method based on template matching |
| CN111539446B (en) * | 2020-03-04 | 2023-10-03 | 南京航空航天大学 | Template matching-based 2D laser hole site detection method |
| CN111444811A (en) * | 2020-03-23 | 2020-07-24 | 复旦大学 | Method for detecting three-dimensional point cloud target |
| CN111444811B (en) * | 2020-03-23 | 2023-04-28 | 复旦大学 | Three-dimensional point cloud target detection method |
| CN112215763A (en) * | 2020-09-01 | 2021-01-12 | 青岛秀山移动测量有限公司 | Planar internal missing point cloud repairing method based on image data assistance |
| CN112215763B (en) * | 2020-09-01 | 2023-02-24 | 青岛秀山移动测量有限公司 | Planar internal missing point cloud repairing method based on image data assistance |
| CN112465903A (en) * | 2020-12-21 | 2021-03-09 | 上海交通大学宁波人工智能研究院 | 6DOF object attitude estimation method based on deep learning point cloud matching |
| CN112529891A (en) * | 2020-12-21 | 2021-03-19 | 深圳辰视智能科技有限公司 | Hollow hole identification and contour detection method and device based on point cloud and storage medium |
| CN112529891B (en) * | 2020-12-21 | 2024-03-19 | 深圳辰视智能科技有限公司 | Method and device for identifying hollow holes and detecting contours based on point cloud and storage medium |
| CN114266798B (en) * | 2021-12-14 | 2023-09-22 | 上海应用技术大学 | Point cloud restoration method based on boundary point iteration |
| CN114266798A (en) * | 2021-12-14 | 2022-04-01 | 上海应用技术大学 | Point cloud repairing method based on boundary point iteration |
| CN114333482A (en) * | 2022-01-07 | 2022-04-12 | 山东众阳健康科技集团有限公司 | Virtual anatomy teaching system based on mixed reality technology |
| CN114333482B (en) * | 2022-01-07 | 2024-05-24 | 众阳健康科技集团有限公司 | Virtual anatomy teaching system based on mixed reality technology |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109147040B (en) | 2020-06-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109147040A (en) | Human body dot cloud hole method for repairing and mending based on template | |
| CN109767463B (en) | Automatic registration method for three-dimensional point cloud | |
| CN104036544B (en) | A kind of building roof method for reconstructing based on on-board LiDAR data | |
| CN102592136B (en) | Three-dimensional human face recognition method based on intermediate frequency information in geometry image | |
| CN106250895B (en) | A kind of remote sensing image region of interest area detecting method | |
| CN114972377A (en) | 3D point cloud segmentation method and device based on moving least square method and hyper-voxels | |
| CN107330901A (en) | A kind of object component decomposition method based on skeleton | |
| CN105678747A (en) | Tooth mesh model automatic segmentation method based on principal curvature | |
| CN109034131A (en) | A kind of semi-automatic face key point mask method and storage medium | |
| CN112200905A (en) | Three-dimensional face completion method | |
| Ma et al. | Kinematic skeleton extraction from 3D articulated models | |
| CN114511575B (en) | Grabbing method for high-reflectivity object based on image segmentation positioning auxiliary point cloud registration | |
| CN109829489A (en) | A kind of cultural relic fragments recombination method and device based on multilayer feature | |
| CN117237428B (en) | Data registration method, device and medium for three-dimensional point cloud | |
| CN112614233B (en) | Fragment splicing method under VR virtual environment | |
| CN110288517A (en) | Skeleton line extracting method based on projection matching group | |
| CN117314786A (en) | Three-dimensional grid repairing method based on deep learning | |
| da Fontoura Costa | Robust skeletonization through exact Euclidean distance transform and its application to neuromorphometry | |
| CN108257118A (en) | A kind of fracture adhesion dividing method based on normal direction corrosion and random walk | |
| Omidalizarandi et al. | Segmentation and classification of point clouds from dense aerial image matching | |
| Palmer et al. | Locating boundaries of textured regions | |
| CN109544578B (en) | Three-dimensional fragment recombination restoration method based on maximum similarity matching | |
| CN116740077A (en) | Plant leaf segmentation method based on three-dimensional point cloud | |
| CN115909099A (en) | Side slope dangerous rock identification and monitoring method based on unmanned aerial vehicle inspection | |
| CN107356968A (en) | Three-dimensional level set tomography curved surface extraction method based on crease |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |