CN109147040A - Human body dot cloud hole method for repairing and mending based on template - Google Patents

Human body dot cloud hole method for repairing and mending based on template Download PDF

Info

Publication number
CN109147040A
CN109147040A CN201810988717.XA CN201810988717A CN109147040A CN 109147040 A CN109147040 A CN 109147040A CN 201810988717 A CN201810988717 A CN 201810988717A CN 109147040 A CN109147040 A CN 109147040A
Authority
CN
China
Prior art keywords
point cloud
template
point
human body
hole
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201810988717.XA
Other languages
Chinese (zh)
Other versions
CN109147040B (en
Inventor
冯结青
康鋆鹏
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Zhejiang University ZJU
Original Assignee
Zhejiang University ZJU
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Zhejiang University ZJU filed Critical Zhejiang University ZJU
Priority to CN201810988717.XA priority Critical patent/CN109147040B/en
Publication of CN109147040A publication Critical patent/CN109147040A/en
Application granted granted Critical
Publication of CN109147040B publication Critical patent/CN109147040B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
    • G06T7/33—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10028—Range image; Depth image; 3D point clouds
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20212—Image combination
    • G06T2207/20221—Image fusion; Image merging
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30196—Human being; Person

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Computer Graphics (AREA)
  • Geometry (AREA)
  • Software Systems (AREA)
  • Image Processing (AREA)

Abstract

本发明公开了一种基于模板的人体点云孔洞修补方法，包括：根据点云中的孔洞区域对人体网格模型模板进行分割，并对模板与点云中的一些关键对应特征点进行标记；提取带有置信度的人体点云扩张边界点集；将分割后的模板与人体点云局部进行刚性配准；将刚性配准后模板变形，使之与人体点云局部相匹配；提取变形后的模板上的顶点构成模板点云，并移除模板点云与原始人体点云重叠的部分；将去重叠模板点云重采样到原始人体点云密度，并平滑融合到原始人体点云孔洞部分。本发明提供的人体点云孔洞修补方法能对人体进行三维建模时遇到的点云孔洞进行修补并保证孔洞修补结果符合人体形状先验知识，具有修补孔洞区域过渡平滑、运算效率高的优点。

The invention discloses a template-based human body point cloud hole repairing method, comprising: segmenting a human body mesh model template according to the hole area in the point cloud, and marking some key corresponding feature points in the template and the point cloud; Extract the human body point cloud expansion boundary point set with confidence; perform rigid registration of the segmented template with the human body point cloud part; deform the rigidly registered template to match it with the human body point cloud part; extract the deformed template The vertices on the template form the template point cloud, and remove the overlapping part of the template point cloud and the original human point cloud; resample the de-overlapping template point cloud to the density of the original human point cloud, and smoothly merge it into the hole part of the original human point cloud . The human body point cloud hole repairing method provided by the invention can repair the point cloud holes encountered in the three-dimensional modeling of the human body and ensure that the hole repair result conforms to the prior knowledge of the human body shape, and has the advantages of smooth transition of the repaired hole area and high computing efficiency. .

Description

Human body dot cloud hole method for repairing and mending based on template
Technical field
The present invention relates to the field of three-dimension modeling towards real-world object in computer graphics, and in particular to one kind is based on The human body dot cloud hole method for repairing and mending of template.
Background technique
Equipment is obtained using three dimensional surface data, and various fields have been widely used in the object modeling of real world In, a large amount of manual modelling operability can be saved and capture the details of more multi-model.And use three dimensional surface data data acquisition Equipment model having and be paid close attention to due to it has huge application value by people to human body.It is set using three-dimensional data acquisition Standby, such as triplex scanner, the three dimensional surface data of collected object is usually point cloud data, is by some discrete three The set of dimension point composition.It generallys use three dimensional surface data and obtains the original human body data cloud that equipment obtains, because being set by acquisition Standby visual angle limitation, human body surface rise and fall generate block, the influence for the factors such as regional area difference in reflectivity is larger, lead to people Body surface face subregion is unable to measure, and incomplete measurement data forms hole.Such as under oxter, hip with regions such as soles be compared with For common typical perforated, the hole in these regions has the characteristics that missing area is big, in irregular shape.These are by endless Entire data formed hole not only influence beauty, but also the quality of the subsequent processings such as mesh reconstruction, texture mapping can be caused compared with It is big to influence.Therefore, it needs rationally to be filled up using absent region of the holes repairing technology to data before practical application, it is ensured that The integrality of data.
A considerable amount of various hole repair algorithms are currently, there are, such as the Hole Repairing in Triangular Meshes that Liepa is proposed is calculated The hole repair algorithm etc. based on Poisson resurfacing that method, Centin et al. are proposed.In Publication No. CN107610061A State's patent document discloses a kind of guarantor's edge point cloud hole repair method based on two-dimensional projection, by determining orderly dot cloud hole side Boundary's point protects edge lengths projection in the fit Plane of dot cloud hole boundary point, then by subpoint using edge lengths sciagraphy is protected It is sequentially connected, is formed to protect plane polygon view field of the edge lengths subpoint as vertex；Then area is carried out to view field Domain is decomposed and point setization filling, bent using Interpolation Property of Radial Basis Function mapping techniques construction dot cloud hole according to dot cloud hole boundary point Face, then filling point reflection is mapped on the dot cloud hole curved surface of construction, complete the repairing of dot cloud hole.However including above-mentioned algorithm Inside, there are sizable deficiencies when handling large area, irregular holes for existing many algorithms.This is primarily due to these Algorithm is to be estimated by the information of hole zone boundary neighborhood filling up for perforated, the hole area filled up if necessary The original surface undulation in domain is larger, and perforated boundary neighborhood, which is not enough to provide, generates accurate hole repairing information needed.
On the other hand, the algorithm that fitting algorithm, such as Allen based on template deformation et al. proposes and Stoll et al. mention Algorithm out is fitted collection point cloud using a complete template, and exports the template after fitting, reaches and closes to large area hole Manage the effect of filling.However these methods are not only local comprising hole using the whole point cloud of complete body templates fitting Region, this makes the robustness of algorithm lower, is easily trapped into locally optimal solution, causes fitting result that cannot be bonded input point everywhere Cloud can destroy the data accuracy of non-perforated；In addition, the calculation amount being fitted to whole point cloud is larger, so that algorithm Operation efficiency it is lower.
Summary of the invention
The purpose of the present invention is to provide a kind of human body dot cloud hole method for repairing and mending based on template, people provided by the invention The dot cloud hole that body dot cloud hole method for repairing and mending encounters when can carry out three-dimensional modeling to human body is repaired and guarantees that hole is repaired It mends result and meets body shape priori knowledge, have the advantages that repair perforated transitions smooth and operation efficiency is high.
The technical solution adopted by the present invention are as follows:
A kind of human body dot cloud hole method for repairing and mending based on template, comprising:
(1) human body grid model template is split according to the perforated in original human body point cloud, and to segmentation after Human body grid model template be marked with the crucial character pair point in original human body point cloud；
(2) the original human body point cloud from original human body data reduction with confidence level expands boundary point set；
(3) by the human body grid model template and original human body point cloud local progress Rigid Registration after segmentation；
(4) the human body grid model template deformation after the Rigid Registration for obtaining step (3) is allowed to and original human body point cloud Part matches；
(5) vertex in extraction step (4) in deformed human body grid model template constitutes template point cloud, and according to arriving The method of boundary point set distance threshold removes part Chong Die with original human body point cloud in template point cloud, obtains overlapping template point Cloud；
(6) go overlapping template point cloud resampling to original human body point cloud density for what step (5) obtained, and smooth blend arrives Complete cloud of hole has been filled up in original human body dot cloud hole part.
In step (2), the method that the original human body point cloud with confidence level expands boundary point set is extracted are as follows:
(2-1) carries out boundary point differentiation using original point set of the combination distinguishing criterion to human body point cloud: using maximum angle Amount criterion, semi-disc measurement criterion and shape measurements criterion calculate separately boundary point probability, and weighted sum obtains probability-weighted value, Probability-weighted value is compared with given threshold value, then thinks that the point is boundary point greater than threshold value, obtains boundary point set
(2-2) is to boundary point setIt is expanded according to distance threshold, and calculates each pixel confidence cp, obtain with confidence The original human body point cloud of degree expands boundary point set.
In step (2-1), the probability-weighted value are as follows:
P (p)=w1P1(p)+w2P2(p)+w3P3(p)
P1(p)、P2(p) and P3It (p) is quasi- using maximum angular measurement criterion, semi-disc measurement criterion and shape measurements respectively The boundary point probability then calculated, w1、w2And w3For the corresponding weight of each criterion.
In step (2-1), the basic thought of maximum angular measurement criterion is: if the adjoint point collection N (p) of data point p is in office Biggish notch is formd between subpoint in portion's tangent plane, then it is assumed that data point p is boundary point；Conversely, if N (p) was at p weeks It encloses and is uniformly distributed, then it is assumed that p is interior point.
First according to data point p and its adjoint point NpConstruct local tangential plane of the point set curved surface at p；Then by adjoint point collection Np It projects in tangent plane, obtains subpoint p 'i.Next each subpoint p ' is calculatediThe angle that line between p point is formed Gather { αi, it is ranked up, maximum angle β=max (α is found outi), boundary point probability value is finally obtained according to the following formula:
In step (2-1), semi-disc measurement criterion is that hole boundary point is judged by the local space structure of point set 's.On two-dimensional manifold, a point p on curved surface is taken, if the point is interior point, neighborhood homeomorphism is in disk, then point p and its neighborhood Average point μpDeviation should very little；, whereas if the point is boundary point, neighborhood homeomorphism is in semi-disc, then μpIt should be able to be with p Position have relatively large deviation.
The adjoint point N of data point ppProjection in local tangential plane may be considered on two-dimensional manifold, by number to be determined The neighboring mean value μ of strong point ppIt is compared with the centroid position of the ideal semi-disc in tangent plane, and with distance between the two To measure the probability that p point belongs to boundary point.Wherein, using gaussian kernel function
To calculate neighborhood NpWeighted average μp, wherein σ depend on data point p to its adjoint point average distance rp, usually It takesThus it can calculate
And to μpCalculate its and plane on projectionIt is then available that boundary is belonged to according to semi-disc criterion measurement p The probability value of point:
In step (2-1), shape measurements criterion is the adjoint point N using data point ppConstruct corresponding covariance matrix Cp, And boundary point is detected according to three of them characteristic value.For hole boundary point and internal point, association that neighborhood is constituted Three feature Distribution values of variance matrix are different, so as to differentiate accordingly to boundary point.Its basic step is as follows:
First according to data point p and its adjoint point NpConstruct corresponding covariance matrix
Obtain three of them eigenvalue λ0≥λ1≥λ2>=0, composition decision vector is normalized to characteristic valueWherein α=λ0+λ1+λ2.The then corresponding four kinds of different decision vector distribution characteristics of the type of point p, point The type of p is boundary point, interior point, angle point/noise spot, the point on straight line, and corresponding decision vector is respectively Λ4=(1,0,0).
Four kinds of distribution characteristics points of Λ can just form the triangle T comprising all probable values of ΛΛ, can pass through Calculating space kernel function belongs to certain type of probability to data point and estimates near characteristic point.Use Gaussian kernel gσ, Variance isBy normalization, it can be deduced that data point belongs to the general of boundary point Rate value:
In step (2-2), expands and calculate each pixel confidence cpMethod are as follows:
A distance threshold τ is provided in advanced, and confidence interval [cL,cU], in the distance d≤τ for arriving holedRange Interior, closer to hole boundary point, confidence level is lower, and boundary pixel confidence is set as cL；Further away from the point of hole, confidence level It is higher；Distance is more than τdPoint, confidence level is set as maximum cU；Define point set any pointDistance to hole is should Point arrives boundary point setThe minimum value of the distance of middle all the points, i.e.,
Then expand boundary point setAre as follows:
And on point set any point p confidence level cpMeet about dpFunctional relation,
cp=C (dp)=min (f (dp)(cU-cL)+cL,cU)
The valuation functions f (d) of distance to confidence level is that codomain is [0,1] monotonically increasing function,
By seeking original point setAll the points confidence level, available boundary point set?On with τdFor maximum The expansion point set of distanceAnd have
In step (4), the mode of calculation template deformation is that iteration carries out corresponding points lookup and minimizes with minor function:
Iteration is restrained until T, i.e., iterates to calculate out twice | | T(k+1)-T(k)| | < ε, ε are the error threshold pre-set Value.
Wherein, α, β, γ are pre-set parameter, V=[v1,…,vn]TThe square constituted for apex coordinates all in template Battle array, U=[u1,…,un]TFor the matrix that the corresponding points coordinate that vertex in V is searched in cloud is constituted, W=diag (w1,…,wn) be template to point cloud each group of corresponding points weighted value, Metzler matrix indicates the adjacent pass in template between vertex System, for connection vertex to (vi,vj) side r, M r row nonzero term be Mri=-1, Mrj=1,For Kronecker product, I4 For 4 × 4 unit matrixs, VLWith ULRespectively the row of correspondence markings point in V is taken out, and the row of the corresponding points in U is taken out, The matrix of composition.
Deform the principle of fitting are as follows: for each vertex v in templateiThere is corresponding 3 × 4 affine transformation square respectively Battle array Ti, each affine transformation matrix includes 12 freedom degrees.Target is to find one group of affine transformation, can be by body templates song FaceIn all the points be moved to deformed curved surfaceOn, so thatThe reference surface defined with human body point cloudSufficiently connect Closely.
Enable T=[T1,…,Tn]TFor the affine transformation matrix T of vertex correspondence each in templateiThe matrix of the 4n connected into × 3, The mode for calculating T is that iterative search deforms rear pattern plate vertex TiviCorresponding points u on to cloudi, and minimize above-mentioned energy letter Number.
When minimizing energy function, it is noted that A is sequency spectrum, and Hessian matrix is nonsingular, therefore this is One well-posed problem.By making the derivative of E (T) equal to 0, available system of linear equations, and E (T) is solved in T=(ATA)- 1ATB obtains minimum value.
In step (4), when being iterated optimization, to weighted value wiFollowing strategy is taken to be adjusted:
(4-1) is if uiFor the boundary point on original point cloud, then it is assumed that viLack part on corresponding original point cloud, will be right Answer matched weighted value wiIt is set to 0 (thinking that there is no corresponding points in original point cloud)；
(4-2) is if TiviWith uiNormal vector angle it is excessive (be greater than specified threshold), then it is assumed that matching is unreliable, will be right Answer matched weighted value wiIt is set to 0；
(4-3) for other matchings, use obtained in step (2) on point cloud every pixel confidence after normalization as Weighted value, i.e.,
Because searching a point T on the deforming template being calculated using closest approach when being iterated optimizationiviTo a cloud On corresponding points uiMatching result be not necessarily reliably, therefore take strategy it is adjusted.
Deformation fitting is carried out for each perforated by using the local template of segmentation respectively, we can reduce list The scale of secondary Solve problems.And this method can be eliminated when the template far from perforated is fitted input point cloud pair The contribution of energy function, so that the convergence rate of local fit is faster, error is smaller.
It further include accelerating convergence by multiresolution method and in optimization process in iterative process in step (4) Different phase uses different weight parameters to energy term.
Because above-mentioned energy function E (T) has higher locality, convergence rate is slower, is easily trapped into local minimum, right This can accelerate to restrain by multiresolution method；Simultaneously in the different phase of optimization process, the weight used energy term Parameter is also different.
Described accelerates convergent method by multiresolution method are as follows: by carrying out differentiate to template mesh model more Rate auto-adaptive parameter obtains the corresponding relationship in the version and these versions of different resolution between vertex；First at low point Above-mentioned optimization process is carried out on the down-sampling of resolution template and original point cloud point cloud；Then affine transformation matrix is upsampled to In high-resolution template, optimized as initial transformation, then to high-resolution template.
The method that the different phase in optimization process uses different weight parameters to energy term are as follows: rigid in iteration When beginning, α is gradually increased as iteration carries out using lesser α and biggish β, γ, is missed by minimizing closest approach matching Poor item allow template and human body original point cloud as far as possible close to.
In step (5), deformation is removed using according to the method for the expansion boundary point set distance threshold obtained to step (2) The part Chong Die with human body point cloud on human body network model template afterwards.
The method of the removing template point cloud part Chong Die with original human body point cloud are as follows: for the point v on template point cloudi, make It is searched to obtain its corresponding points u on original human body point cloud with closest approachiIf dist (vi,ui) < τmAndThen will It is rejected；Wherein τmIt is a specified distance threshold,τdIt is Specified expansion radius.
Using boundary expansion algorithm identical with step (2),For original point cloudBoundary point composition set, use Radius τdBoundary is expanded, the expansion boundary point set with certain " thickness " is obtainedWherein τdIt is specified expansion radius.
The distance threshold τmSelection be sufficiently large, enable and correspond to the part of non-perforated well in template It is removed, and because without rejecting against expansion boundary point setPart, point cloud by rejecting and original point cloud exist Hole boundary has a degree of coincidence.
In step (6), the method for the complete cloud for having been filled up hole are as follows:
The expansion boundary point set that (6-1) obtains step (2)Chong Die template point cloud is gone to make with what step (5) obtained For input, is handled using MLS smoothing algorithm, obtain merging point cloud；
(6-2) and by merging point cloud resampling, keeps the averag density of merging point cloud consistent with original point cloud, obtain a cloud
(6-3) will put cloud againBoundary point set is expanded with eliminatingOriginal point cloudMerge, has been filled up hole Complete cloud:
In the present invention, original human body point cloud is also known as original point cloud or human body point cloud.
Compared with prior art, the beneficial effects of the present invention are embodied in following aspect:
(1) present invention is by combining expansion weighting boundary point set and being filled out using the method for template deformation to perforated It fills, using the prior information of template, can handle the hole of complicated shape, and smooth in hole border transition, it can be preferably Real human body is described in the shape in the region.
(2) by being split to manikin, by perforated piecemeal handle, and use multiresolution method with Interim parameter, accelerates the convergence rate of deformation algorithm, and reduces the matched error of deformation.
Detailed description of the invention
Fig. 1 is the flow diagram of human body dot cloud hole method for repairing and mending provided by the invention.
Specific embodiment
Method provided by the invention is further illustrated with reference to the accompanying drawings and embodiments.
As shown in Figure 1, the implementation steps of the human body dot cloud hole method for repairing and mending provided in this embodiment based on template are as follows:
Step (1): human body grid model template is split according to the perforated in cloud, and to template and human body Some crucial character pair points in point cloud are marked.
In the present embodiment, the typical hole in the human body point cloud human body three-dimensional reconstructing system based on passive vision obtained Hole region is divided, such as: under the crown, oxter, hand, hip, sole.And 7-12 group is marked in each cut zone Character pair point.
Step (2): the human body point cloud with confidence level is extracted from original human body point cloud and expands boundary point set.
(2-1) carries out boundary point differentiation using original point set of the combination distinguishing criterion to human body point cloud: using maximum angle Amount criterion, semi-disc measurement criterion and shape measurements criterion calculate separately boundary point probability, and weighted sum obtains probability-weighted value, Probability-weighted value is compared with given threshold value, then thinks that the point is boundary point greater than threshold value, obtains boundary point set
For the every bit in human body point cloud, using calculating maximum angular measurement criterion, semi-disc measurement criterion and shape degree Amount criterion calculates separately the probability P that the point is boundary point1(p)、P2(p)、P3(p), weighted sum:
P (p)=w1P1(p)+w2P2(p)+w3P3(p)
And be compared probability-weighted value with given threshold value, then think that the point is boundary point greater than threshold value.It chooses herein Weight is w1=w2=w3=1/3, threshold value 0.9.
(2-2) is to boundary point setIt is expanded according to distance threshold, and calculates each pixel confidence cp, obtain with confidence The human body point cloud of degree expands boundary point set.
When calculating the expansion boundary point set for having confidence level, according to the equalization point spacing selected distance threshold value of cloud, this reality Apply τ in exampled=12, confidence interval is set as [0,1]；Define point set any pointDistance to hole is the point to boundary Point setThe minimum value of the distance of middle all the points, i.e.,
Then expand boundary point setAre as follows:
And on point set any point p confidence level cpMeet about dpFunctional relation,
Step (3): by the human body grid model template and original human body point cloud local progress Rigid Registration after segmentation.
The template after segmentation first specially is calculated to human body point cloud local using the corresponding points marked in step (1) Thick rigid transformation matrix reuses ICP algorithm and is iterated optimization, obtain more accurate Rigid Registration as initial value.
Step (4): the human body grid model template deformation after the Rigid Registration that step (3) is obtained is allowed to and primitive man Body point cloud local matches.
The mode of calculation template deformation is that iterative search deforms rear pattern plate vertex TiviCorresponding points u on to cloudi, and most Smallization is with minor function:
Wherein, T=[T1,…,Tn]TFor the affine transformation matrix T of vertex correspondence each in templateiThe square of the 4n connected into × 3 Battle array, α, β, γ are pre-set parameter, V=[v1,…,vn]TFor the matrix that apex coordinates all in template are constituted, U= [u1,…,un]TFor the matrix that the corresponding points coordinate that vertex in V is searched in cloud is constituted, W=diag (w1,…,wn) be Template is to the weighted value for each group of corresponding points for putting cloud, and Metzler matrix indicates the syntople in template between vertex, for connection top Point is to (vi,vj) side r, M r row nonzero term be Mri=-1, Mrj=1,For Kronecker product, I4For 4 × 4 unit squares Battle array, VLWith ULRespectively the row of correspondence markings point in V is taken out, and the row of the corresponding points in U is taken out, the matrix of composition.
Closest approach lookup is accelerated using kd-tree, and the minimum of energy function passes through iterative calculation T=(ATA)- 1ATB is carried out, and until T convergence, i.e., is iterated to calculate out twice | | T(k+1)-T(k)| | iteration ends when < ε choose ε=10 herein-6。
When being iterated optimization, take following strategy to weighted value wiIt is adjusted:
1. if uiFor the boundary point on original point cloud, by the weighted value w of Corresponding matchingiIt is set to 0；
2. if TiviWith uiNormal vector angle be greater than threshold value, by the weighted value w of Corresponding matchingiIt is set to 0, the present embodiment In the threshold value position 90 °；
3. for other matchings, use obtained in step (2) on point cloud every pixel confidence as weighted value, i.e. wi=ci。
To accelerate convergence, using multiresolution method: by carrying out multi-resolution adaptive parameter to template mesh model Change, obtains the corresponding relationship in the version and these versions of different resolution between vertex.First in low resolution template and Above-mentioned optimization process is carried out on the down-sampling point cloud of original point cloud, affine transformation matrix is then upsampled to high-resolution template On, it is optimized as initial transformation, then to high-resolution template.In the present embodiment, using the template of Resolutions: face The primary template of low resolution template and full resolution that the piece number halves.
To accelerate convergence, in the different phase of optimization process, different weight parameters is used to energy term.In the present embodiment In, the parameter for having used 8 groups of unique steps to change, wherein α is from 0.4 to 1, and β is from 1 to 0.2, and γ is from 1 to 0.
Step (5): the vertex in extraction step (4) in deformed human body grid model template constitutes template point cloud, and Part Chong Die with original human body point cloud in template point cloud is removed according to the method to boundary point set distance threshold, obtains being overlapped Template point cloud.
Template point cloud is made of all vertex in deformed template.For the point v on template point cloudi, using being based on The closest approach of kd-tree is searched to obtain its corresponding points u on original point cloudi.If dist (vi,ui) < τmAnd Then rejected.Wherein τmIt is preset distance threshold, τ in the present embodimentm=1.5,For the boundary according to step (2) The expansion boundary point set that Extension algorithm obtains.
Step (6): go overlapping template point cloud resampling to original human body point cloud density for what step (5) obtained, and smooth It is fused to original human body dot cloud hole part, has been filled up complete cloud of hole.
The expansion boundary point set that step (2) is obtainedThe template point cloud obtained with step (5) is used as input MLS smoothing algorithm is handled, and obtains merging point cloud, and by its resampling, is kept its equalization point spacing consistent with original point cloud, is obtained To a cloudFinally, by its with eliminate expansion boundary point setOriginal point cloudMerge, has been filled up the complete of hole Integral point cloud:
The foregoing is merely the preferred embodiment of the present invention, protection scope of the present invention is not limited in above-mentioned embodiment party Formula, all technical solutions for belonging to the principle of the invention all belong to the scope of protection of the present invention.For those skilled in the art and Speech, several improvements and modifications carried out without departing from the principles of the present invention, these improvements and modifications also should be regarded as this The protection scope of invention.

Claims (9)

1.一种基于模板的人体点云孔洞修补方法，包括：1. A template-based human point cloud hole repair method, comprising: (1)根据原始人体点云中的孔洞区域对人体网格模型模板进行分割，并对分割后的人体网格模型的模板与原始人体点云中的关键对应特征点进行标记；(1) Segment the human mesh model template according to the hole area in the original human point cloud, and mark the template of the segmented human mesh model and the key corresponding feature points in the original human point cloud; (2)从原始人体点云提取提取带有置信度的原始人体点云扩张边界点集；(2) Extract the original human point cloud expansion boundary point set with confidence from the original human point cloud; (3)将分割后的模板与原始人体点云局部进行刚性配准；(3) Rigidly register the segmented template with the original human point cloud; (4)将步骤(2)得到的刚性配准后的模板变形，使之与原始人体点云局部相匹配；(4) deforming the rigidly registered template obtained in step (2) to partially match the original human point cloud; (5)提取变形后的模板上的顶点构成模板点云，并根据到扩张边界点集距离阈值的方法移除模板点云中与原始人体点云重叠的部分，得到去重叠模板点云；(5) Extract the vertices on the deformed template to form a template point cloud, and remove the overlapping part of the template point cloud with the original human body point cloud according to the method of the distance threshold to the expanded boundary point set to obtain a de-overlapping template point cloud; (6)将步骤(5)得到的去重叠模板点云重采样到原始人体点云密度，并平滑融合到原始人体点云孔洞部分，得到填补了孔洞的完整点云。(6) Resampling the de-overlapping template point cloud obtained in step (5) to the density of the original human point cloud, and smoothly integrating it into the hole part of the original human point cloud to obtain a complete point cloud filled with holes. 2.根据权利要求1所述的基于模板的人体点云孔洞修补方法，其特征在于，在步骤(2)中，提取带有置信度的原始人体点云扩张边界点集的方法为：2. template-based human body point cloud hole repair method according to claim 1, is characterized in that, in step (2), the method for extracting the original human body point cloud expansion boundary point set with confidence is: (2-1)使用最大角度量准则、半圆盘度量准则与形状度量准则分别计算边界点概率，加权求和得到加权概率值，将加权概率值与给定阈值进行比较，大于阈值则认为该点是边界点，得到边界点集 (2-1) Calculate the boundary point probability using the maximum angle criterion, the semi-disk metric criterion and the shape metric criterion, weighted summation to obtain the weighted probability value, and compare the weighted probability value with the given threshold value, if it is greater than the threshold value, it is considered that the The point is the boundary point, get the boundary point set (2-2)对边界点集根据距离阈值进行扩张，并计算各点置信度cp，得到带有置信度的原始人体点云扩张边界点集。(2-2) For the boundary point set Expansion is performed according to the distance threshold, and the confidence degree cp of each point is calculated to obtain the expanded boundary point set of the original human point cloud with confidence degree. 3.根据权利要求2所述的基于模板的人体点云孔洞修补方法，其特征在于，在步骤(2-2)中，扩张并计算各点置信度cp的方法为：3. template-based human body point cloud hole repairing method according to claim 2, is characterized in that, in step (2-2), the method that expands and calculates each point confidence cp is: 预先给出一个距离阈值τd,以及置信度区间[cL,cU]，在到孔洞的距离d≤τd的范围内，越靠近孔洞边界的点，其置信度越低，边界点置信度设为cL；越远离孔洞的点，其置信度越高；距离超过τd的点，其置信度设为最大值cU；定义点集任一点到孔洞的距离为该点到边界点集中所有点的距离的最小值，即A distance threshold τ d and a confidence interval [c L , c U ] are given in advance. Within the range of the distance d≤τ d to the hole, the closer the point to the hole boundary, the lower the confidence, and the confidence of the boundary point. The degree is set as c L ; the point farther away from the hole, the higher its confidence; the point whose distance exceeds τ d , its confidence is set as the maximum value c U ; any point in the defined point set The distance to the hole is the point to the boundary point set The minimum value of the distances of all points in 则扩张边界点集为：Then expand the boundary point set for: 且点集上任一点p的置信度cp满足关于dp的函数关系，And the confidence c p of any point p on the point set satisfies the functional relationship about d p , cp＝C(dp)＝min(f(dp)(cU-cL)+cL,cU)c p =C(d p )=min(f(d p )(c U -c L )+c L ,c U ) 距离到置信度的评估函数f(d)是取值域为[0,1]单调递增函数， The evaluation function f(d) from distance to confidence is a monotonically increasing function whose value range is [0,1], 4.根据权利要求1所述的基于模板的人体点云孔洞修补方法，其特征在于，在步骤(4)中，计算模板变形的方式为迭代进行对应点查找与最小化以下函数：4. template-based human body point cloud hole repair method according to claim 1, is characterized in that, in step (4), the mode of calculating template deformation is to iteratively carry out corresponding point search and minimize following function: 其中，T＝[T1,…,Tn]T为模板上每个顶点对应的仿射变换矩阵Ti连接成的4n×3矩阵，α、β、γ为预先设置的参数，V＝[v1,…,vn]T为模板上所有顶点坐标构成的矩阵，U＝[u1,…,un]T为V中顶点在点云中查找得到的对应点坐标构成的矩阵，W＝diag(w1,…,wn)为模板到点云的每一组对应点的权重值，M矩阵表示模板上顶点之间的邻接关系，对于连接顶点对(vi,vj)的边r，M在r行的非零项为Mri＝-1，Mrj＝1，为Kronecker积，I4为4×4单位矩阵，VL与UL分别为将V中对应标记点的行取出以及将U中的对应点的行取出组成的矩阵；迭代的终止条件为：T收敛，即两次迭代计算出的||T(k+1)-T(k)||＜ε，ε为预先设置好的误差阈值。Among them, T=[T 1 ,...,T n ] T is a 4n×3 matrix formed by connecting the affine transformation matrices T i corresponding to each vertex on the template, α, β, γ are preset parameters, V=[ v 1 ,...,v n ] T is the matrix formed by the coordinates of all the vertices on the template, U=[u 1 ,...,u n ] T is the matrix formed by the coordinates of the corresponding points obtained by searching the vertices in the point cloud in V, W =diag(w 1 ,...,w n ) is the weight value of each group of corresponding points from the template to the point cloud, and the M matrix represents the adjacency relationship between the vertices on the template. For the connected vertex pair (vi , v j ) , The non-zero entries of edge r, M in row r are M ri =-1, M rj =1, is the Kronecker product, I 4 is a 4×4 unit matrix, and VL and UL are the matrices formed by taking out the row corresponding to the marked point in V and taking out the row corresponding to the point in U respectively; the termination condition of the iteration is: T Convergence, that is, ||T (k+1) -T (k) || < ε calculated by two iterations, where ε is a preset error threshold. 5.根据权利要求4所述的基于模板的人体点云孔洞修补方法，其特征在于，在步骤(4)中，在进行迭代优化时，对权重值wi采取以下策略进行调整：5. template-based human body point cloud hole repairing method according to claim 4, is characterized in that, in step (4), when carrying out iterative optimization, the weight value wi is adjusted by adopting the following strategy: (4-1)如果ui为原始点云上的边界点，则认为vi对应原始点云上的缺失部分，将对应匹配的权重值wi置为0；(4-1) If ui is a boundary point on the original point cloud, it is considered that v i corresponds to the missing part on the original point cloud, and the corresponding matching weight value w i is set to 0; (4-2)如果Tivi与ui的法向量夹角过大，则认为匹配不可靠，将对应匹配的权重值wi置为0；(4-2) If the angle between the normal vectors of T i v i and ui is too large, the matching is considered unreliable, and the weight value w i of the corresponding matching is set to 0; (4-3)对于其他匹配，使用步骤(2)中得到的原始人体点云扩张边界点集上每点置信度经过归一化后作为权重值，即 (4-3) For other matching, the confidence of each point on the expanded boundary point set of the original human point cloud obtained in step (2) is normalized as the weight value, namely 6.根据权利要求4所述的基于模板的人体点云孔洞修补方法，其特征在于，在步骤(4)中，迭代过程中还包括通过多分辨率方法来加速收敛和在优化过程的不同阶段对能量项使用不同的权重参数。6. The template-based human body point cloud hole repair method according to claim 4, wherein in step (4), the iterative process also includes accelerating convergence by a multi-resolution method and different stages in the optimization process. Use a different weight parameter for the energy term. 7.根据权利要求6所述的基于模板的人体点云孔洞修补方法，其特征在于，所述的通过多分辨率方法来加速收敛的方法为：通过对模板网格模型进行多分辨率自适应参数化，得到不同分辨率的版本，以及这些版本上顶点之间的对应关系；先在低分辨率模板以及原始人体点云的下采样点云上进行上述优化过程；然后将仿射变换矩阵上采样到高分辨率模板上，作为初始变换，再对高分辨率模板进行优化。7. The template-based human point cloud hole repairing method according to claim 6, wherein the method for accelerating convergence by the multi-resolution method is: performing multi-resolution adaptation on the template grid model Parameterization to obtain versions with different resolutions and the corresponding relationship between vertices on these versions; first perform the above optimization process on the low-resolution template and the down-sampled point cloud of the original human point cloud; then put the affine transformation matrix on the Sampling to the high-resolution template as an initial transformation, and then optimizing the high-resolution template. 8.根据权利要求1所述的基于模板的人体点云孔洞修补方法，其特征在于，在步骤(5)中，移除模板点云与原始人体点云重叠的部分的方法为：对于模板点云上的一点vi，使用最近点查找得到其在原始人体点云上的对应点ui，如果dist(vi,ui)＜τm且则将其剔除；其中τm是指定的一个距离阈值，τd是指定的扩张半径。8. template-based human body point cloud hole repair method according to claim 1, is characterized in that, in step (5), the method for removing the overlapping part of template point cloud and original human body point cloud is: for template point For a point vi on the cloud, use the nearest point to find its corresponding point ui on the original human point cloud, if dist(vi , ui ) <τ m and is eliminated; where τ m is a specified distance threshold, τ d is the specified dilation radius. 9.根据权利要求1所述的基于模板的人体点云孔洞修补方法，其特征在于，在步骤(6)中，所述的得到填补了孔洞的完整点云的方法为：9. template-based human body point cloud hole repairing method according to claim 1, is characterized in that, in step (6), the described method that obtains the complete point cloud that has filled the hole is: (6-1)将步骤(2)得到的扩张边界点集合与步骤(5)得到的去重叠模板点云作为输入，使用MLS平滑算法进行处理，得到融合点云；(6-1) Set the expansion boundary points obtained in step (2) The de-overlapping template point cloud obtained in step (5) is used as input, and the MLS smoothing algorithm is used for processing to obtain a fusion point cloud; (6-2)并将融合点云重采样，使融合点云的平均密度与原始点云密度一致，得到点云 (6-2) Resample the fused point cloud to make the average density of the fused point cloud consistent with the density of the original point cloud, and obtain a point cloud (6-3)再将点云与去除了扩张边界点集的原始人体点云合并，得到填补了孔洞的完整点云： (6-3) Then point cloud with the removal of the expanded boundary point set The original human point cloud of Merge to get a full point cloud with holes filled in:
CN201810988717.XA 2018-08-28 2018-08-28 Template-based human point cloud hole repairing method Active CN109147040B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201810988717.XA CN109147040B (en) 2018-08-28 2018-08-28 Template-based human point cloud hole repairing method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201810988717.XA CN109147040B (en) 2018-08-28 2018-08-28 Template-based human point cloud hole repairing method

Publications (2)

Publication Number Publication Date
CN109147040A true CN109147040A (en) 2019-01-04
CN109147040B CN109147040B (en) 2020-06-30

Family

ID=64828639

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201810988717.XA Active CN109147040B (en) 2018-08-28 2018-08-28 Template-based human point cloud hole repairing method

Country Status (1)

Country Link
CN (1) CN109147040B (en)

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111444811A (en) * 2020-03-23 2020-07-24 复旦大学 A method for target detection of 3D point cloud
CN111539446A (en) * 2020-03-04 2020-08-14 南京航空航天大学 A 2D Laser Hole Detection Method Based on Template Matching
CN112215763A (en) * 2020-09-01 2021-01-12 青岛秀山移动测量有限公司 Planar internal missing point cloud repairing method based on image data assistance
CN112465903A (en) * 2020-12-21 2021-03-09 上海交通大学宁波人工智能研究院 6DOF object attitude estimation method based on deep learning point cloud matching
CN112529891A (en) * 2020-12-21 2021-03-19 深圳辰视智能科技有限公司 Hollow hole identification and contour detection method and device based on point cloud and storage medium
CN114266798A (en) * 2021-12-14 2022-04-01 上海应用技术大学 Point cloud repair method based on boundary point iteration
CN114333482A (en) * 2022-01-07 2022-04-12 山东众阳健康科技集团有限公司 Virtual anatomy teaching system based on mixed reality technology

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1885349A (en) * 2006-07-05 2006-12-27 东南大学 Point cloud hole repairing method for three-dimensional scanning
US8749551B2 (en) * 2009-12-29 2014-06-10 Hong Fu Jin Precision Industry (Shenzhen) Co., Ltd. System and method for constructing triangular mesh surface of point cloud
CN104504660A (en) * 2014-12-23 2015-04-08 北京数码大方科技股份有限公司 Data processing method and data processing device used for removing voids in grid model
CN107464223A (en) * 2017-07-19 2017-12-12 西安理工大学 A kind of dot cloud hole method for repairing and mending based on section
CN107590825A (en) * 2017-07-31 2018-01-16 东南大学 A kind of dot cloud hole method for repairing and mending based on SFM
CN107610061A (en) * 2017-08-30 2018-01-19 西安理工大学 A kind of guarantor's edge point cloud hole repair method based on two-dimensional projection
CN107767457A (en) * 2017-10-09 2018-03-06 东南大学 A kind of STL digital-to-analogue generation methods quickly rebuild based on a cloud

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1885349A (en) * 2006-07-05 2006-12-27 东南大学 Point cloud hole repairing method for three-dimensional scanning
US8749551B2 (en) * 2009-12-29 2014-06-10 Hong Fu Jin Precision Industry (Shenzhen) Co., Ltd. System and method for constructing triangular mesh surface of point cloud
CN104504660A (en) * 2014-12-23 2015-04-08 北京数码大方科技股份有限公司 Data processing method and data processing device used for removing voids in grid model
CN107464223A (en) * 2017-07-19 2017-12-12 西安理工大学 A kind of dot cloud hole method for repairing and mending based on section
CN107590825A (en) * 2017-07-31 2018-01-16 东南大学 A kind of dot cloud hole method for repairing and mending based on SFM
CN107610061A (en) * 2017-08-30 2018-01-19 西安理工大学 A kind of guarantor's edge point cloud hole repair method based on two-dimensional projection
CN107767457A (en) * 2017-10-09 2018-03-06 东南大学 A kind of STL digital-to-analogue generation methods quickly rebuild based on a cloud

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
曾露露: "基于从运动中恢复结构的三维点云孔洞修补算法研究", 《光学学报》 *

Cited By (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111539446A (en) * 2020-03-04 2020-08-14 南京航空航天大学 A 2D Laser Hole Detection Method Based on Template Matching
CN111539446B (en) * 2020-03-04 2023-10-03 南京航空航天大学 Template matching-based 2D laser hole site detection method
CN111444811A (en) * 2020-03-23 2020-07-24 复旦大学 A method for target detection of 3D point cloud
CN111444811B (en) * 2020-03-23 2023-04-28 复旦大学 Three-dimensional point cloud target detection method
CN112215763A (en) * 2020-09-01 2021-01-12 青岛秀山移动测量有限公司 Planar internal missing point cloud repairing method based on image data assistance
CN112215763B (en) * 2020-09-01 2023-02-24 青岛秀山移动测量有限公司 Planar internal missing point cloud repairing method based on image data assistance
CN112465903A (en) * 2020-12-21 2021-03-09 上海交通大学宁波人工智能研究院 6DOF object attitude estimation method based on deep learning point cloud matching
CN112529891A (en) * 2020-12-21 2021-03-19 深圳辰视智能科技有限公司 Hollow hole identification and contour detection method and device based on point cloud and storage medium
CN112529891B (en) * 2020-12-21 2024-03-19 深圳辰视智能科技有限公司 Method and device for identifying hollow holes and detecting contours based on point cloud and storage medium
CN114266798B (en) * 2021-12-14 2023-09-22 上海应用技术大学 Point cloud restoration method based on boundary point iteration
CN114266798A (en) * 2021-12-14 2022-04-01 上海应用技术大学 Point cloud repair method based on boundary point iteration
CN114333482A (en) * 2022-01-07 2022-04-12 山东众阳健康科技集团有限公司 Virtual anatomy teaching system based on mixed reality technology
CN114333482B (en) * 2022-01-07 2024-05-24 众阳健康科技集团有限公司 Virtual anatomy teaching system based on mixed reality technology

Also Published As

Publication number Publication date
CN109147040B (en) 2020-06-30

Similar Documents

Publication Publication Date Title
CN109147040A (en) Human body dot cloud hole method for repairing and mending based on template
Li et al. Feature-preserving 3D mesh simplification for urban buildings
CN109559338B (en) A 3D Point Cloud Registration Method Based on Weighted Principal Component Analysis and M-estimation
CN109767463B (en) Automatic registration method for three-dimensional point cloud
Jeon et al. Road detection in spaceborne SAR images using a genetic algorithm
CN104008553B (en) Crack detection method with image gradient information and watershed method conflated
CN108052942B (en) A visual image recognition method for aircraft flight attitude
CN100559398C (en) Automatic deepness image registration method
CN111986115A (en) Accurate elimination method for laser point cloud noise and redundant data
Zhang et al. A statistical approach for extraction of feature lines from point clouds
CN105574929A (en) Single vegetation three-dimensional modeling method based on ground LiDAR point cloud data
CN111508073B (en) Method for extracting roof contour line of three-dimensional building model
CN107481274B (en) A Robust Reconstruction Method for 3D Crop Point Clouds
WO2012071688A1 (en) Method for analyzing 3d model shape based on perceptual information
CN108830888B (en) Coarse matching method based on improved multi-scale covariance matrix characteristic descriptor
CN107862735B (en) RGBD three-dimensional scene reconstruction method based on structural information
CN105118090A (en) Adaptive point-cloud filtering method for complex terrain structure
CN117237428B (en) Data registration method, device and medium for three-dimensional point cloud
CN112200905A (en) Three-dimensional face completion method
CN118155176B (en) A kind of automatic control method and system of handling robot based on machine vision
CN113409332A (en) Building plane segmentation method based on three-dimensional point cloud
CN109933684A (en) Retrieval method of 3D model of aircraft parts based on pcl library and eigenvalue extraction
CN116763295A (en) Livestock body measurement methods, electronic equipment and storage media
Ma et al. Kinematic skeleton extraction from 3D articulated models
CN115909099A (en) Side slope dangerous rock identification and monitoring method based on unmanned aerial vehicle inspection

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant