CN106204721B - The non-complete face model restorative procedure in part based on photo - Google Patents
The non-complete face model restorative procedure in part based on photo Download PDFInfo
- Publication number
- CN106204721B CN106204721B CN201610510400.6A CN201610510400A CN106204721B CN 106204721 B CN106204721 B CN 106204721B CN 201610510400 A CN201610510400 A CN 201610510400A CN 106204721 B CN106204721 B CN 106204721B
- Authority
- CN
- China
- Prior art keywords
- face model
- target area
- complete face
- area
- complete
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
Abstract
A kind of non-complete face model restorative procedure in part, belongs to technical field of computer vision.The present invention includes the following steps: that the single binocular measuring system of 1) use obtains non-complete face model data, determines corresponding defect area in photo；2) corresponding region in extraction standard face model, i.e., known defect area；3) defect area 3D data are rebuild by local deformation model (LMM) method；4) coordinate, consolidation, obtain complete face model.The present invention overcomes defect areas to be located on the plane of symmetry, it is difficult to using the defect of symmetry approach, promote the scope of application of non-complete face model reparation significantly.
Description
Technical field
It is especially a kind of by noncontact measurement the present invention relates to a kind of non-complete 3D model restorative procedure；Three-dimensional Gravity
Method of the technology for non-complete face model reparation is built, technical field of computer vision is belonged to.
Background technique
Since face has almost symmetrical morphosis, for the restorative procedure of defect part, most people is thought first
To be to search out the ideal plane of symmetry using symmetry characteristic, the intact part of plane of symmetry side is symmetric to lacking for the other side
Part is damaged, to reach reparation purpose.But the selection of symmetric points is relative complex, once suitable symmetric points cannot be chosen, obtains
To result be often difficult it is satisfactory.In addition, can not just be carried out using this method if defect area is located on the plane of symmetry
It repairs.Therefore, a kind of method can be widely applied to face model reparation urgently proposes.
With the rapid development of computer technology, researcher can rebuild 3D face model by photo, this is to non-
Complete face model reparation brings new approaches.In prior art, the Liao Haibin of Wuhan University is in " Rapid 3d Face
Reconstruction by Fusion of Sfs and Local Morphable Model”(Journal of Visual
Communication and Image Representation 23, no.6 (2012): 924-931) one is proposed in a text
The method for having merged shape from shading (SFS) and local deformation model (LMM), can use single photo, accurately
Face model is rebuild, a certain amount of calculating is reduced, reaches the tradeoff between quality and speed.The Minsik of South Korea, Lee is in " A
Robust Real-Time Algorithm for Facial Shape Recovery from a Single Image
Containing Cast Shadow under General,Unknown Lighting”(Pattern Recognition
46, no.1 (2013): 38-44) the deformation model optimization method based on tensor is described in a text.They are by using tensor generation
Number technology calculates the humorous bilinear model of ball.The method provides the reconstructions of pinpoint accuracy, and are suitable for general, unknown
Illumination condition.The Ankur Patel in the U.S. is in " Driving 3D morphable models using shading
Deformation is refined using shade clue in cues " (Pattern Recognition 45, no.5 (2012): 1993-2004) text
Mould shapes estimation.They promote prioritization scheme using this observation.These researchs are all to utilize individual existing face photo,
Face depth information is obtained, three-dimensional face model is rebuild.But how by the 3D data of reconstruction in conjunction with existing three-dimensional information
Get up is that we should consider the problems of now.
Summary of the invention
In order to overcome the drawbacks of the prior art and insufficient, the invention proposes a kind of by noncontact measurement；Three-dimensional Gravity
Build method of the technology for non-complete face model reparation.Existing non-complete three-dimensional information and 2D photo progress depth is whole
It closes, using local deformation model method (LMM), the 3D data rebuild by photo is merged with existing three-dimensional information,
Finally obtain complete face 3D model.
The present invention is achieved through the following technical solutions, the non-complete face model restorative procedure in the part based on photo, including
Following steps:
1) non-complete face model data is obtained using single binocular measuring system, and determines corresponding defective region in photo
Domain；
2) corresponding region in extraction standard face model, i.e., known defect area；
3) defect area 3D data are rebuild by LMM method；
4) coordinate, consolidation, obtain complete face model.
The step 1 obtains non-complete face data using Structure light method, and determines face by artificial comparison method
Defect area before the defect of face in photo.
Corresponding region in extraction standard face model in the step 2, i.e., known defect area include:
Using face model database, using man-machine interaction method on face model locating defects region, then carry out
Resampling obtains dense corresponding relationship:
A) standard 3D face model and the 3D face model of defect are unfolded respectively using cylinder deployment algorithm, obtain 2D line
Manage image；Target area is calculated, i.e., the area of the defect area in non-complete face model establishes the plane template of defect area
To define the quantity and topological structure of resampling；
B) resampling is carried out to defect area using template and iterative algorithm；
C) pixel on the vertex 3D and texture image is matched, it is ensured that the vertex 3D of resampling and target area meet
Dense corresponding relationship.
Being comprised the following processes by LMM method reconstruction defect area 3D data in the step 3:
Target area is rebuild using local deformation model method, target area is represented as a vector:
si=(x1,y1,z1..., xk,yk,zk,…,xn,yn,zn)∈R3n (1)
I=1,2 ..., m, m represent the quantity of target area, and n represents the quantity of target area characteristic point, (xk,yk,zk) be
The coordinate of k-th of characteristic point；A linear subspaces are constituted using the vector of m target area, pass through matrix S=(s1,
s2…sm)∈R3n×mTo indicate；The target defect area s of reconstructionnewIt is indicated by the linear combination of known vector:
Wherein αi∈ [0,1],
In order to reduce the correlation between the different target region sampled, while data volume is reduced, uses principal component point
Analysis method: passing through m'(m'≤m-1) feature vector of a covariance matrix ∑ s indicates the m' column eigenmatrix Q=of target area
(q1,q2…qm'), the corresponding characteristic value of this eigenmatrix be in the highest flight, therefore formula (2) indicate are as follows:
Wherein: β=(β1,β2…βm')T∈Rm',
Formula (3) show the target area rebuild by added on the target area of standard face model Δ s come
It obtains；Using Principal Component Analysis, whole deviation passes through the deviation delta s of crucial characteristic pointfIt calculates and obtains；Target area is special
The correspondence of sign point is expressed as vector sf∈R2n, in which:
sf=Ls, L:R3n→R2n (4)
L is implication relation, is the mapping method for carrying out Feature Selection；Equally, change eigenmatrix Q in L transformation,
And obtain the eigenmatrix based on characteristic point:
It can be obtained according to formula (3) and (4)
Wherein β is regulation coefficient in formula (6).
Coordination, consolidation in the step 4 obtain complete face model and comprise the following processes:
The defect area 3D data of reconstruction and original non-complete face data fusion are obtained into complete defected area
Domain 3D model；In order to seamlessly transit the target area rebuild and original non-complete model in intersection, using radial base letter
Interpolation algorithm RBF is counted to connect the two parts；WithThe point set of non-complete face model is represented, is usedRepresent the point set of the target area rebuild, wherein X=(x, y, z) is the coordinate of point, radial basis function definition
Are as follows:
Wherein: X ∈ V, Xi∈ S, p (X)=c1+c2x+c3y+c4Z is binding item, c1、c2、c3、c4For the coefficient for binding item；
wi∈ R represents the weight of each basic function, φi:R3→ R represents basic function；The basic function of selection are as follows:
Wherein: | | | | represent the Euclidean distance in three-dimensional space；
Assuming thatThe object value of interpolated target area, i.e. the average value of juncture area, obtain with
Lower equation:
F(Xi)=ti (9)
Boundary condition are as follows:
So interpolating function F can be solved by linear equation (9) and (10):
Wherein: Aij=φ (| | Xi-Xj||)Xi, XjThe i-th behavior (1, x of ∈ S, Pi,yi,zi), W=(w1,w2…wN)T, C
=(c1,c2,c3,c4)T, T=(t1,t2,…tN)T；Therefore 3 radial primary function networks: F are obtainedx(X), Fy(X), Fz(X).Most
Coordinated afterwards according to these three radial basis function, consolidation, obtains complete face model.
It is located on the plane of symmetry the beneficial effects of the present invention are: overcoming defect area, it is difficult to using the defect of symmetry approach,
The scope of application of non-complete face model reparation has been promoted significantly.
Detailed description of the invention
Fig. 1 is the flow chart of the non-complete face model restorative procedure in the part based on photo.
Specific embodiment
With reference to the accompanying drawing with non-complete 3D face model restorative procedure, specific implementation of the invention is further retouched
It states.
The present invention includes the following steps:
1) non-complete face model data is obtained using single binocular measuring system, and determines corresponding defective region in photo
Domain.Non- complete face data are obtained using Structure light method, and are determined in defected preceding photo by artificial comparison method
Defect area.
2) corresponding region in extraction standard face model, i.e., known defect area.Using face model database,
Using man-machine interaction method, locating defects region, then progress resampling obtain dense corresponding relationship on face model:
A) standard 3D face model and the 3D face model of defect are unfolded respectively using cylinder deployment algorithm, obtain 2D line
Manage image；Target area is calculated, i.e., the area of the defect area in non-complete face model establishes the plane template of defect area
To define the quantity and topological structure of resampling；
B) resampling is carried out to defect area using template and iterative algorithm；
C) pixel on the vertex 3D and texture image is matched, it is ensured that the vertex 3D of resampling and target area meet
Dense corresponding relationship.
3) defect area 3D data are rebuild by LMM method.Weight is carried out using local deformation model method to target area
It builds, target area is represented as a vector:
si=(x1,y1,z1..., xk,yk,zk,…,xn,yn,zn)∈R3n (1)
I=1,2 ..., m, m represent the quantity of target area, and n represents the quantity of target area characteristic point, (xk,yk,zk) be
The coordinate of k-th of characteristic point；A linear subspaces are constituted using the vector of m target area, pass through matrix S=(s1,
s2…sm)∈R3n×mTo indicate；The target defect area s of reconstructionnewIt is indicated by the linear combination of known vector:
Wherein αi∈ [0,1],
In order to reduce the correlation between the different target region sampled, while data volume is reduced, uses principal component point
Analysis method: passing through m'(m'≤m-1) feature vector of a covariance matrix ∑ s indicates the m' column eigenmatrix Q=of target area
(q1,q2…qm'), the corresponding characteristic value of this eigenmatrix be in the highest flight, therefore formula (2) indicate are as follows:
Wherein: β=(β1,β2…βm')T∈Rm',
Formula (3) show the target area rebuild by added on the target area of standard face model Δ s come
It obtains；Using Principal Component Analysis, whole deviation passes through the deviation delta s of crucial characteristic pointfIt calculates and obtains；Target area is special
The correspondence of sign point is expressed as vector sf∈R2n, in which:
sf=Ls, L:R3n→R2n (4)
L is implication relation, is the mapping method for carrying out Feature Selection；Equally, change eigenmatrix Q in L transformation,
And obtain the eigenmatrix based on characteristic point:
It can be obtained according to formula (3) and (4)
Wherein β is regulation coefficient in formula (6).
4) coordinate, consolidation, obtain complete face model.
The defect area 3D data of reconstruction and original non-complete face data fusion are obtained into complete defected area
Domain 3D model；In order to seamlessly transit the target area rebuild and original non-complete model in intersection, using radial base letter
Interpolation algorithm RBF is counted to connect the two parts；WithThe point set of non-complete face model is represented, is usedRepresent the point set of the target area rebuild, wherein X=(x, y, z) is the coordinate of point, radial basis function definition
Are as follows:
Wherein: X ∈ V, Xi∈ S, p (X)=c1+c2x+c3y+c4Z is binding item, c1、c2、c3、c4For the coefficient for binding item；
wi∈ R represents the weight of each basic function, φi:R3→ R represents basic function；The basic function of selection are as follows:
Wherein: | | | | represent the Euclidean distance in three-dimensional space；
Assuming thatThe object value of interpolated target area, i.e. the average value of juncture area, obtain with
Lower equation:
F(Xi)=ti (9)
Boundary condition are as follows:
So interpolating function F can be solved by linear equation (9) and (10):
Wherein: Aij=φ (| | Xi-Xj||)Xi, XjThe i-th behavior (1, x of ∈ S, Pi,yi,zi), W=(w1,w2…wN)T, C
=(c1,c2,c3,c4)T, T=(t1,t2,…tN)T；Therefore 3 radial primary function networks: F are obtainedx(X), Fy(X), Fz(X).Most
Coordinated afterwards according to these three radial basis function, consolidation, obtains complete face model.
Claims (4)
1. the non-complete face model restorative procedure in part based on photo, which is characterized in that the restorative procedure includes following steps
It is rapid:
1) non-complete face model data is obtained using single binocular measuring system, and determines corresponding defect area in photo；
2) corresponding region in extraction standard face model, i.e., known defect area, comprising:
Using face model database, using man-machine interaction method on face model locating defects region, then adopted again
Sample obtains dense corresponding relationship:
2-a) standard 3D face model and the 3D face model of defect are unfolded respectively using cylinder deployment algorithm, obtain 2D texture
Image；Target area, i.e., the area of the defect area in non-complete face model are calculated, the plane template for establishing defect area comes
Define the quantity and topological structure of resampling；
Resampling 2-b) is carried out to defect area using template and iterative algorithm；
2-c) pixel on the vertex 3D and texture image is matched, it is ensured that the vertex 3D of resampling and target area meet thick
Close corresponding relationship；
3) defect area 3D data are rebuild by LMM method；
4) coordinate, consolidation, obtain complete face model.
2. the non-complete face model restorative procedure in the part according to claim 1 based on photo, characterized in that described
Step 1) obtains non-complete face data using Structure light method, and is determined in defected preceding photo by artificial comparison method
Defect area.
3. the non-complete face model restorative procedure in the part according to claim 1 based on photo, characterized in that the step
It is rapid 3) in by LMM method rebuild defect area 3D data comprise the following processes:
Target area is rebuild using local deformation model method, target area is represented as a vector:
si=(x1,y1,z1..., xk,yk,zk,…,xn,yn,zn)∈R3n(1)
I=1,2 ..., m, m represent the quantity of target area, and n represents the quantity of target area characteristic point, (xk,yk,zk) it is kth
The coordinate of a characteristic point；A linear subspaces are constituted using the vector of m target area, pass through matrix S=(s1,s2…
sm)∈R3n×mTo indicate；The target defect area s of reconstructionnewIt is indicated by the linear combination of known vector:
Wherein αi∈ [0,1],
In order to reduce the correlation between the different target region sampled, while data volume is reduced, uses Principal Component Analysis:
Passing through m'(m'≤m-1) feature vector of a covariance matrix ∑ s indicates the m' column eigenmatrix Q=(q of target area1,
q2…qm'), the corresponding characteristic value of this eigenmatrix be in the highest flight, therefore formula (2) indicate are as follows:
Wherein: β=(β1,β2…βm')T∈Rm’,
Formula (3) shows the target area rebuild by adding a Δ s on the target area of standard face model to obtain；
Using Principal Component Analysis, whole deviation passes through the deviation delta s of crucial characteristic pointfIt calculates and obtains；Target area characteristic point
Correspondence be expressed as vector sf∈R2n, in which:
sf=Ls, L:R3n→R2n(4)
L is implication relation, is the mapping method for carrying out Feature Selection；Equally, change eigenmatrix Q in L transformation, and
Obtain the eigenmatrix based on characteristic point:
It can be obtained according to formula (3) and (4)
Wherein β is regulation coefficient in formula (6).
4. the non-complete face model restorative procedure in the part according to claim 1 based on photo, characterized in that the step
It is rapid 4) in coordination, consolidation, obtain complete face model and comprise the following processes:
The defect area 3D data of reconstruction and original non-complete face data fusion are obtained into complete defected region 3D
Model；In order to seamlessly transit the target area rebuild and original non-complete model in intersection, inserted using radial basis function
Value-based algorithm RBF connects the two parts；WithThe point set of non-complete face model is represented, is usedRepresent the point set of the target area rebuild, wherein X=(x, y, z) is the coordinate of point, radial basis function definition
Are as follows:
Wherein: X ∈ V, Xi∈ S, p (X)=c1+c2x+c3y+c4Z is binding item, c1、c2、c3、c4For the coefficient for binding item；wi∈R
Represent the weight of each basic function, φi:R3→ R represents basic function；The basic function of selection are as follows:
Wherein: | | | | represent the Euclidean distance in three-dimensional space；
Assuming thatIt is the object value of interpolated target area, i.e. the average value of juncture area is obtained with lower section
Journey:
F(Xi)=ti(9)
Boundary condition are as follows:
So interpolating function F can be solved by linear equation (9) and (10):
Wherein: Aij=φ (| | Xi-Xj||)Xi, XjThe i-th behavior (1, x of ∈ S, Pi,yi,zi), W=(w1,w2…wN)T, C=(c1,
c2,c3,c4)T, T=(t1,t2,…tN)T；Therefore 3 radial primary function networks: F are obtainedx(X), Fy(X), Fz(X)；Last basis
The coordination of these three radial basis function, consolidation, obtain complete face model.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610510400.6A CN106204721B (en) | 2016-06-30 | 2016-06-30 | The non-complete face model restorative procedure in part based on photo |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610510400.6A CN106204721B (en) | 2016-06-30 | 2016-06-30 | The non-complete face model restorative procedure in part based on photo |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106204721A CN106204721A (en) | 2016-12-07 |
| CN106204721B true CN106204721B (en) | 2019-04-09 |
Family
ID=57462954
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610510400.6A Active CN106204721B (en) | 2016-06-30 | 2016-06-30 | The non-complete face model restorative procedure in part based on photo |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106204721B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112001859B (en) * | 2020-08-10 | 2024-04-16 | 深思考人工智能科技(上海)有限公司 | Face image restoration method and system |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101292914A (en) * | 2008-06-12 | 2008-10-29 | 上海交通大学 | Symmetrical character maxillofacial prosthesis producing method based on three-dimensional visual sensation measurement |
| CN101292915A (en) * | 2008-06-12 | 2008-10-29 | 上海交通大学 | Asymmetric character maxillofacial prosthesis producing method based on three-dimensional visual sensation measurement |
| US8044661B2 (en) * | 2007-04-04 | 2011-10-25 | Siemens Aktiengesellschaft | Method for determining a three-dimensional reconstruction of an examination object |
| CN102961201A (en) * | 2012-12-13 | 2013-03-13 | 陈若瀚 | Method for manufacturing personalized facial prosthesis by laser scanning and quick molding technologies |
| CN104899923A (en) * | 2015-06-15 | 2015-09-09 | 扬州大学 | Method for constructing facial prosthesis optimized model based on smile expression geometrical characteristic modification |
-
2016
- 2016-06-30 CN CN201610510400.6A patent/CN106204721B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8044661B2 (en) * | 2007-04-04 | 2011-10-25 | Siemens Aktiengesellschaft | Method for determining a three-dimensional reconstruction of an examination object |
| CN101292914A (en) * | 2008-06-12 | 2008-10-29 | 上海交通大学 | Symmetrical character maxillofacial prosthesis producing method based on three-dimensional visual sensation measurement |
| CN101292915A (en) * | 2008-06-12 | 2008-10-29 | 上海交通大学 | Asymmetric character maxillofacial prosthesis producing method based on three-dimensional visual sensation measurement |
| CN102961201A (en) * | 2012-12-13 | 2013-03-13 | 陈若瀚 | Method for manufacturing personalized facial prosthesis by laser scanning and quick molding technologies |
| CN104899923A (en) * | 2015-06-15 | 2015-09-09 | 扬州大学 | Method for constructing facial prosthesis optimized model based on smile expression geometrical characteristic modification |
Non-Patent Citations (5)
| Title |
|---|
| Rapid 3D face reconstruction by fusion of SFS and Local Morphable Model;Hai-bin Liao等;《Journal of Visual Communication and Image Morphable Model》;20120831;第23卷(第6期);摘要、第4页第4.2节 * |
| Reconstruction and Representation of 3D Objects with Radial Basis Functions;J. C. Carr等;《Proceeding SIGGRAPH 01 Proceedings of the 28th annual conference on Computer graphics and interactive techniques》;20011231;全文 * |
| 一种单双目视觉系统结合的三维测量方法;雷彦章等;《光学学报》;20080731;第28卷(第7期);摘要、引言 * |
| 基于形变模型的三维人脸重建方法及其改进;胡永利等;《计算机学报》;20051031;第28卷(第10期);全文 * |
| 汉族人外鼻三维形态数据库的建立与应用;董岩;《中国博士学位论文全文数据库 信息科技辑》;20110615(第2011年第6期);第73-78页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN106204721A (en) | 2016-12-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108596974B (en) | Dynamic scene robot positioning and mapping system and method | |
| CN109872397B (en) | Three-dimensional reconstruction method of airplane parts based on multi-view stereo vision | |
| CN106803267B (en) | Kinect-based indoor scene three-dimensional reconstruction method | |
| CN110363858A (en) | A kind of three-dimensional facial reconstruction method and system | |
| JP4785880B2 (en) | System and method for 3D object recognition | |
| CN108198145A (en) | For the method and apparatus of point cloud data reparation | |
| CN107240129A (en) | Object and indoor small scene based on RGB D camera datas recover and modeling method | |
| CN105139379B (en) | Based on the progressive extracting method of classified and layered airborne Lidar points cloud building top surface | |
| CN109658444B (en) | Regular three-dimensional color point cloud registration method based on multi-modal features | |
| GB2543893A (en) | Methods of generating personalized 3D head models or 3D body models | |
| CN106485690A (en) | Cloud data based on a feature and the autoregistration fusion method of optical image | |
| CN106780619A (en) | A kind of human body dimension measurement method based on Kinect depth cameras | |
| KR20220006653A (en) | 3D model creation method, apparatus, computer device and storage medium | |
| CN109685886A (en) | A kind of distribution three-dimensional scenic modeling method based on mixed reality technology | |
| CN109598794A (en) | The construction method of three-dimension GIS dynamic model | |
| CN113012122B (en) | Category-level 6D pose and size estimation method and device | |
| CN112767531B (en) | Mobile-end-oriented human body model face area modeling method for virtual fitting | |
| CN113178009B (en) | Indoor three-dimensional reconstruction method utilizing point cloud segmentation and grid repair | |
| CN104574432A (en) | Three-dimensional face reconstruction method and three-dimensional face reconstruction system for automatic multi-view-angle face auto-shooting image | |
| CN103826032A (en) | Depth map post-processing method | |
| CN109583377B (en) | Control method and device for pipeline model reconstruction and upper computer | |
| CN111274944A (en) | Three-dimensional face reconstruction method based on single image | |
| CN108010122B (en) | Method and system for reconstructing and measuring three-dimensional model of human body | |
| CN106933976B (en) | Method for establishing human body 3D net model and application thereof in 3D fitting | |
| CN110532865B (en) | Spacecraft structure identification method based on fusion of visible light and laser |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |