CN103366400A - Method for automatically generating three-dimensional head portrait - Google Patents

Abstract

The invention relates to a method for automatically generating a three-dimensional head portrait. The method comprises the following steps of: forming a three-dimensional face database; collecting a three-dimensional hair style database; detecting a face of the input positive face picture by using a face detection algorithm, and positioning characteristic points of the front side of the face by using an active shape model; generating a three-dimensional face model based on the three-dimensional face database, the input face picture and the coordinates of the characteristic points of the face by using a deformable model method; segmenting the hair of the input positive face picture by using a hair method based on a Markov random field; extracting the hair texture according to the hair segmentation result; obtaining a final matched hair model; and combining the face model and the hair model. By means of the technical scheme, the generated head portrait model simultaneously comprises a face area and a hair area, and the hair style is prevented from being manually added; and for modeling of the hair part, direct three-dimensional reconstruction is replaced by using a search technology, and the efficiency can be improved. High fidelity can be guaranteed under the condition that the hair style database is rich enough due to extremely high repetition of the human hair style.

Description

A kind of three-dimensional head portrait automatic generation method

Technical field

The invention belongs to technical field of computer vision, particularly relate to a kind of three-dimensional head portrait automatic generation method.

Background technology

To be that present computer graphics and computer vision field are most basic one of study a question in the three-dimensional head portrait modeling.Three-dimensional headform differentiates in identity, the aspects such as medical treatment is auxiliary, production of film and TV, game making, digital art all have widely application.

The outward appearance of head portrait mainly comprises face and hair two parts.To these two parts, mainly contain at present following several method: 1) based on the means of laser scanning, namely utilize laser scanner to obtain the depth information of object, then be reconstructed; 2) based on the method for structured light, namely build data acquisition platform with equipment such as projector, camera, LED lamps, the grating of the multiple width of projector projects is caught by camera behind the optical grating reflection to body surface, according to the different coding of different grating representatives, can calculate the three-dimensional expression of object; 3) based on the method for multiple pictures or video sequence, the photo of namely taking based on many different angles is with the three-dimensional expression of principle of stereoscopic vision calculating destination object; 4) based on the method for single photo, these class methods are generally extracted useful priori from three-dimensional face database, then remove to infer the corresponding three-dimensional model of people's face in the photo based on single photo.In the three-dimensional face modeling method based on single photo, foremost algorithm is that deformation model (Morphable model) method is (with reference to the mode in the following document: B.Volker, V.Thomas.A Morphable Model For The Synthesis Of 3D Faces.SIGGRAPH, 1999.).

Existing method is each has something to recommend him:

Method major defect based on laser scanning is to carry out short range scanning to entity, and scanning needs the expensive time, and the number of people will keep motionless in the scanning process, so practicality is very poor.In addition, because the black hair has the character of absorbing laser, the method can not be used for the reconstruct of hair portion.

Have relatively high expectations based on structured light with based on the method comparison film registration of multiple pictures, and existing algorithm counting yield is not high enough, so these two class methods are mainly used in laboratory environment, and is not suitable for being applied to daily life.

Although the reconfiguration technique precision based on single photo is short of to some extent, because its ease of use and higher counting yield have larger practicality, welcome by public users.But because shape difference is apart from larger between the different hair styles, the method for based on database priori is not easy to realize the reconstruct to hair, thereby prior art mainly stresses the generation of human face region.Therefore, a lot of systems can only generate " shaven head image " automatically, and hair style will lean on the art designing personnel to manually add substantially.In addition, in generating three-dimensional faceform's process, a lot of system requirements users locate human face characteristic point by hand, are not to reach automatic completely.

Summary of the invention

The invention provides a kind of three-dimensional head portrait Auto scheme, support individual front face photo of input, according to photo content, by processing fully automatically, export corresponding three-dimensional head model, output model not only comprises three-dimensional face, also comprises three-dimensional hair style.

Technical scheme of the present invention may further comprise the steps:

Step 1: three-dimensional face storehouse;

Step 2: collect three-dimensional hair style storehouse;

Step 3: to the front face photo of input, end user's face detection algorithm detects people's face, and uses active shape model people from location face positive feature point;

Step 4: based on human face photo and the face characteristic point coordinate of three-dimensional face storehouse, input, with deformation model method generating three-dimensional faceform;

Step 5: to the front face photo of input, use the hair method segmenting hair based on markov random file;

Step 6: according to the hair segmentation result, extract the hair texture;

Step 7: the Hair model that obtains final coupling;

Step 8: faceform and Hair model is synthetic.

By means of technique scheme, the head portrait model of generation comprises human face region and hair zones simultaneously, avoids manually adding hair style; To the modeling of hair portion, use search technique to replace direct three-dimensionalreconstruction, can raise the efficiency.Can guarantee higher fidelity in the situation that the hair style storehouse is enough abundant, because the multiplicity of human hair style is high.

Description of drawings

Fig. 1 techniqueflow chart of the present invention;

Embodiment

The present invention is described further below in conjunction with accompanying drawing.

Fig. 1 has represented the intermediate result with hair three-dimensional head portrait product process figure and each step based on individual input photo of the present invention.Wherein, the collection in three-dimensional face storehouse and three-dimensional hair style storehouse and be treated to off-line procedure.H ^pThe binary map that three-dimensional hair style obtains in the frontal projection, H ^dThat textural characteristics corresponding to each model expressed vector.I ^pBe the hair shape figure (binary map) of input photo, I ^dThat hair textural characteristics corresponding to input photo expressed vector.

Mainly comprise following seven operations:

1. collect in the three-dimensional face storehouse.Collect 300 three-dimensional face models, each faceform has 100,000 summits.These faceforms are through yardstick normalization, so that the pupil of two eyes of different faceform is positioned at unified position.To each model, 15 reference mark of manual appointment in the skull position.This step 1 off-line process.

2. three-dimensional hair style storehouse collection.Collect 100 three-dimensional Hair models, these hair styles contain daily being seen hair style substantially.Each Hair model is by a shape vector

With a two-dimensional texture map H ^tForm.N wherein _hVariation range be 2500 to 6000, (x _i, y _i, z _i) expression i summit three-dimensional coordinate.To each model, 15 reference mark of manual appointment in the skull position, 15 reference mark of these reference mark and faceform have the position corresponding relation.Each Hair model is carried out projection towards positive face direction, obtain a binary map H ^p, namely the hair zones pixel value is 1, other area pixel value is 0.To every texture maps H ^t, obtain its texture expression H according to Gabor conversion and word bag model ^d(mode of obtaining belongs to very common mode in the prior art, such as the mode in the following document: M.Eitz et al.Sketch-Based Shape Retrieval.SIGGRAPH 2012.).H wherein ^dIt is the vector of one 1000 dimension.Finally, to 100 three-dimensional Hair models, correspondingly obtain 100 textural characteristics and express

With 100 projection binary map , i=1 ..., 100.This step 1 off-line process.

3. people's face detects and positioning feature point.Front face photo I to input, use and detect people's face based on people's face detection algorithm of Boosting (the people's face detection algorithm based on Boosting belongs to the very known algorithm in this area, such as the mode in the following document: P.Viola, M.Jones.Rapid object detection using a boosted cascade of simple features.Computer Vision and Pattern Recognition (CVPR), 2001.), then (active shape model also belongs to the known model of this area location human face characteristic point with active shape model location people's face positive feature point, such as the mode in the following document: S.Milborrow and F.Nicolls.Locating Faciai Features with an Extended Active Shape Model.ECCV, 2008.).

4. three-dimensional face generates.Human face photo and face characteristic point coordinate based on three-dimensional face storehouse, input, (the deformation model method belongs to three-dimensional face model and generates known method with deformation model method generating three-dimensional faceform, such as the mode in the following document: B.Volker, V.Thomas.A Morphable Model For The Synthesis Of3D Faces.SIGGRAPH, 1999.).The three-dimensional face model shape vector F that generates ^s=(x ₁, y ₁, z ₁..., x _n, y _n, z _n) and texture image F ^tRepresent n=100000.Wherein, (x _i, y _i, z _i) expression i summit three-dimensional coordinate.Because each faceform has specified 15 reference mark in the model bank, correspondingly, the faceform of generation also has 15 reference mark.

5. hair is cut apart.Front face photo I to input, (dividing method of markov random file belongs to hair and cuts apart known method to use hair automatic Segmentation hair based on markov random file, such as the mode in the following document: K.-C.Lee, D.Anguelov, B.Sumengen, S.B.Gokturk.Markov random field models for hair and face segmentation.Automatic Face ﹠amp; Gesture Recognition, 2008.).Segmentation result is for (being designated as I with I bianry image of a size ^p), wherein the hair zones pixel value is 1, other area pixel value is 0.

6. the hair texture extracts.Generate the hair texture maps I of input photo according to the hair segmentation result ^t, namely to each location of pixels (x, y), calculate:

I ^t(x，y)＝I(x，y)·I ^p(x，y) (1)

The correct texture maps I that sends out ^t, obtain its texture expression I according to Gabor conversion and word bag model ^d(Gabor conversion and word bag model all belong to the very known mode that unity and coherence in writing is expressed of obtaining, such as the mode in the following document: M.Eitz et al.Sketch-Based Shape Retrieval.SIGGRAPH 2012.).I wherein ^dIt is the vector of one 1000 dimension.

7. hair style is mated.Calculate respectively I ^pWith

I=1 ..., the Hausdorff distance in 100 between every image, and find out 10 images of its middle distance minimum

I '=1,10 (it is a kind of known apart from account form that Hausdorff belongs to, such as mode listed in the following document: R.T.Rockafellar, R.J.B.Wets.Variational Analysis, Springer-Verlag, 2005, ISBN 3-540-62772-3, ISBN 978-3-540-62772-2, pg.117.).

Note I '=1 ..., 10 in the hair database corresponding texture express vector fractional integration series and be not

I '=1 ..., 10, then calculate respectively I ^dWith

I '=1 ..., the Euclidean distance between 10, and find out model subscript i corresponding to minor increment ^*, namely

$i^{*}=\arg \mathrm{mi}{n}_{i\′}D(I^d,H_{i\′}^d)---\left(2\right)$

I in the three-dimensional hair style storehouse then ^*Individual model is final Matching Model.

8. faceform and Hair model are synthetic.To the three-dimensional face model that generates, suppose that its three-dimensional coordinate at 15 reference mark of skull position is respectively

I=1 ... 15, and their corresponding reference mark in Hair model are respectively I=1 ... 15.Obtain affine transformation matrix T=[A b by finding the solution following equation]:

$\left[\begin{array}{cccc}{x}_1^f& x_2^f& \·\·\·& x_{15}^f\\ y_1^f& y_2^f& \·\·\·& y_{15}^f\\ z_1^f& z_2^f& \·\·\·& z_{15}^f\\ 1& 1& \·\·\·& 1\end{array}\right]=\left[\begin{array}{cc}A& b\\ 0\·\·\·0& 1\end{array}\right]\left[\begin{array}{cccc}{x}_1^h& x_2^h& \·\·\·& x_{15}^h\\ y_1^h& y_2^h& \·\·\·& y_{15}^h\\ z_1^h& z_2^h& \·\·\·& z_{15}^h\\ 1& 1& \·\·\·& 1\end{array}\right]---\left(3\right)$

Wherein A is the matrix of 3 * 3 sizes, and b is the vector of 3 * 1 sizes, and namely T is the matrix of 3 * 4 sizes.Then, following affined transformation is all carried out on all summits in the hair style model:

$\left(\begin{array}{c}{x^{\′}}_i\\ {y^{\′}}_i\\ {z^{\′}}_i\end{array}\right)=A\left(\begin{array}{c}{x}_i\\ y_i\\ z_i\end{array}\right)+b,i=1,\·\·\·,n_h.$

By conversion like this, the faceform of Hair model and generation relatively coordinates on size and relative position, no longer needs manual setting.Hair model shape vector after the note conversion is $H^{{\′}^s}=({x^{\′}}_1,{y^{\′}}_1,{z^{\′}}_1,\·\·\·,{x^{\′}}_{n_h},{y^{\′}}_{n_h},{z^{\′}}_{n_h})$ , then the final head model that generates of this method comprises the shape vector H ' of hair portion ^s, the people face part shape vector F ^sAnd the texture maps H of their correspondences ^tAnd F ^tH ^t

This programme has passed through emulation experiment.To single photo, the processing time on ordinary individual's computer is about 20 seconds, whole-process automatic, and the three-dimensional head portrait model of generation is quite true to nature, can satisfy a lot of application request.

Claims (9)

1. a three-dimensional head portrait automatic generation method is characterized in that, may further comprise the steps:

Step 1: collect the three-dimensional face storehouse;

Step 2: collect three-dimensional hair style storehouse;

Step 3: to the front face photo I of input, end user's face detection algorithm detects people's face, and uses active shape model people from location face positive feature point;

Step 4: based on the human face photo I of described three-dimensional face storehouse, described input and the coordinate of described human face characteristic point, with deformation model method generating three-dimensional faceform;

Step 5: the front face photo I to input, use the hair method segmenting hair based on markov random file;

Step 6: according to the hair segmentation result, extract the hair texture;

Step 7: the Hair model that obtains final coupling;

Step 8: faceform and Hair model is synthetic.

2. three-dimensional head portrait automatic generation method according to claim 1, it is characterized in that: described step 1 specifically comprises: collect 300 three-dimensional face models, each faceform has 100,000 summits, these faceforms are through yardstick normalization, so that the pupil of two eyes of different faceform is positioned at unified position; To each model, 15 reference mark of manual appointment in the skull position.

3. three-dimensional head portrait automatic generation method according to claim 2, it is characterized in that: described step 2 specifically comprises: collect 100 three-dimensional Hair models, each Hair model is by a shape vector With a two-dimensional texture map H ^tForm, wherein n _hVariation range be 2500 to 6000, (x _i, y _i, z _i) expression i summit three-dimensional coordinate; To each model, 15 reference mark of manual appointment in the skull position, 15 reference mark of these reference mark and faceform have the position corresponding relation; Each Hair model is carried out projection towards positive face direction, obtain a binary map H ^p, namely the hair zones pixel value is 1, other area pixel value is 0; To every texture maps H ^t, obtain its texture expression H according to Gabor conversion and word bag model ^d, H wherein ^dIt is the vector of one 1000 dimension; Finally, to 100 three-dimensional Hair models, correspondingly obtain 100 textural characteristics and express

With 100 projection binary map

, i=1 ..., 100.

4. three-dimensional head portrait automatic generation method according to claim 3 is characterized in that: in the described step 4: the three-dimensional face model of generation shape vector F ^s=(x ₁, y ₁, z ₁..., x _n, y _n, z _n) and texture image F ^tRepresent, n=100000, wherein, (x _i, y _i, z _i) expression i summit three-dimensional coordinate; Because each faceform has specified 15 reference mark in the model bank, correspondingly, the faceform of generation also has 15 reference mark.

5. three-dimensional head portrait automatic generation method according to claim 4 is characterized in that: in the described step 5: segmentation result is designated as I for I bianry image of a size ^p, wherein the hair zones pixel value is 1, other area pixel value is 0.

6. three-dimensional head portrait automatic generation method according to claim 5, it is characterized in that: described step 6 is specially:

Generate the hair texture maps I of input photo according to the hair segmentation result ^t, namely to each location of pixels (x, y), calculate

I ^t(x，y)＝I(x，y)·I ^p(x，y) (1)

The correct texture maps I that sends out ^t, obtain its texture expression I according to Gabor conversion and word bag model ^d, I wherein ^dIt is the vector of one 1000 dimension.

7. three-dimensional head portrait automatic generation method according to claim 6, it is characterized in that: described step 7 is specially:

Calculate respectively I ^pWith

I=1 ..., the Hausdorff distance in 100 between every image, and find out 10 images of its middle distance minimum

I '=1 ..., 10;

Note

I '=1 ..., 10 in the hair database corresponding texture express vector fractional integration series and be not

I '=1 ..., 10, then calculate respectively I ^dWith I '=1 ..., the Euclidean distance between 10, and find out model subscript i corresponding to minor increment ^*, namely

$i^{*}=\arg \mathrm{mi}{n}_{i^{\′}}D(I^d,H_{i^{\′}}^d)---\left(2\right)$

I in the three-dimensional hair style storehouse then ^*Individual model is final Matching Model.

8. three-dimensional head portrait automatic generation method according to claim 7, it is characterized in that: described step 8 is specially:

To the three-dimensional face model that generates, suppose that its three-dimensional coordinate at 15 reference mark of skull position is respectively

I=1 ... 15, and their corresponding reference mark in Hair model are respectively

I=1 ... 15.Obtain affine transformation matrix T=[A b by finding the solution following equation]:

$\left[\begin{array}{cccc}{x}_1^f& x_2^f& \·\·\·& x_{15}^f\\ y_1^f& y_2^f& \·\·\·& y_{15}^f\\ z_1^f& z_2^f& \·\·\·& z_{15}^f\\ 1& 1& \·\·\·& 1\end{array}\right]=\left[\begin{array}{cc}A& b\\ 0\·\·\·0& 1\end{array}\right]\left[\begin{array}{cccc}{x}_1^h& x_2^h& \·\·\·& x_{15}^h\\ y_1^h& y_2^h& \·\·\·& y_{15}^h\\ z_1^h& z_2^h& \·\·\·& z_{15}^h\\ 1& 1& \·\·\·& 1\end{array}\right]---\left(3\right)$

Wherein A is the matrix of 3 * 3 sizes, and b is the vector of 3 * 1 sizes, and namely T is the matrix of 3 * 4 sizes;

Then, following affined transformation is all carried out on all summits in the hair style model:

$\left(\begin{array}{c}{x^{\′}}_i\\ {y^{\′}}_i\\ {z^{\′}}_i\end{array}\right)=A\left(\begin{array}{c}{x}_i\\ y_i\\ z_i\end{array}\right)+b,i=1,\·\·\·,n_h$

By conversion like this, the faceform of Hair model and generation relatively coordinates on size and relative position, no longer needs manual setting;

Hair model shape vector after the note conversion is , then the final head model that generates of this method comprises the shape vector H ' of hair portion ^s, the people face part shape vector F ^sAnd the texture maps H of their correspondences ^tAnd F ^t

9. described three-dimensional head portrait automatic generation method one of according to claim 1-8, it is characterized in that: described step 1 and step 2 are off-line process.

Images