CN104200505A - Cartoon-type animation generation method for human face video image - Google Patents

Abstract

The invention discloses a cartoon-type animation generation method for a human face video image. The cartoon-type animation generation method comprises the following steps of: reading a human face video to be processed and extracting a key frame of the human face video to obtain a formula described in the specification; reading a key frame image and carrying out illustration processing to obtain an illustration image; reading the illustration image and carrying out human face feature positioning to obtain a face feature point set, and calculating the ratio of each portion of the human face according to face feature points, and carrying out feature discovery on the human face; carrying out overall deformation on the illustration image to obtain an image fti(x,y); carrying out partial exaggeration on the image fti(x,y) to obtain a final cartoon image; processing S frame by frame to obtain a key frame sequence image with a cartoon effect; reading a key frame sequence D, and carrying out middle frame generation between every two key frames by a cross decomposition method to obtain the human face video image. According to the cartoon-type animation generation method disclosed by the invention, the generated cartoon has a comedy and does not lose the original romantic charm by adopting a global-partial combination method, thus the problems of inconsistent styles and face position disproportionality of the existing cartoon are solved, and the generated illustration image is more natural and rich in artistic perception.

Description

A kind of caricature formula animation producing method of face video image

Technical field

The invention belongs to digital image processing techniques field, relate to a kind of caricature formula animation producing method of face video image.

Background technology

Current human-face cartoon generation technique only carries out simple caricature to face and synthesizes, but compared with artist's Freehandhand-drawing effect, face exaggeration to specific style and synthetic not overripened, although proposed many methods, but have unsatisfactory place, great majority cannot be applied to actual demand to form distinctive human-face cartoon animation.

The gordian technique that human-face cartoon is drawn may be summarized to be: the automatic generation of portrait and the distortion of portrait exaggeration.The automatic generation of portrait roughly can be divided into the face drawing portrait based on image, the large class of face drawing portrait two detecting based on face, although can generate the facial image of specific style, but face still has larger redundancy, stereoscopic sensation shortcoming, there is certain limitation, therefore can't be lively describe face.The distortion exaggeration of portrait roughly can be divided into four classes: based on the method for man-machine interaction; Based on the method for sample template; Based on the regular method of drawing; Based on the method for statistical learning.Method based on man-machine interaction is difficult to operation for the domestic consumer lacking experience.Method based on sample template can only represent average level, and that can not exaggerate shows face characteristic.Based on the regular method of drawing and the large length consuming time of method workload based on statistical learning.These methods can generate the facial image with caricature effect in certain degree, but do not consider the proportionate relationship of face entirety, make exaggeration lose original romantic charm, artistic feeling shortcoming, the result that more difficult acquisition is satisfied.

Summary of the invention

The object of this invention is to provide a kind of caricature formula animation producing method of face video image, make the illustrated painting of generation more natural, be rich in artistic feeling.

The technical solution adopted in the present invention is, a kind of caricature formula animation producing method of face video image is specifically implemented according to following steps:

Step 1, reads in pending face video S={F ₁(x, y), F ₂(x, y) ..., F _n(x, y) }, and its extraction key frame is obtained wherein n is the length of face sequence of frames of video, and m is the number of the key frame of extraction;

Step 2, reads in key frame images f _i(x, y), and it is carried out to illustrated painting processing obtain illustrated painting image f _i' (x, y);

Step 3, reads in illustrated painting image f _i' (x, y), and it is carried out to face characteristic location, obtain face feature point set P=(x ₁, y ₁, x ₂, y ₂..., x _p, y _p), and calculate the ratio at the each position of face face according to face feature point, face is carried out to characteristic discover;

Step 4, to illustrated painting image f _i' (x, y) carry out bulk deformation, obtains image f _ti(x, y);

Step 5, to image f _ti(x, y) carries out part exaggeration, obtains final cartoon image f _pi(x, y);

Step 6 is right process frame by frame, obtain having the keyframe sequence image of caricature effect: D={f _p1(x, y), f _p2(x, y) ..., f _pm(x, y) };

Step 7, reads in keyframe sequence D, between every two key frames, adopts cross decomposition method to carry out intermediate frame generation, obtains the caricature formula animation sequence image of face video image.

Feature of the present invention is also,

The concrete grammar of step 1 is that establishing pending face sequence of video images is S={F ₁(x, y), F ₂(x, y) ..., F _n(x, y) }, adopt dual threshold algorithm to carry out key-frame extraction to S, obtain keyframe sequence image $\tilde{S}=\{f_1(x,y),f_2(x,y),...,f_m(x,y)\}.$

The concrete grammar of step 2 is,

Step 2.1, to image f _i(x, y) carries out USM sharpening (sharpening of Unsharp Mask Unsharp Mask) and processes, and asking the image after sharpening according to formula (1) is t (x, y);

$\left\{\begin{array}{c}h(x,y)=f_i(x,y)*g(x,y)\\ t(x,y)=(1+d)f_i(x,y)-(d-\mathrm{\η})h(x,y)\end{array}\right.---\left(1\right)$

G (x, y)=exp ((x in formula (1) ²+ y ²)/2 σ ²), σ is level and smooth scale parameter; D gets 4.0, is to adjust from f _ithe amount of the smooth picture deducting in (x, y); η gets 0.3, is adjustment experience factor;

Step 2.2, facial image t (x after traversal sharpening, y) each pixel, using 128 as global threshold, the pixel that is greater than threshold value is set to 255,0 and threshold value between pixel be set to the σ power of this pixel value, equal 0 pixel value constant, σ value is 1.21, obtains illustrated painting image f _i' (x, y).

The concrete grammar of step 3 is:

Step 3.1, by the f obtaining in step 2 _i' (x, y), use Adaboost algorithm (adaptive boosting algorithm) to carry out face detection, determine human face region; Then use the face characteristic location detection software STASM increasing income based on active shape model ASM to position face facial zone unique point, obtain face feature point set P=(x ₁, y ₁, x ₂, y ₂..., x _p, y _p), wherein p=1,2 ..., 77;

Step 3.2, the ratio of calculating the each position of face face according to point set P, comprises the wide width=x of face ₁₂-x ₀, the long length=y of face ₆-y ₁₄, the wide eyew=x of human eye ₃₀-x ₃₄, people's face width mouthw=x ₆₅-x ₅₉, with the width facew=x of face of face same level ₉-x ₃, face length breadth ratio k _f=length/width, human eye width accounts for the width ratio k of same level face _e=eyew/width, face width accounts for the ratio k of same level face width _m=mouthw/facew, the upper presiding judge in face three front yards is top=y ₂₂-y ₁₄, middle presiding judge is middle=y ₅₅-y ₂₂, lower presiding judge is bottom=y ₆-y ₅₅;

Step 3.3, characteristic discover: establishing standard faces length breadth ratio is 5/4, and establishes d _f=k _f-5/4 (d _ffor the irrelevance of face length breadth ratio and standard faces length breadth ratio), if d _f< 0.073 is wide face; If 0.073≤d _f< 0.18 is long face; Otherwise be standard face;

If for long face be divided into following four kinds of situation: a, three front yards are isometric, three presiding judge's degree do poorly between two, error is all less than 2; B, upper presiding judge, i.e. top > middle and top > bottom; C, middle presiding judge, i.e. middle > top and middle > bottom; D, lower presiding judge, i.e. bottom > top and bottom > middle;

The wide ratio of the accurate human eye Kuan Zhan of bidding same level face is 3/10, and establishes d _e=k _e-3/10 (d _efor the irrelevance of the wide ratio of human eye Kuan Zhan same level face and standard value), if-0.5 < d _e<-0.4, is pigsney, otherwise is oxeye;

It is 1/2 that the accurate face width of bidding accounts for the wide ratio of same level face, and establishes d _m=k _m-1/2 (d _mfor face width accounts for the irrelevance of the wide ratio of same level face and standard value), if-0.07 < d _m<-0.05, is little face, otherwise is large face.

The concrete grammar of step 4 is:

According to different shapes of face and three front yard situations, to illustrated painting image f _i' (x, y) carry out integral transformation and obtain image f _ti(x, y), transform method is to use bilinear interpolation and affined transformation, concrete change situation is as follows:

Step 4.1, if wide face carries out reduced overall to face, keeps picture traverse constant, and height is 0.618 times of source images;

Step 4.2, if long face and be the isometric situations in three front yards carries out integrally stretching to face, keeps picture traverse constant, and height is 1.618 times of source images;

Step 4.3, if long face and upper presiding judge's situation is carried out the long stretching of longitudinal trapezoidal upper base to integral image, carries out 4 affined transformations to image, specifically by f _i' (x, y) four summits are respectively (0,0), (width, 0), (0, length), (width, length) image conversion is to (0,0), (width, 0), (0.2 × width, length), (0.8 × width, length), in the trapezoid area forming, remainder is filled by white;

Step 4.4, if long face and middle presiding judge's situation, integral image is carried out to Shen word stretching, go up 1/3rd long stretchings of going to the bottom, in 1/3rd Uniform Tensions, lower 1/3rd upper bases are long to stretch, specifically: upper 1/3rd long stretchings of going to the bottom are four summits to be respectively to (0,0), (width, 0), form image-region transform to 4 for (0.2 × width, 0), (0.8 × width, 0), the region forming; In 1/3rd Uniform Tensions be that four summits are respectively $(0,\frac{1}{3}\&times;\mathrm{length}),(\mathrm{width},\frac{1}{3}\&times;\mathrm{length}),(0,\frac{2}{3}\&times;\mathrm{length}),(\mathrm{width},\frac{2}{3}\&times;\mathrm{length})$ 4 of transforming to of image-region that form are $(0,\frac{1}{3}\&times;2.618\&times;\mathrm{length}),(\mathrm{width},\frac{1}{3}\&times;2.618\&times;\mathrm{length})$ The region forming; The long stretching of lower 1/3rd upper bases is that four summits are respectively 4 of transforming to of image-region that (0, length), (width, length) form are $(0,\frac{1}{3}\&times;2.618\&times;\mathrm{length}),(\mathrm{width},\frac{1}{3}\&times;2.618\&times;\mathrm{length}),$ $(0.2\&times;\mathrm{width},\frac{1}{3}\&times;3.618\&times;\mathrm{length}),(0.8\&times;\mathrm{width},\frac{1}{3}\&times;3.618\&times;\mathrm{length})$ The region forming;

Step 4.5, if long face and lower presiding judge's situation is carried out longitudinally trapezoidal long stretching of going to the bottom of entirety to integral image, carries out 4 affined transformations to image, specifically by f _i' (x, y) four summits are respectively (0,0), (width, 0), (0, length), (width, length) transform to (0.2 × width, 0), (0.8 × width, 0), (0, length), (width, length), in the trapezoid area forming, remainder is filled by white.

The concrete grammar of step 5 is:

Step 5.1, according to 4 respective coordinates before and after integral transformation in step 4, calculates the transformation matrix that the affined transformation that adopts in integral transformation is used $T=\left[\begin{array}{ccc}{a}_{11}& a_{12}& a_{13}\\ a_{21}& a_{22}& a_{23}\\ a_{31}& a_{32}& a_{33}\end{array}\right];$

Step 5.2 is found out respectively the rectangular area that upper and lower, left and right four point coordinate of eyes before integral transformation and face form from unique point set P;

Step 5.3, finds out eyes after bulk deformation and the rectangular area at face place, is specially: for wide face situation, keep former eyes and face rectangular area width constant, hypermutation is original 0.618 times; For the isometric situation of long face and three front yards, keep former eyes and face rectangular area width constant, hypermutation is original 1.618 times; For trapezoidal stretching and Shen word pulled out condition, the coordinate according to after formula (2) computational transformation:

$\left\{\begin{array}{c}{x}^{\&prime;}=\frac{a_{11}x+a_{12}y+a_{13}}{a_{31}x+a_{32}y+a_{33}}\\ y^{\&prime;}=\frac{a_{21}x+a_{22}y+a_{23}}{a_{31}x+a_{32}y+a_{33}}\end{array}\right.---\left(2\right)$

In formula (2), (x, y) is the coordinate before converting, (x ', y ') be the coordinate after conversion, the boundary rectangle of four points after changes persuing is changed, the as a whole eyes after distortion and the rectangular area at face place;

Step 5.4, has obtained eyes after integral transformation and the rectangular area coordinate at face place, eyes and the corresponding rectangular area of face is stretched or is compressed, to realize the local exaggerated deformation in these regions; For oxeye or large face, keep rectangle width constant, by the length of rectangle divided by 0.8; For pigsney or little face, keep rectangle width constant, the length of rectangle is multiplied by 0.8; Realize the exaggerated deformation to eyes and these regional areas of face, obtained final cartoon image f _pi(x, y).

The concrete grammar of step 6 is: right process frame by frame, for the f inputting ₁(x, y), f ₂(x, y) ..., f _m(x, y) carries out the processing of step 2 to step 5 one by one, obtains respectively corresponding f _p1(x, y), f _p2(x, y) ..., f _pm(x, y), the keyframe sequence with caricature effect is: D={f _p1(x, y), f _p2(x, y) ..., f _pm(x, y) }.

The concrete grammar of step 7 is, reads in keyframe sequence D, adopts cross decomposition method to carry out intermediate frame generation at every two key frames, and computing formula is suc as formula (3):

F _pu(x,y)＝(1-t)f _pi(x,y)+tf _p(i+1)(x,y) (3)

F in formula (3) _pu(x, y) is at key frame f _pi(x, y) and key frame f _{p (i+1)}the intermediate frame image generating between (x, y), wherein the value of t is from 0 to 1.

The invention has the beneficial effects as follows, the caricature formula animation producing method of face video image of the present invention, aspect the generation of face illustrated painting, is further improved the image after USM sharpening, can obviously remove the redundancy on face, make face portrait more succinct; Use STASM can utilize fully face facial information to locate accurately unique point, be conducive to follow-up shape of face judgement and local exaggeration; The method that adopts whole and part to combine, can more well hold face's ratio, make the caricature generating there is happiness sense and not lose original romantic charm, solved the problem that existing cartoon style is inconsistent, facial positions is out of proportion, make the illustrated painting of generation more natural, be rich in artistic feeling.

Brief description of the drawings

Fig. 1 is the process flow diagram of the caricature formula animation producing method of face video image of the present invention.

Embodiment

Below in conjunction with the drawings and specific embodiments, the present invention is described in detail.

The caricature formula animation producing method of a kind of face video image of the present invention, referring to Fig. 1, specifically implement according to following steps:

Step 1, reads in pending face video S={F ₁(x, y), F ₂(x, y) ..., F _n(x, y) }, and its extraction key frame is obtained wherein n is the length of face sequence of frames of video, and m is the number of the key frame of extraction; (F ₁(x, y), F ₂(x, y), F _n(x, y) is respectively the picture frame of pending face video, f ₁(x, y), f ₂(x, y), f _m(x, y) is respectively the key frame after extraction, the horizontal ordinate that x is picture frame, the ordinate that y is picture frame);

Concrete grammar is that establishing pending face sequence of video images is S={F ₁(x, y), F ₂(x, y) ..., F _n(x, y) }, adopt dual threshold algorithm to carry out key-frame extraction to S, obtain keyframe sequence image $\tilde{S}=\{f_1(x,y),f_2(x,y),...,f_m(x,y)\};$

The detailed process of dual threshold algorithm is:

Suppose Dis (H _t, H _t+1) be the histogram difference of picture frame, T _cand T _gbe two threshold values, and T _g< T _c, Ac (j) is cumulative error, the specific algorithm of dual threshold relative method is as follows so:

(1) if Dis is (H _t, H _t+1) > T _c, between two images, there is sudden change.

(2) if Dis is (H _t, H _t+1) < T _c, and Dis (H _t, H _t+1) > T _g, for gradual change starts, set a gradual change mark.

(3) if Ac (j) > is T _c, gradual change finishes; If Ac (j) < is T _c, continue cumulative error Ac (j)=Ac (j-1)+Dis (H _t, H _t+1).

(4) if the poor Dis (H of next frame _t, H _t+1) < T _gtime, illustrate that the gradual change of assert is not genuine gradual change, cancel gradual change beginning flag, repeat (1);

Step 2, reads in key frame images f _i(x, y), and it is carried out to illustrated painting processing obtain illustrated painting image f _i' (x, y); Concrete grammar is,

Step 2.1, to image f _i(x, y) carries out USM sharpening (sharpening of Unsharp Mask Unsharp Mask) and processes, and asking the image after sharpening according to formula (1) is t (x, y);

$\left\{\begin{array}{c}h(x,y)=f_i(x,y)*g(x,y)\\ t(x,y)=(1+d)f_i(x,y)-(d-\mathrm{\&eta;})h(x,y)\end{array}\right.---\left(1\right)$

G (x, y)=exp ((x in formula (1) ²+ y ²)/2 σ ²), σ is level and smooth scale parameter; D gets 4.0, is to adjust from f _ithe amount of the smooth picture deducting in (x, y); η gets 0.3, is adjustment experience factor;

Step 2.2, facial image t (x after traversal sharpening, y) each pixel, using 128 as global threshold, the pixel that is greater than threshold value is set to 255,0 and threshold value between pixel be set to the σ power of this pixel value, equal 0 pixel value constant, σ value is 1.21, obtains illustrated painting image f _i' (x, y);

Step 3, reads in illustrated painting image f _i' (x, y), and it is carried out to face characteristic location, obtain face feature point set P=(x ₁, y ₁, x ₂, y ₂..., x _p, y _p), and calculate the ratio at the each position of face face according to face feature point, face is carried out to characteristic discover;

Concrete grammar is:

Step 3.1, by the f obtaining in step 2 _i' (x, y), use Adaboost algorithm (adaptive boosting algorithm) to carry out face detection, determine human face region; Then use the face characteristic location detection software STASM increasing income based on active shape model ASM to position face facial zone unique point, obtain face feature point set P=(x ₁, y ₁, x ₂, y ₂..., x _p, y _p), wherein p=1,2 ..., 77;

Step 3.2, the ratio of calculating the each position of face face according to point set P, comprises the wide width=x of face ₁₂-x ₀, the long length=y of face ₆-y ₁₄, the wide eyew=x of human eye ₃₀-x ₃₄, people's face width mouthw=x ₆₅-x ₅₉, with the width facew=x of face of face same level ₉-x ₃, face length breadth ratio k _f=length/width, human eye width accounts for the width ratio k of same level face _e=eyew/width, face width accounts for the ratio k of same level face width _m=mouthw/facew, the upper presiding judge in face three front yards is top=y ₂₂-y ₁₄, middle presiding judge is middle=y ₅₅-y ₂₂, lower presiding judge is bottom=y ₆-y ₅₅;

Step 3.3, characteristic discover: establishing standard faces length breadth ratio is 5/4, and establishes d _f=k _f-5/4 (d _ffor the irrelevance of face length breadth ratio and standard faces length breadth ratio), if d _f< 0.073 is wide face; If 0.073≤d _f< 0.18 is long face; Otherwise be standard face;

If for long face be divided into following four kinds of situation: a, three front yards are isometric, three presiding judge's degree do poorly between two, error is all less than 2; B, upper presiding judge, i.e. top > middle and top > bottom; C, middle presiding judge, i.e. middle > top and middle > bottom; D, lower presiding judge, i.e. bottom > top and bottom > middle;

The wide ratio of the accurate human eye Kuan Zhan of bidding same level face is 3/10, and establishes d _e=k _e-3/10 (d _efor the irrelevance of the wide ratio of human eye Kuan Zhan same level face and standard value), if-0.5 < d _e<-0.4, is pigsney, otherwise is oxeye;

It is 1/2 that the accurate face width of bidding accounts for the wide ratio of same level face, and establishes d _m=k _m-1/2 (d _mfor face width accounts for the irrelevance of the wide ratio of same level face and standard value), if-0.07 < d _m<-0.05, is little face, otherwise is large face;

Step 4, to illustrated painting image f _i' (x, y) carry out bulk deformation, obtains image f _ti(x, y); Concrete grammar is,

According to different shapes of face and three front yard situations, to illustrated painting image f _i' (x, y) carry out integral transformation and obtain image f _ti(x, y), transform method is to use bilinear interpolation and affined transformation, concrete change situation is as follows:

Step 4.1, if wide face carries out reduced overall to face, keeps picture traverse constant, and height is 0.618 times of source images;

Step 4.2, if long face and be the isometric situations in three front yards carries out integrally stretching to face, keeps picture traverse constant, and height is 1.618 times of source images;

Step 4.3, if long face and upper presiding judge's situation is carried out the long stretching of longitudinal trapezoidal upper base to integral image, carries out 4 affined transformations to image, specifically by f _i' (x, y) four summits are respectively (0,0), (width, 0), (0, length), (width, length) image conversion is to (0,0), (width, 0), (0.2 × width, length), (0.8 × width, length), in the trapezoid area forming, remainder is filled by white;

Step 4.4, if long face and middle presiding judge's situation, integral image is carried out to Shen word stretching, go up 1/3rd long stretchings of going to the bottom, in 1/3rd Uniform Tensions, lower 1/3rd upper bases are long to stretch, specifically: upper 1/3rd long stretchings of going to the bottom are four summits to be respectively to (0,0), (width, 0), form image-region transform to 4 for (0.2 × width, 0), (0.8 × width, 0), the region forming; In 1/3rd Uniform Tensions be that four summits are respectively $(0,\frac{1}{3}\&times;\mathrm{length}),(\mathrm{width},\frac{1}{3}\&times;\mathrm{length}),(0,\frac{2}{3}\&times;\mathrm{length}),(\mathrm{width},\frac{2}{3}\&times;\mathrm{length})$ 4 of transforming to of image-region that form are $(0,\frac{1}{3}\&times;2.618\&times;\mathrm{length}),(\mathrm{width},\frac{1}{3}\&times;2.618\&times;\mathrm{length})$ The region forming; The long stretching of lower 1/3rd upper bases is that four summits are respectively 4 of transforming to of image-region that (0, length), (width, length) form are $(0,\frac{1}{3}\&times;2.618\&times;\mathrm{length}),(\mathrm{width},\frac{1}{3}\&times;2.618\&times;\mathrm{length}),$ $(0.2\&times;\mathrm{width},\frac{1}{3}\&times;3.618\&times;\mathrm{length}),(0.8\&times;\mathrm{width},\frac{1}{3}\&times;3.618\&times;\mathrm{length})$ The region forming;

Step 4.5, if long face and lower presiding judge's situation is carried out longitudinally trapezoidal long stretching of going to the bottom of entirety to integral image, carries out 4 affined transformations to image, specifically by f _i' (x, y) four summits are respectively (0,0), (width, 0), (0, length), (width, length) transform to (0.2 × width, 0), (0.8 × width, 0), (0, length), (width, length), in the trapezoid area forming, remainder is filled by white;

Step 4.6, does not carry out integral transformation to standard shape of face;

Step 5, to image f _ti(x, y) carries out part exaggeration, obtains final cartoon image f _pi(x, y); Concrete grammar is:

Step 5.1, according to 4 respective coordinates before and after integral transformation in step 4, calculates the transformation matrix that the affined transformation that adopts in integral transformation is used $T=\left[\begin{array}{ccc}{a}_{11}& a_{12}& a_{13}\\ a_{21}& a_{22}& a_{23}\\ a_{31}& a_{32}& a_{33}\end{array}\right];$

Step 5.2 is found out respectively the rectangular area that upper and lower, left and right four point coordinate of eyes before integral transformation and face form from unique point set P;

Step 5.3, finds out eyes after bulk deformation and the rectangular area at face place, is specially: for wide face situation, keep former eyes and face rectangular area width constant, hypermutation is original 0.618 times; For the isometric situation of long face and three front yards, keep former eyes and face rectangular area width constant, hypermutation is original 1.618 times; For trapezoidal stretching and Shen word pulled out condition, the coordinate according to after formula (2) computational transformation:

$\left\{\begin{array}{c}{x}^{\&prime;}=\frac{a_{11}x+a_{12}y+a_{13}}{a_{31}x+a_{32}y+a_{33}}\\ y^{\&prime;}=\frac{a_{21}x+a_{22}y+a_{23}}{a_{31}x+a_{32}y+a_{33}}\end{array}\right.---\left(2\right)$

In formula (2), (x, y) is the coordinate before converting, (x ', y ') be the coordinate after conversion, the boundary rectangle of four points after changes persuing is changed, the as a whole eyes after distortion and the rectangular area at face place;

Step 5.4, has obtained eyes after integral transformation and the rectangular area coordinate at face place, eyes and the corresponding rectangular area of face is stretched or is compressed, to realize the local exaggerated deformation in these regions; For oxeye or large face, keep rectangle width constant, by the length of rectangle divided by 0.8; For pigsney or little face, keep rectangle width constant, the length of rectangle is multiplied by 0.8; Realize the exaggerated deformation to eyes and these regional areas of face, obtained final cartoon image f _pi(x, y);

Step 6 is right process frame by frame, the keyframe sequence image that obtains having caricature effect is: D={f _p1(x, y), f _p2(x, y) ..., f _pm(x, y) };

For the f inputting ₁(x, y), f ₂(x, y) ..., f _m(x, y) carries out the processing of step 2 to step 5 one by one, obtains respectively corresponding f _p1(x, y), f _p2(x, y) ..., f _pm(x, y), the keyframe sequence with caricature effect is: D={f _p1(x, y), f _p2(x, y) ..., f _pm(x, y) };

Step 7, reads in keyframe sequence D, between every two key frames, adopts cross decomposition method to carry out intermediate frame generation, obtains the caricature formula animation sequence image of face video image;

Concrete grammar is, reads in keyframe sequence D, adopts cross decomposition method to carry out intermediate frame generation at every two key frames, and computing formula is suc as formula (3):

F _pu(x,y)＝(1-t)f _pi(x,y)+tf _p(i+1)(x,y) (3)

F in formula (3) _pu(x, y) is at key frame f _pi(x, y) and key frame f _{p (i+1)}the intermediate frame image generating between (x, y), wherein the value of t is from 0 to 1.

The caricature formula animation producing method of face video image of the present invention, aspect the generation of face illustrated painting, is further improved the image after USM sharpening, can obviously remove the redundancy on face, makes face portrait more succinct; Use STASM can utilize fully face facial information to locate accurately unique point, be conducive to follow-up shape of face judgement and local exaggeration; The method that adopts whole and part to combine, can more well hold face's ratio, make the caricature generating there is happiness sense and not lose original romantic charm, solved the problem that existing cartoon style is inconsistent, facial positions is out of proportion, make the illustrated painting of generation more natural, be rich in artistic feeling.

Claims (8)

1. a caricature formula animation producing method for face video image, is characterized in that, specifically implements according to following steps:

Step 1, reads in pending face video S={F ₁(x, y), F ₂(x, y) ..., F _n(x, y) }, and its extraction key frame is obtained wherein n is the length of face sequence of frames of video, and m is the number of the key frame of extraction;

Step 2, reads in key frame images f _i(x, y), and it is carried out to illustrated painting processing obtain illustrated painting image f _i' (x, y);

Step 3, reads in illustrated painting image f _i' (x, y), and it is carried out to face characteristic location, obtain face feature point set P=(x ₁, y ₁, x ₂, y ₂..., x _p, y _p), and calculate the ratio at the each position of face face according to face feature point, face is carried out to characteristic discover;

Step 4, to illustrated painting image f _i' (x, y) carry out bulk deformation, obtains image f _ti(x, y);

Step 5, to image f _ti(x, y) carries out part exaggeration, obtains final cartoon image f _pi(x, y);

Step 6 is right process frame by frame, the keyframe sequence image that obtains having caricature effect is: D={f _p1(x, y), f _p2(x, y) ..., f _pm(x, y) };

Step 7, reads in keyframe sequence D, between every two key frames, adopts cross decomposition method to carry out intermediate frame generation, obtains the caricature formula animation sequence image of face video image.

2. the caricature formula animation producing method of face video image according to claim 1, is characterized in that, the concrete grammar of step 1 is that establishing pending face sequence of video images is S={F ₁(x, y), F ₂(x, y) ..., F _n(x, y) }, adopt dual threshold algorithm to carry out key-frame extraction to S, obtain keyframe sequence image $\tilde{S}=\{f_1(x,y),f_2(x,y),...,f_m(x,y)\}.$

3. the caricature formula animation producing method of face video image according to claim 1 and 2, is characterized in that, the concrete grammar of step 2 is,

Step 2.1, to image f _i(x, y) carries out USM sharpening processing, and asking the image after sharpening according to formula (1) is t (x, y);

$\left\{\begin{array}{c}h(x,y)=f_i(x,y)*g(x,y)\\ t(x,y)=(1+d)f_i(x,y)-(d-\mathrm{\&eta;})h(x,y)\end{array}\right.---\left(1\right)$

G (x, y)=exp ((x in formula (1) ²+ y ²)/2 σ ²), σ is level and smooth scale parameter; D gets 4.0, is to adjust from f _ithe amount of the smooth picture deducting in (x, y); η gets 0.3, is adjustment experience factor;

Step 2.2, facial image t (x after traversal sharpening, y) each pixel, using 128 as global threshold, the pixel that is greater than threshold value is set to 255,0 and threshold value between pixel be set to the σ power of this pixel value, equal 0 pixel value constant, σ value is 1.21, obtains illustrated painting image f _i' (x, y).

4. the caricature formula animation producing method of face video image according to claim 3, is characterized in that, the concrete grammar of step 3 is:

Step 3.1, by the f obtaining in step 2 _i' (x, y), use Adaboost algorithm to carry out face detection, determine human face region; Then use the face characteristic location detection software STASM increasing income based on active shape model ASM to position face facial zone unique point, obtain face feature point set P=(x ₁, y ₁, x ₂, y ₂..., x _p, y _p), wherein p=1,2 ..., 77;

Step 3.2, the ratio of calculating the each position of face face according to point set P, comprises the wide width=x of face ₁₂-x ₀, the long length=y of face ₆-y ₁₄, the wide eyew=x of human eye ₃₀-x ₃₄, people's face width mouthw=x ₆₅-x ₅₉, with the width facew=x of face of face same level ₉-x ₃, face length breadth ratio k _f=length/width, human eye width accounts for the width ratio k of same level face _e=eyew/width, face width accounts for the ratio k of same level face width _m=mouthw/facew, the upper presiding judge in face three front yards is top=y ₂₂-y ₁₄, middle presiding judge is middle=y ₅₅-y ₂₂, lower presiding judge is bottom=y ₆-y ₅₅;

Step 3.3, characteristic discover: establishing standard faces length breadth ratio is 5/4, and establishes d _f=k _f-5/4, d _ffor the irrelevance of face length breadth ratio and standard faces length breadth ratio, if d _f< 0.073 is wide face; If 0.073≤d _f< 0.18 is long face; Otherwise be standard face;

If for long face be divided into following four kinds of situation: a, three front yards are isometric, three presiding judge's degree do poorly between two, error is all less than 2; B, upper presiding judge, i.e. top > middle and top > bottom; C, middle presiding judge, i.e. middle > top and middle > bottom; D, lower presiding judge, i.e. bottom > top and bottom > middle;

The wide ratio of the accurate human eye Kuan Zhan of bidding same level face is 3/10, and establishes d _e=k _e-3/10, d _efor the irrelevance of the wide ratio of human eye Kuan Zhan same level face and standard value, if-0.5 < d _e<-0.4, is pigsney, otherwise is oxeye;

It is 1/2 that the accurate face width of bidding accounts for the wide ratio of same level face, and establishes d _m=k _m-1/2, d _mfor face width accounts for the irrelevance of the wide ratio of same level face and standard value, if-0.07 < d _m<-0.05, is little face, otherwise is large face.

5. the caricature formula animation producing method of face video image according to claim 4, is characterized in that, the concrete grammar of step 4 is:

According to different shapes of face and three front yard situations, to illustrated painting image f _i' (x, y) carry out integral transformation and obtain image f _ti(x, y), transform method is to use bilinear interpolation and affined transformation, concrete change situation is as follows:

Step 4.1, if wide face carries out reduced overall to face, keeps picture traverse constant, and height is 0.618 times of source images;

Step 4.2, if long face and be the isometric situations in three front yards carries out integrally stretching to face, keeps picture traverse constant, and height is 1.618 times of source images;

Step 4.3, if long face and upper presiding judge's situation is carried out the long stretching of longitudinal trapezoidal upper base to integral image, carries out 4 affined transformations to image, specifically by f _i' (x, y) four summits are respectively (0,0), (width, 0), (0, length), (width, length) image conversion is to (0,0), (width, 0), (0.2 × width, length), (0.8 × width, length), in the trapezoid area forming, remainder is filled by white;

Step 4.4, if long face and middle presiding judge's situation, integral image is carried out to Shen word stretching, go up 1/3rd long stretchings of going to the bottom, in 1/3rd Uniform Tensions, lower 1/3rd upper bases are long to stretch, specifically: upper 1/3rd long stretchings of going to the bottom are four summits to be respectively to (0,0), (width, 0), form image-region transform to 4 for (0.2 × width, 0), (0.8 × width, 0), the region forming; In 1/3rd Uniform Tensions be that four summits are respectively $(0,\frac{1}{3}\&times;\mathrm{length}),(\mathrm{width},\frac{1}{3}\&times;\mathrm{length}),(0,\frac{2}{3}\&times;\mathrm{length}),(\mathrm{width},\frac{2}{3}\&times;\mathrm{length})$ 4 of transforming to of image-region that form are $(0,\frac{1}{3}\&times;2.618\&times;\mathrm{length}),(\mathrm{width},\frac{1}{3}\&times;2.618\&times;\mathrm{length})$ The region forming; The long stretching of lower 1/3rd upper bases is that four summits are respectively 4 of transforming to of image-region that (0, length), (width, length) form are $(0,\frac{1}{3}\&times;2.618\&times;\mathrm{length}),(\mathrm{width},\frac{1}{3}\&times;2.618\&times;\mathrm{length}),$ $(0.2\&times;\mathrm{width},\frac{1}{3}\&times;3.618\&times;\mathrm{length}),(0.8\&times;\mathrm{width},\frac{1}{3}\&times;3.618\&times;\mathrm{length})$ The region forming;

Step 4.5, if long face and lower presiding judge's situation is carried out longitudinally trapezoidal long stretching of going to the bottom of entirety to integral image, carries out 4 affined transformations to image, specifically by f _i' (x, y) four summits are respectively (0,0), (width, 0), (0, length), (width, length) transform to (0.2 × width, 0), (0.8 × width, 0), (0, length), (width, length), in the trapezoid area forming, remainder is filled by white.

6. the caricature formula animation producing method of face video image according to claim 5, is characterized in that, the concrete grammar of step 5 is:

Step 5.1, according to 4 respective coordinates before and after integral transformation in step 4, calculates the transformation matrix that the affined transformation that adopts in integral transformation is used $T=\left[\begin{array}{ccc}{a}_{11}& a_{12}& a_{13}\\ a_{21}& a_{22}& a_{23}\\ a_{31}& a_{32}& a_{33}\end{array}\right];$

Step 5.2 is found out respectively the rectangular area that upper and lower, left and right four point coordinate of eyes before integral transformation and face form from unique point set P;

Step 5.3, finds out eyes after bulk deformation and the rectangular area at face place, is specially: for wide face situation, keep former eyes and face rectangular area width constant, hypermutation is original 0.618 times; For the isometric situation of long face and three front yards, keep former eyes and face rectangular area width constant, hypermutation is original 1.618 times; For trapezoidal stretching and Shen word pulled out condition, the coordinate according to after formula (2) computational transformation:

$\left\{\begin{array}{c}{x}^{\&prime;}=\frac{a_{11}x+a_{12}y+a_{13}}{a_{31}x+a_{32}y+a_{33}}\\ y^{\&prime;}=\frac{a_{21}x+a_{22}y+a_{23}}{a_{31}x+a_{32}y+a_{33}}\end{array}\right.---\left(2\right)$

In formula (2), (x, y) is the coordinate before converting, (x ', y ') be the coordinate after conversion, the boundary rectangle of four points after changes persuing is changed, the as a whole eyes after distortion and the rectangular area at face place;

Step 5.4, has obtained eyes after integral transformation and the rectangular area coordinate at face place, eyes and the corresponding rectangular area of face is stretched or is compressed, to realize the local exaggerated deformation in these regions; For oxeye or large face, keep rectangle width constant, by the length of rectangle divided by 0.8; For pigsney or little face, keep rectangle width constant, the length of rectangle is multiplied by 0.8; Realize the exaggerated deformation to eyes and these regional areas of face, obtained final cartoon image f _pi(x, y).

7. the caricature formula animation producing method of face video image according to claim 1, is characterized in that, the concrete grammar of step 6 is: right process frame by frame, for the f inputting ₁(x, y), f ₂(x, y) ..., f _m(x, y) carries out the processing of step 2 to step 5 one by one, obtains respectively corresponding f _p1(x, y), f _p2(x, y) ..., f _pm(x, y), the keyframe sequence with caricature effect is: D={f _p1(x, y), f _p2(x, y) ..., f _pm(x, y) }.

8. the caricature formula animation producing method of face video image according to claim 6, it is characterized in that, the concrete grammar of step 7 is to read in keyframe sequence D, adopt cross decomposition method to carry out intermediate frame generation at every two key frames, computing formula is suc as formula (3):

F _pu(x,y)＝(1-t)f _pi(x,y)+tf _p(i+1)(x,y) (3)

F in formula (3) _pu(x, y) is at key frame f _pi(x, y) and key frame f _{p (i+1)}the intermediate frame image generating between (x, y), wherein the value of t is from 0 to 1.