CN100383807C - Feature point positioning method combined with active shape model and quick active appearance model - Google Patents

Abstract

The present invention relates to a characteristic point positioning method combined with movable shape models and fast movable appearance models in the technical field of image processing. The present invention comprises the procedures: 1. Lucas AAM models established, initial parameters are calculated, and initial positions of models are given; 2. ASM models are established; 3. the initial positions of models obtained in the procedure (1) and a Lucas AAM method are used for searching characteristic points of human faces; 4. characteristic points of human faces searched by the Lucas AAM models are used as initial positions, and new positions of characteristic points are searched in an ASM method. The present invention combines the ASM method and the Lucas AAM method together to carry out the positioning of characteristic points of human faces, and in this way, the defect of low accuracy of ASM global search can be compensated by the advantage of favorable effect of Lucas AAM global search. Meanwhile, the characteristic of poor local search of Lucas AAM can be compensated by the advantage of favorable effect of ASM local search. Meanwhile, because the Lucas AAM method is carried out slowly, the speed of the whole characteristic point search can also be improved greatly after the ASM method is combined.

Description

The characteristic point positioning method of combining movement shape and Fast Activities display model

Technical field

What the present invention relates to is a kind of method of technical field of image processing, specifically is the characteristic point positioning method of a kind of combining movement shape and Fast Activities display model.

Background technology

The research in people's face field receives the concern of increasing researcher in recent years, and the face characteristic point location is the key link of whole people's face research field, and the accuracy of face characteristic point location directly affects the various aspects of follow-up people's face research.

Find by prior art documents, " moving shape model-its training and application " that T.F.Cootes etc. delivered on " computer vision and image understanding " (the 38th page of first phase nineteen ninety-five), ASM (moving shape model) method that this article proposes, in this method, when carrying out the search of unique point reposition, on perpendicular to the one-dimensional profile on the direction of former and later two unique point lines, find the center of the sub-profile that makes the mahalanobis distance minimum and set the reposition that this center is current unique point, but this method is very responsive to the initial position of unique point, if the initial position of current unique point is when its target location, search precision is higher, and when wide position, initial position, search precision can sharply descend, so the Local Search precision height of this method, and the global search precision is lower.Iain Matthews etc. " the movable appearance model review " on " the international periodical of computer vision " (2004 the 2nd phase the 135th page), delivered simultaneously, this article has proposed a kind of fast face characteristic point positioning method that utilizes image alignment method principle to carry out the AAM search, this method has extraordinary global search effect, even initial characteristics point wide position, this method also can search near the target location.But the Local Search effect of this method is not as the ASM method, and as a rule, this method can not fully accurately be located this unique point.

Summary of the invention

The present invention is directed to the deficiencies in the prior art, a kind of man face characteristic point positioning method in conjunction with ASM and Lucas AAM is proposed, it is combined ASM and two methods of Lucas AAM (Fast Activities display model) carry out the face characteristic point location, the shortcoming that ASM global search precision is low can be compensated by the effective advantage of Lucas AAM global search like this, meanwhile, the characteristics of Lucas AAM Local Search difference also can be compensated by the effective advantage of ASM Local Search, therefore the two is combined and can make up for each other's deficiencies and learn from each other, Sou Suo characteristic point position can be very accurate like this.Simultaneously, because Lucas AAM method is relatively slow, incorporated the speed that also can improve the full feature point search after the ASM method greatly.

The present invention is achieved by the following technical solutions, comprises the steps:

(1) set up Lucas AAM model, calculate initial parameter, provide the initial position of model;

(2) set up the local grain of ASM model and unique point;

(3) initial position of the model that obtains with step (1) and with Lucas AAM method seeker face characteristic point;

(4) unique point that arrives with Lucas AAM pattern search is as initial position, with ASM method search characteristics point.

In the described step (1), set up Lucas AAM model, be meant: at first select the principal character point of k people's face on each training sample image of training set, the shape of this k unique point composition can be by a vector x (i)=[x ₁, x ₂..., x _k, y ₁, y ₂..., y _k] represent, unique point with identical numbering has been represented identical feature in different images, n training sample image makes their represented shapes the most approaching on size, direction and position just to n shape vector should be arranged thereby calibrate this n vector then.Then the shape vector after n the calibration is carried out PCA (pivot analysis) and handle, finally any one shape can be expressed as $x=\stackrel{\‾}{x}+\mathrm{Pb},$ Wherein $b=P^T\·(x-\stackrel{\‾}{x}),$ B has represented the situation of change of preceding t maximum pattern, has so just set up Lucas AAM shape.Set up the characteristic point position and the average shape of training sample image then with piecewise linearity affine deformation method

Between mapping relations, and training sample image is deformed to average shape with this relation

And the gray-scale value of each picture element in the average shape after the distortion pulled into a vector, i.e. the texture of this training sample image, the length of this texture is average shape

The number of inner picture element, n training sample image are carried out pivot analysis to n texture vector then and are handled just to n texture vector should be arranged, and finally any one texture can be expressed as $A\left(x\right)=A_0\left(x\right)+\underset{i=1}{\overset{m}{\mathrm{\Σ}}}{\mathrm{\λ}}_i{A}_i\left(x\right),\∀x\∈\stackrel{\‾}{x},$ So just set up Lucas AAM texture model.

In the described step (1), calculate initial parameter, be meant: compute gradient decline image

Wherein

Be the gradient of average texture, and

It is the Jacobian of piecewise linearity affine deformation.According to $H=\underset{x}{\mathrm{\Σ}}{[\▿A_0\frac{\∂W}{\∂p}]}^T[\▿A_0\frac{\∂W}{\∂p}]$ Calculate Hessian (a kind of Jacobi matrix) matrix H.

In the described step (1), the initial position of computation model is meant: find two positions with the variance projection function on facial image, and set in two point coordinate and be [x1, Y1].To above-mentioned average shape model of trying to achieve

The center of calculating four unique points around the eyeball of the left and right sides respectively is as left and right sides eye position, thereby obtains two middle point coordinate [X2, Y2], then whole average shape model

Translation [X1-X2, Y1-Y2] so just obtains the initial position of model, thereby can be used for search.

Described step (2) is meant: the foundation for the ASM model is the same with the foundation of shape in the previous step.Also need to set up its local grain for each unique point in the training sample image, be that m pixel respectively selected on the both sides, center on perpendicular to former and later two unique point line directions of current unique point promptly with current unique point, calculate this (2m+1) thus the gray-scale value derivative of individual pixel and normalization obtain a profile (vector that is made of the derivative of the gray-scale value of pixel).The profile that remembers j unique point in i the shape vector is g _Ij, then j unique point profile's is average, $\stackrel{\‾}{g_j}=\frac{1}{n}\underset{i=1}{\overset{n}{\mathrm{\Σ}}}{g}_{\mathrm{ij}},$ Its variance is $C_j=\frac{1}{n}\underset{i=1}{\overset{n}{\mathrm{\Σ}}}(g_{\mathrm{ij}}-\stackrel{\‾}{g_j})\·{(g_{\mathrm{ij}}-\stackrel{\‾}{g_j})}^T,$ K unique point all calculated the average and variance of its profile, thereby just obtained the local grain of k unique point.

Described step (3) is meant: the initial position of the model that obtains with step (1) is searched for Lucas AAM method, and concrete steps are as follows:

A) by current p basis $x=\stackrel{\‾}{x}+\mathrm{Pb}$ Calculated characteristics point position is arrived the deformation texture that current characteristic point position surrounds with the linear affine deformation method of segmentation And obtain texture vector I (W (x; P)).

B) calculated difference image I (W (x; P))-A ₀(x).

C) calculate ${\mathrm{\Σ}}_x{[\▿A_0\frac{\∂W}{\∂p}]}^T\left[I\left(W(x;p)\right)-A_0\left(x\right)\right].$

D) calculate $\mathrm{\Δp}=H^{-1}{\mathrm{\Σ}}_x{[\▿A_0\frac{\∂W}{\∂p}]}^T[I\left(W(x;p)\right)-A_0\left(x\right)].$

E) by formula W (x; P) ← W (x; P) ο W (x; Δ p) ^-1Renewal obtains new P value.

After iterating, by formula $x=\stackrel{\‾}{x}+\mathrm{Pb}$ Obtain new shape, i.e. the position of unique point.

Described step (4) is meant: as initial position, and utilize the ASM searching method to carry out unique point search in image with the Search Results that obtains in the previous step, this search procedure mainly is that the variation by affined transformation and parameter b realizes.Specifically realize by following two steps that iterate:

A) calculate the reposition of each unique point

At first initial ASM model is covered on the image, for j unique point in the model, be that the individual pixel of 1 (1 greater than m) is respectively selected on the both sides, center on perpendicular to its former and later two unique point line directions with it, thereby the gray-scale value derivative and the normalization of calculating this 1 pixel then obtain a profile, in this new profile, get length and be designated as temp (P), define an energy function for the sub-profile of (2*m+1) $f_j\left(p\right)=(\mathrm{temp}\left(P\right)-\stackrel{\‾}{g_j})\·C_j^{-1}\·{(\mathrm{temp}\left(P\right)-\stackrel{\‾}{g_j})}^T,$ With this energy function pass judgment on current sub-profile with

Between similarity, select to make f _j(p) Zui Xiao position is as the reposition of this unique point, and calculates it and change dX _j, each unique point is all carried out such calculating just obtains k change in location dX _i, i=1,2 ..., k, and form a vectorial dX=(dX ₁, dX ₂..., dX _k).

B) renewal of parameter in the affined transformation and b

By formula X=M (s, θ) [x]+X _c: M (s (1+ds), (θ+d θ)) [x+dx]+(X _c+ dX _c)=(X+dX), M (s (1+ds), (θ+d θ)) [x+dx]=M (s, θ) [x]+dX+X _c-(X _c+ dX _c), by formula $x=\stackrel{\‾}{x}+\mathrm{Pb},$ Now wish to find db to make $x+\mathrm{dx}=\stackrel{\‾}{x}+P(b+\mathrm{db}),$ By formula $x=\stackrel{\‾}{x}+\mathrm{Pb}$ Can get db=P ^-1Dx so just can make following renewal: X to parameter _c=X _c+ w _tDX _c, Y _c=Y _c+ w _tDY _c, θ=θ+w _θD θ, b=b+W _bDb, w in the formula _i, w _θ, w _s, W _bBe to be used for the weights that controlled variable changes, like this by formula $x=\stackrel{\‾}{x}+\mathrm{Pb}$ Can obtain new shape.

The man face characteristic point positioning method in conjunction with ASM and these two kinds of methods of Lucas AAM that the present invention proposes has very high precision.Owing to use Lucas AAM method to carry out the coarse search of unique point at the beginning, this can find the approximate location of unique point, and then with faceted search before the ASM method to the position carry out the essence search of unique point as initial position, search precision is very high under this prerequisite, but also is difficult for being absorbed in local minimum.With man face characteristic point positioning method and original ASM method in conjunction with ASM and these two kinds of methods of Lucas AAM that the face database contrast the present invention who takes proposes, the average error of both positioning feature point of front and back is respectively 2.1 pixels and 4.5 pixels.Experiment shows that the method that the present invention proposes is more accurate than further feature independent positioning method.Replace Lucas AAM method with the ASM method simultaneously in subsequent searches, its speed also is greatly improved.

Description of drawings

Fig. 1 is the facial image that indicates unique point.

Fig. 2 is the result of eye location.

Fig. 3 is the initial position of unique point.

The result of Fig. 4 for obtaining after the Lucas AAM search.

The result of Fig. 5 for obtaining after the ASM search.

Embodiment

Below in conjunction with a specific embodiment technical scheme of the present invention is described in further detail.

The image that embodiment adopts is from the facial image database of taking.Whole implement process is as follows:

1. from face database, select 500 facial images of having marked unique point to set up the ASM model.Marked the facial image of unique point, as shown in Figure 1.Corresponding 500 shape vectors and texture vector are done the foundation that Lucas AAM model has promptly been finished in the pivot analysis processing respectively.Any one shape can be expressed as like this $x=\stackrel{\‾}{x}+\mathrm{Pb},$ Any one texture can be expressed as simultaneously $A\left(x\right)=A_0\left(x\right)+\underset{i=1}{\overset{m}{\mathrm{\Σ}}}{\mathrm{\λ}}_i{A}_i\left(x\right),\∀x\∈\stackrel{\‾}{x}.$

Calculate initial parameter: compute gradient decline image

Wherein

Be the gradient of average texture, and

It is the Jacobian of the linear affine deformation of segmentation.According to $H=\underset{x}{\mathrm{\Σ}}{[\▿A_0\frac{\∂W}{\∂p}]}^T[\▿A_0\frac{\∂W}{\∂p}]$ Calculate the Hessian matrix H.

The initial position of computation model: two positions about finding with the variance projection function on the facial image, they are respectively [274,229] and [371,228], as shown in Figure 2, thereby can obtain point coordinate in two [322.5,228.5].To above-mentioned average shape model of trying to achieve

The center of calculating left and right sides eyeball four unique points on every side respectively is as left and right sides eye position, and they are respectively [52.79 ,-48.76] and [52.03 ,-49.22], thereby obtain two middle point coordinate [0.37 ,-48.99], then whole average shape model

Translation [322.5-(0.37), 228.5-(48.99)] so just obtains the initial position of model, as shown in Figure 3.

2. the same with the foundation of shape in the previous step for the foundation of ASM model.Also need to set up its local grain for each unique point in the training sample image, be that 5 pixels are respectively selected on the both sides, center on perpendicular to former and later two unique point line directions of current unique point promptly with current unique point, calculate this 11 (2*5+1) thus the gray-scale value derivative of individual pixel and normalization obtain a profile.The profile that remembers j unique point in i the shape vector is g _Ij, then j unique point profile's is average, $\stackrel{\‾}{g_j}=\frac{1}{n}\underset{i=1}{\overset{n}{\mathrm{\Σ}}}{g}_{\mathrm{ij}},$ Its variance is $C_j=\frac{1}{n}\underset{i=1}{\overset{n}{\mathrm{\Σ}}}(g_{\mathrm{ij}}-\stackrel{\‾}{g_j})\·{(g_{\mathrm{ij}}-\stackrel{\‾}{g_j})}^T,$ 60 unique points are all calculated the average and variance of its profile, thereby just obtained the local grain of 60 unique points.

With resulting initial position in the first step as reference position, search for Lucas AAM method, iteration is 30 times altogether, Search Results is as shown in Figure 4.

With the Search Results that obtains in the previous step as initial position, and utilize the ASM searching method in image, to carry out unique point search.Just can finally locate 60 unique points through 5 step iteration, as shown in Figure 5.

By experiment, can find that method proposed by the invention all is greatly improved than two kinds of original methods on precision and speed.

Claims (7)

1. the characteristic point positioning method of combining movement shape and Fast Activities display model is characterized in that, comprises the steps:

(1) set up Lucas AAM model, calculate initial parameter, provide the initial position of model;

(2) set up the local grain of ASM model and unique point;

(3) initial position of the model that obtains with step (1) and with Lucas AAM method seeker face characteristic point;

(4) unique point that arrives with Lucas AAM pattern search is as initial position, with ASM method search characteristics point.

2. the characteristic point positioning method of combining movement shape according to claim 1 and Fast Activities display model, it is characterized in that, in the described step (1), set up Lucas AAM model, be meant: at first select k people's face principal character point on each sample image of training set, the shape of this k unique point composition is by a vector x (i)=[x ₁, x ₂..., x _k, y ₁, y ₂..., y _k] represent, the unique point of identical numbering has been represented identical feature in different images, n sample image is just to there being n shape vector, calibrating this n vector makes their represented shapes the most approaching on size, direction and position, shape vector after n the calibration is carried out pivot analysis handle, any one shape all is expressed as $x=\stackrel{\‾}{x}+\mathrm{Pb}$ , wherein $b=P^T\·(x-\stackrel{\‾}{x})$ , b has represented the situation of change of a preceding t max model, and this has just set up Lucas AAM shape; The unique point of setting up sample image with piecewise linearity affine deformation method with Between mapping relations, and sample image is deformed to average shape with this relation

, and the gray-scale value of each picture element in the average shape after the distortion pulled into a vector, i.e. the texture of this training sample image, the length of this texture is average shape

The number of inner picture element, n training sample image are carried out pivot analysis to n texture vector then and are handled just to n texture vector should be arranged, and finally any one texture all is expressed as $A\left(x\right)=A_0\left(x\right)+\underset{i=1}{\overset{m}{\mathrm{\Σ}}}{\mathrm{\λ}}_i{A}_i\left(x\right),\∀x\∈\stackrel{\‾}{x}$ , so just set up Lucas AAM texture model.

3. the characteristic point positioning method of combining movement shape according to claim 1 and Fast Activities display model is characterized in that, in the described step (1), calculates initial parameter, is meant: compute gradient decline image

Be the average texture gradient, Be the Jacobian of piecewise linearity affine deformation, by $H=\underset{x}{\mathrm{\Σ}}{[\▿A_0\frac{\∂W}{\∂p}]}^T[\▿A_0\frac{\∂W}{\∂p}]$ Calculate the Hessian matrix.

4. the characteristic point positioning method of combining movement shape according to claim 1 and Fast Activities display model, it is characterized in that, in the described step (1), the initial position of computation model, be meant: find two positions with the variance projection function, and set in two point coordinate and be [X1, Y1], to above-mentioned average shape model of trying to achieve

, the center of calculating four unique points around the eyeball of the left and right sides respectively is as left and right sides eye position, thereby obtains two middle point coordinate [X2, Y2], then whole average shape model Translation [X1-X2, Y1-Y2] so just obtains the initial position of model.

5. the characteristic point positioning method of combining movement shape according to claim 1 and Fast Activities display model, it is characterized in that, described step (2), be meant: the foundation for the ASM model is the same with the foundation of shape in the previous step, also need to set up its local grain for each unique point in the training sample image, be that m pixel respectively selected on the both sides, center on perpendicular to former and later two unique point line directions of current unique point promptly with current unique point, thereby the gray-scale value derivative and the normalization of calculating this 2m+1 pixel obtain a profile, remember that the profile of j unique point in i the shape vector is g _Ij, then j unique point profile's is average, $\stackrel{\‾}{g_j}=\frac{1}{n}\underset{i=1}{\overset{n}{\mathrm{\Σ}}}{g}_{\mathrm{ij}}$ , its variance is $C_j=\frac{1}{n}\underset{i=1}{\overset{n}{\mathrm{\Σ}}}(g_{\mathrm{ij}}-\stackrel{\‾}{g_j})\·{(g_{\mathrm{ij}}-\stackrel{\‾}{g_j})}^T$ , k unique point all calculated the average and variance of its profile, thereby just obtained the local grain of k unique point.

6. the characteristic point positioning method of combining movement shape according to claim 1 and Fast Activities display model is characterized in that, described step (3) is meant: the initial position with model is searched for Lucas AAM method, and concrete steps are as follows:

A) by current p basis $x=\stackrel{\‾}{x}+\mathrm{Pb}$ Calculate current characteristic point position, the deformation texture that current unique point is surrounded with the linear affine deformation method of segmentation arrives

, obtain a texture vector I (W (x; P));

B) calculated difference image I (W (x; P))-A ₀(x);

C) calculate ${\mathrm{\Σ}}_x{[\▿A_0\frac{\∂W}{\∂p}]}^T[I\left(W(x;p)\right)-A_0\left(x\right)];$

D) calculate $\mathrm{\Δp}=H^{-1}{\mathrm{\Σ}}_x{[\▿A_0\frac{\∂W}{\∂p}]}^T[I\left(W(x;p)\right)-A_0\left(x\right)];$

E) by formula W (x; P) ← W (x; P) oW (x; Δ p) ^-1Renewal obtains new P value;

After iterating, by formula $x=\stackrel{\‾}{x}+\mathrm{Pb}$ Obtain new shape, i.e. the position of unique point.

7. the characteristic point positioning method of combining movement shape according to claim 1 and Fast Activities display model, it is characterized in that, described step (4), be meant: the result who searches with Lucas AAM is as initial position, and utilize the ASM searching method in image, to carry out the unique point search, this search procedure mainly is that the variation by affined transformation and parameter b realizes, specifically realizes by following two steps that iterate:

A) calculate the reposition of each unique point

For j unique point in the model, be that 1 pixel is respectively selected on the both sides, center on perpendicular to its former and later two unique point line directions with it, 1 greater than m, thereby the gray-scale value derivative and the normalization of calculating this 1 pixel then obtain a profile, in this new profile, get length and be designated as temp (P), define an energy function for the sub-profile of (2*m+1) $f_j\left(p\right)=(\mathrm{temp}\left(P\right)-\stackrel{\‾}{g_j})\·C_j^{-1}\·{(\mathrm{temp}\left(P\right)-\stackrel{\‾}{g_j})}^T$ , with this energy function pass judgment on current sub-profile with

Between similarity, select to make f _j(p) Zui Xiao position is as the reposition of this unique point, and calculates it and change dX _j, each unique point is all carried out such calculating just obtains k change in location dX _i, i=1,2 ..., k, and form a vectorial dX=(dX ₁, dX ₂..., dX _k);

B) renewal of parameter in the affined transformation and b is by formula X=M (s, θ) [x]+X _c: M (s (1+ds), (θ+d θ)) [x+dx]+(X _c+ dX _c)=(X+dX), M (s (1+ds), (θ+d θ)) [x+dx]=M (s, θ) [x]+dX+X _c-(X _c+ dX _c), by formula $x=\stackrel{\‾}{x}+\mathrm{Pb},$ Now wish to find db to make $x+\mathrm{dx}=\stackrel{\‾}{x}+P(b+\mathrm{db}),$ By formula $x=\stackrel{\‾}{x}+\mathrm{Pb}$ Get db=P ^-1Dx so just can make following renewal: X to parameter _c=X _c+ w _tDX _c, Y _c=Y _c+ w _tDY _c, θ=θ+w _θD θ, b=b+W _bDb, w in the formula _t, w _θ, w _s, W _bBe to be used for the weights that controlled variable changes, by formula $x=\stackrel{\‾}{x}+\mathrm{Pb}$ Get shape newly.

Images