CN100461204C - Method for recognizing facial expression based on 2D partial least square method - Google Patents

Abstract

A method for identifying expression of face based on 2-D bias least square method includes dividing a training sample pattern to be seven types of expressions and dividing them to be a numbers of sub-blocks in equal size, picking up vein character of each sub-block by utilizing LBP operator to form character matrix of partial vein, carrying out character pick-up on partial vein character matrix by 2-D bias least square method to form seven types of expression template data for finalizing train process and using the most close face as expression attribution of inputted image.

Description

A kind of human facial expression recognition method based on the 2 D partial least square method

Technical field

The present invention relates to a kind of human facial expression recognition method, particularly a kind of image local feature extracting method based on the 2 D partial least square method.

Background technology

(Local Binary Pattern, LBP) operator is that a fixed size is 3 * 3 rectangular block to local binary, altogether corresponding to 9 gray-scale values.8 gray-scale values are all around compared with the center gray-scale value, sub-piece more than or equal to the center gray-scale value is represented by 1, otherwise then by 0 expression, 8 binary values reading according to clockwise direction are as the eigenwert of this 3 * 3 rectangular block, and final encoded radio is exactly the LBP value on this aspect.This operator can be measured and extract the texture information of local adjacent domain in the gray level image.Therefore, the LBP operator has been widely used in fields such as Texture classification, image retrieval, facial image analysis.

Traditional human facial expression recognition method based on the LBP operator roughly can be divided into two kinds.The one, sample image is divided into the plurality of sub piece, utilize the LBP operator that each sub-piece is carried out texture feature extraction, the LBP feature with all sub-pieces in the sub-picture is together in series as the characteristic of this this image of duplicate sample then.At last, utilize simple nearest neighbor classifier to obtain classification results.Though it is fast that this method has feature extraction speed, the advantage that computation complexity is low.But its defective also is significantly, at first the LBP feature of each sub-piece directly is together in series to characterize sample image and destroyed the structural information of each sub-piece in original image; Secondly after the intact image texture features of LBP operator extraction, just adopt nearest neighbor classifier to classify and make classification results too coarse, be difficult to obtain gratifying recognition correct rate.Two are to use the sub-piece of variable-size, and in conjunction with learning algorithms such as AdaBoost, in the training process of these class methods, the AdaBoost algorithm can be chosen the plurality of sub piece for the human facial expression recognition most critical automatically, and forms final strong classifier and finish classification feature.This method can obtain very high recognition correct rate owing to adopted the sorting algorithm of exquisite powerful (State-of-the-art).But the distinct issues of such algorithm are that the training of AdaBoost algorithm is extremely consuming time, and the sorter of a stalwartness of training needs several days at least, even the time in several weeks, and this has just seriously restricted the widespread use of these class methods.

Summary of the invention

The present invention just is to overcome above-mentioned defective, development, a kind of human facial expression recognition method based on the 2 D partial least square method of exploitation.

Technical scheme of the present invention is:

A kind of human facial expression recognition method based on the 2 D partial least square method, its major technique step is as follows:

A) with the training sample image collection according to seven kinds of expression classification, promptly indignation, detest, frightened, glad, neutral, sad and surprised;

B) sample image is divided into equal-sized m sub-piece;

C) utilize the textural characteristics of each sub-piece of LBP operator extraction;

D) the LBP textural characteristics that every width of cloth sample image is extracted constitutes a local grain eigenmatrix;

E) adopt adaptive weighted mechanism, promptly give weights to m sub-piece, the computing method of weights are: all neutral facial expression images in the training sample are obtained neutral average face, and extract the local grain eigenmatrix A=[a of this neutrality average face ₁..., a _m], then obtain corresponding local grain eigenmatrix respectively for all espressiove images $H_i=[h_1^i,\·\·\·,h_m^i],$ Utilize card side's distance calculation to go out all espressiove images and the LBP histogram difference of neutral average face image on certain height piece then, and with the adaptive weighting of the LBP histogram difference on certain height piece as this sub-piece, as follows:

$e_j=\underset{i}{\mathrm{\Σ}}{x}^2(a_j,h_j^i),(j=1,\·\·\·,m)$

x ²It is the expression formula of card side's distance;

F) the local grain eigenmatrix that uses the 2 D partial least square method that all sample images are obtained carries out the statistical nature extraction, forms seven kinds of template datas that expression is corresponding, finishes training process;

G) when new images is imported, according to b) to f) step finishes the texture of new images, the extraction of statistical nature, the template data of seven kinds of expressions that form with step f) is measured then, is the expression ownership of this width of cloth input picture near the person.

It is 2 D partial least square method (Two-dimensional Partial Least Square that advantage of the present invention and effect are to use traditional partial least square method, hereinafter to be referred as 2DPLS), and utilize it to combine the local feature of expressing one's feelings with the local binary operator and extract, and a kind of improved image local feature extracting method has been proposed.

Specific as follows:

One. the present invention is divided into the plurality of sub piece with facial image, utilizes the textural characteristics in the sub-piece of LBP operator extraction, and the textural characteristics that all sub-pieces in every width of cloth image are obtained is combined into a local grain eigenmatrix in order to characterize this width of cloth facial image.This building method not only can extract the local grain feature of facial image effectively, can also keep the spatial information of each part in integral body, simultaneously the difference of the LBP feature of trying to achieve according to corresponding sub block in espressiove image and the neutral face image has designed a kind of adaptive weighted mechanism, makes the more objective and accurate of weight more artificial setting that each sub-piece obtained.

Two. the present invention has adopted more accurate but has calculated easy dimension reduction method local textural characteristics matrix is carried out the discriminant information extraction.Because facial image is converted into the local grain eigenmatrix, by conventional P LS method is extended to 2DPLS, just can handle the sample of matrix form.In the 2DPLS method, structure to class members's relational matrix has carried out corresponding modification, the scalar of representing a sample in the classic method is expanded to a diagonal matrix make its matrix form that adapts to sample, and the weights that utilize adaptive weighted method to obtain embody the difference of contained information importance in the different sub-pieces.Simultaneously, for the singular problem of class members's relation, the generalized inverse analytic solution of its covariance matrix have also been derived.

Three. the present invention obviously is better than traditional second class LBP method aspect time consumption for training.When training sample set was 70 width of cloth images, it was finished training process and only needs 4 seconds, then needs several hours based on traditional second class LBP method, and being more or less the same of both recognition correct rates.

Description of drawings

Fig. 1---basic LBP operator figure among the present invention.

Fig. 2---mesoscale of the present invention is the expansion LBP operator figure of (8,2) and (16,2).

Fig. 3---among the present invention based on topography's feature extracting method synoptic diagram of 2DPLS.

Fig. 4---the Expression Recognition rate figure that obtains under the different sub-piece division conditions among the present invention.

Fig. 5---the Expression Recognition accuracy figure that relevant experiment with the people obtains.

Fig. 6---with the Expression Recognition accuracy figure of the irrelevant experiment acquisition of people.

Embodiment

One, at first sets up the 2 D partial least square method

1.1 partial least square method (Partial Least Square, PLS)

Being provided with two class means is zero stochastic variable { r ₁, r ₂..., r _n..., r _i∈ R ^p, { s ₁, s ₂..., s _n..., s _i∈ R ^q, the target of PLS method will find a pair of projecting direction (weight vector) α and β exactly, makes projection $\tilde{r}={\mathrm{\α}}^{T}r,\tilde{s}={\mathrm{\β}}^{T}s$ Satisfy:

1)

With

The variation information that comprises each independent variable as much as possible, promptly $\mathrm{Var}\left(\tilde{r}\right)\→\max ,$ $\mathrm{Var}\left(\tilde{s}\right)\→\max .$

2) With

Between degree of correlation reach maximum, promptly $\mathrm{\ρ}(\tilde{r},\tilde{s})\→\max .$

Comprehensive above 2 constraint conditions 1), 2) can draw, the PLS method is according to maximization

With

Covariance try to achieve the best projection direction, that is:

$\arg \underset{{\mathrm{\α}}^T\mathrm{\α}=1,{\mathrm{\β}}^T\mathrm{\β}=1}{\max }\left\{\mathrm{var}\left({\mathrm{\α}}^{T}r\right)\mathrm{\ρ}{({\mathrm{\α}}^{T}r,{\mathrm{\β}}^{T}s)}^2\mathrm{var}\left({\mathrm{\β}}^{T}s\right)\right\}=\arg \underset{{\mathrm{\α}}^T\mathrm{\α}=1,{\mathrm{\β}}^T\mathrm{\β}=1}{\max }\left\{{\left[\mathrm{cov}({\mathrm{\α}}^{T}r,{\mathrm{\β}}^{T}s)\right]}^2\right\}---\left(2\right)$

From above-mentioned modeling process about PLS as can be seen, constraint condition 1), 2) embodied the thought of principal component analysis (PCA) (PCA) and canonical correlation analysis (CCA) respectively, so the PLS method can be regarded as the CCA method of carrying out respectively after the PCA in r and s space.Consider that s is that the class members of arteface concerns variable in solving the pattern classification problem, formula (2) correspondingly is revised as:

$\arg \underset{{\mathrm{\α}}^T\mathrm{\α}=1,{\mathrm{\β}}^T\mathrm{\β}=1}{\max }\left\{\frac{{\left[\mathrm{cov}({\mathrm{\α}}^{T}r,{\mathrm{\β}}^{T}s)\right]}^2}{\mathrm{var}\left({\mathrm{\β}}^{T}s\right)}\right\}---\left(3\right)$

Following formula is a constrained extreme-value problem, can be found the solution by method of Lagrange multipliers.Can obtain following secular equation group through calculating us:

$\left\{\begin{array}{c}{\mathrm{\Σ}}_{\mathrm{rs}}{\mathrm{\Σ}}_{\mathrm{ss}}^{-1}{\mathrm{\Σ}}_{\mathrm{sr}}\mathrm{\α}=\mathrm{\λ\α}\\ {\mathrm{\Σ}}_{\mathrm{sr}}{\mathrm{\Σ}}_{\mathrm{rr}}^{-1}{\mathrm{\Σ}}_{\mathrm{rs}}\mathrm{\β}=\mathrm{\λ\β}\end{array}\right.---\left(4\right)$

Wherein, ∑ _Rr, ∑ _Ss, ∑ _Rs=∑ _Sr ^TBe respectively stochastic variable r, the auto-covariance matrix of s and cross covariance matrix.

Obviously, we can obtain satisfied 1 and 2 the projecting direction that retrains by finding the solution secular equation group (4).In addition, we can try to achieve α and β respectively to should be noted that through type (4), but for pattern recognition problem, it is significant having only α.

1.2 the 2 D partial least square method (Two-dimensional PLS, 2DPLS)

(Two-dimensional PLS, 2DPLS) the maximum difference with conventional P LS method is to use matrix form to characterize sample image to the two dimension least square method.If $X=[x_{11},x_{12},\&CenterDot;\&CenterDot;\&CenterDot;x_{1{\mathrm{<figure-callout\; id="1"\; label="n"\; filenames="C200710019405E00121.png"\; state="\{\{state\}\}">n</figure-callout>}}_1},x_{21},x_{22},\&CenterDot;\&CenterDot;\&CenterDot;,x_{2{\mathrm{<figure-callout\; id="2"\; label="n"\; filenames="C200710019405E00121.png"\; state="\{\{state\}\}">n</figure-callout>}}_2},\&CenterDot;\&CenterDot;\&CenterDot;x_{C{n}_c}]$ Be sample data, wherein x _Ij, (i=1 ..., C; J=1 ..., n _i) be that size is the matrix of h * l, n _i, (i=1 ..., C) be the interior sample size of i class, then total N=n in the sample set ₁+ n ₂+ n _CIndividual sample image.Therefore, the average of sample is $\stackrel{\&OverBar;}{X}=(1/N){\mathrm{\&Sigma;}}_{i=1}^C{\mathrm{\&Sigma;}}_{j=1}^{n_i}{x}_{\mathrm{ij}}.$ For the pattern classification problem, the sample data collection can be used as one group of stochastic variable in the 2DPLS method, is called the sample data matrix, and another group stochastic variable then needs arteface.

Next utilize the information of the affiliated classification of sample image to construct another group stochastic variable, be called class members's relational matrix.In traditional C CA method, class members's relational matrix has the building method of following two kinds of equivalences usually:

　　　　

Wherein

Be that element is 1 n entirely _i* 1 column vector is represented to comprise n in the i class _iIndividual sample. Be that element is 0 n entirely _i* 1 column vector represents not comprise in other kinds sample in the i class.In other words formula (5) is represented class members's relation table, and which kind of a certain sample belongs to, and then puts 1 in the relevant position of class members's relational matrix, other then be 0.So class members's relational matrix of structure has well shown each sample and all kinds of attaching relations.For the 2DCCA method, scalar characterizes sample image because it uses matrix, therefore according to the form of sample data matrix X class members's relational matrix has been carried out following modification on the basis of formula (5):

For the 2DPLS method, scalar characterizes sample data because it uses matrix, therefore according to the form of sample data matrix X class members's relational matrix has been carried out following modification equally on the basis of formula (5):

　　　

Wherein, P _iExpression belongs to the sample in the i class.But each sample is pairing is not numeral 1 but the unit matrix I that size is l * l _iTherefore, P _iCan be expressed as by matrix I:

Such structure makes class members's relational matrix can not only embody the spatial information that each sample and all kinds of subordinate relation have also kept sample data.In order to try to achieve the average of class members's relational matrix on the two-dimensional sense, we are rewritten as matrix Y $Y=[y_{11},\&CenterDot;\&CenterDot;\&CenterDot;,y_{1{\mathrm{<figure-callout\; id="1"\; label="n"\; filenames="C200710019405E00121.png"\; state="\{\{state\}\}">n</figure-callout>}}_1},\&CenterDot;\&CenterDot;\&CenterDot;,y_{C{n}_C}],$ Y wherein _IjIt is (the matrix of l * C) * l.Therefore, the average of class members's relational matrix is $\stackrel{\&OverBar;}{Y}=(1/N){\mathrm{\&Sigma;}}_{i=1}^C{\mathrm{\&Sigma;}}_{j=1}^{n_i}{y}_{\mathrm{ij}}.$

Behind the structure of finishing sample data matrix X and class members's relational matrix Y, can try to achieve the best projection direction w of 2DPLS according to formula (6) _XAnd w _Y:

$\arg \max \left\{\frac{{\left[\mathrm{cov}(w_X^{T}X,w_Y^{T}Y)\right]}^2}{\mathrm{var}\left(w_Y^{T}Y\right)}\right\}---\left(8\right)$

Equally, formula (8) also is the extreme-value problem of a belt restraining.Adopt the method for Lagrange multipliers can be in the hope of following feature problem system of equations:

$\left\{\begin{array}{c}{\mathrm{\&Sigma;}}_{\mathrm{XY}}{\mathrm{\&Sigma;}}_{\mathrm{YY}}^{-1}{\mathrm{\&Sigma;}}_{\mathrm{YX}}{w}_X=\mathrm{\&lambda;}{w}_X\\ {\mathrm{\&Sigma;}}_{\mathrm{YX}}{\mathrm{\&Sigma;}}_{\mathrm{XX}}^{-1}{\mathrm{\&Sigma;}}_{\mathrm{XY}}{w}_Y=\mathrm{\&lambda;}{w}_Y\end{array}\right.---\left(9\right)$

Wherein, ${\mathrm{\&Sigma;}}_{\mathrm{XX}}={\mathrm{\&Sigma;}}_{i=1}^C{\mathrm{\&Sigma;}}_{j=1}^{n_i}(x_{\mathrm{ij}}-\stackrel{\&OverBar;}{X}){(x_{\mathrm{ij}}-\stackrel{\&OverBar;}{X})}^T$ With ${\mathrm{\&Sigma;}}_{\mathrm{YY}}={\mathrm{\&Sigma;}}_{i=1}^C{\mathrm{\&Sigma;}}_{j=1}^{n_i}(y_{\mathrm{ij}}-\stackrel{\&OverBar;}{Y}){(y_{\mathrm{ij}}-\stackrel{\&OverBar;}{Y})}^T$ Be the auto-covariance matrix of sample data matrix X and class members's relational matrix Y, ${\mathrm{\&Sigma;}}_{\mathrm{XY}}={\mathrm{\&Sigma;}}_{\mathrm{YX}}^T={\mathrm{\&Sigma;}}_{i=1}^C{\mathrm{\&Sigma;}}_{j=1}^{n_i}(x_{\mathrm{ij}}-\stackrel{\&OverBar;}{X}){(y_{\mathrm{ij}}-\stackrel{\&OverBar;}{Y})}^T$ It then is the cross covariance matrix of two matrixes.

From above derivation as can be seen, in the 2DPLS method, still to face class members's relational matrix Y and auto-covariance matrix ∑ thereof _YYTherefore the problem of full rank need not find the solution the matrix ∑ _YYGeneralized inverse.Equally, according to because matrix Y and matrix ∑ _YYThe singularity of matrix form, try to achieve the matrix ∑ _YYGeneralized inverse For:

${\mathrm{\&Sigma;}}_{\mathrm{YY}}^g={\left[\begin{array}{cc}{\mathrm{\&Sigma;}}_{\mathrm{ZZ}}^g& 0_{\{h\&times;(C-1)\}\&times;h}\\ 0_{h\&times;\{h\&times;(C-1)\}}& 0_{h\&times;h}\end{array}\right]}_{(l\&times;C)\&times;(l\&times;C)}---\left(10\right)$

Wherein, ${\mathrm{\&Sigma;}}_{\mathrm{ZZ}}^g=((1/n_C)R_{C-1}{R}_{C-1}^T+M^g)$ Generalized inverse for the auto-covariance matrix of matrix z in the formula (10).

Two, the selection of sample image neutron piece number

Divide the scope that sub-piece number has directly determined LBP operator extraction local grain feature on the facial image, the selection of sub-piece number is a part and the whole compromise problem that combines.This experiment utilizes the selection to the determinant piece number of the division experience of the different sizes of facial image.Consider effective embodiment of local feature and the feasibility on the algorithm, facial image is divided into 16,36,64,144 respectively, 200 sub-pieces, promptly every block size is respectively 30 * 30, and 20 * 20,15 * 15,10 * 10,6 * 6.Training set then is each picked at random 1 width of cloth image from everyone difference expression, and totally 70 width of cloth images are formed, and are remaining as test set.Carry out 5 average Expression Recognition accuracy of obtaining of experiment as shown in Figure 4 according to above strategy.From table as can be seen, group piece number is 64, and promptly each sub-block size is 15 * 15 o'clock, and the recognition correct rate that is obtained is the highest.This result has proved that the selection of sub-piece number need take all factors into consideration local and whole relation, and is excessive as the fruit piece, then lost the locality meaning, and sub-piece is too little, then can not guarantee enough statistical significances.Therefore, in following experiment, selecting size is that 15 * 15 sub-piece carries out five equilibrium to image.

Three, the local binary operator (Local Binary Pattern, LBP)

The LBP operator is that a fixed size is 3 * 3 rectangular block, altogether corresponding to 9 gray-scale values; 8 gray-scale values are all around compared with the center gray-scale value, sub-piece more than or equal to the center gray-scale value is represented by 1, otherwise then by 0 expression, 8 binary values reading according to clockwise direction are as the eigenwert of this 3 * 3 rectangular block, final encoded radio is exactly the LBP value on this aspect, as shown in Figure 1.Mathematical description is: definition of T=t (g _c, g ₀G _P-1) represent textures windows to be processed, wherein g _cThe central point of expression window, g ₀G _P-1P-1 point around the expression central point.Deducting the texture image that obtains behind the gray-scale value of central point is T=t (g _c, g ₀-g _c..., g _P-1-g _c), if think that the gray scale of central point and gray scale on every side are independently, T ≈ t (g can be write as in so above-mentioned texture image unit ₀-g _c... g _P-1-g _c), and then can be write as T=t (s (g again ₀-g _c), s (g ₁-g _c) ..., s (g _P-1-g _c)), x wherein〉0 o'clock, s (x)=1; X≤0 o'clock, s (x)=0 by giving the factor of a scale-of-two 2p of each distribution, obtains the numerical value LBP of the LBP of this final window at last _{P, R}=∑ ^PS (g _c-g _p) 2 ^p, wherein, P, R classify, and expression is counted and the size of radius

In order to improve the limitation of the textural characteristics that can't extract large-scale structure that initial LBP operator exists, use of a kind of main expansion of the rectangular block of the adjacent sub-blocks of varying number and different size as the LBP operator.Fig. 2 is the example of the LBP operator of two expansions, and wherein (P R) is illustrated in and has P sample point on the circumference that radius is R.The ability of the local grain that the extraction different characteristic of counting of the window of different sizes, different numbers is described is also different, and the content of describing local grain is also different.

In addition, when adjacent sub-blocks quantity and radius size are very big, have most of very limited in the feature that extracts to the description of texture.Therefore, Ojala has proposed another kind of expansion to the LBP operator and has been called " evenly pattern (Uniform Pattern) ".This time, the numerical value of LBP of each window was not the net result of absolute binary code, but calculate in each window the situation of change of code element in the binary code, different situations of change has been represented different local features, for example 00000000 has the zero code element to change, and 00011110 has two code elements to change.Concrete mathematical description is:

${\mathrm{LBP}}_{P,\mathrm{<figure-callout\; id="2"\; label="R\; riu"\; filenames="C200710019405E00121.png"\; state="\{\{state\}\}">R</figure-callout>}}^{\mathrm{riu}}=\left\{\begin{array}{cc}{\mathrm{\&Sigma;}}_{p=0}^{p=1}s(g_p-g_c)& \mathrm{if\; U}({\mathrm{LBP}}_{P,R}\&le;2)\\ P+1& \mathrm{otherwise}\end{array}\right.---\left(11\right)$

Wherein

$U\left({\mathrm{LBP}}_{P,R}\right)=|s(g_{P-1}-g_c)-s(g_0-g_c)|+{\mathrm{\&Sigma;}}_{p=1}^{p-1}|s(g_{P-1}-g_c)-s(g_0-g_c)|---\left(12\right)$

Subscript riu2 is the mark of an employing " evenly pattern ".Timo Ahonen is through studies show that, in 59 " Unified coding " of LBP, the probability that often occurs on the FERET storehouse is 79.3%.Experimental result proves that evenly pattern can effectively be described most textural characteristics in the picture of publishing picture, and reduces the quantity of feature greatly.

Analyze this quasi-mode classification problem for facial sex, need measure to determine the classification ownership of test data the difference between test data and the training data.Because the LBP feature characterizes with represented as histograms, uses nearest neighbor classifier to finish above function usually.The present invention uses chi method (x ²) measure the histogrammic difference of LBP, as follows:

${\mathrm{\&chi;}}^2(S,M)=\underset{i}{\mathrm{\&Sigma;}}\frac{{(S_i-M_i)}^2}{S_i+M_i}---\left(13\right)$

Wherein S and M represent two LBP histograms respectively.

Four, the corresponding local grain eigenmatrix of foundation and sample image

Sample image is divided into m sub-piece, uses the LBP histogram h in the sub-piece of LBP operator extraction _i∈ R ^k, and m LBP histogram of this image of duplicate sample formed the local grain eigenmatrix of k * m dimension, as shown in Figure 3.As above structure not only can effectively extract the local grain feature of facial image, can also keep the spatial information of local feature in image.

Five, adaptive weighted mechanism and statistical nature extract

In addition, the characteristic information that comprises in m the sub-piece in the image obviously is different for the importance of expression classification, so we have designed a kind of adaptive weighted mechanism and give rational weights to m sub-piece.At first all neutral facial expression images in the training sample are obtained neutral average face, and extract the local grain eigenmatrix A=[a of this neutrality average face ₁..., a _m], then obtain corresponding local grain eigenmatrix respectively for all espressiove images $H_i=[h_1^i,\&CenterDot;\&CenterDot;\&CenterDot;,h_m^i].$ Utilize card side's distance calculation to go out all espressiove images and LBP histogram (promptly be a certain row of the corresponding topical textural characteristics matrix) difference of neutral average face image on certain height piece then, gained difference and as the adaptive weighting of this sub-piece.As follows:

$e_j=\underset{i}{\mathrm{\&Sigma;}}{x}^2(a_j,h_j^i),(j=1,\&CenterDot;\&CenterDot;\&CenterDot;,m)---\left(1\right)$

We use the 2DPLS method that the sample image that characterizes with local grain eigenmatrix form is carried out dimensionality reduction.When structure class members relational matrix, the diagonal entry of unit matrix is to embody the different importance of different sub-pieces for Expression Recognition in the adaptive weighting alternate form (6) that will be tried to achieve by formula (1).According to secular equation group (9), we just can try to achieve eigenmatrix w _xFrom this matrix, choose maximum several features value characteristic of correspondence vector structure projection matrix W and be used for dimension-reduction treatment.

Six, the judgement that the input picture expression is belonged to

Suppose local grain eigenmatrix H that size is h * l of input, can obtain matrix behind the dimensionality reduction by following equation

For:

$\tilde{H}=W^{T}H---\left(14\right)$

Wherein, if from matrix w _xIn chosen d vector, then projection matrix W and matrix

Size be respectively l * d and d * l.When differentiating, utilize card side's distance matrix metric

And the distance between training sample behind the dimensionality reduction, determine the expression of input picture according to nearest neighbouring rule.

Seven, experimental result and analysis

Two kinds of experiments: with relating to persons, irrelevant with the people.With the experiment emphasis of relating to persons be that because if same individual's shape of face is accurately located the well-done words of work, remaining difference mainly is exactly the difference of expressing one's feelings on the face on the discriminability of test expression itself.What on the contrary, the experiment that has nothing to do with the people was mainly tested is the popularization problem of Expression Recognition problem on standard expression storehouse and non-standard expression storehouse.

Experiment is based on Japanese female face (JAFFE) database of expressing one's feelings.When extracting, all adopts local grain LBP (8, the 1) operator of " evenly pattern ".All experiments under the condition of 1GB internal memory, use the MATLAB7.0 software platform to realize all at P4 3.0G CPU.In order to reduce the be bold influence of factors such as little and illumination variation of people's face position, people in the image, before experiment, image has been carried out following pre-service:

1. with human face region segmented extraction in the image, and unified size is 120 * 120.

2. the centre coordinate based on eyes and mouth carries out normalization to the position of people's face.In order to guarantee accuracy, centre coordinate is all by manual demarcation.

3. image is carried out illumination compensation and histogram equalization.

7.1 the experiment relevant with the people

The purpose of relevant experiment with the people is the discriminability of the same individual different table feelings of test.Training set is each picked at random 1 width of cloth image from everyone difference expression, and totally 70 width of cloth images are formed, and are remaining as test set.And using this paper method and classic method to carry out 5 experiments, the averaging of income recognition correct rate is respectively 87.86% and 78.41%.Figure 5 shows that the recognition correct rate that every class expression obtains in this experiment, the recognition correct rate that obtained of this paper method is apparently higher than traditional LBP method as can be seen, this raising comes from this paper method and has adopted 2DPLS that local textural characteristics matrix has been carried out the dimension-reduction treatment that helps differentiating, can obtain more expression discriminant information than directly utilizing the LBP histogram to carry out template matches in traditional LBP method.Angry, glad, surprised as seen from the figure simultaneously discrimination is comparatively desirable, and this is that the LBP operator is easy to catch texture difference therebetween because this several expressions variation on local detail is bigger.Detest, frightened and sad discrimination is then lower, this is because the performance of the face of these three kinds of expressions is quite similar, even manually all easily it is obscured.In addition, for the training set basis that 70 width of cloth facial images constitute, this paper method is finished training process and is only needed 4 seconds, finds out that thus this paper method does not obviously increase computing cost when significantly improving recognition correct rate.

7.2 the experiment that has nothing to do with the people

The experiment that has nothing to do with the people is in order to test this paper method at the generalization ability aspect the Expression Recognition, irrelevant with the people is exactly that image in the test set and the image in the training set are from different objects, so all facial expression images that we select 4 people at random are remaining as test sample book as training sample.Same this paper method and the classic method used carried out 5 experiments, and experimental result as shown in Figure 6.Can find by Fig. 6: the first, to compare with the experiment relevant with the people, recognition correct rate all descends to some extent.The difference of this explanation different people on face shape can produce a very large impact Expression Recognition.The second, the recognition correct rate that this paper method is obtained in this experiment still surpasses traditional LBP method comprehensively, has proved that further the local feature that adopts based on 2DPLS extracts the validity of expression feature extraction and the superiority of more traditional LBP method.The 3rd, be again that indignation, happiness and surprised three kinds of expressions have obtained the highest recognition correct rate in this experiment, this has illustrated that the big more expression generalization ability of facial deformation is good more.

Claims (3)

1. human facial expression recognition method based on the 2 D partial least square method, its step is as follows:

A) with the training sample image collection according to seven kinds of expression classification, promptly indignation, detest, frightened, glad, neutral, sad and surprised;

B) sample image is divided into equal-sized m sub-piece;

C) utilize the textural characteristics of each sub-piece of LBP operator extraction;

D) the LBP textural characteristics that every width of cloth sample image is extracted constitutes a local grain eigenmatrix;

E) adopt adaptive weighted mechanism, promptly give weights to m sub-piece, the computing method of weights are: all neutral facial expression images in the training sample are obtained neutral average face, and extract the local grain eigenmatrix A=[a of this neutrality average face ₁..., a _m], then obtain corresponding local grain eigenmatrix respectively for all espressiove images $H_i=[h_1^i,\&CenterDot;\&CenterDot;\&CenterDot;,h_m^i],$ Utilize card side's distance calculation to go out all espressiove images and the LBP histogram difference of neutral average face image on certain height piece then, and with the adaptive weighting of the LBP histogram difference on certain height piece as this sub-piece, as follows:

$e_j=\underset{i}{\mathrm{\&Sigma;}}{x}^2(a_j,h_j^i),(j=1,\&CenterDot;\&CenterDot;\&CenterDot;,m)$

x ²It is the expression formula of card side's distance;

F) the local grain eigenmatrix that uses the 2 D partial least square method that all sample images are obtained carries out the statistical nature extraction, forms seven kinds of template datas that expression is corresponding, finishes training process;

G) when new images is imported, according to b) to f) step finishes the texture of new images, the extraction of statistical nature, the template data of seven kinds of expressions that form with step f) is measured then, is the expression ownership of this width of cloth input picture near the person.

2. a kind of human facial expression recognition method based on the 2 D partial least square method according to claim 1 is characterized in that the selection of the neutron of sample image described in step b) piece number, and facial image is divided into 64 sub-pieces, and promptly every block size is 15 * 15.

3. a kind of human facial expression recognition method according to claim 1 based on the 2 D partial least square method, the method for building up that it is characterized in that local grain eigenmatrix in the step d) is: sample image is divided into m sub-piece, use the LBP histogram in the sub-piece of LBP operator extraction, and m the LBP histogram of this image of duplicate sample formed the local grain eigenmatrix of k * m dimension.

Images