CN101908149A - Method for identifying facial expressions from human face image sequence - Google Patents

Abstract

The invention relates to a method for identifying facial expressions from a human face image sequence, belonging to the technical field of analyzing and identifying human facial expressions. The method of the invention comprises the following steps of: firstly, adopting a method for tracing feature points, sequentially extracting the displacement amount of the normalized facial key point and the length of the special geometrical characteristic for each frame image of the expression image sequence, and combining the data to form a characteristic column vector; secondly, sequentially arranging all characteristic column vectors of the sequence to form a characteristic matrix, wherein each characteristic matrix represents a facial expression image sequence; finally, comparing the similarities among the characteristic matrixes by using a canonical correlation analysis method, thereby determining the human face images to be identified into one of the basic expressions of happiness, sadness, fear, hate, surprise and anger. In the invention, the canonical correlation analysis method is successfully applied to identifying the human facial expressions, the dynamic information in the expression generation course is utilized effectively and the higher recognition rate and the shorter CPU computation time are acquired.

Description

A kind of method of from human face image sequence, discerning countenance

Technical field

The present invention relates to a kind of method of from human face image sequence, discerning countenance, belong to human face expression analysis and recognition technology field.

Background technology

Along with the fast development of computer technology, human face expression analysis automatically and recognition technology will make countenance become a new channel (channel) of man-machine interaction, and allow reciprocal process become more natural and effective.Countenance analysis and identification comprise three basic problems: 1. how to find in image and people from location face; 2. how from detected face image or face image sequence, to extract effective expressive features; 3. how to design appropriate sorting technique and come Identification Lists affectionate person class.In recent years, having a lot of research work to be devoted to discern human face expression: Cohn etc. from image sequence has proposed a kind of method based on light stream and has discerned slight change in the countenance in document " Feature-Point Tracking by Optical FlowDiscriminates Subtle Differences in Facial Expression " (Int ' l Conf.Automatic Face and Gesture Recognition, pp.396-401 (1998)).(IVCNZ 2008 at document " Facial expression recognition from image sequences using optimizedfeature selection " for people such as Laj evardi, New Zealand, Page (s): 1-6 (2008)) discloses a kind of Feature Selection process in, used naive Bayesian (NB) sorter to discern the method for human face expression by optimizing.Sun Zhengxing etc. document " towards the LSVM algorithm of video sequence expression classification " (computer-aided design (CAD) and graphics journal. the 21st the volume, the 4th phase (2009)) disclose a kind of employing in and from video human face, extracted the expression geometric properties based on the moving shape model (ASM) of a tracking, and the method for using local support vector machine (LSVM) sorter that expression is classified.The shortcoming of these methods is: it only extracts feature from peak value expression frame, ignored the important time domain multidate information that the expression production process comprises, so the accuracy rate of its identification is not high.

In addition, " Active and Dynamic Information Fusion for Facial ExpressionUnderstanding from Image Sequences " (IEEE TRANSACTIONS ON PATTERNANALYSIS AND MACHINE INTELLIGENCE, VOL.27, NO.5, MAY 2005), " Manifoldbased analysis of facial expression " (Image and Vision Computing 24 (2006) 605-614), " Facial expression recognition from video sequences:temporal and static modeling " (Computer Vision and Image Understanding91 (2003) 160-187) though etc. the method that proposes of the document time domain multidate information that used the expression production process to comprise, its calculation of complex, computing cost is big.

The important prior art that the present invention uses is: and canonical correlation analysis (CanonicalCorrelation Analysis, CCA).

Canonical correlation analysis is the classical tool in the statistical study, and it can be used to measure the linear relationship between two or more data sets.Canonical correlation coefficient is defined as two d dimensional linear subspace L ₁And L ₂Between leading role θ _iCosine value:

${\cos \mathrm{\θ}}_i=\underset{u_i\∈L_1,v_i\∈L_2}{\max }{u_i}^T{v}_i(1\≤i\≤d);$

Wherein:

${u_i}^T{u}_i={v_i}^T{v}_i=1,{u_i}^T{u}_j={v_i}^T{v}_j=0(i\≠j);$

Parameter d is represented the linear subspaces dimension.

The canonical correlation analysis technology successfully has been applied to fields such as image set coupling, people's face or object identification in recent years, and therefore the canonical correlation analysis technology being used to solve the Expression Recognition problem is a simple but effective method theoretically.But, in the Expression Recognition problem, the face image difference of same individual different table feelings is also little, even the image difference of two opposite expressions neither be very big, so simply the canonical correlation analysis technology is applied in the Expression Recognition, can not obtains effect preferably.So far, also do not find pertinent literature and the practical application that the canonical correlation analysis technology is used for human face expression identification.

Summary of the invention

The objective of the invention is to have proposed a kind of method of from human face image sequence, discerning countenance in order to overcome the deficiency that prior art exists.The present invention uses method that face feature point follows the tracks of to extract the length of normalized face key point displacement and particular geometric feature successively at each two field picture in the facial expression image sequence, and these data are formed a characteristic series vector; All characteristic series vectors in the sequence are arranged in order and are formed an eigenmatrix, and each eigenmatrix is represented a facial expression image sequence; Utilize the similarity between the canonical correlation analysis method comparative feature matrix then, thereby facial image to be identified is defined as one of six kinds of basic facial expressions (happiness, sadness, fear, detest, surprised and angry).

The objective of the invention is to be achieved through the following technical solutions.

A kind of method of from human face image sequence, discerning countenance, its concrete operations step is:

Step 1, selection image sequence

Select the image sequence of representing six kinds of basic facial expressions such as happiness, sadness, fear, detest, surprised and anger from the countenance database, the quantity of the image sequence of every kind of basic facial expression is more than 20; From each facial expression image sequence, choose m (m 〉=10, and m is a positive integer) two field picture, and each facial expression image sequence finishes to the peak value facial expression image from neutral facial expression image.

Step 2, sign face feature point

On the basis of step 1, sign face feature point; Be specially:

The 1st step: identify 20 face feature points successively in first two field picture in each facial expression image sequence; Wherein, the 1st, 2 unique points lay respectively at the brows position of the right eyebrow and left side eyebrow; 3rd, 4 unique points lay respectively at the eyebrow tail position of the right eyebrow and left side eyebrow; 5th, 6 unique points lay respectively at the inner eye corner position of the right eyes and left side eyes; 7th, 8 unique points lay respectively at the minimum point of the right eyes and left side eyes; 9th, 10 unique points lay respectively at the tail of the eye position of the right eyes and left side eyes; 11st, 12 unique points lay respectively at the peak of the right eyes and left side eyes; 13rd, 14 unique points lay respectively at right-most position of the wing of nose and the left-most position of the wing of nose; The 15th unique point is positioned at the nose position; 16th, 17 unique points lay respectively at the right-most position of the corners of the mouth and the left-most position of the corners of the mouth; 18th, 19 unique points lay respectively at lip center line and the crossing highs and lows of lip outline line; The 20th unique point is positioned at face's center line and the crossing minimum point of face mask line.

The method of 20 face feature points of described sign includes but not limited to: 1. identify manually; 2. adopt people such as Vukdadinovic at document " Fully automatic facial feature point detectionusing gabor feature based boosted classifiers " (Proc.IEEE Int ' l Conf.on Systems, Man and Cybernetics, pp.1692-1698 (2005)) the enhancing classifier methods based on the Gabor feature that proposes in realizes the automatic location to 20 face feature points.

The 2nd step: the people's who proposes in document " Anthropometry of the Head and Face " (NewYork:Raven Press (1994)) according to Farkas the eyes and the statistics of cheek, nose and cheek spatial relationship calculate the position of the 21st, 22 unique points in first two field picture in each facial expression image sequence; 21st, 22 unique points lay respectively at the cheekbone position of right side cheek and left side cheek.

The 3rd step: adopt people such as Patras document " Particle filtering with factorizedlikelihoods for tracking facial features " (Proc.Int ' l Conf.AutomaticFace ﹠amp; Gesture Recognition, pp.97-102 (2004)) but in the particle filter tracking method that proposes based on the likelihood function factorization, according to the position of 22 unique points in first two field picture in each facial expression image sequence, follow the tracks of 22 face feature points in the subsequent frame image in each facial expression image sequence.

The 4th step: adopt affine transformation method to adjust the position and the size of people's face in the image, the equal and opposite in direction of people's face in the same image sequence and position are consistent.Be specially:

At first make the line maintenance level of two inner eye corner points in first two field picture in each image sequence; Then, according to the position of these 3 points of uppermost point of two inner eye corner points in first two field picture in the image sequence of place and philtrum, 22 face feature points in remaining each frame are done mapping and normalized; After handling through affined transformation, in the same image sequence people of all images be bold little equate and the position of these 3 points of uppermost point of two inner eye corner points and philtrum and first frame in the position consistency of these 3 points.

Step 3, extraction countenance feature

On the basis of step 2 operation, from every image, extract the countenance feature successively; Be specially:

The 1st step: set up the Xb-Yb coordinate system, this coordinate system is an initial point with the lower left corner of every image, and level is the Xb axle to right, and direction is the Yb axle straight up; Obtain on every image 22 characteristic point coordinates value (xb successively according to 22 unique point place locations of pixels on every image based on the Xb-Yb coordinate system _i, yb _i) and the coordinate figure (x of the uppermost point of people's face philtrum of every image _Origin, y _Origin), wherein, i=1～22, and i is a positive integer;

The 2nd step: set up the X-Y coordinate system, this coordinate system is an initial point with the uppermost point of people's face philtrum of every image, and level is an X-axis to right, and direction is a Y-axis straight up; Obtain 22 characteristic point coordinates value (x on every image by formula 1 and formula 2 based on the X-Y coordinate system _i, y _i);

x _i＝xb _i-x _origin (1)

y _i＝yb _i-y _origin (2)

The 3rd step: the horizontal ordinate displacement that obtains 22 unique points of every image by formula 3 and formula 4

The ordinate displacement

${\mathrm{\Δx}}_i^{\′}=x_i-{\stackrel{\‾}{x}}_i---\left(3\right)$

${\mathrm{\Δy}}_i^{\′}=y_i-{\stackrel{\‾}{y}}_i---\left(4\right)$

Wherein, Be respectively the abscissa value and the ordinate value of the character pair point of first two field picture in this image place image sequence.

The 4th step: obtain the horizontal ordinate displacement Δ x after 22 unique point normalization of every image by formula 5 and formula 6 successively _i, ordinate displacement Δ y _i

${\mathrm{\Δx}}_i={\mathrm{\Δx}}_i^{\′}/x_{\mathrm{base}}---\left(5\right)$

${\mathrm{\Δy}}_i={\mathrm{\Δy}}_i^{\′}/y_{\mathrm{base}}---\left(6\right)$

Wherein, x _Base=x ₆-x ₅, y _Base=y ₆-y ₅x ₅And x ₆Be respectively the abscissa value of the 5th, 6 unique points of this image; y ₅And y ₆Be respectively the ordinate value of the 5th, 6 unique points of this image.

The 5th step: obtain 10 geometric distance feature mfv in every image ₁～mfv ₁₀Be specially:

Obtain eyes stretching degree value mfv according to formula 7 ₁:

mfv ₁＝((y ₁₁-y ₇)+(y ₁₂-y ₈))/2 (7)

Wherein, y ₇, y ₈, y ₁₁, y ₁₂Be respectively the ordinate value of the 7th, 8,11,12 unique points of this image.

Obtain eye widths value mfv according to formula 8 ₂:

mfv ₂＝((x ₅-x ₉)+(x ₁₀-x ₆))/2 (8)

Wherein, x ₅, x ₆, x ₉, x ₁₀Be respectively the abscissa value of the 5th, 6,9,10 unique points of this image.

Obtain brows height value mfv according to formula 9 ₃:

mfv ₃＝(y ₁+y ₂)/2 (9)

Wherein, y ₁, y ₂Be respectively the ordinate value of the 1st, 2 unique points of this image.

Obtain eyebrow tail height value mfv according to formula 10 ₄:

mfv ₄＝(y ₃+y ₄)/2 (10)

Wherein, y ₃, y ₄Be respectively the ordinate value of the 3rd, 4 unique points of this image.

Obtain eyebrow width value mfv according to formula 11 ₅:

mfv ₅＝((x ₁-x ₃)+(x ₄-x ₂))/2 (11)

Wherein, x ₁, x ₂, x ₃, x ₄Be respectively the abscissa value of the 1st, 2,3,4 unique points of this image.

Obtain mouth stretching degree value mfv according to formula 12 ₆:

mfv ₆＝y ₁₈-y ₁₉ (12)

Wherein, y ₁₈, y ₁₉Be respectively the ordinate value of the 18th, 19 unique points of this image.

Obtain mouth width value mfv according to formula 13 ₇:

mfv ₇＝x ₁₇-x ₁₆ (13)

Wherein, x ₁₆, x ₁₇Be respectively the abscissa value of the 16th, 17 unique points of this image.

Obtain nose-corners of the mouth distance value mfv according to formula 14 ₈:

mfv ₈＝((y ₁₅-y ₁₆)+(y ₁₅-y ₁₇))/2 (14)

Wherein, y ₁₅, y ₁₆, y ₁₇Be respectively the ordinate value of the 15th, 16,17 unique points of this image.

Obtain eyes-cheek distance value mfv according to formula 15 ₉:

mfv ₉＝(((y ₁₁+y ₇)/2-y ₂₁)+((y ₁₂+y ₈)/2-y ₂₂))/2 (15)

Wherein, y ₂₁, y ₂₂Be respectively the ordinate value of the 21st, 22 unique points of this image.

Obtain nose-chin distance value mfv according to formula 16 ₁₀:

mfv ₁₀＝y ₁₅-y ₂₀ (16)

Wherein, y ₁₅, y ₂₀Be respectively the ordinate value of the 15th, 20 unique points of this image.

The 6th step: by 10 geometric distance feature mfv in 17 pairs of every images of formula ₁～mfv ₁₀Carry out normalized;

${\mathrm{mfv}}_j={\mathrm{mfv}}_j/{\stackrel{\‾}{\mathrm{mfv}}}_j---\left(17\right)$

Wherein, j=1～10, and j is a positive integer;

Be geometric distance feature corresponding in first two field picture in this image place image sequence.

The 7th step: 10 geometric distance feature mfv that use every image respectively ₁～mfv ₁₀And the horizontal ordinate displacement Δ x that obtains after 22 unique point normalizeds of every width of cloth image _i, ordinate displacement Δ y _iConstitute 54 dimensional vector z of expression one width of cloth face image expression information _kWherein: z _k∈ R ⁵⁴, 1≤k≤m, R represents real number);

The 8th step: use characteristic matrix Z={z ₁, z ₂..., z _m∈ R ^{54 * m}Represent a facial expression image sequence; Z wherein ₁The neutral expression of expression, z _mExpression peak value expression.

Step 4, treat the test pattern sequence and classify;

On the basis of step 3, adopt canonical correlation analysis to treat the test pattern sequence and classify; Be specially:

The 1st step: the image sequence of every kind of basic facial expression choosing in the step 1 is divided into 2 parts at random, and a part is as training data, and a part is as test data; The quantity of training data is Q, Q 〉=20, and Q is a positive integer; Article one, training data is a facial expression image sequence; Article one, test data also is a facial expression image sequence.

The 2nd step: go on foot the training data that obtains at the 1st, people such as employing T.-K.Kim are at document " Discriminative Learning and Recognition of Image Set Classes UsingCanonical Correlations " (IEEE Transactions On Pattern Analysis AndMachine Intelligence, Vol.29, No.6 (2007)) the canonical correlation discriminant analysis method that proposes in is handled, and obtains transformation matrix T ∈ R ^{54 * n}, n＜54, and n is a positive integer; Use transformation matrix T that the eigenmatrix Z of the image in the pantomimia image sequence of choosing in the step 1 (comprising training data and test data) is changed then, obtain Z '=T ^TZ.

The 3rd step: facial expression image sequence of picked at random from the test data described in the 1st step, the eigenmatrix Z ' that calculates this facial expression image sequence and the canonical correlation coefficient of the eigenmatrix Z ' of each training data with.

The 4th the step: the 3rd the step result the basis on, calculate respectively this facial expression image sequence and every kind of basic facial expression canonical correlation coefficient and mean value; Choose the expression of 6 minimum value correspondences in the mean value, as classification results.

Through above-mentioned steps, can finish the Expression Recognition for the treatment of the test pattern sequence.

Beneficial effect

Compare with existing recognition methods, a kind of method of discerning countenance from human face image sequence of the present invention successfully applies to the canonical correlation analysis method in the human face expression identification, effectively utilized the multidate information in the expression production process, and obtained high recognition and less CPU operation time.

Description of drawings

Fig. 1 is 7 two field pictures in certain 15 frame image sequence in the specific embodiment of the present invention;

Fig. 2 is the unique point in the facial image and Xb-Yb coordinate system in the specific embodiment of the present invention, X-Y coordinate system synoptic diagram;

Fig. 3 is the general structure frame synoptic diagram of the inventive method.

Embodiment

The present invention is described in detail below in conjunction with the drawings and specific embodiments.

Use Cohn-Kanade countenance database in the present embodiment, therefrom select 212 image sequences of 50 people of six kinds of basic facial expressions such as representing happiness, sadness, fear, detest, surprised and anger.From each facial expression image sequence, respectively choose 15 two field pictures,, finish to the peak value facial expression image from neutral facial expression image.Figure 1 shows that 7 frames in certain 15 frame image sequence wherein.22 unique points in each facial image and Xb-Yb coordinate system, X-Y coordinate system are as shown in Figure 2.The image sequence of selecting 35 people wherein is as training set, and remaining image sequence guarantees and individual irrelevant human face expression classifying quality with this as test set.Each test all uses the test of picked at random and training set to carry out 5 times, calculates its average result.In study and assorting process, the parameter d of expression linear subspaces dimension is made as 10, and the parameter n of expression transformation matrix T dimension is made as 20.

The general structure frame synoptic diagram of the inventive method as shown in Figure 3.Use the inventive method, the confusion matrix of recognition result is as shown in table 1.The diagonal element of matrix is the number percent that countenance is correctly classified, and nondiagonal element is corresponding to the number percent of mis-classification.The average accuracy rate of this paper method has surpassed 90%.

The confusion matrix (%) of table 1 this paper method recognition result

	Glad	Sad	Frightened	Detest	Surprised	Angry
							Glad	95.2	0	2.6	0	2.2	0
Sad	0	92.5	0	2.8	0	4.7
							Frightened	8.8	0	87.2	0	0	4.0
Detest	0	1.5	7.4	91.1	0	0
							Surprised	5.2	0	1.2	0	90.5	3.1
Angry	0	10.3	0	4.2	0	85.5

For explanation effect of the present invention, use identical data, adopt optimization feature selection approach and LSVM method to test respectively, its result is as shown in table 2.

The comparison (%) of table 2 distinct methods discrimination

	Glad	Sad	Frightened	Detest	Surprised	Angry	Average accuracy rate
								This paper method	95.2	92.5	87.2	91.1	90.5	85.5	90.3
Optimize feature selecting	76.3	67.6	55	100	78	100	79.5
								LSVM	91.3	86.4	89.6	88.3	92.5	86.5	89.1

Test shows that the inventive method has higher accuracy rate, and from the operation steps of the inventive method, the inventive method is simple as can be seen.

The above only is a preferred implementation of the present invention; should be understood that; for those skilled in the art; under the prerequisite that does not break away from the principle of the invention; can also make some improvement; perhaps part technical characterictic wherein is equal to replacement, these improvement and replace and also should be considered as protection scope of the present invention.

Claims (3)

1. One kind from human face image sequence identification countenance method, it is characterized in that: its concrete operations step is:

   Step 1, selection image sequence

   Select the image sequence of representing six kinds of basic facial expressions such as happiness, sadness, fear, detest, surprised and anger from the countenance database, the quantity of the image sequence of every kind of basic facial expression is more than 20; From each facial expression image sequence, choose the m two field picture, wherein: m 〉=10, and m is a positive integer; And each facial expression image sequence finishes to the peak value facial expression image from neutral facial expression image;

   Step 2, sign face feature point

   On the basis of step 1, sign face feature point; Be specially:

   The 1st step: identify 20 face feature points successively in first two field picture in each facial expression image sequence; Wherein, the 1st, 2 unique points lay respectively at the brows position of the right eyebrow and left side eyebrow; 3rd, 4 unique points lay respectively at the eyebrow tail position of the right eyebrow and left side eyebrow; 5th, 6 unique points lay respectively at the inner eye corner position of the right eyes and left side eyes; 7th, 8 unique points lay respectively at the minimum point of the right eyes and left side eyes; 9th, 10 unique points lay respectively at the tail of the eye position of the right eyes and left side eyes; 11st, 12 unique points lay respectively at the peak of the right eyes and left side eyes; 13rd, 14 unique points lay respectively at right-most position of the wing of nose and the left-most position of the wing of nose; The 15th unique point is positioned at the nose position; 16th, 17 unique points lay respectively at the right-most position of the corners of the mouth and the left-most position of the corners of the mouth; 18th, 19 unique points lay respectively at lip center line and the crossing highs and lows of lip outline line; The 20th unique point is positioned at face's center line and the crossing minimum point of face mask line;

   The 2nd step: the people's who proposes in document " Anthropometry of the Head and Face " according to Farkas the eyes and the statistics of cheek, nose and cheek spatial relationship calculate the position of the 21st, 22 unique points in first two field picture in each facial expression image sequence; 21st, 22 unique points lay respectively at the cheekbone position of right side cheek and left side cheek;

   The 3rd step: adopt people such as Patras at document " Particle filtering with factorizedlikelihoods for tracking facial features " but in the particle filter tracking method based on the likelihood function factorization of proposition, according to the position of 22 unique points in first two field picture in each facial expression image sequence, follow the tracks of 22 face feature points in the subsequent frame image in each facial expression image sequence;

   The 4th step: adopt affine transformation method to adjust the position and the size of people's face in the image, the equal and opposite in direction of people's face in the same image sequence and position are consistent;

   Step 3, extraction countenance feature

   On the basis of step 2 operation, from every image, extract the countenance feature successively; Be specially:

   The 1st step: set up the Xb-Yb coordinate system, this coordinate system is an initial point with the lower left corner of every image, and level is the Xb axle to right, and direction is the Yb axle straight up; Obtain on every image 22 characteristic point coordinates value (xb successively according to 22 unique point place locations of pixels on every image based on the Xb-Yb coordinate system _i, yb _i) and the coordinate figure (x of the uppermost point of people's face philtrum of every image _Origin, y _Origin), wherein, i=1～22, and i is a positive integer;

   The 2nd step: set up the X-Y coordinate system, this coordinate system is an initial point with the uppermost point of people's face philtrum of every image, and level is an X-axis to right, and direction is a Y-axis straight up; Obtain 22 characteristic point coordinates value (x on every image by formula 1 and formula 2 based on the X-Y coordinate system _i, y _i);

   x _i＝xb _i-x _origin (1)

   y _i＝yb _i-y _origin (2)

   The 3rd step: the horizontal ordinate displacement that obtains 22 unique points of every image by formula 3 and formula 4

   

   The ordinate displacement

   

   ${\mathrm{\Δx}}_i^{\′}=x_i-{\stackrel{\‾}{x}}_i---\left(3\right)$

   ${\mathrm{\Δy}}_i^{\′}=y_i-{\stackrel{\‾}{y}}_i---\left(4\right)$

   Wherein,

   

   Be respectively the abscissa value and the ordinate value of the character pair point of first two field picture in this image place image sequence;

   The 4th step: obtain the horizontal ordinate displacement Δ x after 22 unique point normalization of every image by formula 5 and formula 6 successively _i, ordinate displacement Δ y _i

   ${\mathrm{\Δx}}_i={\mathrm{\Δx}}_i^{\′}/x_{\mathrm{base}}---\left(5\right)$

   ${\mathrm{\Δy}}_i={\mathrm{\Δy}}_i^{\′}/y_{\mathrm{base}}---\left(6\right)$

   Wherein, x _Base=x ₆-x ₅, y _Base=y ₆-y ₅x ₅And x ₆Be respectively the abscissa value of the 5th, 6 unique points of this image; y ₅And y ₆Be respectively the ordinate value of the 5th, 6 unique points of this image;

   The 5th step: obtain 10 geometric distance feature mfv in every image ₁～mfv ₁₀Be specially:

   Obtain eyes stretching degree value mfv according to formula 7 ₁:

   mfv ₁＝((y ₁₁-y ₇)+(y ₁₂-y ₈))/2 (7)

   Wherein, y ₇, y ₈, y ₁₁, y ₁₂Be respectively the ordinate value of the 7th, 8,11,12 unique points of this image;

   Obtain eye widths value mfv according to formula 8 ₂:

   mfv ₂＝((x ₅-x ₉)+(x ₁₀-x ₆))/2 (8)

   Wherein, x ₅, x ₆, x ₉, x ₁₀Be respectively the abscissa value of the 5th, 6,9,10 unique points of this image;

   Obtain brows height value mfv according to formula 9 ₃:

   mfv ₃＝(y ₁+y ₂)/2 (9)

   Wherein, y ₁, y ₂Be respectively the ordinate value of the 1st, 2 unique points of this image;

   Obtain eyebrow tail height value mfv according to formula 10 ₄:

   mfv ₄＝(y ₃+y ₄)/2 (10)

   Wherein, y ₃, y ₄Be respectively the ordinate value of the 3rd, 4 unique points of this image;

   Obtain eyebrow width value mfv according to formula 11 ₅:

   mfv ₅＝((x ₁-x ₃)+(x ₄-x ₂))/2 (11)

   Wherein, x ₁, x ₂, x ₃, x ₄Be respectively the abscissa value of the 1st, 2,3,4 unique points of this image;

   Obtain mouth stretching degree value mfv according to formula 12 ₆:

   mfv ₆＝y ₁₈-y ₁₉ (12)

   Wherein, y ₁₈, y ₁₉Be respectively the ordinate value of the 18th, 19 unique points of this image;

   Obtain mouth width value mfv according to formula 13 ₇:

   mfv ₇＝x ₁₇-x ₁₆ (13)

   Wherein, x ₁₆, x ₁₇Be respectively the abscissa value of the 16th, 17 unique points of this image;

   Obtain nose-corners of the mouth distance value mfv according to formula 14 ₈:

   mfv ₈＝((y ₁₅-y ₁₆)+(y ₁₅-y ₁₇))/2 (14)

   Wherein, y ₁₅, y ₁₆, y ₁₇Be respectively the ordinate value of the 15th, 16,17 unique points of this image;

   Obtain eyes-cheek distance value mfv according to formula 15 ₉

   mfv ₉＝(((y ₁₁+y ₇)/2-y ₂₁)+((y ₁₂+y ₈)/2-y ₂₂))/2 (15)

   Wherein, y ₂₁, y ₂₂Be respectively the ordinate value of the 21st, 22 unique points of this image;

   Obtain nose-chin distance value mfv according to formula 16 ₁₀:

   mfv ₁₀＝y ₁₅-y ₂₀ (16)

   Wherein, y ₁₅, y ₂₀Be respectively the ordinate value of the 15th, 20 unique points of this image;

   The 6th step: by 10 geometric distance feature mfv in 17 pairs of every images of formula ₁～mfv ₁₀Carry out normalized;

   ${\mathrm{mfv}}_j={\mathrm{mfv}}_j/{\stackrel{\‾}{\mathrm{mfv}}}_j---\left(17\right)$

   Wherein, j=1～10, and j is a positive integer;

   

   Be geometric distance feature corresponding in first two field picture in this image place image sequence;

   The 7th step: 10 geometric distance feature mfv that use every image respectively ₁～mfv ₁₀And the horizontal ordinate displacement Δ x that obtains after 22 unique point normalizeds of every width of cloth image _i, ordinate displacement Δ y _iConstitute 54 dimensional vector z of expression one width of cloth face image expression information _kWherein: z _k∈ R ⁵⁴, 1≤k≤m, R represents real number);

   The 8th step: use characteristic matrix Z={z ₁, z ₂..., z _m∈ R ^{54 * m}Represent a facial expression image sequence; Z wherein ₁The neutral expression of expression, z _mExpression peak value expression;

   Step 4, treat the test pattern sequence and classify

   On the basis of step 3, adopt canonical correlation analysis to treat the test pattern sequence and classify; Be specially:

   The 1st step: the image sequence of every kind of basic facial expression choosing in the step 1 is divided into 2 parts at random, and a part is as training data, and a part is as test data; The quantity of training data is Q, Q 〉=20, and Q is a positive integer; Article one, training data is a facial expression image sequence; Article one, test data also is a facial expression image sequence;

   The 2nd step: go on foot the training data that obtains at the 1st, the canonical correlation discriminant analysis method that people such as employing T.-K.Kim propose in document " Discriminative Learning and Recognition of Image Set Classes UsingCanoni cal Correlations " is handled, and obtains transformation matrix T ∈ R ^{54 * n}, n＜54, and n is a positive integer; Use transformation matrix T that the eigenmatrix Z of the image in the pantomimia image sequence of choosing in the step 1 is changed then, obtain Z '=T ^TZ;

   The 3rd step: facial expression image sequence of picked at random from the test data described in the 1st step, the eigenmatrix Z ' that calculates this facial expression image sequence and the canonical correlation coefficient of the eigenmatrix Z ' of each training data with;

   The 4th the step: the 3rd the step result the basis on, calculate respectively this facial expression image sequence and every kind of basic facial expression canonical correlation coefficient and mean value; Choose the expression of 6 minimum value correspondences in the mean value, as classification results;

   Through above-mentioned steps, can finish the Expression Recognition for the treatment of the test pattern sequence.
2. 2. a kind of method of discerning countenance from human face image sequence as claimed in claim 1 is characterized in that: the method for 20 face feature points of sign includes but not limited in first two field picture described in the 1st step of step 2 in each facial expression image sequence: 1. identify manually; 2. the enhancing classifier methods based on the Gabor feature that adopts people such as Vukdadinovic to propose in document " Fully automatic facial feature point detection using gaborfeature based boosted classifiers " realizes the automatic location to 20 face feature points.
3. 3. a kind of method of from human face image sequence, discerning countenance as claimed in claim 1, it is characterized in that: step 2 adopts affine transformation method to adjust the position and the size of people's face in the image described in the 4th step, the equal and opposite in direction and the position of people's face in the same image sequence are consistent, are specially:

   At first make the line maintenance level of two inner eye corner points in first two field picture in each image sequence; Then, according to the position of these 3 points of uppermost point of two inner eye corner points in first two field picture in the image sequence of place and philtrum, 22 face feature points in remaining each frame are done mapping and normalized; After handling through affined transformation, in the same image sequence people of all images be bold little equate and the position of these 3 points of uppermost point of two inner eye corner points and philtrum and first frame in the position consistency of these 3 points.

Images