CN101369310A - Robust human face expression recognition method - Google Patents

Abstract

The invention provides a robust face expression identification method, wherein a face image is reconstructed through robust principal component analysis, and significance analysis is performed on difference images of the original face images and the reconstructed face images, to detect a blocked area, then the images at the blocked area are reconstructed to remove screen, finally expression sorting is performed on the face images after removing the screen, to obtain expression identification result. The invention has better ability of removing the screen for each face screen, which is of importance for increasing face expression recognition rate under screen, and is a feasible robust face expression identification method.

Description

A kind of facial expression recognizing method of robust

(1) technical field

The present invention relates to a kind of mode identification method, especially relate to a kind of facial expression recognizing method of robust.Belonging to human facial expression information extracts and the identification field.

(2) background technology

Human face expression identification generally is divided into the identification of facial action and the identification of emotion.For example, part Study person discerns based on the single of face action coded system from human face expression and mixes motor unit and carry out.And most researchers from human face expression, discern the people happiness, surprised, sad, emotion such as fear.Because it is non-rigid motion that human face expression changes, and is subjected to influences such as individual difference, visual angle change, illumination, human face expression identification is a difficult task, and there be limited evidence currently of has the human face expression recognition system that can be applied to actual environment.

The identification of in the past human face expression often is confined to controlled condition, and for example background is single, illumination is consistent, no head motion etc., and therefore, the human face expression under the controlled condition is discerned can reach high recognition.But seldom there is the researcher that the robust human face Expression Recognition under the uncontrolled condition is studied.Since 21 century, a few studies person begins one's study to the facial expression recognizing method that blocks, illumination, posture, image resolution ratio etc. have robustness.Wherein, for blocking that the facial expression recognizing method with robustness mainly contains how much masks of method, local space of adopting local feature and the method extracted based on the method for the facial movement model of state and based on the Gabor wavelet character etc., but seldom there is the researcher people's face to be removed the Expression Recognition of carrying out robust after blocking.Have Robust identifying method under the situation of blocking less and do not remove the deficiency of shelter at people face, the present invention proposes a kind of new robust human face expression recognition method.

(3) summary of the invention

The objective of the invention is: have the situation of blocking not possess the deficiency of robustness at existing facial expression recognizing method to face, propose a kind of facial expression recognizing method of robust, it can make people's face that the higher Expression Recognition effect of acquisition under the situation of blocking is being arranged.

The facial expression recognizing method of a kind of robust of the present invention, by the robust principal component analysis (PCA) facial image is reconstructed, and the error image of the facial image after original facial image and the reconstruct carried out significance analysis, detect occlusion area, be reconstructed with removal according to image then and block occlusion area, facial image after at last removal being blocked carries out expression classification, obtains the Expression Recognition result.

The facial expression recognizing method of a kind of robust of the present invention, its step is as follows:

Step 1: not containing the L class human face expression image normalization that blocks with N is data matrix C _i∈ R ^{M * n}(i=1 ... M), as training sample, the many classification AdaBoost method training facial expression classifier that adopt husky Pierre people such as (Schapire) to propose.

Step 2: M comprised contain that to block and do not contain the L class human face expression image normalization that blocks be data matrix A _i∈ R ^{M * n}(i=1 ... M), as training sample.Make s=m * n, with A _iExpand into one dimension column data vector d _i∈ R ^{S * 1}(i=1 ... M), constitute input matrix D=[d ₁d ₂D _M] ∈ R ^{S * M}, (Robust Principle Component Analysis, RPCA) method obtain robust mean vector μ ∈ R in the robust principal component analysis (PCA) of adopting Fei Nanduo (Fernando) to propose ^{S * 1}With robust latent vector B ∈ R ^{S * k}, k＜M.

Step 3: with human face expression image normalization data matrix P ∈ R to be identified ^{M * n}

Step 4: P is expanded into one dimension column data vector d ∈ R ^{S * 1}, suc as formula the reconstruct vector d of (1) compute vector d _Rec∈ R ^{S * 1}, and it is deformed into data matrix P ' ∈ R ^{M * n}

d _Rec=μ+BB ^T(d-μ) formula (1)

Step 5: calculate facial image matrix P ' and error image matrix E ∈ R after the reconstruct with the facial image matrix P of expression to be identified ^{M * n}, as the formula (2).

E=|P '-P| formula (2)

Step 6: establishing scanning window R, high (1≤h＜m), wide is that (1≤w＜n), the upper left corner coordinate of window are (x1, y1) (0≤x1＜n, 0≤y1＜m), h, w, x1, y1 are traveled through, and satisfy constraint condition as the formula (3) to w for h.Scanning window R to error image carries out conspicuousness detection (as the formula (4)), significantly is worth H _{E, R}

0≤x1+w≤n and 0≤y1+h≤m and 2*w*h＜m*n formula (3)

$H_{E,R}=-\underset{i}{\mathrm{\Σ}}{P}_{E,R}\left(e_i\right){\log }_2{P}_{E,R}\left(e_i\right)$ Formula (4)

P wherein _{E, R}(e _i) refer to that error image matrix E is e in scanning window R value _i(0≤e _i≤ 255) probability.

Step ₇: to the remarkable value H of all scanning window R _{E, R}Get maximum significantly value H _Max=max{H _{E, R}, and judge occlusion area.As the formula (5), if significantly be worth H _MaxGreater than the threshold value H that presets ₀, with H _MaxRelevant zone is judged as occlusion area, does not have occlusion area otherwise be judged to be.

Formula (5)

Step 8: the occlusion area to human face expression image array P is reconstructed, as the formula (6).If R _OcclusionBe not empty, jump to step 4; If R _OcclusionBe sky, continue execution in step 9.

$P(x,y)=\left\{\begin{array}{cc}P(x,y)& (x,y)\∉R_{\mathrm{occlusion}}\\ P\′(x,y)& (x,y)\∈R_{\mathrm{occlusion}}\end{array}\right.$ Formula (6)

Step 9:, obtain the human face expression recognition result with the input of human face expression image array P as step 1 training gained facial expression classifier.

Good effect of the present invention and advantage are:

1. the present invention has carried out removing and blocked processing containing the human face expression image that blocks, and is significant for the human face expression discrimination that raising is blocked under the situation;

2. the present invention has preferably to remove to the various faces situation of blocking and blocks ability, is a kind of facial expression recognizing method of feasible robust.

(4) description of drawings

Fig. 1 method step block scheme

(5) specific implementation method

See shown in Figure 1, the facial expression recognizing method of a kind of robust of the present invention, its step is as follows:

Step 1: not containing the L class human face expression image normalization that blocks with N is data matrix C _i∈ R ^{M * n}(i=1 ... M), as training sample, the many classification AdaBoost method training facial expression classifier that adopt husky Pierre people such as (Schapire) to propose.

Step 2: M comprised contain that to block and do not contain the L class human face expression image normalization that blocks be data matrix A _i∈ R ^{M * n}(i=1 ... M), as training sample.Make s=m * n, with A _iExpand into one dimension column data vector d _i∈ R ^{S * 1}(i=1 ... M), constitute input matrix D=[d ₁d ₂D _M] ∈ R ^{S * M}, (Robust Principle Component Analysis, RPCA) method obtain robust mean vector μ ∈ R in the robust principal component analysis (PCA) of adopting Fei Nanduo (Fernando) to propose ^{S * 1}With robust latent vector B ∈ R ^{S * k}, k＜M.

Step 3: with human face expression image normalization data matrix P ∈ R to be identified ^{M * n}

Step 4: P is expanded into one dimension column data vector d ∈ R ^{S * 1}, suc as formula the reconstruct vector d of (1) compute vector d _Rec∈ R ^{S * 1}, and it is deformed into data matrix P ' ∈ R ^{M * n}

d _Rec=μ+BB ^T(d-μ) formula (1)

Step 5: calculate facial image matrix P ' and error image matrix E ∈ R after the reconstruct with the facial image matrix P of expression to be identified ^{M * n}, as the formula (2).

E=|P '-P| formula (2)

Step 6: establishing scanning window R, high (1≤h＜m), wide is that (1≤w＜n), the upper left corner coordinate of window are (x1, y1) (0≤x1＜n, 0≤y1＜m), h, w, x1, y1 are traveled through, and satisfy constraint condition as the formula (3) to w for h.Scanning window R to error image carries out conspicuousness detection (as the formula (4)), significantly is worth H _{E, R}

0≤x1+w≤n and 0≤y1+h≤m and 2*w*h＜m*n formula (3)

$H_{E,R}=-\underset{i}{\mathrm{\Σ}}{P}_{E,R}\left(e_i\right){\log }_2{P}_{E,R}\left(e_i\right)$ Formula (4)

P wherein _{E, R}(e _i) refer to that error image matrix E is e in scanning window R value _i(0≤e _i≤ 255) probability.

Step 7: to the remarkable value H of all scanning window R _{E, R}Get maximum significantly value H _Max=max{H _{E, R}, and judge occlusion area.As the formula (5), if significantly be worth H _MaxGreater than the threshold value H that presets ₀, with H _MaxRelevant zone is judged as occlusion area, does not have occlusion area otherwise be judged to be.

Formula (5)

Step 8: the occlusion area to human face expression image array P is reconstructed, as the formula (6).If R _OcclusionBe not empty, jump to step 4; If R _OcclusionBe sky, continue execution in step 9.

$P(x,y)=\left\{\begin{array}{cc}P(x,y)& (x,y)\∉R_{\mathrm{occlusion}}\\ P\′(x,y)& (x,y)\∈R_{\mathrm{occlusion}}\end{array}\right.$ Formula (6)

Step 9:, obtain the human face expression recognition result with the input of human face expression image array P as step 1 training gained facial expression classifier.

Claims (1)

1. the facial expression recognizing method of a robust is characterized in that, this recognition methods step is as follows:

Step 1: not containing the L class human face expression image normalization that blocks with N is data matrix C _i∈ R ^{M * n}(i=1 ... M), as training sample, adopting husky Pierre is many classification AdaBoost method training facial expression classifier that Schapire proposes;

Step 2: M comprised contain that to block and do not contain the L class human face expression image normalization that blocks be data matrix A _i∈ R ^{M * n}(i=1 ... M), as training sample, make s=m * n, with A _iExpand into one dimension column data vector d _i∈ R ^{S * 1}(i=1 ... M), constitute input matrix D=[d ₁d ₂D _M] ∈ R ^{S * M}, the expense south of employing is that the robust principal component analysis (PCA) that Fernando proposes is Robust Principle Component Analysis more, the RPCA method obtains robust mean vector μ ∈ R ^{S * 1}With robust latent vector B ∈ R ^{S * k}, k＜M;

Step 3: with human face expression image normalization data matrix P ∈ R to be identified ^{M * n}

Step 4: P is expanded into one dimension column data vector d ∈ R ^{S * 1}, suc as formula the reconstruct vector d of (1) compute vector d _Rec∈ R ^{S * 1}, and it is deformed into data matrix P ' ∈ R ^{M * n}

d _Rec=μ+BB ^T(d-μ) formula (1)

Step 5: calculate facial image matrix P ' and error image matrix E ∈ R after the reconstruct with the facial image matrix P of expression to be identified ^{M * n}, as the formula (2);

E=|P '-P| formula (2)

Step 6: establish scanning window R high for h (1≤h＜m), wide is that (1≤w＜n), the upper left corner coordinate of window are (x1, y1) (0≤x1＜n, 0≤y1＜m), h, w, x1, y1 are traveled through, and satisfy constraint condition as the formula (3) to w; Scanning window R to error image carries out the conspicuousness detection, as the formula (4), significantly is worth H _{E, R}

0≤x1+w≤n and 0≤y1+h≤m and 2*w*h＜m*n formula (3)

$H_{E,R}=-\underset{i}{\mathrm{\Σ}}{P}_{E,R}\left(e_i\right){\log }_2{P}_{E,R}\left(e_i\right)$ Formula (4)

Wherein _{PE, R}(e _i) refer to that error image matrix E is e in scanning window R value _i(0≤e _i≤ 255) probability;

Step 7: to the remarkable value H of all scanning window R _{E, R}Get maximum significantly value H _Max=max{H _{E, R}, and judge occlusion area, and as the formula (5), if significantly be worth H _MaxGreater than the threshold value H that presets ₀, with H _MaxRelevant zone is judged as occlusion area, does not have occlusion area otherwise be judged to be;

Formula (5)

Step 8: the occlusion area to human face expression image array P is reconstructed, as the formula (6); If R _OcclusionBe not empty, jump to step 4; If R _OcclusionBe sky, continue execution in step 9;

$P(x,y)=\left\{\begin{array}{cc}P(x,y)& (x,y)\∉R_{\mathrm{occlusion}}\\ P\′(x,y)& (x,y)\∈R_{\mathrm{occlusion}}\end{array}\right.$ Formula (6)

Step 9:, obtain the human face expression recognition result with the input of human face expression image array P as step 1 training gained facial expression classifier.

Images