CN108268838A - Facial expression recognizing method and facial expression recognition system - Google Patents

Abstract

The present application discloses a facial expression recognition method, which includes: detecting a human face from an original image; performing face alignment and feature point positioning on the detected human face; extracting facial feature information from a human face image; The characteristic data of facial expressions are classified to realize facial expression recognition. This application uses face detection, feature point location, feature extraction, and expression classification to predict the maximum possibility of human facial expressions, which ensures the accuracy of expression recognition and has a wide range of application prospects.

Description

Facial expression recognition method and human facial expression recognition system

technical field

The present application relates to a facial expression recognition method and a human facial expression recognition system, which belong to the technical field of human facial expression recognition.

Background technique

The generation of human emotions is a very complicated psychological process, and the expression of emotions is accompanied by a variety of expressions. There are mainly three expressions that are often used by computer scientists for research: facial expressions, voice, and actions. Among these three ways of expressing emotions, expressions contribute up to 55% of the emotion. With the increasing application of human-computer interaction technology, facial expression recognition technology is of great significance in the field of human-computer interaction. As one of the main research methods in the field of pattern recognition and machine learning, a large number of facial expression recognition algorithms have been proposed.

However, facial expression recognition technology also has its weaknesses: 1. Different facial expression changes: facial expressions will be different according to the differences in different people's expressions; 2. The same person's context changes: the same person's expression in real life 3. External conditions, such as: background, lighting, angle, distance, etc., have a great influence on expression recognition. All of the above will affect the accuracy of facial expression recognition.

Contents of the invention

In view of the inaccurate recognition of facial expressions in the prior art, which affects the accuracy of facial expression recognition, the purpose of this application is to provide a method and system for facial expression recognition, which can realize accurate recognition of facial expressions.

To achieve the above object, the present invention provides a method for recognizing facial expressions.

The facial expression recognition method is characterized in that, comprising:

Detect faces from raw images;

Perform face alignment and feature point positioning on the detected faces;

Extract facial feature information from face images;

According to the acquired feature data, expression classification is carried out to realize facial expression recognition.

The human face detection: that is to detect the existence of human faces from the original images of various scenes, and accurately separate the human face regions.

Further, the detection of human face from the original image includes:

Scan the original image progressively based on local binary patterns to obtain a response image;

Adopt AdaBoost algorithm to carry out human face detection to described response image, detect the existence of human face;

The AdaBoost algorithm is used for human eye detection, and the face area is separated.

Optionally, the multi-scale detection is performed according to 1.25-0.9 in the detection process using the AdaBoost algorithm.

Further, said performing face alignment and feature point location on the detected faces includes:

A locally constrained model is used to annotate facial feature points.

Optionally, use the local constraint model to mark the facial feature points, and after obtaining the coordinates of the feature points, select an area that reflects the differences between various expressions, and extract the two expressions based on deformation and motion. types of features;

Recursive feature elimination and linear vector machine are used for feature evaluation, and the selected features are further selected for feature selection.

Further, the extraction of facial feature information from the face image includes:

Select the area that reflects the difference between various expressions, and extract two types of features: deformation-based expression features and motion-based expression features;

Recursive feature elimination and linear vector machine are used for feature evaluation, and the selected features are further selected for feature selection.

Optionally, the regions reflecting the differences among various expressions include eyes, nose tip, corners of mouth, eyebrows, and contour points of various parts of human face.

Further, the extraction of facial feature information from the face image further includes: performing feature selection on the extracted facial feature information, obtaining a subset of facial features, and saving the facial feature information for expression recognition.

Further, according to the acquired feature data, performing expression classification to realize facial expression recognition includes:

According to the extracted facial feature information, select samples, use prior knowledge to train the expression classifier, and each sample corresponds to the corresponding expression label;

Through the expression classifier, the expression classification is realized by using the least squares rule.

Further, said performing facial expression classification according to the acquired feature data to realize facial expression recognition also includes:

The base vector space is created by using the expression features of the known labels, and the expression to be tested is projected into this space to judge the expression category and perform facial expression recognition.

As a specific implementation, the method for recognizing facial expressions includes the following steps: (1) detecting a human face from an original image; (2) performing face alignment and feature points on the detected human face Positioning; (3) Extract facial feature information from the face image; (4) According to the acquired feature data, perform expression classification and realize facial expression recognition.

Wherein, step (1) further comprises: (11) progressively scanning original image based on partial binary pattern, obtains a response image; (12) adopts AdaBoost algorithm to carry out face detection to described response image, detects the existence of human face; (13) The AdaBoost algorithm is used for human eye detection, and the face area is separated.

Further, AdaBoost algorithm is used for face detection or human eye detection to perform multi-scale detection according to 1.25-0.9.

Step (2) further includes: using a local constraint model to mark facial feature points.

Step (3) further includes: (31) selecting the mouth, eyebrows, and eyes as three main areas that reflect the differences between various expressions, and extracting two types of features based on deformation-based expression features and motion-based expression features; ( 32) Use recursive feature elimination and linear vector machine for feature evaluation, and further perform feature selection on the selected features.

Further, feature selection is performed on the extracted facial feature information, a subset of facial features is obtained, and the facial feature information is saved for expression recognition.

Step (4) further includes: (41) selecting samples according to the extracted facial feature information, and using prior knowledge to train an expression classifier, each sample corresponds to a corresponding expression label; (42) through the expression classifier, using least squares Rules to achieve expression classification.

Further, the base vector space is created with the expression features of the known labels, and the expression to be tested is projected into this space to judge the expression category and perform facial expression recognition.

Another aspect of the present application provides a facial expression recognition system, characterized in that the system includes: a human face detection module, a feature point positioning module, a feature extraction module, and a facial expression recognition module;

The human facial expression recognition module is used to detect the human face from the original image;

The feature point positioning module is connected to the face detection module, and is used for face alignment and feature point positioning of the detected faces;

The feature extraction module is connected with the feature point positioning module, and is used to extract facial feature information from the face image;

The facial expression recognition module is connected with the feature extraction module, and is used to predict the maximum possibility of the facial expression data to be recognized through the trained expression classifier according to the extracted facial feature information, and find out the possibility The highest expression category, realizing facial expression recognition.

Optionally, the face detection module progressively scans the original image based on local binary patterns to obtain a response image;

Adopt AdaBoost algorithm to carry out human face detection to described response image, detect the existence of human face;

The AdaBoost algorithm is used for human eye detection, and the face area is separated.

Optionally, the multi-scale detection is performed according to 1.25-0.9 in the detection process using the AdaBoost algorithm.

Optionally, the feature point positioning module uses a local constraint model to mark facial feature points.

Optionally, the feature extraction module selects regions reflecting the differences between various types of expressions, and extracts two types of features: deformation-based expression features and motion-based expression features;

Recursive feature elimination and linear vector machine are used for feature evaluation, and the selected features are further selected for feature selection.

Optionally, the regions reflecting the differences among various expressions include at least one of mouth, eyebrows, eyes, and nose tip.

Optionally, the feature extraction module performs feature selection on the extracted facial feature information, acquires a subset of facial features, and saves the facial feature information for expression recognition.

Optionally, the facial expression recognition module performs expression classification according to the acquired feature data, and realizing facial expression recognition includes: selecting samples according to the extracted facial feature information, and using prior knowledge to train an expression classifier, each sample Corresponding to the corresponding emoticon label;

Through the expression classifier, the expression classification is realized by using the least squares rule.

Optionally, the facial expression recognition module uses the expression features of known labels to create a base vector space, and the expression to be tested judges the expression category by projecting its features into this space to perform facial expression recognition.

The beneficial effect that this application can produce comprises:

This application uses face detection, feature point location, feature extraction, and expression classification to predict the maximum possibility of human facial expressions, which ensures the accuracy of expression recognition and has a wide range of application prospects.

Description of drawings

FIG. 1 is a schematic flow chart of the face recognition method described in the present application.

FIG. 2 is a schematic diagram of the architecture of the face recognition system described in this application.

Detailed ways

The present application is described in detail below in conjunction with the examples, but the present application is not limited to these examples.

Example 1

The facial expression recognition method and system provided by the present invention will be described in detail below in conjunction with the accompanying drawings.

Referring to FIG. 1 , it is a schematic flow chart of the facial expression recognition method of the present invention. The method comprises the following steps: S11: detecting the human face from the original image; S12: performing face alignment and feature point positioning on the detected human face; S13: extracting facial feature information from the human face image; S14: according to The acquired feature data is used to classify facial expressions and realize facial expression recognition. The above steps will be described in detail below in conjunction with the accompanying drawings.

S11: Detect the human face from the original image.

Face detection: It detects the existence of human faces from the original images of various scenes, and accurately separates the human face area. As a preferred embodiment, step S11 can further be completed by the following steps: 11) progressively scan the original image based on the partial binary pattern to obtain a response image; 12) use the AdaBoost algorithm to perform face detection on the response image, and detect Existence of human face; 13) AdaBoost algorithm is used for human eye detection, and the human face area is separated.

As an effective texture description operator, Local Binary Pattern (LBP) has excellent ability to describe local texture features of images. Applying the LBP operator process is similar to the template operation in the filtering process, and scans the original image line by line; for each pixel in the original image, the gray value of the point is used as the threshold value, and the surrounding 3×3 8 fields are processed. Binarization: according to a certain order, the result of binarization is composed into an 8-bit binary number, and the value of this binary number (0~255) is used as the response of this point.

As shown in Table 1, the corresponding gray value of the original image in one embodiment, for the center point of the 3×3 area in Table 1, use its gray value of 88 as the threshold, binarize its 8 fields, and start from the upper left The point starts to follow the clockwise direction (the order can be arbitrary, but it must be unified) and the binarization result is composed of a binary number 10001011, which is 139 in decimal, as the response of the center. After the entire progressive scanning process is over, a LBP response image is obtained, which can be used as a feature of follow-up work; the corresponding gray value of the obtained response image is shown in Table 2.

180 52 5 213 88 79 158 84 156

The corresponding gray value of the original image in Table 1-Example.

1 0 0 1 139 0 1 0 1

The response image obtained in Table 2 corresponds to the gray value.

The AdaBoost algorithm was proposed by Freund and Schapire based on the online distribution algorithm. The AdaBoost algorithm allows the designer to continuously add new weak classifiers until a predetermined small enough error rate is reached. In the AdaBoost algorithm, each training sample is assigned a weight, indicating the probability that it is selected into the training set by a component classifier. If a certain sample point has been classified accurately, then its probability of being selected will be reduced in constructing the next training set; on the contrary, if a certain sample point has not been correctly classified, its weight will be increased. Through T rounds of training, the AdaBoost algorithm can focus on those more difficult samples and synthesize a strong classifier for target detection.

The AdaBoost algorithm is described as follows:

1) Given a calibrated training sample set (x ₁ , y ₁ ), (x ₂ , y ₂ ),...(x _L , y _L ). Among them, g _j ( _xi ) represents the j-th Haar-Like feature of the i-th training image, x _i ∈ X, represents the input training sample, y _i ∈ Y={1,-1} represents the true and false samples respectively .

2) Initialize the weights w _{1, i} = 1/2m, 1/2n, where m and n represent the data of true and false samples respectively, and the total number of samples L=m+n.

3) For T rounds of training, For t=1,2,...,T.

Normalize the weights for all samples:

For the jth Haar-Like feature in each sample, a simple classifier can be obtained, which is to determine the threshold θ _j and bias P _j so that the error ε _j reaches the minimum:

in,

The bias P _j determines the direction of the inequality, and there are only two cases of ±1.

Among the determined simple classifiers, find a weak classifier h _t with the smallest error ε _t .

4) Update the weights of all samples:

Among them, β _t =ε _t /(1-ε _t ), if x _i is correctly classified by h _i , then e _i =0, otherwise e _i =1.

5) The final strong classifier obtained is:

where α _t =ln(1/β _t ) is scaled in terms of the prediction error of h _t .

So far, through the above steps, the face can be detected. During the detection process, multi-scale detection can be performed according to 1.25-0.9, and finally the windows are merged and the results are output.

On the basis of detected faces, the AdaBoost algorithm is used for human eye detection. The basic principle of human eye detection is the same as that of face detection, and will not be repeated here. In the process of human eye detection, multi-scale detection can be carried out according to 1.25-0.9, and the elimination mechanism can be established (according to the position, size and other characteristics of human eyes).

S12: Perform face alignment and feature point positioning on the detected faces.

Feature point positioning: According to the input face image, the key feature points of the face are automatically located, such as eyes, nose tip, mouth corner points, eyebrows, and contour points of various parts of the face. As a preferred implementation manner, step S12 can be further completed by using the following steps: using a local constraint model to mark facial feature points.

The Local Constraint Model (CLM) completes face feature point detection by initializing the position of the average face, and then letting the feature points of each average face search and match on its neighborhood position. The whole process is divided into two stages: model building stage and point fitting stage. The model construction phase can be subdivided into the construction of two different models: shape model construction and patch model construction. Shape model construction is to model the shape of the face model, which describes the guidelines followed by shape changes. The Patch model is to model the neighborhood around each feature point, establish a feature point matching criterion, and judge the best match of feature points.

The locally constrained model (CLM) algorithm is described as follows:

1) Shape model construction

Computes the aligned average shape of all face samples in the training set. Suppose there are M pictures, each picture has N feature points, and the coordinates of each feature point are assumed to be ( _xi , y _i ), the vector composed of the coordinates of N feature points on an image is represented by x=[x ₁ y ₁ x ₂ y ₂ … x _N y _N ] ^T , the average face of all images can be obtained:

Calculate the difference between the shape vector of each sample image and the average face, and a zero-mean shape change matrix X can be obtained:

The main components that determine the shape change of the face can be obtained by performing PCA transformation on the matrix X, namely

Obtain the main eigenvalue λ _i and the corresponding eigenvector p _i . Because the eigenvectors corresponding to the larger eigenvalues generally contain the main information of the sample, the eigenvectors corresponding to the largest k eigenvalues are selected to form an orthogonal matrix P=(p ₁ ,p ₂ ,…,p _k ).

The weight vector of shape change b=(b ₁ ,b ₂ ,…,b _k ) ^T , each component of b represents its magnitude in the direction of the corresponding feature vector:

Then for any face test image, its sample shape vector can be expressed as:

2) Patch model construction

Assuming that there are M face images in the training sample, select N key feature points of the face on each image, select a fixed-size patch area around each feature point, and mark the patch area containing the feature points as a positive sample; then A patch of the same size is intercepted in the non-feature point area and marked as a negative sample.

Assuming that each feature point has a total of r patches, it is composed of a vector (x ⁽¹⁾ ,x ⁽²⁾ ,…x ^(r) ) ^T , and each image in the sample set has Then the output is only positive samples and negative samples, that is, the patch is a feature point area and a non-feature point area. Then y ⁽ⁱ⁾ ={-1,1}i=1,2,...r, where y ⁽ⁱ⁾ =1 is a positive sample label, and y ⁽ⁱ⁾ =-1 is a negative sample label. Then the trained linear support vector machine is:

Among them, _xi represents the subspace vector of the sample set, that is, the support vector, α _i is the weight coefficient, M _s is the number of support vectors for each feature point, and b is the offset. Available:

y ⁽ⁱ⁾ = w ^T x ⁽ⁱ⁾ + θ

w ^T = [w ₁ w ₂ … w _n ] is the weight coefficient of each support vector, and θ is the offset. In this way, a patch model is established for each feature point.

3) Point fitting

A similar response map is generated for each feature point, identified as R(X,Y), by performing a local search within the restricted area of the currently estimated feature point location.

Fit a quadratic function to the response graph, assuming that R(X,Y) obtains the maximum value at (x ₀ , y ₀ ) in the neighborhood, we fit a function to this position so that the position and the maximum value R (X, Y) correspond one to one. A quadratic function can be described as follows:

r(x,y)=a(xx ₀ ) ² +b(yy ₀ ) ² +c

Among them, a, b, and c are the coefficients of the quadratic function, and the solution method is to minimize the error between the quadratic function r(x,y) and R(X,Y), that is, to complete a least squares calculation:

With the parameters a, b, and c, then r(x, y) is an objective cost function about the position of the feature point, and then a deformation constraint cost function is added to form the objective function for feature point search. The objective function is as follows Show:

Each time the objective function is optimized to obtain a new feature point position, and then iteratively updated until it converges to the maximum value, the face point fitting is completed.

S13: Extract facial feature information from the face image.

Feature extraction: that is, to extract representative feature information of the face from the normalized face image. As a preferred implementation, step S13 can be further completed by the following steps: (31) Select the mouth, eyebrows, and eyes as three main areas that reflect the differences between various expressions, and extract deformation-based expression features and motion-based expressions Two types of features of features; (32) Use recursive feature elimination and linear vector machine for feature evaluation, and further feature selection for the selected features.

Using the local constraint model to mark the facial feature points, after obtaining the coordinates of the feature points, select the shape features of the three main areas of the mouth, eyebrows, and eyes, that is, calculate the relevant slope information between the key points in these three areas, and extract the information based on Deformed facial features. At the same time, the key points in the three areas are tracked, the corresponding displacement information is extracted, and the distance information between the specific feature points of the expression picture is extracted, and the distance is compared with the calm picture to obtain the change information of the distance. facial features.

Recursive feature elimination and linear vector machine are used for feature evaluation, and the weight value calculated by support vector machine is used as the sorting criterion to further denoise the selected features.

The feature selection algorithm is described as follows:

Input: training sample set l is the number of categories

Output: Ranked set of features R

1) Initialize the original feature set S = {1, 2, ..., D}, feature sorting set R = []

2) Generate (l(l-1))/2 training samples:

in the training sample Find the pairwise combinations of different categories to get the final training samples:

Loop the process until S=[]:

3) Obtain l training sub-samples X _j (j=1,2,...,(l(l-1))/2);

Use X _j to train the support vector machine respectively, and get w _j (j=1,2,…,l) respectively;

Computing the sorting criterion score

Find the feature with the smallest ranking criterion score

Update feature set R={p}∪R;

Remove this feature in S S=S/p.

S14: According to the acquired feature data, perform expression classification to realize facial expression recognition.

Classification: That is, human expressions are roughly divided into seven categories, namely joy, anger, sadness, disgust, surprise, fear, and neutrality. As a preferred embodiment, step S14 can further be completed by following steps: (41) select samples according to the extracted facial feature information, and utilize prior knowledge to train an expression classifier, and each sample corresponds to a corresponding expression label; (42) Through the expression classifier, the expression classification is realized by using the least squares rule.

Training expression classifier: Use the support vector machine algorithm to train the extracted facial features, and an expression classifier will be obtained after the training is completed.

Support vector machine (SVM) algorithm description:

input training set Where x _i ∈ R ^D , y _i ∈ {+1,-1}, x _i is the i-th sample, N is the sample size, and D is the number of sample features. SVM looks for the optimal classification hyperplane w·x+b=0.

The optimization problem that SVM needs to solve is:

sty _i (w x _i +b)≥1-ξ _i i=1,2,…,N

ξ _i ≥0,i=1,2,…,N

The original problem can be transformed into a dual problem:

Among them, α _i is the Lagrangian multiplier.

The final solution of w is:

The discriminant function of SVM is:

Expression classification: Input the extracted facial feature information into the trained classifier, and let the classifier give an expression prediction value. That is, the rule of least squares is applied to find the best function match to the data by minimizing the sum of the squares of the errors. At this point, a complete face recognition process is completed.

With reference to Fig. 2, the structural representation of facial expression recognition system of the present invention; Described system comprises: a human face detection module 21, a feature point positioning module 22, a feature extraction module 23 and a facial expression recognition module 24.

The human face detection module 21 is used to detect human faces from the original image. The face detection module 21 can scan the original image progressively based on local binary patterns to obtain a response image; then use the AdaBoost algorithm to perform face detection on the response image to detect the presence of a human face; then use the AdaBoost algorithm to perform human face detection. Eye detection to separate the face area. For the specific implementation of face detection, refer to the aforementioned method flow, which will not be repeated here.

The feature point location module 22 is connected to the face detection module 21 and is used for face alignment and feature point location for the detected faces. The local constraint model is used to mark the facial feature points, and the key facial feature points are located, such as eyes, nose tip, mouth corner points, eyebrows and contour points of various parts of the face. For the specific implementation of feature point positioning, refer to the aforementioned method flow, which will not be repeated here.

The feature extraction module 23 is connected to the feature point location module 22, and is used to extract facial feature information from the face image. The feature extraction module 23 can extract two types of features based on deformation-based expression features and motion-based expression features by selecting three main areas that reflect the differences between various expressions: mouth, eyebrows, and eyes; Feature elimination and linear vector machine for feature evaluation, and further feature selection for the selected features. In the feature extraction stage, feature selection is performed on the extracted facial feature information, a subset of facial features is obtained, and the facial feature information is saved for expression recognition. For the specific implementation manner, refer to the aforementioned method flow, which will not be repeated here.

The face recognition module 24 is connected to the feature extraction module 23, and is used to classify facial expressions according to the acquired feature data to realize facial expression recognition. The feature extraction module 24 can select samples according to the facial feature information extracted, and utilize prior knowledge to train an expression classifier, and each sample corresponds to a corresponding expression label; afterwards, the expression classifier adopts the least squares rule to realize expression classification . The classification process is to use the expression features of known labels to create a base vector space, and the expression to be tested judges the expression category by projecting its features into this space, and performs facial expression recognition. For its specific implementation, refer to the aforementioned method flow, which will not be repeated here.

Embodiment 2 facial expression recognition method

The method for facial expression recognition in the present embodiment is as follows:

Step 11: Detect faces from the original image;

In this step, a specific implementation manner includes step 101 , step 102 and step 103 .

Step 101: Scan the original image line by line based on local binary patterns to obtain a response image.

Step 102: Using the AdaBoost algorithm to perform face detection on the response image to detect the presence of a human face.

Step 103: Use the AdaBoost algorithm to detect human eyes, and separate the human face area.

In a specific manner, AdaBoost algorithm is used to perform multi-scale detection according to 1.25-0.9 in the process of face detection or human eye detection.

Step 12: Perform face alignment and feature point positioning on the detected faces;

In this step, a specific implementation manner is: using a local constraint model to mark facial feature points.

Step 13: Extract facial feature information from the face image;

In this step, a specific implementation manner includes step 301 and step 302 .

Step 301: Select the mouth, eyebrows, and eyes as three main areas that reflect the differences between various expressions, and extract two types of features: deformation-based expression features and motion-based expression features;

In this step, another specific implementation method is: select the eyes, the tip of the nose, the corners of the mouth, the eyebrows, and the contour points of each part of the face to reflect the differences between various expressions, and extract the expression features based on deformation and based on Two types of features for expressive features of motion;

Step 302: Use recursive feature elimination and linear vector machine for feature evaluation, and further perform feature selection on the selected features.

In a specific implementation manner, feature selection is performed on the extracted facial feature information, a subset of facial features is obtained, and the facial feature information is saved for expression recognition.

Step 14: According to the acquired feature data, perform expression classification to realize facial expression recognition.

In this step, a specific implementation manner includes step 401 and step 402 .

Step 401: Select samples according to the extracted facial feature information, and use prior knowledge to train an expression classifier, each sample corresponds to a corresponding expression label;

Step 402: Using the expression classifier and adopting the least squares rule, the expression classification is realized.

In a specific implementation, the expression features of known labels are used to create a base vector space, and the expression to be tested is projected into this space to determine the expression category and perform facial expression recognition.

Various algorithms involved in this embodiment are the same as those in Embodiment 1.

Embodiment 3 facial expression recognition system

The facial expression recognition system in this embodiment includes: a human face detection module, a feature point positioning module, a feature extraction module, and a facial expression recognition module;

The human facial expression recognition module is used to detect the human face from the original image;

In a specific implementation manner, the face detection module scans the original image progressively based on local binary patterns to obtain a response image;

Adopt AdaBoost algorithm to carry out human face detection to described response image, detect the existence of human face;

The AdaBoost algorithm is used for human eye detection, and the face area is separated.

In one specific implementation manner, the multi-scale detection is performed according to 1.25-0.9 in the detection process using the AdaBoost algorithm.

The feature point positioning module is connected to the face detection module, and is used for face alignment and feature point positioning of the detected faces;

In a specific implementation manner, the feature point location module uses a local constraint model to mark facial feature points.

The feature extraction module is connected with the feature point positioning module, and is used to extract facial feature information from the face image;

In a specific implementation manner, the feature extraction module selects regions that reflect the differences between various types of expressions, and extracts two types of features: deformation-based expression features and motion-based expression features;

Recursive feature elimination and linear vector machine are used for feature evaluation, and the selected features are further selected for feature selection.

In one specific implementation manner, the regions selected to reflect the differences among various expressions include eyes, nose tip, mouth corners, eyebrows, and outline points of various parts of the human face.

The facial expression recognition module is connected with the feature extraction module, and is used to predict the maximum possibility of the facial expression data to be recognized through the trained expression classifier according to the extracted facial feature information, and find out the possibility The highest expression category, realizing facial expression recognition;

In a specific implementation manner: the facial expression recognition module performs expression classification according to the acquired feature data, and realizing facial expression recognition includes: selecting samples according to the extracted facial feature information, and using prior knowledge to train an expression classifier , each sample corresponds to the corresponding expression label;

Through the expression classifier, the expression classification is realized by using the least squares rule.

In a specific embodiment, the facial expression recognition module performs expression classification according to the acquired feature data, and realizing facial expression recognition includes: selecting samples according to the extracted facial feature information, and using prior knowledge to train an expression classifier , each sample corresponds to the corresponding expression label;

Through the expression classifier, the expression classification is realized by using the least squares rule;

The human facial expression recognition module uses the expression features of known labels to create a base vector space, and the expression to be tested judges the expression category by projecting its features into this space, and performs human facial expression recognition.

Various algorithms involved in this embodiment are the same as those in Embodiment 1.

The above are only a few embodiments of the application, and do not limit the application in any form. Although the application is disclosed as above with preferred embodiments, it is not intended to limit the application. Any skilled person familiar with this field, Without departing from the scope of the technical solution of the present application, any changes or modifications made using the technical content disclosed above are equivalent to equivalent implementation cases, and all belong to the scope of the technical solution.

Claims (10)

1. A facial expression recognition method, characterized in that, comprising: Detect faces from raw images; Perform face alignment and feature point positioning on the detected faces; Extract facial feature information from face images; According to the acquired feature data, expression classification is carried out to realize facial expression recognition. 2. method according to claim 1, is characterized in that, described detecting people's face from original image comprises: Scan the original image progressively based on local binary patterns to obtain a response image; Adopt AdaBoost algorithm to carry out human face detection to described response image, detect the existence of human face; Use the AdaBoost algorithm to detect human eyes and separate the face area; Preferably, the multi-scale detection is carried out according to 1.25-0.9 in the detection process using the AdaBoost algorithm. 3. The method according to claim 1, wherein said face alignment and feature point location are carried out to the detected faces comprising: A locally constrained model is used to annotate facial feature points. 4. method according to claim 1, is characterized in that, described extracting facial feature information from face image comprises: Select the area that reflects the difference between various expressions, and extract two types of features: deformation-based expression features and motion-based expression features; Use recursive feature elimination and linear vector machine for feature evaluation, and further feature selection for the selected features; Preferably, the regions that reflect the differences between various expressions include eyes, nose tip, mouth corners, eyebrows, and contour points of various parts of the human face; Preferably, the extracting facial feature information from the face image further includes: performing feature selection on the extracted facial feature information, obtaining a subset of facial features, and saving the facial feature information for expression recognition. 5. method according to claim 1, is characterized in that, described according to the feature data that obtains, carries out expression classification, realizes facial expression recognition and comprises: According to the extracted facial feature information, select samples, use prior knowledge to train the expression classifier, and each sample corresponds to the corresponding expression label; Through the expression classifier, the expression classification is realized by using the least squares rule; Preferably, said according to the acquired characteristic data, carries out expression classification, realizes facial expression recognition and also includes: The base vector space is created by using the expression features of the known labels, and the expression to be tested is projected into this space to judge the expression category and perform facial expression recognition. 6. a facial expression recognition system, characterized in that, said system comprises: a human face detection module, a feature point location module, a feature extraction module, a facial expression recognition module; The human facial expression recognition module is used to detect the human face from the original image; The feature point positioning module is connected to the face detection module, and is used for face alignment and feature point positioning of the detected faces; The feature extraction module is connected with the feature point positioning module, and is used to extract facial feature information from the face image; The facial expression recognition module is connected with the feature extraction module, and is used to predict the maximum possibility of the facial expression data to be recognized through the trained expression classifier according to the extracted facial feature information, and find out the possibility The highest expression category, realizing facial expression recognition. 7. The system according to claim 6, wherein the face detection module scans the original image progressively based on local binary patterns to obtain a response image; Adopt AdaBoost algorithm to carry out human face detection to described response image, detect the existence of human face; The AdaBoost algorithm is used for human eye detection, and the face area is separated. 8. The system according to claim 6, wherein the feature point location module uses a local constraint model to mark facial feature points. 9. system according to claim 6, is characterized in that, described feature extraction module selects the region that embodies the difference between various expressions, extracts the feature of two types of expression features based on deformation and based on motion ; Use recursive feature elimination and linear vector machine for feature evaluation, and further feature selection for the selected features; Preferably, the regions that reflect the differences between various expressions include eyes, nose tip, mouth corners, eyebrows, and contour points of various parts of the human face; Preferably, the feature extraction module performs feature selection on the extracted facial feature information, obtains a subset of facial features, and saves the facial feature information for expression recognition. 10. system according to claim 6, it is characterized in that, described human facial expression recognition module carries out expression classification according to the feature data that obtains, and realizing human facial expression recognition comprises: according to the facial characteristic information of extraction, select sample, utilize Prior knowledge training expression classifier, each sample corresponds to the corresponding expression label; Through the expression classifier, the expression classification is realized by using the least squares rule; Preferably, the facial expression recognition module uses the expression features of known labels to create a base vector space, and the expression to be tested is projected into this space to determine the expression category and perform facial expression recognition.