CN107169413A - A kind of human facial expression recognition method of feature based block weight - Google Patents

Abstract

The invention relates to a facial expression recognition method based on feature block weighting. The operation steps of this method are as follows: 1) Extract the Gabor texture features and geometric features of the expression picture; 2) Use the PCA algorithm to reduce the feature dimension for the extracted Gabor texture features, align the extracted geometric features in blocks, and divide the geometric features into mouth Three feature blocks of head, left eye, and right eye, and use the Procrustes Analysis method to align each geometric feature; 3) The Gabor texture feature after PCA dimensionality reduction is fused with the three geometric feature blocks after Procrustes Analysis to form Fusion features; 4) Input the fusion features into the Bp neural network with weighted feature blocks, train the neural network, and seek appropriate weight coefficients for each layer. The invention improves the commonality of the expression geometric features, and solves the problem that different feature forms and different regional features of the face have different contribution rates to expression recognition.

Description

A Facial Expression Recognition Method Based on Feature Block Weighting

technical field

The invention relates to a facial expression recognition technology, in particular to a method of weighting each feature block based on the feature block and weighting a Bp (backpropagation) neural network.

Background technique

The biggest problem facing the research on facial expression recognition is how to improve the accuracy of facial expression recognition. Due to the influence of different regions, races, face sizes, skin colors, cultures, etc., the current facial expression recognition methods do not have good performance. Versatility, not robust to different people.

The feature extraction of facial expressions is very critical to the recognition of facial expressions. Different feature extraction methods represent features from different perspectives. However, different features have different contribution rates to the recognition of facial expressions. In order to distinguish the feature importance of different features and different regions of the face, many scholars assign weight factors to each dimension feature based on the method of weight analysis, and adopt optimization principles such as maximizing inter-class distance and minimizing intra-class distance to optimize the weight factor. Find, and use this to distinguish the contribution rate of different features to expression recognition, and improve the recognition rate of facial expressions. However, these methods all face the following three problems:

1. The number of feature dimensions extracted from expression images is as high as tens of thousands. The weighting of each dimension of features will inevitably lead to a large number of weight factors. Finding weight factors will inevitably increase the calculation pressure, resulting in insufficient real-time performance.

2. Weighting each dimension feature separately will inevitably cause each feature to lose its original representation.

3. The optimization search of the weight factor and the classifier are two independent processes. The quality of the weight factor must be detected by the classifier, and it is good for the classifier to classify correctly.

Based on the above requirements, the present invention proposes a facial expression recognition method based on feature block weighting. For problems 1 and 2, it is proposed to weight features of different forms and different regions of the face based on the feature block level. Aiming at problem 3, a weighted Bp neural network is proposed, and the optimization of the weight factor is carried out simultaneously with the optimization of the weights and thresholds of each layer of the neural network.

Contents of the invention

In view of the defects in the prior art, the purpose of the present invention is to propose a facial expression recognition method based on feature block weighting to solve the problem that different forms of features and features of different regions of the face have different contribution rates to facial expression recognition.

In order to achieve the above object, design of the present invention is:

This facial expression recognition method based on feature block weighting includes facial Gabor feature extraction, facial geometric feature extraction, block alignment, and Bp neural network based on feature block weighting.

A Gabor filter is constructed to extract the Gabor texture features of facial expressions, and PCA is used to reduce the dimensionality of the Gabor features due to the high dimensionality of Gabor features. The Face++ function library is used to extract the position of facial key points as geometric features. For geometric features, due to the difference in facial position and size, facial geometric features need to be aligned to reduce the impact of inaccurate positioning and different sizes. Many scholars will The geometric features of the face are aligned using Procrustes Analysis, which has achieved good results. We know that human beings judge expressions mainly rely on the different shapes of the mouth and eyes. The changes of the mouth and eyes are independent of each other and do not interfere with each other. Therefore, this method proposes to divide the geometric features of the face into three geometric feature blocks of left eye, right eye, and mouth, and use Procrustes Analysis to align them respectively, which is different from the overall alignment of the face. This method reduces the interference between each feature block, Compared to aligning the geometric features of the entire face, the alignment of the sample geometric features is better. This operation is beneficial to solve the problems of low recognition rate caused by different face sizes and facial organ sizes of different people.

Aiming at the problem that different areas of the face and different facial expression feature representations have different contribution rates to expression recognition, the traditional method weights each dimension feature, and optimizes iterations based on the principles of maximizing inter-class distance and minimizing intra-class distance. However, this method has three disadvantages: 1. It destroys the original feature representation and the relationship between features, and the weighting of each feature will inevitably lose the overall advantage. 2. Weighting each dimension feature separately will inevitably cause each feature to lose its original representation. 3. The optimization of feature weights and the classifier are a separate process. Therefore, for the first and second shortcomings: this method proposes a concept based on feature block weighting, and uses the Gabor feature of facial expression, the geometric feature of the left eye, the geometric feature of the right eye, and the geometric feature of the mouth as four independent features Blocks, each feature block is weighted based on the feature block. For the third shortcoming: a Bp neural network based on feature block weighting is proposed. Combined with the Bp neural network, a weight layer is added before the input layer of the neural network. This weight layer realizes the weighting of each feature block, and the feature The weighting process of the block is combined with the classifier, through the training of the training samples, the weight factor of the weight layer is searched and optimized, and the weighting of the feature block is realized.

According to above-mentioned inventive concept, the present invention adopts following technical scheme:

A facial expression recognition method based on feature block weighting, characterized in that the operation steps are as follows: 1) extract Gabor texture features and geometric features of expression pictures; 2) use PCA algorithm to reduce feature dimension for extracted Gabor texture features, and extract The geometric features are aligned in blocks, and the geometric features are divided into three feature blocks of the mouth, left eye, and right eye, and the Procrustes Analysis method is used to align each geometric feature; 3) Gabor texture features after PCA dimensionality reduction and The three geometric feature blocks after Procrustes Analysis are fused to form a fused feature; 4) The fused feature is input to the Bp neural network weighted by the feature block, and the neural network is trained to find the appropriate weight coefficient of each layer.

The above-mentioned Gabor texture and geometric features of extracting facial expressions are: adopt Gabor filter to extract the Gabor texture features of facial expressions, adopt Face++ function library to extract geometric features of facial expressions.

The above-mentioned alignment of geometric feature blocks is: divide the geometric features into left eye geometric feature blocks, right eye geometric feature blocks and mouth geometric feature blocks, and then use Procrustes Analysis to perform alignment processing on each feature block.

The above-mentioned feature fusion is: arrange and combine the Gabor feature and each geometric feature block in the form of a column vector.

The above-mentioned Bp neural network method of feature block weighting is: a weight layer is added before the input layer of the neural network, and the weight layer contains four weight factors, and these four weight factors are combined with the parameters of each layer of the Bp neural network Training optimization to achieve weighting of the four feature blocks.

Compared with the prior art, the present invention has the following obvious outstanding substantive features and significant technological progress: the commonality of the geometric features of expressions is improved, and the problem of different feature representation forms and features of different regions of the face on the different contribution rates to expression recognition is solved. , thereby improving the recognition accuracy of facial expressions.

Description of drawings

FIG. 1 is an overall flow diagram of an embodiment of the present invention.

FIG. 2 is a structural diagram of a weighted Bp neural network according to an embodiment of the present invention.

Fig. 3 is a flow chart of calculating the input features of the weighted Bp neural network according to the embodiment of the present invention.

detailed description

The preferred embodiments of the present invention will be described in detail below in conjunction with the accompanying drawings.

Embodiment one:

Referring to Fig. 1, this facial expression recognition method based on feature block weighting is characterized in that the operation steps are as follows: 1) extracting the Gabor texture features and geometric features of the expression picture; 2) using the PCA algorithm to reduce the feature dimension for the extracted Gabor texture features , align the extracted geometric features in blocks, divide the geometric features into three feature blocks of mouth, left eye, and right eye, and use the Procrustes Analysis method to align each geometric feature; 3) Gabor after PCA dimensionality reduction The texture feature is fused with the three geometric feature blocks after Procrustes Analysis to form the fused feature; 4) The fused feature is input to the Bp neural network weighted by the feature block, and the neural network is trained to find the appropriate weight coefficient of each layer.

Embodiment two:

This embodiment is basically the same as Embodiment 1, and the special features are as follows:

The Gabor texture and the geometric feature of the described extraction expression picture are: adopt the Gabor filter to extract the Gabor texture feature of the expression image, adopt the Face++ function library to extract the geometric feature of the expression image.

The alignment of the geometric feature blocks is: divide the geometric features into left eye geometric feature blocks, right eye geometric feature blocks and mouth geometric feature blocks, and then use Procrustes Analysis to perform alignment processing on each feature block.

The feature fusion is: arranging and combining the Gabor feature and each geometric feature block in the form of a column vector.

The Bp neural network method of described characteristic block weighting is: before the input layer of neural network, increased weight layer, weight layer comprises four weight factors, and these four weight factors are carried out with the parameter of each layer of Bp neural network Train and optimize together to realize the weighting of the four feature blocks.

Embodiment three:

As shown in Figure 1, the Gabor filter is used to extract the Gabor features of facial expressions. Since the feature dimensions of Gabor features are large, in high-dimensional feature representations, these features are usually linearly correlated and contain more useless or less useful features. Variables, therefore, the PCA algorithm is used to perform feature selection on the extracted Gabor features. Use the Face++ function library to extract the facial geometric features of the expression image. Due to the difference in the structure and size of the face, the position and size of the eyes and mouth are different, the extracted facial expression geometric features are divided into mouth, left eye, and right eye geometric features. block, and then perform Procrustes Analysis on each feature block of facial features separately. The extracted Gabor features are fused with the features after the block alignment of the geometric features to form the fused features. The fusion method is as follows:

In the above formula, F represents the fused feature, and F _g represents the Gabor feature after PCA dimensionality reduction. F _l , F _r , and F _m represent the geometric features of the left eye, right eye, and mouth after Procrustes Analysis, respectively.

For each feature block in the fusion feature, the weight is defined based on the feature block, and a certain weight is assigned to the features of each region as a whole. This method divides the extracted features into four independent feature blocks: Gabor texture feature block, left eye The body geometric feature block, the right eye geometric feature block and the mouth geometric feature block are respectively regarded as independent wholes and given certain weights. The definition rules of weight factors are as follows:

When the Bp neural network accepts the input feature vector, it treats each feature variable equally, but in fact, different facial expression feature regions have different contribution rates to expression recognition. Therefore, this method proposes a Bp neural network with feature block weighting, as shown in the figure 2. The feature block weighted neural network adds a layer of weight layer before the input layer of the original Bp neural network. The weight layer is composed of four weight factors defined by the above two formulas, respectively defining the weight factors of the four feature blocks . The weight layer will first weight each feature block, and then input the weighted features to the input layer, followed by the hidden layer, and finally the output layer, which is the forward propagation of the input features. After that, according to the calculation error, the weight and threshold of the output layer are updated, but the weights of the hidden layer, input layer, and weight layer are updated, that is, the backpropagation of the error. The specific flow chart is shown in Figure 3: After the weights and thresholds of the network are initialized, the input features are first weighted to the feature block, and then input to the input layer of the neural network, and the results of each layer are calculated layer by layer, analyzed and compared The difference between the actual output and the expected output, and then update the weights of each layer backwards. The calculation method of the weight layer is as follows:

The four feature blocks are multiplied by the corresponding weight factors, and the four feature blocks are weighted based on feature blocks. The formula is as follows:

In the above formula, it is the fusion feature after feature block weighting. F is the input feature of the weight layer, the output of the weight layer, and will be used as the input of the input layer. After that, the calculation will be carried out step by step according to the calculation steps of the Bp neural network, and the weight coefficients of each layer will be updated iteratively until the error requirements are met.

Claims (5)

1. a kind of human facial expression recognition method of feature based block weight, it is characterised in that operating procedure is as follows：

1) the Gabor textural characteristics and geometric properties of expression picture are extracted；

2) use PCA algorithms to reduce characteristic dimension the Gabor textural characteristics of extraction, the geometric properties piecemeal of extraction alignd, Geometric properties are divided into mouth, left eye, three characteristic blocks of right eye, and Procrustes Analysis methods are respectively adopted by respectively Individual geometric properties are alignd；

3) the Gabor textural characteristics after PCA dimensionality reductions are melted with three geometric properties blocks after Procrustes Analysis Close, constitute fusion feature；

4) fusion feature is input to the Bp neutral nets of characteristic block weight, neutral net is trained, seeks suitable Each layer weight coefficient.

2. the human facial expression recognition method of feature based block weight according to claim 1, it is characterised in that the step It is rapid 1) extract expression picture Gabor textures and geometric properties be：The Gabor lines of facial expression image are extracted using Gabor filter Feature is managed, the geometric properties of facial expression image are extracted using Face++ function libraries.

3. the human facial expression recognition method of feature based block weight according to claim 1, it is characterised in that the step It is rapid 2) in geometric properties block alignment be：Geometric properties are divided into left eye geometric properties block, right eye geometric properties block and mouth several What characteristic block, Procrustes Analysis are then respectively adopted to each characteristic block and carry out registration process.

4. the human facial expression recognition method of feature based block weight according to claim 1, it is characterised in that the step It is rapid 3) in Fusion Features be：Gabor characteristic and each geometric properties block are subjected to permutation and combination in the way of column vector.

5. the human facial expression recognition method of feature based block weight according to claim 1, it is characterised in that the step It is rapid 4) in the Bp neural net methods of characteristic block weight be：Weight layer, weight are added before the input layer of neutral net Layer includes four weight factors, and this four weight factors are trained into optimization together with the parameter progress of each layer of Bp neutral nets, Realize the weight to four characteristic blocks.