CN111814713A - An expression recognition method based on BN parameter transfer learning - Google Patents

Abstract

The invention relates to the technical field of target recognition, and discloses an expression recognition method based on BN parameter migration learning. The invention fully utilizes the transfer learning mechanism to apply the learning knowledge in a certain field to different but related fields, can effectively solve the problem of insufficient sample data volume of facial expression modeling caused by illumination, shooting angle and the like in facial expression recognition, reduces the influence of insufficient sample number on parameter learning precision and recognition result, and can be widely applied to the environment with noisy, uncertain and difficult acquisition of a large amount of human face target data.

Description

An expression recognition method based on BN parameter transfer learning

technical field

The invention relates to the application field of target recognition in artificial intelligence, image engineering, management science and engineering, in particular to an expression recognition method based on BN parameter transfer learning.

Background technique

Bayesian network (Bayesian network, BN) has practical application value in uncertainty modeling and decision support. Bayesian network parameter learning is to obtain all parameters through sample data and prior knowledge when the structure is known. The process of conditional probability distribution of network nodes.

After the problem domain is transformed into the BN model representation, the BN theory can be used to complete the reasoning task. Among them, the Junction tree algorithm is one of the BN accurate inference algorithms with fast calculation speed and the most widely used. Because BN organically combines the theoretical achievements of probability theory and graph theory, it is an effective method to solve the problem of uncertainty and incomplete information reasoning, and it is an ideal tool that can be applied to facial expression recognition.

The parameter learning of the BN model refers to the problem of estimating the parameters of the BN model under the premise that the structure of the BN model is known. Currently, Maximum Likelihood Estimation (MLE) provides a method for evaluating model parameters given observational data, that is, the model has been determined and the parameters are unknown. When the data is sufficient, the maximum likelihood estimation algorithm is usually used to obtain better parameter learning accuracy. Maximum A Posterior (MAP) estimates are point estimates of hard-to-observe quantities obtained from empirical data. Similar to maximum likelihood estimation, maximum a posteriori estimation incorporates the prior distribution of the quantity to be estimated.

Facial expression recognition is an interdisciplinary subject that integrates artificial intelligence, neurology, and computers. It has a wide range of applications in psychoanalysis, clinical medicine, vehicle monitoring, and business. Facial expressions refer to various emotions expressed through changes in facial muscles, eye muscles and mouth muscles. With the in-depth study of facial expressions, Ekman further improved human facial expressions, and proposed a facial action coding system based on Action Units (AUs), and analyzed the motion characteristics of these motor units to illustrate the correlation with them. Facial expressions (Source: P. Ekman, W. V. Friesen, J. C. Hager, Facial Action Coding System, A Human Face, Salt Lake City, UT, 2002.).

In view of the problem that the feature samples obtained in the process of facial expression recognition are scarce, the existing literature proposes a facial expression recognition method based on Bayesian network modeling under small data sets (see: Guo Wenqiang, Gao Wenqiang, Xiao Qinkun, Xu Cheng, Li Mengran. Facial expression recognition based on BN modeling under small dataset [J]. Science Technology and Engineering, 2018, 18(35):179-183). The method first extracts the geometric features and HOG features of facial expression images, and then forms an action unit (AU) label sample set through feature fusion and normalization. It is transformed into a set of constraints between BN conditional probabilities, and then convex optimization is introduced to maximize the solution to complete the estimation of BN model parameters. However, the acquisition of expert experience is often subject to considerable subjectivity, which is not conducive to the estimation of BN parameters.

The principle of transfer learning is to apply knowledge and experience in one field to other scenarios, train and analyze labeled samples in one or more task fields (source fields), obtain parameter models of such tasks, and then apply them to In another related task domain (target domain), the classification of data from another domain is done.

When transfer learning processes these two related and common tasks, it does not need to process the data of the two tasks in the source domain and the target domain separately, but uses the experience and knowledge of pattern recognition on one of the task data. to process another task data. In this way, in the process of BN parameter learning, the inaccuracy of the learning result caused by the difference of one of the task data can be avoided, especially the influence of the subjectivity of expert experience can be avoided.

SUMMARY OF THE INVENTION

The present invention provides an expression recognition method based on BN parameter transfer learning, which can solve the above problems in the prior art.

The present invention provides an expression recognition method based on BN parameter transfer learning, comprising the following steps:

S1. Obtain the facial activity unit AU;

S2. Determine whether the facial expression BN is modeled;

Judging whether the modeling flag BN_Flag is "true", the initial value is set to "false", if BN_Flag is "true", indicating that BN has been modeled, then jump to S7, enter the recognition process; otherwise, execute S3, enter the construction mold process;

S3. Obtain the sample AU required for BN modeling;

S31. Determine the number of categories E_num and categories for facial expression recognition;

S32, extract the sample data set of the required facial activity unit AU according to the expression of each category;

S4. Determine the target/source domain network BN model structure diagram of facial expression recognition;

Through the obtained sample data set of the facial activity unit AU and the relationship between the facial expression and the activity unit AU as a priori information, the model structure diagram G1 for establishing the target network facial expression recognition BN is determined, and the source domain is determined. Model structure diagram G2 of BN for network facial expression recognition;

S5, source domain BN parameter learning;

S51, obtain the sample data set M (S _n ) of each source domain face activity unit AU;

S52, calculate the source weight coefficient k(n) of each source domain sample value accounting for the total source domain sample value, as shown in formula (1):

Among them, the sum of each source weight coefficient is 1, and the value of the weight coefficient is any real number between [0, 1];

S53, using the method of maximum likelihood estimation MLE to obtain the parameter θ _ni of each source domain BN model, where i is the ith node in the sub-nodes of the source domain BN model;

S54, fuse the parameters θ _ni of each source domain BN model to obtain the total source domain BN parameter θ _Si ;

θ _Si =∑ _n k(n)θ _ni (2);

S6. Obtain the BN parameters of the facial expression in the target domain;

S61, using the method of maximum a posteriori probability estimation MAP to obtain the parameter θ _Ti of the initial BN model of the target domain, where i is the ith node in the sub-nodes of the BN model of the target domain;

S62, calculate the final parameter θ _i of the target domain facial expression BN according to the weight factor;

θ _i = _α1θTi + _α2θsi (3)

Among them, α1 and α2 are weighting factors, α1+α2=1;

S63. Set the BN_Flag flag to "true" to complete the BN modeling; return to S1;

S7, facial expression recognition;

S71, set the facial expression attribute probability threshold Ψ;

S72, put the facial expression recognition evidence AU into the constructed BN model, and use the joint tree inference algorithm to perform BN inference to obtain the facial expression attribute probability Ψ';

S73. Discriminate facial expressions;

If the facial expression attribute probability Ψ' is greater than or equal to the threshold Ψ, output the facial expression attribute, that is, the facial expression recognition result, otherwise, re-acquire a new AU data set.

The specific steps of extracting the facial activity unit AU according to each type of expression in the step S32 include:

S321, obtaining the geometric features of the facial expression on the facial expression image of the human face through the CLM algorithm;

S322, extracting the texture feature of the facial expression by using the HOG algorithm on the facial expression image of the human face;

S323, performing feature fusion and normalization processing on the geometric features and texture features of the facial expressions to obtain the fusion features of the facial expressions;

S324. Use the support vector machine SVM to classify the fusion features of the facial expressions, and obtain the dataset M(t) of the facial activity unit AU in the target domain and the dataset M(S _n ) of the facial activity unit AU in the source domain, n=1 ,2,...,q, q is the number of source fields, and q is a natural number.

The specific steps of obtaining the geometric features of facial expressions in the step S321 include:

S3211. Obtain the feature point positioning information of the facial expression image;

The CLM facial feature point localization algorithm is used to locate the feature points and extract the geometric features of the facial expressions.

The specific steps of determining the structure diagram of the facial expression recognition target/source network BN model in the step S4 include:

S41, determine the face expression recognition BN node;

Determine the parent node and child node of the facial expression recognition BN;

S42. Determine the directed acyclic graph of the facial expression recognition BN;

Directed edges are used to connect the parent and child nodes of the face expression recognition BN in turn, and the BN model structure diagram G1 of the face expression recognition target network is determined to be established, and the face expression recognition source network BN model structure diagram G2 is established.

The number of categories E_num for facial expression recognition in the step S11 is 6, including: "happy", "surprised", "scared", "angry", "disgusted" and "sad".

Compared with the prior art, the beneficial effects of the present invention are:

(1) Transfer learning is used to solve the problem of low recognition accuracy when the BN experimental training data set is insufficient (small data problem) in the process of face expression recognition in the target domain. Transfer learning is to obtain data and information from similar fields to solve learning problems in fields with insufficient data. Through the knowledge in the source domain data, the learning method of constructing the learning model on the target domain improves the recognition accuracy.

(2) The influence of subjectivity of expert experience on the learning accuracy of BN parameters can be avoided. The knowledge learned in the source task is transferred to the target task, and the learning algorithm in the source task is based on the classic sufficient samples MLE, MAP and other algorithms, and its learning mechanism is based on a data-driven method, thus avoiding the subjective learning of BN parameters. The effect of expert experience on learning accuracy.

Description of drawings

FIG. 1 is a flowchart of an expression recognition method based on a migration mechanism provided by the present invention.

FIG. 2 is a structural diagram of a face expression recognition target domain BN model provided by the present invention.

FIG. 3 is a structural diagram of a face expression recognition source domain BN model provided by the present invention.

FIG. 4 is a flow chart of learning BN parameters of facial expressions under the source network provided by the present invention.

FIG. 5 is a flow chart of learning BN parameters of facial expressions under the target network provided by the present invention.

FIG. 6 is six basic expression diagrams of the CK data set provided by the embodiment of the present invention.

FIG. 7 shows six basic facial expressions of the FER2013 dataset provided by an embodiment of the present invention.

FIG. 8 is a facial feature point location map provided by an embodiment of the present invention.

Detailed ways

A specific embodiment of the present invention will be described in detail below with reference to the accompanying drawings 1-8, but it should be understood that the protection scope of the present invention is not limited by the specific embodiment.

The embodiment of the present invention provides an expression recognition method based on BN parameter transfer learning. The facial expression recognition BN model structure is constructed according to the relationship between the facial expression and the AU label, and then the BN parameters and the face target calculated by the face source domain data set are used. Domain data set BN initial parameters, obtain the final facial expression recognition BN parameters according to the migration mechanism, and use the inference algorithm in BN theory to perform BN inference to recognize facial expressions. The invention makes full use of the transfer learning mechanism to apply the knowledge learned in a certain field to different but related fields, and can effectively solve the problem of insufficient facial expression modeling sample data due to illumination, shooting angle, etc. in facial expression recognition. It reduces the influence of insufficient number of samples on parameter learning accuracy and recognition results, and can be widely used in noisy, uncertain and difficult to obtain a large amount of face target data environment.

As shown in Fig. 1, a kind of expression recognition method based on BN parameter transfer learning provided by the present invention comprises the following steps:

S1: Get facial activity unit (AU);

Refer to: Valstar M F, Almaev T, Girard J M, et al. FERA2015-second facial expression recognition and analysis challenge[C]//IEEE, International Conference and Workshops on Automatic Face and GestureRecognition.2015:1-8, Among them, the action unit AU6 represents whether the "lower mouth contour is upturned" occurs or not, and AU25 represents the occurrence of the event of "splitting the lips and exposing the teeth", etc.;

Optionally, use the Openface open source tool (source: Baltrusaitis T, Robinson P, Morency LP. OpenFace: An open source facial behavior analysis toolkit[C]//2016IEEEWinter Conference on Applications of Computer Vision(WACV).IEEE, 2016.) , perform the CLM algorithm on the input picture information to obtain the geometric features of the facial expression, and the HOG algorithm extracts the HOG features of the facial expression through feature fusion and normalization to obtain the fusion features of the facial expression, and uses the support vector machine (SVM) to carry out Classify to obtain a dataset M(t) of target-domain face active units (AU) and a dataset M(S _n ) of source-domain face active units (AU) n=1, 2, . . . , q. q is the number of source fields, and q is a natural number.

S2: Determine whether the facial expression BN has been modeled.

Determine whether the BN structure modeling flag BN_Flag is "true". The initial value of BN_Flag is set to "false", BN_Flag=0 (0 is "false"). If modeling, BN_Flag=1 (1 is "true"), then jump to S7 to perform facial expression recognition. Otherwise, execute S3 and enter the BN modeling process;

S3: Obtain the sample AU required for BN modeling.

S31: Determine the number of categories E_num for facial expression recognition; in this example, E_num is 6. That is, it corresponds to six types of expressions: "happy", "surprised", "frightened", "angry", "disgusted" and "sad".

S32: extract facial activity unit AU according to each type of expression;

Optionally, use the CLM algorithm and the HOG algorithm to obtain the geometric features and texture features of the facial expression on the facial expression image, obtain the fusion features of the facial expression through feature fusion and normalization, and use the support vector machine (SVM) to obtain the fusion features of the facial expression. Perform classification to obtain a dataset M(t) of target-domain face active units (AU) and a dataset M(S _n ) of source-domain face active units (AU) n=1, 2, . . . , q. q is the number of source fields, and q is a natural number.

S4: determine the BN model structure diagram of the target (source) network for facial expression recognition;

S41: Determination of the face expression recognition BN node.

Optionally, select the "Expression" node to identify the parent node of the BN (the node to be queried); "AU1", "AU2", "AU4", "AU5", "AU6", "AU7", "AU9" , "AU12", "AU15", "AU17", "AU23", "AU25" and other nodes are the child nodes (evidence nodes) of BN for facial expression recognition.

S42: Determine the directed acyclic graph of the facial expression recognition BN.

Directed edges are used to connect parent nodes and child nodes in turn, that is, Expression is used as the nock tail of 12 directed edges in turn, and the arrows point to AU1, AU2, AU4, AU5, AU6, AU7, AU9, AU12, AU15, AU17, AU23, AU25. Determined to establish the target facial expression recognition BN model structure G1, the source domain facial expression recognition BN model structure G2;

S5: source domain BN parameter learning;

S51: Calculate the small sample number threshold C(n) of each source domain network according to the BN model of the source network for face expression recognition (for the determination of the small sample number threshold required for BN model learning, see: D. Koller, and N. Friedman, Probabilistic Graphical Models: Principles and Techniques-Adaptive Computation and Machine Learning, United States of America, the MIT Press, 2009.)

S52: Determine whether the data set M(Sn) of the source domain face activity unit (AU) is greater than C( _n ), if greater than C(n), execute S122, otherwise use the S10 method to continue to obtain M( _Sn ).

S53: Calculate the source weight coefficient k(n) of each source domain sample value accounting for the total source domain sample value; as shown in formula (1).

The sum of the weight coefficients of each source is 1, and the value of the weight coefficients is any real number between [0, 1]. where M(S _n ) is each source domain AU sample set (n=1,2,...,q);

S54: Use the maximum likelihood estimation (MLE) method to learn the parameters of each source domain BN model

θ _ni (i is the ith node of the child node of the source domain BN model),

S55: the BN parameter θ _ni of the fusion source domain;

θ _Si =∑ _n k(n)θ _ni (2)

S6: Obtain the BN parameters of the facial expression in the target domain;

S61: adopt the method of maximum a posteriori probability estimation (MAP) to learn the parameter θ _Ti of the initial BN model of the target domain;

S62: calculate the final parameter θ _i of the target BN according to the weight factor;

θ _i = _α1θTi + _α2θsi (3)

where α1 and α2 are weighting factors, α1+α2=1;

Optionally, α1 is set to 0.7.

S63: Set the BN_Flag flag to "true", indicating that the BN modeling has been completed; return to S1 to obtain the facial expression recognition evidence AU;

If BN_Flag=True, after returning to S1 to obtain the facial expression AU, it will not model again, but enter S7 for expression recognition.

S7: face expression recognition;

S71: Set the facial expression attribute probability threshold Ψ.

Optionally, Ψ is 0.7.

S72: utilize the reasoning algorithm in the BN theory to carry out BN reasoning to obtain the facial expression attribute probability Ψ';

Optionally, the inference algorithm adopts the associative tree inference algorithm.

S73: Discriminate facial expressions. If the facial expression attribute probability Ψ' is greater than or equal to the threshold Ψ, output the facial expression attribute, that is, the facial expression recognition result. Otherwise, re-acquire new AU evidence samples.

Example

The source of the data set in this example consists of two parts:

(1) CK database (source: Swati NigamRajiv SinghEmailauthorA.K.Misra.Efficient facial expression recognition using histogram oforiented gradients in wavelet domain,Multimedia Tools and Applications,Vol.77(21)(2018)28725-28747.) in the available part The subject is 97 undergraduates in introductory psychology courses. Their ages ranged from 18 to 30 years old. 65% were female, 15% African American, and 3% Asian or Latino. Including six basic expressions, it is a relatively common facial expression dataset at present. As shown in Figure 6.

(2) FER2013 dataset

FER2013 facial expression dataset (source: Hong-Wei Ng, Viet Dung Nguyen, Vassilios Vonikakis, Stefan Winkler, Deep learning for emotion recognition on small datasets using transfer learning, 2015.) consists of 35886 facial expression images, including test images , Announcement Verification Graph and Private Verification Graph. Each image is composed of grayscale images with a fixed size of 48x48. As shown in Figure 7.

The target database CK for facial expression recognition is very extensive in number, but the number of participants (97 people) and interpersonal similarity are quite limited. At the same time, the database is obtained in a laboratory environment and cannot fully simulate real scenes. In order to improve the robustness of the algorithm, that is, the recognition accuracy, a transfer learning mechanism is proposed. The source domain database FER2013 is used to learn the knowledge in the task, thereby enhancing the expression classification of the target database CK.

In this embodiment, the CK library is used as the target domain data set of the experiment, and 240 images (the last few frames of the image sequence) are selected for each expression, of which 120 expression images are used as the training set for parameter learning, and the BN-based parameter migration can be used. Learning method, obtain BN parameters, and then complete expression recognition, the remaining 120 images of CK data set are used as test set. Taking the FER2013 database as the source domain, 300 datasets were selected as the source domain dataset for the experiment, and 300 groups of AU sample data were selected for each expression for parameter model migration.

The present invention relates to an expression recognition method based on a migration mechanism. In this embodiment, the source domain data set M(S _n )=300; the target domain data set used for training M(t)=120, and then the BN parameter is performed. Migrate to get the final BN parameters of facial expressions. The test data set is 120 groups;

Include the following steps:

S1: Get facial activity unit (AU);

Optionally, use the Openface open source tool (source: Baltrusaitis T, Robinson P, Morency LP. OpenFace: An open source facial behavior analysis toolkit[C]//2016IEEEWinter Conference on Applications of Computer Vision(WACV).IEEE, 2016.) , perform the CLM algorithm on the input image information to obtain the geometric features of the facial expressions, and the HOG algorithm extracts the HOG features of the facial expressions through feature fusion and normalization to obtain the fusion features of the facial expressions. Classification to obtain a dataset M(t)=120 of target domain face active units (AU) and a dataset M(sn) n=1, 2, . . . , q of source domain face active units (AU). q is the number of source fields, and q is a natural number. In this example, the number of datasets in the source domain is q=1, that is, there is one dataset M(s1) in the source domain.

S2: Determine whether the facial expression BN has been modeled.

When this method is used for the first time, the initial value BN_Flag=0, that is, BN is not modeled, and S3 needs to be executed to carry out BN modeling.

S3: Obtain the sample AU required for BN modeling

S31: Determine the number of categories E_num for facial expression recognition; in this example, E_num is 6. That is, it corresponds to six types of expressions: "happy", "surprised", "frightened", "angry", "disgusted" and "sad".

S32: Extract the facial activity unit AU according to each type of expression.

Using the Openface open source tool, the CLM algorithm is performed on the input facial expression image information to obtain the geometric features of the facial expression, and the HOG algorithm extracts the reference (Guo Wenqiang, Gao Wenqiang, etc. Facial expression recognition based on Bayesian network modeling under small data sets [ J]. Science Technology and Engineering, 2018, 018(035): 179-183.) The algorithm obtains the geometric features of the facial expression, and the HOG algorithm extracts the HOG features of the facial expression through feature fusion and normalization to obtain the facial expression The fusion features of , are classified by support vector machine (SVM), and the data set M(t) of the target domain face active unit (AU) and the data set M(S ₁ ) of the source domain face active unit (AU) are obtained:

S321: Acquire the feature point positioning information of the facial expression image. Optionally, use the CLM facial feature point positioning algorithm to locate feature points and extract the geometric features of facial expressions; as shown in Figure 8, the partial coordinates of the 68 feature points in a certain expression image in the target domain in the image are:

{260.892 243.672; 263.012 269.343; 266.038 293.998; 270.955 317.406

281.108 340.161; 299.33 357.192; 326.176 367.506; 353.233 376.735

...

407.442 305.409; 387.881 304.42; 379.841 306.079; 372.826 306.621}

S322: Obtain the AU sample data set. Optionally, the facial expression image is processed by feature fusion and normalization, and is classified by a support vector machine (SVM) to obtain an AU sample data set. The source domain is M(S ₁ ) and; the target domain is M(t ); get part of the facial expression AU sample label dataset as shown in Table 1. The six basic expressions in this example are: "happy", "surprised", "frightened", "angry", "disgusted" and "sad".

Table 1 AU sample label dataset

S4: determine the BN model structure diagram of the target (source) network for facial expression recognition;

From the obtained AU sample data set, it is determined that the target facial expression recognition BN model structure G1 and the source domain facial expression recognition BN model structure G2 are established;

The BN model structures of the target domain and source domain are obtained as shown in Figure 2 and Figure 3, respectively. The number and direction of this node are determined according to the relationship between facial expressions and AUs in Table 2. Among them, the parent node Expression_S and Expression_T nodes represent the state of facial expressions, including six kinds of expression states; 12 AUs are used as child nodes, and each AU has two value events, namely "doesn't happen" and "occurs". "1" and "2" are used to represent the two states of "occurrence" and "non-occurrence" of this AU node; when "Expression_S=happy", AU6=1, AU12=1, and other AU labels are 0. (1 means that the current AU occurs, 0 means that the AU does not occur). Obviously, the state of facial expressions can be characterized by the AU state.

S5: source domain BN parameter learning;

S51: In this embodiment, the source domain data set adopts the FER2013 data set, wherein the data set M(S ₁ )=300;

S52: Calculate the source weight coefficient k(n) of each source domain sample value accounting for the total source domain sample value;

In this embodiment, a source network model is adopted. According to the above formula (1),

S53: Obtain source domain BN parameters according to maximum likelihood estimation;

In this embodiment, 300 groups of AUs in the FER2013 data set are selected as the source domain data set, and the parameters of the BN node i can be obtained according to the MAP. When i=1,

Other i=2...12, the method is the same.

S54: Calculate the BN parameter θ _S1 of the total source domain; as shown in the above formula (2), the result is as follows.

S6: Obtain the BN parameters of the facial expression in the target domain;

S61: adopt the method of maximum a posteriori probability estimation to learn the parameter θ _Ti of the initial BN model of the target domain;

In this embodiment, the AUs of 120 images in the CK dataset are selected as the target domain dataset, which can be obtained according to the maximum a posteriori estimation. When i=1, the initial BN parameter of the target domain is the same for other i=2...12 methods.

S62: Calculate the target domain BN parameter θ _i , as shown in the parameter fusion formula (3).

θ _i = _α1θTi + _α2θsi (3)

In this embodiment, α1=0.7 and α2=0.3. According to the above formula (3), it can be calculated that when i=1, that is, the parameters of the first child node of the target domain BN

S63: Set the BN_Flag flag to "true", indicating that the BN modeling has been completed; return to S1 to obtain the facial expression recognition evidence AU;

Since BN_Flag=true, after returning to S1 to obtain the facial expression AU, it will not be modeled again, that is, skipping S3 to S6, and entering S7 for expression recognition.

S7: facial expression recognition;

S71: set the facial expression attribute probability threshold Ψ=0.7;

S72: in the BN model, the observation evidence ev to be identified is obtained by D, and the inference algorithm in the BN theory is used to carry out BN inference to obtain the facial expression attribute probability Ψ';

For example, in this embodiment, a set of processing data of “angry” is input, and the evidence to be observed ev=[1 1 2 1 1 21 1 1 2 2 1], where “1”, “2” represent angry The AU label sample set obtained by feature processing for expressions. When input ev=[1 1 2 1 1 2 1 1 1 2 2 1], we can get Ψ'=0.975 by using the Junction Tree inference algorithm;

S73: Discriminate facial expressions.

In this embodiment, it can be obtained that Ψ'=0.975>Ψ, that is, the output facial expression attribute is "angry".

Input the observation evidence ev to be diagnosed, and use the joint tree algorithm to infer the results as shown in the table.

The attribute probability of "angry" is Ψ'=0.975; while Expression_T is the attribute probability of other expressions as shown in Table 3.

Table 3 The general inference results of expression attributes

target attribute hapiness Fear disgust sad surprise Attribute probability Ψ' 0.000 0.002 0.012 0.010 0.001

Similarly, input the 6 basic expressions of the 120 groups of test set AU label datasets of the CK dataset, according to the expression recognition method based on BN parameter transfer learning proposed in this embodiment, the expression recognition results can be obtained, as shown in Table 4.

Table 4 Expression classification results of CK data

The invention proposes an expression recognition method based on a migration mechanism. The facial expression recognition BN model structure is constructed according to the relationship between facial expressions and action unit labels, and then the BN parameters calculated by the face source domain data set and the face target domain data set are used. BN initial parameters, obtain the final facial expression recognition BN parameters according to the migration mechanism, and use the inference algorithm in BN theory to perform BN inference to recognize facial expressions. The invention makes full use of the transfer learning mechanism to apply the knowledge learned in a certain field to different but related fields, and can effectively solve the problem of insufficient facial expression modeling sample data due to illumination, shooting angle, etc. in facial expression recognition. It reduces the influence of insufficient number of samples on parameter learning accuracy and recognition results, and can be widely used in environments that are noisy, uncertain, and difficult to obtain a large amount of face target data.

The invention uses migration learning to solve the problem that the recognition accuracy rate is low when the BN experimental training data set data is insufficient (small data problem) in the face expression recognition process of the target domain. Transfer learning is to obtain data and information from similar fields to solve learning problems in fields with insufficient data. Through the knowledge in the source domain data, the learning method of constructing the learning model on the target domain improves the recognition accuracy.

The present invention can avoid the influence of the subjectivity of expert experience on the learning accuracy of BN parameters. The knowledge learned in the source task is transferred to the target task, and the learning algorithm in the source task is based on the classic sufficient samples MLE, MAP and other algorithms, and its learning mechanism is based on the data-driven method, thus avoiding the subjective learning of BN parameters. The effect of expert experience on learning accuracy.

The above disclosures are only a few specific embodiments of the present invention, however, the embodiments of the present invention are not limited thereto, and any changes that can be conceived by those skilled in the art should fall within the protection scope of the present invention.

Claims (5)

1. A facial expression recognition method based on BN parameter transfer learning is characterized by comprising the following steps:

s1, acquiring a face activity unit AU;

s2, judging whether the facial expression BN is modeled or not;

judging whether the modeling Flag BN _ Flag is 'true', setting the initial value to be 'false', if the BN _ Flag is 'true', indicating that the BN is modeled, jumping to S7, and entering an identification process; otherwise, executing S3 and entering the modeling process;

s3, obtaining a sample AU required by BN modeling;

s31, determining the number E _ num and the type of the facial expression recognition;

s32, extracting a sample data set of a required facial activity unit AU according to the expression of each category;

s4, determining a facial expression recognition target/source domain network BN model structure diagram;

determining a model structure chart G1 for establishing a target network human face expression recognition BN and determining a model structure chart G2 for a source domain network human face expression recognition BN by using the acquired sample data set of the face activity unit AU and the relationship between the human face expression and the activity unit AU as prior information;

s5, learning source domain BN parameters;

s51, obtaining sample data set M of each source area face activity unit AU (S)_n)；

S52, calculating a source weight coefficient k (n) of each source domain sample value in the total source domain sample value, and using a formula (1) to show that:

wherein, the sum of the weight coefficients of all the sources is 1, and the values of the weight coefficients are all arbitrary real numbers between [0,1 ];

s53, obtaining parameter theta of each source domain BN model by adopting a Maximum Likelihood Estimation (MLE) method_niI is the ith node in the source domain BN model child nodes;

s54, fusing parameters theta of BN models in various source domains_niObtaining the total source domain BN parameter theta_Si；

θ_Si＝∑_nk(n)θ_ni(2)；

S6, acquiring BN parameters of the facial expressions of the target domain;

s61, obtaining parameter theta of target domain initial BN model by adopting maximum posterior probability MAP estimation method_TiI is the ith node in the target domain BN model child nodes;

s62, calculating the final parameter theta of the target domain facial expression BN according to the weight factor_i；

θ_i＝α1θ_Ti+α2θ_si(3)

Wherein α 1 and α 2 are weighting factors, α 1+ α 2 ═ 1;

s63, setting the BN _ Flag to be true, and finishing the modeling of the BN; returning to S1;

s7, recognizing facial expressions;

s71, setting a facial expression attribute probability threshold psi;

s72, putting the facial expression recognition evidence AU into the constructed BN model, and carrying out BN inference by using a joint tree inference algorithm to obtain a facial expression attribute probability psi';

s73, judging facial expressions;

and if the probability Ψ' of the facial expression attributes is greater than or equal to the threshold Ψ, outputting the facial expression attributes, namely the facial expression recognition result, and otherwise, acquiring the new AU data set again.

2. The BN parameter migration learning-based expression recognition method as claimed in claim 1, wherein the step S32 of extracting facial activity units AU according to each type of expression includes:

s321, obtaining geometric characteristics of the facial expression of the human face through a CLM algorithm on the facial expression image of the human face;

s322, extracting the texture features of the facial expressions from the facial expression images of the human faces through an HOG algorithm;

s323, carrying out feature fusion and normalization processing on the geometric features and the textural features of the facial expression to obtain fusion features of the facial expression;

s324, classifying the fusion characteristics of the facial expressions by using a Support Vector Machine (SVM) to obtain a data set M (t) of a target domain facial activity unit AU and a data set M (S) of a source domain facial activity unit AU_n) N is 1,2, the. q, q is the number of source domains, and q is a natural number.

3. The expression recognition method based on BN parameter migration learning of claim 2, wherein the specific step of obtaining the geometric features of the facial expression in step S321 includes:

s3211, obtaining positioning information of feature points of the expression image;

and (5) positioning the feature points by utilizing a CLM facial feature point positioning algorithm, and extracting the geometric features of the facial expression.

4. The expression recognition method based on BN parameter migration learning of claim 1, wherein the specific step of determining the facial expression recognition target/source network BN model structure diagram in step S4 includes:

s41, determining a BN node for recognizing the facial expressions;

determining a father node and a child node of the BN;

s42, determining a directed acyclic graph of the BN (facial expression recognition);

and sequentially connecting the father node and the child node of the BN by using the directed edge, determining to establish a BN model structure diagram G1 of the target network for facial expression recognition, and establishing a BN model structure diagram G2 of the source network for facial expression recognition.

5. The expression recognition method based on BN parameter migration learning of claim 1, wherein the number of categories E _ num of facial expression recognition in step S11 is 6, including: "happy", "surprised", "fear", "angry", "disgust" and "sad" 6 types of expressions.

Images