CN110717434A - Expression recognition method based on feature separation - Google Patents

Expression recognition method based on feature separation Download PDF

Info

Publication number
CN110717434A
CN110717434A CN201910941100.7A CN201910941100A CN110717434A CN 110717434 A CN110717434 A CN 110717434A CN 201910941100 A CN201910941100 A CN 201910941100A CN 110717434 A CN110717434 A CN 110717434A
Authority
CN
China
Prior art keywords
expression
sample
feature
loss
separation
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201910941100.7A
Other languages
Chinese (zh)
Other versions
CN110717434B (en
Inventor
谢龙汉
杨烈
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
South China University of Technology SCUT
Original Assignee
South China University of Technology SCUT
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by South China University of Technology SCUT filed Critical South China University of Technology SCUT
Priority to CN201910941100.7A priority Critical patent/CN110717434B/en
Publication of CN110717434A publication Critical patent/CN110717434A/en
Application granted granted Critical
Publication of CN110717434B publication Critical patent/CN110717434B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/174—Facial expression recognition
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/168—Feature extraction; Face representation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/172—Classification, e.g. identification
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
    • Y02T10/00—Road transport of goods or passengers
    • Y02T10/10—Internal combustion engine [ICE] based vehicles
    • Y02T10/40—Engine management systems

Landscapes

  • Engineering & Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • General Health & Medical Sciences (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Image Analysis (AREA)

Abstract

The invention discloses an expression recognition method based on feature separation. The method separates the features related to the expression from the features unrelated to the expression by using a feature separation framework based on a generated countermeasure network, and then ignores the features unrelated to the expression and only performs expression recognition according to the features related to the expression. The method provides a characteristic separation framework based on generation of a countermeasure network, which consists of a generator G and a discriminator D; wherein the generator G comprises two parts, an encoder Gen and a decoder Gde. In the training process, the framework enables the features really determining the expression to be converged to the expression feature part and the features irrelevant to the expression to be converged to the irrelevant feature part through partial feature exchange and the constraints of center loss, content loss, counter loss and classification loss. In the testing stage, the classifier is trained only according to the expression features to perform expression recognition. Through feature separation, the method can overcome the interference of other irrelevant factors in the expression recognition process, and improve the accuracy of expression recognition.

Description

Expression recognition method based on feature separation
Technical Field
The invention relates to the field of facial expression recognition, in particular to an expression recognition method based on feature separation.
Background
Facial expression is one of the important ways that humans express feelings and intentions. Facial expression recognition has high potential application value in social robots, medical instruments, fatigue driving monitoring and many other human-computer interaction systems, and therefore, a great deal of research is carried out on the facial expression recognition. The traditional method is based on artificially designed features or performs shallow learning to recognize facial expressions, the recognition energy is very limited, and with the development of deep learning, more and more researches adopt a deep learning method to recognize facial expressions, so that the development of the field of facial expression recognition is greatly promoted. Although deep learning has strong feature learning ability and achieves good effect in the task of facial expression recognition, some problems still exist in practical application, mainly including: (1) there are large individual differences between different subjects; (2) the facial pose and the illumination condition of the human face in an actual application scene are different; (3) occlusion problems and the like also exist in practical application scenes.
In recent years, facial expression recognition based on deep learning has been advanced, but the problem of individual difference still remains one of the important factors limiting the improvement of the accuracy rate of facial expression recognition. In the expression recognition task, it is generally desirable that the distance of the images of the same expression in the feature space is smaller than the distance between different expressions. However, due to the fact that different individuals have great differences in age, sex, hair style, skin color, growth, character and the like, the distance between different expressions of the same individual is smaller than the distance between the same expression of different individuals, so that the expression recognition of different individuals is incorrect.
Disclosure of Invention
In order to solve the problem of individual difference in the expression recognition process and avoid interference of irrelevant features on an expression recognition task, the invention provides a feature separation framework based on generation of a countermeasure network. The frame can separate the characteristics related to the expression from the characteristics unrelated to the expression, neglect the characteristics unrelated to the expression and only recognize the expression according to the characteristics related to the expression, thereby overcoming the interference of the unrelated factors and improving the accuracy of the expression recognition.
The purpose of the invention is realized by at least one of the following technical solutions.
An expression recognition method based on feature separation comprises the following steps:
s1, a feature separation frame is provided, and the frame is used for performing expression feature exchange on two input images;
s2, training a feature separation framework through the constraint of various loss functions to ensure that the features really determining the expression are converged into expression feature vectors during feature separation, and the features irrelevant to the expression are converged into irrelevant feature vectors;
and S3, after the training of the feature separation frame is finished, training a classifier according to the expression feature vectors separated by the feature separation frame, and recognizing the expression.
Further, the step S1 specifically includes the following steps:
s1.1, first, a feature separation framework based on generation of a countermeasure network is proposed, which is composed of a generator G and a discriminator D, where the generator G includes an encoder Gen and a decoder Gde;
s1.2, arbitrarily reading two images from a training set to serve as an input first sample and an input second sample, and then respectively performing feature extraction on the input first sample and the input second sample by using an encoder Gen;
s1.3, dividing the extracted feature vector into an expression feature vector and an irrelevant feature vector according to a set proportion to obtain a first expression feature vector, a second expression feature vector, a first irrelevant feature vector and a second irrelevant feature vector; then exchanging the first expression feature vector and the second expression feature vector, and keeping the first irrelevant feature vector and the second irrelevant feature vector unchanged;
s1.4, generating two new images, namely a first generation sample and a second generation sample, by using a decoder Gde according to the feature vectors recombined after exchange;
s1.5, inputting the generated first generated sample and the corresponding real sample, namely the third sample, into a discriminator D, inputting the generated second generated sample and the corresponding real sample, namely the fourth sample, into the discriminator D, and simultaneously carrying out true and false discrimination and classification.
Further, the step S2 includes the following steps:
s2.1, in order to make the characteristics irrelevant to the expression converge into irrelevant vectors, introducing content loss Lcon(ii) a The first sample and the third sample and the second sample and the fourth sample are different in expression, and other characteristics are the same, and the generator G can converge characteristics which are irrelevant to the expression into an invariant characteristic vector which does not participate in exchange through the constraint of content loss between the constraint samples, namely the third sample and the fourth sample, and the generated image;
s2.2, in order to make the features related to the expression converge into the expression feature vector, introducing classification loss; loss of classification from real images during training of a feature separation framework
Figure BDA0002222925690000021
The discriminator D is optimized, so that the classification capability of the discriminator D is improved; while the loss of classification from the generated imageOptimizing a generator G to enable the generator G to realize expression exchange of the generated image through feature exchange, so that the information determining the expression of the generated image is converged into the expression feature vector participating in the exchange;
s2.3, in order to make the generated image very close to the real sample, thereby leading the content loss and the classification loss to better converge, introducing the counter loss Ladv；
S2.4, in order to reduce the distance of the expression features of the same type of samples in the feature space and improve the separation purity of the expression features, the expression recognition accuracy is improved, and the central loss L is introducedcen；
S2.5, finally, according to the comprehensive loss function L of the generator GGSum discriminant D's combined loss function LDTo train a feature separation framework, LG and LDRespectively as follows:
Figure BDA0002222925690000023
wherein ,λcls,λcon and λcenThe weights of the classification loss, the content loss and the center loss, respectively, in the final loss function, need to be determined by a large number of experiments.
Further, in step S2.1, content is lost LconThe expression of (a) is as follows:
Figure BDA0002222925690000025
wherein G (x) represents a generated sample obtained from the input sample x, and
Figure BDA0002222925690000026
representing constrained samples corresponding to calculated content loss for the generated image, ExRepresenting the mathematical expectation from the input sample x.
Further, in step S2.2,
Figure BDA0002222925690000027
and
Figure BDA0002222925690000028
are respectively:
Figure BDA0002222925690000029
Figure BDA00022229256900000210
where x represents the input sample, c represents the class of sample x, G (x) represents the generated sample from sample x, Dcls(c | x) represents the probability that the discriminator D recognizes the input sample x as the class c, Ex,cIndicating a mathematical expectation from the input sample x and the class c of the sample x.
Further, step (ii)In S2.3, the loss L is resistedadvThe expression of (a) is as follows:
Ladv＝Ex[logDsrc(xo)]+Ex[log(1-Dsrc(G(xi)))]；
wherein ,xi and xoInput samples and constraint samples, G (x), respectivelyi) Is based on input sample xiResulting generated image, Dsrc(x) Probability of judging sample x as a true sample for discriminator D, ExRepresenting the mathematical expectation from the input sample x.
Further, in step S2.4, the center loss LcenThe expression of (a) is as follows:
Figure BDA0002222925690000031
wherein ,eiRepresenting the expressive features, y, of the sample iiA category of the sample i is represented,
Figure BDA0002222925690000032
representing a central feature vector of a corresponding category of the sample i in the k training; the initial value of the central feature vector of each expression is a random value, and in the training process, the updating process of the central feature vector of each expression is as follows:
Figure BDA0002222925690000033
Figure BDA0002222925690000034
wherein ,
Figure BDA0002222925690000035
Figure BDA0002222925690000036
represents the average distance between the jth sample and the corresponding central vector in the t iteration,
Figure BDA0002222925690000037
and
Figure BDA0002222925690000038
respectively representing the central vectors of the ith sample in the processes of the t iteration and the t +1 iteration, wherein alpha is the updated learning rate of the central vector, alpha belongs to (0,1), and the specific value of alpha needs to be determined through a large number of experiments.
Further, the step S3 specifically includes the following steps:
s3.1, after training of a feature separation frame is completed, firstly, performing feature extraction and separation on any input sample by using a trained encoder Gen, then training a simple convolutional neural network as a classifier according to the separated expression feature vector, wherein the classifier uses cross entropy loss as an optimization objective function;
and S3.2, after the training of the classifier is finished, reading in test samples from the test set, firstly, using the previously trained encoder Gen to extract and separate the features of each test sample, and then using the classifier to recognize the expressions according to the separated expression feature vectors.
Compared with the prior art, the invention has the advantages that:
the invention provides a feature separation framework based on generation of a countermeasure network, which can separate features related to expressions and features unrelated to the expressions, ignore the features unrelated to the expressions and only recognize the expressions according to the features related to the expressions, thereby overcoming the interference of the unrelated features to the expression recognition process and improving the accuracy of the expression recognition.
Drawings
Fig. 1 is a diagram of a feature separation framework according to an embodiment of the present invention.
Fig. 2 is a network structure diagram for expression recognition in the testing phase according to the embodiment of the present invention.
Detailed Description
The practice of the present invention will be further illustrated by the following examples and drawings, but the practice and protection of the present invention is not limited thereto.
An expression recognition method based on feature separation comprises the following steps:
s1, a feature separation frame is provided, and the frame is used for performing expression feature exchange on two input images, and the method specifically comprises the following steps:
s1.1, first, a feature separation framework based on generation of a countermeasure network is proposed, which is composed of a generator G and a discriminator D, wherein the generator G comprises an encoder Gen and a decoder Gde. In this example, the network structure of the generator is shown in table 1, and the network structure of the discriminator is shown in table 2.
TABLE 1 network architecture of generators
Figure BDA0002222925690000041
In table 1, the convolution module contains one convolution layer, one instance normalization layer, one ReLU activation function layer, and one Dropout layer, and h and w are the height and width of the input image, respectively.
TABLE 2 network architecture of arbiter
Figure BDA0002222925690000042
Figure BDA0002222925690000051
In table 2, the discriminating module includes a convolution layer, a leakage relu activation function layer, and a Dropout layer, and h and w are the height and width of the input image, respectively.
S1.2, arbitrarily inputting two images from a training set as a first sample and a second sample which are input, wherein the size of an input picture is 128 x 128 in the embodiment, and then respectively performing feature extraction on the input first sample and the input second sample by using an encoder Gen.
S1.3, dividing the extracted feature vector into an expression feature vector and an irrelevant feature vector according to a proportion, wherein the feature separation ratio in the example is 124:900, and obtaining a first expression feature vector, a second expression feature vector, a first irrelevant feature vector and a second irrelevant feature vector; and then exchanging the first expression feature vector and the second expression feature vector, wherein the first irrelevant feature vector and the second irrelevant feature vector are kept unchanged.
S1.4, generating two new images, namely a first generation sample and a second generation sample, by using a decoder Gde according to the feature vectors recombined after exchange;
s1.5, inputting the generated first generated sample and the corresponding real sample, namely the third sample, into a discriminator D, inputting the generated second generated sample and the corresponding real sample, namely the fourth sample, into the discriminator D, and simultaneously carrying out true and false discrimination and classification.
S2, training a feature separation framework through the constraint of various loss functions to ensure that the features really determining the expression are converged into expression feature vectors during feature separation, and the features irrelevant to the expression are converged into irrelevant feature vectors, and the specific steps are as follows:
s2.1, in order to make the characteristics irrelevant to the expression converge into irrelevant vectors, introducing content loss Lcon(ii) a The first sample, the second sample, the third sample and the fourth sample are different in expression, other characteristics are the same, and the generator G can converge characteristics irrelevant to the expression into an invariant characteristic vector which does not participate in exchange through the constraint of content loss between the constraint samples, namely the third sample and the fourth sample, and the generated image; content loss LconThe expression of (a) is as follows:
Figure BDA0002222925690000052
wherein G (x) represents a generated sample obtained from the input sample x, and
Figure BDA0002222925690000053
representing constrained samples corresponding to calculated content loss for the generated image, ExRepresenting the mathematical expectation from the input sample x.
S2.2, in order to make the features related to the expression converge into the expression feature vector, introducing classification loss; loss of classification from real images during training of a feature separation framework
Figure BDA0002222925690000054
The discriminator D is optimized, so that the classification capability of the discriminator D is improved; while the loss of classification from the generated image
Figure BDA0002222925690000055
Optimizing a generator G to enable the generator G to realize expression exchange of the generated image through feature exchange, so that the information determining the expression of the generated image is converged into the expression feature vector participating in the exchange;
Figure BDA0002222925690000056
and
Figure BDA0002222925690000061
are respectively:
Figure BDA0002222925690000062
where x represents the input sample, c represents the class of sample x, G (x) represents the generated sample from sample x, Dcls(c | x) represents the probability that the discriminator D recognizes the input sample x as the class c, Ex,cIndicating a mathematical expectation from the input sample x and the class c of the sample x.
S2.3, in order to make the generated image very close to the real sample, thereby leading the content loss and the classification loss to better converge, introducing the counter loss Ladv(ii) a Against loss LadvThe expression of (a) is as follows:
Ladv＝Ex[logDsrc(xo)]+Ex[log(1-Dsrc(G(xi)))]；
wherein ,xi and xoInput samples and constraint samples, G (x), respectivelyi) Is based on input sample xiResulting generated image, Dsrc(x) Probability of judging sample x as a true sample for discriminator D, ExRepresenting the mathematical expectation from the input sample x.
S2.4, in order to reduce the distance of the expression features of the same type of samples in the feature space and improve the separation purity of the expression features, the expression recognition accuracy is improved, and the central loss L is introducedcen(ii) a Center loss LcenThe expression of (a) is as follows:
Figure BDA0002222925690000064
wherein ,eiRepresenting the expressive features, y, of the sample iiA category of the sample i is represented,
Figure BDA0002222925690000065
representing a central feature vector of a corresponding category of the sample i in the k training; the initial value of the central feature vector of each expression is a random value, and in the training process, the updating process of the central feature vector of each expression is as follows:
Figure BDA0002222925690000067
wherein ,
Figure BDA0002222925690000068
Figure BDA00022229256900000614
represents the average distance between the jth sample and the corresponding central vector in the t iteration,and
Figure BDA00022229256900000611
respectively representing the central vectors of the ith sample in the processes of the tth iteration and the t +1 th iteration, wherein alpha is the learning rate (alpha belongs to (0,1)) updated by the central vectors, and the specific value of the central vectors needs to be determined through a large number of experiments;
s2.5, finally, according to the comprehensive loss function L of the generator GGSum discriminant D's combined loss function LDTo train a feature separation framework, LG and LDRespectively as follows:
Figure BDA00022229256900000612
Figure BDA00022229256900000613
wherein ,λcls,λcon and λcenThe weights of the classification loss, the content loss and the central loss in the final loss function are respectively, and through a large number of experiments, the values of the weights in the example are respectively as follows: lambda [ alpha ]cls＝3,λcon＝10，λcen＝10。
And S3, after the training of the feature separation frame is finished, training a classifier according to the expression feature vectors separated by the feature separation frame, and recognizing the expression, wherein as shown in FIG. 2, the specific steps are as follows:
s3.1, after training of a feature separation frame is completed, firstly, performing feature extraction and separation on any input sample by using a trained encoder Gen, then training a simple convolutional neural network as a classifier according to the separated expression feature vector, wherein the network structure of the simple convolutional neural network is shown in a table 3, and the classifier uses cross entropy loss as an optimization objective function;
TABLE 3 network architecture of convolutional neural networks
Figure BDA0002222925690000071
The classification module in the table contains a convolution layer, a LeakyReLU activation function layer and a Dropout layer, where h and w are the height and width of the input image, respectively.
And S3.2, after the training of the classifier is finished, reading in test samples from the test set, firstly, using the previously trained encoder Gen to extract and separate the features of each test sample, and then using the classifier to recognize the expressions according to the separated expression feature vectors.
The frame can separate the features related to the expression from the features unrelated to the expression, neglect the features unrelated to the expression and only recognize the expression according to the features related to the expression, thereby overcoming the interference of the unrelated features to the expression recognition process and improving the accuracy of the expression recognition.

Claims (8)

1. An expression recognition method based on feature separation is characterized by comprising the following steps:
s1, a feature separation frame is provided, and the frame is used for performing expression feature exchange on two input images;
s2, training a feature separation framework through the constraint of various loss functions to ensure that the features really determining the expression are converged into expression feature vectors during feature separation, and the features irrelevant to the expression are converged into irrelevant feature vectors;
and S3, after the training of the feature separation frame is finished, training a classifier according to the expression feature vectors separated by the feature separation frame, and recognizing the expression.
2. The expression recognition method based on feature separation according to claim 1, wherein the step S1 specifically includes the following steps:
s1.1, first, a feature separation framework based on generation of a countermeasure network is proposed, which is composed of a generator G and a discriminator D, where the generator G includes an encoder Gen and a decoder Gde;
s1.2, arbitrarily reading two images from a training set to serve as an input first sample and an input second sample, and then respectively performing feature extraction on the input first sample and the input second sample by using an encoder Gen;
s1.3, dividing the extracted feature vector into an expression feature vector and an irrelevant feature vector according to a set proportion to obtain a first expression feature vector, a second expression feature vector, a first irrelevant feature vector and a second irrelevant feature vector; then exchanging the first expression feature vector and the second expression feature vector, and keeping the first irrelevant feature vector and the second irrelevant feature vector unchanged;
s1.4, generating two new images, namely a first generation sample and a second generation sample, by using a decoder Gde according to the feature vectors recombined after exchange;
s1.5, inputting the generated first generated sample and the corresponding real sample, namely the third sample, into a discriminator D, inputting the generated second generated sample and the corresponding real sample, namely the fourth sample, into the discriminator D, and simultaneously carrying out true and false discrimination and classification.
3. The expression recognition method based on feature separation according to claim 1, wherein the step S2 comprises the following steps:
s2.1, in order to make the characteristics irrelevant to the expression converge into irrelevant vectors, introducing content loss Lcon(ii) a The first sample and the third sample and the second sample and the fourth sample are different in expression, and other characteristics are the same, and the generator G can converge characteristics which are irrelevant to the expression into an invariant characteristic vector which does not participate in exchange through the constraint of content loss between the constraint samples, namely the third sample and the fourth sample, and the generated image;
s2.2, in order to make the features related to the expression converge into the expression feature vector, introducing classification loss; loss of classification from real images during training of a feature separation framework
Figure FDA0002222925680000011
To optimize the discriminator D and thereby improve the discriminator DA classification capability; while the loss of classification from the generated image
Figure FDA0002222925680000012
Optimizing a generator G to enable the generator G to realize expression exchange of the generated image through feature exchange, so that the information determining the expression of the generated image is converged into the expression feature vector participating in the exchange;
s2.3, in order to make the generated image very close to the real sample, thereby leading the content loss and the classification loss to better converge, introducing the counter loss Ladv；
S2.4, in order to reduce the distance of the expression features of the same type of samples in the feature space and improve the separation purity of the expression features, the expression recognition accuracy is improved, and the central loss L is introducedcen；
S2.5, finally, according to the comprehensive loss function L of the generator GGSum discriminant D's combined loss function LDTo train a feature separation framework, LG and LDRespectively as follows:
Figure FDA0002222925680000013
Figure FDA0002222925680000021
wherein ,λcls，λcon and λcenThe weights of the classification loss, the content loss and the center loss, respectively, in the final loss function, need to be determined by a large number of experiments.
4. The method for recognizing expressions based on feature separation as claimed in claim 3, wherein in step S2.1, the content is lost by LconThe expression of (a) is as follows:
wherein G (x) represents a generated sample obtained from the input sample x, andrepresenting constrained samples corresponding to calculated content loss for the generated image, ExRepresenting the mathematical expectation from the input sample x.
5. The expression recognition method based on feature separation as claimed in claim 3, wherein in step S2.2,
Figure FDA0002222925680000023
and
Figure FDA0002222925680000024
are respectively:
Figure FDA0002222925680000025
Figure FDA0002222925680000026
where x represents the input sample, c represents the class of sample x, G (x) represents the generated sample from sample x, Dcls(c | x) represents the probability that the discriminator D recognizes the input sample x as the class c, Ex，cIndicating a mathematical expectation from the input sample x and the class c of the sample x.
6. The expression recognition method based on feature separation as claimed in claim 3, wherein in step S2.3, the confrontation loss LadvThe expression of (a) is as follows:
Ladv＝Ex[logDsrc(xo)]+Ex[log(1-Dsrc(G(xi)))]；
wherein ,xi and xoInput samples and constraint samples, G (x), respectivelyi) Is based on input sample xiResulting generated image, Dsrc(x) Probability of judging sample x as a true sample for discriminator D, ExRepresenting the mathematical expectation from the input sample x.
7. The expression recognition method based on feature separation as claimed in claim 3, wherein in step S2.4, the center loss L iscenThe expression of (a) is as follows:
Figure FDA0002222925680000027
wherein ,eiRepresenting the expressive features, y, of the sample iiA category of the sample i is represented,
Figure FDA0002222925680000028
representing a central feature vector of a corresponding category of the sample i in the k training; the initial value of the central feature vector of each expression is a random value, and in the training process, the updating process of the central feature vector of each expression is as follows:
Figure FDA0002222925680000029
Figure FDA00022229256800000210
wherein ,
Figure FDA0002222925680000031
Figure FDA0002222925680000032
represents the average distance between the jth sample and the corresponding central vector in the t iteration,
Figure FDA0002222925680000033
and
Figure FDA0002222925680000034
respectively representing the central vectors of the ith sample in the processes of the t iteration and the t +1 iteration, wherein alpha is the updated learning rate of the central vector, alpha belongs to (0,1), and the specific value of alpha needs to be determined through a large number of experiments.
8. The expression recognition method based on feature separation according to claim 1, wherein the step S3 specifically includes the following steps:
s3.1, after training of a feature separation frame is completed, firstly, performing feature extraction and separation on any input sample by using a trained encoder Gen, then training a simple convolutional neural network as a classifier according to the separated expression feature vector, wherein the classifier uses cross entropy loss as an optimization objective function;
and S3.2, after the training of the classifier is finished, reading in test samples from the test set, firstly, using the previously trained encoder Gen to extract and separate the features of each test sample, and then using the classifier to recognize the expressions according to the separated expression feature vectors.
CN201910941100.7A 2019-09-30 2019-09-30 Expression recognition method based on feature separation Active CN110717434B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201910941100.7A CN110717434B (en) 2019-09-30 2019-09-30 Expression recognition method based on feature separation

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201910941100.7A CN110717434B (en) 2019-09-30 2019-09-30 Expression recognition method based on feature separation

Publications (2)

Publication Number Publication Date
CN110717434A true CN110717434A (en) 2020-01-21
CN110717434B CN110717434B (en) 2023-05-23

Family

ID=69212138

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201910941100.7A Active CN110717434B (en) 2019-09-30 2019-09-30 Expression recognition method based on feature separation

Country Status (1)

Country Link
CN (1) CN110717434B (en)

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20190042952A1 (en) * 2017-08-03 2019-02-07 Beijing University Of Technology Multi-task Semi-Supervised Online Sequential Extreme Learning Method for Emotion Judgment of User
CN109508669A (en) * 2018-11-09 2019-03-22 厦门大学 A kind of facial expression recognizing method based on production confrontation network

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20190042952A1 (en) * 2017-08-03 2019-02-07 Beijing University Of Technology Multi-task Semi-Supervised Online Sequential Extreme Learning Method for Emotion Judgment of User
CN109508669A (en) * 2018-11-09 2019-03-22 厦门大学 A kind of facial expression recognizing method based on production confrontation network

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
胡敏;余胜男;王晓华;: "基于约束性循环一致生成对抗网络的人脸表情识别方法" *

Also Published As

Publication number Publication date
CN110717434B (en) 2023-05-23

Similar Documents

Publication Publication Date Title
Li et al. Lightweight attention convolutional neural network for retinal vessel image segmentation
CN108537743B (en) Face image enhancement method based on generation countermeasure network
CN108615010B (en) Facial expression recognition method based on parallel convolution neural network feature map fusion
Sargin et al. Audiovisual synchronization and fusion using canonical correlation analysis
He et al. Multi view facial action unit detection based on CNN and BLSTM-RNN
CN114398961B (en) Visual question-answering method based on multi-mode depth feature fusion and model thereof
CN111274921B (en) Method for recognizing human body behaviors by using gesture mask
CN111523462A (en) Video sequence list situation recognition system and method based on self-attention enhanced CNN
CN112784929B (en) Small sample image classification method and device based on double-element group expansion
CN111709306A (en) Double-current network behavior identification method based on multilevel space-time feature fusion enhancement
Huang et al. End-to-end continuous emotion recognition from video using 3D ConvLSTM networks
CN110826462A (en) Human body behavior identification method of non-local double-current convolutional neural network model
CN110969073B (en) Facial expression recognition method based on feature fusion and BP neural network
CN112115796A (en) Attention mechanism-based three-dimensional convolution micro-expression recognition algorithm
CN114299542A (en) Video pedestrian re-identification method based on multi-scale feature fusion
Garg et al. Facial expression recognition & classification using hybridization of ICA, GA, and neural network for human-computer interaction
CN111401116B (en) Bimodal emotion recognition method based on enhanced convolution and space-time LSTM network
Yin et al. Dynamic difference learning with spatio-temporal correlation for deepfake video detection
CN114511912A (en) Cross-library micro-expression recognition method and device based on double-current convolutional neural network
CN110415261B (en) Expression animation conversion method and system for regional training
Liu et al. Discriminative Feature Representation Based on Cascaded Attention Network with Adversarial Joint Loss for Speech Emotion Recognition.
CN113850182A (en) Action identification method based on DAMR-3 DNet
Wei et al. A survey of facial expression recognition based on deep learning
CN115797827A (en) ViT human body behavior identification method based on double-current network architecture
CN116884072A (en) Facial expression recognition method based on multi-level and multi-scale attention mechanism

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant