CN110163098A - Based on the facial expression recognition model construction of depth of seam division network and recognition methods - Google Patents
Based on the facial expression recognition model construction of depth of seam division network and recognition methods Download PDFInfo
- Publication number
- CN110163098A CN110163098A CN201910307984.0A CN201910307984A CN110163098A CN 110163098 A CN110163098 A CN 110163098A CN 201910307984 A CN201910307984 A CN 201910307984A CN 110163098 A CN110163098 A CN 110163098A
- Authority
- CN
- China
- Prior art keywords
- recognition
- expression
- face
- label
- facial
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000008921 facial expression Effects 0.000 title claims abstract description 57
- 238000000034 method Methods 0.000 title claims abstract description 51
- 238000010276 construction Methods 0.000 title abstract description 7
- 230000014509 gene expression Effects 0.000 claims abstract description 96
- 208000004929 Facial Paralysis Diseases 0.000 claims abstract description 36
- 208000036826 VIIth nerve paralysis Diseases 0.000 claims abstract description 36
- 230000001815 facial effect Effects 0.000 claims description 13
- 238000012549 training Methods 0.000 claims description 12
- 238000010606 normalization Methods 0.000 claims description 3
- 230000006870 function Effects 0.000 description 34
- 238000013528 artificial neural network Methods 0.000 description 8
- 230000005484 gravity Effects 0.000 description 6
- 210000004709 eyebrow Anatomy 0.000 description 5
- 238000002474 experimental method Methods 0.000 description 4
- 230000007423 decrease Effects 0.000 description 3
- 230000008451 emotion Effects 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 241000208340 Araliaceae Species 0.000 description 2
- 235000005035 Panax pseudoginseng ssp. pseudoginseng Nutrition 0.000 description 2
- 235000003140 Panax quinquefolius Nutrition 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 235000008434 ginseng Nutrition 0.000 description 2
- 210000003128 head Anatomy 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 238000005457 optimization Methods 0.000 description 2
- 238000012360 testing method Methods 0.000 description 2
- 238000013473 artificial intelligence Methods 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000002939 conjugate gradient method Methods 0.000 description 1
- 238000013527 convolutional neural network Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000011478 gradient descent method Methods 0.000 description 1
- 239000004615 ingredient Substances 0.000 description 1
- 238000012804 iterative process Methods 0.000 description 1
- 238000003062 neural network model Methods 0.000 description 1
- 210000002569 neuron Anatomy 0.000 description 1
- 238000011176 pooling Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/243—Classification techniques relating to the number of classes
- G06F18/24323—Tree-organised classifiers
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/174—Facial expression recognition
- G06V40/175—Static expression
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Artificial Intelligence (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Oral & Maxillofacial Surgery (AREA)
- Human Computer Interaction (AREA)
- Multimedia (AREA)
- Image Analysis (AREA)
- Image Processing (AREA)
Abstract
The invention discloses a kind of based on the facial expression recognition model construction of depth of seam division network and recognition methods, it proposes the double-deck Tree Classifier and uses a facial expression recognition classifier and multiple recognition of face classifiers, high-rise learning tasks lay particular emphasis on the identification of human face expression, and each low layer learning tasks lay particular emphasis on recognition of face (face with identical expression).Furthermore, layered structure can be used for determining the distribution of the quantity and low-level nodes (face) of the slubbing point (human face expression) in each learning tasks, it should be wherein assigned in identical learning tasks with the face of identical human face expression, method provided by the invention is in facial paralysis Expression Recognition and extends in conventional facial expression recognition problem and has higher accuracy rate.
Description
Technical field
The present invention relates to facial image recognition methods, and in particular to a kind of facial expression recognition based on depth of seam division network
Model construction and recognition methods.
Background technique
With the development of the related disciplines such as computer technology, artificial intelligence technology, the degree of automation of entire society is continuous
It improves, people are increasingly strong for the demand of the human-computer interaction of similar person to person's exchange way.If computer and robot energy
It is enough that there is the ability for understanding and showing emotion as the mankind, it will fundamentally change the relationship between people and computer, make to count
Calculation machine can be serviced preferably for the mankind.Expression Recognition is the basis of affective comprehension, is the premise of computer understanding people's emotion.
If realizing that computer will fundamentally change the relationship of people and computer to the understanding of human face expression with identification, this will be to future
Human-computer interaction generates important meaning.
The existing expression recognition method based on deep neural network is absorbed in Expression Recognition individual task mostly, does not consider
The influence of facial shape otherness, such as facial paralysis Expression Recognition, the purpose is to ignore facial paralysis ingredient and identify expressive components,
And facial paralysis is identified, facial paralysis expressive components should be ignored, and emphasis identifies facial paralysis, however, in real world facial paralysis expression with
The many factors weave ins such as facial shape, head pose, the exterior light photograph of Different Individual, while the difference of facial shape is also
It will lead to facial paralysis expression and generate variation.
Therefore expression recognition method in the prior art can only identify facial expression mostly, can not be simultaneously to face and table
Feelings are identified.
Summary of the invention
The purpose of the present invention is to provide a kind of based on the facial expression recognition model construction of depth of seam division network and identification
Method can not simultaneously identify face and expression to solve expression recognition method in the prior art.
In order to realize above-mentioned task, the invention adopts the following technical scheme:
A kind of facial expression recognition model building method based on depth of seam division network executes according to the following steps:
Step 1 obtains the image that several include face, obtains original image set；
Step 2 carries out again after concentrating every width original image to carry out the pretreatment of human face region interception to the original image
The normalization of picture size obtains facial expression image collection；
Step 3 obtains the corresponding label of facial expression image concentration each image, obtains tally set；The label includes table
Feelings label and face label；
Step 4 trains depth of seam division using the tally set as output using the facial expression image collection as input
Network obtains facial expression recognition model；
The depth of seam division network includes the multiple convolutional layers set gradually, multiple full articulamentums and the double-deck tree
Classifier；
The double-deck Tree Classifier includes Expression Recognition classification layer and recognition of face classification layer, the Expression Recognition
Layer of classifying exports expression label, and the recognition of face classification layer exports face label.
Further, face table is obtained using error back propagation method training depth of seam division network in the step 4
Feelings identification model.
Further, it is calculated separately using Formulas I with Formula II using when error back propagation method training depth of seam division network
The partial derivative of Expression Recognition classification layer weight parameter and the partial derivative of recognition of face classification layer weight parameter:
Wherein,For the partial derivative of Expression Recognition classification layer weight parameter, ￡ is loss function, and W is Expression Recognition classification
Layer weight parameter, xeFor Expression Recognition classify layer input feature vector,For the local derviation of recognition of face classification layer weight parameter
Number, V are recognition of face classification layer weight parameter, xfFor the feature vector of recognition of face classification layer input；
The loss function ￡ is obtained using formula III:
Wherein, VnIndicate the weight parameter of n-th of face label of recognition of face classification layer, VjIndicate recognition of face classification
The weight parameter of j-th of face label of layer, j=1,2 ..., N, n=1,2 ..., N, N indicate face total number of labels, and N is positive whole
Number, bfIndicate recognition of face offset parameter, bfFor natural number；
WmIndicate the weight parameter of m-th of expression label of Expression Recognition classification layer, WiIndicate the of Expression Recognition classification layer
The weight parameter of i expression label, i=1,2 ..., M, m=1,2 ..., M, M indicate expression label sum, and M is positive integer, be
Indicate Expression Recognition offset parameter, beFor natural number.
Further, depth of seam division network described in the step 4 include 5 convolutional layers set gradually, 3 it is complete
Articulamentum and 1 double-deck Tree Classifier.
Further, the face label in the step 3 includes facial paralysis label and non-facial paralysis label.
A kind of facial expression recognizing method based on depth of seam division network, using step A- step B to face to be identified
Image is handled, and recognition result is obtained:
Step A, the facial image to be identified is handled using the method for step 2, obtains facial expression image；
Step B, the step A facial expression image obtained is identified using facial expression recognition model, obtains recognition result.
The present invention has following technical effect that compared with prior art
1, provided by the invention a kind of based on the facial expression recognition model construction of depth of seam division network and recognition methods, it adopts
The plane softmax classifier in deep neural network is replaced with the double-deck Tree Classifier, it is poor to solve human individual's facial shape
The different influence that facial expression recognition is generated；It is used in conjunction with expression label and face label in the depth of seam division network of proposition,
It is identified while realization to expression and face, and improves the accuracy rate of identification；
2, provided by the invention a kind of based on the facial expression recognition model construction of depth of seam division network and recognition methods, lead to
It crosses and devises loss function, it can be with the weight coefficient in combined optimization multistage classifier, so that the function during each iteration
The correctly predicted probability of Expression Recognition and recognition of face can be maximized.
Detailed description of the invention
Fig. 1 is the depth of seam division schematic network structure provided in one embodiment of the present of invention.
Specific embodiment
The technical term occurred in the present invention is explained first, in the technology to help to better understand the application
Hold:
The double-deck Tree Classifier: one has the classifier of the double-deck classification layer, passes through the building one double-deck tree in the present invention
Shape structure embodies the relationship between recognition of face and facial expression recognition task, and first layer is Expression Recognition classification, the second layer
For recognition of face classification, input picture is trained in combination with face label and human face expression label, so that study has more
The depth characteristic of discrimination.
Loss function: being that chance event or its value in relation to stochastic variable are mapped as nonnegative real number to indicate that this is random
The function of " risk " or " loss " of event.In the application, loss function is associated usually as learning criterion with optimization problem,
I.e. by minimizing loss function solution and assessment models.
Neural network weight coefficient: the weight coefficient connected between neuron.
The partial derivative of neural network weight coefficient: important ginseng when neural network weight coefficient is updated using gradient descent method
Number.
Expression label: can express the label of human emotion, can be happy, sad and tranquil etc..
Face label: can represent the label of face characteristic, can be about the age, about the colour of skin or about human face's shape
The label of state.
Error backpropagation algorithm: the purpose of back-propagation algorithm is that the method declined by gradient finds one group of energy maximum
Reduce to limit the weight of error, the detailed process of error backpropagation algorithm are as follows: firstly, initialization neural network；Secondly, defeated
Enter training sample pair, calculates each layer output；Then network output error is calculated；Then each layer error signal is calculated；Then pass through
The method of gradient decline adjusts each layer weight, wherein when the method using gradient decline adjusts each layer weight, it is each by calculating
The partial derivative of layer weight coefficient obtains gradient, and along the opposite direction of gradient vector, gradient reduces most fast, it is easier to find function
Minimum value, that is, weighted value；Then it checks whether network overall error reaches required precision, meets, then training terminates；It is unsatisfactory for, then
Input training sample pair is returned, is recycled after calculating each layer output.
Embodiment one
Since many factors such as the facial shape of expression and Different Individual, head pose, exterior light photograph are handed in real world
It is woven in together, while the difference of facial shape also results in expression and generates variation.In order to weaken facial shape to Expression Recognition
It influences, discloses a kind of facial expression recognition model building method based on depth of seam division network in the present embodiment.
Method provided by the invention constructs the double-deck Tree Classifier based on existing depth convolutional neural networks model to replace
The plane softmax classifier of output layer is changed, depth multi-task learning frame is constructed, by utilizing expression label and face label
Common study has more the depth characteristic of sense, knowledge is migrated from related face identification mission, to weaken face
Influence of the form to Expression Recognition improves Expression Recognition performance.
Specifically, method provided by the invention executes according to the following steps:
Step 1 obtains the image that several include face, obtains original image set；
Step 2 carries out again after concentrating every width original image to carry out the pretreatment of human face region interception to the original image
The normalization of picture size obtains facial expression image collection；
In the present embodiment, the face position of original image is intercepted out, and be normalized, picture size is
224*224。
Step 3 obtains the corresponding label of facial expression image concentration each image, obtains tally set；The label includes table
Feelings label and face label；
In the present embodiment, expression label can be the label that can express human feelings sense such as happy, sad and tranquil；People
Face label can be about the age, about the colour of skin, about any label that can represent face characteristic of human face's state, such as
Method provided in this embodiment is applied in facial paralysis Expression Recognition, therefore face label includes facial paralysis label and non-facial paralysis mark
Label；Or method provided in this embodiment is applied to based in facial image fatigue driving identification, face label packet at this time
Include tired label and non-tired label, that is to say, that facial expression recognizing method provided in this embodiment can extend to more
A application environment, compared to a kind of traditional neural network that can only obtain recognition result, recognition methods provided in this embodiment
Two kinds of recognition results can be obtained simultaneously.
In the present embodiment, facial expression recognizing method provided by the invention is applied to facial paralysis Expression Recognition, wherein table
Feelings label includes closing one's eyes, smiling, lift eyebrow, frowning, alarmming nose, showing tooth and drum this 7 kinds of expressions of the cheek, face label include facial paralysis label with
And non-facial paralysis label；Therefore, there is 7 × 2=14 kind for a possibility that its label of width facial expression image.Such as a width expression
Image, label are [facial paralysis is closed one's eyes] or [non-facial paralysis is smiled] etc..
Step 4 trains depth of seam division using the tally set as output using the facial expression image collection as input
Network obtains facial expression recognition model；
The depth of seam division network includes the multiple convolutional layers set gradually, multiple full articulamentums and the double-deck tree
Classifier；
The double-deck Tree Classifier includes Expression Recognition classification layer and recognition of face classification layer, the Expression Recognition
Layer of classifying exports expression label, and the recognition of face classification layer exports face label.
In the present embodiment, depth of seam division network is completed on the basis of deep neural network model A1exNet, such as Fig. 1
It is shown, as a preferred embodiment, depth of seam division network include the 5 layers of convolutional layer set gradually, 3 layers of full articulamentum with
And the double-deck Tree Classifier.
Wherein the 1st hidden layer is convolutional layer, uses the convolution kernel of 96 11x 11x 3, number of nodes are as follows: 55 (W) x 55
(H) (C) x of x 48 2=290400.It (is rolled up according to the parameter of convolutional layer=convolution kernel size x convolution kernel quantity+number of offsets
Long-pending nuclear volume), this layer parameter quantity are as follows: (11*11*3*96)+96=34848.
2nd hidden layer is convolutional layer, using 256 5x5x48 convolution kernels, only with upper one layer of connection of the same GPU, section
Point quantity: 27*27*128*2=186624, number of parameters: (5*5*48*128+128) * 2=307456, last " * 2 " be because
Network layer is evenly distributed on two GPU, first calculates the parameter on single GPU, multiplied by GPU quantity 2.
3rd hidden layer is convolutional layer, using 384 3x3x256 convolution kernels, number of nodes: and 13*13*192*2=64896,
Number of parameters: 3*3*256*384+384=885120.
4th hidden layer is convolutional layer, using 384 3x3x192 convolution kernels, only with upper one layer of connection of the same GPU, section
Point quantity: 13*13*192*2=64896, number of parameters: (3*3*192*192+192) * 2=663936.
5th hidden layer is convolutional layer, using 256 3x3x192 convolution kernels, only with upper one layer of connection of the same GPU, section
Point quantity: 13*13*128*2=43264, number of parameters: (3*3*192*128+128) * 2=442624.
6th hidden layer is full articulamentum, number of nodes 4096.According to the number of parameters of full articulamentum=upper node layer number
Measure the lower node layer quantity+number of offsets (i.e. next layer of number of nodes) of (after pooling) x, number of parameters are as follows: (6*
6*128*2) * 4096+4096=37752832, this number of parameters be far longer than before all convolutional layers number of parameters it
With, that is to say, that the parameter of AlexNet is predominantly located in subsequent full articulamentum.
7th hidden layer is full articulamentum, and number of nodes is expression quantity.
8th hidden layer is full articulamentum, and the number of nodes of each is the quantity of corresponding expression human face.
In the present embodiment, the method at training depth of seam division network can be conjugate gradient method or error backpropagation algorithm
Deng.
In order to improve the accuracy rate and efficiency of recognition methods, it is preferable that instructed in step 4 using error back propagation method
Practice depth of seam division network, obtains facial expression recognition model.
In the present embodiment, due to devising the double-deck Tree Classifier, wherein Expression Recognition classification layer lays particular emphasis on human face expression
Identification, the classification results of each Expression Recognition classification layer have corresponded to a recognition of face classification layer.In addition, layered structure is available
In the distribution for the quantity and low-level nodes (face) for determining the slubbing point (human face expression) in each learning tasks, wherein having phase
Face with human face expression should be assigned in identical learning tasks.
The purpose of back-propagation algorithm is to find one group by the method for gradient decline to reduce error to the maximum extent
Weight, the partial derivative of weight coefficient constitutes gradient in loss function, and loss function is the function about weight, each by calculating
The partial derivative of layer weight coefficient obtains gradient, and along the opposite direction of gradient vector, gradient reduces most fast, it is easier to find function
Minimum value, it is therefore desirable to partial derivative is sought to weight coefficient, repetition training is carried out along the opposite direction of gradient vector, obtains
The weight of error can be reduced.
Optionally, table is calculated separately using Formulas I and Formula II using when error back propagation method training depth of seam division network
The partial derivative of feelings identification classification layer weight coefficient and the partial derivative of recognition of face classification layer weight coefficient:
Wherein,For the partial derivative of Expression Recognition classification layer weight coefficient, ￡ is loss function, and W is Expression Recognition classification
Layer weight coefficient, xeFor Expression Recognition classify layer input feature vector,For the local derviation of recognition of face classification layer weight coefficient
Number, V are recognition of face classification layer weight coefficient, xfFor the feature vector of recognition of face classification layer input.
The loss function ￡ is obtained using formula III:
Wherein, VnIndicate the weight coefficient of n-th of face label of recognition of face classification layer, VjIndicate recognition of face classification
The weight coefficient of j-th of face label of layer, j=1,2 ..., N, n=1,2 ..., N, N indicate face total number of labels, and N is positive whole
Number, bfIndicate recognition of face offset parameter, bfIt is updated together with weight coefficient V；
WmIndicate the weight coefficient of m-th of expression label of Expression Recognition classification layer, WiIndicate the of Expression Recognition classification layer
The weight coefficient of i expression label, i=1,2 ..., M, m=1,2 ..., M, M indicate expression label sum, and M is positive integer, be
Indicate Expression Recognition offset parameter, beIt is updated together with weight coefficient W.
It in the present embodiment, should be for not since the variation of human face expression will affect the accuracy rate of recognition of face
Same classification task uses specific depth characteristic.
In the present embodiment, specific explanations are carried out to the calculating of partial derivative by taking facial paralysis Expression Recognition as an example, wherein face mark
Label include facial paralysis and non-facial paralysis, and expression label includes eye closing, smile, lift eyebrow, frown, alarmming nose, shows tooth and the drum cheek, therefore M=
7, N=2；
Firstly, calculating loss function ￡ using formula III:
Calculate the loss function of Expression Recognition classification layer part in loss function:
Wherein, M indicates expression label sum, and in the present embodiment, M=7, m=1 indicate to close one's eyes, and m=2 indicates to smile, m
=3 indicate lift eyebrow, and m=4 expression is frowned, and m=5 indicates to alarm nose, and m=6 shows that tooth, m=7 indicate the drum cheek；I=1 indicates to close one's eyes,
I=2 indicates to smile, and i=3 indicates lift eyebrow, and i=4 expression is frowned, and i=5 indicates to alarm nose, and i=6 shows that tooth, i=7 indicate drum
The cheek；In the present embodiment, xeFor the output vector of the last one full articulamentum, i.e., the face table obtained by preceding 8 layers of neural network
The depth characteristic vector of feelings.
Summation indicates to seek weight coefficient * feature vector+offset parameter summation of all expression labels below fraction, point
It is weight coefficient * feature vector+offset parameter for seeking any one expression label above formula, whole fraction is for seeking
The weight coefficient * feature vector of weight coefficient * feature vector+offset parameter of any one expression label and all expression labels+
The specific gravity of the summation of offset parameter；After calculating the corresponding rate of specific gravity of 7 labels, after taking logarithm to each rate of specific gravity, 7 ratios are sought
It is negated after the summation of weight values logarithm, is exactly the loss function of Expression Recognition classification layer part in loss function.
Calculate the loss function of recognition of face classification layer part in loss function:
In the present embodiment, when carrying out facial paralysis Expression Recognition, N=2, n=1 indicate facial paralysis, and n=2 indicates non-facial paralysis, j=1
Indicate facial paralysis, j=2 indicates non-facial paralysis；In the present embodiment, xfIt is defeated for Expression Recognition classification layer and the last one full articulamentum
Feature vector out, i.e. the depth characteristic vector of recognition of face.
It is consistent with the Expression Recognition classification loss function of layer part is calculated, it is corresponding by seeking 2 face labels
After rate of specific gravity, logarithm is taken to each rate of specific gravity, is negated after seeking the summation of 2 rate of specific gravity logarithms, is exactly face in loss function
The loss function of identification classification layer part；
By in loss function Expression Recognition classification layer part loss function and loss function in recognition of face classify layer portion
The loss function divided adds up, and obtains loss function ￡.
Secondly, updating Expression Recognition classification layer weight coefficient W using Formulas I, comprising:
Step I, it calculatesWithOverall error is obtained to the partial derivative of Expression Recognition classification layer weight coefficient W
Step II, it pressesMode realizes the update of Expression Recognition classification layer weight coefficient W, wherein taking
Learning coefficient η=0.5；
Learning coefficient η is a factor for guaranteeing weighted value smooth change, avoids crossing most when close to minimum value
Small value, this factor are commonly referred to as learning rate, take 0 < η < 1.
When practical calculating, with asking difference to replace seeking partial derivative, the difference of the input and output by calculating every layer obtains corresponding ginseng
Several partial derivatives, and then realize the update of coefficient W.
Finally, updating recognition of face classification layer weight coefficient V using Formula II, comprising:
Step one calculatesWithOverall error is obtained to the partial derivative of V
Step two is pressedMode realizes the update of recognition of face classification layer weight coefficient V, wherein taking
Practise coefficient μ=0.5.
In the present embodiment, W, V, b when specific implementatione、bfCalculating be on Pytorch platform realize simultaneously update,
Pytorch platform can calculate the partial derivative of corresponding parameter, and then to W, V, b by the difference of every layer of input and output of calculatinge、bf
Parameter is all updated.
The sentence of calculating is mainly as follows:
Ecriterion=nn.CrossEntropyLoss () .cuda (args.gpu) # expression classifier loss function
Fcriterion=nn.CrossEntropyLoss () .cuda (args.gpu) # face classification device loss function
E_output, f_output=model (Imag) # do forward process to input image data Imag, are exported
output
Loss=ecriterion (e_output, e_target)+fcriterion (f_output, f_target_
Var) # calculates separately the loss between two classifiers output and target
Loss.backward () # reverse procedure calculates gradient of the loss about each parameter
The calculated gradient of optimizer.step () # is updated parameter
Loss function provided in this embodiment is different from traditional backpropagation, only pays the utmost attention in each iterative process
Relevant learning tasks, loss function only consider the associated weight coefficient being layered in Tree Classifier and other learning tasks
Prediction probability prevents in the training process far from global optimum.
Embodiment two
A kind of facial expression recognizing method based on depth of seam division network is disclosed in the present embodiment, to people to be identified
Face image obtains recognition result using step A- step B:
Step A, the facial image to be identified is handled using the method for step 2 in embodiment one, is obtained
Facial expression image；
Step B, the step A facial expression image obtained is identified using the facial expression recognition model in embodiment one,
Obtain recognition result.
In the present embodiment, it is tested on HP work station, workstation configuration Intel i5-7400CPU, has 8G
The Nvidia GTX1080i video card of memory, 16G memory, 512G SSD hard disk.In addition, all experiments are all in PyTorch platform
Upper progress.
In this experiment, it is tested using facial paralysis expression data library, the data-base recording the seven of 49 facial paralysis patients
Kind facial paralysis expression (closes one's eyes, smiles, lift eyebrow, frowning, alarmming nose, showing tooth and rouse the cheek).Since resulting facial paralysis data are limited, each
People corresponds to each facial expressions and acts and there was only a picture, therefore can only carry out the reality of face-independent in this experiment
It tests.This experiment selects 268 facial paralysis expression samples as training sample altogether, and 40 samples are as test sample.
Facial paralysis expression data library is known using recognition methods provided by the invention and the identification of conventional depth learning network
Not, accuracy rate comparison is shown in Table 1.
The accuracy rate of distinct methods on 1 facial paralysis expression data library of table
From table 1 it follows that hierarchy structure model proposed by the present invention is available one in the identification of facial paralysis expression
Fixed promotion, relative to VGG-fintune, accuracy rate improves 5%, improves relative to Inception V3 accuracy rate
2.5%, relative to Resnet 34, accuracy rate improves 5%, and recognition effect is promoted obvious.
Claims (6)
1. a kind of facial expression recognition model building method based on depth of seam division network, which is characterized in that according to the following steps
It executes:
Step 1 obtains the image that several include face, obtains original image set；
Step 2 carries out image after concentrating every width original image to carry out the pretreatment of human face region interception to the original image again
The normalization of size obtains facial expression image collection；
Step 3 obtains the corresponding label of facial expression image concentration each image, obtains tally set；The label includes expression mark
Label and face label；
Step 4 trains depth of seam division network using the tally set as output using the facial expression image collection as input,
Obtain facial expression recognition model；
The depth of seam division network includes the multiple convolutional layers set gradually, multiple full articulamentums and the double-deck tree classification
Device；
The double-deck Tree Classifier includes Expression Recognition classification layer and recognition of face classification layer, the Expression Recognition classification
Layer output expression label, the recognition of face classification layer export face label.
2. as described in claim 1 based on the facial expression recognition model building method of depth of seam division network, which is characterized in that
Using error back propagation method training depth of seam division network in the step 4, facial expression recognition model is obtained.
3. as claimed in claim 2 based on the facial expression recognition model building method of depth of seam division network, which is characterized in that
Expression Recognition classification layer weight is calculated separately using Formulas I and Formula II using when error back propagation method training depth of seam division network
The partial derivative of the partial derivative of parameter and recognition of face classification layer weight parameter:
Wherein,For Expression Recognition classify layer weight parameter partial derivative,For loss function, W is Expression Recognition classification layer power
Weight parameter, xeFor Expression Recognition classify layer input feature vector,For the partial derivative of recognition of face classification layer weight parameter, V
For recognition of face classification layer weight parameter, xfFor the feature vector of recognition of face classification layer input；
The loss function ￡ is obtained using formula III:
Wherein, VnIndicate the weight parameter of n-th of face label of recognition of face classification layer, VjIndicate recognition of face classification layer
The weight parameter of j-th of face label, j=1,2 ..., N, n=1,2 ..., N, N indicate face total number of labels, and N is positive integer,
bfIndicate recognition of face offset parameter, bfFor natural number；
WmIndicate the weight parameter of m-th of expression label of Expression Recognition classification layer, WiIndicate i-th of Expression Recognition classification layer
The weight parameter of expression label, i=1,2 ..., M, m=1,2 ..., M, M indicate expression label sum, and M is positive integer, beIt indicates
Expression Recognition offset parameter, beFor natural number.
4. as described in claim 1 based on the facial expression recognition model building method of depth of seam division network, which is characterized in that
Depth of seam division network described in the step 4 includes 5 convolutional layers set gradually, 3 full articulamentums and 1 bilayer
Tree Classifier.
5. as described in claim 1 based on the facial expression recognition model building method of depth of seam division network, which is characterized in that
Face label in the step 3 includes facial paralysis label and non-facial paralysis label.
6. a kind of facial expression recognizing method based on depth of seam division network, which is characterized in that treat knowledge using step A- step B
Other facial image is handled, and recognition result is obtained:
Step A, the facial image to be identified is handled using the method for step 2 in claim 1, obtains expression
Image；
Step B, the table that step A is obtained using facial expression recognition model described in any one of claim 1-5 claim
Feelings image is identified, recognition result is obtained.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910307984.0A CN110163098A (en) | 2019-04-17 | 2019-04-17 | Based on the facial expression recognition model construction of depth of seam division network and recognition methods |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910307984.0A CN110163098A (en) | 2019-04-17 | 2019-04-17 | Based on the facial expression recognition model construction of depth of seam division network and recognition methods |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110163098A true CN110163098A (en) | 2019-08-23 |
Family
ID=67639391
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910307984.0A Pending CN110163098A (en) | 2019-04-17 | 2019-04-17 | Based on the facial expression recognition model construction of depth of seam division network and recognition methods |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110163098A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111126180A (en) * | 2019-12-06 | 2020-05-08 | 四川大学 | Facial paralysis severity automatic detection system based on computer vision |
| CN112001213A (en) * | 2020-04-25 | 2020-11-27 | 深圳德技创新实业有限公司 | Accurate facial paralysis degree evaluation method and device based on 3D point cloud segmentation |
| CN112768065A (en) * | 2021-01-29 | 2021-05-07 | 北京大学口腔医学院 | Facial paralysis grading diagnosis method and device based on artificial intelligence |
| CN116051682A (en) * | 2022-11-30 | 2023-05-02 | 四川省中车铁投轨道交通有限公司 | Intelligent tramcar chassis fault detection method and system |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2016026063A1 (en) * | 2014-08-21 | 2016-02-25 | Xiaoou Tang | A method and a system for facial landmark detection based on multi-task |
| KR20160037423A (en) * | 2014-09-29 | 2016-04-06 | 동명대학교산학협력단 | A Hybrid Method based on Dynamic Compensatory Fuzzy Neural Network Algorithm for Face Recognition |
| CN106803069A (en) * | 2016-12-29 | 2017-06-06 | 南京邮电大学 | Crowd's level of happiness recognition methods based on deep learning |
| CN109376692A (en) * | 2018-11-22 | 2019-02-22 | 河海大学常州校区 | Migration convolution neural network method towards facial expression recognition |
| CN109583322A (en) * | 2018-11-09 | 2019-04-05 | 长沙小钴科技有限公司 | A kind of recognition of face depth network training method and system |
-
2019
- 2019-04-17 CN CN201910307984.0A patent/CN110163098A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2016026063A1 (en) * | 2014-08-21 | 2016-02-25 | Xiaoou Tang | A method and a system for facial landmark detection based on multi-task |
| KR20160037423A (en) * | 2014-09-29 | 2016-04-06 | 동명대학교산학협력단 | A Hybrid Method based on Dynamic Compensatory Fuzzy Neural Network Algorithm for Face Recognition |
| CN106803069A (en) * | 2016-12-29 | 2017-06-06 | 南京邮电大学 | Crowd's level of happiness recognition methods based on deep learning |
| CN109583322A (en) * | 2018-11-09 | 2019-04-05 | 长沙小钴科技有限公司 | A kind of recognition of face depth network training method and system |
| CN109376692A (en) * | 2018-11-22 | 2019-02-22 | 河海大学常州校区 | Migration convolution neural network method towards facial expression recognition |
Non-Patent Citations (1)
| Title |
|---|
| 彭先霖等: "基于多任务深度卷积神经网络的人脸/面瘫表情识别方法", 《西北大学学报(自然科学版)》 * |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111126180A (en) * | 2019-12-06 | 2020-05-08 | 四川大学 | Facial paralysis severity automatic detection system based on computer vision |
| CN112001213A (en) * | 2020-04-25 | 2020-11-27 | 深圳德技创新实业有限公司 | Accurate facial paralysis degree evaluation method and device based on 3D point cloud segmentation |
| CN112001213B (en) * | 2020-04-25 | 2024-04-12 | 深圳德技创新实业有限公司 | Accurate facial paralysis degree evaluation method and device based on 3D point cloud segmentation |
| CN112768065A (en) * | 2021-01-29 | 2021-05-07 | 北京大学口腔医学院 | Facial paralysis grading diagnosis method and device based on artificial intelligence |
| CN116051682A (en) * | 2022-11-30 | 2023-05-02 | 四川省中车铁投轨道交通有限公司 | Intelligent tramcar chassis fault detection method and system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110321833B (en) | Human body behavior identification method based on convolutional neural network and cyclic neural network | |
| US20190042952A1 (en) | Multi-task Semi-Supervised Online Sequential Extreme Learning Method for Emotion Judgment of User | |
| EP4242923A2 (en) | Compact language-free facial expression embedding and novel triplet training scheme | |
| Kadam et al. | CNN model for image classification on MNIST and fashion-MNIST dataset | |
| CN110163098A (en) | Based on the facial expression recognition model construction of depth of seam division network and recognition methods | |
| Wu et al. | Enhancing TripleGAN for semi-supervised conditional instance synthesis and classification | |
| CN109522925A (en) | A kind of image-recognizing method, device and storage medium | |
| Ali et al. | Facial emotion detection using neural network | |
| Ocquaye et al. | Dual exclusive attentive transfer for unsupervised deep convolutional domain adaptation in speech emotion recognition | |
| CN113392766A (en) | Attention mechanism-based facial expression recognition method | |
| CN108446676A (en) | Facial image age method of discrimination based on orderly coding and multilayer accidental projection | |
| Jiang et al. | Heterogenous-view occluded expression data recognition based on cycle-consistent adversarial network and K-SVD dictionary learning under intelligent cooperative robot environment | |
| CN109508640A (en) | A kind of crowd's sentiment analysis method, apparatus and storage medium | |
| CN110414626A (en) | A kind of pig variety ecotype method, apparatus and computer readable storage medium | |
| Depuru et al. | Convolutional neural network based human emotion recognition system: A deep learning approach | |
| CN114492634B (en) | Fine granularity equipment picture classification and identification method and system | |
| CN116258990A (en) | Cross-modal affinity-based small sample reference video target segmentation method | |
| He et al. | Global and local fusion ensemble network for facial expression recognition | |
| Wang et al. | Fusion network for face-based age estimation | |
| CN111783852B (en) | Method for adaptively generating image description based on deep reinforcement learning | |
| Halkias et al. | Sparse penalty in deep belief networks: using the mixed norm constraint | |
| Tripathi | Facial emotion recognition using convolutional neural network | |
| CN109583406B (en) | Facial expression recognition method based on feature attention mechanism | |
| He et al. | Dual multi-task network with bridge-temporal-attention for student emotion recognition via classroom video | |
| Mallet et al. | Hybrid Deepfake Detection Utilizing MLP and LSTM |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20190823 |
|
| RJ01 | Rejection of invention patent application after publication |